2. Methodology

Download

Image – Inclusive Justice Activity

To conduct the eJNS survey, adult (18+) participants were recruited through two strategies. First, the survey was advertised on Facebook and Instagram (Meta platforms). Ads were run through USAID Justicia Inclusiva’s Meta account. The ads targeted people living in the 76 municipalities included in the Justicia Inclusiva programme.

Secondly, the survey was disseminated via WhatsApp by Justicia Inclusiva’s regional teams and civil society organisations in 76 municipalities. This approach is similar to snowball sampling⁵, although people were not instructed to share the survey once completed.

Both of these sampling methods are not representative of the population, as respondents are self-selected and do not follow the true distribution of the population geographically, by age, gender, and so on. Exogenous factors like online presence and internet penetration also affect the types of participants obtained.

Each gathering method (social media advertising and via WhatsApp) had a unique survey link. The surveys themselves are exactly the same, but this allowed the research team to keep the responses separate, ensuring the ability to discern and properly identify each survey group.

To motivate participants to take part in the survey on social media, an incentive was provided whereby 50 participants were randomly selected to receive a cellular data package of 24 gigabytes. The survey was advertised between 31 August – 6 October 2023.

After data collection was complete, three follow-up qualitative interviews were conducted with randomly selected participants from each of the three regions of the Justicia Inclusiva programme⁶. The interview participants were solely selected from the Social Media group of respondents. Interviewees were asked about their preferences and attitudes towards legal actors, their experiences throughout their justice journeys, and their perception of justice in their communities. The purpose of these follow-up interviews was to give some qualitative context to the data collected.

The Survey

Participants were presented with a list of 14 legal problem categories and asked to indicate those they had experienced in the preceding 12 months. These 14 macro categories were created based on the list of 101 problems used in the JNS survey and validated with experts during adaptation meetings⁷. Each problem category was paired with a visual icon to help people identify and better understand the category.

Additionally, some categories included examples to further illustrate the problems they were meant to represent. The categories were:

Land (ex: displacement, boundaries, titles)
Domestic violence
Housing
Neighbour disputes
Employment (ex: non-payment of salaries, work accident)
Family (ex: divorce, child support, inheritances)
Access to health, pensions, education and public services⁸

Crime and misdemeanours (ex: homicide, robbery, extorsion, cattle rustling)
Public order and armed conflict (ex: kidnapping or illegal detention, physical violence, forced disappearance)
Purchase of goods and services (public and private)
Traffic (ex: accidents, unfair fines)
Debt and money
Documentation (ex: citizenship card, military ID, migratory status)
Corruption

After this selection, a series of questions were asked for each individual problem to learn about its seriousness, resolution status, and the fairness of the resolution, if the problem had been resolved. This information gives insights into the problems people experience and seek fair solutions for, as well as the number of problems which achieve some kind of resolution.

In the second section, the survey automatically selected the problem that the participant ranked the highest in terms of seriousness (on a scale of 1-10). This problem was then the focus of the remaining survey questions. If two or more problems were ranked the same, the survey randomly selected one to continue with. For their most serious problem people reported on who they went to for help to try and solve their problem. If the person went to anyone for help, then a series of questions was asked to learn about the helpfulness of these actors.

Limitations

As is the case with many online and self-selection surveys, it is likely that the eJNS data better represents younger, urban populations, who have access to the internet and are more likely to use social media platforms. Furthermore, because respondents decide whether or not to click on the survey, there is a possibility that those who complete it are more likely to have legal problems, or bad experiences with the justice system and are eager to share these. For these reasons, there is no guarantee of a representative sample for the eJNS, which may lead to biases in the data.

Thus, while the eJNS approach presents the justice experiences of specific people, in specific places, its results are not generalisable towards broader populations. When analysing and using the eJNS data, caution is needed and it must always be accompanied by other contextual considerations. When possible, results from online surveys should be contrasted with more traditional and representative surveys.

Finally, it should be noted that some sub-samples throughout the survey are very small, making it difficult to compare across groups as proportions are distorted. To avoid such distortions, the results presented here mainly reflect at least 50 responses.

Sample Characteristics

The overall gender breakdown of the sample is 64% female and 36% male. The social media sample group has a slightly more even distribution of 57% female and 43% male.

As expected, due to the nature of the sampling method, the majority of participants self-identify as living in urban areas (71%).
The average age is 42 years old. Throughout this report, participants are separated into four age categories: Youth (18-24); Young adulthood (25-39); Middle adulthood (40-64); and Seniors (65+).
42% of Colombian respondents self-identify as an internally displaced person⁹ (IDP).
The majority, 54%, report being able to cover their basic financial needs.

In 2022, the World Bank reported that 73% of individuals in Colombia use the internet¹⁰. This is quite high compared to other countries where HiiL has conducted similar eJNS surveys. The team had hoped this would increase the response rates of the social media administered survey. However, the rate was comparable to other eJNS surveys HiiL has conducted via social media. Unfortunately, internet usage does not necessarily correlate to higher response rates. There are many factors which could have impacted the sample size including, but not limited to:

Scepticism regarding the survey’s purpose and use;
mistrust of those administering the survey, as it came from a Facebook page;
concern as to whether the anonymity of the survey would be respected;
caution by people on the subject of justice as the survey was advertised during an electoral period;
lacking legitimacy due to being an electronic survey; and
survey length.

References

[5] This is a non-probability sampling method in which the contacts of a specific group are used to obtain new participants in a study. In this case, the contact group used was the regional teams of Inclusive Justice, who then shared the survey with civil society organisations in the municipalities. These organisations, in turn, shared the survey with other individuals.

[6] Coastal Savannah (Bolívar, Montería, Sucre and Bajo Cauca in Antioquia), North (Urabá in Antioquia and Chocó), and South (Valle del Cauca, Cauca, Nariño and Putumayo)

[7] We want to give special thanks to Sara Palacio Gaviria, project coordinator at Universidad EAFIT; Andrés Ham González, associate professor at Universidad de los Andes; Ángela Guarín, assistant professor at Universidad de los Andes; and the Inclusive Justice team of specialists who participated and contributed in the adaptation process of this survey.

[8] This category is presented as “State Services” in the charts throughout this report.

[9] The eJNS explicitly asked about self-recognition of displacement status. Participants could indicate whether they do recognise themselves as internally displaced, whether they do not, or if they preferred not to answer. The report uses this variable to explore differences between internally displaced and non-displaced populations. It is possible that several of the findings for the internally displaced population are also applicable to victims of the armed conflict in general.

[10] https://data.worldbank.org/indicator/IT.NET.USER.ZS?end=2022&locations=CO&start=2014