Home Treatment Guidelines Guideline Method

Guideline Method

Justice problems are as common as diseases. Family problems, threats of violence between neighbours, non-payment of wages or land grabbing, they require high-quality treatments. Evidence-based guidelines can help justice workers decide how to best assist people when they cope with conflict or crime.

This page explains in 11 steps how evidence-based ways of working can be developed in a local (country) setting, whilst benefiting from international best practices and research about what works. We explain the process and methodology for guideline development, which is inspired by the methods for developing guidelines in the health sector and for corrections.

Photo by Tara Winstead from Pexels

The guideline development process starts by gathering a diverse group of local practitioners and justice experts. This group is referred to as the guideline’s Committee of Experts (CoE). The Committee of Experts co-creates the guideline, performs quality control, and helps to create local ownership. Assembling and engaging a Committee of Experts at an early stage helps to increase buy-in from local justice workers.

The Committee is composed of twelve to eighteen members. The Committee should include academic experts, legal professionals, judges, government officials, NGO workers, and users of the justice system. A Chair is appointed with full endorsement of the Committee. The Chair will serve in that capacity for three years, and may extend his or her term on the Committee for up to six years.

The Committee of Experts also reviews and advises on:

a. Overall scope and purpose of the guideline

b. Stakeholder involvement

c. Rigour of development

i. Systematic methods were used to gather evidence
ii. Criteria for selecting and grading evidence are clearly described
iii. The methods for formulating and categorizing recommendations are clearly defined
iv. The desirable and undesirable outcomes for recommendations have been clearly described
v. Links between the evidence base and recommendations are explicit
vi. The guideline has been reviewed externally prior to publication

d. Clarity of presentation

e. Applicability

f. Editorial independence

HiiL develops a comprehensive list of topics to include in the guideline. Topics are subject matters that people need to address in order to be able to solve their particular justice problem.

We make a distinction between three different categories of topics:

Issues: These are the substantive topics that people have to deal with regularly.

For example, in order for people to be able to separate they must agree on housing arrangements.

Process: These relate to the procedural steps that people must take to solve their justice problem.

For example, in order for people to be able to separate they should receive a diagnosis from a neutral third party.

Complications: These include complications that need extra attention.

For example, a complication in separation could be the existence of domestic violence (how to deal with that?).

The list of topics is based on years of experience and interaction with justice workers and users of the justice sector. HiiL has conducted extensive research on what the most prominent problems are and what kind of solutions people are looking for.^[1] From this initial list, the Committee of Experts identifies the topics that they believe should be prioritized. Additional topics can be included if the Committee considers it necessary. Based on the Committee’s input, the resources and staff available, and HiiL’s mission and values, HiiL decides on the final list of topics to be covered by the guideline.

Helpful legal advice can take multiple forms: procedural knowledge, advice that helps with communication, adjusts behaviour, and provides reassurance.

^[1] HiiL, Justice Needs and Satisfaction in Uganda, accessible via: https://www.hiil.org/projects/justice-needs-and-satisfaction-in-uganda/ and HiiL, Understanding Justice Needs, The Elephant in the Courtroom, p. 54, accessible via: https://www.hiil.org/projects/understanding-justice-needs-the-elephant-in-the-courtroom/

Identifying and working towards needs-based outcomes is essential for delivering high-quality justice^[2]. Outcomes suggested by research (victimology, criminology, empirical research on effective interventions) and practice are listed in the following table.

Outcomes per type of justice problems:

Family: 1. Children are cared for 2. Secure housing for all 3. Secure incomes for all 4. No violence 5. Respectful communication 6. Division of debts 7. Division of property

Land: 1. Compensation for loss of income or property 2. Sharing of benefits 3. Allocation of land ownership 4. Agreement on the use of land 5. Protection

Neighbour: 1. Respectful communication 2. Less nuisance 3. Repair or compensation 4. Repairing relationships 5. Solutions for border issues (buildings, fences, trees)

Employment: 1. Respect for achievements 2. Financial compensation 3. Adjustment of roles 4. Employment prospects 5. Maintaining good relationships 6. Timeline

Crime: 1. Understanding what happened 2. Return of property, repair or compensation 3. Perpetrator is caught 4. Protection, preventing it will happen again 5. Apology or explanation 6. Punishment 7. Both parties being able to move on

HiiL leads a workshop with the CoE to create a common understanding of the outcomes that people need in their local context, backed by international literature research on user needs. The goal is to create a shared understanding of the relevant outcomes in the local context. The outcomes stated above are tested with the CoE to ensure that it is in line with the local context and needs. A common understanding of outcomes will help to identify what practice-based evidence and evidence-based practice we need to identify (steps 4 and 5).

^[2] OECD, Equal Access to Justice for Inclusive Growth, p. 190, accessible via: http://www.oecd.org/gov/equal-access-to-justice-for-inclusive-growth-597f5b7f-en.htm and Task Force on Justice, Justice for All, accessible via: https://www.justice.sdg16.plus/report

After the outcomes are defined, we identify what local justice workers from the informal and formal sectors consider best practices for resolving their most pressing justice problems. We organize workshops and invite government, local, and civil society leaders to share their experiences on what works. The workshops are conducted in mixed groups of around 20 participants of justice workers from the formal and informal sectors from a specific area. It is recommended that at least 5 separate workshops take place in 5 different locations, collecting information from around 100 practitioners. Each workshop is facilitated as full-day workshops and ample time is given to not only collect suggested best practices on specific interventions, but also how they relate to each other.

Workshops are conducted according to a standard format:

5. Group sessions: Government, local, and civil society leaders share their experiences
6. Share feedback in interactive setting
7. First working session: Define best practices for issues
8. Second working session: Define best practices for process
9. Third working session: Define best practices for complications
10. Reflect on the workshop and conclude

In order to ensure that workshops are representative of the national population, HiiL conducts several workshops throughout the country the guideline is being developed for. Geographical coverage depends on the resources available, and is determined in partnership with the Committee of Experts.

There are many different steps in the path to resolving a justice problem. Each problem is broken down into a broad range of topics (such as mediation, adjudication, arrangements for raising children etc.) For each of these topics a number of possible interventions can be identified to help prevent or resolve the problem. Internationally available literature contains evidence which supports or invalidates interventions to justice issues. We test these interventions, rate the quality of evidence that underlies them, and define actionable recommendations. The following steps explain this process.

i. First literature search: Identifying the most common interventions

HiiL conducts a first literature search to identify all possible interventions for each topic. This is done in accordance with the search strategy (explained in step three). For each topic, the team selects the two or three most effective interventions. Two interventions are then compared with each other in PICO format and – if applicable – similarly compared to a third intervention.

ii. Defining the PICO question^[3]

In order to assess the effectiveness of interventions, HiiL compares selected interventions using PICO questions. The PICO-approach is used in the medical sector to help define the effectiveness of an intervention. PICO stands for:

These four elements should always be present in the PICO questions. The standard structure of a PICO question is: For [population/person], is [intervention 1] more effective than [compared intervention 2] for [outcome/goal]?

iii. Search strategy^[4] and literature selection^[5]

After identifying the PICO question, HiiL reviews and selects the literature. HiiL first defines which keywords are used for the literature search.

The literature search is conducted in a hierarchical manner. HiiL starts with looking into existing evidence-based guidelines, systematic reviews and meta-analyses. In case this results in insufficient evidence, the search extends to randomized controlled trials. If this is not available, the team searches for observational studies and empirical research. Lastly, if needed, HiiL gathers relevant opinions by international experts.

iv. Assessing and grading the evidence^[6]

The literature contains evidence. Following the selection of literature, the quality of the evidence is evaluated. HiiL grades the quality of evidence for each PICO question, based on the GRADE (Grading of Recommendations Assessment, Development and Evaluation) system. The GRADE-manual (accessible on www.guidelinedevelopment.org/handbook) provides an elaborate description of this method.

GRADE is used extensively by the medical sector for developing guidelines. The GRADE system provides a definition of the quality of evidence. The quality of evidence reflects the level of confidence in the effect of the intervention on people’s well-being. The evidence is graded according to a three-step process.

v. Making recommendations^[7]

The recommendations can be established following the answer to the PICO question and assessment of the quality of evidence by applying GRADE.

The strength of the recommendation depends on whether the desirable effects of an intervention outweigh the undesirable effects, and on the strength of evidence.

^[3] How to phrase a PICO question is explained in: Schunemann, Brozek, Guyatt and Oxman, GRADE Handbook, Chapter 2, accessible via: http://gdt.guidelinedevelopment.org/app/handbook/handbook.html#h.1yd7iwhn8pxp

^[4] The search strategy is similar to the strategy used in developing guidelines for family doctors in The Netherlands, accessible via: https://www.nhg.org/sites/default/files/content/nhg_org/uploads/handleiding_ontwikkelen_nhg-behandelrichtlijnen_0.pdf

^[5] The literature selection-process is similar to the selection-process used in developing guidelines for family doctors in The Netherlands, accessible via: https://www.nhg.org/sites/default/files/content/nhg_org/uploads/handleiding_ontwikkelen_nhg-behandelrichtlijnen_0.pdf

^[6] The assessment and grading of evidence in the medical sector is explained in: Schunemann, Brozek, Guyatt and Oxman, GRADE Handbook, Chapter 5, accessible via: http://gdt.guidelinedevelopment.org/app/handbook/handbook.html#h.1yd7iwhn8pxp

^[7] How to go from evidence to recommendations in medical guidelines is explained in: Schunemann, Brozek, Guyatt and Oxman, GRADE Handbook, Chapter 6, accessible via: http://gdt.guidelinedevelopment.org/app/handbook/handbook.html#h.1yd7iwhn8pxp

We make a first draft of the guideline by combining practice-based evidence and evidence-based practice. During this process we:

Compatible practices are highlighted and categorized as ‘best practices in line with international literature’, whereas incompatible suggested practices are categorized as ‘other suggested practices’.

b. Include an annex where we elaborate on and assess the evidence base and identify remaining gaps in international literature. Identifying the gaps in literature helps research institutes to identify where further research is needed;
c. Draft comprehensive descriptions of the interventions tested, so that the resulting recommendations are clearly understood. These descriptions are drawn directly from the literature;
d. Categorize the “strength” of recommendations, taking into account the quality of the evidence and (un)desired effects of the interventions on the target group. As previously mentioned, there are four categories of recommendations (Strongly Recommended, Recommended, Context-specific Recommendation and Not Recommended). The strength of a given recommendation is intended to inform whether and to what extent it is applied by justice workers and users.

HiiL submits the guideline to the Committee of Experts for review. The Committee decides whether the recommendations are acceptable within the local/national context. They report their findings within three months of submission. If the Committee of Experts determines that a recommendation is incompatible with local practice, the Committee and HiiL collectively review the recommendation and determine whether it should remain the same, be modified, or be removed from the guideline entirely.

HiiL develops a version of the guideline for justice users and a professional version for justice workers. Depending on user needs, available resources, and the target audience, these versions are put into digital form (by means of an app), hardcopy, or both.

Together with local designers, HiiL develops visual infographics for every recommendation. Culturally-competent visuals are essential because they increase the user-friendliness of the guideline and make it accessible to a broader audience. Once the visuals are included and the review is completed, HiiL can finalize the prototype and develop a dissemination plan in partnership with the Committee of Experts. The dissemination plan is dependent on local context and will therefore vary across guidelines.

The prototype dissemination strategy depends on:

The target group of the guideline
The format of the guideline (app or hardcopy)
Available budget
Timeline
Volume of the guideline
Dissemination channels

Three to six months after dissemination, HiiL assesses whether the guideline is making a meaningful difference in the well-being of justice users. We also want to gain an understanding of how the guideline is used and how it fits into the daily practice of justice workers. Data collection methods vary depending on the project circumstances and dissemination strategy, but may include surveys, interviews, focus group discussions, or direct observation. Using these methods, and in partnership with the Committee of Experts, HiiL tests whether the guideline is creating changes in the knowledge, attitudes, and practices of the target community.

We ask:

This feedback from justice users and workers directly informs improvements to the recommendations in the guideline as well as the way that information is presented and disseminated. We incorporate the feedback into the next version of the prototype.

The guideline is a living document. This means that the recommendations are regularly reviewed, updated, and improved to reflect developments in the field. It is recommended that updates happen on a yearly basis. Both new findings that support previous research and new findings that contradict it are added to the evidence base and used to reassess the accuracy of the guideline. The strength of the recommendations and the recommendations themselves are modified accordingly.

Initially, HiiL is responsible for updating and improving upon the guideline. Over time, local ownership is built to take ownership of this process. The role of HiiL is gradually reduced to a solely supervisory/quality control one. The process for creating local ownership is explained separately in the sustainability strategy.

In addition to supervising the improvement of existing guidelines, HiiL develops guidelines for new populations and new categories of justice problems. The development strategy of new guidelines is informed by data highlighting the most urgent justice problems in countries around the world.

Implementation is an important element for the success of any guideline^[8]. National implementation can take place once HiiL and the Committee of Experts have ensured the recommendations contained in the guideline are desirable and based on high-level quality evidence. Moreover, the support and willingness from formal as well as informal institutions working on solving justice problems is crucial to the success of the countrywide implementation.

Each guideline and each country needs its own implementation plan that takes into consideration the local needs. We already know from the medical sector that new guidelines take several years to be implemented and become common practice. In the justice sector this can be even harder as a paradigm shift needs to happen to convince justice workers and institutions to work with a guideline approach.

Guideline implementation is facilitated by the support of local leaders. Justice workers in influential roles are involved from the beginning of the guideline development process (in the CoE and the early workshops). By involving them early and creating ownership they are asked to commit to working with the guidelines and promoting an evidence-based approach to justice within their respective organisations.

It is important to involve all those who have contributed to the development of the guideline in the CoE as well as in the workshops. They can be champions of the evidence-based approach and be drivers of the implementation process. Feedback is collected every 3 months to understand how/if the guideline is being used and whether there are any potential barriers or nottle-necks that need to be addressed.

Possible barriers to guideline implementation that need to be addressed are:

The Committee of Experts tracks implementation progress and works together with HiiL to modify the implementation plan where needed.

^[8] Rauh, Arnold, Braga, Curca, Eckert, Fröbe, Karamouzis, Lakatos and Molitor, Challenge of implementing clinical practice guidelines. Getting ESMO’s guidelines even closer to the bedside: introducing the ESMO Practising Oncologists’ checklists and knowledge and practice questions, accessible via: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6069906/

Justice Dashboard

Justice Dashboard

Guideline Method

1. Establish a committee of experts

The Committee of Experts also reviews and advises on:

2. Define the topics for the guideline

3. Create common understanding of outcomes with justice users

Workshops are conducted according to a standard format:

4. Learn from the experiences of justice workers (collect practice-based evidence)

5. Collect evidence from the literature and define recommendations (evidence-based practice)

6. Assess compatibility of recommended practices with international evidence base and provide context

7. Experts review the first draft of the guideline

The prototype dissemination strategy depends on:

8. Create and disseminate a prototype

9. Test the prototype

We ask:

10. Improve the guideline continuously

11. Countrywide implementation

Possible barriers to guideline implementation that need to be addressed are:

Contents:

Justice Dashboard

Contact us

Follow us

Navigation