About the Xchange prevention registry

Background

Xchange is an online registry of evidence-based prevention programmes. In its first phase, the registry will make available manualised interventions that European evaluation studies show have beneficial outcomes relating to substance use. For programmes of US origin, the registry also provides the evidence level of the evaluation studies conducted there (Blueprint ratings).

The initial research for eligible prevention programmes and the rating process was done within the project: “Communities That Care (CTC) European Network: Making CTC work at the European level” (1/2013 – 12/2015), supported by the “Prevention of and Fight against Crime Programme” (European Commission - Directorate General Home Affairs).

In addition to providing information on the effectiveness of programmes, the registry also offers information on the experiences of professionals who have implemented these programmes in individual European countries. This enables decision-makers to assess the ease with which programmes can be implemented in different social, cultural and organisational contexts.

The registry is the centrepiece of a growing network of national registries. Through Xchange, an interested visitor can access additional details on a specific programme in the national language within a local registry, or access through a national registry the European implementation experiences of a given programme.

What is the scope of the registry and what kind of programmes are included?

At present, Xchange contains 20+ manualised prevention programmes, all of which address substance use-related problems. The programme must be still active (i.e. currently in use in at least one EU country) and be judged to be beneficial in Europe by at least one European evaluation study. In future, Xchange will be expanded to include programmes that target other risky behaviours and possibly environmental interventions at local level.

Why such a European registry?

The Council of the EU’s Minimum quality standards in drug demand reduction in the EU demands that prevention professionals have access to knowledge on effective prevention programmes. A growing number of programmes are considered effective at reducing substance use and related problems under carefully controlled conditions. However, access to evidence‐based prevention programmes is still limited and they remain under‐utilised compared to prevention strategies with no empirical evidence for effectiveness. Xchange aims to address this challenge by providing access to such programmes.

An additional challenge is that, even when effective programmes are used, they are not being implemented with quality and fidelity in real-life settings, often because they do not fit into the existing prevention culture and infrastructure. Xchange provides decisional support by continuously compiling information on implementation experiences with the programmes contained within it.

Users can access information on the effectiveness of a programme and on the extent to which it can be implemented. The real-life scenario in Europe’s multiple cultures is that a highly effective programme may not be easy to implement in a given country, while a less effective programme may be easier to implement. Users of Xchange will have more tools for balanced decision-making at their fingertips because the registry also allows for selecting programmes according to targeted outcomes or the risk and protective factors it addresses.

A unique feature of Xchange is that it allows national high-ranking programmes to be promoted in this European registry and, inversely, allows international visitors to find more details about local adaptations in national languages. If you would like to share your experience of implementing one of the programmes in the registry, you are invited to contribute by filling in the online implementation questionnaire (https://drugusersurvey.limequery.com/index.php/839193/lang-en).

How does a programme get into our registry?

Here we use the quality of the evaluation studies for the programmes as entry criteria. We consider studies with the following characteristics

Intervention Criteria

Mandatory:

  • goal definition: The outcomes of the intervention are clearly defined
  • target group: The intended subjects to receive the intervention are clearly identified
  • conceptual quality: The intended subjects to receive the intervention are clearly identified
  • risk and protective factors that the programme seeks to change must be identified
  • the programme's theoretical rationale or logic model must be explained
  • intervention documentation: there is a documentation of the intended intervention structure, content and delivery process
  • programme focus: the programmes should focus on substance use, delinquent, violent and related behaviours.

Evaluation Design

Mandatory: the programme has been evaluated in Europe

Mandatory: The programme has been evaluated by at least one randomized controlled trail (RCT) OR quasi-experimental evaluations of good quality, such as:

  • sample characteristics: Clear statement of the demographic characteristics of the population participating in the intervention and in the control group(s)
  • intervention characteristics: there is documentation of what participants usually received in the intervention
  • evaluation quality: Appropriate assignment to the intervention
    • valid and reliable measures, description of administrative and archival indicators, specification of data collection method
    • analysis is based on ‘intention-to-treat’
    • appropriate statistical analysis, control for baseline differences
    • test for equivalence between intervention and control group at baseline

Not mandatory for inclusion:

  • long-term follow-up
  • sample size and effect size: Population and any specific subgroups with whom the program has been demonstrated to be effective are stated; reporting of effect sizes, along with significance levels
  • Reporting of relevant conditions under which the effectiveness was found to vary
  • Attrition Bias: No evidence of significant differential attrition

Implementation and dissemination: not required for inclusion, but used for the ratings

  • implementation readiness/accessibility: available are materials (e.g., programme manuals, brochures, information for administrators) that could facilitate but not directly assist with programje implementation
  • existing guidelines (replicability): available are materials (e.g. tested training curricula, mechanisms for ongoing supervision and coaching) that could facilitate but not directly assist with programme implementation
  • training and instruction is available
  • materials (e.g., protocols for gathering process and/or outcome data, ongoing monitoring of intervention fidelity, supervision/training feedback) that could facilitate but not directly assist with program implementation are available
  • dissemination readiness: explicit processes for insuring the program gets to the right persons
    • support / technical assistance during implementation is available
    • financial resources required to deliver the intervention are specified
    • human resources required to deliver the intervention are specified
  • European implementability: there should be indicators why the programme can be implemented in different European countries
  • transferability across countries: the programme is implemented and evaluated in more than 2 European countries
  • transferability across settings: the programme is implemented and evaluated in more settings

How do we get to the rating of each programme?

We asssess intervention impact across the different evaluations of programme versions in Europe:

  • significant positive impact: the level of evidence of a consistent and statistically significant beneficial impact on at least one primary outcome, across different studies and/or programme versions. See also 'about Xchange ratings' below
  • absence of iatrogenic effects: There is an absence of iatrogenic effects for intervention participants

About Xchange ratings?

More information on the ratings used are provided below.

Beneficial: Interventions for which significant effects for most outcomes(1) are in favour of the intervention as found in randomised controlled trials (RCTs) or experimental design studies of good quality in Europe. An intervention ranked as ‘beneficial’ is suitable for most contexts.

Likely to be beneficial: Interventions for which the effects for most outcomes1 are in favour of the intervention as found in evaluation studies of acceptable quality in Europe and for which the evidence is therefore limited. An intervention ranked as ‘likely to be beneficial’ is suitable for most contexts, with some discretion.

Likely to be partially beneficial: Interventions for which the effects for some outcomes(2) are in favour of the intervention as found in evaluation studies of acceptable quality in Europe and for which the evidence is therefore limited and partial. An intervention ranked as ‘likely to be partially beneficial’ is suitable with caution, and should be tested in more contexts.

Unknown effectiveness: Interventions for which the effects for a few outcomes(3) are in favour of the intervention as found in evaluation studies of acceptable quality in Europe, making it difficult to assess if they are effective or not.

Evidence of ineffectiveness: Interventions that gave no or negative results in evaluation studies of acceptable quality in Europe.

Not included in Xchange are interventions for which there are no evaluation studies of acceptable quality in Europe, notwithstanding ratings of their effectiveness in other continents.

(1) Generally this would be more than 50% of measures. It would also be based on most relevant, e.g. some effects on child outcomes, and not only effects on parent outcomes

(2) For example there is an effect only on parent outcomes or only about 30% of outcomes

(3) There is a positive effect on less than 20% of outcomes or a negative effect

Why can I not find a programme that I know or have heard of?

There are a number of possible reasons for this, including:

  • It has not been entered yet into the registry. We have contacted the programme leaders of top level entries in our previous EDDRA registry and offered them to provide the additional information and evidence needed in order to put their programme into evidence Xchange. The REITOX National Focal Points and the network of the EUSPR are looking for eligible programmes and for 2018 we are also aiming at including more programmes from National Registries of evidence-based programmes in Germany, Spain, France, Italy, Netherlands, UK and Poland, if they fulfil the entry criteria of Xchange
  • The studies in Europe related to the programme do not fulfil the criteria above. If the quality of the studies in Europe - even of well-known programmes - is not satisfactory, these programme don't appear in Xchange, even though they might be listed in International Registries.

What if I don’t agree with the evidence rating of a programme?

The registry is dynamic and constantly evolving: if you disagree with an evidence rating or are aware of new good quality studies which review the behavioural outcomes of any of the programmes included in Xchange, please send them to evidence-Xchange@emcdda.europa.eu

These will be discussed at the next meeting of the Review Board of the Xchange registry and will be taken into account when rating the evidence of the respective entry. The board currently consists of one member of the EMCDDA Scientific Committee, members of the European Society for Prevention Research, EMCDDA staff members and appointed experts. The list of current members can be found on the Xchange partners page

What next?

The registry will be expanded to address an increasing range of solution-oriented interventions (e.g. prevention of overdose in the community; prevention of violence and crime and prevention of radicalisation).

Explore all resources in the Best practice portal

 Xchange

Collaborations and partnerships in best practice

logo of the cochrane collaboration  grade logo   grade logo  Health Evidence Network, WHO Europe 

Loading