About EDDRA quality levels
The purpose of EDDRA is to compile evaluated practices in drug demand reduction implemented in Member States of the European Union. In order to take into account differences in the quality of evaluated practices, a set of criteria were developed in 2007. These criteria focus on the quality assessment of intervention evaluation. They are used to categorise evaluated projects according to three quality levels:
- Level 1 projects
- Level 2 projects: promising projects
- Level 3: top level projects
Level 2 and 3 are determined via a points system.
The criteria focus primarily on implementation and evaluation design based on the logic model and data collection. Criteria that help to judge the strength of the available evidence on the effectiveness of the intervention will be introduced once the majority of submitted projects are rated as promising or top level projects.
Criteria for Level 1 projects
All projects that meet the following entry criteria will be added to EDDRA:
- The project has been evaluated
- It has a theory base/assumptions that are clearly related to its objectives
- It has clear evaluation indicators in place that relate to its objectives and initial situation
- It contains a clear description of the evaluation design
- The project must be at least one year old
Level 2 and 3 EDDRA Projects
| Logic model | Points | Evaluation | Points | Additional information/deliveries | Points |
|---|---|---|---|---|---|
|
Specific objectives exist
|
1
|
Process evaluation* |
2
|
Coordination with other services and programmes* |
2
|
|
Specific objectives are linked to indicators
|
1
|
-
|
|
-
|
|
|
Indicators reduce the objectives into one or more quantifiable dimensions |
1
|
-
|
|
-
|
|
|
Specific objectives connect to initial situation
|
1
|
-
|
|
-
|
|
|
The presented results refer to the formulated objectives*
|
1
|
Outcome evaluation: Follow-up assessment |
2
|
Instruments used for outcome evaluation are available or delivered to EMCDDA |
2
|
|
Outcome evaluation results available
|
1
|
Pre-post design, no comparison group (naturalistic) |
4
|
Instruments used for outcome evaluation are new (not yet in the EIB) |
1
|
|
The working hypothesis presented links to the initial situation* |
1
|
Pre-post design AND comparison group (quasi-experimental) |
8
(12) 1
|
AND are validated instrument(s) |
1
|
|
The working hypothesis is based on evidence (references to controlled trials at least)* |
2
|
Pre-post design AND comparison group AND randomisation (RCT) |
12
|
Project manual is available and delivered to the EMCDDA* |
2
|
|
The working hypothesis links to the specific objectives and the indicators* |
2
|
Outcome evaluation with modified instrument based on a validated instrument |
2
|
-
|
|
|
Activities (programme contents) fit to objectives*
|
1
|
Outcome evaluation with validated instrument
|
4
|
-
|
|
|
Activities fit to objectives and working hypothesis*
|
1
|
-
|
|
-
|
|
|
Max. sum of points
|
13
|
Max. sum of points
|
18
|
Max. sum of points
|
8
|
* Criteria that can be met also by process evaluations. The points system is arranged in a way that also projects that only conducted a process evaluation can achieve the required points to become a level 2 (promising) project.
1 if this is the best feasible design for that setting
Cut-off point for level 2: >12 (“Promising project”)
Cut-off point for level 3: >28 (“Top level” projects)
