EMCDDA Home
  • EN
Search

Program Domowych Detektywow

Quality level: 3

Process evaluation

Results

The process evaluation was based on a combination of qualitative and quantitative methods: - post-test questionnaires for students (n=226) and parents (n=158);- two group interviews with teachers (n=10); and - analyses of programme documentation.The evaluation found that the programme had been fully implemented in all intervention schools. According to self-report data from both students and parents, over 90 % of the students participated in the booklet activities, most frequently with their mothers. Similar rates were identified from the teachers' classroom records. Girls, pupils in two-parent families and 'good' students were significantly more likely to complete more booklets. The rate of participation in the family evening was also high, with 74 % of students attending, 56 % with at least one parent. Teachers were given two alternative methods of selecting peer leaders in the classroom: election from a whole group of students; or election from small, pre-selected groups. Although most of the selections were based on student popularity, group interviews with teachers established that the peer-leader election procedure differed from class to class. Being a peer leader was perceived by students as an honour. In the teachers' opinions, the trained peer leaders were very motivated and fully engaged in the programme activities. Although they experienced some difficulties with discipline during small group activities, they generally performed their tasks well or very well.The process evaluation also identified some significant differences in the quality of the programme delivery. Students from three classes reported significantly lower rates of participation in the activities and much lower rates of satisfaction. Analyses of documentation and interviews with teachers confirmed that in these three classes there had been some defects in the programme implementation - such as shortening the classroom activities and poor preparation of the family evening - as well as inadequate co-operation between school staff involved in the programme, insufficient parental acceptance of the programme and lack of commitment by the school authority.Quality of program delivery was assessed by: focus group interviews with teachers, phone interviews with randomly selected parents, student's feedback (questionnaire). Process evaluation indicated that programme was successfully implemented in majority of classes, but in some participating classes there were important defects in quality of programme delivery evidenced by low level of students satisfaction, low level of teachers motivation, and difficulties in completing all key programme elements (e.g. family evenings).

Outcome evaluation

Results

The first outcome evaluation was focused on immediate outcomes and adopted a quasi-experimental design. The study sample (n=440) consisted of fifth-grade pupils in ten primary schools from Mokot�w, a district of Warsaw. The school authorities accepted random assignment to either the intervention or reference group. Five schools were assigned to the intervention group (n=231) and five to the reference group (n=209), which was offered the programme the following term. The evaluation was based on a self-report questionnaire. Pupils were assessed in their classrooms by trained staff at pre-test (September 1998) and at post-test (December 1998). Of the students monitored at post-test, 11 % dropped out leaving 393 or 89 %, and there were no differences in attrition rates between the intervention and reference groups. Analyses were performed using multiple analyse of variance (Manova) with an individual subject as the unit of analysis. Comparison of the differences using the six main scales suggested a significant reduction in alcohol use (p<0.02) and intention to drink (p<0.03) among students in the intervention group. The second outcome evaluation (copmpleted in 2003) was focused on delayed outcomes (4-month follow-up) and also utilized quasi-experimental design. The study sample (n= 800) consisted of fourth and fifth grade students of eight primary schools in Warsaw. The schools were randomly assigned either to the intervention or reference condition. In this second evaluation no effect of the programme was found in terms of students' alcohol use and intention to drink, except a group of fourth-graders who demonstrated some favourable changes in these behavioural variables.


The first outcome evaluation showed that the programme was also effective in increasing parent-child communication about the effects of both alcohol and cigarette use (p<0.03). There were no significant differences on two scales concerning peer norms and resistance skills. The results of the second evaluation (4-month follow-up) confirmed programme effectiveness in terms of deceasing students' pro-alcohol attitudes.


The programme was effective in increasing pupils' knowledge about the consequences of alcohol drinking (p<0.001). These results were confirmed in the second evaluation done 4 months after completing the programme.

References

1. Bobrowski K. (2004) Ocena odroczonych efektow Programu Domowych Detektywow mierzonych po czterech miesiacach od zakonczenia programu (The Slick Tracy Home Detectives Program outcome evaluation - a four-month follow-up), Alkoholizm iNarkomania 18(1-2), 61-76. 2. Ostaszewski, K., Bobrowski, K., Borucka, A., Okulicz-Kozaryn, K., Pisarska,A., Perry, C., Williams, C. (1998) 'Program Domowych Detektywow. Adaptacja amerykanskiego programu profilaktyki alkoholowej dla mlodziezy we wczesnym okresie dojrzewania' ('A Polish adaptation of the US alcohol primary prevention programme for young adolescents'), Alkoholizm i Narkomania, 3, 339-60. 3. Ostaszewski K., Bobrowski K, Borucka A., Okulicz-Kozaryn K., Pisarska, A. (2000) : Chapter 7. Evaluating innovative drug-prevention programmes: Lessons learned. [in:] Evaluation - a key tool for improving drug prevention. EMCDDA Scientific Monograph Series No 5, European Commission, EMCDDA, 75-85. 4. Perry, C., Williams, C., Veblen-Mortenson, S., Toomey, T., Komro, K.,Anstine,P., McGovern, P., Finnegan, J., Forster, J., Wagenaar, A.,.Wolfson, M. (1996)'Outcomes of a community-wide alcohol use prevention program during early adolescence: Project Northland', American Journal of Public Health, 86, 956-65. 5. Williams, C., Perry, C., Dudovitz, B., Veblen-Mortenson, S., Anstine, P.,Komro, K. A., Toomey, T. L. (1995a) 'A home-based prevention program for sixth-grade alcohol use: Results from Project Northland', Journal of Primary Prevention, 16 ,125-47.

The PDF contains the full intervention description including additional contact information.

Survey

Dear visitor,

We're currently conducting a short survey of this resource (EDDRA: Exchange on Drug Demand Reduction Action). The survey will take no more than 2 minutes and we would be really grateful for your contribution.

No, thanks — maybe some other time

Yes, I'll take the survey!

Many thanks,

The EDDRA team

Print

Page last updated: Friday, 20 January 2012