prevention

The programme has been evaluated in one cluster-randomised trial (RCT) and in one quasi-experimental design in The Netherlands. An adapted version for special education students was studied in a quasi-experimental trial.

The first evaluation study (QED) (Cuijpers et al. 2002) enrolled five regions of municipal health services in the Netherlands. In order to be included in the study, schools had to have an active committee coordinating the drug prevention activities at the school. Students of nine experimental (N = 1156) schools were compared with students of three control schools (N = 774). The groups were compared before the intervention started, 1 year later, 2 years later and 3 years later. The primary outcomes were self-reported frequency of and lifetime use of tobacco, alcohol and marijuana. Two years after the start of the intervention, only effects on alcohol use were found. There was no clear evidence for any effects on attitude towards substance use and on self-efficacy. A secondary analysis studied differential effects in various segments of the student population (Smit et al., 2003). Multivariate logistic regression analysis showed that the effect of the intervention on alcohol use was less favourable in students who disliked school.

The RCT (Malmberg et al. 2014) studied two intervention conditions: the integral programme compared to a limited e-learning module. Structured digital questionnaires were administered pre-intervention and at 32 months follow-up. The primary outcome measures were new incidences of alcohol (life-time and 1-month prevalence), tobacco (life-time and 1-month prevalence) and marijuana use (life-time prevalence). There were no statistically significant intervention effects on any outcome. The study concluded that the programme was either ineffective or implemented inadequately. A secondary analysis (Malmberg et al. 2015) explored whether boys, adolescents of lower educational backgrounds, or adolescents high on personality risk traits would benefit more from the HSD program than others. The results showed that sex, education level and personality characteristics of the participants did not moderate the intervention effects.

A QED (Turhan et al. 2017) aimed at studying HSD’s effectiveness on tobacco and alcohol use in Dutch secondary special education (SE) schools, and whether this depends upon subtypes of SE schools and the level of implementation. Thirteen secondary SE schools spread throughout the Netherlands participated. 363 students were allocated arbitrarily or depending on teacher motivation to either intervention condition (n = 205) or usual curriculum (n = 158). There was no clear evidence for any effects on attitude towards substance use or self-efficacy. While no effects of HSD-SE were found for SEL (learning disabilities/developmental disorders) students, significant adverse effects were found in SEB student (behavioural/emotional difficulties) on both a cognitive and a behavioural alcohol-related outcome measure. SEB students distinguish themselves from SEL students in displaying more oppositional and defiant behaviour, leading to counteracting peer interactions during class. The pilot study concluded that within special education, substance use interventions may need to be targeted at school subtypes, as these may have harmful effects among students with behavioural difficulties.

Although HSD includes a parent component (conference providing parenting skills specifically relevant for preventing alcohol and tobacco use) this is not evaluated in any of the above studies. Nevertheless, Koning et al. 2009 demonstrated in a RCT that combining only the alcohol module of the HSD (student component) programme with a brief parental meeting (parent component) that was integrated in an existing general parents’ meeting at the beginning of each school year (year 1, 2 and 3 in high school) was effective. That is, onset of weekly drinking was postponed up to 52 months after baseline through an increase in strict rule-setting about alcohol and adolescents’ level of self-control. The student only or parent only components were not effective. This was different from the original HSD program in three ways 1) only alcohol use was a target, 2) parents meeting was integrated in general parents meeting which yielded high attendance rates and 3) a school committee, regulations and early detection were not included., in which parental rule-setting behaviour was encouraged as an effective strategy, helped to reduce adolescents’ alcohol use. The latter study was not included in the Xchange evaluation because it is not considered the core of the HSD programme.

The effect of changing bar closing hours have been studied in experimental evaluation designs in Norway (Rossow et al. 2012) and The Netherlands (De Goeij et al., 2012). Conducting an RCT is not possible because of the environmental nature of these measures.

Rossow and colleagues (2012) estimated the effect on violence of small changes in closing hours for on-premise alcohol sales and assessed whether a possible effect is symmetrical. The study demonstrated that each additional 1-hour extension to the opening times of premises selling alcohol was associated with a 16% increase in violent crime. The quasi-experimental evaluation study drew on data from 18 Norwegian cities that extended or restricted the closing hours for on-premise alcohol sales with a maximum of 2 hours. Closing hours were measured in terms of the latest permitted hour of on-premise trading, ranging from 1 a.m. to 3 a.m. The outcome measure comprised police-reported assaults that occurred in the city centre between 10 p.m. and 5 a.m. at weekends. Assaults outside the city centre during the same time window functioned both as a proxy for potential confounders and as a control variable. The data spanned a period of ten years (2000–2010) and included 774 observations. Outcomes from the main analyses suggested that each 1-hour extension of closing hours was associated with a statistically significant increase of 4.8 assaults (95% CI 2.60, 6.99) per 100 000 inhabitants per quarter (i.e. an increase of about 16%). Findings indicate that the effect is symmetrical (extending or restricting). These findings were consistent across three different modelling techniques.

De Goeij and colleagues (2015) studied the implementation of a new alcohol policy in Amsterdam allowing alcohol outlets in two of the five nightlife areas to extend their closing times from 1 April 2009 onwards. The study found that a 1-hour extension of alcohol outlet closing times in some of Amsterdam’s nightlife areas was associated with 34% more alcohol-related injuries. They investigated how levels and trends of hospital reported alcohol-related injuries changed after implementation of this alcohol policy by comparing areas with extended closing times to those without. A before-and-after evaluation compared changes in alcohol-related injuries between intervention and control areas. Participant alcohol-related ambulance attendances were compared between control and intervention areas between 1 April 2006 and 1 April 2009 (respectively n=544 and n=499) and between 1 April 2009 and 1 April 2011 (respectively, n=357 and n=480). The results demonstrate that after 1 April 2009, intervention areas showed a larger change in the level of alcohol-related injuries than control areas [incidence rate ratio 1.34, 95% confidence interval (CI) =1.12, 1.61], but trends remained stable in all areas. This increase was only statistically significant for the following subgroups: 2.00–5.59 a.m., weekend days, men, individuals aged 25–34 years, and people transported to a hospital. However, the increase did not differ between subgroups with statistical significance.

The programme has been evaluated in randomized controlled trials in Finland, United Kingdom (2020), the Netherlands (2020), and in Italy (2016). Evaluations are underway in Estonia, Greece and UK (Clarkson et al., 2022). The intervention is also implemented in Spain (Lopez-Catalan et al. 2022) and Belgium. The latter two evaluations are not considered in the Xchange rating because of methodological issues.

KiVa was developed and first evaluated in Finland. Subsequently, two thirds of all Finnish comprehensive schools started implementing KiVa IN 2011. For this first evaluation, 78 schools were randomly assigned to intervention and control conditions. The first phase (2007-2008 with 8237 pupils, aged 10-12 years in Grades 4-6 in 78 schools) demonstrated significant reductions in pupil reported bullying and victimisation after one academic year (Kärnä et al., 2011). KiVa was found to significantly reduce (by 17-30% in comparison to control schools) both peer- and self-reported bullying and victimization. The odds of being a victim were about 1.5-1.8 times higher and being a bully 1.2-1.3 times higher for control school’s students than for an intervention school students.

Reductions occurred in all nine forms of bullying examined (including physical, verbal, and cyber-victimisation; Salmivalli, Kärnä, & Poskiparta, 2011). In phase two (2008-2009 with children aged 7-15 years, Grades 1-9) victimisation and bullying reduced by approximately a third for intervention schools. Increased empathy and self-efficacy in supporting and defending victims, and reductions in bully reinforcing behaviour were also reported (Salmivalli & Poskiparta, 2012). Furthermore, anxiety and depression decreased, peer perceptions improved (Williford et al., 2012) and school liking, academic motivation, and performance increased (Salmivalli, Garandeau, & Veenestra, 2012) (Clarkson et al. 2019).

A secondary analysis of this Finnish data (Garandeau et al. 2022) applied multilevel structural equation modelling analyses in pre-test and post-test (1 year later) data in the same sample (n=15,103, 399 control and 462 intervention classrooms from 140 schools). The study showed that KiVa had a positive effect after nine months on affective empathy, but not cognitive empathy, independent of students’ gender, initial levels of empathy, bullying, or popularity, nor of school type or classroom bullying norms.

The age group 10-12 years showed the best result in Finland (Unit 2 lesson curriculum) (Kärnä et al., 2011). Indeed, KiVa’s anti-bullying work has been found to be more challenging in Finnish secondary than in its primary schools (Kärnä et al., 2013).

The Italian RCT (Nocentini & Menesini, 2016) involved 2042 students in grades 4 and 6 (approx. 8 to 11 years old) in 13 randomly assigned school to intervention group (KiVa) and to control group (usual school provision). The study focused on different outcomes, such as bullying, victimisation, pro-bullying attitudes, pro-victim attitudes, and empathy towards victims. Multilevel models showed that KiVa reduced bullying and victimisation and increased pro-victim attitudes and empathy toward the victim in grade 4, with ES = 0.24 to -.40. In grade 6, KiVa reduced bullying, victimisation and pro-bullying attitudes, the effect was smaller, but also significant (d>= 0.20). The study showed that the odds of being a victim were 1.93 times higher for a control group than for intervention group.

The Dutch RCT (Huitsing, 2020) evaluated KiVa and Kiva+ among 4383 students in grades 3 – 4 (aged 8-9) from 98 schools who volunteered to participate in the research. The study collected outcome data at five time points over a period of three years. At the baseline, two-third of the participating schools were randomly assigned to the intervention group (KiVa or KiVa+, with an additional intervention component of network feedback to teachers) and one-third to the control group (waiting list, case as usual). The study showed that self-reported victimisation and bullying reduced more strongly in KiVa-schools compared with control schools, and with stronger effects after two school years of implementation. Moreover, it showed that the odds of being a victim were 1.29 – 1.63 times higher for control group, and the odds of being a bully were 1.19 – 1.66 higher than for KiVa students. There were no significant differences between KiVa and KiVa+.

The British two-arm waitlist control cluster RCT (Axford et al., 2020) involved 3214 students (aged 7-11) in 22 primary schools. The schools were randomly allocated to the intervention group and waitlist control group (usual school provision) with a 1:1 ratio. The outcomes targeted were student-reported victimisation and bullying perpetration, teacher reported child behaviour and emotional well-being, and school absenteeism (administrative reports). There was no statistically significant effect on the primary outcome of child-reported victimisation or on the secondary outcomes. The impact on victimisation was not moderated by gender, age or victimisation status at baseline. The trial found insufficient evidence to conclude that KiVa had an effect on the primary outcome. The programme has been rated as Promising by Blueprints for Healthy Youth Development based on the review of studies conducted worldwide.

The Talk about alcohol has been studied by making use of a longitudinal quasi-experimental design (Lynch et al., 2013; Lynch et al., 2015)

The first evaluation study used a QED, multilevel modelling and mainly descriptive statistics (Lynch et al. 2013). The study investigated change over time for the two groups by carrying out a self-report questionnaire survey of students at three time points across the school years 2011-12 and 2012-13. There was some variation in each responding sample, as some students will have been present or absent at different times, although there was a core group of 1924 in the intervention group and 1741 in the comparison group who responded to all three surveys; still more than the 1500 students at each time point that were calculated as required for robust analysis of change in outcomes over time. The positive response rate and very minor attrition over the course of the evaluation indicates programme loyalty in schools. Rates of change between the two groups for the onset of drinking, knowledge of alcohol and its effects; and frequency of drinking were studied.  Significantly fewer students in the intervention group than in the comparison had ever had an alcoholic drink by the time of the third survey (effect size 0.000), significantly greater increase for students in the intervention group on the knowledge component at T2 (effect size 0.17) and T3 (effect size 0.15), no evidence of a statistically significant difference in frequency of drinking (amongst those who drank alcohol) or in terms of prevalence of drinking to get drunk at this stage.

The second evaluation study consisted of a longer-term follow up based in the same data (Lynch, 2015) to determine whether the proportion of students in the intervention group who had ever had a drink was still significantly lower than in the comparison group; whether a significant difference in knowledge still existed; and if a significant difference in frequency of drinking and drinking to get drunk emerged when students were older. Only 18 out the 33 schools at baseline participated in the fourth survey (age 15-16). Over half of the participants dropped out: n=900 in intervention group at T4 (2142 at T0), n=1146 at T4 (2268 at T0) but no attrition or intent to treat analysis was conducted. Multilevel modelling showed significantly fewer students in the intervention group compared to control group had ever had an alcoholic drink by the time of the fourth survey but there was no evidence of a statistically significant difference in knowledge component nor in frequency of drinking (amongst those who drank alcohol) or in terms of prevalence of drinking to get drunk at this stage. Considering that both studies are predominantly based on regression models it is difficult to establish a nuanced picture of the effects of this intervention without having goodness of fit indices of these models and confidence intervals of the coefficients.