Family dysfunction ruins lives. We all know this; many of us will have personally seen its corrosive effects. Even if we’ve been fortunate enough to avoid this personal knowledge, we can’t escape the effects staring up at us from the pages of our newspapers. Two of the problems that have received the most attention in recent years are child abuse and neglect, and conduct disorder (severe anti-social behaviour) in children. Although these problems are frightening in their severity and scale, the good news is that although we can’t solve them completely, we can do something about them. Early intervention programmes offer a chance of breaking, even preventing, their cycle, if the programmes are truly effective. Unfortunately, not all of them are.
We’re all familiar with the problem of abuse and neglect, but conduct disorder may need a little more introduction. Experts define it as “antisocial, disruptive, aggressive, dishonest and related behaviours” in young children. The significance is that these behaviours “are precursors of a wide range of adverse outcomes in adulthood,” such as “criminal offending, imprisonment, alcohol and substance abuse, teen pregnancy, mental health problems, suicidal behaviours and poor physical health. In addition, these problems are associated with limited educational achievement, higher welfare dependence and limited earnings.” Conduct disorder is found in around five to ten percent of children, and 15 to 20 percent of Maori children. It’s an expensive problem—“the public cost of providing services to children with severe conduct problems is about 10 times the cost for children without conduct problems.”
The broad social impact of these problems is obvious, and will touch us all in some way. In addition to the compassion we should feel for children brought up in these environments, considerations of economics and social stability also compel us to act.
Recognising the urgency of the problems and the growing political interest in intervention programmes, we investigated interventions that have the greatest potential to respond to the twin problems of conduct disorder and abuse and neglect. We wanted to identify types of interventions that can be effective at treating these problems, and whether specific programmes currently funded by the Government are actually effective.
We chose to focus on two programme types—home visitation and parent management training programmes. These programmes are capable of being very effective, and can be used with young children for early intervention. There are other types for young children that are effective, and still others that can be used for later intervention. All these types should also be included in an overall package of interventions, but it was outside the scope of our investigation to consider all of them in detail.
We found that the Government is currently funding several home visitation and parent management training programmes. However, the bulk of the funding (around $37 million) goes to programmes that have not been shown to be effective. The largest recipient is Family Start, a home visitation programme. By contrast, only about $2 million goes to effective programmes (though there was some additional funding that couldn’t be quantified).
Underpinning this research was the question of programme evaluation. Evaluations are what reveal whether a programme is really effective; that is, whether it really makes a difference to the outcomes it is supposed to. Rigorous evaluations therefore tell us whether a programme is worth investing in and continuing, or whether it is a waste of resources. When it comes to measuring a programme’s effect on outcomes, a randomised controlled trial is generally the best test of effectiveness. However, most of the New Zealand programmes we reviewed either had not been evaluated with sufficient rigour to prove effectiveness, or the evaluations that had been done did not show promising results.
An intervention called Scared Straight provides a good case study of why we should be careful to conduct rigorous evaluations. The Government’s Advisory Group on Conduct Problems describes it this way: “In this programme young offenders were exposed to the realities of prison life in an attempt to ‘scare them straight.’ Initial, uncontrolled evaluations of the programme provided positive reports and all participants were of the view that the programme was beneficial. It was only when the results of randomised trials became available that it became apparent that the programme was, in fact, harmful and increased risks of crime.”
So what makes an effective home visitation or parent training programme? To begin with, such programmes should be evidence-based. That is, they should be based on research into the causes of the problems they target, and they should be subject to further research to show that they actually make a difference to those problems. Programmes should pass this test of design and evaluation before they are widely implemented.
When they are implemented more broadly, they should generally be implemented with fidelity to the design that has been proven to work. However, some degree of programme flexibility can be beneficial, particularly with home visitation programmes, as it allows programme workers to respond to the particular needs confronting them. It is also important that the programmes, as designed, are capable of being implemented and being effective in “real world” settings, as opposed to simply being effective in tightly controlled clinical trials.
When programmes are rolled out more widely, this must be done carefully and enough time must be allowed for implementation to be done well. A recent independent review of Family Start found that its “implementation was rushed.” The reviewer commented that, “Ministerial expectations of speed drove the process,” and that, “The variability in programme fidelity today can be linked to the speed of the initial roll-out and consequent expansions. Programme specifications were not well-developed at the beginning … This was compounded by additional expansions to a programme that officials reported clearly needed improvement.”
Programmes also need to be adequately staffed by appropriately trained and qualified professionals. A 2005 evaluation of Family Start found that only 40% of staff surveyed had a “recognised educational qualification” at that time, though it should be noted that the Ministry of Social Development introduced new requirements for staff to have tertiary qualifications following that review.
Programmes should also be capable of engaging the families they seek to serve in the programme, so that parents are motivated to change. However, all programmes face the common difficulty that “those most in need tend to be the most difficult to engage in treatment, and the first to drop out.”
It also seems that the best home visitation and parenting programmes are family-focused, rather than focusing solely on parents or solely on children. They should also be appropriate to the culture of the families they seek to serve; in some cases this may mean adapting a programme. Finally, they should be cost-effective, although sometimes this is difficult to measure precisely and there may not be any guarantees that cost-effectiveness will hold for each individual participant. Ideally, however, it would be possible to see that a dollar spent on the programme saves more than a dollar in later social costs that the programme has prevented.
Despite its importance, programme funding is not always closely connected to evidence of effectiveness from rigorous evaluations. In a lecture hosted by Maxim Institute, Professor David Fergusson spoke about “barriers to the implementation of evidence-based programmes.” Professor Fergusson is the developer of Early Start, a Christchurch-based home visitation programme which has demonstrated effectiveness in a randomised controlled trial. Early Start is recognised internationally as an exemplary programme. An independent New Zealand reviewer recently described Early Start as a programme that “maintains a strong link to research, has set benchmarks for itself and focuses on consistent delivery and ‘fidelity’ to programme design.”
Professor Fergusson identified three barriers. The first is “public service practices,” the way in which government agencies develop programmes. He described it like this: “A whole group of people, who are variously known as stakeholders, are invited in to the government to give their advice about what works. Everybody’s view is treated as being equally valid, so those who have got the evidence are just as valid as those who have got strong opinions, and all of this knowledge is written on a whiteboard by a junior policy analyst …. the whole pastiche is mixed into a report, and tipped out onto the plate in an ugly, greasy stew which is called policy. Then, having done that, what happens next is, ‘cause government agencies don’t implement programmes, they actually invite providers, who actually were never part of the development process, to tender for this programme and run it. And after that, someone says ‘well what about some evaluation?’ That’s how Family Start was developed, exactly. That’s the exact trajectory. I have caricatured it a bit, but those are the contours.”
The second barrier relates to research practice. Researchers, said Professor Fergusson, get their professional reward from journal publications, not from making policy. Because of this, and because of “the difficulties of dealing with the [policy] environment,” researchers do not often “engage vigorously with government agencies … And not only that, if you actually upset too many people you might get your funding threatened.” The end-point of research has to change, he said, from simply publishing a paper, to “publish[ing] a paper to change the world, to move things forward.”
The third barrier involves cultural dimensions. Professor Fergusson identified “a big divide about methodology, about ownership, whether Western science can be validly applied to Maori, Pacific and Asian [populations].” Here, there is no easy resolution to hand.
In addition to these barriers, when the Government is footing the bill, Ministers will want to make political capital out of the programme. However, careful, incremental programme development and evaluation is not the stuff of banner headlines. As the Family Start experience suggests, the pressure to do something now to justify re-election, exacerbated by a short electoral cycle, creates the temptation to rush the programme at the expense of programme quality. This has to change. Professor Fergusson offers a salutary caution: “Unless we move things forward, we are wasting money, and we are wasting the public’s hope.”
To read Maxim Institute’s report, Broken Boughs: The role of effective family interventions, click here: http://www.maxim.org.nz/files/pdf/broken_boughs.pdf
To listen to Professor David Fergusson’s lecture, Stepping In and Stepping Up: Effective interventions for conduct problems in childhood, click here: http://www.maxim.org.nz/index.cfm/media/article?id=1966