"STOP THE TOP": NTA RESPONSE... VS THE EVIDENCE

Dr Jason Luty revealed research in the March 2010 issue of Addiction Today showing that the National Treatment Agency’s Top Treatment Outcome Profile form is “criminally invalid”. The NTA’s Colin Bradbury gives his defence below - and Addiction Today editor Deirdre Boyd balances the scales by laying out some of the facts ascertaining the validity of NTA claims. Would readers sum up the cases with a verdict of “fit for purpose” or “criminally invalid”?

Read the pdf:
Download Addiction Today 124-NTA Top defence vs facts

"All the research evidence states that drug treatment is effective in reducing drug use and cutting crime," writes Bradbury, head of delivery for the National Treatment Agency for Substance Misuse. "This is clear from the National Treatment Outcomes Research Study – Ntors – in 2001 through to the Drug Treatment Outcomes Research Study – Dtors – in 2009. The purpose of the Treatment Outcomes Profile – Top – was not to supplement or second-guess this national body of knowledge, but to enable local clinicians to assess the progress of individual clients against their own care plan in the course of a treatment programme."

"There are so many inaccuracies and diversions in Colin Bradbury’s defence of Top and the NTA delivery that it is hard to constrain the corrections in one page. But, for transparency, let’s try," writes Boyd, editor of Addiction Today and CEO of the Addiction Recovery Foundation.

"The statement that Top should not supplement Ntors or Dtors is irrelevant to our case. But it does remind me of a grave distortion in both these surveys: that the outcomes of detoxification are mixed up with those from ‘rehabs’ or residential rehabilitation.  For the uninitiated, detox is merely purging the body temporarily of chemicals, like the ancient romans deliberately vomiting so that they could continue their orgies of eating and drinking. In good practice, patients are referred into treatment after detox. Such treatment can be rehab or other treatment indicated by a care plan.  So detox is not treatment.

"Yet both Ntors and Dtors confused the outcomes of detox with rehab, for reasons still to be satisfactorily explained. Clearly, when comparing this joint-outcome with methadone maintenance – the other option in Ntors – the gaps in outcomes were artificially narrowed.

"However, Ntors at least acknowledges that the most difficult, chronic cases went to rehab; Top is incapable of such diagnostic recognition.

Bradbury does go on to say that Top should “enable local clinicians to assess the progress of individual clients against their own care plan in the course of a treatment programme”. This it emphatically does not do... “The NTA continues to collect more and more statistics in order to hide the truth,” one hard-pressed CEO emailed me. “Year on year for the past five years, our agency – an abstinence based rehabilitation project – receives more calls from those suffering with addictive disorders and families stating that they have been told that rehab is not an option by their local Drug Action Teams. They cannot get help. They report that they are simply assessed, assessed and assessed again. Great for gathering statistics but poor service for people.”

Kathy Gyngell writes more on Ntors and Dtors in the May issue of Addiction Today.

REVIEW BY PEERS

Bradbury: The National Treatment Agency started developing this 20-item outcomes tool in 2006 and it was subsequently validated in the peer-reviewed journal Addiction. Top covers outcomes for the four key domains set out in national care planning guidance: drug use, injecting risk behaviour, crime and health, and social functioning.

Boyd: Let’s put to one side that the authors of the ‘peer review’ of Top included its own author, paid by the NTA in the first instance to produce it almost overnight – after all, if you cannot sing your own praises, who will?
The key question here is: what was it reviewed to be capable of? “Peer reviewed as fit for purpose,” the NTA proclaimed in 2008 in a taxpayer-funded PR programme. The trusting might think that “fit for purpose” means effectiveness in implementing Bradbury’s earlier statement. Perhaps that Top asks the best questions to monitor/feedback client progress? That Top questions elicit top clinical feedback? No... the review was not about that. Rather, it confirmed that the Top questions  are “fit for purpose” to elicit “reliable” rather than ambiguous responses.
Here it should be noted that Top is a self-report form without independent audit.  

LANCET PUBLICATION

Bradbury: As a by-product of this local process, Top does give us an extra insight into how treatment is delivered. National Top data submitted so far is already having a significant impact in increasing the international knowledge base of the effectiveness of treatment. For example, Top outcomes data for over 14,000 individuals with heroin and crack cocaine problems was published by The Lancet in a paper last autumn.

Boyd: The erroneous data published in the Lancet – written by NTA employees rather than independent reviewers, with “abstinence” meaning from only one drug not all drugs, for only a short time, in an approach costing 4-6 times more than some rehabs – can be found by clicking here. Where is the compliance to the hippocratic oath of “first do no harm”?

CRIME QUESTIONS

Bradbury: The stakeholder group which advised the NTA in the development of Top was always aware that crime questions in an outcomes tool designed for clinical use would be contentious. However, the NTA consulted with the British Medical Association, the General Medical Council, and the Department of Health’s Digital Policy Unit, responsible for protecting the confidentiality of patient records. All approved the crime items on the Top. Agreed minutes of these meetings are published on the NTA website.
The four items that are concerned with crime were carefully constructed and tested with over 1,000 clients in the process that was eventually published in Addiction, and was shown to be eminently workable in practice.

Boyd:  Dr Jason Luty covered this in the March issue of Addiction Today.  Dr David Best also published research showing how figures are massaged - for details, click here. One treatment provider CEO confirms his experience of what Best’s research states. “Bradbury misses the point re honest answering of Top forms. There is a vested interest in being dishonest.  Admitting to increased drug use or crime can result in consequences – being taken off methadone or told that they are not motivated and thus not ready for rehab. He comments on the relationship between clinician and service user. These forms act as a barrier as opposed to improving the therapeutic relationship. The forms contribute to the ‘sausage factory’ experience as opposed to real individualised care, listening and support offered in a therapeutic relationship.”

TOP DEPLETES RESOURCES

Boyd: "Bradbury states that this tool was to enable local clinicians to assess progress. So an audit ought to be undertaken asking local clinicians if it is helping to assess progress,” one typically aggrieved treatment-centre CEO wrote to me. “It certainly does not assist our agency in assessing progress. It wastes time that could be spent doing worthwhile activity with service users. The only local clinicians who speak up for it are those whose funding depends on keeping Drug Action Teams and the NTA happy as opposed to their service users. Dare I suggest that these agencies are well skilled in massaging figures.”
Other organisations have had to hire extra admin staff to administer the Top form, so that precious resources are drained from their core work of helping patients to recover.
“We have to fill in over 1,000 Top forms on a regular basis,” said one NHS clinical director. “It costs us money. That might not be so bad if only Top fed back some useful information to us which could help our patients and our procedures. It does not give us any data.”

ASKING QUESTIONS

Bradbury: Dr Luty claimed that his audit of the Top showed “patients are simply not answering the questions properly”. I would suggest that instead of blaming his patients for not answering questions properly, Dr Luty should consider whether the questions around crime – and all other domains – are being asked in a sensitive and appropriate way.
The NTA has always stressed that no client should be forced to give information that they do not wish to. However, if an individual tells a clinician two pieces of information which are obviously contradictory – for example, that they are not working or committing crime, but are spending £900 a month on drugs – then this should be explored further.
The fact that an individual does not feel able to tell the truth is important clinical information and should not be glossed over.
In cases where a client persists in giving contradictory information such as this, the NTA’s guidance is clear: rather than submitting obviously false information, “Not Answered” should be entered for the appropriate Top item.

Boyd: This is already addressed under the heading of Crime Questions. 

WHAT OUTCOME MEASURES?

Bradbury: A recent Ipsos Mori opinion poll disclosed that the public’s chief expectation of drug treatment is that it should reduce crime and antisocial behaviour in their area. There is already a significant body of evidence to say that drug treatment helps reduce acquisitive crime.
Even if Addiction Today got its wish to “stop the Top,” any national outcome measure will inevitably rely on the clinician and client having an honest and open conversation on what is happening in that individual’s life.
Given the tough public spending decisions that lie ahead, I would suggest that unless the treatment field is able to demonstrate its worth in delivering meaningful outcomes for both the client and the community (including a crime-reduction dividend), then the unprecedented increases in funding that have been forthcoming in the past few years may well be at jeopardy. We know that the public expect nothing less.

Boyd: To address this crisis, Addiction Today/the Addiction Recovery Foundation is working with a select number of think-tanks to create valid outcome measures. We have been offering our expertise and time free of charge - a fraction of the £20million feeding into the NTA annually.
I leave the last word to another reader:  “I hope that, when Colin Bradbury makes threats about the field demonstrating its worth, he includes the NTA. The easiest way to save money without affecting outcomes is to disband this counterproductive, expensive quango.”

Comments

[the step aside consultancy]

I read this with great interest. Until 2006 I worked for a very well known & established organisation. Part of my job was to manage the outcomes. I was told to 'massage' the figures!! in order to satisfy the DAT & NTA at this point I resigned! after all those helping addicts are supposed to be promoting a certain modicom of honsty.
My organisation, totally independent & specialists in the treatment of alcoholism, currently has a success rate of 53%. Should anyone query these numbers then all they have to do is speak to the clients who have experienced our programmes. TOTAL HONESTY with no recognition from our PCT.
Regards.
Thalia Shannon-Eyers.

[JD]

I was working for the organisation that was involved in designing and piloting the TOP form and went to various training days organised in order to elicite apparently discussion from staff as to what they thought about the early top form. There was little, if any, enthusiasm and frankly out spoken disdain of this insipid tool. It was clear that nothing we said was going to be taken on board, even points like measuring outcomes for going to rehab or complete abstinence and quality of life without drugs or social support networks in peoples lives.
It seemed to be more an exercise in back patting between the TOP delevopment people and the aforesaid organisation. note these training/introducions to tops were held at the organisations head office.
This provider is so well known and powerful now and has very close links with the nta and apparently a turn over of 60 million per year (Said by on of the CEO's at a staff induction day).