Addiction Today: "WHAT ARE YOUR SUCCESS RATES?"

THE BROADWAY INVENTORY HAS ANSWERS

Treatment organisations are often asked about their success rates – but what
is success? Peter Smith and Dr Jennifer Parker trialled a tool which measures the development of characteristics playing a significant role in the acquisition and maintenance of quality recovery. They share their findings.

Print-friendly version:
Download AddictionToday-Broadway Inventory introduction

The Broadway Inventory itself:
Download Broadway Inventory

Almost without fail, commissioners or other interested groups coming in contact with our organisations for the first time are likely to ask ‘what is your success rate’. Without much verifiable, objective and statistically meaningful data to refer to we may well offer retention rate as an implicit measure of ‘success’, on the basis that ‘completion of treatment’ is a good predictor for this.

However, whilst this measure may be simple and easily understood, it can hide a myriad of variables beneath the surface that make it meaningless as an instrument for comparing treatment programmes, if this is what is really being sought.

It may be of some use for an individual organisation wishing to make broad comparisons from one year to another, especially where programme changes have been made, but the measure does not take in to account differences between centres where length of treatment programme and admission or discharge policy, client profiles or size of unit may vary. In addition it may also be open to a subjective interpretation of the term resulting for instance in a service user leaving a programme prematurely determined as an ‘early completion’ rather than a non-completion.

With this in mind and in an attempt to deliver a more meaningful treatment impact measurement, for a number of years Broadway relied on the Christo Inventory for Substance-misuse Services (CISS; George Christo, 1999). Our original belief existed that CISS might become a universal outcome measurement tool, the results from which commissioners would rely on and might be required by some for the purpose of contract compliance. This choice was supported by the additional suggestion that it could act as an internal auditing process.

After a number of years of systematically carrying out the exercise with all service users, whilst fighting a rear guard action from counsellors questioning why they were doing it and having never been asked by care managers, commissioners or anyone else about either an individual’s CISS score or outcomes measured by it, the collective decision was made to abandon the practice and apply the time saved to therapeutic activities.

As an organisation that delivers abstinence-based residential treatment programmes ranging from one week detoxification and up to 24 weeks' treatment (combined first- and second-stage care phases), the CISS was of limited value. But, at least CISS is optional and some providers choose to continue using it as it proves useful for their particular needs.

The introduction of TOP by the NTA in 2007 has done nothing to improve the situation and to help service providers deliver any meaningful information for purchasers and neither has it helped providers in evaluating the impact of treatment programmes. As a broad measure of progress over time or for the NTA’s own purposes, perhaps it has some use, but for anyone else it is of little value and I believe that if organisations were given the choice, few would choose to continue using it.

With this background in mind we began to explore ways of identifying the impact of our treatment approach on the individuals we were working with. The weekly meetings with service users and exit questionnaires were already in place to gather information regarding routines and specific programme elements, but missing was a method for measuring the extent to which the treatment being delivered might help individuals to change in ways that would support their recovery goal. The result of this process was the Broadway Inventory (BI), a tool aimed at measuring the development of characteristics that play a significant role in the acquisition and maintenance of a quality recovery.

The intention was to create a simple, easily understood, self-administered format that could be presented on one sheet of paper, completed within a few minutes and in so doing, maximise return rates. With limited administration, bureaucracy, counsellor time or patient time requirements, counsellor bias was removed and patient compliance was minimised. Ultimately what appears to have emerged is an instrument that has a variety of uses and valid in a variety of settings.

Firstly, the BI can be used as a therapeutic tool with the client. Following completion of the ‘exit’ half of the BI and by bringing both portions together to one of the last sessions, the counsellor can show the client, in a very clear and visual way, just how much progress they have made.

Secondly, the BI can be used to identify the areas where further work might need to be directed in any continuing treatment programme and therefore aid the care planning process.

Thirdly, outcomes from the BI collected from a significant number or over a period of time can be used to identify programme strengths and weaknesses in order that resources can be specifically targeted at programme elements that might benefit from enhancement.

Fourthly, information gathered from the BI can be used to produce an individual or organisational outcome ‘factor’, the Treatment Impact Factor (TIF), which could provide commissioners and other interested parties with an objective measure of treatment effectiveness. This may prove to be useful if ‘Payment By Results’ becomes a reality.

An initial study carried out with patients at Broadway Lodge, using an early version of the BI indicated that it could deliver useful and reliable data. With the support of Dr Jennifer Parker from University of the West of England (UWE), an analysis of data collected from the initial version of the BI demonstrated its validity in identifying treatment impact features. However it also emerged that additional information could be retrieved with an analysis of factors such as age, gender, length of treatment and addiction or drug ‘type’. We are also of the belief that with amendments that have since been made, the BI would be applicable across different addictions, a range of treatment modalities and as relevant to individual counselling session(s) as a longer lasting treatment programme. Further considerations may result in the BI being developed for wider application including its use with family members.

Broadway Inventory's preliminary findings and description of the service users who completed it:
Download AddictionToday-Broadway Inventory introduction.

Advantages of the Broadway Inventory

The BI has been shown to capture the subjective change in service users' perceptions of themselves, and the world they will re-enter, as ‘recovering’ people. Each person’s individual responses can be used to formulate a plan of action for the months, or even years ahead. Areas of strength and areas for improvement can be clearly identified and used to support the client and provide further help where necessary. This pilot study shows two broad arenas that should be considered in early recovery: a) attitude towards society/the world and b) attitude towards the self.

There was some indication that women may have different issues to their male counterparts, particularly in relation to using cognitions and self worth. However, there was a ratio of 2:1 men to women. This may have affected scores, and possibly significant differences that have been described. Conversely, in direct relation to the previous comment, this data set has high ecological validity and is a fair description of the treatment population generally. Men usually outnumber women in treatment facilities (for all sorts of reasons). Based on these findings we strongly advocate using the BI as part of the admission and discharge procedure.

FUTURE DIRECTIONS

Based on these preliminary findings two courses of action have been taken. Firstly, two further scales have been added to the BI: ‘Self-belief’ and ‘Trust levels’. It is hoped that these additions will help further improve the reliability of the BI and bridge the gap between ‘self’ and ‘the world’. Secondly, a ‘Treatment Impact Factor’ (TIF) has been developed to score the outcomes for measurement purposes with a proposed qualitative judgement of outcomes for standardisation purposes.

A larger trial with the addition of a control group is currently underway to further develop and refine the BI and the TIF. In addition, we will be working on the development of a computer programme that will provide users of the BI with an aid to scoring and analysing data to measure the impact the treatment process has had and to help identify links and correlations that might otherwise not be so apparent.
Finally, we assert that the TIF is a meaningful and manageable measure of treatment impact and the BI a valid measure of second order indicators of recovery (Dayton, 2007). Therefore, the BI allows access to the person’s desire to grow and develop rather than first order changes, such as reduction in using or in illegal or other behaviours, as seen in previous measures of impact evaluation. The final statement must reflect the idea that the service user’s internal world is the best indicator of whether they will successfully recover from chemical dependency: the BI can be used to structure and measure this and to help the client move forwards from treatment.

REFERENCES

Christo, G. (1999). CISS: keeping it sweet and simple. Addiction Today Vol 11, No 61, pp14 – 15

National Treatment Agency (2007) Treatment Outcome Profile (TOP). London: National Treatment Agency for Substance Misuse

Dayton, T. (2007). Emotional Sobriety: From Relationship Trauma to Resilience and Balance. Florida: Health Communications Inc.

Procedure

Any new client should be provided with the BI as near to the beginning of the treatment process as is possible and given a second copy of the BI to complete as near to the end of treatment as possible.

The definition of terms is provided to help limit the involvement of a third party, and is therefore intended to be basic without being patronising, whilst still remaining meaningful.

The process by which the completed forms are collected is at the discretion of individual organisation and instructions for this should be made clear. Some minor changes to the BI may need to be made by organisations for this purpose.

The Treatment Impact Factor (TIF) is calculated by adding together the total number of points of progress made along the continuum for each of the ten factors. As there are ten points for each of the ten factors, this will result in a percentage change. Where there is a negative movement, this should be subtracted in the calculation.

TIF score:     0 – 10%    notable change
                 10% - 20%    significant positive change
                 Over   20%    substantial positive change

A more close examination and analysis of data collected through the BI may be helpful in identifying treatment impact for sub groups and in revealing patterns and correlations that might otherwise go undetected.

The BI is available from the download page at www.broadwaylodge.co.uk

Any comments, feedback or questions regarding the BI will be most welcome and can be addressed to petersmith@broadwaylodge.org.uk

The Broadway Inventory
(BI)

Initials…………… Male or Female Age ……. Date ………………..
(please circle)

Type of addiction ………………………….. Admission or Discharge
(if drug, put MAIN drug of choice) (please circle)

Physical Health
Healthy                                                                                                                    Unhealthy
l……….l……….l……….l……….l……….l……….l……….l……….l……….l
Anxiety Level
High                                                                                                                             Low
l……….l……….l……….l……….l……….l……….l……….l……….l……….l
Self Belief
Strong                                                                                                                         Weak
l……….l……….l……….l……….l……….l……….l……….l……….l……….l
Cravings
Frequent                                                                                                                      Rare
l……….l……….l……….l……….l……….l……….l……….l……….l……….l
Social Functioning
Good                                                                                                                            Poor
l……….l……….l……….l……….l……….l……….l……….l……….l……….l
Attitudes
Positive                                                                                                                  Negative
l……….l……….l……….l……….l……….l……….l……….l……….l……….l
Self Worth
Low                                                                                                                            High
l……….l……….l……….l……….l……….l……….l……….l……….l……….l
Trust Levels
High                                                                                                                           Low
l……….l……….l……….l……….l……….l……….l……….l……….l……….l
Self Management
Good                                                                                                                          Poor
l……….l……….l……….l……….l……….l……….l……….l……….l……….l
Outlook on Life
Negative                                                                                                                  Positive
l……….l……….l……….l……….l……….l……….l……….l……….l……….l

Guide To Completing This Form

This information will be treated in confidence and will be used to help improve our treatment services.

Please put a cross on each of the lines on the other side of this page to show where you are at the moment in relation to each heading.

PHYSICAL HEALTH
This is about how healthy you are from a medical point of view.

ANXIETY LEVEL
How anxious are you in yourself as a general rule.

SELF BELIEF
How much belief you have in your ability to succeed or achieve things.

CRAVINGS
How often do you have a strong desire to use or act out the addiction.

SOCIAL FUNCTIONING
This means how well you get on with other people, how you feel about meeting and talking to new people or how good you are at forming relationships with others.

ATTITUDES
This relates to your level of willingness to consider different ways of looking at things, to be open to compromise and accept the views of others.

SELF MANAGEMENT
This is about how good you are at looking after yourself and planning and dealing with things in an organised and responsible way.

SELF WORTH
This is about how you see yourself and your belief about how other people might see you.

TRUST LEVELS
The level of confidence or belief you have in other people, a Higher Power or the treatment programme to be able to help you.

OUTLOOK ON LIFE
This refers to the view you have about your situation and what you believe the future holds.

Once completed please return to Reception. Thank you for your help.