RSAT Program Evaluations



EXAMPLE:  A TECHNICAL ASSISTANCE REPORT TO AN RSAT FUNDED FACILITY

This report follows two days at the treatment facility—a typical days due to both the admission of a large group of new clients with all that entails and the absence of the senior administrator. We, Janelle Prueter and Fred Zackon, spent roughly half of our time observing program operations and chatting with clients and staff and the other half learning about the programming through discussion with senior staff and reviewing program materials, curricula, etc. Prior to the visit, we had preparatory discussions with some staff members and reviewed program descriptors and materials made available to us by them and via the agency website.  
 

 


 

Commendations 

 
Several features of the program struck us as notably propitious, and could well become even more valuable in the years ahead.   
 
Committed Staff:  The staff, volunteers, and interns whom we met all seemed authentically engaged in their work and highly motivated towards doing their best. No less important, they are led by a senior staff that seems open to thoughtful adaptation and respectful of both staff and clients. 
 
Strong Community Support:  The senior staff have been resourceful and energetic in securing support from and ties to the community. A broad consortium of local non-­- profits, businesses, and public agencies have contributed with personnel and in-­-kind contributions. The agency has successfully pursued other grants and funding opportunities to fill the gaps inside and outside of the program, e.g., aftercare services; new transitional housing, etc. 
 
Calm and Respectful Environment: Staff and clients assert that open hostility is infrequent and threats or acts of violence all but non-­-existent. What this says about treatment effectiveness is unclear, but it is a good sign and it suggests that the peer community exercises significant social control. One contributor to the benign atmosphere might be that virtually all client activity, save for the bathroom, is visible from almost any point in the large room. Nonetheless, clients were adamant that even when occasionally their only nighttime supervision came from a CO in a glass office/cage outside the unit, all remained well.
 
A healthy approach to incorporating health in the program: A TC that says it is “faith-­-based” can have special appeal and power for many people. But it might be or grow insensitive to its clients and the needs of recovery too, by virtue of both the intense authority issues that often arise in TCs, and inflexible religious attitudes. The agency is not one of those. Treatment programs that offer spiritual practices and guidance to those who so chose, that explain clearly and perhaps demonstrate the value of faith, and that nonetheless respect individual spiritual beliefs or lack thereof—qualities we see in the agency—are worthy of emulation. 
 
A growing continuum of care: Again, senior staff leadership and community support are elaborating an appropriate set of treatment modalities: including family case management and counseling, and a newly established aftercare services.
 
A growing graduate peer group: The agency claims that for clients who have completed the entire program, 3 yr. recidivism rates are 20%. A small sample size and other factors (see below) caution against drawing conclusions from this outstanding measure. Even so, the roughly forty graduates constitute an invaluable resource for recruiting clients, building community support, enriching the therapeutic milieu, helping to train staff, substantiating and delivering program content, forming constructive bonds with clients, and helping to lead their way into mainstream society. We understand that many graduates already actively serve the program in some fashion, and many others could do so in ways yet to be determined.
 
 

 


 

Primary Recommendations

 
These recommendations propose steps that we think the agency would find timely, practical, and efficacious. They are each meant to build upon program features that have served the program well to this point. If a few agency practices are somewhat antiquated (e.g., wearing signs as a “learning experience”), we encountered nothing that appeared inappropriate or in need of corrective action.
 
Other than assigning them either primary or secondary status, we do not rank the recommendations by importance or by a sequence for implementation. Nor do we offer particular strategies for implementing the recommendations; that would go beyond the scope of our consultation. And the agency community can probably decide how best to proceed without much help. For tools that can help with several recommendations, however, we cite two institutional sources: Texas Christian University’s institute for Behavioral Research (www.ibr.tcu.edu/) and NiaTx (www.niatx.net). The former offers relatively simple and validated instruments for capturing crucial client data; the latter focuses on means and methods for fostering treatment programming change.
 
We only note that oftentimes big and important progress requires a conservative step­-wise approach.  Especially when, as with the agency, the current structure is sound and there is no urgent need for change. Paradoxically, perhaps, when conditions are stable can be a good time to move ahead, because stability allows for gradual adaptation. “Fixes” on the other hand, are often undertaken with more urgency and less deliberation. Seeing no need for big fixes, our sense is that the agency is in good shape to not only consider upgrades and enhancements, but also to start making them real. These, then, are our recommendations.
 
Streamline intake and assessment. Legal, mental health and medical eligibility is well documented and clear, but there appears to be a great deal of data duplication in intake, psychosocial, and case management. While getting the story more than once can turn up inconsistencies or get a more fine-­grained profile, it is unclear whether and how that impacts subsequent treatment. And separate clinical and case management files may be a barrier to integrating information. We recommend unifying and consolidating the process with fewer tools. We especially recommend using one tool that can be staged and also used for reassessment: the ASI for example or the TCU Intake instrument. To screen for substance abuse as part of eligibility process the TCU Drug Screen II might be a good choice. (It is free and normed on the CJS population.)
 
Consider recruiting “higher risk” offenders. Currently the agency screens the offender pool for highly motivated clients with relatively good family support and recovery prospects. Once admitted, these clients experience a high intensity program. Dominant voices in the treatment research and public policy arena these days cite evidence that more­-highly motivated and supported the prospective client, the less they may need intensive treatment. And, that needless long intense programs for those clients can lead to poorer outcomes for some, premature departures for others, and in other cases success that was likely without intensive treatment. Also, greater public savings from crime-reduction result from partial success with more challenging/higher-­risk offenders than greater success with low risk offenders. It might be prudent to partner with the Drug Court and bring in the Reentry Council to review other CJS treatment interventions, ones that might be implemented or adapted to work towards the best match between the client profile and the treatment particulars.
 
In any case the agency would only strengthen its impact with clear parameters for defining its target population, having specific admission criteria that are sensitive to the realities of that population, and interventions that are appropriate to that population. This might seem obvious enough, but it is neither simple nor easy in practice. Note: it is not necessary to stick to one narrowly defined target population; a community with a range of personalities and risk ratings has potential advantages. But continuing attention need be paid to how two or more “streams” of clients affect the program, and quick adjustments may be called for.
 
Condition phase advancements on individual achievement. The current system of cohort advancement inevitably reduces client incentives to do one’s best and degrades, at least a little, staff focus on individual achievement. Enabling a client to join a more advanced peer group, the approach favored in TCs generally, can create a positive culture of achievement. Reducing the necessary minimum LOS also frees up resources. Current best-­practice wisdom suggests that 6 months can be an acceptable minimum time in intensive residential treatment. THE AGENCY could establish, for example, 2 months as the minimum time per phase, anticipating that 2 to 4 months per phase would be the norm. (Clients could not only be held back pending progress, but set back should their behaviors warrant). Any move to reduce program duration implies consultation with the criminal justice system. An agreed plan is likely as long as appropriate treatment objectives per phase are clearly defined and clearly achieved, and that where necessary the client has a prolonged final phase of supervision in transitioning to life in the mainstream community.
 
Establish clear recovery-­based treatment objectives for each phase. The agency offers clients an array of useful written materials and exercises and related group discussion sessions: e.g. Change Company publications, Purpose Driven Life materials, relapse prevention activities. Client attention to the materials and group participation is tracked, but measurable learning and appropriate practice and use of the content is not. Nor is the learning consistently reinforced throughout the program environment. These are vital conditions for durable treatment gains. But they require that the core elements of a “recovery curriculum” be made clear to everyone; that the particular concepts, practices, and skills at issue be carefully defined; that the respective means to teaching and instilling them be identified and targeted; and that their achievement be made as measurable as practical.
 
Towards this end, we urge extended consultation among agency staff, knowledgeable associates, the local community of maturely recovering drug­-abusers, and if possible professionals who have researched developmental recovery among substance abusing offenders. Out of this work agency staff would determine specific sets of knowledge, skills, and so on that would be appropriate to each client phase.  While selections may need revisiting now and then, focusing hard on a sound set of unambiguous learning objectives will grow staff competence and even mastery in teaching them ever more effectively.
 
These core achievements—e.g., understand key scientific facts about addiction and important things known about recovery, participate regularly in a social activity that strengthens recovery and that you can continue after release, master at least 3 skills that can help keep you out of trouble and that you can do easily wherever you are—should be distributed across the treatment phases with some developmental logic. Consistent with current best practice standards, cognitive or cognitive behavioral skills—which are themselves of central importance for relapse prevention—need to be taught with robust modeling, rehearsal, and practice application. All staff and interns should be fully familiar with all the core requirements and how they are taught and the purposes they serve. Discussion about, comments upon, and lots of good, bad, and indifferent enactments or core practices and precepts should become prominent features of the agency as a learning community.
 
Strengthen and enhance staff, intern, and volunteer training. This also involves several components. For one, we urge developing a unified set of standard operating procedures that can be relied upon by all staff, interns, and volunteers.  Most procedures are already well defined. What’s needed is to consolidate and organize them for ready reference. We also see benefits from enhancing the “onboarding” process—i.e., building strong, capable, and concerted teams using standard procedures for staff orientation. We recommend creating protocols to observe staff and interns to ensure they are competent in their various functions—run groups, work with individuals, perform documentation, etc.—so they can be “signed-­-off” as competent to do them independently.
 
Crucial to any such training is attention to issues related to working with a criminal justice population and the boundaries that may apply to interactions with clients. We also heard requests for more attention to the challenge of being an agent of both therapy and institutional authority, which implies greater focus on the operational definition of “rational authority.”
 
We urge sharpening policies and practices around providing ongoing clinical supervision: when it happens, by whom, what’s expected, etc. While some of this in place—intern group clinical supervision, for example—more is needed. The ideal is to build and maintain a culture of clinical supervision and a regime of well-­-defined standards.
 
 

 


 

Secondary Recommendations & Suggestions

 
Deepen staff insight into the client experience. The need starts with client Orientation, which as we observed it, was over­-loaded with a premature run-down of the program’s many (necessary) rules and sanctions.  Focusing more on client testimony, Q and A, and various experiential engagement activities on Day One, Session One could help make Orientation more welcoming.  We also suggest exploring NiaTx’s approaches to simulating the client experience for staff by putting select staff through client processes to determine what is and what is not likely to have a desired impact.  We suggest that the first process to focus on be intake and orientation.
 
Once more TCU offers a useful tool. The Client Evaluation of Self in Treatment (CEST) tracks engagement, motivation, psychological functioning, social functioning and criminal thinking. It could be used in conjunction with mid-term evaluations to track progress and provide additional information for phase progression. Some sections combined into their Inmate Pre-­Release Assessment (IPASS) can be used to determine risk of relapse and recidivism prior to release into the community and then can inform aftercare plan. The Engagement scale also tracks effectiveness of counselors for supportive, not punitive, purposes. Not least we suggest if at all possible interviewing Interview clients that leave the program to help improve retention.
 
On the walls, less is more. The client­-produced sayings and aphorisms that cover most of the visible wall space seems overwhelming at first, but rather quickly fades to wallpaper. Rather like the whole being less than the sum of its parts. We suggest limiting signs and posters to key themes, identified core learning, and only the most outstanding inspirational or instructive sayings. To get continued value from the existing collection of sayings, perhaps they could all be catalogued as a menu of themes; have clients choose several for a seminar and/or add something new to the catalogue, which would always be available for client perusal.
 
Consider having more NA in lieu of AA meetings. NA is an important context for enabling clients to identify with the culture of abstinence and recovery.  Alcoholism recovery of course is real recovery and the necessary focus of many clients. But for many criminal justice system (CJS) substance abusers, groups dominated by older mainstream or middle-­class participants can be problematic. For most CJS folks, at least at first, NA fits better with its ready reference to poly-­-drug use and the criminal street life and most of all a more familiar set of personalities.
 
Ensure ready reference skills. Only professional researchers and scholars need to stay highly current in the field of substance abuse treatment. And given the time and possible expense it used to take, few practitioners could do more than make occasional efforts to catch up a little. No more, of course. Not only can Google Scholar, for example, turn up thousands of relevant web pages (and many full articles as well as abstracts), many leading on-line science publications have addiction or substance-abuse subsections, and there are dozens of professionally respected addiction and offender-specific journals, magazines, and newsletters. Although few if any important issues in our field might be fully “settled,” keeping an eye on the literature helps one stay humble, remain sensitive to important treatment factors, improve existing skills and practices, and learn more about what one is already doing and its impact. At least one staff person should be a capable and frequent investigator of findings about the science and art of substance abuse treatment and recovery.
 
To request follow-­-up on the above, please give either of us a call. You can also visit www.RSAT-­TTA.com for relevant reference materials, post questions to peers, and more. Our bios are there too, as well as contact information.