Feedback Informed Treatment

Why Measure Therapeutic Alliance?

As psychologists we often get in the same routine for years once we figure out what’s working for us. This post will hopefully help you find a new way of figuring out what is actually working for clients! Introducing the Agnew Relationship Measure- 5 (ARM-5), which is included in all NovoPsych subscriptions.

Feedback Informed Treatment (FIT) 

The ARM-5 is based on the concept of Feedback Informed Treatment (FIT), which involves routinely and most importantly formally seeking feedback from clients about the process of therapy and the working relationship with the therapist. Most NovoPsych users routinely measure outcomes (symptoms or psychological wellbeing), but a real opportunity lies in measuring therapeutic alliance. After all, therapeutic alliance is among the best predictors of outcomes.

I’m aware there is a diversity of views about FIT; some people swear by it and others have theoretical objections to measuring therapeutic alliance. I’m not here to evangelise, but feel that the ARM-5 can be a helpful tool for a therapist’s toolbox. I’d be interested to get your feedback on FIT, either way.

ARM-5 Quick Guide

Measures: Therapeutic bond, sense of partnership, and confidence in the therapy.

Helpful for: Gaining feedback from clients about how sessions are going.

Comparable measures: The ARM-5 is similar to the Session Rating Scale (SRS), but with better psychometric properties. For this reason it is recommended over the SRS.

Agnew Relationship Measure – 5

88866800-6438-4525-a097-6952eb3e9d54

Description

The ARM-5 is designed to be used close to the end of a therapy session at regular intervals (either every session, 2nd session or every 3rd session). Given that therapeutic alliance is among the best predictors of treatment success, this tool can help clinicians identify risk of dropout as well as track any ruptures/repairs in the alliance. During a course of CBT, the ARM-5 has been shown to be equivalent to other scales such as the Session Rating Scale (SRS) in terms of predicting client outcomes (Bouchard, 2018).

The ARM-5 assesses three dimensions of the alliance known to be important for treatment efficacy

1.   Bond

2.   Partnership

3.   Confidence in therapy

Results and Interpretation 

Below is a graph of the therapeutic alliance from session to session. You can see a rupture clearly in the fourth session, followed by a recovery. We can also see that the client isn’t feeling a strong sense of partnership with the therapist. That’s helpful to know!

a60339ab-4ac6-4b73-be07-f616f3a7f917

Each time the ARM-5 is administered NovoPsych will compute and graph scores automatically. Scores are presented as “mean scores”, indicating the average responses (from 1 to 7). In addition, a percentile rank is presented using the mean and standard deviation from the Cahil et al. (2011) sample, indicating the level of alliance compared to a normative sample. 

Three Components of Therapeutic Alliance

Bond (item 1) is the measure that encompasses the classic dimensions of client-therapist bond and feelings of therapist supportiveness. 

Partnership (items 2 and 3) measures agreement on tasks, and agreement on goals. 

Confidence (items 4 and 5) measures the client’s confidence in the treatment approach, as well as the perceived confidence the therapist has in their own techniques. Confidence has been identified as the strongest predictor of positive outcome, reflecting the therapist’s and client’s joint sense of progress and investment.

If you’d like to get feedback from clients but the prospect of administering the ARM-5 in session feels a bit awkward, then I’d suggest administering the Client Satisfaction Survey, a 10 item survey asking about a client’s experience of the counselling service more generally. Many psychology practices administer the Client Satisfaction Survey post discharge (or at dropout), which they find very valuable for improving their service.

I hope you find this information helpful!

References:

Jane Cahill , William B. Stiles , Michael Barkham , Gillian E. Hardy , Gregory Stone , Roxane Agnew-Davies & Gisela Unsworth (2012) Two short forms of the Agnew Relationship Measure: The ARM-5 and ARM-12, Psychotherapy Research, 22:3, 241-255, DOI: 10.1080/10503307.2011.643253

Practice-Based Evidence

Using DASS-21 Data for
Practice-Based Evidence

Some of the most positive feedback I hear from NovoPsych users is the function that graphs results over time so you can visually monitor your client’s symptoms changing from session to session. This is a valuable tool to build insight about individual clients, but wouldn’t it be great to see how your entire client cohort is doing?

The good news is that you can. Let me show you how one NovoPsych user had used DASS-21 data.

Dr Nathan Castle consistently used NovoPsych to administer the DASS-21 almost every session. Then, after a year of data collection, he exported the data and crunched the numbers. As the data below shows, there was a clear difference between DASS-21 scores in the first session compared to discharge. And the more sessions someone attended, the more they benefited.

So that’s excellent, Dr Castle’s clients are improving, but they might be improving by chance, right? How do we know that the benefit was actually statistically significant? Well there is a way to determine that, its called the reliable change index, and this is what it showed:

Psychological therapy effectivness

This is great news, 67% of clients are seeing reliable improvement! And for the real stats nerds among us, let’s take a look at Dr Nathan Castle’s effect size (Cohen’s D)

RCI DASS

This statistical information shows Nathan is a highly effective therapist. Of course, the key to learning how to improve any skill is feedback. NovoPsych helps clinicians get this feedback, either by symptom scales like the DASS-21 or therapeutic alliance scales like the ARM-5.

So where to from here? Well firstly I’d like to get your feedbackWould you be interested in having automated tools built into NovoPsych to give you a view on how your clients are doing as a group? At the moment, completing this analysis requires data to be exported from NovoPsych, but we would love to make the process simpler for you, if there is significant interest. 

Currently NovoPsych users have the option of doing the above analysis via a more manual method. The process involves exporting data from NovoPsych (currently a $110 fee) into your favourite statistics package. If you’re not statistically inclined you can contact Dr Nathan Castle who has offered his consulting services to NovoPsych users.

Feedback informed supervision

I hope you’ve found the above analysis of NovoPsych data helpful, or even inspiring. I can’t help but think the future for mental health interventions is not just evidence-based practice, but also practice-based evidence.

Take care,

Dr Ben Buchanan

Co-Founder and Director of NovoPsych.