Outcome Data

"The data are always friendly" - Irene Elkin  

Below are a sample of my clinical outcome data since 2011.  I support the "open data" movement towards making clinical outcome data available to consumers, payers, policy-makers, and researchers (after identifying information has been removed from the data).

Below is a poster that describes a representative sample of my psychotherapy outcome data from two years of practice at the University of Alaska Fairbanks Student Health and Counseling Center.  This poster was created by Data Visualization expert Andy Kirk


Tony Rousmaniere Therapy Outcome Data

2011 Private Practice data (San Francisco & Palo Alto, California)

  • Total # of clients in dataset: 42
  • Avg # of sessions: 8.58
  • Total # of clients with more than one session: 38 (90%)
  • Single session clients: 4 (10%)
  • Start in clinical range (of clients with 2+ sessions): 25 (66%)

Clinical Outcomes

  • Ended Year in Clinical Change: 22 (88% of clients starting in clinical range)
  • Ended Year in Clinical Recovery: 20 (80% of clients starting in clinical range)
  • Ended Year in Clinical Deterioration: 2 (8% of clients starting in clinical range)
  • Average change in ORS score: 7.52  (clients starting in clinical range)
  • Cohen's D effect size for clients starting in clinical range: 1.22.  (Pre-tx std dev=6.14, mean intake ORS=22.27, mean last session ORS=29.79, n=38)

Duration of Treatment

  • Avg # of sessions for low distress clients: 5.27
  • Avg # of sessions for high distress clients: 10.74
  • Avg # of sessions for clinical change to occur: 3.27

  

2012-2013 University Counseling Center data  (University of Alaska Fairbanks)

Outcome Questionnaire-45 (OQ-45) data

  • Total # of clients in dataset: 102
  • Avg # of sessions:  4.60
  • Total # of clients with more than one session: 73 (72%)
  • Single session clients: 29 (28%)
  • Start in clinical range (of clients with 2+ sessions): 53 (73%)

Clinical Outcomes

  • Ended Year in Clinical Change: 24 (45% of clients starting in clinical range)
  • Ended Year in Clinical Recovery:  13 (25% of clients starting in clinical range)
  • Ended Year in Clinical Deterioration: 3 (6% of clients starting in clinical range) 
  • Average change in OQ score: 15.06  (clients starting in clinical range)
  • Cohen's D effect size for clients starting in clinical range: .82.  (pre-tx std dev=18.25, mean intake OQ=86.30, mean last session OQ=71.24, n=53)           

Duration of Treatment

  • Avg # of sessions for low distress clients: 4.24
  • Avg # of sessions for high distress clients: 4.78


Outcome Rating Scale (ORS) data

  • Total # of clients in dataset:  89
  • Avg # of sessions: 5.37
  • Total # of clients with more than one session: 65 (73%)
  • Single session clients: 24 (27%)
  • Start in clinical range (of clients with 2+ sessions): 42 (65%)

Clinical Outcomes

  • Ended Year in Clinical Change: 20 (48% of clients starting in clinical range)
  • Ended Year in Clinical Recovery:  11 (26% of clients starting in clinical range)
  • Ended Year in Clinical Deterioration: 4 (6% of clients starting in clinical range)   
  • Average change in ORS score: 4.84  (clients starting in clinical range)
  • Cohen's D effect size for clients starting in clinical range: .97.  (pre-tx std dev=4.98, mean intake ORS=15.63, mean last session ORS=20.47, n=42)

Duration of Treatment

  • Avg # of sessions for low distress clients: 5.61
  • Avg # of sessions for high distress clients: 5.24
  • Avg # of sessions for clinical change to occur: 2.60
  • Maximum # of sessions for clinical change: 5.00


2014-2015:  I do not have data for these years because my counseling center at the University of Alaska Fairbanks switched to a different outcome measurement program (the Counseling Center Assessment of Psychological Symptoms - CCAPS) which does not facilitate analysis of outcome data at the clinician level.  The CCAPS was a better clinical tool for helping our clients, because it uses data specifically for university counseling centers.  


2015-2016 Private Practice data (Seattle) - Preliminary
I moved to Seattle and started a private practice in the Fall of 2015.  Following is the data from my practice as of the end of the summer of 2016.  The dataset is still small (n=8 clients) because (a) I work part-time as a therapist, (b) some clients have not completed treatment, (c) some clients have opted-out of having their data included in this analysis, and (d) I do not include data from clients who start treatment above the "clinical cutoff" (see below for an explanation.)  This data should be considered preliminary because there are not sufficient data for a statistically reliable gauge of the work.  Ideally, there should be data from at least 30 clients to provide statistical reliability.  This data will be updated annually.

The outcome measure I used from 2015-216 was the Behavioral Health Measure-20 (BHM-20). 

  • Clients in data: 15
  • Clients who started in clinical range (below "clinical cutoff"): 9 (60%)
  • Range of sessions: 2 to 50
  • Clients with more than one session: 100%
  • Single session clients: 0

Clinical Outcomes

  • Clients who met the BHM-20 "recovery" benchmark: 4 (44% of clients starting in clinical range)
  • Clients who met the BHM-20 "improved" benchmark: 7 (78% of clients starting in clinical range)
  • Clients who showed no clinically significant change on the BHM-20: 2 (25% of clients starting in clinical range)
  • Clients who deteriorated, according to BHM-20 scores: 1 (12.5% of clients starting in clinical range)
  • Average change in BHM-20 score: .52 

(I was unable to calculate a Cohens-D effect size from the BHM-20 data.)


How I Practice

Individuals are about 90% of my practice, and couples about 10%.  The therapeutic modalities I primarily use are Short-Term Psychodynamic Psychotherapy (STPP), Intensive Short-Term Dynamic Psychotherapy (ISTDP) and Cognitive-Behavioral Therapy (CBT).

Outcome Measurement and Client Response

I have used two outcome measures.  The Outcome Rating System (ORS) is an ultra-brief measure that is free to use.  Information about the ORS is here and here.   Clients completed the ORS at the beginning of each session, on a laptop computer or with paper/pencil.  It takes about 30 to 60 seconds to complete. 

I also used the Outcome Questionnaire-45 to track outcomes (OQ-45).  Information about the OQ-45 is here.  Clients completed the OQ at the beginning of each session on a computer in the waiting room.  It takes about 3-5 minutes to complete.

Consent for tracking outcomes is obtained at the beginning of treatment.  This is voluntary for clients, and about 5% of clients refused consent.  Consent includes agreement for the outcomes to be used for research and publication, after being de-identified.

After reviewing my cases, I consider the ORS and OQ results to be valid for about 85% of my cases.  For a few cases I think both measures underrepresented the amount of positive change, especially for clients who started treatment with less clinical distress.  For a few clients the ORS overrepresented the amount of positive change, in my judgement.  

When I asked clients their thoughts about the outcome measurement, most clients were curious about their results but said they gave more weight to my subjective clinical impression of their case.  A few clients were skeptical of the validity of standardized outcome measurements, and a few clients refused to complete the measures because of this concern.   In studies, clinicians have reported that a leading reason for not collecting outcome data is that clients do not want to, or concerns about harming the therapeutic alliance.  I did not find this to be the case, but rather found that discussing outcomes with some clients strengthened the therapeutic alliance.  My guess is that clinician attitudes towards the outcome measures strongly affect clients’ reactions to the measures.

What About Change for "Low-Distress" Clients?

There are no outcome measures that are good at assessing the amount of change in psychotherapy for clients who start therapy with low levels of distress, termed the "clinical cutoff" in the psychotherapy research literature.  (Clients who start therapy above the clinical cutoff of 25 points on the ORS or below 64 points on the OQ).  For this reason, I do not have reliable quantitative data on the amount of change for low-distress clients.  

Limitations of the Data and Outcome Measurement

It is difficult to assess how good these results are, because there are no available data from therapists working in the same locations as I worked.  Likewise, it is not possible to compare my 2011 outcome data to my 2012+ outcome data because the location and populations were so different.  For example. it is possible that the $150 out-of-pocket fee most of my clients paid for therapy in private practice increased their motivation for change, thereby boosting the outcome data.   Another validity concern is that I do not yet have follow-up data, so I do not know if gains will be lasting. 

Therapy outcome measures are not perfect.  Multiple clients report negative change on outcome measures when in fact their symptoms improved.  Likewise, paper forms can never describe the rich complexities of growth and healing in therapy.  Given these limitations, I still think it is important for therapists to track their clients' outcomes with unbiased measures.  While the measures should never be used alone for clinical decisions, they are one important source of information regarding client progress or deterioration, and are can be helpful for assessing overall therapist effectiveness, when combined with a range of other assessment tools.


Popular outcome measures

Online forums and communities that discuss outcome measures and provide training

Software to help measure outcomes



Writing on measuring psychotherapy effectiveness

Reaching for Excellence: Feedback-Informed Treatment in Practice.  (in press).  Miller, S.D., Prescott, D., & Maeschalck, S. (2017).  Washington, DC: APA Books.

Outcomes Management, Reimbursement, and the Future of Psychotherapy.  G.S. (Jeb) Brown & Takuya Minami.  (2010).  In: The heart and soul of change: Delivering what works in therapy (2nd ed.). Duncan, Miller, Wampold, & Hubble,  (Eds.); pp. 267-297. Washington, DC, US: American Psychological Association.

The heroic client: A revolutionary way to improve effectiveness through client-directed, outcome-informed therapy. (2004). Duncan,B.L., Miller, S.D., & Sparks, J.A.  San Francisco: Jossey-Bass.

Developing and Delivering Practice-Based Evidence: A Guide for the Psychological Therapies. (2010). Barkham, M., Hardy, G.E., & Mellor-Clark, J.  New York, NY:  Wiley.