Outcome Data

"The data are always friendly" - Irene Elkin  

Below are a sample of my clinical outcome data since 2011.  I support the "open data" movement towards making clinical outcome data available to consumers, payers, policy-makers, and researchers (after identifying information has been removed from the data).

Below is a poster that describes a representative sample of my psychotherapy outcome data from two years of practice at the University of Alaska Fairbanks Student Health and Counseling Center.  This poster was created by Data Visualization expert Andy Kirk


Tony Rousmaniere Therapy Outcome Data

2011 Private Practice data (San Francisco & Palo Alto, California)

  • Total # of clients in dataset: 42
  • Avg # of sessions: 8.58
  • Total # of clients with more than one session: 38 (90%)
  • Single session clients: 4 (10%)
  • Start in clinical range (of clients with 2+ sessions): 25 (66%)

Clinical Outcomes

  • Ended Year in Clinical Change: 22 (88% of clients starting in clinical range)
  • Ended Year in Clinical Recovery: 20 (80% of clients starting in clinical range)
  • Ended Year in Clinical Deterioration: 2 (8% of clients starting in clinical range)
  • Average change in ORS score: 7.52  (clients starting in clinical range)
  • Cohen's D effect size for clients starting in clinical range: 1.22.  (Pre-tx std dev=6.14, mean intake ORS=22.27, mean last session ORS=29.79, n=38)

Duration of Treatment

  • Avg # of sessions for low distress clients: 5.27
  • Avg # of sessions for high distress clients: 10.74
  • Avg # of sessions for clinical change to occur: 3.27

  

2012-2013 University Counseling Center data  (University of Alaska Fairbanks)

Outcome Questionnaire-45 (OQ-45) data

  • Total # of clients in dataset: 102
  • Avg # of sessions:  4.60
  • Total # of clients with more than one session: 73 (72%)
  • Single session clients: 29 (28%)
  • Start in clinical range (of clients with 2+ sessions): 53 (73%)

Clinical Outcomes

  • Ended Year in Clinical Change: 24 (45% of clients starting in clinical range)
  • Ended Year in Clinical Recovery:  13 (25% of clients starting in clinical range)
  • Ended Year in Clinical Deterioration: 3 (6% of clients starting in clinical range) 
  • Average change in OQ score: 15.06  (clients starting in clinical range)
  • Cohen's D effect size for clients starting in clinical range: .82.  (pre-tx std dev=18.25, mean intake OQ=86.30, mean last session OQ=71.24, n=53)           

Duration of Treatment

  • Avg # of sessions for low distress clients: 4.24
  • Avg # of sessions for high distress clients: 4.78


Outcome Rating Scale (ORS) data

  • Total # of clients in dataset:  89
  • Avg # of sessions: 5.37
  • Total # of clients with more than one session: 65 (73%)
  • Single session clients: 24 (27%)
  • Start in clinical range (of clients with 2+ sessions): 42 (65%)

Clinical Outcomes

  • Ended Year in Clinical Change: 20 (48% of clients starting in clinical range)
  • Ended Year in Clinical Recovery:  11 (26% of clients starting in clinical range)
  • Ended Year in Clinical Deterioration: 4 (6% of clients starting in clinical range)   
  • Average change in ORS score: 4.84  (clients starting in clinical range)
  • Cohen's D effect size for clients starting in clinical range: .97.  (pre-tx std dev=4.98, mean intake ORS=15.63, mean last session ORS=20.47, n=42)

Duration of Treatment

  • Avg # of sessions for low distress clients: 5.61
  • Avg # of sessions for high distress clients: 5.24
  • Avg # of sessions for clinical change to occur: 2.60
  • Maximum # of sessions for clinical change: 5.00


2014-2015:  I do not have data for these years because my counseling center at the University of Alaska Fairbanks switched to a different outcome measurement program (the Counseling Center Assessment of Psychological Symptoms - CCAPS) which does not facilitate analysis of outcome data at the clinician level.  We started using the CCAPS because (a) it integrated better with our electronic records software, (b) was easier for counselors and medical staff to use for consultations, and (c) was designed specifically for university counseling centers.  


2015-2017 Private Practice data (Seattle) - Preliminary

I moved to Seattle and started a private practice in the Fall of 2015. Following is the data from my practice from Fall 2015 to Winter 2017. The dataset is still small because I work part-time as a therapist, some clients have opted-out of having their data included in this analysis, and I do not include data from clients who start treatment above the "clinical cutoff" (see below for an explanation.) This data should be considered preliminary because there are not sufficient data for a statistically reliable gauge of the work. Ideally, there should be data from at least 30 clients to provide statistical reliability. This data will be updated annually.

Notes: Since 2015 my primary professional focus has gradually shifted towards more writing, research, leading workshops and webinars, and providing consultation for agencies and training programs. This means I have worked with fewer new clients and had less time for my own professional development (clinical training, expert consultation, and deliberate practice.) Research from the science of expertise suggests is reasonable to expect that my clinical effectiveness will plateau and decline over time given this career trajectory.

The outcome measure I used from 2015-2016 was the Behavioral Health Measure-20 (BHM-20). Since January 2016 I have used the Outcome Questionnaire-45.

  • Total # of clients with data: 27 
  • Clients who started in clinical range (below "clinical cutoff"): 18 (66%) 
  • Range of sessions: 1 to over 100 
  • Clients with more than one session: 24 (89%) 
  • Clients who did not return after first session: 3 (11%) 
  • Average # of sessions: 21

Clinical Outcomes

  • Clients who met the BHM-20 or OQ-45 "recovery" or “improved” benchmark: 11 (73% of clients starting in clinical range and had two or more sessions. Nine of these clients were “recovery”, and two were “improved”.)
  • Clients who showed no clinically significant change on the BHM-20 or OQ-45: 4 (27% of clients starting in clinical range and had two or more sessions.)
  • Clients who deteriorated, according to BHM-20 or OQ-45 scores: 0 (0%)

How I Practice

Individuals are about 90% of my practice, and couples about 10%.  The therapeutic modalities I primarily use are Short-Term Psychodynamic Psychotherapy (STPP), Intensive Short-Term Dynamic Psychotherapy (ISTDP), and Cognitive-Behavioral Therapy (CBT).

Outcome Measurement and Client Response

I have used two outcome measures.  The Outcome Rating System (ORS) is an ultra-brief measure that is free to use.  Information about the ORS is here and here.   Clients completed the ORS at the beginning of each session, on a laptop computer or with paper/pencil.  It takes about 30 to 60 seconds to complete. 

I also used the Outcome Questionnaire-45 to track outcomes (OQ-45).  Information about the OQ-45 is here.  Clients completed the OQ at the beginning of each session on a computer in the waiting room.  It takes about 3-5 minutes to complete.

Consent for tracking outcomes is obtained at the beginning of treatment.  This is voluntary for clients, and about 5% of clients refused consent.  Consent includes agreement for the outcomes to be used for research and publication, after being de-identified.

After reviewing my cases, I consider the ORS and OQ results to be valid for about 80% of my cases.  For a few cases I think both measures underrepresented the amount of positive change, especially for clients who started treatment with less clinical distress.  For a few clients the ORS overrepresented the amount of positive change, in my judgement.  

When I asked clients their thoughts about the outcome measurement, most clients were curious about their results but said they gave more weight to my subjective clinical impression of their case.  A few clients were skeptical of the validity of standardized outcome measurements, and a few clients refused to complete the measures because of this concern.   In studies, clinicians have reported that a leading reason for not collecting outcome data is that clients do not want to, or concerns about harming the therapeutic alliance.  I did not find this to be the case, but rather found that discussing outcomes with some clients strengthened the therapeutic alliance.  My guess is that clinician attitudes towards the outcome measures strongly affect clients’ reactions to the measures.

What About Change for "Low-Distress" Clients?

There are no outcome measures that are good at assessing the amount of change in psychotherapy for clients who start therapy with low levels of distress, termed the "clinical cutoff" in the psychotherapy research literature.  (Clients who start therapy above the clinical cutoff of 25 points on the ORS or below 64 points on the OQ).  For this reason, I do not have reliable quantitative data on the amount of change for low-distress clients.  

Limitations of the Data and Outcome Measurement

It is difficult to assess how good these results are, because there are no available data from therapists working in the same locations as I worked.  Likewise, it is not possible to compare outcome data from my different work locations because the client populations are different.  Another validity concern is that I do not have follow-up outcome data, so I do not know if gains from treatment are lasting. 

Therapy outcome measures are not perfect.  Multiple clients report negative change on outcome measures when in fact their symptoms improved.  Likewise, paper forms can never describe the rich complexities of growth and healing in therapy.  Given these limitations, I still think it is important for therapists to track their clients' outcomes with unbiased measures.  While the measures should never be used alone for clinical decisions, they are one important source of information regarding client progress or deterioration, and are can be helpful for assessing overall therapist effectiveness, when combined with a range of other assessment tools.


Popular outcome measures

Online forums and communities that discuss outcome measures and provide training

Software to help measure outcomes



Books on measuring psychotherapy effectiveness

Feedback-Informed Treatment in Practice: Reaching for Excellence. (2017). Prescott, D., Miller, S.D., & Maeschalck, S. Washington, DC: APA Books.

Outcomes Management, Reimbursement, and the Future of Psychotherapy. G.S. (Jeb) Brown & Takuya Minami.  (2010).  In: The heart and soul of change: Delivering what works in therapy (2nd ed.). Duncan, Miller, Wampold, & Hubble,  (Eds.); pp. 267-297. Washington, DC, US: American Psychological Association.

The heroic client: A revolutionary way to improve effectiveness through client-directed, outcome-informed therapy. (2004). Duncan,B.L., Miller, S.D., & Sparks, J.A.  San Francisco: Jossey-Bass.

Developing and Delivering Practice-Based Evidence: A Guide for the Psychological Therapies. (2010). Barkham, M., Hardy, G.E., & Mellor-Clark, J.  New York, NY:  Wiley.