CiMH Palette of Measures Evaluation Training: Youth Outcome Questionnaires (YOQs)

Cricket Mitchell, PhD CIMH Evaluation Consultant

Palette of Measures Evaluation: What You Will Need (slide 1 of 2) • Palette of Measures Data Entry Shell v2 (Excel file) – Developed by CiMH and customized for each participating agency – Holds all data for clients served thru the Palette of Measures project – Demographics, service delivery information, pre- and post- outcome measure data • Palette of Measures Data Dictionary v2 (Word document) – A guide for using the associated data entry shell – Defines each column in the excel file 2

Palette of Measures Evaluation: What You Will Need (slide 2 of 2) • Outcome measures from the two-pronged approach – General Outcome Measure – Target-Specific Outcome Measure(s) –

For example…

• Youth Outcome Questionnaire © (YOQ) and Youth Outcome Questionnaire – Self Report © (YOQ-SR) – Provided to agencies participating in the Palette of Measures project under CiMH’s statewide license 3

Overview of Training

• Brief Overview of Palette of Measures evaluation protocol • Youth Outcome Questionnaires functioning – Administration – Scoring – Clinical Utility data submissions – Data entry: YOQ/YOQ-SR – Data submissions to CiMH © (YOQ & YOQ-SR): General outcome measures of youth mental health • Instructions for Palette of Measures data entry and – Data entry: Demographics & Services 4

Brief Overview of Palette of Measures Evaluation Protocol

5

Outcome Assessment

• Palette of Measures providers will track outcomes using data from pre- and post- administrations of standardized measures of functioning • Pre- and Post- a “dose” of treatment / an intervention interval – General measure of youth mental health functioning (e.g., YOQ/YOQ-SR, CANS, Ohio Scales) – Target-specific measure linked to focus of treatment/intervention (e.g., AQ, ECBI, PHQ-9, PTSD-RI, RCADS) • Providers may choose to administer mid-course assessments as well – e.g., at 3-month intervals in usual care 6

A note about the use of standardized assessment measures… (slide 1 of 2) • Assessment is the beginning of developing a relationship with the child and family – Demonstrates a desire to know what the child and family are experiencing – By incorporating standardized assessment measures of functioning, the efficiency and thoroughness of assessment is enhanced 7

A note about the use of standardized assessment measures… (slide 2 of 2) • Using standardized assessment measures of functioning… – Assists in initial clinical impressions – Provides valuable information to guide treatment/interventions – Assesses sufficiency of treatment delivered – Demonstrates treatment-related improvements in child functioning 8

Youth Outcome Questionnaires

©

(YOQ & YOQ-SR)

9

CiMH YOQ & YOQ-SR Training

• Information on the administration, scoring, and clinical utility of the YOQ & YOQ-SR was obtained from each measure’s respective Administration and Scoring Manual published by OQ Measures, LLC 10

YOQ & YOQ-SR Description

• General measure of functioning • Assesses the global mental health functioning of children – Parent/caregiver report for children ages 4-18 – Self-report for adolescents 12-18 – Adult self-report is also available for 19+ • Available in English and Spanish 11

YOQ & YOQ-SR Description

• 5-15 minutes to complete • Sensitive to clinical change in short periods of time – Regardless of treatment modality, diagnosis, or discipline of the treating professional • Valid and reliable • Provided to partner agencies under CiMH’s statewide license – Partnering agencies sign a User’s Agreement and return it to CIMH 12

YOQ & YOQ-SR Description

• Parallel versions of the same measure • 64 items • 5-point Likert scale response options* – Never or Almost Never – Rarely – Sometimes – Frequently – Almost Always or Always *response values vary by item • Six Scale Scores – Intrapersonal Distress (ID) – Somatic (S) – Interpersonal Relations (IR) – Social Problems (SP) – Behavioral Dysfunction (BD) – Critical Items (CI) • Total Score 13

YOQ & YOQ-SR Administration

• Administer pre- and post- a dose of treatment, or an intervention interval – YOQ-SR completed by adolescent client – YOQ completed by mother/mother figure or father/father figure (when available) • Both can be completed for clinical purposes; however, only one should be entered for outcome tracking purposes – Select parent/caregiver who spends most time with client 14

Let’s take a look at the YOQ-SR...

15

YOQ & YOQ-SR Administration

• “… during the past 7 days.” • Ask parents and youth to fill out the questionnaires as honestly as possible – Informants can easily be influenced by the attitude of the person administering the scale – Let them know that this questionnaire will help you, as a clinician, better understand how the child is doing overall • Ask parents and youth to complete all items 16

YOQ & YOQ-SR Scoring

• For each item, transfer the value corresponding to the selected response into the box at the right-hand side of the page titled, ‘For Office Use Only’ – Each item loads onto one of the six scales (e.g., ID, SP, IR) – Note that some items have negative response option values • Sum the items in each scale on Side1 – Add the numbers in all boxes under the heading ID and enter that subtotal into the ID box at the bottom of the page – Repeat for each scale • Sum the items in each scale on Side 2 • Transfer the subtotals from Side 1 to Side 2 • Sum the subtotals to determine Scale Scores • Sum the Scale Scores to determine Total Score – Note that it is possible to have negative values for scores 17

YOQ & YOQ-SR Scoring

• Missing Data (items that are left blank) – If 5 or more items are missing, consider the questionnaire invalid – Substitute a mean item response for the missing item • Determine in which scale the missing item belongs • Add up the other items in that scale, and determine their average • Substitute the average score for the missing response 18

Clinical Utility of the YOQ & YOQ-SR

• Total Score – Possible scores range from -16 to 240 – Clinical cutpoints • 46 or higher on the YOQ • 47 or higher on the YOQ-SR –

Lower

scores indicate more functioning

normative

, non clinical, aspects of general mental health • Elevations on certain scales indicate areas of specific distress for the child/youth 19

Clinical Utility of the YOQ & YOQ-SR

• Intrapersonal Distress (ID) Scale – Possible scores range from -4 to 68 – Clinical cutpoints • 16 or higher on the YOQ • 17 or higher on the YOQ-SR – Assesses the amount of

emotional distress in the child/youth, including anxiety, depression, fearfulness, hopelessness, and self-harm

– High scores indicate a considerable degree of intrapersonal distress in the child/youth 20

Clinical Utility of the YOQ & YOQ-SR

• Somatic (S) Scale – Possible scores range from 0 to 32 – Clinical cutpoints • 5 or higher on the YOQ • 6 or higher on the YOQ-SR – Indicates change in

somatic distress or physical complaints

– High scores indicate the parent/caregiver is aware of, or the youth is experiencing, a high number of somatic symptoms; while low scores indicate either absence or unawareness of them 21

Clinical Utility of the YOQ & YOQ-SR

• Interpersonal Relations (IR) Scale – Possible scores range from -6 to 34 – Clinical cutpoints • 4 or higher on the YOQ • 3 or higher on the YOQ-SR – Assesses issues relevant to the

peers child/youth’s relationship with parents, other adults, and

– High scores indicate significant interpersonal difficulty; while low scores reflect a cooperative, pleasant interpersonal demeanor 22

Clinical Utility of the YOQ & YOQ-SR

• Social Problems (SP) Scale – Possible scores range from -2 to 68 – Clinical cutpoints • 3 or higher on the YOQ & YOQ-SR – Assesses

problems that are socially related including aggression and delinquency

– A feature of these items is that they are slow to change; whereas, content tapped by many of the other scales often changes over a period of time as a result of treatment intervention 23

Clinical Utility of the YOQ & YOQ-SR

• Behavioral Dysfunction (BD) Scale – Possible scores range from -4 to 40 – Clinical cutpoints • 12 or higher on the YOQ • 11 or higher on the YOQ-SR – Assesses

inattention, hyperactivity, impulsivity, concentration, ability to organize tasks, and ability to handle frustration

24

Clinical Utility of the YOQ & YOQ-SR

• Critical Items (CI) Scale – Possible scores range from 0 to 36 – Clinical cutpoints • 5 or higher on the YOQ • 6 or higher on the YOQ-SR – Assesses areas such as

paranoia, obsessive-compulsive behaviors, hallucination, delusions, suicide, mania, and eating disorders

25

Clinical Utility of the YOQ & YOQ-SR

• Assesses a variety of specific areas of difficulty in child/youth mental health functioning • Assists in initial clinical impressions • Provides valuable information to guide treatment/interventions • Comparisons of pre/post scores reveal areas of clinical improvement as well as areas of potential unmet need 26

Summary of YOQ/YOQ-SR Score Ranges and Clinical Cutpoints

YOQ/YOQ-SR Scale Intrapersonal Distress (ID) Range of possible scores -4 to 68 Clinical Cutpoint for YOQ Clinical Cutpoint for YOQ-SR 16 17 Somatic (S) Interpersonal Relations (IR) Social Problems (SP) Behavioral Dysfunction (BD) Critical Items (CI) Total Score 0 to 32 -6 to 34 -2 to 30 -4 to 40 0 to 36 -16 to 240 5 4 3 12 5 46 6 3 3 11 6 47 27

Palette of Measures Data Entry and Data Submissions

28

Palette of Measures Data Entry Shell

• There is a separate spreadsheet in the excel workbook (aka database) for each type of data: – Demographics & Services – Pre- General outcome measure – Post- General outcome measure – Pre- Target-specific measure(s) – Post- Target-specific outcome measure(s) • Specific outcome measure spreadsheets included in each agency’s database varies across Palette of Measures project participants 29

Palette of Measures Data Entry Shell

• In addition to the spreadsheets that hold data...

– There is an

Instructions

spreadsheet • Basic data entry instructions • Contact information for T.A. (Cricket Mitchell) – There is a

Data Lists

spreadsheet at the end of the workbook that you will not use • Data Lists populate the pull-down menus in other spreadsheets 30

Palette of Measures Data Entry: YOQ/YOQ-SR

Pre-YOQs Client ID# Date of Assessment PARENT/CAREGIVER REPORT: (ages 4-18) YOQ (Scale Scores and Total Score) YOQ1-ID YOQ1-S YOQ1-IR YOQ1-SP YOQ1-BD YOQ1-CI YOQ1-Total … •There is a separate spreadsheet for Pre-YOQ data, Post-YOQ data, and Mid-YOQ data •In each spreadsheet, there is a separate field for each of the YOQ and YOQ-SR Scale Scores and Total Scores •In the event of missing data, leave the fields blank/empty. Do not enter text into any of the fields.

31

Palette of Measures Data Entry: YOQ/YOQ-SR

Mid-YOQs Client ID# Date of Assessment Assessment YOQ (Scale Scores and Total Score) YOQM YOQM YOQM YOQM YOQM Interval ID S IR SP BD YOQM CI YOQM Total •For agencies who will conduct mid-treatment assessments, indicate the Assessment Interval in the Mid-YOQs spreadsheet by selecting from the available pull-down menu •e.g., 1 st mid-treatment assessment, 2 nd mid-treatment assessment 32

But, before you enter any outcome data, you’ll enter Demographics & Service Delivery Information...

33

Palette of Measures Data Entry: Demographics & Services

(1 of 2)

Client Information:

Client ID# DOB Primary (DSM-IV code) (DSM-IV code) Gender Ethnicity Language Primary Axis I Secondary Axis I Therapist ID •Use a unique identifier for Client ID# •Categorical variables will have pull-down menus from which you’ll select an option (e.g., gender, ethnicity, language) •Dates should be entered as xx/xx/xxxx •Axis I diagnoses s/b the numeric DSM-IV code •Therapist ID is optional 34

Palette of Measures Data Entry: Demographics & Services

(2 of 2)

Focus 1 of Treatment

Focus1 Date of First Session Date of Last Session Total # Completed of Sessions Focus 1?

(if Focus 1 not completed) Completed (if Services not completed) Reason Services?

Reason •Select Focus from pull-down menu (e.g., anxiety, depression) •The shell will hold data for up to 4 foci, or treatment targets •Enter Date of First Session •The remaining fields are to be completed at the end of treatment targeting this particular focus (e.g., Date of Last, # Sessions) •2 levels of “Completed?” • Treatment targeting this particular focus • Overall service delivery 35

Palette of Measures Data Submissions

(slide 1 of 3) • Data submissions to CiMH will occur twice a year throughout the duration of the project – The end of each May (reflecting all clients served from the initiation of the project through the end of that April) – The end of each December (reflecting all clients served from the initiation of the project through the end of that November) –

Note that this is the anticipated schedule; actual data submission dates may vary slightly

• An email notice will be sent to Palette of Measures site leads approximately one month in advance of each data submission deadline 36

Data Submissions

(slide 2 of 3) • Providers may choose from among the following methods for submitting their Palette of Measures Excel databases to CiMH: – Use YouSendIt, or another secure web-based transfer site, to submit data electronically • YouSendIt ( protected) www.YouSendIt.com

) is a vendor that supports the secure transfer of electronic data (encrypted and password – Mail a password-protected CD to CiMH and submit the password separately (via email or phone) – Email an encrypted, password-protected file(s) to CiMH and submit the password separately (via email or phone) 37

Data Submissions

(slide 3 of 3) • After data are submitted, sites continue to enter new data into the same database – Always reflects an ongoing, historical record of clients served through the Palette of Measures project • Every effort is made to distribute reports within two months of each data submission – Aggregate and site/agency-specific reports 38

Questions

39

The End

Contact Information

•Cricket Mitchell, PhD •Email: [email protected]

•Cell phone: 858-220-6355 40