Research Questions – Overview

Status: November 3, 2009

1. Main Study Questions: Outcomes (reduction in symptoms)

   • • COVARIATES: New vs old client [NEWCLNT]; program type [CIPROGCAT]; OTHERS?

   1. Does providing frequent feedback [can create newvar FBGRP=1 if GROUP = 3 or 4; else=0]on client-based outcome variables to clinicians improve the home-based mental health treatment of youth? That is, are youth in the weekly feedback condition more likely to show a stronger negative rate of change in symptoms as measured by the youth and counselor SFSS and less likely to show an increase in symptoms compared to the 3-months feedback condition (intent to treat main effect of feedback)?
      
      

      • SFSS SUMMARY VARS (vars in CBCL units may be 1^st choice among non-categorical vars):
      • YDSFSTLN SFS-Y: Symptom Scale in CBCL units **
      • YDSFSEXT SFS-Y: Symptom Subscale Externalizing (in CBCL units)****
      • YDSFSINT SFS-Y: Symptom Subscale Internalizing (in CBCL units)****
      • YDSFSTOT SFS-Y: Total Symptom Scale (1-5 scale) **
      • YDSFSEX_NT Non-Transformed Staff externalizing score
      • YDSFSIN_NT Non-Transformed Staff internalizing score
      • ADSFSTLN SFS-A: Symptom Scale in CBCL units **
      • ADSFSEXT SFS-A: Symptom Subscale Externalizing (in CBCL units)****
      • ADSFSINT SFS-A: Symptom Subscale Internalizing (in CBCL units)****
      • ADSFSTOT SFS-A: Total Symptom Scale (1-5 scale) **
      • ADSFSEX_NT Non-Transformed Staff externalizing score
      • ADSFSIN_NT Non-Transformed Staff internalizing score
      • CDSFSTLN SFS-C: Symptom Scale in CBCL units **
      • CDSFSEXT SFS-C: Symptom Subscale Externalizing (in CBCL units)****
      • CDSFSINT SFS-C: Symptom Subscale Internalizing (in CBCL units)****
      • CDSFSTOT SFS-C: Total Symptom Scale (1-5 scale) **
      • CDSFSEX_NT Non-Transformed Staff externalizing score
      • CDSFSIN_NT Non-Transformed Staff internalizing score
      • Categorical Vars:
      • YDSFSCAT ‘YDSFSTLN categorical **
      • YDSFSCAT_EXT ‘YDSFSEXT categorical **
      • YDSFSCAT_INT ‘YDSFSINT categorical **
      • ADSFSCAT ‘ADSFSTLN categorical **
      • ADSFSCAT_EXT ‘ADSFSEXT categorical **
      • ADSFSCAT_INT ‘ADSFSINT categorical **
      • CDSFSCAT ‘CDSFSTLN categorical **
      • CDSFSCAT_EXT ‘CDSFSEXT categorical **
      • CDSFSCAT_INT ‘CDSFSINT categorical **
      • Hypothesis: The frequent feedback group will show more significant improvement and less significant increase in symptoms compared to the 3-months feedback group.
        
   2. Does providing training on common factors [can create newvar TRGRP=1 if GROUP = 2 or 4; else=0] to clinicians and their supervisors improve the home-based mental health treatment of youth? That is, are youth in the common factor training condition more likely to show a stronger negative rate of change in symptoms as measured by the youth and counselor SFSS and less likely to show an increase in symptoms compared to the no training condition (intent to treat main effect of training)?
      
      
      • SFSS: see above
      • Hypothesis: The common factor training group will show more significant improvement and less significant increase in symptoms compared to the no-training group.
   3. Does providing training on common factors in addition to frequent feedback [can create newvar FBTRGRP=1 if GROUP = 4; else=0]enhance the effect of feedback? That is, are youth in the combined common factor training condition and weekly feedback condition more likely to show a stronger negative rate of change in symptoms as measured by the youth and counselor SFSS and less likely to show an increase in symptoms compared to the other 3 conditions (intent to treat interaction effect of feedback and training)?
      
      
      • SFSS: see above
      • 
        Hypothesis: The combined group will show more significant improvement and less significant increase in symptoms compared to the other three groups.
   4. Does reviewing frequent feedback on client-based process and outcome variables to clinicians improve the home-based mental health treatment of youth? That is, are youth in the weekly feedback condition for which the counselor regularly (NEED TO DEFINE) reviewed available reports more likely to show a stronger negative rate of change in symptoms as measured by the youth and counselor SFSS and less likely to show an increase in symptoms compared to those who are 3-months feedback condition and those for whom the counselor did not regularly review the reports (main effect of feedback with adherence taken into account)?
      
      
      • This refers to the “implementation measures,” of which there are many:
      • [IMFBUYBLS -- cumFB_W3CSFS]
      • I think a meeting (or more!) would be best to review what these are and which are best to use.

      • Some vars are global (e.g., over all measures, how many or what ratio of reports were viewed/read) – e.g., [IMFBWUcl, IMFBQUcl]

      • Some vars are measure-specific – e.g.,

        • WU_viewedYSFS, W3_viewedYSFS, W4_viewedYSFS

        • W3_totviewYSFS, W3_totviewASFS, W3_totviewCSFS

        • W4_totviewYSFS, W4_totviewASFS, W4_totviewCSFS

        • cumFB_Q4YSFS, cumFB_Q4ASFS, cumFB_Q4CSFS

      • When focus is on outcomes (e.g., SFSS), vars related to SFSS would perhaps make most sense, though it’s a question about whether FB on measures other than SFSS might also affect SFSS over time (e.g., diffusion of sorts).

      • There are sets of vars to denote how long it was before reports were read –

        • e.g., IMFBWUcl, IMFBW3cl, IMFBW4cl

        • unconstrained = ever read (‘U’ in varname/label);

        • w/in 3 wks of availability (‘3’ in varname/label);

        • w/in 4 wks of availability (‘4’ in varname/label)

      • Note that the 90-day (“quarterly”) groups receive only 90-day FB reports; those in weekly groups receive weekly AND 90-day FB reports.

      • SFSS: see above
      • 
        Hypothesis: The group who reviewed frequent feedback reports regularly will show more significant improvement and less significant increase in symptoms compared to others.
        
   5. Does participating in training on common factors to clinicians and their supervisors improve the home-based mental health treatment of youth? That is, are youth in the common factor training condition whose counselors actively participated in the training (NEED TO DEFINE) more likely to show a stronger negative rate of change in symptoms as measured by the youth and counselor SFSS and less likely to show an increase in symptoms compared to the no training condition and those who did not actively participate (main effect of training with adherence taken into account)?
      
      
      • IMPCF01A DATE CF TEST PASSED: EXPECTANCIES
      • IMPCF01B DATE CF TEST PASSED: COLLAB TX PLANNING
      • IMPCF01C DATE CF TEST PASSED: TA
      • IMPCF01D LATEST DATE OF ANY/ALL READ
      • IMPCF03A MOD COMPLETED
      • IMPCF03B MOD COMPLETED
      • IMPCF03C MOD COMPLETED
      • IMPCF03E # OF 3 MODS COMPLETED
      • IMPCF03D ALL 3 MODS COMPLETED?
      • SFSS: see above
      • 
        Hypothesis: The group whose counselors actively participated in training will show more significant improvement and less significant increase in symptoms compared to the others.

   6. Does the participation in training on common factors in addition to reviewing frequent feedback enhance the effect of feedback? That is, are youth in the combined common factor training condition and weekly feedback condition whose counselors showed good adherence more likely to show a stronger negative rate of change in symptoms as measured by the youth and counselor SFSS and less likely to show an increase in symptoms compared to the other 3 conditions and those with low adherence (interaction effect of feedback and training with adherence taken into account)?
      
      

      • See above
      • 
        Hypothesis: The combined group of counselors with good adherence will show more significant improvement and less significant increase in symptoms compared to the others.
        

2. Possible variability in change patterns
   
   The proposed covariates, new/old client and program type, may not only influence the level and the rate of change but also the pattern of change in severity (SFSS). In general, treatment progress is expected to follow a logarithmic curve: a lot of improvement early on and then it levels of. Thus, the effect of feedback is likely to be different for clients who started CFIT late relative to their treatment start date. In addition, for youth in foster care, the service provider considers it a success when the kids remain stable, that is, they show a rather flat pattern of change. Youth in the virtual resident program may experience a lot of change because of the high severity they start treatment with and the intensity of the treatment. If these different patterns are found, is it appropriate to use the same statistical model for all of them?
   In addition, the comparative effect of weekly feedback should be the strongest during the first three months during which the comparison group does not receive any feedback at all. (CB: perhaps only for “new” clients)
   
   

   1. Is there a detectable difference in the pattern of change in regard to new/old client? [NEWCLNT]

   2. Is there a detectable difference in the pattern of change in regard to program type? [CIPROGCAT]

   3. Is there a detectable stronger effect of feedback during the first three months of CFIT?

      • [CITXBGDT: Client Info Tx Start Date****; other date vars as desired for comparisons –
      • e.g., 1^ST SFSS completed: CLYSFSDT, CLASFSDT, CLCSFSDT]
        

3. Other outcomes: Life Satisfaction and Hopefulness
   
   We have two other outcome variables representing the strength-based approach. It is likely that these two variables are less volatile and change at a slower pace. If there are significant changes over time, it would be interesting to determine how they related to our main outcome variable, the SFSS. Also, it would be interesting to explore if the study conditions have a significant effect on these strength-based outcomes.
   
   

   1. Is there a meaningful change in youth life satisfaction over the course of treatment?

      • [YDBLSTOT; CLYBLSDT]

   2. Is there a meaningful change in youth hopefulness over the course of treatment?

      • [YDCHSTOT; CLYCHSDT]
      • See above:

   3. How are changes in symptoms and functioning (SFSS) over time related to changes in youth life satisfaction?

   4. How are changes in symptoms and functioning (SFSS) over time related to changes in youth hopefulness?

   5. How are youth life satisfaction and hopefulness related to each other over time?

   6. Is there any significant effect of the study conditions on life satisfaction?

   7. Is there any significant effect of the study conditions on hopefulness?
      
      

4. Mechanisms of change: Clinicians and feedback
   
   Our theory is that by providing clinician with feedback about issues in their treatment (outcomes and process), they are aware of these issues and are motivated to address those issues during sessions following the feedback. By having awareness of the issues and by, in turn, addressing them, they will have less unintentional interruptions of treatment and, thus, better outcomes of treatment overall.
   
   

   1. Are SFSS item alerts on a feedback report related to reports of clinicians addressing this issue in subsequent sessions as reported on the session report form? [We need to decide what matches there are between SFSS items and items on the session report form]

      • CB: some on this has been done (by tommaso, particularly, I think)
      • Item Alerts Reports (SFSS):
      • Criterion for “alert” (i.e., value of response) varies by item – see end of doc)
      • Specific Problems Addressed (SRF): CLCSRFDT Date Que was completed; CDSRATE Rate session
      • (Note: Version 1 of SRF had CDSRPB01--CDSRPB09 only; Version 2 had all)
      • CDSRPB01 Behavioral problems
      • CDSRPB02 Family problems
      • CDSRPB03 Harm to self/others
      • CDSRPB04 School/work problems
      • CDSRPB05 Friends/peer problems
      • CDSRPB06 Emotional problems
      • CDSRPB07 Alcohol/substance use
      • CDSRPB08 Strengths of youth/family
      • CDSRPB09 Other option for issues discussed
      • CDSRPB10 Problems w/delinquent behavior
      • CDSRPB11 Client hope for future
      • CDSRPB12 Client motivation for treatment
      • CDSRPB13 Counselor r/ship issues w/client
      • CDSRPB14 Client perceptions on counseling
      • CDSRPB15 Client progress
      • CDSRPB16 Client satisfaction w/services
      • CDSRPB17 Caregiver strain dealing w/youth issues
      • CDSRPB18 Caregiver satisfaction w/life
      • CDSRPB19 Caregiver motivation for youth treatment
      • CDSRPB20 Counselor issues w/caregiver
      • CDSRPB21 Caregiver satisfaction w/services

   2. Are sudden changes in scores (outcomes or process variables) related to reports of clinicians addressing this issue in subsequent sessions as reported on the session report form? [We need to decide what matches there are between scales scores and items on the session report form]

   3. Are reports from clinician that they addressed an issue related to changes on the respective scale representing the issue?
      
      

      • See above

5. Mechanisms of change: Therapy process
   
   There is a debate in the clinical literature to what degree common factors and therapy process variables really matter. The current empirical literature is insufficient to answer these questions. With our unique longitudinal dataset we have the opportunity and responsibility to investigate some important issue in regard to the nature of therapy process variables and their relationship to clinical outcomes over time. It was our assumption that process variables predict outcomes and that we can influence process variables through interventions (i.e., feedback and training).

   1. How does TA behave over time? What is the pattern of change?

      • CB: We are presently fielding a shorter version of the TAQ for Y and A – their items parallel the current C version
      • CLYTAQDT Date Que was completed
      • YDTAQTOT TAQ-Y: Alliance Scale score ****
      • CLATAQDT Date Que was completed
      • ADTAQTOT TAQ-A: Alliance Scale score ****
      • CLCTARDT Date Que was completed
      • CDTAY01 Counselor working r/ship w/client
      • CDTAA02 Counselor working r/ship w/client's caregiver
      • CDTAY03 Counselor perception of client r/ship rating
      • CDTAA04 Counselor perception caregiver r/ship rating

   2. What does really matter in regard to the relationship of TA to outcomes (SFSS)? Is the level, the rate of change, a sudden drop (rupture), agreement of raters (clinician and youth), or synchronicity (divergence and convergence over time)? See above

   3. Does providing frequent feedback on TA have a measurable impact on TA? See above

   4. How does motivation behave over time? Is there a growth factor over time? Are there sudden changes?

      • CLYMYTDT Date Que was completed
      • YDMYTTOT Scale score (total)
      • YDMYTHLP MYT-Y: Motivation Help Seeking Subscale ****
      • YDMYTPRB MYT-Y: Motivation Problem Recognition Subscale ****
      • YDMYTRDY MYT-Y: Motivation Treatment Readiness Subscale ****

   5. In what way is motivation related to outcomes (SFSS)? Is motivation to be in treatment going up when the youth severity increases (problem recognition)? Are those youth who are motivated to be in treatment more likely to take advantage of treatment (change subscale on the counseling impact scale) and, thus, more likely to improve their outcomes?

      • CLYCISDT Date Que was completed
      • YDCISTOT CIS-Y: Counseling Impact Scale ****
      • YDCISCHG CIS-Y: Youth Change Subscale ****
      • YDCISINS CIS-Y: Youth Insight Subscale ****
      • See above

   6. Is a sudden drop in TA (rupture) related to subsequent changes in treatment motivation?

   7. To what degree does motivation at intake matter compared to motivation once treatment started?

   8. Does providing frequent feedback on motivation have a measurable impact on motivation?

   9. How does session impact change over time? Is there a growth factor or are changes from session to session more common?

   10. How is session impact related to outcomes? Are those youth who are improving in regard to outcomes more likely to perceive an impact of counseling compared to those who are not improving? Does a positive perception of impact have an effect on changes in outcomes?

   11. Is session impact related to treatment motivation? That is, are those youth who perceive a positive impact also more motivated to be in treatment compared to those who perceive less of an impact?

   12. Does providing feedback on session impact have a measurable effect on session impact?

   13. Are realistic treatment expectations [needs to be operationalized] related to early developments in TA?

6. Mechanism of change: Caregivers
   
   Caregivers are seen as an important factor in the mental health treatment of youth both in regard to how their well being is related to the youth mental well being and in regard to their participation in treatment.

   1. Is the caregiver’s life satisfaction related to the youth progress in treatment? That is, are youths, whose caregivers are not doing well, less likely to improve?

      • CLASWLDT Date Que was completed
      • ADSWLTOT Scale score

   2. Is the youth mental health status (primarily externalizing behaviors) related to the experience of caregiver strain? And, is caregiver strain related to caregiver life satisfaction?

      • CLACGSDT Date Que was completed
      • ADCGSTOT CGS-A: Caregiver Strain Scale ****
      • ADCGSOBJ CGS-A: CG Objective Subscale ****
      • ADCGSSUB CGS-A: CG Subjective Subscale ****
      • To Be Continued!!

   3. Is the caregivers well-being (life satisfaction
      
      

7. The influence of clinicians
   
   
   
   

8. Implementation
   
   

9. Organizational context and outcomes

OTHER VARS