SOWK 6341 OLLU Validity Threats Mental Conditions Research Discussion
Purpose
Experimental designs are the gold standard of research because they help eliminate any other explanation for why an intervention does or does not work. However, experimental designs are difficult in social service settings for many reasons. For purposes of practicality, we often opt for quasi-experimental designs that introduce limitations to our research (threats to validity). While it is nearly impossible to eliminate every threat to validity, there are ways to mitigate those threats.
Create an original discussion post where you identify two potential threats to validity and craft ways to mitigate those threats.
Discussion: Identifying Threats Mental Health Treatment A nonprofit serving children who have witnessed family violence wants to know if their counselors should use cognitive behavioral therapy (CBT) or trauma-informed cognitive behavioral therapy (TFCBT). They have two counselors trained in CBT who have been working at the agency for 15 years each. They also have two new counselors straight from an MSW program who have been trained in TFCBT. The CBT counselors have a current caseload of 20 children each. All new children will be assigned to the TFCBT counselors. When a child is done with therapy, the counselors will ask the parents if their child’s behaviors have gotten better. They will use that information to decide if they should use TFCBT or CBT.
SOWK 6341 OLLU Validity Threats Mental Conditions Research Discussion
Purpose
This discussion aims to identify potential threats to validity in a proposed study comparing the effectiveness of cognitive behavioral therapy (CBT) and trauma-informed cognitive behavioral therapy (TF-CBT) for children who have witnessed family violence. Experimental designs seek to eliminate alternative explanations for study outcomes, but some threats are difficult to avoid entirely in social service research. While it is impossible to eliminate all threats, mitigation strategies can be employed.
Discussion: Identifying and Mitigating Threats
Two main threats to validity are apparent in the proposed study. The first is selection bias resulting from non-random assignment of clients to treatment conditions. The CBT counselors’ caseloads represent an established client base, while new clients will be assigned to the less experienced TF-CBT counselors. This introduces the possibility that client characteristics rather than the treatment explain any observed differences in outcomes.
To mitigate this threat, researchers could employ stratified random assignment to distribute pre-treatment characteristics evenly across conditions (Creswell & Creswell, 2018).[1] For example, they might match clients on gender, age, trauma history, and symptom severity before randomizing to counselors.
The second threat is expectancy or allegiance effects stemming from counselors’ training backgrounds (Luborsky et al., 1999).[2] The CBT counselors have 15 years of experience with that model versus the TF-CBT counselors’ new training. This could influence therapeutic alliance and delivery fidelity in a way that biases outcomes.
To address allegiance effects, the researchers could implement treatment integrity procedures like session recordings and independent coding/ratings (McHugh & Barlow, 2010).[3] They might also consider a Solomon four-group design adding no-treatment control groups to parse treatment from expectancies (Creswell & Creswell, 2018).[1]
In summary, threats from selection bias and allegiance could be mitigated through random assignment, matching on key variables, treatment integrity monitoring, and control conditions to strengthen the internal validity of the study.
