Article Text

PDF

A checklist and structured lay interview had low agreement but comparable validity for childhood psychiatric disorders
  1. Michael Berger, PhD
  1. Royal Holloway, University of London Surrey, UK

    Statistics from Altmetric.com

    
 
 Question Do a self administered problem checklist and a lay administered structured interview provide comparable levels of validity for estimating the prevalence of childhood psychiatric disorders?

    Design

    A blinded comparison of a self administered problem checklist and a lay administered structured interview which used computer algorithms to determine if DSM-III-R criteria for individual disorders were met.

    Setting

    Public schools in Hamilton, Canada.

    Participants

    From a random sample of 2317 children (age 6 to 16 y), of which 1751 had parental consent to participate in the study, 251 of 329 eligible parent and child pairs were included in the study.

    Description of tests

    A revised version of the Ontario Child Health Study scales (OCHS-R) was completed during a screening phase and again at a later date by participants in the study. The parent version of the Diagnostic Interview for Children and Adolescents, revised (DICA-R), was given to parent and child pairs on 2 occasions a mean of 17 days apart. Interviewers were blind to the OCHS-R responses.

    Main outcome measures

    The disorders of interest were conduct, oppositional defiant, attention deficit hyperactivity, over anxious, separation anxiety, and major depressive disorders. Validity was assessed by examining the discrimination between families who did and did not access help in the previous 6 months using positive predictive values and the area under the receiving operating curve.

    Main results

    DICA-R and OCHS-R classified 5% and 6% of children, respectively, with conduct disorder; 25% and 28%, respectively, with oppositional defiant disorder; 15% and 15%, respectively, with attention deficit hyperactivity disorder; 14% and 11%, respectively, with over anxious disorder; 4% and 5%, respectively, with separation anxiety disorder; and 4% and 5%, respectively, with major depressive disorder. Agreement on cases between instruments was low (kappa range 0.05 [major depressive disorder] to 0.52 [oppositional defiant disorder]). The area under the receiver operating curve for cases seeking help was higher for OCHS-R than DICA-R for conduct disorder (78% v 66%) and oppositional defiant disorder (79% v 74%). There were no differences between the instruments for the positive predictive value of cases seeking help

    Conclusion

    Although agreement on cases was low, the self administered problem checklist, Ontario Child Health Study scales, revised, and the lay administered structured interview, Diagnostic Interview for Children and Adolescents, revised, provided comparable evidence of validity.

    Commentary

    Resources generally preclude the use of the clinical interview for case identification in epidemiological studies, even though it is currently the gold standard. Hence studies of the epidemiological value of other instruments are potentially important. The study by Boyle et al uses and compares 2 alternatives to the “clinician as the instrument” in an epidemiological study.

    In examining the validities of the instruments, a major criterion was discrimination between families who did and did not access help or use consultation. This is a very weak criterion: it is unclear what the subcriteria were or what the proportions of participants for each (consultation or help) were, and no indication is given of when cases were seen, what the diagnosis was, who made it, and how such diagnoses matched with the research instrument classifications.

    Although both instruments were of similar reliability and validity and yielded no appreciable differences on most of the key indices, the authors found substantial disagreements between the 2 instruments in terms of who they identified as cases; this, they believe, was because both instruments contained a substantial amount of error in identifying cases. Clearly, this finding is central and reinforces the need for all epidemiological studies of mental health in children and adolescents at least to have their case detection instruments carefully checked, calibrated, and periodically re-evaluated. The clinician as the instrument of diagnosis should be included in this process. The methods used in this study are some interesting examples of what needs to be done in this regard.

    For me, the most interesting parts of the report are the issues raised by the authors in their discussions at the beginning and end of the report. These issues, which should be addressed by all clinicians and researchers, included methodological issues (ie, low prevalence rates of childhood psychiatric disorder, high sample attrition rate, and limited criteria for testing validity) and an issue of validity (ie, disagreement on cases between instruments).

    View Abstract

    Footnotes

    • Sources of funding: National Health Research and Development Program, Ottawa, Canada; Ontario Ministry of Community and Social Services.

    • For correspondence: Dr M H Boyle, Centre for Studies of Children at Risk, Patterson Building, Chedoke Hospital, Hamilton, Ontario L8N 3Z5, Canada. Fax +1 905 574 6665.

    Request permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.