Identifying children with anxiety disorders using brief versions of the Spence Children's Anxiety Scale for children, parents, and teachers.

Clinic-referred sample

Children in both trials were assessed prior to randomization and treatment group allocation, and this data was used in the current study. As part of this assessment, parents and children completed the Anxiety Disorders Interview Schedule-Child and Parent Interviews (ADIS-IV-C/P) and the SCAS-C/P and SDQ-C/P, and teachers were asked to complete the SCAS-T and SDQ-T.

Strengths and Difficulties Questionnaires (SDQ-C; SDQ-P/T)

The SDQ ( Goodman, 1997 ) provides a broad-based measure of a child’s emotional and behavioral difficulties. The child (SDQ-C) and parent/teacher (SDQ-P/T) report versions include corresponding items addressing a child’s emotional symptoms (five items), peer relationship problems (five items), conduct problems (five items), and hyperactivity/inattention (five items), with strong evidence in support of its psychometric properties both in community ( Goodman et al., 2000 ) and clinic-referred samples ( Goodman, Renfrew, & Mullick, 2000 ). In this study the SDQ-emotional problems scale, internalizing problems scale (emotional + peer relationship problems), conduct problems scale, and externalizing problems scale (conduct + hyperactivity/inattention) were used to examine the convergent and divergent validity of scores on the new brief anxiety measure. The internal consistency for the SDQ scale scores were acceptable-good in the current samples (SDQ-emotional problems scale, child α = .76, parent α = .84, teacher α = .85; SDQ-internalizing problems scale, child α = .76, parent α = .82, teacher α = .82; SDQ-conduct problems scale, child α = .60, parent α = .65, teacher α = .70; SDQ-externalizing problems scale, child α = .74, parent α = .82, teacher α = .85).

ADIS-C/P

The ADIS-C/P was administered with the clinic-referred sample to assess the child’s diagnostic status, including the assessment of DSM–IV anxiety, mood and externalizing disorders. The reliability and validity of the ADIS has been widely reported ( Silverman, Saavedra, & Pina, 2001 ). As per the standard guidelines, overall diagnoses and Clinical Severity Ratings (CSRs; 4–8) were assigned if the child met diagnostic criteria based on either the child or parent report, and the higher of the two CSRs was assigned. The disorder with the highest CSR was assigned as the primary disorder. Assessors were psychology graduates in both trials, and all assessments were discussed with a consensus team for at least the first 20 interviews for each assessor, at which stage the assessor’s reliability was checked (minimum κ = .85). After this point, at least one in six interviews were discussed with a consensus team; and overall reliability within the assessment team in both trials was excellent (child-report diagnosis: κ = 0.98; CSR: intraclass correlation [ICC] = 0.98–0.99; parent-report diagnosis: κ = 0.98; CSR: ICC = 0.97–0.99).

Evaluation of the brief questionnaires

Total scores on the optimal brief versions of the parent/child/teacher SCAS were calculated using the same procedure to deal with missing data as detailed previously for the full length SCAS (total scores reflect the sum of responses to all included items). The following psychometric properties of scores on the optimal brief versions of the parent/child/teacher SCAS were examined in each sample and compared with scores on the full length SCAS-P/C/T: (a) internal consistency (Cronbach’s alpha coefficients); (b) agreement between reporters (Pearson’s r correlation coefficients); (c) convergent and divergent validity (Pearson’s r correlation coefficient between full/brief SCAS scores and SDQ internalizing/emotional/externalizing/conduct scale scores). The capacity of the optimal brief parent/child/teacher SCAS scores to discriminate between children in the clinic-referred sample and children in the community sample was examined for the total sample, and for gender groups using (a) independent sample t tests (and Cohen’s d ), and (b) ROC analyses (as detailed previously, examining both the AUC and the sensitivity/specificity values associated with optimal cut-off scores on the parent/child/teacher brief SCAS). To compare the functioning of the brief SCAS with the full-length SCAS, the capacity of the full-length SCAS to discriminate between the two samples was also analyzed. A series of logistic regressions were used to examine the contribution of each reporter (parent, child, teacher), to determine whether using multiple informants improves the capacity of the brief SCAS scores to identify children in the clinic-referred sample. Using optimal cut-off scores identified in the ROC analyses, the sensitivity and specificity values associated with each combination of reporters (parent + child, parent + teacher, teacher + child, parent + child + teacher) were examined. For each combination of reporters, the sensitivity value reflected the proportion of children in the clinic-referred sample who scored above the optimal cut-off score based one at least one of reporter; and specificity value reflected the proportion of children in the community sample who scored below the optimal cut-off for each reporter. Gender differences on total scores on the brief (and full-length SCAS) within each sample (independent samples t tests) were also examined.

Because the sample sizes were large (>330 in each sample), a conservative p value ( p < .01) was used to indicate a statistical significance. All analyses were conducted using IBM SPSS (Version 21).

Agreement between reporters

Agreement between reporters within each sample are provided in online supplement 2 , indicating similar levels of agreement on the brief SCAS as the full SCAS. For the brief questionnaires, parent–child agreement was the highest (community sample, r = .40, p < .001; clinic-referred sample, r = .34, p < .001) and teacher–child agreement the lowest (community sample, r = .25, p < .001, clinic-referred sample, r = .05, p = .46).

Convergent and divergent validity

Convergent and divergent validity indices for the brief and full SCAS scores within each sample are provided in online supplement 3 . Similar patterns were observed for the brief SCAS scores as for the full SCAS scores, with significantly higher correlations between the brief parent/child/teacher SCAS scores and the SDQ-emotional problems scale scores ( r = .62–.76) and the SDQ-internalizing scale scores ( r = .58-.70), than between the brief parent/child/teacher SCAS scores and the SDQ-conduct problems scale scores ( r = .08-.32) and SDQ-externalizing problems scale scores ( r = .10–.34; z = 4.91–9.16, p < .0001).

ROC analyses

As displayed in Table 6 , the SCAS-P-8 was able to accurately identify children in the clinic-referred sample with an AUC of .86, and using an optimal cut-off score of 7.5, achieved .85 sensitivity and .75 specificity overall (with sensitivity/specificity values of .81/.79 for boys; and .89/.71 for girls). Corresponding sensitivity/specificity values for optimal cut-off scores on the full SCAS-P were 82/.78 (boys, .83/.80; girls, .82/.77).

The SCAS-C-8 also achieved an AUC >.70, both in the total sample and the gender differentiated groups (boys, .74; girls, .70). ROC analyses examining the SCAS-C-8 in the total sample indicated that the optimal cut-off score was 6.5, achieving a sensitivity value of .67, and specificity of .64 (it was not possible to achieve sensitivity >.70, with specificity >.60 for the total sample). The ROC analyses among the gender differentiated groups, however, indicated that the optimal cut-off scores among boys was 5.5, and among girls was 7.5, with respective sensitivity/specificity values of .73/.70, and .64/.63. The full child SCAS failed to achieve an AUC >.70 in the total sample or among gender differentiated groups, and the optimal cut-off scores on full child SCAS achieved similar sensitivity to the SCAS-C-8 (boys, .71; girls, .61), but with lower specificity (boys, .61; girls, .55).

The SCAS-T-8 achieved an AUC of .76, and the optimal cut-off score of 4.5 in the total sample was associated with a sensitivity value of .70, and specificity of .73. Analyses among gender differentiated groups indicated the optimal cut-off score on the SCAS-T-8 among boys was 3.5 (sensitivity/specificity, .74/.64), and among girls was 4.5 (sensitivity/specificity, .73/.69). Optimal cut-off scores on the SCAS-T-20 achieved sensitivity/specificity values of .71/.71 among boys, and .74/.64 among girls.

Using multiple reporters and the contribution of each reporter

Findings from the series of Logistic Regressions using different combinations of the SCAS-P-8, SCAS-C-8 and SCAS-T-8 scores to predict whether the child was in the community or clinic-referred sample are displayed in Table 7 . Among the models including two reporters, using parent report (SCAS-P-8) and teacher report (SCAS-T-8) explained the most variance (Nagelkerk, .54, Cox & Snell, .40); and scores on both the SCAS-P-8 and SCAS-T-8 were uniquely associated with sample (odds ratio, 1.40 and 1.18, respectively). Replacing the teacher report (SCAS-T-8) with the child report (SCAS-C-8) only slightly reduced the total amount of variance explained (Nagelkerk, .47, Cox & Snell, .35), although in this parent + child model, the SCAS-C-8 score was not significantly associated with the sample. Using teacher report (SCAS-T-8) and child report (SCAS-C-8) explained the least variance of all of the models (Nagelkerk, .33, Cox & Snell, .24), but both the SCAS-T-8 score and SCAS-C-8 score made small significant contributions (odds ratio = 1.28 and 1.12, respectively). In the model including all three reporters, higher scores on the SCAS-P-8 best predicted whether participants were in the community or clinic-referred sample (odds ratio = 1.39), and the SCAS-T-8 score also made a significant unique contribution (odds ratio = 1.17), but the SCAS-C-8 did not (odds ratio = 1.02).

As displayed in online supplement 4 , the brief SCAS scores accurately identified >89% of children in the clinic-referred sample when multiple reporters were used, with the highest sensitivity achieved when all three brief questionnaires are combined (.97), and lowest when teacher and child report are combined (.89). The brief SCAS specificity was reduced when multiple reporters were combined; ranging from .54 (parent + teacher and parent + child) to .42 (parent + teacher + child) based on the optimal cut-off points identified in Table 6 .