The utility of the SCAS-C/P to detect specific anxiety disorders among clinically anxious children.

Procedure

Data collected as part of the pretreatment assessment in the Genes for Treatment study was used in the current study (see Hudson et al., 2015 for further details). Children ( N = 1,438) completed the SCAS-C, and mothers ( n = 1,438) and fathers ( n = 953) completed the SCAS-P. The ADIS-C/P was used to assign anxiety and comorbid diagnoses, and associated CSRs in all sites except at Bochum, where the Diagnostisches Interview bei psychischen Störungen im Kindes- und Jugendalter (Kinder-DIPS) [ Schneider, Unnewehr, & Margraf, 2009 ] was used. 1 All trials were approved by site-specific research ethics committees. Parents provided consent, and children provided assent.

Anxiety Disorders Interview Schedule

Diagnostic status was assessed using the ADIS-C/P-IV ( Silverman & Albano, 1996 ) across all sites, with the exception of Bochum, where the German equivalent, Kinder-DIPS ( Schneider, Unnewehr, & Margraf, 2009 ) was used. The ADIS-C/P consists of independent parent and child interviews, and its reliability and validity are widely reported ( Silverman, Saavedra, & Pina, 2001 ). The presence and severity of anxiety disorders were assessed across all sites; and all sites (with the exception of Bergen) also assessed comorbid mood and externalizing disorders with this interview. Diagnoses were assigned if a child met the DSM–IV ( American Psychiatric Association, 1994 ) criteria and received a CSR of 4 or above, based on the composite parent and child report (see Hudson et al., 2015 , for further details). As per interview schedule guidance, when there were discrepancies between the child and parent reports, diagnoses were assigned if symptoms were reported by either the child or the parent, and the higher CSR was assigned as the overall CSR. Good interrater reliability (κ ≥ 0.8) for clinician-assigned diagnoses within samples used in this study are reported elsewhere ( Creswell, Apetroaia, Murray, & Cooper, 2013 ; Hudson et al., 2014 ; Lyneham, Abbott, & Rapee, 2007 ).

Data Analytic Approach

The ability of four SCAS-C/P subscales (Separation Anxiety, Social Phobia, Generalized Anxiety, Physical Injury Fears) to identify corresponding DSM–5 ( American Psychiatric Association, 2013 ) anxiety disorders (separation anxiety disorder; social anxiety disorder; generalized anxiety disorder; specific phobia) was examined. There were not sufficient panic disorder (2.2%) or agoraphobia (1.6%) diagnoses to examine the functioning of the Panic/Agoraphobia subscale.

Analyses examining only child- and/or mother-report SCAS subscale scores included the total sample ( N = 1,438), and the subsample for which father report was available ( n = 953) was used for analyses that included father-report SCAS subscale scores.

There are different published norms and t scores for preadolescent girls and boys for the SCAS (available on www.scaswebsite.com ), and therefore it is likely that optimal subscale cutoff scores designed to detect the corresponding disorders will similarly vary for girls and boys. To determine whether it was appropriate to consider girls and boys separately in subsequent analyses, first, gender differences on these four SCAS-C/P subscale scores (independent samples t tests) were examined for each reporter (child, mother, father; see Online Supplement 1 of the online supplemental materials). Significant gender differences ( p < .05) were observed on all child-report subscales, three mother-report subscales (Separation Anxiety, Generalized Anxiety, Physical Injury Fears), and the father-report Separation Anxiety subscale. To allow a consistent approach across analyses, girls and boys were considered separately in all subsequent analyses.

The capacity of each of the four SCAS-C/P subscales (based on child, mother, and father reports) to discriminate between children with and without the related anxiety disorder was examined using (a) independent sample t tests (and Cohen’s d ), and (b) receiver operating characteristics (ROCs). ROC analyses produce an area under the curve (AUC) statistic, ranging from 1.0 (indicating perfect classification of children with/without the disorder) to .50 (indicating chance-level classification of children with/without the disorder). In line with previous studies using ROC analyses to examine child anxiety measures ( van Gastel & Ferdinand, 2008 ; Villabø, Gere, Torgersen, March, & Kendall, 2012 ), a minimum threshold of an AUC of .70 was used to indicate that the SCAS-C/P subscale was at least moderately accurate in identifying the corresponding anxiety disorder. In cases in which the AUC was >.70, the sensitivity (correct classification of children with the target anxiety disorder) and specificity (correct classification of children without the target anxiety disorder) values for alternative cutoff scores were also examined. Identifying optimal cutoff scores involves a trade-off between sensitivity and specificity. With a focus on identifying the target disorder (and not missing cases), sensitivity was prioritized, and the optimal cutoff score reflected the score with sensitivity >.80 and specificity >.70. If it was not possible to achieve this .80/.70 combination, cutoff scores with lower sensitivity values (<.80) and specificity values >.60 were considered. For optimal cutoff scores, overall correct classification (i.e., number and percentage correctly classified) was also calculated.

Agreement between child–mother, child–father, and mother–father report on the four subscale scores was examined using Pearson correlations. Four logistic regressions were then used to examine the unique contribution of child, mother, and father reports in identifying the four target anxiety disorders (separation anxiety disorder, social anxiety disorder, generalized anxiety disorder, specific phobia). For each regression model, the corresponding child-mother-father subscale scores were entered using the block-entry method. In cases in which the ROC analyses indicated that the SCAS-C/P subscale was at least moderately accurate at identifying the corresponding anxiety disorder (i.e., AUC >.70), and an optimal cutoff score was identified, the sensitivity/specificity associated with each combination of reporters was also examined. It is possible to combine information from multiple reporters in different ways. In keeping with the standard approach used to combine information from multiple reporters in diagnostic interviews, and with the aim of maximizing the capacity to identify specific disorders, an “OR-rule” was used (i.e., children who scored above the cutoff score for at least one reporter were classed as “above the cutoff” overall). For each combination of reporters (child–mother, child–father, mother–father, child-mother-father), the following were calculated: (a) the proportion of children with the target anxiety disorder who scored above the optimal cutoff score on the corresponding subscale for at least one reporter (sensitivity), and (b) the proportion of children without the target anxiety disorder who scored below the optimal cutoff score on the corresponding subscale for each reporter (specificity). The total number (and percentage) of children who were correctly classified was also calculated, that is, children with the target anxiety disorder who scored above the optimal cutoff score on the corresponding subscale for at least one reporter plus children without the target disorder who scored below the optimal cutoff score on the corresponding subscale for each reporter.

ROC Analyses

ROC analyses for each SCAS-C/P subscale for the three reporters (child, mother, and father) are displayed in Table 3 . The Separation Anxiety subscale (child, mother, and father reports) was able to accurately identify separation anxiety disorders among both girls and boys (AUC = .73–.82). Optimal cutoff scores for each reporter were associated with sensitivity values >.70 (.70–.78) and corresponding specificity values >.60 (.62–.75).

The mother- and father-report Social Phobia subscale was able to accurately identify social anxiety disorders among both girls and boys (AUC = .70–.77). Optimal cutoff scores for mother and father reports were associated with sensitivity values of .70 to .71 among girls and .66 to .67 among boys, with corresponding specificity values of .69 to .71 among girls and .63 to .67 among boys. The child-report Social Phobia subscale achieved an AUC >.70 among girls (AUC = .71) but not boys (AUC = .65). Among girls, the optimal cutoff score on the child-report Social Phobia subscale achieved sensitivity of .67 and specificity of .65.

The Generalized Anxiety subscale was not able to accurately identify children with generalized anxiety disorder (AUC < .70 for child, mother, and father reports). The Physical Injury Fears subscale also failed to identify children with specific phobias (AUC < .70) for child or mother report. The father-reported Physical Injury Fears subscale, however, did achieve an AUC of .70 among girls (but not boys), and the associated optimal cutoff score achieved sensitivity and specificity values of .61 and .71, respectively.

Using Multiple Informants

Correlations between child–mother, child–father, and mother–father report on the four subscales are displayed in Online Supplement 2 of the online supplemental materials. Across all subscale and gender groups, mother–father agreement ranged from .43 to .71, child–mother agreement ranged from .35 to .55, and child–father agreement from .26 to .51. Mother–father correlation coefficients ranged from .67 to .70 for the Separation Anxiety subscale to .43 to .48 for the Generalized Anxiety subscale. Child–mother correlation coefficients were similar on the Separation Anxiety and Physical Injury Fears subscales (.50–.55), and ranged from .35 to .42 for the Social Phobia and Generalized Anxiety subscales. Child-father correlation coefficients ranged from .44 to .51 on the Separation Anxiety and Physical Injury Fears subscales, to .26 to .29 on the Social Phobia and Generalized Anxiety subscales.

Table 4 displays findings from logistic regressions examining the contribution of child, mother, and father reports in identifying separation anxiety disorder, social anxiety disorder, generalized anxiety disorder, and specific phobias. Higher scores on the Separation Anxiety subscale for each reporter were associated with separation anxiety disorder among girls and boys (odds ratio [ OR ] = 1.11–1.27), indicating that each reporter made a unique contribution. The Nagelkerke and Cox and Snell R -squared statistics indicated that the separation anxiety model explained .35 to .46 of the variance among girls and .26 to .35 among boys.

Child-, mother-, and father-reported Social Phobia subscale scores also each made a significant contribution in identifying social anxiety disorders ( OR = 1.10–1.18), and overall, the model explained .25 to .34 of the variance among girls and .18 to .24 among boys.

Higher scores on the Generalized Anxiety subscale were not, however, associated with generalized anxiety disorders based on child, mother, or father reports, and overall the generalized anxiety disorder model explained very little of the variance among girls or boys (Nagelkerke = .03/.02, Cox & Snell = .05/.04). Similarly, child-reported Physical Injury Fears subscale scores were not associated with specific phobias. Both father- and mother-reported Physical Injury Fears subscale scores each made a significant contribution to identifying specific phobias among girls ( OR = 1.24 and 1.12, respectively); and mother report made a significant contribution to identifying specific phobias among boys ( OR = 1.11). Overall, the specific phobia models, however, explained a small amount of the variance (girls, Nagelkerke = .12, Cox & Snell = .17; boys, Nagelkerke = .08, Cox & Snell = .11).

Sensitivity and specificity values associated with using two or more reporters were calculated for subscales when optimal cutoff scores were identified for each reporter (i.e., Separation Anxiety subscale and Social Phobia subscale among girls). As displayed in Table 5 , combining two or three reporters improved the Separation Anxiety subscale sensitivity (.88–.92) but reduced its specificity (.44–.60). This reduction in specificity was less marked for mother–father report (specificity = .57–.60) than either mother–child (.50–.52), father–child (.49), or mother-father-child (.44–.45).

Similarly, combining two or three reporters improved the Social Phobia subscale’s sensitivity among girls (.87–.92) but reduced its specificity (.40-.56). Again, mother–father report produced higher specificity (.56) on the Social Phobia subscale than other reporter combinations.

Discussion

We examined the capacity of the SCAS-C/P subscales to detect specific anxiety disorders (separation anxiety disorder, social anxiety disorder, generalized anxiety disorder, specific phobias) within a large, multisite, clinically anxious sample ( N = 1,438) of children aged 7 to 12 years. There was variation in the extent to which scores on each subscale were able to discriminate between children with and without that corresponding anxiety disorder and the accuracy with which each subscale identified children with the target disorder.

The Separation Anxiety subscale score was able to discriminate between children with and without separation anxiety disorder, with significantly higher scores among children with than without separation anxiety disorder based on child, mother, and father reports ( d = .82–1.31). This subscale also identified children with separation anxiety disorder with a moderate to good level of accuracy across the three reporters (AUC = .73–.82), and the optimal cutoff scores achieved an acceptable sensitivity/specificity balance (>.70/>.60). The Separation Anxiety subscale’s ability to accurately identify separation anxiety disorders in preadolescent children is in line with previous illustrations of its ability to accurately identify recovery from the corresponding anxiety disorder within the same age group ( Evans et al., 2017 ) and its stronger predictive capacity than other SCAS subscales among adolescents ( Olofsdotter et al., 2016 ).

The performance of the Social Phobia subscale showed some variation across reporters. The mother- and father-report Social Phobia subscale score discriminated between children with and without social anxiety disorders, with significantly higher scores among the former ( d = .72–1.02), and also identified children with social anxiety disorder with a moderate level of accuracy (AUC = .70–.77). The optimal cutoff scores on the mother- and father-report Social Phobia subscale achieved acceptable sensitivity/specificity (for girls, .70–.71/.69–.71; for boys, .66–.67/.63–.67). Interestingly, these positive findings in relation to the parent-report Social Phobia subscale contrast with previous findings that the Social Phobia subscale failed to accurately identify recovery from social anxiety disorders ( Evans et al., 2017 ). The parent-report Social Phobia subscale’s utility as an identification tool may therefore be greater than its utility to monitor treatment response. Similar to mother and father reports, the child-report Social Phobia subscale scores were also significantly higher among children with than without social anxiety disorder (girls, d = .77; boys, d = .55). Child report, however, only identified social anxiety disorder with an acceptable level of accuracy among girls (AUC = .71), with sensitivity/specificity values of .67/.65. Previous studies that include adolescents report positive findings in relation to the Social Phobia subscale’s ability to identify social anxiety disorders using both self-report and parent report ( Brown-Jacobsen et al., 2011 ; Olofsdotter et al., 2016 ). The limited capacity of the social anxiety SCAS-C items to discriminate between a clinically anxious and community sample of preadolescent children is, however, reported elsewhere ( Reardon et al., 2018 ). It is therefore possible that preadolescent children have limited ability to differentiate between developmentally appropriate and clinically significant social anxieties, but this ability improves with age.

The capacity of the Generalized Anxiety subscale score to discriminate between children with and without generalized anxiety disorder was limited. Although child-, mother-, and father-reported Generalized Anxiety subscale scores were significantly higher among children with than without generalized anxiety disorder, effect sizes were small ( d = .26–.42). The Generalized Anxiety subscale also failed to accurately identify children with generalized anxiety disorder across reporters (AUC <.70). Doubt surrounding the Generalized Anxiety subscale’s ability to accurately detect generalized anxiety disorder is also reported elsewhere. Brown-Jacobsen et al. (2011) reported poorer performance for the Generalized Anxiety subscale compared with other subscales in relation to the sensitivity/specificity achieved in a sample of children and adolescents, and Nauta et al. (2004) reported similarly high scores on the parent-report Generalized Anxiety subscale among children with generalized anxiety disorder as those with other anxiety disorders. Interestingly the predictive capacity of the MASC Generalized Anxiety subscale is similarly limited ( Villabø et al., 2012 ). There are, however, also more positive illustrations of the capacity of both the SCAS and the RCADS (a derivative of the SCAS) Generalized Anxiety subscales to detect generalized anxiety disorder ( Chorpita, Moffitt, & Gray, 2005 ; Ebesutani et al., 2010 ; Olofsdotter et al., 2016 ), but as these studies include adolescents, it is possible that the SCAS and the RCADS Generalized Anxiety subscales are better able to detect generalized anxiety disorder in adolescents than preadolescent children and that the ability to identify clinically significant levels of worry improves with age.

Given that the SCAS Generalized Anxiety subscale addresses anxiety symptoms that are common across anxiety disorders (general worry, worries about bad things happening, physical symptoms), it may not be surprising that its capacity to discriminate between children with generalized anxiety disorder and children with other anxiety disorders is limited. Indeed, although studies examining the factor structure of the SCAS provide support for a six-correlated-factor model, corresponding to the six subscales ( Orgilés et al., 2016 ), an alternative model with five correlated factors and a higher order Generalized Anxiety factor has also been proposed ( Nauta et al., 2004 ), suggesting the Generalized Anxiety subscale is

Implications

This study has implications for the potential use of the SCAS-C/P subscales to detect specific anxiety disorders in preadolescent children in clinical practice. Findings provide support for the use of the child- and parent-report Separation Anxiety subscale for identifying children with separation anxiety disorders. Findings also support the use of the parent-report Social Phobia subscale for identifying children with social anxiety disorders, and the child-report Social Phobia subscale for identifying girls with social anxiety disorders. This study provides data relating to both mother and father optimal cutoff scores, and so offers potential for application in settings in which only mother or father report is available. When multiple reporters are available, clinicians and researchers will need to weigh the improved capacity to identify the presence of separation anxiety disorder and social anxiety disorder associated with using multiple reporters against the reduced capacity to correctly identify the absence of separation anxiety disorder and social anxiety disorder. These findings are of particular importance to clinical settings in which questionnaire measures are commonly used as a time- and cost-effective means to identify potential diagnoses. Moreover, the RCADS ( Chorpita, Yim, Moffitt, Umemoto, & Francis, 2000 ) is a derivative of the SCAS, and items on the SCAS Separation Anxiety and Social Phobia subscales also appear on the RCADS. These findings therefore have relevance to clinical services that routinely use the RCADS when it would be possible to use the SCAS Separation Anxiety and Social Phobia subscales items to identify children with the corresponding disorders. Importantly, the study suggests that the SCAS Generalized Anxiety subscale and Physical Injury Fears subscale should not be relied upon as measures to specifically identify children with generalized anxiety disorder and specific phobias in clinical populations.

Limitations

There are a few limitations associated with this study. We examined the capacity of SCAS-C/P subscales to detect four anxiety disorders, but there were not a sufficient number of children with either panic disorder or agoraphobia to examine the capacity of the Panic/Agoraphobia subscale to detect children with these disorders. Standard diagnostic interview schedules were used to assess diagnoses, but it was not possible to evaluate interrater reliability for clinician-assigned diagnoses across all sites included in the sample. Generalized anxiety disorder was the most common diagnosis within the sample, and the fact that only a relatively small proportion of children (25%) did not have generalized anxiety disorder may have contributed to the SCAS subscale’s failure to accurately detect this disorder. Moreover, all children in this sample met criteria for at least one anxiety disorder, and therefore we were not able to examine the capacity of the SCAS subscales to discriminate between children with specific anxiety disorders and children without any anxiety disorders. Indeed, the variance on SCAS subscale scores was limited in this study, and it is likely that our results underestimate the capacity of the subscales to detect the target disorders compared with what we may expect to find in a community sample. Similarly, this study examined how well the SCAS can identify specific anxiety disorders within clinical populations, but we were not able to examine its capacity of to discriminate between children with and without any anxiety disorders.

It is also important to acknowledge that the SCAS was designed to assess symptoms consistent with DSM–IV ( American Psychiatric Association, 1994 ) anxiety disorders. The SCAS items addressing obsessive and compulsive behaviors are therefore not consistent with the DSM–5 ( American Psychiatric Association, 2013 ) classification of anxiety disorders, in which obsessive–compulsive disorder is no longer classified as an anxiety disorder; and no SCAS item(s) specifically address the newly classified anxiety disorder, selective mutism. Changes from DSM–IV to DSM–5 in the diagnostic criteria for anxiety disorders were, however, minor and do not alter the relevance of the other SCAS items and subscales. It will nevertheless be important for future research to examine the capacity of the SCAS (or subset of items) to specifically detect children with selective mutism. Indeed, Muris et al. (2017) report the close association between the SCAS Social Anxiety subscale and a new Selective Mutism scale, suggesting the capacity of the SCAS Social Anxiety subscale to detect children with selective mutism warrants investigation.

This study provides support for the ability of the SCAS Separation Anxiety and Social Phobia subscales to identify preadolescent children with separation anxiety disorder and social anxiety disorder in clinical populations, and provides optimal cutoff scores for mother, father, and child reports. It will also be important for future research to evaluate the capacity of mother-, father-, and child-report SCAS subscales to detect specific anxiety disorders among adolescents.