Children's processing and comprehension of complex sentences containing temporal connectives: The influence of memory on the time course of accurate responses.

The Current Study

Children listened to a two-clause sentence containing before or after, with events narrated either in a chronological or reverse order. During the narration, an animation of the event in each clause was shown, separately, on a touch screen monitor. Children were then asked to touch the picture that represented which of the two events happened last. We did not explicitly manipulate the position of the connective but it varied by the nature of our two within-subject factors: order and connective. Therefore, like others (e.g., Pyykkönen & Järvikivi, 2012 ), we can also relate our findings to connective position in the sentence.

We first hypothesized that the reason for the youngest children’s poor comprehension would be that they use a nonlinguistic strategy to compensate for a fragile understanding of the connective ( Clark, 1971 ). Evidence for this would come from above chance performance for chronological sentences, but not for reverse order sentences. For the older children, we predicted a different pattern of performance, because they were expected to have more robust knowledge of the specific meaning of the connectives. Specifically, we expected these children to perform above chance for all sentence types, reflecting their ability to accurately encode the connective. However, we predicted that their accuracy for reverse order sentences would be lower than that for chronological order sentences, because of the higher processing demands of this sentence type ( Just & Carpenter, 1992 ; Van Dyke et al., 2014 ).

Our second and third hypotheses relate to two different accounts: whether memory ( Just & Carpenter, 1992 ) or language knowledge (e.g., Van Dyke et al., 2014 ) best explains processing difficulties. As memory and language skills both typically improve within the age range of interest, we also predict that whichever skill best explains performance should also explain unique variance over and above the effects of age, thus accounting for developmental improvements. Our use of a timed response measure, in addition to accuracy, provides a sensitive means to assess whether different sentence structures differ in processing ease, as has been found for adults ( Münte et al., 1998 ; Ye et al., 2012 ).

If a memory capacity-constrained account (e.g., Just & Carpenter, 1992 ) best explains processing difficulties, children should be more accurate and faster to respond to sentences that place the least demands on working memory. This account predicts the best performance for sentences with a chronological order that are linked by before (medial position) because these permit incremental word by word processing. All other sentence combinations (before-reverse, after-chronological, and after-reverse) carry two features that increase the amount of information that must be held in working memory (reverse order, more difficult connective, initial position). Critically, this pattern of performance will be predicted by an independent measure of memory.

If a language-based account (e.g., Van Dyke et al., 2014 ) best explains processing difficulties, then language knowledge, as measured by performance across connective ( before , after ) and by an independent measure of vocabulary, should modulate how well children can process and comprehend sentence structures that require more computational effort. More specifically, we would expect slower and less accurate responses to reverse order sentences linked by after , and for the pattern of performance to be driven by our measure of vocabulary knowledge. Critically, the influence of these measures of language knowledge would be expected to override the effects of working memory that would be proposed by the memory capacity account ( Just & Carpenter, 1992 ; as demonstrated by Van Dyke et al., 2014 ).

Note that the influence of connective knowledge that is proposed by a language-based account of sentence processing ( Van Dyke et al., 2014 ) differs to that proposed by the first (nonlinguistic strategy) hypothesis ( Clark, 1971 ). The first hypothesis focuses on whether young children display below-chance accuracy for reverse order sentences: this would be a result of using a nonlinguistic strategy, which is in turn a result of not having a basic appreciation for the meaning of the connective. Conversely, the language-based account of sentence processing ( Van Dyke et al., 2014 ) relates to when children perform above - chance at all sentence structures. Therefore, it focuses on the period that follows children’s appreciation for the meaning of the connective, which is a later period of interest to the first hypothesis and relates to a more fine-grained understanding of the connective that can be used to contrast only the predictions of a memory capacity-constrained account ( Just & Carpenter, 1992 ).

Vocabulary

Our measure of receptive vocabulary was the British Picture Vocabulary Scales—III ( Dunn, Dunn, Styles, & Sewell, 2009 ), in which children have to point to one of four pictures that best illustrates the meaning of a word spoken aloud by the researcher. Testing was discontinued when a specified number of errors had been made, as per the guidelines in the manual. Raw vocabulary scores demonstrated age-related improvements: 3- to 4-year-olds = 64.85 (7.99); 4- to 5-year-olds = 78.71 (7.34); 5- to 6-year-olds = 91.26 (6.74); 6- to 7-year-olds = 98.67 (8.56). All children had a standardized score above 85 and the mean scores ( SD ) indicate that each age group was performing at an age-appropriate level: 3- to 4-year-olds = 108.89 (7.44); 4- to 5-year-olds = 104.43 (8.36); 5- to 6-year-olds = 100.56 (5.62); and 6- to 7-year-olds = 98.38 (7.44).

Memory

Each child completed the digit span subtest from the Working Memory Battery for Children ( Pickering & Gathercole, 2001 ) to assess memory. This is the most suitable assessment of memory for our age range, because 4-year-olds perform at floor on more complex measures of working memory ( Gathercole, Pickering, Ambridge, & Wearing, 2004 ). In this task, children were asked to recall a string of digits in the same order that they were spoken by the experimenter. The easiest level comprises strings of two digits, and the number of items in the string is increased once three trials on level were answered correctly. Raw scores were used for the analysis. The raw memory scores (means and standard deviations) demonstrated age-related improvements: 3- to 4-year-olds = 19.11 (3.23); 4- to 5-year-olds = 22.71 (3.14); 5- to 6-year-olds = 25.78 (3.99); 6- to 7-year-olds = 26.81 (3.74). In addition, the standardized scores of memory were within the normal range of 85–115 for each age group: 4- to 5-year-olds = 103.86 (11.00); 5- to 6-year-olds = 108.70 (14.32); and 6- to 7-year-olds = 106.73 (15.84). Standardized scores are not provided for 3- to 4-year-olds. The test–retest reliability reported in the manual for children aged 5 to 7 years is good ( r = .81).

Design

A 4 × 2 × 2 mixed design was used. The between-subjects independent variable was age group (3–4, 4–5, 5–6, and 6–7 years) and the within-subjects variables were order (chronological, reverse order) and connective type ( before , after ). By manipulating order and connective, we also by nature varied the position of the connective (see Table 1 ). The dependent variables were accuracy and response times.

Results

We report the results for accuracy and RTs separately. For each, a series of generalized linear mixed-effects models (GLMMs; Baayen, Davidson, & Bates, 2008 ) were fitted to the data in the R statistics environment ( R Core Team, 2014 ) using glmer (for the binomial accuracy dependent variable) and lmer (for the continuous RT dependent variable) from package lme4 ( Bates, Maechler, & Bolker, 2014 ). This method is essentially an extension of logistic regression, such that it allows both subject and item effects to be simultaneously treated as random. In other words, a GLMM simultaneously controls for (error) variance that is unexpectedly caused by specific items and specific participants rather than by the fixed effects themselves.

The aim for each model was to have a maximal random effects structure: random intercepts for subjects and items, and random slopes where applicable to the design ( Barr, Levy, Scheepers, & Tily, 2013 ). However, this process highlighted the problems associated with obtaining a maximum model that have been recently outlined by Bates, Kliegl, Vasishth, and Baayen (2015) . Specifically, the information in typical data (i.e., the number of observations per subject and per item) is not sufficient to support the complexity of maximum models. As a consequence of this, our most complex models failed to converge. Using the recommendations of Bates et al. (2015) , fixed and random effects were incrementally added to a minimal model and were justified by using the likelihood ratio test ( Pinheiro & Bates, 2000 ) for comparing models. In addition, the models were pruned so that nonsignificant factors were removed.

We removed 10 children from the analysis: 4 who performed at ceiling across the four sentences (100%), 5 who were identified as outliers in by-age by-sentence box plots, and 1 who was identified as the single outlier in by-age box plots of our independent measure of memory. This did not alter the main findings. Therefore, we report the main effects and interactions of memory, vocabulary, age, order and connective on the accuracy of responses by 98 children.

An initial model ( Table A1 ; see the Appendix ) was built that only examined the effects of age, order and connective. This showed no difference between accuracy for before and after sentences, and no interaction effects between variables (all p s > .15). Therefore, following recommendations to allow more complex models to be clearly interpretable and to be better supported by the data (see Bates et al., 2015 ), these nonsignificant effects were pruned. The pruning of nonsignificant factors did not alter the reported findings ( Table A2 ; see the Appendix ) and, together with the removal of data points, ensured a normal distribution of the data that, in turn, allowed convergence of the final reported model that incorporated the effects of memory and vocabulary (see Table 2 ). Memory and vocabulary were strongly correlated ( r = .69), so were both centered. The addition of memory, χ 2 (2) = 7.23, p < .03, and vocabulary, χ 2 (2) = 7.23, p < .03, both improved the fit of the pruned model ( Table A2 ; see the Appendix ).

The inferential statistics are presented in Table 2 . The first column provides the parameter estimates ( b ), which can be interpreted the same way as a regression, such that each shows the change in the log odds accuracy of response associated with each fixed effect on the dependent variable. A positive value indicates that the effect will benefit accuracy, whereas a negative value indicates that the effect will hinder accuracy. The by-age group mean (and standard deviation) accuracy scores for each sentence type are shown in Figure 1 . There was a significant and sizable effect of order, because chronological sentences were comprehended more accurately than reverse order sentences. There was also a main effect of memory, because children with higher working memory scores were significantly more accurate on the sentence comprehension task. There were no significant interactions between the variables. The influence of memory was over and above age and vocabulary, which were both nonsignificant. This contrasts with the finding reported in the initial models that had not incorporated memory and vocabulary ( Table A1 and Table A2 ; see the Appendix ): These had reported a main effect of age, with each of the three older age groups performing significantly more accurate than the 3- to 4-year-olds. This indicates that the effects of age in those initial models served as a proxy for the role of memory.

We also investigated a possible trade-off between accuracy and RTs. However, the fit of the final reported model (see Table 2 ), was not improved when RTs were added as a fixed effect covariate, χ 2 (2) = 0.34, p < .84 or as item-wise random intercepts, χ 2 (1) = 0.83, p < .36. Similarly, these additions did not significantly improve the fit of the models reported in the Appendix ( Table A1 and Table A2 ), all p s > .90.

We followed up the main effect of order with one-sample t tests to examine whether each age-group performed above chance for chronological compared to reverse order sentences. Our youngest two age groups performed above chance for before-chronological sentences (3- to 4-year olds: t [26] = 2.93, p < .01; 4- to 5-year olds: t [27] = 4.21, p < .01) and after-chronological sentences, (3- to 4-year olds: t [26] = 2.82, p < .01; 4- to 5-year olds: t [27] = 5.82, p < .01). However, these children were not above chance level for before-reverse sentences (3- to 4-year olds: t [26] = −1.60, p = .94; 4- to 5-year olds: t [27] = −0.85, p = .80), or after-reverse sentences (3- to 4-year olds: t [26] = −1.17, p = .87; 4- to 5-year olds: t [27] = −1.38, p = .09). This pattern of performance indicates that their inaccuracy for reverse order sentences was likely a result of their fragile understanding for the meaning of before and after. Conversely, despite performing less accurately for revers

RT Analysis

We did not include responses by 3- to 4-year-olds because their longer RTs suggested that they were not able to follow the instruction to respond as quickly as possible. The 1,816 correct responses by 4- to 7-year-olds were screened following recommendations from Baayen and Milin (2010) to remove potential distortions from the norm and improve the convergence of models. We first removed extreme RTs that exceeded 2.5 standard deviations past the overall mean (49 responses over 9.5 s). Second, we removed remaining outliers that were more than 2.5 standard deviations above the mean response by subject (54 responses) and by item (42 further responses). Thus, a total of 8% of the original data points were removed as outliers. In addition, the data of one 6- to 7-year-old was removed because they were identified as an outlier in by-age box plots of our independent measure of memory. The mean (and standard deviation) RTs in seconds by age-group were 1.75 (1.40) for 4- to-5-year-olds, 1.19 (1.17) for 5- to 6-year-olds, and 1.11 (1.27) for 6- to 7-year-olds. Mean RTs for all correct responses in each experimental condition are presented in Figure 2 . Nontransformed means are reported for ease of interpretation. When 3- to- 4-year-olds were screened using this method, their RTs were 2.96 (2.20) s, hence their exclusion.

A square root transformation was used for the inferential analysis so that the data were normally distributed. As in the accuracy analysis, an initial model was built which did not incorporate memory and vocabulary as covariates ( Table A4 ; see the Appendix ). However, the RT model was not pruned, because age, order and connective each had either a significant main effect or were involved in an interaction. The same pattern of findings was found in a model of nontransformed RTs (see Table A3 ; see the Appendix ), but our final model (see Table 3 ) reports the square root transformation because the normal distribution reduced the stress on the model and, in turn, allowed the convergence of the additional effects of (centered) memory and (centered) vocabulary. In GLMMs of data with a continuous dependent variable, it is custom to present t-values and confidence intervals rather than p values because, for reasons beyond the current study, the statistical function lmer (from package lme4 ; Bates et al., 2012) does not provide p values. Reliably, a significant effect is indicated by a t-value exceeding 2, and when confidence intervals do not pass zero ( Baayen, 2008 ).

Table 3 summarizes the main effects and interactions of memory, age, order and connective on RTs. Similar to the accuracy analysis, there was no main effect of age once memory was added as a covariate, indicating that working memory was driving the developmental improvement in the processing of sentences overall. In contrast to the analysis of the accuracy data, there was a main effect of connective: RTs to sentences with before were faster than for sentences with after . Also in contrast to the analysis of accuracy data, the main effect of order was not significant: RTs to chronological sentences were not significantly different to those for reverse order sentences.

The main effect of connective was qualified by a three-way interaction between age, order and connective. The influence of age on the effects of order and connective indicates a developmental improvement in the processing of sentences. Therefore, the interaction was broken down by age. This is reported in Table 4 with by age-group models of the effect of order in a subset of each connective. The RTs by 4- to 6-year-old’s were significantly influenced by an interaction between order and connective, whereas older children’s RTs were not. In the 4- to 6-year-olds, there was a main effect of order for before sentences, but not for after sentences. Specifically, before-chronological sentences were responded to significantly faster than before-reverse sentences, whereas RTs to chronological and reverse order sentences containing after did not differ.

In line with the accuracy data, the addition of memory to the model significantly improved the fit of the data, χ2(4) = 11.43, p = .02. Children with higher memory capacity made faster (correct) responses overall. Most notably, there was a significant two-way interaction between memory and order, and also one between memory and connective. These interactions indicate that memory predicted the effects of both connective and order. Vocabulary did not improve the fit of the data, χ2(4) = 6.53, p = .16. Therefore, we do not report models of RTs that incorporate vocabulary. This indicates that processing times were driven by memory capacity rather than vocabulary per se .

Discussion

This study was designed to identify the reasons why children continue to experience difficulties in comprehending sentences containing before and after beyond the age that they have begun to display an early competence for these connectives. In general, there were developmental improvements in performance, such that sentences were understood more accurately and processed more quickly by older children. In relation to event order, children were less accurate at comprehending reverse order compared to chronological sentences. Our experimental manipulation of sentence type, together with independent measures of memory and language knowledge, enabled us to test between different theoretical accounts of children’s difficulties with such sentences. The precise pattern of findings indicates different reasons for this effect in younger and older children. As discussed subsequently, the evidence suggests that younger children’s performance with reverse order sentences was limited because they displayed little or no understanding of the connective and instead relied on a nonlinguistic strategy ( Clark, 1971 ). In contrast, older children’s overall performance indicated that they knew the meanings of the two connectives. A consideration of the pattern of performance and how this was related to individual differences in memory and language skills, suggests that older children’s performance was limited by the processing demands of these sentences ( Just & Carpenter, 1992 ; Van Dyke et al., 2014 ). We first examine the findings of the accuracy analysis and then turn to the analysis of RTs, and discuss why variability in children’s processing of these sentences is best explained by a memory capacity-constrained account (e.g., Just & Carpenter, 1992 ).

Our findings for response accuracy are convergent with the developmental findings reported by previous studies of children’s comprehension of sentences with temporal connectives ( Blything et al., 2015 ; Clark, 1971 ; Pyykkönen & Järvikivi, 2012 ). Children aged 3 to 5 years performed above chance on chronological sentences, but not for reverse order sentences. This difference indicates that they did not take full advantage of the event order that is signaled by the connective and compensated for this by defaulting to an expectation that language order maps onto the actual order of events ( Clark, 1971 ). The 5- to- 7-year-olds performed above chance for all sentence types, which reflects an appreciation for the meaning of the connectives. However, they were in general poorer on reverse order sentences. Because older children displayed an appreciation for the meaning of the connectives, one reason for the lower accuracy for reverse order sentences is that these sentences have higher processing costs ( Pyykkönen & Järvikivi, 2012 ).

Performance on the accuracy task was best explained by memory rather than chronological age or vocabulary. This finding provides partial support for the memory capacity-constrained account ( Just & Carpenter, 1992 ). That is, performance was driven by whether children’s memory capacity was sufficient to cope with the processing demands of our sentences in general. However, the account is only partially supported because the inaccurate comprehension of reverse order compared to chronological sentences did not interact with memory. We argue that the absence of this interaction could be attributed to the task requirement to provide speeded responses. When children are required to respond quickly, they have less time to reflect on and revise the representation that they have constructed and stored in memory (see Marinis, 2010 ). As a result, the ability to accurately store and manipulate the contents of memory may have a weaker influence on accuracy. Therefore, we turn to our RT measure, to better understand our pattern of data and the processing difficulties experienced by children with these sentence types.

RTs were analyzed for only correct responses to determine if different connectives or structures differed in ease of processing. Thus, the pattern of data cannot be compared directly with the accuracy data. The RT analyses indicate that, even when sentences with temporal connectives are comprehended correctly, some are more difficult to process than others (e.g., Cain & Nash, 2011 ; Ye et al., 2012 ). The RT data support the memory capacity-constrained account ( Just & Carpenter, 1992 ). Children responded most quickly to chronological order sentences linked by before (medial position), which allow incremental word by word processing; and more slowly to before-reverse sentences, which do not afford incremental processing. There was no effect of order for sentences containing after. After-chronological sentences (initial position, later acquired connective) sentences and after-reverse sentences (reverse order, later acquired connective) each carry two features associated with taxing information to be held in working memory, and do not permit incremental processing. This may be the reason for the absence of RT differences between these two sentence types.

Importantly, the incorporation of memory significantly improved the fit of the model for RTs, whereas vocabulary did not. Moreover, the main effect of age was no longer significant when memory was added to the model. Instead, the main effect of memory can account for developmental improvements in the processing of these sentences. This suggests that, as in the accuracy findings, age effects were partly a proxy for the influence of memory. Of particular note, the variation in RTs across our sentence structures was predicted by our independent measure of memory span. This indicates that demands on working memory are driving these effects. That is, children with higher working memory spans are better able to cope with the higher memory demands of difficult sentences, and so experience fewer problems, as do adults ( Just & Carpenter, 1992 ).

In turn, the support we provide for