# A08 · Statistical analysis specification

Version 1.0 ·15 October 2026. Prospective study; zero participants. The public Python tool implements the corpus audit, artificial-data simulations and a complete-cohort mean-score estimator. It does **not** implement multiple imputation, crossed-effects fitting, reliability intervals or human case validation. No such fitted result is reported.

## Estimand, assignment and measurement

160 adults, four pre-allocation strata of40;20assigned to each arm in every stratum. Strata cross leadership/practitioner-support role and lower/higher prior research-agent use(see instruments). Twelve fixed cases per participant; two independent masked human raters score six 0–4 dimensions. Participant outcome=mean of twelve case totals after averaging rater totals. Scale0–24. Equal-stratum intention-to-treat contrast; conditions defined by frozen interface+training+source manifest. No per-protocol exclusion for not using the tool.

Independent custodian generates20 of 40assignment within each stratum after enrollment/baseline lock, records seed/procedure in a restricted audit file and releases assignments. No actual allocation or recruited roster exists in this package. Case order balanced independently across arms using predetermined order lists; preserve order variable. Pilot recruitment/raters are separate.

## Complete-cohort primary estimator

For stratum h=1,…,4, let ybar_h1 and ybar_h0 be participant means; estimate tau=(1/4)sum_h(ybar_h1−ybar_h0). For each of eight cells c, define a_c=s²_c/(16n_c). Estimated variance U=sum_c a_c. Welch–Satterthwaite degrees of freedom nu=U²/[sum_c a_c²/(n_c−1)]. Report tau,SE=sqrt(U),95%tinterval and two-sided p. This is an approximate Neyman sampling/randomization inference, not an exact Fisher test. Actual complete input requires20 per arm/stratum,160 unique participant IDs and caseCount12. Synthetic test fixtures can be smaller with at least2per cell. Unequal variances are permitted; zero/infinite variance, duplicates, unknown arms and out-of-range means raise errors.

The article’s primary target is the recruited cohort on the fixed cases. The design does not sample districts or all educational questions randomly. A case-level response is not an independent participant. No claim of national representativeness or student-effect identification follows.

## Missingness and observed noncompletion

A deliberately unanswered brief during a functioning, otherwise completed timed task is an observed zero only under the locked instrument rule. Technical loss, withdrawal and interrupted sessions are missing, not zeros. Reasons recorded before unmasking. No outcome is reconstructed from private logs after consent withdrawal when retention is not permitted.

Confirmatory missing-data analysis:50 imputations of missing case totals using a bounded multilevel predictive model. Use truncated normal likelihood0–24, fixed assignment, four-stratum indicators, centered baseline literacy, case fixed effects, case-order position and assignment×case terms; participant random intercept. Use four chains,1,000 warm-up+1,000 retained draws/chain, fixed independent seeds; standardize continuous predictors; normal(0,2.5) priors on standardized coefficients, intercept normal(12,6), half-normal(0,6) SD priors. Generate50 separated posterior predictive completed datasets only after convergence checks(Rhat<1.01, effective sample size>400 for arm/variance terms, no divergent transitions). These are planned settings, not executed fits or proof of calibration. Pilot/preregister any justified model simplification before outcome inspection; failed diagnostics lead to reported unresolved inference rather than silent complete-case substitution.

Calculate participant mean and complete-cohort contrast/variance in each completed dataset. Rubin pooling: Qbar=meanQ_m; Ubar=meanU_m; B=sample variance ofQ_m; T=Ubar+(1+1/M)B. Use Barnard–Rubin small-sample adjusted degrees of freedom with complete-data nu_com set to the harmonic mean of the M complete-data Welch dfs; report this choice and a normal-reference sensitivity. Missing-data sensitivity shifts each imputed treatment case downward and control case upward by1then2points, truncating to0–24; include reversed shifts, arm-specific patterns and0/24extreme bounds. Complete-case contrast secondary whenever missingness exists. Report arm-specific missingness and denominator before all effect estimates.

## Secondary and exploratory analyses

Report sentinel-error proportions and absolute arm differences with participant-level resampling intervals(2,000 stratified bootstrap samples preserving arm/stratum), separated by error category and rater/adjudication. No equivalence or noninferiority claim without a separately justified bound. Confidence calibration:mean squared difference between confidence probability and a locked binary adequacy threshold(total≥18 AND no adjudicated sentinel error), with cluster-respecting resampling. Threshold is proposed, must be validated/frozen before recruitment. Secondary time outcomes include total and source-verification time; report medians, differences and bootstrap intervals. Satisfaction is descriptive.

A secondary crossed-effects model includes fixed arm, stratum, order and case-domain effects; random participant and case intercepts and, if identifiable, treatment-by-case random effect. Ordinal/dimension sensitivity models are exploratory; singular fits are reported rather than used to select a favorable model. Interactions by case domain and baseline experience are exploratory with descriptive intervals. Two secondary test families(fidelity subdimensions; process/time outcomes) use Benjamini–Hochberg within-family adjustment if formal tests are presented. Sentinel errors remain prominently reported with counts/intervals regardless of multiplicity or primary significance. No claim of a randomized mediator effect.

Inter-rater agreement:ICC(A,2)absolute agreement of mean two-rater totals, with participant-respecting bootstrap intervals; dimension cross-tabulations and ordinal agreement summaries. Rater-arm guesses after scoring are descriptive. Third-rater review does not overwrite selected primary scores. All raters score all cases independently.

## Executed artificial-data simulations

Seed20261015; NumPy 2.3.5, SciPy 1.17.0;10,000 replicates for each of nine combinations N 80/160/240 and within-person correlation rho.10/.30/.50. Twelve standardized cases/person, marginal variance1; person-mean variance=rho+(1−rho)/12. Generate independent Gaussian participant means within four equal strata and two arms. Effect=.25caseSD. Apply the above cell-variance estimator and Welch df per replicate. Report power, null rejection rates and MCSEsqrt(p(1−p)/R). No dropout, bounded-score ceiling, case-treatment heterogeneity or source effects. These results are design sensitivity, not treatment evidence. Analyze variance/case feasibility in a pilot before final target commitment; any design change registered before allocation.

## Corpus audit and version rules

Query and36-record boundary are in corpus-audit.json. Parse only actual author contributions, body, article-meta permissions and actual DOI identifiers. Resolve ALI license_ref as well as xlink references; exclude data-only CC0 from article permission. Generic unversioned CC BY remains unresolved. NC/ND and mixed conflicts excluded. Exact core/JATS DOI agreement required; F1000 .2/.3conflict retained. Screen retraction/withdrawal/EoC signals only in documented fields, preserving corrections separately. These checks do not certify integrity or validate a paper agent. No source full text or figure distributed. All original identities retained.

## Decision rules and reporting

Single confirmatory endpoint, two-sided alpha .05. Practical threshold and acceptable sentinel-error/workload bounds must be chosen with independent human stakeholders before registration; none invented as an empirically validated threshold here. Publish null and adverse results, confidence intervals, all exclusions, source/agent manifest and amendments. Do not claim educational/clinical effectiveness from this benchmark. No AI-generated peer-review report can authorize publication. A08 is a new unreviewed protocol; conceptual A03 review is not inherited.