The next EdTech question is how the claim was tested

01

An evidence label can conceal several different questions

A product described as evidence-based may be supported by a study, but the description still leaves a school with work to do. What was tested, with whom, for which outcome and under what conditions? A finding about a particular use is not automatically a finding about the whole product. A result obtained with intensive support is not automatically an account of routine operation. Independent leaders need those distinctions before placing institutional weight on the claim.

The October development builds directly on the May partnership we examined earlier. On 22 October EEF uploaded an invitation to tender for a practical guide to evaluating EdTech and AI tools. It follows DfE’s September programme guidance. This is a development in how evidence will be generated and assessed. It is not an announcement that the programme has already demonstrated the effectiveness of particular products.12

02

The method needs to match the decision

The invitation describes work on evaluation design, implementation, outcomes and engagement with developers. The fact that these questions are being addressed is consequential. A useful evaluation has to balance rigour with the pace at which tools change, but speed does not remove the need to define what the study can establish. The school should be able to see the question behind the reported result.1

School Connection’s assessment is that leaders should start with the decision they face. Are they deciding whether to investigate a use, undertake a limited trial or extend an approach across the school? Each requires evidence proportionate to the consequences. Treating an early study as an automatic mandate for a full introduction across the school can give it more authority than its design supports. Dismissing all preliminary evidence can also prevent useful learning.

03

Preserve the difference between the programme stages

The May announcement described an evidence pipeline from rapid evaluations towards more robust testing of promising approaches. The September programme material develops the wider programme. These are stages in research and implementation, not a single universal approval route for school software. Participation, selection for testing and a positive finding must remain distinguishable when a product describes its relationship with the programme.23

An independent school should therefore ask for the relevant study or method rather than rely on a programme logo or a broad statement of involvement. That request is not an allegation that a developer is misleading anyone. It is ordinary scrutiny of the claim on which the school is considering relying.

The resulting explanation should make clear which uncertainty the evidence reduces and which important questions remain open.

04

A workload claim needs the whole work process

For an administrative use, the reported outcome may concern staff time. Leaders should ask what was counted and whose work changed. Drafting, checking, correction, approval and communication can sit with different people. A reduction in one stage can be real without producing the same reduction across the process.

The evaluation should not be expected to answer questions outside its scope, but the school needs to identify those omissions before estimating a local benefit.

The governing issue is especially clear where use of a tool changes professional responsibility. A member of staff may remain accountable for the final output even when a system produces the draft. If verification work rises, a headline saving can become less meaningful. This is an analytical mechanism to examine, not a finding that AI necessarily adds workload. The evidence should determine the conclusion rather than the school’s enthusiasm for or suspicion of the technology.

05

Transfer depends on the conditions the school can sustain

EEF’s implementation guidance emphasises the context and process through which an approach becomes part of daily work. Applied to a technology study, the implication is that leaders should understand the support, expertise and time surrounding the observed result. A school may be able to reproduce those conditions, adapt them or conclude that the proposed use is not currently feasible. Each can be a reasonable evidence-led judgement.4

The analysis should include differences in age range, organisation and operating model without assuming independent schools are either uniquely advantaged or outside the relevance of wider research. A small school, an all-through institution and a group can face different implementation constraints. Their decisions should retain those differences. A study’s setting is a material part of its meaning, not a detail to remove from a sales summary.

06

Evidence should remain connected to purpose and safeguards

EEF’s earlier digital guidance asks schools to establish the intended educational improvement before introducing technology. That principle also helps prevent an evaluation discussion from becoming an isolated technical exercise. The school should ask whether the outcome studied matters to its purpose and whether the proposed use preserves the responsibilities attached to the activity. A measured benefit does not remove separate safeguarding, privacy or contractual questions.5

Our October continuity analysis examines another part of this picture: what happens when systems or assumptions fail. Evaluation and continuity should inform each other. If the use becomes important to school work, leaders should understand the evidence for relying on it and the consequence if it becomes unavailable or changes materially. The school is making an organisational decision, not merely choosing which interface staff prefer.

07

The next report should make uncertainty smaller

The October commission is valuable because it makes method an explicit part of the developing evidence programme. School Connection will follow the completed guide and subsequent studies, preserving their populations, designs and limitations. We will distinguish findings from research plans and report material changes in the strength of evidence. A later result should develop the earlier discussion rather than simply repeat that technology has potential.

A relevant study would provide stronger support for adoption if it measured a meaningful outcome and explained implementation sufficiently for a school to assess whether the result could transfer. Selection, endorsement or a narrow demonstration would provide weaker support if the claim could not be traced further.

No product was evaluated for this editorial. The useful leadership habit is to ask how the claim was tested, then decide what that test permits the school to conclude.