Before expanding an education AI pilot, learn how teachers actually use and correct the material. General enthusiasm about AI is weaker evidence than a recent preparation task with visible inputs, edits, and classroom constraints.
Separate product usability, generated-content quality, technical reliability, and teaching outcomes. Feedback about one category should not be converted into an unmeasured claim about another.
Ask about the completed workflow
Invite teachers to describe the last material they prepared, source type, time constraints, edits required, and what they used in class. Ask which parts they discarded and why, without implying that positive feedback is the desired answer.
Keep participation and data sharing appropriate to the setting. Use non-identifying task examples where possible and avoid collecting student records merely to make feedback more detailed.
Review a fictional lesson-preparation pilot
Suppose a teacher imports a chapter, generates a lesson, rewrites several definitions, removes an impractical activity, and exports a quiz. The useful follow-up asks what made those changes necessary and whether the final pack solved the original preparation problem.
Distinguish extraction errors from model additions and editing friction. A teacher who likes the final lesson may still have spent substantial correction effort. Record that effort rather than treating completed export as automatic success.
Test feedback prompts with a small authorised group first. Include teachers who stopped using the tool or could not complete setup, not only those who reached a polished result.
Turn findings into bounded decisions
Group repeated problems while preserving important differences in subject, device, source quality, and classroom environment. A request for more features may be a workaround for an unclear existing step.
Choose whether to improve one workflow, narrow supported sources, repair installation, revise review controls, or investigate another audience. Write the next hypothesis and the evidence that would justify expansion.
The tradeoff is depth versus sample breadth. A few detailed task reviews can reveal concrete defects but cannot establish widespread demand or learning impact. A broader survey can show patterns while losing the context needed to interpret them. Use each method for the question it can support.
The ChalkLens portfolio account documents a built teaching workflow and explicitly leaves classroom adoption, learning outcomes, and time savings unmeasured. This pilot plan is proposed guidance, not a report of teacher research already conducted.
Questions about teacher feedback
Should the pilot ask whether AI improved learning?
Do not infer learning outcomes from a general opinion question. That claim needs a separate appropriate evaluation design and evidence.
Are feature requests useful?
Yes, as clues. Ask for the recent task and obstacle behind the request before making it a requirement.
What should an expansion review contain?
Include completed and abandoned tasks, correction reasons, operating limits, sample context, and next decisions. Bring the findings to an education pilot discussion.