A model's published capability is not a guarantee that your application completes the customer's task correctly. The app also depends on extraction, prompt construction, resource management, output handling, and the user's review workflow.
Describe product promises at the level you can verify. A feature that drafts a quiz from selected text is different from a claim that it produces accurate curriculum-ready assessments without review.
Trace the layers behind the claim
Separate model documentation, runtime support, application implementation, task evaluation, and customer outcomes. Each needs its own evidence. A supported format does not establish extraction quality, and a generated answer does not establish learning improvement.
Use current official references such as the Gemma documentation and LiteRT-LM overview for model and runtime context. Do not transfer a published benchmark into an application claim without an equivalent test.
Review a fictional lesson-generator description
Suppose a product page says the app creates a draft explanation, activity, and quiz from a selected chapter. Acceptance evidence should show supported source input, preserved page context, complete output sections, editable drafts, and teacher review.
A stronger claim that every answer is grounded requires a reviewed question set with source spans and unsupported-answer cases. A claim of time savings requires an actual comparison that includes review and correction effort. Neither follows automatically from local inference.
Create a claim table with wording, intended audience, supporting artefact, limitations, and owner. Mark unmeasured claims explicitly rather than quietly converting aspiration into fact.
Keep feature guarantees narrow and useful
Define supported documents, languages, devices, offline prerequisites, and failure behaviour. Explain when the app asks for a smaller source selection or cannot answer from the material provided.
Human review should be a real product step where promised. Preserve original source and generated draft separately so users can check changes rather than accepting a polished result by default.
The tradeoff is persuasive breadth versus trustworthy specificity. Broad claims may attract interest but create expectations the implementation cannot support. A precise promise can still be compelling when it names a complete useful workflow and its evidence.
Revisit claims after model, runtime, extraction, or interface changes. A previously evaluated configuration does not permanently validate every later release.
The ChalkLens case study describes a built local teaching workflow and states that classroom adoption, learning outcomes, and time savings have not been measured there. That is the appropriate boundary between implementation and outcome evidence.
Questions about local AI claims
Can a model card prove an app feature works?
It provides relevant capability and limitation context. The application still needs task-specific acceptance and operating evidence.
Should every feature promise include a benchmark?
Not necessarily. Some promises need functional evidence, while performance or outcome claims need appropriate measurements and disclosed conditions.
What should a buyer review first?
Review one exact promise, its demonstration, acceptance cases, and stated limits. Bring unsupported wording to a product evidence review.