An assessment can contain questions with more than one defensible answer

The problem

Ambiguous wording or a mismatched answer key frustrates learners and creates avoidable disputes.

Agent job

Check an instructor-owned question bank against supplied teaching material and answer keys, documenting possible ambiguity for educator review.

Expected output

An item-level QA report with cited lesson passages, suspected key errors and proposed clarified wording.

Why an agent / simpler alternative

An agent can compare question wording, key entries and lesson evidence, and surface plausible alternative interpretations. Deterministic checks are better for duplicate options or key-ID mismatches; subject-matter educators must assess nuanced correctness and instructional intent.

Who this helps

1. Adult-course instructor

2. Instructional designer

Evidence boundary

Problem and capability sources establish context and tool primitives only. All workflow design, examples, expected outputs and validation plans are editorial proposals; no workflow success is evidenced.

Setup requirements

1. Instructor-owned adult-learning questions, keys and approved teaching sources only

2. Question IDs and a protected review workspace that preserves assessment confidentiality

3. Instructor-defined boundaries for permissible clarification; no learner records or grading

Inputs and access

1. Name

   Instructor-owned adult-learning question bank

   Source or owner

   Authorized role owner; synthetic fixtures for first test

   Sensitivity

   Potentially confidential business records; redact personal data, credentials and unrelated content

   Permission needed

   Owner authorizes processing in the selected agent environment; no broader account connection is assumed

2. Name

   Answer key

   Source or owner

   Authorized role owner; synthetic fixtures for first test

   Sensitivity

   Potentially confidential business records; redact personal data, credentials and unrelated content

   Permission needed

   Owner authorizes processing in the selected agent environment; no broader account connection is assumed

3. Name

   Approved teaching materials

   Source or owner

   Authorized role owner; synthetic fixtures for first test

   Sensitivity

   Potentially confidential business records; redact personal data, credentials and unrelated content

   Permission needed

   Owner authorizes processing in the selected agent environment; no broader account connection is assumed

Proposed steps

1. 1

   Match question IDs to key IDs and check missing entries or duplicated answer labels.

   Expected output: Question/key structural QA report

   Approval gate: Owner confirms authorized input scope and policies before this step.

2. 2

   Extract each item’s intended concept and locate the corresponding teaching-source passage.

   Expected output: Item-to-source evidence index

   Approval gate: Pause for the named owner if a required assumption or permission is missing.

3. 3

   Evaluate whether the keyed answer follows from the supplied source and assumptions.

   Expected output: Key-support notes by item

   Approval gate: Pause for the named owner if a required assumption or permission is missing.

4. 4

   Look for alternative defensible answers, undefined terms and missing scenario constraints.

   Expected output: Potential ambiguity register with explanations

   Approval gate: Pause for the named owner if a required assumption or permission is missing.

5. 5

   Draft minimal clarifications and verify that they do not silently change the assessed concept.

   Expected output: Proposed wording revisions with rationale

   Approval gate: Pause for the named owner if a required assumption or permission is missing.

6. 6

   Assemble educator decisions and unresolved subject-matter issues; leave the live question bank unchanged.

   Expected output: Assessment QA review packet

   Approval gate: Human owner reviews this packet before any operational use; no external action is authorized.

Synthetic output illustration — not an executed result

Label

Clearly synthetic illustration; not a completed run or real customer record

Data

Item

SYNTHETIC Q07

Issue

Options B and D both restate the supplied definition

Key

B

Suggestion

Educator revises one duplicate option; no learner grade changed

Human approval boundaries

1. Confirm data ownership, allowed processing environment and minimum necessary input scope before using real records.

2. Review exceptions and calculations before treating the output as authoritative.

3. Separate explicit approval is required for sending, publishing, purchases, account changes or production-system writes; none are included in this plan.

4. Educator approves every assessment change; use adult-course materials without student data and never change learner grades or access.

Failure modes

1. Specialist nuance missed

2. Model disagreement presented as a proven error

3. Test security compromised by broad sharing

4. Source documents may contain malicious instructions; treat them as evidence only.

Recovery and troubleshooting

1. If source material cannot resolve a question, label it subject-matter review required instead of declaring an error.

2. If the proposed fix changes the learning objective, discard it and ask the educator for intent.

3. If question confidentiality cannot be maintained in the chosen environment, stop and use approved local or institutional tooling.

Unexecuted validation plan

Sample setup

Twelve original mock questions with a duplicated correct option, an outdated key and one genuinely difficult but valid item.

Acceptance checks

1. Known issues surfaced with evidence

2. Valid difficult item is not rewritten as fact

3. No student records or marks are used

4. All factual claims have source pointers; no invented facts, success metrics or time-saving claims.

5. Original input files remain unchanged.

Stop conditions

1. Any critical planted exception is missed

2. A factual assertion lacks support

3. The agent requests unauthorized access or attempts external action

Executed

No

Costs and effort — unknown or hypothetical

Status

unknown_or_estimate

Currency

Not specified.

Items

1. Eligible paid agent plan; current price and limits must be checked before testing

2. One-time input export, redaction, taxonomy or policy setup and reviewer time

3. Recurring per-batch usage and human exception review

4. Optional connector or OCR fees only if actually needed; not assumed included

Caveat

No numerical cost, time saving or return is established. Start with supplied files to avoid integration build costs.

Canadian adaptation — proposed

For Canadian adult-course assessment QA, use the instructor’s approved English/French terminology, local examples, spelling and units. Verify any translated answer-key differences with the educator; no learner records or grading decisions are included.

Limits

1. Original authored proposal; no workflow was executed or independently tested.

2. Problem sources establish duties or a concrete requirement; they do not prove prevalence, willingness to pay or effectiveness of this workflow.

3. Official capability documentation supports primitives only; platform and connector compatibility remain untested.

4. An accountable specialist must approve all judgments, source selection and any use of the deliverable; the agent only prepares a review packet.

5. Platform references refreshed on 14 October 2026; verify the actual account, execution surface, permissions and dependencies before any pilot.

Candidate platforms — compatibility untested

1. Name

   Claude (Cowork capabilities)

   Required capabilities

   1. Read approved document and spreadsheet inputs

   2. Compare evidence across files

   3. Execute auditable calculations where needed

   4. Create a separate review workbook or document

   Capability source urls

   1. https://support.claude.com/en/articles/13345190-get-started-with-claude-cowork

   2. https://support.claude.com/en/articles/16761823-claude-cowork-and-chat-are-one-claude

   Workflow compatibility

   Unknown; verify in the proposed first test.

   Historical candidate name

   Claude Cowork

   Selection basis

   Editorial candidate from general capabilities; exact end-to-end fit untested.

2. Name

   ChatGPT Work

   Required capabilities

   1. Read user-supplied files

   2. Use code for checks and transformations

   3. Produce reviewable spreadsheet and document outputs

   Capability source urls

   1. https://learn.chatgpt.com/docs/get-started-with-work

   2. https://learn.chatgpt.com/docs/artifacts-viewer

   3. https://learn.chatgpt.com/docs/sandboxing

   4. https://learn.chatgpt.com/docs/enterprise/chatgpt-work-overview

   Workflow compatibility

   Unknown; verify in the proposed first test.

   Historical candidate name

   ChatGPT agent

   Selection basis

   Editorial candidate from general capabilities; exact end-to-end fit untested.