Coordinate paper reproduction and follow-up experiments

Why use this

1. A research team reproducing a paper can lose track of whether a failure came from data, implementation, environment or the original claim. Proposed fixes may then be tested without preserving a clean baseline for comparison.

2. This job could coordinate claims, attempts and follow-up questions in a sourced research brief. Keeping failed attempts and deviations visible helps the researcher decide which experiment is justified next without presenting partial replication as independent confirmation.

Why an agent / simpler alternative

Useful judgment

Separate claim verification from reproduction, identify dependencies and compare proposed fixes against the intended baseline.

Simpler alternative

Use a single experiment notebook when one researcher can manage the whole reproduction scope.

Unsuitable when

The workflow would exceed compute limits, alter the baseline silently or publish claims before qualified review.

Proposed inputs

1. Name

   Paper/claim brief

   Owner

   Research lead

   Minimum scope

   Authorized paper sections and specific reproduction targets

   Quality check

   Distinguish central claims from secondary observations

2. Name

   Experiment manifest

   Owner

   Research team

   Minimum scope

   Data provenance, environment, parameters and planned artifacts

   Quality check

   Track deviations and preserve failed attempts

Access and setup — proposed or unknown

Proposed minimum

1. Use a synthetic paper claim and experiment ledger; no downloads, experiments, merges or compute allocation.

Compatibility unknowns

1. Current data/code availability, environment reproducibility, tool access and runtime compatibility are unknown.

External actions requiring approval

1. Data use, experimentation, resource spend, sharing, publishing, merging and deletion require explicit authorized scope.

Expected deliverable and review checks

Proposed artifact

Research-plan candidate with claim map, stage ownership, failure categories and next-experiment rationale.

Critical checks

1. Link every conclusion to evidence; separate proposed fixes from observed results; preserve baseline and resource constraints.

Missing output evidence

This overlay adds no new output inspection, execution or validation; consult the referenced canonical source outcome limits.

Failure signals and proposed responses

1. Signal

   A fix changes more than one experimental factor

   Why it matters

   Any improvement becomes difficult to interpret

   Proposed response

   Separate changes or explicitly record the confound.

2. Signal

   A failed run is omitted from the brief

   Why it matters

   The evidence becomes selectively optimistic

   Proposed response

   Retain its status, reason and available artifacts.

Effort and cost — unknown

Setup work

1. Define reproduction target, baseline and evidence handoffs.

Ongoing review

1. Research lead reviews failures and approves justified follow-up experiments.

Unknowns

1. Data availability, reproducibility barriers, compute duration and costs are unknown.

First check — proposed, unexecuted

Status

proposed_unexecuted

Synthetic fixture

Invent a claim, baseline manifest and three proposed reruns with one hidden parameter change and one missing failure record.

Acceptance checks

1. Flag both gaps; distinguish plans from results; run no experiment and assert no replication.

Stop conditions

1. Stop if data rights, baseline identity or resource authorization is unresolved.

Limits

1. A research coordination brief cannot establish reproducibility, causal improvement or validity of a scientific claim.