Reproducibility
Why it matters
A computational claim is useful only when another learner can reconstruct the same model, inputs, method and evidence.
Learning objectives
- Create a complete experiment manifest with units, method and expected evidence.
- Distinguish repeatability, reproducibility and independent validation.
Prerequisites
Courses 6.1 and 7.1; prior canonical classes where applicable.
Concept and explanation
A reproducibility record binds a question to a model version, code, parameters, units, numerical method, seed when random, expected outputs, tests and limitations. Re-running the same code is repeatability; reproducing from the record tests whether the work is independently reconstructable.
Key terms
manifest; provenance; repeatability; reproducibility; seed; environment; version; audit trail
Domain-specific notation and representation
Record QUESTION → MODEL v1 → INPUTS/UNITS → METHOD → EXPECTED EVIDENCE → TESTS → LIMITS. Runtime and package versions remain unclaimed because PYTHON_RUNTIME is NOT_VERIFIED.
Worked computational experiments
Worked experiment 1
Question: Can a cooling calculation be reconstructed from a manifest?
Expectation / hypothesis: With fixed parameters and time points, the expected temperatures are identical.
Model: Newton cooling: T(t)=20+60 exp(-0.1t).
Inputs and parameters: t=[0,10,20] s; ambient 20 °C; initial 80 °C; k=0.1 s⁻¹.
Baseline: Model v1 with k=0.1 and unchanged units.
Experiment design: Compute only declared times and compare with hand evaluation at t=0.
Code / algorithm:
from math import exp
times=[0,10,20]
result=[20+60*exp(-0.1*t) for t in times]
print([round(x,2) for x in result])Expected output — not executed:
Expected/precomputed: [80.0, 42.07, 28.12]Results: Temperature decreases toward 20 °C; the manifest preserves every input.
Verification and validation: Verify t=0 gives 80 exactly; validate only against suitable cooling data for an intended use.
Sensitivity and uncertainty: Changing k changes cooling rate; measurement/model uncertainty is not removed.
Interpretation: The record supports reconstruction, not physical validation.
Limitations: Constant ambient temperature and lumped body assumption.
Reproducibility: model=v1; code hash pending; units SI; method=closed form; runtime NOT_VERIFIED.
Worked experiment 2
Question: Does a fixed seed make a stochastic estimate repeatable?
Expectation / hypothesis: The same seed and sample count produce the same expected sequence in one specified generator.
Model: Estimate a Bernoulli proportion from 8 declared draws.
Inputs and parameters: seed=17; n=8; threshold=0.5.
Baseline: Seed 17 and n=8 are fixed.
Experiment design: Record generator, seed, sample count and statistic before inspecting results.
Code / algorithm:
import random
r=random.Random(17)
x=[r.random()<0.5 for _ in range(8)]
print(x, sum(x)/8)Expected output — not executed:
Expected/precomputed for Python's documented generator: [False, False, False, True, False, False, False, True] 0.25Results: Two successes yield 0.25.
Verification and validation: Verify arithmetic 2/8; validation requires a justified probability model and adequate sample.
Sensitivity and uncertainty: A different seed changes sampling variability, not numerical error.
Interpretation: A seed aids repetition but does not make n=8 sufficient.
Limitations: Generator details and small sample constrain transfer.
Reproducibility: seed=17; n=8; statistic=mean; expected result labeled; runtime NOT_VERIFIED.
Common mistake / invalid experiment
Failure: reporting results without the parameter file, units, seed or model version prevents reconstruction and hides changes between runs.
Flawed evidence
run_model() # parameters and version not recordedCorrection and recheck
config={'model':'v1','k_per_s':0.1,'times_s':[0,10,20]}
assert config['times_s'][0] == 0Guided practice
Guided activity 1
Complete the question, expected evidence, inputs, parameters, controls, baseline and metric table.
Guided activity 2
Select verification evidence, independent validation evidence, sensitivity checks and uncertainty statements.
A/B/C/D practice
A — Foundation
Audit a manifest for missing units.
B — Application
Write a reproducibility record for a deterministic case.
C — Reasoning and error detection
Repair an unrecorded-seed experiment.
D — Challenge / transfer
Explain why reproducible is not automatically valid.
Validation and reproducibility
CODE → PREDICT → TRACE → EXPECTED OUTPUT → DEBUG → TEST → VALIDATE. Successful execution is not claimed and would not by itself validate the model.
Knowledge check and summary
- Identify question, output, inputs, parameters and controls.
- Compare baseline and controlled case with a declared metric.
- Separate verification from validation evidence.
- State sensitivity, uncertainty, limitations and reproducibility.
Mastery criterion: 4/4 checks plus two worked experiments, two guided activities, four A/B/C/D responses and one corrected failure with fresh reassessment.
Related laboratory
Next class: Parameter sweeps
Sources and references
- Academy-original bilingual instructional material.
- National Academies — Reproducibility and Replicability in Science.
- NIST/SEMATECH e-Handbook of Statistical Methods.
- Official Python, NumPy, SciPy and Matplotlib documentation where referenced.
Science & Mathematics Tutor
Context: Course 7.2 · Module 1 · Class 7.2.1. Ask for help with the active experiment, code trace, error diagnosis, validation or related lab. The Tutor follows the selected language and does not fabricate execution.
Academic review
VERIFIED · MOBILE QA = ENVIRONMENT_BLOCKED
Mathematical Foundations format parity
MATHEMATICAL_FOUNDATIONS_FORMAT_PARITY = PASS
The current Mathematical Foundations implementation is the UI/UX authority. Canonical identities govern academic structure, and approved academic content governs instruction.
- Standard
- ACADEMY_STANDARD_LESSON_EXPERIENCE_V1
- Class version
- v1
- Format review
- 2026-09-19
- Reference
- Mathematical Foundations
Version and review history
- 7.2.1 · v1 · 2026-09-19 — Canonical identity and approved academic content preserved.
- DEEP FORMAT REBUILD · 2026-09-20 — Module, class, laboratory, practice, assessment, Tutor and review presentation rebuilt to the Mathematical Foundations experience.
- MOBILE QA — ENVIRONMENT_BLOCKED