Quantum Computing Experiment Reproducibility Checklist for Developers
reproducibilityquantum experimentsSDK versionsdeveloper workflowquantum software

Quantum Computing Experiment Reproducibility Checklist for Developers

SSmart Qubit Editorial Team
2026-11-13
6 min read

A practical reproducibility checklist for quantum experiments, covering environments, seeds, circuits, backends, metadata, results, and baselines.

A reproducible quantum computing experiment is one that a teammate, reviewer, or future version of you can rerun and understand. This checklist covers the practical details that most often get lost: SDK versions, random seeds, circuit construction, transpilation, backend settings, shot counts, calibration context, outputs, and classical baselines.

Overview

Quantum experiments are unusually sensitive to their execution context. Two programs can contain the same logical circuit but produce different results because they use different SDK releases, compiler passes, random seeds, simulator options, hardware backends, calibration states, or measurement settings. Reproducibility does not always mean that every hardware result will be identical. It means that another person can reconstruct the workflow, identify the variables, and explain any remaining variation.

Start by separating your experiment into four layers:

  • Problem definition: the input data, objective, hypothesis, success metric, and classical comparison.
  • Circuit definition: the logical operations, qubit mapping assumptions, measurements, and circuit parameters.
  • Execution: the SDK, transpiler settings, simulator or hardware backend, shots, noise model, and runtime configuration.
  • Analysis: raw results, post-processing code, plots, statistical summaries, and conclusions.

Record each layer separately. A notebook that only contains the final histogram is difficult to audit, even when the code appears complete. For broader guidance on structuring a complete workflow, see our guide to hybrid quantum-classical workflows.

Checklist by scenario

Before starting a local simulator experiment

  • Create a project repository with a clear entry point, such as run_experiment.py, and keep exploratory notebooks alongside, rather than instead of, executable scripts.
  • Record the language version and exact package versions. Use the SDK's supported dependency or lockfile workflow where available, and commit the resulting environment file.
  • Write down the experiment question, expected output, circuit size, number of shots, and success metric in a README or configuration file.
  • Set random seeds for every component that uses randomness, including circuit generation, parameter initialisation, simulator sampling, data splitting, and classical optimisation. Record the seed values in the run metadata.
  • Export the logical circuits in a portable representation where practical, and save the human-readable circuit diagram as an additional inspection aid.
  • Store simulator configuration, including whether the run is statevector, sampling-based, noisy, or another supported mode. Do not assume that a simulator's default settings remain unchanged across releases.

Before running on quantum hardware

  • Record the provider, backend name, backend identifier if available, and the time of submission. A friendly display name alone may not be sufficient to identify the execution target later.
  • Save the requested shot count, measurement options, job or run identifier, and any queue or runtime settings that affect execution.
  • Capture the transpilation or compilation configuration: optimisation level, basis gates if explicitly set, coupling or connectivity assumptions, initial layout, routing strategy, and any seed used by stochastic compilation.
  • Save both the pre-transpilation logical circuit and the final circuit submitted to the backend. Include depth, gate counts, two-qubit gate counts, and the final qubit mapping when these are available.
  • Record the backend status and relevant calibration context at execution time. A full calibration snapshot may not always be accessible, but save the metadata supplied by the provider and note what was unavailable.
  • Run a control or reference circuit where appropriate. A simple circuit, identity-like circuit, or classically predictable case can help distinguish algorithmic behaviour from execution problems.

Hardware results should be treated as time-dependent observations, not as permanent properties of a circuit. The quantum hardware benchmarking workflow provides a useful companion process for comparing backend behaviour.

Before sharing results with a team

  • Commit the code, configuration, environment file, and circuit exports used for the reported run.
  • Store raw counts or expectation values separately from processed tables and figures. Never make the chart the only copy of the result.
  • Include a machine-readable metadata file for each run. JSON or YAML is suitable if the fields are documented and consistently named.
  • Provide a reproduction command and a short explanation of expected outputs, tolerances, and known sources of variation.
  • Include a classical baseline using the same input data and evaluation metric. A quantum result is difficult to interpret without a reference method.
  • Document failed runs, retries, discarded circuits, and any manual changes made after execution.

What to double-check

Use the following metadata fields as a minimum run record. Adapt the names to your tooling, but avoid leaving important values only in notebook cells.

{
  "experiment_id": "short-descriptive-name",
  "commit": "version-control-reference",
  "language_version": "recorded-version",
  "sdk_versions": {"primary_sdk": "recorded-version"},
  "seed": 12345,
  "circuit_source": "path-or-export-reference",
  "transpilation": {
    "optimisation_level": "recorded-setting",
    "layout": "recorded-setting",
    "transpiler_seed": 67890
  },
  "backend": {
    "type": "simulator-or-hardware",
    "name": "backend-reference",
    "shots": 1000
  },
  "execution_time": "timestamp-with-timezone",
  "job_id": "provider-reference",
  "calibration_reference": "snapshot-or-metadata-reference",
  "input_reference": "dataset-version-or-hash",
  "result_file": "path-to-raw-output",
  "baseline": "method-and-version"
}

Check that your seed actually controls the operation you think it controls. A seed passed to a circuit generator may not control a simulator sampler or a classical optimiser. Likewise, a fixed simulator seed cannot make hardware measurements deterministic. If a provider manages part of the workflow remotely, record the provider's job identifier and the settings exposed by its API rather than implying control over hidden components.

Check circuit identity at each stage. A logical circuit, a transpiled circuit, and a circuit reconstructed from an exported file may differ in gate ordering, qubit ordering, parameter precision, or measurement mapping. Compare diagrams, serialised circuits, qubit maps, and summary metrics before comparing results.

Finally, inspect the classical side of the experiment. Pin data preprocessing, feature ordering, train-test splits, optimiser settings, stopping criteria, and metric calculations. This is especially important for hybrid experiments and quantum machine learning; the PennyLane hybrid workflow guide offers related implementation context.

Common mistakes

  • Saving only the notebook: notebooks often contain hidden state, execution order problems, and unrecorded manual edits. Convert the final workflow into a clean script or documented command.
  • Recording a package range instead of an exact environment: a requirement such as a minimum version does not prove which dependency combination produced the result. Commit a lockfile or an equivalent resolved environment record.
  • Keeping only processed results: rounded probabilities, plotted averages, and filtered counts cannot replace raw provider output.
  • Assuming the backend name is enough: hardware status, qubit mapping, compiler configuration, and execution time can materially affect the run.
  • Changing transpilation defaults silently: optimisation settings can alter depth, gate counts, and noise exposure. Treat compiler configuration as part of the experimental method.
  • Confusing repeated shots with repeated experiments: shots sample one submitted circuit configuration, while separate jobs may encounter different compilation or hardware conditions. Label both clearly.
  • Ignoring failed and partial runs: retries can create selection bias if only successful jobs are retained. Keep a run log with status and reason for exclusion.
  • Comparing unlike baselines: use the same data, split, metric, and evaluation procedure when comparing a quantum circuit with a classical method.

When SDKs or providers change, consult the relevant migration notes and re-run a small validation experiment before trusting an older pipeline. Our SDK comparison and migration checklist can help when a project moves between Qiskit, Cirq, and PennyLane.

When to revisit

Revisit this checklist at the start of every new experiment, before sharing a result externally, and whenever a run moves from a simulator to hardware. It is also worth returning to after an SDK upgrade, provider API change, compiler change, backend change, operating-system rebuild, or alteration to the data pipeline.

For team planning, schedule a review at least twice a year even if the project appears stable. Verify that installation instructions still work, environment files can recreate the project, provider limits and authentication steps have not changed, and stored circuit exports can still be read. Avoid hard-coding assumptions about current defaults; state them explicitly in configuration and documentation.

A practical final gate is the clean-machine test:

  1. Clone the repository into a fresh environment.
  2. Install from the committed environment definition.
  3. Run the documented command with the recorded configuration.
  4. Confirm that the logical circuit, submitted circuit, raw result, and baseline are produced.
  5. Compare the output using a declared tolerance rather than requiring identical hardware counts.
  6. Record any difference and update the README, metadata schema, or code before the next experiment.

Use this process as a lightweight research contract. It makes quantum software experiments easier to review, debug, migrate, and extend, whether you are comparing simulators, testing a backend, or preparing a result for a wider engineering team. For related decisions about simulator choice and execution environments, see the quantum simulator comparison guide.

Related Topics

#reproducibility#quantum experiments#SDK versions#developer workflow#quantum software
S

Smart Qubit Editorial Team

Quantum Computing Editors

Senior editor and content strategist. Writing about technology, design, and the future of digital media. Follow along for deep dives into the industry's moving parts.