API handoff: source record

Inspect the original answers and check the verifier repair without rerunning an agent. This reviewed set contains 21 files (291,856 bytes).

Study recorded 8 September 2026File index as JSON →

On this page

Recheck without a model

Use a shell with curl, shasum and Python 3. Review the source files before executing their verifier code. No evx installation, account, model call or private repository is needed. The download commands create a new directory and check every file’s SHA-256.

Download the reviewed files and verify their hashes

set -eu
# Use a new directory. Existing directories are never overwritten by this recipe.
mkdir 'api-handoff-review'
cd 'api-handoff-review'
mkdir -p 'baseline'
mkdir -p 'candidate'
mkdir -p 'baseline/production'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/predeclared-plan.json' --output 'predeclared-plan.json'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/posthoc-plan.json' --output 'posthoc-plan.json'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/manifest.json' --output 'manifest.json'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/expected.json' --output 'expected.json'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/result.json' --output 'result.json'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/final-answers.json' --output 'final-answers.json'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/adjudication.json' --output 'adjudication.json'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/verify.py' --output 'verify.py'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/verify_v2.py' --output 'verify_v2.py'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/test_verifiers.py' --output 'test_verifiers.py'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/test_rescore.py' --output 'test_rescore.py'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/rescore.py' --output 'rescore.py'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/capture.json' --output 'capture.json'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/baseline--AGENTS.md' --output 'baseline/AGENTS.md'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/candidate--AGENTS.md' --output 'candidate/AGENTS.md'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/baseline--ANSWER_FORMAT.json' --output 'baseline/ANSWER_FORMAT.json'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/baseline--production--full.md' --output 'baseline/production/full.md'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/baseline--production--openapi.json' --output 'baseline/production/openapi.json'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/baseline--production--auth-study.json' --output 'baseline/production/auth-study.json'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/baseline--production--dispatch-study.json' --output 'baseline/production/dispatch-study.json'
curl --fail --silent --show-error --max-time 30 'https://evals.ax/research/artifacts/production-api-handoff-2026-09-08/baseline--production--http-observations.json' --output 'baseline/production/http-observations.json'
shasum -a 256 --check <<'EVX_REVIEWED_SHA256'
02007ff7ffb749456ea5223ccb17e7076cba21b5cf2cae9031ef8494dd817b50  predeclared-plan.json
bd3c4f51c67bb8b8e7c4384ec93e42fcc51cdca3ef23240354ada9b84548f53c  posthoc-plan.json
90e387c2f08e7bb3d8f8ad7f821497082fe9ebe5be2a51eb07bc5fbb75cbeb98  manifest.json
079463aea0501865527bf30c9603310b0b0e29c49f39e4407b1b74679af61cd8  expected.json
3176505ee2526d46d9425cb5b0b548a09313db9812c29938efcd7f32f67db7ba  result.json
eda49a8dbcefaf99a740cb5af40b28a1a8ce131edbacef9727a27a7a62e0c1ff  final-answers.json
3d9d94c649e889820e1d51d9baca275c958eca3903a99002894570cdea274484  adjudication.json
a528400d84d516370f0f999b9eb6f2f66a8793907ccc4603df92d32f673cbdfd  verify.py
ca0bacae64bd085f8d60cc5dc54a6834012547606c4483928360e33729068567  verify_v2.py
ed237f6e1b301a8cac5a358d90f1ac47108737181439a4c9875048fe01320f6f  test_verifiers.py
bcbef94414d4070cab84470e7e5c7c39324e8c5f3591fa113467930736db289b  test_rescore.py
6ccf30e0a5ee315cc5c58b67a917745db5f8b3132199c531dd1a205a5dd0af1d  rescore.py
b0e487299067e2f1400426661084d31e95f6ec4e1211b3aae549354779c1877a  capture.json
92a44a43adf6fe1e3286b660ec62aec512807420d9df5429bc0ef397b3e53539  baseline/AGENTS.md
89866816978bd316544680a29142ec801b57053edd7eacea4bb20e912ca31b3b  candidate/AGENTS.md
cff99bbb75cdc36d10c8cf0d9f5f06ce5da08db3839d61dd23d2d1862343ba10  baseline/ANSWER_FORMAT.json
2381575b7512e2b8f37ee488ef2be7aeedeafaf9fe4a0c27bc61127443a7f6e7  baseline/production/full.md
6f0bb187bc2bb8e3e0bedbd74bfbe487b62edb3212ca456f08158b084d7f288f  baseline/production/openapi.json
cb8e1cb5db90f9cad1c4d62b568389b5b619e65bc93b260e1616ae94371fa0ae  baseline/production/auth-study.json
2510d98e57db4b9d833e5adac79bb716757d5ea4ce9b5f6fb0bd1438bbf95d67  baseline/production/dispatch-study.json
b669b3c3e5078b73ceffa6f5e0e68d35227d8a99e71c9e9562e19ab3687bd36e  baseline/production/http-observations.json
EVX_REVIEWED_SHA256

In that same shell, after the download and hash checks succeed, run:

python3 test_verifiers.py
python3 test_rescore.py
python3 rescore.py > reassessment.json

The tests must pass. The separate reassessment.json reports all eight answers: original passes remain 0/4 in each condition; post-hoc accepted answers are 2/4 baseline and 4/4 candidate. The original result.json stays unchanged. No provider traces are required for this check.

If a download or hash check fails, stop before running the verifier. Keep the partial directory for inspection and retry from a new directory. These commands do not recreate the original agent execution or prove the verifier covers every valid answer.

Recorded protocol

The declared task, controls and two entry points. The capture records which public responses were available on 8 September 2026.

predeclared-plan.json4,669 bytes
SHA-256
02007ff7ffb749456ea5223ccb17e7076cba21b5cf2cae9031ef8494dd817b50
manifest.json19,822 bytes
SHA-256
90e387c2f08e7bb3d8f8ad7f821497082fe9ebe5be2a51eb07bc5fbb75cbeb98
capture.json1,441 bytes
SHA-256
b0e487299067e2f1400426661084d31e95f6ec4e1211b3aae549354779c1877a
baseline/AGENTS.md659 bytes
SHA-256
92a44a43adf6fe1e3286b660ec62aec512807420d9df5429bc0ef397b3e53539
candidate/AGENTS.md1,410 bytes
SHA-256
89866816978bd316544680a29142ec801b57053edd7eacea4bb20e912ca31b3b
baseline/ANSWER_FORMAT.json2,257 bytes
SHA-256
cff99bbb75cdc36d10c8cf0d9f5f06ce5da08db3839d61dd23d2d1862343ba10

Original answers and verifier v1

All eight final answers, their output hashes and the original result. Version 1 had a known false-negative defect; its recorded 0/4 versus 0/4 verdicts remain unchanged.

expected.json3,279 bytes
SHA-256
079463aea0501865527bf30c9603310b0b0e29c49f39e4407b1b74679af61cd8
result.json34,861 bytes
SHA-256
3176505ee2526d46d9425cb5b0b548a09313db9812c29938efcd7f32f67db7ba
final-answers.json34,885 bytes
SHA-256
eda49a8dbcefaf99a740cb5af40b28a1a8ce131edbacef9727a27a7a62e0c1ff
verify.py5,805 bytes
SHA-256
a528400d84d516370f0f999b9eb6f2f66a8793907ccc4603df92d32f673cbdfd

Separate post-hoc reassessment

Version 2, its controls and the recorded repair plan. It accepted 2/4 baseline and 4/4 candidate answers. This does not establish a navigation benefit.

posthoc-plan.json1,500 bytes
SHA-256
bd3c4f51c67bb8b8e7c4384ec93e42fcc51cdca3ef23240354ada9b84548f53c
adjudication.json15,846 bytes
SHA-256
3d9d94c649e889820e1d51d9baca275c958eca3903a99002894570cdea274484
verify_v2.py2,508 bytes
SHA-256
ca0bacae64bd085f8d60cc5dc54a6834012547606c4483928360e33729068567
test_verifiers.py2,014 bytes
SHA-256
ed237f6e1b301a8cac5a358d90f1ac47108737181439a4c9875048fe01320f6f
test_rescore.py2,913 bytes
SHA-256
bcbef94414d4070cab84470e7e5c7c39324e8c5f3591fa113467930736db289b
rescore.py4,649 bytes
SHA-256
6ccf30e0a5ee315cc5c58b67a917745db5f8b3132199c531dd1a205a5dd0af1d

Frozen public inputs

Historical response files used by both conditions. These API and CLI examples describe the captured version; use current documentation for account operations today.

baseline/production/full.md44,281 bytes
SHA-256
2381575b7512e2b8f37ee488ef2be7aeedeafaf9fe4a0c27bc61127443a7f6e7
baseline/production/openapi.json100,823 bytes
SHA-256
6f0bb187bc2bb8e3e0bedbd74bfbe487b62edb3212ca456f08158b084d7f288f
baseline/production/auth-study.json4,287 bytes
SHA-256
cb8e1cb5db90f9cad1c4d62b568389b5b619e65bc93b260e1616ae94371fa0ae
baseline/production/dispatch-study.json3,238 bytes
SHA-256
2510d98e57db4b9d833e5adac79bb716757d5ea4ce9b5f6fb0bd1438bbf95d67
baseline/production/http-observations.json709 bytes
SHA-256
b669b3c3e5078b73ceffa6f5e0e68d35227d8a99e71c9e9562e19ab3687bd36e

Provenance and limits

Publication is limited to these exact reviewed bytes. The source revision is8f750ca02b2c17f51da29e343057ae259c1e754e; original pre-execution inputs were committed at8d86dda1be42db57df4c8b82cecd1749ad04bc42. The JSON index records each original relative path, role, byte length and digest.

Only the two entry-point files differ between conditions. The candidate’s other response files were identical to the baseline copies supplied here. This download is sufficient for reassessment, and is not a complete new agent-run distribution.

Raw provider streams, host traces and private authentication source are outside this public set. Final answers contain facts and quotations from the frozen public inputs. Hashes identify the recorded bytes; they do not independently attest execution, complete model context or research validity. Dollar cost remains unknown.

Current experiment instructions →