dennisthiessen/signal-platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: isolate production universe in capacity research

Browse Source
This commit is contained in:
dennisthiessen
2026-08-05 21:00:19 +02:00
parent 23fe39fd78
commit 6fc82ae857
6 changed files with 349 additions and 97613 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/research/portfolio-capacity-bracket.md
+19 -1
View File
@@ -22,6 +22,11 @@ Because the current ~505-name production membership is projected backward,
paired arm-versus-control differences are the primary evidence. Absolute
profitability is descriptive and survivorship-biased.
Implementation correction: the first completed v1 artifact at commit `23fe39f`
incorrectly allowed the snapshot's broad rank-only universe to submit trades.
That artifact is invalid, is removed from the branch, and must not be used for
strategy conclusions. Runner v2 fixes the construction/ranking partition below.
## Frozen arms
1. **cap10_incumbent:** exact production-style cap-10 control, no displacement.
@@ -37,6 +42,12 @@ live-like full-universe residual-momentum/low-volatility 80/20 rank, activation
threshold 80, normal gate-reset re-entry, close fill, 3×ATR trail, 30-session
maximum hold, 1% risk, and costs of 0.10% and 0.20% per fill.
Every priced symbol contributes to the daily cross-sectional rank. Only symbols
not listed in the snapshot's `research_rank_only` side table may submit trade
setups to any arm. The resulting construction universe must contain 450-600
symbols (expected approximately 505); validation fails outside that frozen
guardrail or when the side table references unknown ticker symbols.
The daily replay uses zero outcome horizon: setup and rank observations continue
through the snapshot's last session because portfolio simulation, unlike outcome
grading, does not require 30 future bars.
@@ -83,7 +94,8 @@ positions liquidate at the last measurement close with costs.
The validate-only mode must print realized cohort counts and fail unless both
protocols contain the seven annual clusters 2019–2025 and every warm anchor has
at least 12 seeds.
at least 12 seeds. It must also print ranking, rank-only, and tradable symbol
counts plus the raw, removed, and retained qualified-long counts.
## Reporting
@@ -120,6 +132,12 @@ atomically and resume verifies a fingerprint over the implementation commit,
this specification hash, snapshot SHA-256, cache key, arm definitions, costs,
and cohort manifest. An authoritative run refuses a dirty worktree.
The existing v1 candidate/rank cache is intentionally reusable: its
full-universe current-day ranks are correct. Runner v2 derives a fingerprinted
construction view by removing qualified rows whose symbols are rank-only. V2
uses a versioned checkpoint directory, so invalid v1 portfolio cells are never
resumed and the expensive daily rank replay does not need to run again.
The loader reads only ticker ID/symbol and the OHLCV columns used by replay, so
snapshots created before SEC metadata added `tickers.cik`, `tickers.sic`, and
`tickers.sic_description` remain valid. Do not migrate or alter the research
reports/portfolio-construction-prod505-capacity-bracket-daily-v1.json
-97519
View File
File diff suppressed because it is too large Load Diff
reports/portfolio-construction-prod505-capacity-bracket-daily-v1.md
-74
View File
@@ -1,74 +0,0 @@
# Focused daily portfolio-capacity matrix
Generated: 2026-08-05T18:12:50.577686+00:00
## Question
The current daily Phase A control admitted 472 trades and rejected 519 qualified opportunities because the ten-slot book was full. This run brackets the economic cost of that binding constraint; it has no formal promotion gate.
> Universe caveat: today's production membership is projected backward. Use paired arm-versus-control differences, not absolute profitability, for construction conclusions.
## Paired annual medians
### Empty Book — 0.10% per fill
| Arm | ΔEV net R | 90% context | ΔCalmar | 90% context |
|---|---:|---:|---:|---:|
| cap10_incumbent | 0.000 | [0.000, 0.000] | 0.000 | [0.000, 0.000] |
| cash_unbounded | 0.137 | [0.120, 0.185] | 0.000 | [-0.010, 0.270] |
| cap10_weekly_top10 | 0.092 | [0.040, 0.117] | 0.000 | [-0.040, 0.000] |
| cap15_incumbent | 0.098 | [0.022, 0.183] | 0.000 | [-0.010, 0.160] |
| Arm | ΔPF | ΔGain-to-Pain | ΔSortino | ΔCAGR pp | ΔMaxDD pp |
|---|---:|---:|---:|---:|---:|
| cap10_incumbent | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
| cash_unbounded | 0.094 | 0.068 | 0.313 | 0.800 | 3.800 |
| cap10_weekly_top10 | -0.035 | -0.052 | -0.196 | -5.700 | 2.850 |
| cap15_incumbent | 0.037 | 0.017 | 0.168 | -0.050 | 2.150 |
### Warm Book — 0.10% per fill
| Arm | ΔEV net R | 90% context | ΔCalmar | 90% context |
|---|---:|---:|---:|---:|
| cap10_incumbent | 0.000 | [0.000, 0.000] | 0.000 | [0.000, 0.000] |
| cash_unbounded | 0.119 | [-0.014, 0.214] | 0.000 | [-0.045, 0.150] |
| cap10_weekly_top10 | 0.068 | [0.063, 0.154] | 0.000 | [-0.110, 0.020] |
| cap15_incumbent | 0.063 | [-0.104, 0.145] | 0.000 | [-0.020, 0.110] |
| Arm | ΔPF | ΔGain-to-Pain | ΔSortino | ΔCAGR pp | ΔMaxDD pp |
|---|---:|---:|---:|---:|---:|
| cap10_incumbent | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
| cash_unbounded | 0.069 | 0.018 | -0.921 | -4.500 | 0.000 |
| cap10_weekly_top10 | 0.000 | -0.026 | -0.869 | -4.100 | 3.900 |
| cap15_incumbent | 0.035 | 0.006 | -0.994 | -0.900 | 1.050 |
## Warm-seed initialization dispersion
| Arm | Cost/fill | Median EV IQR ratio | Median Calmar IQR ratio |
|---|---:|---:|---:|
| cap10_incumbent | 0.10% | n/a | n/a |
| cash_unbounded | 0.10% | n/a | n/a |
| cap10_weekly_top10 | 0.10% | n/a | n/a |
| cap15_incumbent | 0.10% | n/a | n/a |
| cap10_incumbent | 0.20% | n/a | n/a |
| cash_unbounded | 0.20% | n/a | n/a |
| cap10_weekly_top10 | 0.20% | n/a | n/a |
| cap15_incumbent | 0.20% | n/a | n/a |
## Capacity and operations — 0.10% per fill
| Arm | Median trades | Median blocked | Median positions | Peak | Turnover | Min-risk rejects |
|---|---:|---:|---:|---:|---:|---:|
| cap10_incumbent | 212.0 | 98.4% | 10.00 | 10 | 14.72 | 0 |
| cash_unbounded | 340.0 | 0.0% | 18.24 | 41 | 19.04 | 2719474 |
| cap10_weekly_top10 | 303.0 | 97.1% | 9.99 | 10 | 18.07 | 0 |
| cap15_incumbent | 327.0 | 97.3% | 14.78 | 15 | 17.30 | 0 |
## Weekly-ranking opportunity set
- Median fresh entrant pool: 47.0.
- Median zero-entrant fraction: 0.000.
- Replacements across reported paths: 15081.
- Same-symbol re-entries within 10 sessions: 4081.
Bootstrap intervals above resample seven annual summaries and are descriptive context only. They are not gates or independent-population confidence claims.
scripts/portfolio_capacity_research.py
+9 -2
View File
@@ -225,8 +225,15 @@ def percentile(values: Iterable[float], probability: float) -> float | None:
def iqr(values: Iterable[float]) -> float | None:
q25 = percentile(values, 0.25)
q75 = percentile(values, 0.75)
clean: list[float] = []
for value in values:
if value is None:
continue
parsed = float(value)
if math.isfinite(parsed):
clean.append(parsed)
q25 = percentile(clean, 0.25)
q75 = percentile(clean, 0.75)
if q25 is None or q75 is None:
return None
return q75 - q25
scripts/run_portfolio_construction_matrix.py
+165 -14
View File
@@ -48,7 +48,10 @@ from scripts.research_rankings import _live_universe_rank_map # noqa: E402
CACHE_VERSION = 'portfolio-capacity-candidates-v1-zero-horizon'
RUNNER_VERSION = 'portfolio-capacity-bracket-v1'
RUNNER_VERSION = 'portfolio-capacity-bracket-v2'
CONSTRUCTION_VIEW_VERSION = 'production-book-filter-v1'
MIN_PRODUCTION_UNIVERSE = 450
MAX_PRODUCTION_UNIVERSE = 600
SPEC_PATH = ROOT / 'docs' / 'research' / 'portfolio-capacity-bracket.md'
DEFAULT_RUN_ID = 'prod505-capacity-bracket-daily-v1'
_WORKER_CONTEXT: dict[str, Any] | None = None
@@ -103,6 +106,67 @@ def _json_hash(value: Any) -> str:
return hashlib.sha256(payload).hexdigest()
def _construction_universe_errors(manifest: dict[str, Any]) -> list[str]:
errors: list[str] = []
construction_rows = int(manifest['construction_ticker_rows'])
construction_priced = int(manifest['construction_symbols_with_prices'])
ranking_rows = int(manifest['ranking_ticker_rows'])
rank_only_rows = int(manifest['rank_only_ticker_rows'])
unknown_rank_only = int(manifest['rank_only_unknown_symbols'])
if not MIN_PRODUCTION_UNIVERSE <= construction_rows <= MAX_PRODUCTION_UNIVERSE:
errors.append(
'construction ticker universe must contain '
f'{MIN_PRODUCTION_UNIVERSE}-{MAX_PRODUCTION_UNIVERSE} symbols; '
f'found {construction_rows}'
)
if not MIN_PRODUCTION_UNIVERSE <= construction_priced <= MAX_PRODUCTION_UNIVERSE:
errors.append(
'priced construction universe must contain '
f'{MIN_PRODUCTION_UNIVERSE}-{MAX_PRODUCTION_UNIVERSE} symbols; '
f'found {construction_priced}'
)
if construction_rows + rank_only_rows != ranking_rows:
errors.append(
'construction and rank-only ticker partitions do not cover the '
'ranking universe'
)
if unknown_rank_only:
errors.append(
f'research_rank_only contains {unknown_rank_only} symbols absent '
'from tickers'
)
return errors
def _construction_candidate_view(
cached: dict[str, Any],
snapshot_data: dict[str, Any],
) -> dict[str, Any]:
'''Filter a reusable full-universe rank cache to production setup symbols.'''
allowed = set(snapshot_data['construction_symbols'])
raw_candidates = list(cached['qualified_candidates'])
qualified = [
row for row in raw_candidates if str(row['symbol']) in allowed
]
view_key = {
'version': CONSTRUCTION_VIEW_VERSION,
'base_cache_key_hash': _json_hash(cached['key']),
'construction_universe_manifest': (
snapshot_data['construction_universe_manifest']
),
}
return {
**cached,
'qualified_candidates': qualified,
'raw_full_universe_qualified_long_count': len(raw_candidates),
'qualified_long_count': len(qualified),
'filtered_rank_only_qualified_long_count': (
len(raw_candidates) - len(qualified)
),
'construction_view_key': view_key,
}
def _atomic_json(path: Path, value: Any) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
temporary = path.with_suffix(path.suffix + '.tmp')
@@ -292,12 +356,34 @@ async def _load_snapshot(
f'loaded prices: {index}/{len(symbols)}',
flush=True,
)
rank_only_symbols = await bt._load_research_rank_only_symbols(db)
finally:
await engine.dispose()
if not prices or not benchmark_closes:
raise SystemExit('Snapshot has no usable prices or benchmark history')
ticker_symbols = set(symbols)
price_symbols = set(prices)
known_rank_only = ticker_symbols & rank_only_symbols
unknown_rank_only = rank_only_symbols - ticker_symbols
construction_symbols = ticker_symbols - known_rank_only
priced_construction_symbols = price_symbols & construction_symbols
priced_rank_only_symbols = price_symbols & known_rank_only
construction_universe_manifest = {
'ranking_ticker_rows': len(ticker_symbols),
'ranking_symbols_with_prices': len(price_symbols),
'ranking_symbols_sha256': _json_hash(sorted(ticker_symbols)),
'construction_ticker_rows': len(construction_symbols),
'construction_symbols_with_prices': len(priced_construction_symbols),
'construction_symbols_sha256': _json_hash(sorted(construction_symbols)),
'rank_only_ticker_rows': len(known_rank_only),
'rank_only_symbols_with_prices': len(priced_rank_only_symbols),
'rank_only_symbols_sha256': _json_hash(sorted(known_rank_only)),
'rank_only_unknown_symbols': len(unknown_rank_only),
'rank_only_unknown_symbols_sha256': _json_hash(sorted(unknown_rank_only)),
}
strategy = next(
row
for row in bt.PORTFOLIO_MONITOR_STRATEGIES
@@ -350,11 +436,13 @@ async def _load_snapshot(
'benchmark_closes': benchmark_closes,
'prices': prices,
'symbols': symbols,
'construction_symbols': construction_symbols,
'universe_manifest': {
'ticker_rows': len(symbols),
'symbols_with_prices': len(prices),
'symbols_sha256': _json_hash(sorted(symbols)),
},
'construction_universe_manifest': construction_universe_manifest,
'ranking_key': ranking_key,
'exit_policy': exit_policy,
'hold_days': hold_days,
@@ -639,21 +727,51 @@ def _markdown(report: dict[str, Any]) -> str:
'## Paired annual medians',
'',
]
validation = report.get('validation') or {}
construction = validation.get('construction_universe_manifest') or {}
coverage = validation.get('candidate_rank_coverage') or {}
if construction:
construction_count = construction['construction_symbols_with_prices']
rank_only_count = construction['rank_only_symbols_with_prices']
ranking_count = construction['ranking_symbols_with_prices']
qualified_count = coverage.get('construction_qualified_longs')
filtered_count = coverage.get('filtered_rank_only_qualified_longs')
insertion = lines.index('## Paired annual medians')
lines[insertion:insertion] = [
'## Validated universes',
'',
f'- Tradable setup symbols with prices: '
f'{construction_count}.',
f'- Rank-only symbols with prices: '
f'{rank_only_count}.',
f'- Full ranking symbols with prices: '
f'{ranking_count}.',
f'- Tradable qualified longs: '
f'{qualified_count}.',
f'- Rank-only qualified rows removed: '
f'{filtered_count}.',
'',
]
paired = report['analysis']['paired_per_year']
for protocol in ('empty_book', 'warm_book'):
for cost, protocol in (
(cost, protocol)
for cost in COSTS_PER_SIDE_PCT
for protocol in ('empty_book', 'warm_book')
):
lines.extend([
f'### {protocol.replace("_", " ").title()} — 0.10% per fill',
'',
'| Arm | ΔEV net R | 90% context | ΔCalmar | 90% context |',
'|---|---:|---:|---:|---:|',
])
lines[-4] = lines[-4].replace('0.10%', f'{cost:.2f}%')
for arm in ARMS:
row = next(
item
for item in paired
if item['arm_id'] == arm['id']
and item['protocol'] == protocol
and float(item['cost_per_side_pct']) == 0.1
and float(item['cost_per_side_pct']) == float(cost)
)
ev = row['headline']['ev_net_r']
calmar = row['headline']['calmar']
@@ -676,7 +794,7 @@ def _markdown(report: dict[str, Any]) -> str:
for item in paired
if item['arm_id'] == arm['id']
and item['protocol'] == protocol
and float(item['cost_per_side_pct']) == 0.1
and float(item['cost_per_side_pct']) == float(cost)
)
headline = row['headline']
lines.append(
@@ -767,7 +885,7 @@ async def _main() -> None:
if args.candidate_cache
else ROOT / 'reports' / '.cache' / f'{args.run_id}-candidates.pkl'
)
candidate_cache = _build_candidate_cache(
base_candidate_cache = _build_candidate_cache(
snapshot_data,
snapshot=snapshot,
snapshot_sha256=snapshot_sha256,
@@ -775,11 +893,19 @@ async def _main() -> None:
workers=workers,
quiet=bool(args.quiet),
)
candidate_cache = _construction_candidate_view(
base_candidate_cache,
snapshot_data,
)
cohort_manifest = build_cohort_manifest(
snapshot_data['benchmark_closes'].keys()
)
cohort_errors = validate_cohort_manifest(cohort_manifest)
universe_errors = _construction_universe_errors(
snapshot_data['construction_universe_manifest']
)
validation_errors = [*universe_errors, *cohort_errors]
cells = build_cells(cohort_manifest)
validation_payload = {
'runner_version': RUNNER_VERSION,
@@ -794,9 +920,20 @@ async def _main() -> None:
'first': candidate_cache['rank_first_date'],
'last': candidate_cache['rank_last_date'],
'observations': candidate_cache['rank_observation_count'],
'qualified_longs': candidate_cache['qualified_long_count'],
'raw_full_universe_qualified_longs': candidate_cache[
'raw_full_universe_qualified_long_count'
],
'filtered_rank_only_qualified_longs': candidate_cache[
'filtered_rank_only_qualified_long_count'
],
'construction_qualified_longs': candidate_cache[
'qualified_long_count'
],
},
'universe_manifest': snapshot_data['universe_manifest'],
'construction_universe_manifest': (
snapshot_data['construction_universe_manifest']
),
'empty_cluster_counts': cohort_manifest['empty_cluster_counts'],
'warm_seed_counts': cohort_manifest['warm_seed_counts'],
'empty_cluster_count': cohort_manifest['empty_cluster_count'],
@@ -805,12 +942,15 @@ async def _main() -> None:
'matrix_cells': len(cells),
'cache_path': str(cache_path.resolve()),
'cache_key_hash': _json_hash(candidate_cache['key']),
'errors': cohort_errors,
'construction_view_key_hash': _json_hash(
candidate_cache['construction_view_key']
),
'errors': validation_errors,
}
print(json.dumps(validation_payload, indent=2, sort_keys=True), flush=True)
if cohort_errors:
if validation_errors:
raise SystemExit(
'Cohort validation failed; revise and re-hash the specification'
'Research validation failed; do not start the authoritative run'
)
if args.validate_only:
return
@@ -823,7 +963,11 @@ async def _main() -> None:
'snapshot_sha256': snapshot_sha256,
'specification_sha256': specification_sha256,
'candidate_cache_key': candidate_cache['key'],
'construction_view_key': candidate_cache['construction_view_key'],
'universe_manifest': snapshot_data['universe_manifest'],
'construction_universe_manifest': (
snapshot_data['construction_universe_manifest']
),
'cohort_manifest': cohort_manifest,
'arms': list(ARMS),
'costs_per_side_pct': list(COSTS_PER_SIDE_PCT),
@@ -836,7 +980,10 @@ async def _main() -> None:
checkpoint_dir = (
Path(args.checkpoint)
if args.checkpoint
else ROOT / 'reports' / '.cache' / f'{args.run_id}-checkpoint'
else ROOT
/ 'reports'
/ '.cache'
/ f'{args.run_id}-{RUNNER_VERSION}-checkpoint'
)
completed = _checkpoint_state(
checkpoint_dir,
@@ -914,7 +1061,7 @@ async def _main() -> None:
key=lambda row: str(row['cell_id']),
)
analysis = aggregate_results(result_cells)
requirements_path = ROOT / 'requirements.txt'
dependency_manifest = ROOT / 'pyproject.toml'
report: dict[str, Any] = {
'run_id': args.run_id,
'status': 'complete',
@@ -946,9 +1093,10 @@ async def _main() -> None:
'environment': {
'python': sys.version,
'platform': platform.platform(),
'requirements_sha256': (
_sha256_file(requirements_path)
if requirements_path.exists()
'dependency_manifest': str(dependency_manifest.relative_to(ROOT)),
'dependency_manifest_sha256': (
_sha256_file(dependency_manifest)
if dependency_manifest.exists()
else None
),
'command': [sys.executable, *sys.argv],
@@ -956,6 +1104,9 @@ async def _main() -> None:
'validation': validation_payload,
'runtime_config': snapshot_data['runtime_config'],
'universe_manifest': snapshot_data['universe_manifest'],
'construction_universe_manifest': (
snapshot_data['construction_universe_manifest']
),
'candidate_cache': {
key: value
for key, value in candidate_cache.items()
tests/unit/test_portfolio_capacity_research.py
+156 -3
View File
@@ -1,6 +1,7 @@
from __future__ import annotations
import asyncio
import pickle
import sqlite3
from datetime import date, timedelta
@@ -13,12 +14,18 @@ from scripts.portfolio_capacity_research import (
bootstrap_median_interval,
build_cells,
build_cohort_manifest,
iqr,
summarize_simulation,
validate_cohort_manifest,
)
from scripts.run_portfolio_construction_matrix import (
CACHE_VERSION,
_assert_clean_worktree,
_build_candidate_cache,
_checkpoint_state,
_construction_candidate_view,
_construction_universe_errors,
_json_hash,
_load_snapshot,
_markdown,
_operational_summary,
@@ -126,11 +133,18 @@ def test_load_snapshot_accepts_pre_sec_ticker_schema(tmp_path, monkeypatch):
volume BIGINT NOT NULL,
created_at DATETIME NOT NULL
);
CREATE TABLE research_rank_only (
symbol VARCHAR(10) PRIMARY KEY
);
INSERT INTO tickers VALUES
(1, 'LEGACY', 'Legacy Co', '2024-01-01 00:00:00');
(1, 'LEGACY', 'Legacy Co', '2024-01-01 00:00:00'),
(2, 'RANK', 'Rank Only Co', '2024-01-01 00:00:00');
INSERT INTO ohlcv_records VALUES
(1, 1, '2024-01-02', 100, 102, 99, 101, 1000000,
'2024-01-02 00:00:00'),
(2, 2, '2024-01-02', 50, 51, 49, 50, 500000,
'2024-01-02 00:00:00');
INSERT INTO research_rank_only VALUES ('RANK');
'''
)
@@ -167,7 +181,8 @@ def test_load_snapshot_accepts_pre_sec_ticker_schema(tmp_path, monkeypatch):
loaded = asyncio.run(_load_snapshot(snapshot, quiet=True))
assert loaded['symbols'] == ['LEGACY']
assert loaded['symbols'] == ['LEGACY', 'RANK']
assert loaded['construction_symbols'] == {'LEGACY'}
assert loaded['prices']['LEGACY'] == (
[date(2024, 1, 2).toordinal()],
[100.0],
@@ -176,6 +191,11 @@ def test_load_snapshot_accepts_pre_sec_ticker_schema(tmp_path, monkeypatch):
[101.0],
[1_000_000],
)
assert loaded['prices']['RANK'][4] == [50.0]
assert loaded['construction_universe_manifest'][
'construction_ticker_rows'
] == 1
assert loaded['construction_universe_manifest']['rank_only_ticker_rows'] == 1
with sqlite3.connect(snapshot) as connection:
columns = {
row[1] for row in connection.execute('PRAGMA table_info(tickers)')
@@ -183,6 +203,111 @@ def test_load_snapshot_accepts_pre_sec_ticker_schema(tmp_path, monkeypatch):
assert {'cik', 'sic', 'sic_description'}.isdisjoint(columns)
def test_construction_view_filters_rank_only_rows_without_rebuilding_cache():
manifest = {
'ranking_ticker_rows': 506,
'ranking_symbols_with_prices': 506,
'construction_ticker_rows': 505,
'construction_symbols_with_prices': 505,
'rank_only_ticker_rows': 1,
'rank_only_symbols_with_prices': 1,
'rank_only_unknown_symbols': 0,
}
cached = {
'key': {'version': 'existing-broad-cache'},
'qualified_candidates': [
{'symbol': 'PROD', 'date': '2025-01-02'},
{'symbol': 'RANK', 'date': '2025-01-02'},
],
'qualified_long_count': 2,
'daily_rank_map': {
('RANK', '2025-01-02'): {'strategy_rank': 99.0},
},
}
view = _construction_candidate_view(
cached,
{
'construction_symbols': {'PROD'},
'construction_universe_manifest': manifest,
},
)
assert [row['symbol'] for row in view['qualified_candidates']] == ['PROD']
assert view['raw_full_universe_qualified_long_count'] == 2
assert view['filtered_rank_only_qualified_long_count'] == 1
assert view['qualified_long_count'] == 1
assert ('RANK', '2025-01-02') in view['daily_rank_map']
assert len(cached['qualified_candidates']) == 2
def test_existing_broad_candidate_cache_key_remains_reusable(tmp_path, monkeypatch):
snapshot = tmp_path / 'research.sqlite'
snapshot.write_bytes(b'snapshot-placeholder')
cache_path = tmp_path / 'broad-cache.pkl'
snapshot_data = {
'recommendation_config': {'rr': 3.0},
'activation': {'min_momentum_percentile': 80.0},
'runtime_config': {'ranking_key': 'test'},
'universe_manifest': {
'ticker_rows': 4655,
'symbols_with_prices': 4654,
'symbols_sha256': 'symbols',
},
}
key = {
'version': CACHE_VERSION,
'snapshot': str(snapshot.resolve()),
'snapshot_sha256': 'snapshot-hash',
'cadence': 'daily',
'outcome_horizon_sessions': 0,
'recommendation_config_hash': _json_hash(
snapshot_data['recommendation_config']
),
'activation_hash': _json_hash(snapshot_data['activation']),
'runtime_config': snapshot_data['runtime_config'],
'universe_manifest': snapshot_data['universe_manifest'],
}
cached = {'key': key, 'qualified_candidates': [{'symbol': 'PROD'}]}
cache_path.write_bytes(pickle.dumps(cached))
monkeypatch.setattr(
bt,
'_replay_candidates_for_period',
lambda *_args: pytest.fail('existing cache should avoid replay'),
)
loaded = _build_candidate_cache(
snapshot_data,
snapshot=snapshot,
snapshot_sha256='snapshot-hash',
cache_path=cache_path,
workers=1,
quiet=True,
)
assert loaded == cached
def test_construction_universe_guard_rejects_leaked_broad_book():
valid = {
'ranking_ticker_rows': 4655,
'construction_ticker_rows': 506,
'construction_symbols_with_prices': 506,
'rank_only_ticker_rows': 4149,
'rank_only_unknown_symbols': 0,
}
assert _construction_universe_errors(valid) == []
leaked = {
**valid,
'construction_ticker_rows': 4655,
'construction_symbols_with_prices': 4654,
'rank_only_ticker_rows': 0,
}
errors = _construction_universe_errors(leaked)
assert any('450-600' in error for error in errors)
def test_unbounded_count_and_effective_risk_floor():
start = date(2025, 1, 6)
ords = [start.toordinal() + offset for offset in range(4)]
@@ -504,6 +629,10 @@ def test_simple_cluster_bootstrap_is_deterministic_and_not_a_gate():
assert first['p05'] <= first['point'] <= first['p95']
def test_iqr_materializes_generator_before_both_quantiles():
assert iqr(value for value in (0.0, 1.0, 2.0, 3.0)) == pytest.approx(1.5)
def test_aggregate_reports_paired_years_and_separate_warm_iqrs():
cells: list[dict] = []
for cost in (0.1, 0.2):
@@ -579,16 +708,40 @@ def test_aggregate_reports_paired_years_and_separate_warm_iqrs():
)
assert set(cash_warm['headline']) == {'ev_net_r', 'calmar'}
assert 'D' not in cash_warm
assert cash_warm['headline']['ev_net_r']['median_iqr_ratio'] == pytest.approx(
1.0
)
assert cash_warm['headline']['calmar']['median_iqr_ratio'] == pytest.approx(
1.0
)
assert cash_warm['headline']['ev_net_r']['bootstrap_90']['n'] == 7
markdown = _markdown({
'generated_at': '2026-08-05T00:00:00Z',
'analysis': report,
'operational_summary': _operational_summary(cells),
'validation': {
'construction_universe_manifest': {
'construction_symbols_with_prices': 506,
'rank_only_symbols_with_prices': 4148,
'ranking_symbols_with_prices': 4654,
},
'candidate_rank_coverage': {
'construction_qualified_longs': 5000,
'filtered_rank_only_qualified_longs': 137000,
},
},
})
assert 'ΔGain-to-Pain' in markdown
assert '0.10% per fill' in markdown
assert '0.20% per fill' in markdown
assert 'Tradable setup symbols with prices: 506.' in markdown
assert 'Rank-only qualified rows removed: 137000.' in markdown
assert 'formal promotion gate' in markdown
def test_synthetic_worker_matrix_covers_four_arms_protocols_and_costs():
def test_synthetic_worker_matrix_covers_four_arms_protocols_and_costs(monkeypatch):
monkeypatch.setenv('BACKTEST_SNAPSHOT_OFFLINE', '0')
monkeypatch.setenv('BACKTEST_ALLOW_SPAWN', '0')
start = date(2025, 1, 6)
sessions = _business_days(start, date(2025, 1, 17))
ords = [session.toordinal() for session in sessions]