fix: isolate production universe in capacity research
This commit is contained in:
@@ -22,6 +22,11 @@ Because the current ~505-name production membership is projected backward,
|
|||||||
paired arm-versus-control differences are the primary evidence. Absolute
|
paired arm-versus-control differences are the primary evidence. Absolute
|
||||||
profitability is descriptive and survivorship-biased.
|
profitability is descriptive and survivorship-biased.
|
||||||
|
|
||||||
|
Implementation correction: the first completed v1 artifact at commit `23fe39f`
|
||||||
|
incorrectly allowed the snapshot's broad rank-only universe to submit trades.
|
||||||
|
That artifact is invalid, is removed from the branch, and must not be used for
|
||||||
|
strategy conclusions. Runner v2 fixes the construction/ranking partition below.
|
||||||
|
|
||||||
## Frozen arms
|
## Frozen arms
|
||||||
|
|
||||||
1. **cap10_incumbent:** exact production-style cap-10 control, no displacement.
|
1. **cap10_incumbent:** exact production-style cap-10 control, no displacement.
|
||||||
@@ -37,6 +42,12 @@ live-like full-universe residual-momentum/low-volatility 80/20 rank, activation
|
|||||||
threshold 80, normal gate-reset re-entry, close fill, 3×ATR trail, 30-session
|
threshold 80, normal gate-reset re-entry, close fill, 3×ATR trail, 30-session
|
||||||
maximum hold, 1% risk, and costs of 0.10% and 0.20% per fill.
|
maximum hold, 1% risk, and costs of 0.10% and 0.20% per fill.
|
||||||
|
|
||||||
|
Every priced symbol contributes to the daily cross-sectional rank. Only symbols
|
||||||
|
not listed in the snapshot's `research_rank_only` side table may submit trade
|
||||||
|
setups to any arm. The resulting construction universe must contain 450-600
|
||||||
|
symbols (expected approximately 505); validation fails outside that frozen
|
||||||
|
guardrail or when the side table references unknown ticker symbols.
|
||||||
|
|
||||||
The daily replay uses zero outcome horizon: setup and rank observations continue
|
The daily replay uses zero outcome horizon: setup and rank observations continue
|
||||||
through the snapshot's last session because portfolio simulation, unlike outcome
|
through the snapshot's last session because portfolio simulation, unlike outcome
|
||||||
grading, does not require 30 future bars.
|
grading, does not require 30 future bars.
|
||||||
@@ -83,7 +94,8 @@ positions liquidate at the last measurement close with costs.
|
|||||||
|
|
||||||
The validate-only mode must print realized cohort counts and fail unless both
|
The validate-only mode must print realized cohort counts and fail unless both
|
||||||
protocols contain the seven annual clusters 2019–2025 and every warm anchor has
|
protocols contain the seven annual clusters 2019–2025 and every warm anchor has
|
||||||
at least 12 seeds.
|
at least 12 seeds. It must also print ranking, rank-only, and tradable symbol
|
||||||
|
counts plus the raw, removed, and retained qualified-long counts.
|
||||||
|
|
||||||
## Reporting
|
## Reporting
|
||||||
|
|
||||||
@@ -120,6 +132,12 @@ atomically and resume verifies a fingerprint over the implementation commit,
|
|||||||
this specification hash, snapshot SHA-256, cache key, arm definitions, costs,
|
this specification hash, snapshot SHA-256, cache key, arm definitions, costs,
|
||||||
and cohort manifest. An authoritative run refuses a dirty worktree.
|
and cohort manifest. An authoritative run refuses a dirty worktree.
|
||||||
|
|
||||||
|
The existing v1 candidate/rank cache is intentionally reusable: its
|
||||||
|
full-universe current-day ranks are correct. Runner v2 derives a fingerprinted
|
||||||
|
construction view by removing qualified rows whose symbols are rank-only. V2
|
||||||
|
uses a versioned checkpoint directory, so invalid v1 portfolio cells are never
|
||||||
|
resumed and the expensive daily rank replay does not need to run again.
|
||||||
|
|
||||||
The loader reads only ticker ID/symbol and the OHLCV columns used by replay, so
|
The loader reads only ticker ID/symbol and the OHLCV columns used by replay, so
|
||||||
snapshots created before SEC metadata added `tickers.cik`, `tickers.sic`, and
|
snapshots created before SEC metadata added `tickers.cik`, `tickers.sic`, and
|
||||||
`tickers.sic_description` remain valid. Do not migrate or alter the research
|
`tickers.sic_description` remain valid. Do not migrate or alter the research
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@@ -1,74 +0,0 @@
|
|||||||
# Focused daily portfolio-capacity matrix
|
|
||||||
|
|
||||||
Generated: 2026-08-05T18:12:50.577686+00:00
|
|
||||||
|
|
||||||
## Question
|
|
||||||
|
|
||||||
The current daily Phase A control admitted 472 trades and rejected 519 qualified opportunities because the ten-slot book was full. This run brackets the economic cost of that binding constraint; it has no formal promotion gate.
|
|
||||||
|
|
||||||
> Universe caveat: today's production membership is projected backward. Use paired arm-versus-control differences, not absolute profitability, for construction conclusions.
|
|
||||||
|
|
||||||
## Paired annual medians
|
|
||||||
|
|
||||||
### Empty Book — 0.10% per fill
|
|
||||||
|
|
||||||
| Arm | ΔEV net R | 90% context | ΔCalmar | 90% context |
|
|
||||||
|---|---:|---:|---:|---:|
|
|
||||||
| cap10_incumbent | 0.000 | [0.000, 0.000] | 0.000 | [0.000, 0.000] |
|
|
||||||
| cash_unbounded | 0.137 | [0.120, 0.185] | 0.000 | [-0.010, 0.270] |
|
|
||||||
| cap10_weekly_top10 | 0.092 | [0.040, 0.117] | 0.000 | [-0.040, 0.000] |
|
|
||||||
| cap15_incumbent | 0.098 | [0.022, 0.183] | 0.000 | [-0.010, 0.160] |
|
|
||||||
|
|
||||||
| Arm | ΔPF | ΔGain-to-Pain | ΔSortino | ΔCAGR pp | ΔMaxDD pp |
|
|
||||||
|---|---:|---:|---:|---:|---:|
|
|
||||||
| cap10_incumbent | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
|
|
||||||
| cash_unbounded | 0.094 | 0.068 | 0.313 | 0.800 | 3.800 |
|
|
||||||
| cap10_weekly_top10 | -0.035 | -0.052 | -0.196 | -5.700 | 2.850 |
|
|
||||||
| cap15_incumbent | 0.037 | 0.017 | 0.168 | -0.050 | 2.150 |
|
|
||||||
|
|
||||||
### Warm Book — 0.10% per fill
|
|
||||||
|
|
||||||
| Arm | ΔEV net R | 90% context | ΔCalmar | 90% context |
|
|
||||||
|---|---:|---:|---:|---:|
|
|
||||||
| cap10_incumbent | 0.000 | [0.000, 0.000] | 0.000 | [0.000, 0.000] |
|
|
||||||
| cash_unbounded | 0.119 | [-0.014, 0.214] | 0.000 | [-0.045, 0.150] |
|
|
||||||
| cap10_weekly_top10 | 0.068 | [0.063, 0.154] | 0.000 | [-0.110, 0.020] |
|
|
||||||
| cap15_incumbent | 0.063 | [-0.104, 0.145] | 0.000 | [-0.020, 0.110] |
|
|
||||||
|
|
||||||
| Arm | ΔPF | ΔGain-to-Pain | ΔSortino | ΔCAGR pp | ΔMaxDD pp |
|
|
||||||
|---|---:|---:|---:|---:|---:|
|
|
||||||
| cap10_incumbent | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
|
|
||||||
| cash_unbounded | 0.069 | 0.018 | -0.921 | -4.500 | 0.000 |
|
|
||||||
| cap10_weekly_top10 | 0.000 | -0.026 | -0.869 | -4.100 | 3.900 |
|
|
||||||
| cap15_incumbent | 0.035 | 0.006 | -0.994 | -0.900 | 1.050 |
|
|
||||||
|
|
||||||
## Warm-seed initialization dispersion
|
|
||||||
|
|
||||||
| Arm | Cost/fill | Median EV IQR ratio | Median Calmar IQR ratio |
|
|
||||||
|---|---:|---:|---:|
|
|
||||||
| cap10_incumbent | 0.10% | n/a | n/a |
|
|
||||||
| cash_unbounded | 0.10% | n/a | n/a |
|
|
||||||
| cap10_weekly_top10 | 0.10% | n/a | n/a |
|
|
||||||
| cap15_incumbent | 0.10% | n/a | n/a |
|
|
||||||
| cap10_incumbent | 0.20% | n/a | n/a |
|
|
||||||
| cash_unbounded | 0.20% | n/a | n/a |
|
|
||||||
| cap10_weekly_top10 | 0.20% | n/a | n/a |
|
|
||||||
| cap15_incumbent | 0.20% | n/a | n/a |
|
|
||||||
|
|
||||||
## Capacity and operations — 0.10% per fill
|
|
||||||
|
|
||||||
| Arm | Median trades | Median blocked | Median positions | Peak | Turnover | Min-risk rejects |
|
|
||||||
|---|---:|---:|---:|---:|---:|---:|
|
|
||||||
| cap10_incumbent | 212.0 | 98.4% | 10.00 | 10 | 14.72 | 0 |
|
|
||||||
| cash_unbounded | 340.0 | 0.0% | 18.24 | 41 | 19.04 | 2719474 |
|
|
||||||
| cap10_weekly_top10 | 303.0 | 97.1% | 9.99 | 10 | 18.07 | 0 |
|
|
||||||
| cap15_incumbent | 327.0 | 97.3% | 14.78 | 15 | 17.30 | 0 |
|
|
||||||
|
|
||||||
## Weekly-ranking opportunity set
|
|
||||||
|
|
||||||
- Median fresh entrant pool: 47.0.
|
|
||||||
- Median zero-entrant fraction: 0.000.
|
|
||||||
- Replacements across reported paths: 15081.
|
|
||||||
- Same-symbol re-entries within 10 sessions: 4081.
|
|
||||||
|
|
||||||
Bootstrap intervals above resample seven annual summaries and are descriptive context only. They are not gates or independent-population confidence claims.
|
|
||||||
@@ -225,8 +225,15 @@ def percentile(values: Iterable[float], probability: float) -> float | None:
|
|||||||
|
|
||||||
|
|
||||||
def iqr(values: Iterable[float]) -> float | None:
|
def iqr(values: Iterable[float]) -> float | None:
|
||||||
q25 = percentile(values, 0.25)
|
clean: list[float] = []
|
||||||
q75 = percentile(values, 0.75)
|
for value in values:
|
||||||
|
if value is None:
|
||||||
|
continue
|
||||||
|
parsed = float(value)
|
||||||
|
if math.isfinite(parsed):
|
||||||
|
clean.append(parsed)
|
||||||
|
q25 = percentile(clean, 0.25)
|
||||||
|
q75 = percentile(clean, 0.75)
|
||||||
if q25 is None or q75 is None:
|
if q25 is None or q75 is None:
|
||||||
return None
|
return None
|
||||||
return q75 - q25
|
return q75 - q25
|
||||||
|
|||||||
@@ -48,7 +48,10 @@ from scripts.research_rankings import _live_universe_rank_map # noqa: E402
|
|||||||
|
|
||||||
|
|
||||||
CACHE_VERSION = 'portfolio-capacity-candidates-v1-zero-horizon'
|
CACHE_VERSION = 'portfolio-capacity-candidates-v1-zero-horizon'
|
||||||
RUNNER_VERSION = 'portfolio-capacity-bracket-v1'
|
RUNNER_VERSION = 'portfolio-capacity-bracket-v2'
|
||||||
|
CONSTRUCTION_VIEW_VERSION = 'production-book-filter-v1'
|
||||||
|
MIN_PRODUCTION_UNIVERSE = 450
|
||||||
|
MAX_PRODUCTION_UNIVERSE = 600
|
||||||
SPEC_PATH = ROOT / 'docs' / 'research' / 'portfolio-capacity-bracket.md'
|
SPEC_PATH = ROOT / 'docs' / 'research' / 'portfolio-capacity-bracket.md'
|
||||||
DEFAULT_RUN_ID = 'prod505-capacity-bracket-daily-v1'
|
DEFAULT_RUN_ID = 'prod505-capacity-bracket-daily-v1'
|
||||||
_WORKER_CONTEXT: dict[str, Any] | None = None
|
_WORKER_CONTEXT: dict[str, Any] | None = None
|
||||||
@@ -103,6 +106,67 @@ def _json_hash(value: Any) -> str:
|
|||||||
return hashlib.sha256(payload).hexdigest()
|
return hashlib.sha256(payload).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _construction_universe_errors(manifest: dict[str, Any]) -> list[str]:
|
||||||
|
errors: list[str] = []
|
||||||
|
construction_rows = int(manifest['construction_ticker_rows'])
|
||||||
|
construction_priced = int(manifest['construction_symbols_with_prices'])
|
||||||
|
ranking_rows = int(manifest['ranking_ticker_rows'])
|
||||||
|
rank_only_rows = int(manifest['rank_only_ticker_rows'])
|
||||||
|
unknown_rank_only = int(manifest['rank_only_unknown_symbols'])
|
||||||
|
if not MIN_PRODUCTION_UNIVERSE <= construction_rows <= MAX_PRODUCTION_UNIVERSE:
|
||||||
|
errors.append(
|
||||||
|
'construction ticker universe must contain '
|
||||||
|
f'{MIN_PRODUCTION_UNIVERSE}-{MAX_PRODUCTION_UNIVERSE} symbols; '
|
||||||
|
f'found {construction_rows}'
|
||||||
|
)
|
||||||
|
if not MIN_PRODUCTION_UNIVERSE <= construction_priced <= MAX_PRODUCTION_UNIVERSE:
|
||||||
|
errors.append(
|
||||||
|
'priced construction universe must contain '
|
||||||
|
f'{MIN_PRODUCTION_UNIVERSE}-{MAX_PRODUCTION_UNIVERSE} symbols; '
|
||||||
|
f'found {construction_priced}'
|
||||||
|
)
|
||||||
|
if construction_rows + rank_only_rows != ranking_rows:
|
||||||
|
errors.append(
|
||||||
|
'construction and rank-only ticker partitions do not cover the '
|
||||||
|
'ranking universe'
|
||||||
|
)
|
||||||
|
if unknown_rank_only:
|
||||||
|
errors.append(
|
||||||
|
f'research_rank_only contains {unknown_rank_only} symbols absent '
|
||||||
|
'from tickers'
|
||||||
|
)
|
||||||
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
def _construction_candidate_view(
|
||||||
|
cached: dict[str, Any],
|
||||||
|
snapshot_data: dict[str, Any],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
'''Filter a reusable full-universe rank cache to production setup symbols.'''
|
||||||
|
allowed = set(snapshot_data['construction_symbols'])
|
||||||
|
raw_candidates = list(cached['qualified_candidates'])
|
||||||
|
qualified = [
|
||||||
|
row for row in raw_candidates if str(row['symbol']) in allowed
|
||||||
|
]
|
||||||
|
view_key = {
|
||||||
|
'version': CONSTRUCTION_VIEW_VERSION,
|
||||||
|
'base_cache_key_hash': _json_hash(cached['key']),
|
||||||
|
'construction_universe_manifest': (
|
||||||
|
snapshot_data['construction_universe_manifest']
|
||||||
|
),
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
**cached,
|
||||||
|
'qualified_candidates': qualified,
|
||||||
|
'raw_full_universe_qualified_long_count': len(raw_candidates),
|
||||||
|
'qualified_long_count': len(qualified),
|
||||||
|
'filtered_rank_only_qualified_long_count': (
|
||||||
|
len(raw_candidates) - len(qualified)
|
||||||
|
),
|
||||||
|
'construction_view_key': view_key,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def _atomic_json(path: Path, value: Any) -> None:
|
def _atomic_json(path: Path, value: Any) -> None:
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
temporary = path.with_suffix(path.suffix + '.tmp')
|
temporary = path.with_suffix(path.suffix + '.tmp')
|
||||||
@@ -292,12 +356,34 @@ async def _load_snapshot(
|
|||||||
f'loaded prices: {index}/{len(symbols)}',
|
f'loaded prices: {index}/{len(symbols)}',
|
||||||
flush=True,
|
flush=True,
|
||||||
)
|
)
|
||||||
|
rank_only_symbols = await bt._load_research_rank_only_symbols(db)
|
||||||
finally:
|
finally:
|
||||||
await engine.dispose()
|
await engine.dispose()
|
||||||
|
|
||||||
if not prices or not benchmark_closes:
|
if not prices or not benchmark_closes:
|
||||||
raise SystemExit('Snapshot has no usable prices or benchmark history')
|
raise SystemExit('Snapshot has no usable prices or benchmark history')
|
||||||
|
|
||||||
|
ticker_symbols = set(symbols)
|
||||||
|
price_symbols = set(prices)
|
||||||
|
known_rank_only = ticker_symbols & rank_only_symbols
|
||||||
|
unknown_rank_only = rank_only_symbols - ticker_symbols
|
||||||
|
construction_symbols = ticker_symbols - known_rank_only
|
||||||
|
priced_construction_symbols = price_symbols & construction_symbols
|
||||||
|
priced_rank_only_symbols = price_symbols & known_rank_only
|
||||||
|
construction_universe_manifest = {
|
||||||
|
'ranking_ticker_rows': len(ticker_symbols),
|
||||||
|
'ranking_symbols_with_prices': len(price_symbols),
|
||||||
|
'ranking_symbols_sha256': _json_hash(sorted(ticker_symbols)),
|
||||||
|
'construction_ticker_rows': len(construction_symbols),
|
||||||
|
'construction_symbols_with_prices': len(priced_construction_symbols),
|
||||||
|
'construction_symbols_sha256': _json_hash(sorted(construction_symbols)),
|
||||||
|
'rank_only_ticker_rows': len(known_rank_only),
|
||||||
|
'rank_only_symbols_with_prices': len(priced_rank_only_symbols),
|
||||||
|
'rank_only_symbols_sha256': _json_hash(sorted(known_rank_only)),
|
||||||
|
'rank_only_unknown_symbols': len(unknown_rank_only),
|
||||||
|
'rank_only_unknown_symbols_sha256': _json_hash(sorted(unknown_rank_only)),
|
||||||
|
}
|
||||||
|
|
||||||
strategy = next(
|
strategy = next(
|
||||||
row
|
row
|
||||||
for row in bt.PORTFOLIO_MONITOR_STRATEGIES
|
for row in bt.PORTFOLIO_MONITOR_STRATEGIES
|
||||||
@@ -350,11 +436,13 @@ async def _load_snapshot(
|
|||||||
'benchmark_closes': benchmark_closes,
|
'benchmark_closes': benchmark_closes,
|
||||||
'prices': prices,
|
'prices': prices,
|
||||||
'symbols': symbols,
|
'symbols': symbols,
|
||||||
|
'construction_symbols': construction_symbols,
|
||||||
'universe_manifest': {
|
'universe_manifest': {
|
||||||
'ticker_rows': len(symbols),
|
'ticker_rows': len(symbols),
|
||||||
'symbols_with_prices': len(prices),
|
'symbols_with_prices': len(prices),
|
||||||
'symbols_sha256': _json_hash(sorted(symbols)),
|
'symbols_sha256': _json_hash(sorted(symbols)),
|
||||||
},
|
},
|
||||||
|
'construction_universe_manifest': construction_universe_manifest,
|
||||||
'ranking_key': ranking_key,
|
'ranking_key': ranking_key,
|
||||||
'exit_policy': exit_policy,
|
'exit_policy': exit_policy,
|
||||||
'hold_days': hold_days,
|
'hold_days': hold_days,
|
||||||
@@ -639,21 +727,51 @@ def _markdown(report: dict[str, Any]) -> str:
|
|||||||
'## Paired annual medians',
|
'## Paired annual medians',
|
||||||
'',
|
'',
|
||||||
]
|
]
|
||||||
|
validation = report.get('validation') or {}
|
||||||
|
construction = validation.get('construction_universe_manifest') or {}
|
||||||
|
coverage = validation.get('candidate_rank_coverage') or {}
|
||||||
|
if construction:
|
||||||
|
construction_count = construction['construction_symbols_with_prices']
|
||||||
|
rank_only_count = construction['rank_only_symbols_with_prices']
|
||||||
|
ranking_count = construction['ranking_symbols_with_prices']
|
||||||
|
qualified_count = coverage.get('construction_qualified_longs')
|
||||||
|
filtered_count = coverage.get('filtered_rank_only_qualified_longs')
|
||||||
|
insertion = lines.index('## Paired annual medians')
|
||||||
|
lines[insertion:insertion] = [
|
||||||
|
'## Validated universes',
|
||||||
|
'',
|
||||||
|
f'- Tradable setup symbols with prices: '
|
||||||
|
f'{construction_count}.',
|
||||||
|
f'- Rank-only symbols with prices: '
|
||||||
|
f'{rank_only_count}.',
|
||||||
|
f'- Full ranking symbols with prices: '
|
||||||
|
f'{ranking_count}.',
|
||||||
|
f'- Tradable qualified longs: '
|
||||||
|
f'{qualified_count}.',
|
||||||
|
f'- Rank-only qualified rows removed: '
|
||||||
|
f'{filtered_count}.',
|
||||||
|
'',
|
||||||
|
]
|
||||||
paired = report['analysis']['paired_per_year']
|
paired = report['analysis']['paired_per_year']
|
||||||
for protocol in ('empty_book', 'warm_book'):
|
for cost, protocol in (
|
||||||
|
(cost, protocol)
|
||||||
|
for cost in COSTS_PER_SIDE_PCT
|
||||||
|
for protocol in ('empty_book', 'warm_book')
|
||||||
|
):
|
||||||
lines.extend([
|
lines.extend([
|
||||||
f'### {protocol.replace("_", " ").title()} — 0.10% per fill',
|
f'### {protocol.replace("_", " ").title()} — 0.10% per fill',
|
||||||
'',
|
'',
|
||||||
'| Arm | ΔEV net R | 90% context | ΔCalmar | 90% context |',
|
'| Arm | ΔEV net R | 90% context | ΔCalmar | 90% context |',
|
||||||
'|---|---:|---:|---:|---:|',
|
'|---|---:|---:|---:|---:|',
|
||||||
])
|
])
|
||||||
|
lines[-4] = lines[-4].replace('0.10%', f'{cost:.2f}%')
|
||||||
for arm in ARMS:
|
for arm in ARMS:
|
||||||
row = next(
|
row = next(
|
||||||
item
|
item
|
||||||
for item in paired
|
for item in paired
|
||||||
if item['arm_id'] == arm['id']
|
if item['arm_id'] == arm['id']
|
||||||
and item['protocol'] == protocol
|
and item['protocol'] == protocol
|
||||||
and float(item['cost_per_side_pct']) == 0.1
|
and float(item['cost_per_side_pct']) == float(cost)
|
||||||
)
|
)
|
||||||
ev = row['headline']['ev_net_r']
|
ev = row['headline']['ev_net_r']
|
||||||
calmar = row['headline']['calmar']
|
calmar = row['headline']['calmar']
|
||||||
@@ -676,7 +794,7 @@ def _markdown(report: dict[str, Any]) -> str:
|
|||||||
for item in paired
|
for item in paired
|
||||||
if item['arm_id'] == arm['id']
|
if item['arm_id'] == arm['id']
|
||||||
and item['protocol'] == protocol
|
and item['protocol'] == protocol
|
||||||
and float(item['cost_per_side_pct']) == 0.1
|
and float(item['cost_per_side_pct']) == float(cost)
|
||||||
)
|
)
|
||||||
headline = row['headline']
|
headline = row['headline']
|
||||||
lines.append(
|
lines.append(
|
||||||
@@ -767,7 +885,7 @@ async def _main() -> None:
|
|||||||
if args.candidate_cache
|
if args.candidate_cache
|
||||||
else ROOT / 'reports' / '.cache' / f'{args.run_id}-candidates.pkl'
|
else ROOT / 'reports' / '.cache' / f'{args.run_id}-candidates.pkl'
|
||||||
)
|
)
|
||||||
candidate_cache = _build_candidate_cache(
|
base_candidate_cache = _build_candidate_cache(
|
||||||
snapshot_data,
|
snapshot_data,
|
||||||
snapshot=snapshot,
|
snapshot=snapshot,
|
||||||
snapshot_sha256=snapshot_sha256,
|
snapshot_sha256=snapshot_sha256,
|
||||||
@@ -775,11 +893,19 @@ async def _main() -> None:
|
|||||||
workers=workers,
|
workers=workers,
|
||||||
quiet=bool(args.quiet),
|
quiet=bool(args.quiet),
|
||||||
)
|
)
|
||||||
|
candidate_cache = _construction_candidate_view(
|
||||||
|
base_candidate_cache,
|
||||||
|
snapshot_data,
|
||||||
|
)
|
||||||
|
|
||||||
cohort_manifest = build_cohort_manifest(
|
cohort_manifest = build_cohort_manifest(
|
||||||
snapshot_data['benchmark_closes'].keys()
|
snapshot_data['benchmark_closes'].keys()
|
||||||
)
|
)
|
||||||
cohort_errors = validate_cohort_manifest(cohort_manifest)
|
cohort_errors = validate_cohort_manifest(cohort_manifest)
|
||||||
|
universe_errors = _construction_universe_errors(
|
||||||
|
snapshot_data['construction_universe_manifest']
|
||||||
|
)
|
||||||
|
validation_errors = [*universe_errors, *cohort_errors]
|
||||||
cells = build_cells(cohort_manifest)
|
cells = build_cells(cohort_manifest)
|
||||||
validation_payload = {
|
validation_payload = {
|
||||||
'runner_version': RUNNER_VERSION,
|
'runner_version': RUNNER_VERSION,
|
||||||
@@ -794,9 +920,20 @@ async def _main() -> None:
|
|||||||
'first': candidate_cache['rank_first_date'],
|
'first': candidate_cache['rank_first_date'],
|
||||||
'last': candidate_cache['rank_last_date'],
|
'last': candidate_cache['rank_last_date'],
|
||||||
'observations': candidate_cache['rank_observation_count'],
|
'observations': candidate_cache['rank_observation_count'],
|
||||||
'qualified_longs': candidate_cache['qualified_long_count'],
|
'raw_full_universe_qualified_longs': candidate_cache[
|
||||||
|
'raw_full_universe_qualified_long_count'
|
||||||
|
],
|
||||||
|
'filtered_rank_only_qualified_longs': candidate_cache[
|
||||||
|
'filtered_rank_only_qualified_long_count'
|
||||||
|
],
|
||||||
|
'construction_qualified_longs': candidate_cache[
|
||||||
|
'qualified_long_count'
|
||||||
|
],
|
||||||
},
|
},
|
||||||
'universe_manifest': snapshot_data['universe_manifest'],
|
'universe_manifest': snapshot_data['universe_manifest'],
|
||||||
|
'construction_universe_manifest': (
|
||||||
|
snapshot_data['construction_universe_manifest']
|
||||||
|
),
|
||||||
'empty_cluster_counts': cohort_manifest['empty_cluster_counts'],
|
'empty_cluster_counts': cohort_manifest['empty_cluster_counts'],
|
||||||
'warm_seed_counts': cohort_manifest['warm_seed_counts'],
|
'warm_seed_counts': cohort_manifest['warm_seed_counts'],
|
||||||
'empty_cluster_count': cohort_manifest['empty_cluster_count'],
|
'empty_cluster_count': cohort_manifest['empty_cluster_count'],
|
||||||
@@ -805,12 +942,15 @@ async def _main() -> None:
|
|||||||
'matrix_cells': len(cells),
|
'matrix_cells': len(cells),
|
||||||
'cache_path': str(cache_path.resolve()),
|
'cache_path': str(cache_path.resolve()),
|
||||||
'cache_key_hash': _json_hash(candidate_cache['key']),
|
'cache_key_hash': _json_hash(candidate_cache['key']),
|
||||||
'errors': cohort_errors,
|
'construction_view_key_hash': _json_hash(
|
||||||
|
candidate_cache['construction_view_key']
|
||||||
|
),
|
||||||
|
'errors': validation_errors,
|
||||||
}
|
}
|
||||||
print(json.dumps(validation_payload, indent=2, sort_keys=True), flush=True)
|
print(json.dumps(validation_payload, indent=2, sort_keys=True), flush=True)
|
||||||
if cohort_errors:
|
if validation_errors:
|
||||||
raise SystemExit(
|
raise SystemExit(
|
||||||
'Cohort validation failed; revise and re-hash the specification'
|
'Research validation failed; do not start the authoritative run'
|
||||||
)
|
)
|
||||||
if args.validate_only:
|
if args.validate_only:
|
||||||
return
|
return
|
||||||
@@ -823,7 +963,11 @@ async def _main() -> None:
|
|||||||
'snapshot_sha256': snapshot_sha256,
|
'snapshot_sha256': snapshot_sha256,
|
||||||
'specification_sha256': specification_sha256,
|
'specification_sha256': specification_sha256,
|
||||||
'candidate_cache_key': candidate_cache['key'],
|
'candidate_cache_key': candidate_cache['key'],
|
||||||
|
'construction_view_key': candidate_cache['construction_view_key'],
|
||||||
'universe_manifest': snapshot_data['universe_manifest'],
|
'universe_manifest': snapshot_data['universe_manifest'],
|
||||||
|
'construction_universe_manifest': (
|
||||||
|
snapshot_data['construction_universe_manifest']
|
||||||
|
),
|
||||||
'cohort_manifest': cohort_manifest,
|
'cohort_manifest': cohort_manifest,
|
||||||
'arms': list(ARMS),
|
'arms': list(ARMS),
|
||||||
'costs_per_side_pct': list(COSTS_PER_SIDE_PCT),
|
'costs_per_side_pct': list(COSTS_PER_SIDE_PCT),
|
||||||
@@ -836,7 +980,10 @@ async def _main() -> None:
|
|||||||
checkpoint_dir = (
|
checkpoint_dir = (
|
||||||
Path(args.checkpoint)
|
Path(args.checkpoint)
|
||||||
if args.checkpoint
|
if args.checkpoint
|
||||||
else ROOT / 'reports' / '.cache' / f'{args.run_id}-checkpoint'
|
else ROOT
|
||||||
|
/ 'reports'
|
||||||
|
/ '.cache'
|
||||||
|
/ f'{args.run_id}-{RUNNER_VERSION}-checkpoint'
|
||||||
)
|
)
|
||||||
completed = _checkpoint_state(
|
completed = _checkpoint_state(
|
||||||
checkpoint_dir,
|
checkpoint_dir,
|
||||||
@@ -914,7 +1061,7 @@ async def _main() -> None:
|
|||||||
key=lambda row: str(row['cell_id']),
|
key=lambda row: str(row['cell_id']),
|
||||||
)
|
)
|
||||||
analysis = aggregate_results(result_cells)
|
analysis = aggregate_results(result_cells)
|
||||||
requirements_path = ROOT / 'requirements.txt'
|
dependency_manifest = ROOT / 'pyproject.toml'
|
||||||
report: dict[str, Any] = {
|
report: dict[str, Any] = {
|
||||||
'run_id': args.run_id,
|
'run_id': args.run_id,
|
||||||
'status': 'complete',
|
'status': 'complete',
|
||||||
@@ -946,9 +1093,10 @@ async def _main() -> None:
|
|||||||
'environment': {
|
'environment': {
|
||||||
'python': sys.version,
|
'python': sys.version,
|
||||||
'platform': platform.platform(),
|
'platform': platform.platform(),
|
||||||
'requirements_sha256': (
|
'dependency_manifest': str(dependency_manifest.relative_to(ROOT)),
|
||||||
_sha256_file(requirements_path)
|
'dependency_manifest_sha256': (
|
||||||
if requirements_path.exists()
|
_sha256_file(dependency_manifest)
|
||||||
|
if dependency_manifest.exists()
|
||||||
else None
|
else None
|
||||||
),
|
),
|
||||||
'command': [sys.executable, *sys.argv],
|
'command': [sys.executable, *sys.argv],
|
||||||
@@ -956,6 +1104,9 @@ async def _main() -> None:
|
|||||||
'validation': validation_payload,
|
'validation': validation_payload,
|
||||||
'runtime_config': snapshot_data['runtime_config'],
|
'runtime_config': snapshot_data['runtime_config'],
|
||||||
'universe_manifest': snapshot_data['universe_manifest'],
|
'universe_manifest': snapshot_data['universe_manifest'],
|
||||||
|
'construction_universe_manifest': (
|
||||||
|
snapshot_data['construction_universe_manifest']
|
||||||
|
),
|
||||||
'candidate_cache': {
|
'candidate_cache': {
|
||||||
key: value
|
key: value
|
||||||
for key, value in candidate_cache.items()
|
for key, value in candidate_cache.items()
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
|
import pickle
|
||||||
import sqlite3
|
import sqlite3
|
||||||
from datetime import date, timedelta
|
from datetime import date, timedelta
|
||||||
|
|
||||||
@@ -13,12 +14,18 @@ from scripts.portfolio_capacity_research import (
|
|||||||
bootstrap_median_interval,
|
bootstrap_median_interval,
|
||||||
build_cells,
|
build_cells,
|
||||||
build_cohort_manifest,
|
build_cohort_manifest,
|
||||||
|
iqr,
|
||||||
summarize_simulation,
|
summarize_simulation,
|
||||||
validate_cohort_manifest,
|
validate_cohort_manifest,
|
||||||
)
|
)
|
||||||
from scripts.run_portfolio_construction_matrix import (
|
from scripts.run_portfolio_construction_matrix import (
|
||||||
|
CACHE_VERSION,
|
||||||
_assert_clean_worktree,
|
_assert_clean_worktree,
|
||||||
|
_build_candidate_cache,
|
||||||
_checkpoint_state,
|
_checkpoint_state,
|
||||||
|
_construction_candidate_view,
|
||||||
|
_construction_universe_errors,
|
||||||
|
_json_hash,
|
||||||
_load_snapshot,
|
_load_snapshot,
|
||||||
_markdown,
|
_markdown,
|
||||||
_operational_summary,
|
_operational_summary,
|
||||||
@@ -126,11 +133,18 @@ def test_load_snapshot_accepts_pre_sec_ticker_schema(tmp_path, monkeypatch):
|
|||||||
volume BIGINT NOT NULL,
|
volume BIGINT NOT NULL,
|
||||||
created_at DATETIME NOT NULL
|
created_at DATETIME NOT NULL
|
||||||
);
|
);
|
||||||
|
CREATE TABLE research_rank_only (
|
||||||
|
symbol VARCHAR(10) PRIMARY KEY
|
||||||
|
);
|
||||||
INSERT INTO tickers VALUES
|
INSERT INTO tickers VALUES
|
||||||
(1, 'LEGACY', 'Legacy Co', '2024-01-01 00:00:00');
|
(1, 'LEGACY', 'Legacy Co', '2024-01-01 00:00:00'),
|
||||||
|
(2, 'RANK', 'Rank Only Co', '2024-01-01 00:00:00');
|
||||||
INSERT INTO ohlcv_records VALUES
|
INSERT INTO ohlcv_records VALUES
|
||||||
(1, 1, '2024-01-02', 100, 102, 99, 101, 1000000,
|
(1, 1, '2024-01-02', 100, 102, 99, 101, 1000000,
|
||||||
|
'2024-01-02 00:00:00'),
|
||||||
|
(2, 2, '2024-01-02', 50, 51, 49, 50, 500000,
|
||||||
'2024-01-02 00:00:00');
|
'2024-01-02 00:00:00');
|
||||||
|
INSERT INTO research_rank_only VALUES ('RANK');
|
||||||
'''
|
'''
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -167,7 +181,8 @@ def test_load_snapshot_accepts_pre_sec_ticker_schema(tmp_path, monkeypatch):
|
|||||||
|
|
||||||
loaded = asyncio.run(_load_snapshot(snapshot, quiet=True))
|
loaded = asyncio.run(_load_snapshot(snapshot, quiet=True))
|
||||||
|
|
||||||
assert loaded['symbols'] == ['LEGACY']
|
assert loaded['symbols'] == ['LEGACY', 'RANK']
|
||||||
|
assert loaded['construction_symbols'] == {'LEGACY'}
|
||||||
assert loaded['prices']['LEGACY'] == (
|
assert loaded['prices']['LEGACY'] == (
|
||||||
[date(2024, 1, 2).toordinal()],
|
[date(2024, 1, 2).toordinal()],
|
||||||
[100.0],
|
[100.0],
|
||||||
@@ -176,6 +191,11 @@ def test_load_snapshot_accepts_pre_sec_ticker_schema(tmp_path, monkeypatch):
|
|||||||
[101.0],
|
[101.0],
|
||||||
[1_000_000],
|
[1_000_000],
|
||||||
)
|
)
|
||||||
|
assert loaded['prices']['RANK'][4] == [50.0]
|
||||||
|
assert loaded['construction_universe_manifest'][
|
||||||
|
'construction_ticker_rows'
|
||||||
|
] == 1
|
||||||
|
assert loaded['construction_universe_manifest']['rank_only_ticker_rows'] == 1
|
||||||
with sqlite3.connect(snapshot) as connection:
|
with sqlite3.connect(snapshot) as connection:
|
||||||
columns = {
|
columns = {
|
||||||
row[1] for row in connection.execute('PRAGMA table_info(tickers)')
|
row[1] for row in connection.execute('PRAGMA table_info(tickers)')
|
||||||
@@ -183,6 +203,111 @@ def test_load_snapshot_accepts_pre_sec_ticker_schema(tmp_path, monkeypatch):
|
|||||||
assert {'cik', 'sic', 'sic_description'}.isdisjoint(columns)
|
assert {'cik', 'sic', 'sic_description'}.isdisjoint(columns)
|
||||||
|
|
||||||
|
|
||||||
|
def test_construction_view_filters_rank_only_rows_without_rebuilding_cache():
|
||||||
|
manifest = {
|
||||||
|
'ranking_ticker_rows': 506,
|
||||||
|
'ranking_symbols_with_prices': 506,
|
||||||
|
'construction_ticker_rows': 505,
|
||||||
|
'construction_symbols_with_prices': 505,
|
||||||
|
'rank_only_ticker_rows': 1,
|
||||||
|
'rank_only_symbols_with_prices': 1,
|
||||||
|
'rank_only_unknown_symbols': 0,
|
||||||
|
}
|
||||||
|
cached = {
|
||||||
|
'key': {'version': 'existing-broad-cache'},
|
||||||
|
'qualified_candidates': [
|
||||||
|
{'symbol': 'PROD', 'date': '2025-01-02'},
|
||||||
|
{'symbol': 'RANK', 'date': '2025-01-02'},
|
||||||
|
],
|
||||||
|
'qualified_long_count': 2,
|
||||||
|
'daily_rank_map': {
|
||||||
|
('RANK', '2025-01-02'): {'strategy_rank': 99.0},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
view = _construction_candidate_view(
|
||||||
|
cached,
|
||||||
|
{
|
||||||
|
'construction_symbols': {'PROD'},
|
||||||
|
'construction_universe_manifest': manifest,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert [row['symbol'] for row in view['qualified_candidates']] == ['PROD']
|
||||||
|
assert view['raw_full_universe_qualified_long_count'] == 2
|
||||||
|
assert view['filtered_rank_only_qualified_long_count'] == 1
|
||||||
|
assert view['qualified_long_count'] == 1
|
||||||
|
assert ('RANK', '2025-01-02') in view['daily_rank_map']
|
||||||
|
assert len(cached['qualified_candidates']) == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_existing_broad_candidate_cache_key_remains_reusable(tmp_path, monkeypatch):
|
||||||
|
snapshot = tmp_path / 'research.sqlite'
|
||||||
|
snapshot.write_bytes(b'snapshot-placeholder')
|
||||||
|
cache_path = tmp_path / 'broad-cache.pkl'
|
||||||
|
snapshot_data = {
|
||||||
|
'recommendation_config': {'rr': 3.0},
|
||||||
|
'activation': {'min_momentum_percentile': 80.0},
|
||||||
|
'runtime_config': {'ranking_key': 'test'},
|
||||||
|
'universe_manifest': {
|
||||||
|
'ticker_rows': 4655,
|
||||||
|
'symbols_with_prices': 4654,
|
||||||
|
'symbols_sha256': 'symbols',
|
||||||
|
},
|
||||||
|
}
|
||||||
|
key = {
|
||||||
|
'version': CACHE_VERSION,
|
||||||
|
'snapshot': str(snapshot.resolve()),
|
||||||
|
'snapshot_sha256': 'snapshot-hash',
|
||||||
|
'cadence': 'daily',
|
||||||
|
'outcome_horizon_sessions': 0,
|
||||||
|
'recommendation_config_hash': _json_hash(
|
||||||
|
snapshot_data['recommendation_config']
|
||||||
|
),
|
||||||
|
'activation_hash': _json_hash(snapshot_data['activation']),
|
||||||
|
'runtime_config': snapshot_data['runtime_config'],
|
||||||
|
'universe_manifest': snapshot_data['universe_manifest'],
|
||||||
|
}
|
||||||
|
cached = {'key': key, 'qualified_candidates': [{'symbol': 'PROD'}]}
|
||||||
|
cache_path.write_bytes(pickle.dumps(cached))
|
||||||
|
monkeypatch.setattr(
|
||||||
|
bt,
|
||||||
|
'_replay_candidates_for_period',
|
||||||
|
lambda *_args: pytest.fail('existing cache should avoid replay'),
|
||||||
|
)
|
||||||
|
|
||||||
|
loaded = _build_candidate_cache(
|
||||||
|
snapshot_data,
|
||||||
|
snapshot=snapshot,
|
||||||
|
snapshot_sha256='snapshot-hash',
|
||||||
|
cache_path=cache_path,
|
||||||
|
workers=1,
|
||||||
|
quiet=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert loaded == cached
|
||||||
|
|
||||||
|
|
||||||
|
def test_construction_universe_guard_rejects_leaked_broad_book():
|
||||||
|
valid = {
|
||||||
|
'ranking_ticker_rows': 4655,
|
||||||
|
'construction_ticker_rows': 506,
|
||||||
|
'construction_symbols_with_prices': 506,
|
||||||
|
'rank_only_ticker_rows': 4149,
|
||||||
|
'rank_only_unknown_symbols': 0,
|
||||||
|
}
|
||||||
|
assert _construction_universe_errors(valid) == []
|
||||||
|
|
||||||
|
leaked = {
|
||||||
|
**valid,
|
||||||
|
'construction_ticker_rows': 4655,
|
||||||
|
'construction_symbols_with_prices': 4654,
|
||||||
|
'rank_only_ticker_rows': 0,
|
||||||
|
}
|
||||||
|
errors = _construction_universe_errors(leaked)
|
||||||
|
assert any('450-600' in error for error in errors)
|
||||||
|
|
||||||
|
|
||||||
def test_unbounded_count_and_effective_risk_floor():
|
def test_unbounded_count_and_effective_risk_floor():
|
||||||
start = date(2025, 1, 6)
|
start = date(2025, 1, 6)
|
||||||
ords = [start.toordinal() + offset for offset in range(4)]
|
ords = [start.toordinal() + offset for offset in range(4)]
|
||||||
@@ -504,6 +629,10 @@ def test_simple_cluster_bootstrap_is_deterministic_and_not_a_gate():
|
|||||||
assert first['p05'] <= first['point'] <= first['p95']
|
assert first['p05'] <= first['point'] <= first['p95']
|
||||||
|
|
||||||
|
|
||||||
|
def test_iqr_materializes_generator_before_both_quantiles():
|
||||||
|
assert iqr(value for value in (0.0, 1.0, 2.0, 3.0)) == pytest.approx(1.5)
|
||||||
|
|
||||||
|
|
||||||
def test_aggregate_reports_paired_years_and_separate_warm_iqrs():
|
def test_aggregate_reports_paired_years_and_separate_warm_iqrs():
|
||||||
cells: list[dict] = []
|
cells: list[dict] = []
|
||||||
for cost in (0.1, 0.2):
|
for cost in (0.1, 0.2):
|
||||||
@@ -579,16 +708,40 @@ def test_aggregate_reports_paired_years_and_separate_warm_iqrs():
|
|||||||
)
|
)
|
||||||
assert set(cash_warm['headline']) == {'ev_net_r', 'calmar'}
|
assert set(cash_warm['headline']) == {'ev_net_r', 'calmar'}
|
||||||
assert 'D' not in cash_warm
|
assert 'D' not in cash_warm
|
||||||
|
assert cash_warm['headline']['ev_net_r']['median_iqr_ratio'] == pytest.approx(
|
||||||
|
1.0
|
||||||
|
)
|
||||||
|
assert cash_warm['headline']['calmar']['median_iqr_ratio'] == pytest.approx(
|
||||||
|
1.0
|
||||||
|
)
|
||||||
|
assert cash_warm['headline']['ev_net_r']['bootstrap_90']['n'] == 7
|
||||||
markdown = _markdown({
|
markdown = _markdown({
|
||||||
'generated_at': '2026-08-05T00:00:00Z',
|
'generated_at': '2026-08-05T00:00:00Z',
|
||||||
'analysis': report,
|
'analysis': report,
|
||||||
'operational_summary': _operational_summary(cells),
|
'operational_summary': _operational_summary(cells),
|
||||||
|
'validation': {
|
||||||
|
'construction_universe_manifest': {
|
||||||
|
'construction_symbols_with_prices': 506,
|
||||||
|
'rank_only_symbols_with_prices': 4148,
|
||||||
|
'ranking_symbols_with_prices': 4654,
|
||||||
|
},
|
||||||
|
'candidate_rank_coverage': {
|
||||||
|
'construction_qualified_longs': 5000,
|
||||||
|
'filtered_rank_only_qualified_longs': 137000,
|
||||||
|
},
|
||||||
|
},
|
||||||
})
|
})
|
||||||
assert 'ΔGain-to-Pain' in markdown
|
assert 'ΔGain-to-Pain' in markdown
|
||||||
|
assert '0.10% per fill' in markdown
|
||||||
|
assert '0.20% per fill' in markdown
|
||||||
|
assert 'Tradable setup symbols with prices: 506.' in markdown
|
||||||
|
assert 'Rank-only qualified rows removed: 137000.' in markdown
|
||||||
assert 'formal promotion gate' in markdown
|
assert 'formal promotion gate' in markdown
|
||||||
|
|
||||||
|
|
||||||
def test_synthetic_worker_matrix_covers_four_arms_protocols_and_costs():
|
def test_synthetic_worker_matrix_covers_four_arms_protocols_and_costs(monkeypatch):
|
||||||
|
monkeypatch.setenv('BACKTEST_SNAPSHOT_OFFLINE', '0')
|
||||||
|
monkeypatch.setenv('BACKTEST_ALLOW_SPAWN', '0')
|
||||||
start = date(2025, 1, 6)
|
start = date(2025, 1, 6)
|
||||||
sessions = _business_days(start, date(2025, 1, 17))
|
sessions = _business_days(start, date(2025, 1, 17))
|
||||||
ords = [session.toordinal() for session in sessions]
|
ords = [session.toordinal() for session in sessions]
|
||||||
|
|||||||
Reference in New Issue
Block a user