dennisthiessenandClaude Opus 5
f5d4b516ab
Brings the durable artifacts of research/portfolio-capacity-rebalancing onto main so the rationale for raising the count cap lives with the code that cites it. The matrix runner, the research simulator hooks and the study's unit tests are deliberately left behind; they remain at tag research/portfolio-capacity-final. Corrects conclusions that were reached on EV per trade and are now superseded: the findings doc's decisions 1 (keep cap 10) and 4 (run the risk-floor A/B) are struck through and answered in a new correction section, and the research README and phase-A matrix entries are updated to match. The frozen specification itself is untouched -- its recorded SHA-256 f1e37783 still verifies. effective-risk-floor-ab.md is retained but marked CLOSED/NEGATIVE: the study it proposes is already answered by cap15 vs cash_unbounded (-0.753pp CAGR while EV/trade rises), and its EV-based pass rule would have shipped it. scripts/research_rankings.py replaces a fourth copy of the historical rank-map helper; run_research_matrix, run_execution_recovery_matrix and run_daily_reentry_matrix now share it. The shared version adds a duplicate observation guard and a deterministic symbol tie-break the copies lacked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
170 lines
8.2 KiB
Markdown
170 lines
8.2 KiB
Markdown
# Portfolio-capacity bracket — frozen specification
|
||
|
||
Date frozen: 2026-08-05
|
||
Branch: research/portfolio-capacity-rebalancing
|
||
Runner: scripts/run_portfolio_construction_matrix.py
|
||
|
||
## Question and motivation
|
||
|
||
The daily Phase A production control (a0_control: close fill, 30-session
|
||
maximum hold, 1% fixed-fractional risk, no correlation or volatility overlay)
|
||
recorded 472 trades and 519 otherwise qualified entries rejected because the
|
||
ten-position book was full. The blocked share is 519 / (519 + 472) = 52.4%.
|
||
The book is therefore materially arrival-order constrained.
|
||
|
||
This supersedes the older statement that the ten-slot cap never bound. That
|
||
statement came from a shorter, weekly, pre-gate-reset replay and is not evidence
|
||
about the current daily strategy.
|
||
|
||
The study brackets the value of capacity before tuning replacement details. It
|
||
does not contain a formal promotion rule or automatically change production.
|
||
Because the current ~505-name production membership is projected backward,
|
||
paired arm-versus-control differences are the primary evidence. Absolute
|
||
profitability is descriptive and survivorship-biased.
|
||
|
||
Implementation correction: the first completed v1 artifact at commit `23fe39f`
|
||
incorrectly allowed the snapshot's broad rank-only universe to submit trades.
|
||
That artifact is invalid, is removed from the branch, and must not be used for
|
||
strategy conclusions. Runner v2 fixes the construction/ranking partition below.
|
||
|
||
## Frozen arms
|
||
|
||
1. **cap10_incumbent:** exact production-style cap-10 control, no displacement.
|
||
2. **cash_unbounded:** no position-count cap; cash/no leverage and the existing
|
||
20% per-position notional ceiling remain. Reject an entry if actual initial
|
||
stop-risk after cash/notional sizing is below 0.5% of marked equity.
|
||
3. **cap10_weekly_top10:** on the final trading session of each ISO week, rank
|
||
holdings plus fresh same-day qualified entrants and retain the top ten.
|
||
4. **cap15_incumbent:** cap 15, no displacement.
|
||
|
||
All arms use the frozen Phase A control configuration: daily candidate replay,
|
||
live-like full-universe residual-momentum/low-volatility 80/20 rank, activation
|
||
threshold 80, normal gate-reset re-entry, close fill, 3×ATR trail, 30-session
|
||
maximum hold, 1% risk, and costs of 0.10% and 0.20% per fill.
|
||
|
||
Every priced symbol contributes to the daily cross-sectional rank. Only symbols
|
||
not listed in the snapshot's `research_rank_only` side table may submit trade
|
||
setups to any arm. The resulting construction universe must contain 450-600
|
||
symbols (expected approximately 505); validation fails outside that frozen
|
||
guardrail or when the side table references unknown ticker symbols.
|
||
|
||
The daily replay uses zero outcome horizon: setup and rank observations continue
|
||
through the snapshot's last session because portfolio simulation, unlike outcome
|
||
grading, does not require 30 future bars.
|
||
|
||
Control-parity note: a direct main-versus-branch comparison found identical
|
||
total return, CAGR, maximum drawdown, and Sharpe. The branch intentionally
|
||
changes only the first calendar year's `yearly_returns` convention: it starts
|
||
from initial capital rather than equity after the first session, so day-one
|
||
entry costs are now charged to year one. Older reports can therefore show a
|
||
different first-year contextual return without a strategy-performance
|
||
regression. New trade-detail and measurement-start fields are additive.
|
||
|
||
### Weekly-selection mechanics
|
||
|
||
- Ordinary exits run before entries/rebalancing.
|
||
- Open slots may still fill from daily qualified entries during the week.
|
||
- On the final ISO-week session, current holdings and that day's fresh qualified
|
||
entrants use the full-universe strategy_rank for that same date.
|
||
- Stored entry-day rank is never used.
|
||
- Holdings with missing current rank/data are protected and consume a slot;
|
||
entrants missing rank are ineligible.
|
||
- Incumbents win exact rank ties; symbol is the deterministic final tie-breaker.
|
||
- Rebalance exits pay costs and bypass cooldown/post-stop state.
|
||
- Report entrant-pool sizes, replacements, turnover, and same-symbol re-entry
|
||
within 5/10/20 sessions.
|
||
|
||
## Frozen cohorts
|
||
|
||
research.sqlite is expected to cover 2016-01-04 through 2026-07-17. Residual
|
||
momentum requires 252 benchmark sessions. Empty-book starts additionally require
|
||
504 prior scoring sessions and 252 forward measurement sessions.
|
||
|
||
- **Empty book:** first eligible session of each month, approximately January
|
||
2019 through July 2025; start with no positions and measure 252 sessions.
|
||
- **Warm book:** first session of each year 2019–2025 is the measurement anchor.
|
||
Seed the portfolio on the first session of every ISO week falling 63–126
|
||
trading sessions before the anchor, carry all positions and gate-reset state
|
||
forward, and measure the same 252-session anchor window.
|
||
|
||
Warm portfolio returns reset to marked equity immediately before the anchor
|
||
session. P&L after the anchor from carried positions belongs to portfolio
|
||
returns, while trade EV includes only entries on or after the anchor. Remaining
|
||
positions liquidate at the last measurement close with costs.
|
||
|
||
The validate-only mode must print realized cohort counts and fail unless both
|
||
protocols contain the seven annual clusters 2019–2025 and every warm anchor has
|
||
at least 12 seeds. It must also print ranking, rank-only, and tradable symbol
|
||
counts plus the raw, removed, and retained qualified-long counts.
|
||
|
||
## Reporting
|
||
|
||
Primary reported measures:
|
||
|
||
- net EV per trade in R, with costs and actual initial stop-risk dollars;
|
||
- Calmar (CAGR / max drawdown);
|
||
- profit factor on net trade R;
|
||
- Gain-to-Pain (sum of all monthly returns / absolute sum of negative months);
|
||
- Sortino using daily returns and zero target.
|
||
|
||
Also report total return/CAGR, maximum drawdown, Sharpe, win rate, time
|
||
underwater, exposure, cash, average/peak positions, sessions at capacity,
|
||
turnover, costs, qualified/admitted/blocked opportunities, and minimum-risk
|
||
rejections.
|
||
|
||
For each arm/protocol/cost/metric, pair identical paths with cap10_incumbent,
|
||
take the median paired delta within each start year or annual anchor, show all
|
||
seven cluster values, and headline their median.
|
||
|
||
Initialization dispersion is reported separately for EV and Calmar: calculate
|
||
the seed-path IQR within each warm anchor, divide by the paired control IQR, show
|
||
all seven ratios, and headline their median. Do not combine them into a composite.
|
||
|
||
For context only, run a deterministic 10,000-replicate cluster bootstrap over
|
||
the seven paired annual summaries and report the central 90% percentile interval
|
||
for median EV and Calmar deltas and warm IQR ratios. These intervals are not
|
||
promotion gates, independent-population confidence claims, or formal inference.
|
||
|
||
## Reproducibility and execution
|
||
|
||
Candidate replay/ranks cache under reports/.cache; each matrix cell checkpoints
|
||
atomically and resume verifies a fingerprint over the implementation commit,
|
||
this specification hash, snapshot SHA-256, cache key, arm definitions, costs,
|
||
and cohort manifest. An authoritative run refuses a dirty worktree.
|
||
|
||
The existing v1 candidate/rank cache is intentionally reusable: its
|
||
full-universe current-day ranks are correct. Runner v2 derives a fingerprinted
|
||
construction view by removing qualified rows whose symbols are rank-only. V2
|
||
uses a versioned checkpoint directory, so invalid v1 portfolio cells are never
|
||
resumed and the expensive daily rank replay does not need to run again.
|
||
|
||
The loader reads only ticker ID/symbol and the OHLCV columns used by replay, so
|
||
snapshots created before SEC metadata added `tickers.cik`, `tickers.sic`, and
|
||
`tickers.sic_description` remain valid. Do not migrate or alter the research
|
||
snapshot: its original SHA-256 is part of the run fingerprint.
|
||
|
||
macOS environment setup from the repository root (zsh):
|
||
|
||
python3 -m venv .venv
|
||
./.venv/bin/python -m pip install -e '.[dev]'
|
||
|
||
Preflight:
|
||
|
||
./.venv/bin/python scripts/run_portfolio_construction_matrix.py \
|
||
backtest_snapshots/research.sqlite \
|
||
--run-id prod505-capacity-bracket-daily-v1 \
|
||
--workers auto \
|
||
--resume \
|
||
--validate-only
|
||
|
||
Authoritative run:
|
||
|
||
./.venv/bin/python scripts/run_portfolio_construction_matrix.py \
|
||
backtest_snapshots/research.sqlite \
|
||
--run-id prod505-capacity-bracket-daily-v1 \
|
||
--workers auto \
|
||
--resume
|
||
|
||
Commit only the compact final JSON and Markdown reports. Raw curves, trades,
|
||
candidate caches, and checkpoints remain ignored.
|