Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f5d4b516ab | ||
|
|
3ff0fd9f1c |
@@ -1701,7 +1701,14 @@ def _gate_ablation(candidates: list[dict], activation: dict, threshold: float) -
|
||||
# the QUALIFIED setups at their detection close, best momentum first while
|
||||
# slots and cash allow.
|
||||
SIM_STARTING_CAPITAL = 10_000.0
|
||||
SIM_MAX_POSITIONS = 10
|
||||
# Headroom, not a target: the count cap should never bind. The capacity study
|
||||
# (reports/portfolio-construction-prod505-capacity-bracket-daily-v1) showed a book
|
||||
# that never hits the count cap earns +1.1pp CAGR over the old 10 (51 cohorts of 175
|
||||
# better, 2 worse) at unchanged drawdown, because the blocked entries were as good as
|
||||
# the taken ones — capacity costs trade COUNT, not trade quality. The real ceiling is
|
||||
# cash plus SIM_NOTIONAL_CAP, which saturates the book near 12 positions, so 15/20/None
|
||||
# are the same experiment. Judge any future change here on CAGR, never on EV per trade.
|
||||
SIM_MAX_POSITIONS = 15
|
||||
SIM_RISK_PER_TRADE = 0.01 # fraction of equity risked per position (entry→stop)
|
||||
SIM_NOTIONAL_CAP = 0.20 # max fraction of equity per position (no margin)
|
||||
_EULER_MASCHERONI = 0.5772156649015329
|
||||
|
||||
@@ -40,10 +40,12 @@ KEY_CAPACITY = "shadow_book_capacity"
|
||||
KEY_RISK_PCT = "shadow_book_risk_pct"
|
||||
KEY_START_EQUITY = "shadow_book_start_equity"
|
||||
|
||||
# Matches the validated configuration: 10-position book, 1% fixed-fractional
|
||||
# risk. Start equity is only a sizing base — comparisons are drawn in percent
|
||||
# and R-multiples, never in raw currency.
|
||||
DEFAULT_CAPACITY = 10
|
||||
# Matches the validated configuration: 1% fixed-fractional risk, and a count cap
|
||||
# set as headroom rather than a target — see backtest_service.SIM_MAX_POSITIONS,
|
||||
# which this must track. NOTIONAL_CAP below saturates the book near 12 positions,
|
||||
# so the count cap should simply never bind. Start equity is only a sizing base —
|
||||
# comparisons are drawn in percent and R-multiples, never in raw currency.
|
||||
DEFAULT_CAPACITY = 15
|
||||
DEFAULT_RISK_PCT = 1.0
|
||||
DEFAULT_START_EQUITY = 100_000.0
|
||||
|
||||
|
||||
+18
-2
@@ -25,7 +25,7 @@ score, Structural S/R, the Gate Target Ladder, sentiment, fundamentals) is
|
||||
| 1.5× ATR initial stop | Real exit | Cuts losers fast |
|
||||
| 3× ATR trailing stop, 30-day max hold | Real exit | Best Sharpe of every exit tested |
|
||||
| Post-stop normal gate reset | Re-entry policy | Stop always closes; a later gate failure and subsequent fresh qualification define the next signal episode. The selected study arm reached Sharpe 1.77 / CAGR 48.3% at capacity 10; live scan-before-outcome timing is stricter (Sharpe 1.68 / CAGR 44.8% analogue). [Full study](post-stop-reentry.md) |
|
||||
| Max 10 concurrent positions, 1% risk per trade | Sizing | Cap never binds in practice |
|
||||
| Max **15** concurrent positions, 1% risk per trade | Sizing | Raised from 10 (2026-08-05) so the count cap never binds: +1.075pp CAGR paired, 51 paths better / 2 worse, drawdown unchanged. Cash plus the 20% notional cap saturates the book near 12. [Findings](portfolio-capacity-bracket-findings.md#correction-2026-08-05-ev-per-trade-was-the-wrong-lens) |
|
||||
| Structural S/R | Human-facing product context | Clean, capped zones for charts and alerts; not read by the scanner |
|
||||
| Gate Target Ladder | Screening machinery | Volume-free transient proposals preserve the production candidate set exactly; never an exit |
|
||||
|
||||
@@ -61,7 +61,7 @@ invites overfitting.
|
||||
|---|---|
|
||||
| ATR trail multiple {1.5–4.0} | **Keep 3.0** — ≤2.0 whipsaws out the right tail; ≥2.5 is a plateau |
|
||||
| Momentum lookback (6-1, 3-1, 12-7 Novy-Marx, composites) | **Keep residual 12-1** — the others have IC ≈ 0 or weaker t-stats |
|
||||
| Selection cutoff {70…90} × book size {10, 15, 20} | **Keep 80 × 10** — monotonically worse in both directions |
|
||||
| Selection cutoff {70…90} × book size {10, 15, 20} | **Keep cutoff 80; book size now 15** — the focused daily bracket found cap 15 worth +1.075pp CAGR (the weekly replay's contrary reading was EV-per-trade). Weekly rank replacement hurt. [Findings](portfolio-capacity-bracket-findings.md#correction-2026-08-05-ev-per-trade-was-the-wrong-lens) |
|
||||
| Position sizing (equal-weight, inverse-vol, risk-% sweep) | **Keep 1% fixed-fractional** |
|
||||
| Primary-target probability floor | **Keep 20%** — pruned lottery targets, 1,428 → 1,089 qualified, lifted Sharpe |
|
||||
| Primary-target R:R selector | **Keep 1.5** — target choice is intentionally independent of the later 2.0 activation floor |
|
||||
@@ -146,6 +146,7 @@ knobs.
|
||||
| **Broader universe** | Composition changes factor signs (fip tug-of-war); vol-tilt on breadth is only a **directional hypothesis** (auth. −0.048 / t −1.36) | Any prod broaden must re-validate 80/20 tilt; offline research only; research.sqlite requires completion manifest |
|
||||
| **Forward paper-trade record** | The only true out-of-sample evidence the snapshot cannot give | Time; mark entries at actual near-close fill once ops ships |
|
||||
| **Better target model for clear-air names** | The return is demonstrably there (#2 wins on raw CAGR in *both* train and test); it's the *flat* 3× ATR target that makes it too expensive in risk | Needs a per-name model, not a constant k×ATR |
|
||||
| **Minimum effective-risk floor** | ⛔ CLOSED NEGATIVE, not run. The floor lifts EV/trade (+0.032) and PF (+0.073) *by deleting trades* — 11.4 fewer per path, never one more — and costs **−0.753pp CAGR**, −0.047 Sharpe, −0.051 Calmar | Do not run the A/B; its EV-based pass rule would have shipped it. [Withdrawn specification](effective-risk-floor-ab.md) / [findings](portfolio-capacity-bracket-findings.md#correction-2026-08-05-ev-per-trade-was-the-wrong-lens) |
|
||||
|
||||
---
|
||||
|
||||
@@ -197,4 +198,19 @@ qualification. The [daily re-entry matrix](post-stop-reentry.md) supports this
|
||||
for the current 10-position book, but not as a universal rule for other
|
||||
portfolio capacities.
|
||||
|
||||
Capacity is closed **positive**: the count cap was raised 10 → 15 so it no longer
|
||||
binds, worth **+1.075pp CAGR** paired across 175 paths (51 better, 2 worse) at
|
||||
unchanged drawdown. Fifteen is headroom, not a target — cap15 peaked at 12 with
|
||||
zero full-book skips, so cash plus the 20% notional cap is the real ceiling.
|
||||
|
||||
An earlier reading of this run concluded "keep cap 10, added only 0.0018 R/trade."
|
||||
That was **EV per trade**, which is the wrong metric for a treatment that changes
|
||||
trade *count*: flat EV/trade means the blocked entries were as good as the taken
|
||||
ones, so refusing them cost their whole contribution to return. Weekly
|
||||
current-rank replacement remains rejected (−0.043 EV R, 24% churn). The 0.5%
|
||||
effective-risk-floor A/B is **closed negative** without being run — it costs
|
||||
0.75pp of CAGR while raising EV/trade, and its frozen pass rule would have shipped
|
||||
it. See the [frozen specification](portfolio-capacity-bracket.md) and the
|
||||
[capacity findings](portfolio-capacity-bracket-findings.md#correction-2026-08-05-ev-per-trade-was-the-wrong-lens).
|
||||
|
||||
The next real evidence is **forward**, not backward: the live paper-trade record.
|
||||
|
||||
@@ -0,0 +1,143 @@
|
||||
# Effective initial-risk floor A/B - frozen specification
|
||||
|
||||
> ## ⛔ CLOSED 2026-08-05 — NEGATIVE. DO NOT RUN.
|
||||
>
|
||||
> This A/B was never executed because the capacity-bracket run already contains
|
||||
> it. `cap15_incumbent` (peak 12, zero blocked, no floor) and `cash_unbounded`
|
||||
> (peak 12, floor) have the same effective capacity and differ essentially only
|
||||
> by `min_initial_risk_fraction`. Paired over 175 paths, the 0.5% floor gives
|
||||
> **EV/trade +0.032 and profit factor +0.073, but CAGR −0.753pp, total return
|
||||
> −0.765pp, Sharpe −0.047, Calmar −0.051**, and it removes 11.4 trades per path
|
||||
> while never adding one (174 worse / 0 better).
|
||||
>
|
||||
> **The pass rule below is unsafe.** It promotes on paired EV, and the floor
|
||||
> raises EV per trade *precisely by deleting trades* that were net positive
|
||||
> contributors — so this specification would have shipped a change costing
|
||||
> 0.75pp of CAGR. Any successor study must decide on CAGR/total return and treat
|
||||
> EV per trade as a diagnostic.
|
||||
>
|
||||
> See [portfolio-capacity-bracket-findings.md](portfolio-capacity-bracket-findings.md#correction-2026-08-05-ev-per-trade-was-the-wrong-lens).
|
||||
> Retained as a record of what was specified and why it was withdrawn.
|
||||
|
||||
Date frozen: 2026-08-05
|
||||
Branch: research/portfolio-capacity-rebalancing (deleted; tag `research/portfolio-capacity-final`)
|
||||
Runner: scripts/run_portfolio_construction_matrix.py (not on main; see tag)
|
||||
Study ID: risk-floor-ab
|
||||
|
||||
## Question
|
||||
|
||||
Does rejecting an otherwise qualified cap-10 entry when its actual initial
|
||||
stop-risk after cash and notional sizing is below 0.5% of marked equity improve
|
||||
trade selection?
|
||||
|
||||
The completed capacity bracket cannot answer this. Its cash_unbounded arm
|
||||
removed the count cap and applied the 0.5% floor simultaneously. In the 70 paths
|
||||
where the control cap never bound, that arm still raised mean EV from 0.328 to
|
||||
0.399 R and profit factor from 1.60 to 1.75 while trades fell about 8% and
|
||||
exposure stayed nearly flat. Capacity was a no-op in those paths, so the floor
|
||||
is the plausible cause, but the prior arm remains confounded.
|
||||
|
||||
This A/B changes only the floor. It has no formal promotion gate and does not
|
||||
automatically change production.
|
||||
|
||||
## Frozen arms
|
||||
|
||||
1. cap10_incumbent: current production-style cap-10 control, with no minimum
|
||||
effective-risk floor.
|
||||
2. cap10_min_risk_005: the same cap-10 strategy, rejecting an entry only when
|
||||
actual initial stop-risk after cash/notional sizing is below 0.5% of marked
|
||||
equity.
|
||||
|
||||
Both arms have max_positions=10, weekly replacement disabled, 1% target risk
|
||||
per trade, and identical admission ordering. The only differing simulator
|
||||
argument is min_initial_risk_fraction: None versus 0.005.
|
||||
|
||||
All other settings remain the frozen daily Phase A control: current production
|
||||
construction universe, full-universe residual-momentum/low-volatility 80/20
|
||||
rank, threshold 80, normal gate-reset re-entry, close fills, 3x ATR trail,
|
||||
30-session maximum hold, 20% per-position notional ceiling, no leverage, and
|
||||
costs of 0.10% and 0.20% per fill.
|
||||
|
||||
Every priced symbol contributes to the daily cross-sectional rank. Rank-only
|
||||
symbols cannot submit trades. Validation retains the 450-600-symbol production
|
||||
construction guardrail and the legacy-snapshot column-scoped loader.
|
||||
|
||||
## Frozen cohorts
|
||||
|
||||
Reuse the completed bracket's point-in-time daily candidate/rank cache and
|
||||
cohort manifest:
|
||||
|
||||
- Empty book: first eligible session of each month in 2019-2025, with 504 prior
|
||||
scoring sessions and 252 measurement sessions. This is the primary start-date
|
||||
evidence.
|
||||
- Warm book: weekly seeds 63-126 sessions before each 2019-2025 annual anchor,
|
||||
with state carried into the same 252-session measurement window. This is a
|
||||
state-carrying replication, not independent evidence.
|
||||
|
||||
The expected realization is 78 empty-book paths, 97 warm paths, seven annual
|
||||
clusters in each protocol, two costs, two arms, and 700 cells.
|
||||
|
||||
Do not use warm-seed IQR as evidence. Six of seven completed-bracket anchors
|
||||
were structurally degenerate because fractional sizing is scale invariant and
|
||||
the 30-session maximum hold washed out books before anchors. The 2023 exception
|
||||
shows that state carrying itself works.
|
||||
|
||||
## Reporting and interpretation
|
||||
|
||||
For every protocol and cost, pair identical paths. Report:
|
||||
|
||||
- mean, median, P25, and P75 paired net-EV changes in R;
|
||||
- positive-path and bit-identical-path fractions;
|
||||
- the median paired delta within each year and the median across seven years;
|
||||
- simple 90% cluster-bootstrap context for EV and Calmar, with no CI gate;
|
||||
- mean paired PF, Gain-to-Pain, Sortino, Calmar/MAR, CAGR, maximum drawdown,
|
||||
total return, and Sharpe changes;
|
||||
- trades, floor rejections, holding time, cash, gross exposure, average/peak
|
||||
positions, turnover, and costs.
|
||||
|
||||
Means and identical-path fractions must appear beside medians so inert cohorts
|
||||
cannot turn a left- or right-skewed treatment into a misleading zero headline.
|
||||
For these 252-session windows, the implementation's full-window Calmar is CAGR
|
||||
divided by maximum drawdown, the same numeric definition commonly called MAR;
|
||||
do not present the duplicate label as a second independent metric.
|
||||
|
||||
Today's production membership is projected backward. Use paired differences
|
||||
for the treatment conclusion; absolute profitability remains descriptive and
|
||||
survivorship-biased. Empty and warm protocols cover the same seven market years
|
||||
and must not be interpreted as independent replications.
|
||||
|
||||
Interpretation is deliberately simple:
|
||||
|
||||
- a positive result means the isolated floor improves the paired EV
|
||||
distribution without an economically important loss of total-return or
|
||||
drawdown quality;
|
||||
- a negative result closes the floor;
|
||||
- mixed EV/portfolio-quality results are reported as a trade-off, not forced
|
||||
through a composite score.
|
||||
|
||||
## Reproducibility and macOS execution
|
||||
|
||||
The authoritative run refuses a dirty worktree. Its fingerprint includes the
|
||||
implementation commit, this specification hash, snapshot hash, candidate-cache
|
||||
key, construction view, cohort manifest, arm definitions, costs, and study
|
||||
version. Cells checkpoint atomically and --resume verifies the fingerprint.
|
||||
|
||||
From the repository root on macOS:
|
||||
|
||||
python3 -m venv .venv
|
||||
./.venv/bin/python -m pip install -e '.[dev]'
|
||||
|
||||
Preflight, reusing the completed bracket's candidate/rank cache:
|
||||
|
||||
./.venv/bin/python scripts/run_portfolio_construction_matrix.py + backtest_snapshots/research.sqlite + --study risk-floor-ab + --run-id prod505-effective-risk-floor-ab-daily-v1 + --candidate-cache reports/.cache/prod505-capacity-bracket-daily-v1-candidates.pkl + --workers 8 + --resume + --validate-only
|
||||
|
||||
Authoritative run:
|
||||
|
||||
./.venv/bin/python scripts/run_portfolio_construction_matrix.py + backtest_snapshots/research.sqlite + --study risk-floor-ab + --run-id prod505-effective-risk-floor-ab-daily-v1 + --candidate-cache reports/.cache/prod505-capacity-bracket-daily-v1-candidates.pkl + --workers 8 + --resume
|
||||
|
||||
On an M2 Pro, eight workers is the explicit high-utilization setting. Use six
|
||||
instead on a memory-constrained machine; auto intentionally caps itself at six.
|
||||
Changing worker count does not change the fingerprint or results.
|
||||
|
||||
Commit only the compact final JSON and Markdown reports. Candidate caches,
|
||||
checkpoints, raw curves, and trade ledgers remain ignored.
|
||||
@@ -28,6 +28,18 @@ Mechanics guards confirmed before reading results: calendar truncation asserted
|
||||
| **Validation** | **1.68** | **0.72** | **41.6%** | **20.9%** | **1.99** | **239** |
|
||||
| Full (close-fill) | 1.77 | 0.50 | 48.3% | 21.6% | 2.23 | 472 |
|
||||
|
||||
**Capacity correction (2026-08-05):** the full close-fill control also records
|
||||
skipped_book_full = 519 versus 472 admitted trades, so the ten-slot book
|
||||
refuses 52.4% of admitted+blocked qualified opportunities. The older weekly
|
||||
claim that the cap never bound is stale and does not apply to this daily
|
||||
gate-reset configuration. Capacity was isolated in the
|
||||
[focused bracket study](portfolio-capacity-bracket.md) and **resolved: the count
|
||||
cap was raised 10 → 15 so it no longer binds (+1.075pp CAGR paired, 51 paths
|
||||
better / 2 worse, drawdown unchanged).** Note that the blocked *count* was a poor
|
||||
guide in both directions — one path had 244 blocked entries and relieving all of
|
||||
them moved CAGR by −0.1pp. See the
|
||||
[findings correction](portfolio-capacity-bracket-findings.md#correction-2026-08-05-ev-per-trade-was-the-wrong-lens).
|
||||
|
||||
Validation SE ≈ 0.72 — almost no arm clears a 1-SE delta.
|
||||
|
||||
---
|
||||
|
||||
@@ -0,0 +1,219 @@
|
||||
# Portfolio-capacity bracket — findings
|
||||
|
||||
Date interpreted: 2026-08-05
|
||||
|
||||
Status: **SUPERSEDED IN PART — see [Correction](#correction-2026-08-05-ev-per-trade-was-the-wrong-lens)
|
||||
at the foot of this document before acting on anything here.** Weekly replacement
|
||||
is closed as a negative result and that still holds. The capacity decision below
|
||||
("keep cap 10") and the recommendation to run the effective-risk-floor A/B were
|
||||
both reached on EV per trade and are **reversed** by the correction: the count cap
|
||||
was raised so it no longer binds, and the floor A/B is closed as negative.
|
||||
|
||||
> The runner (`scripts/run_portfolio_construction_matrix.py`), the research
|
||||
> simulator hooks, and the study's unit tests were deliberately not merged to
|
||||
> main. They live at tag `research/portfolio-capacity-final`.
|
||||
|
||||
This document interprets the frozen v2 run without modifying its generated
|
||||
outputs:
|
||||
|
||||
- result commit: `24482c6`;
|
||||
- simulation source commit: `6fc82ae8574de9104c83273e018391e75a5f8ac6`;
|
||||
- frozen specification SHA-256:
|
||||
`f1e37783cf6d157ecc827d48211fa45da16f0a0ac19cd23686b3902d347a1898`;
|
||||
- JSON SHA-256:
|
||||
`2435875667097db7416a0d96f412db81d2f2d09ba053748c9f2cfb8a0cba4417`;
|
||||
- Markdown SHA-256:
|
||||
`dc3f5de25eb0a156ce51d0025c90e04ac0977e9502dec47bcf1b25bdcf609c81`.
|
||||
|
||||
The run completed 78 empty-book paths, 97 warm-seed paths, seven annual
|
||||
clusters under both protocols, two cost levels, four arms, and 1,400 cells with
|
||||
no validation errors. The construction universe was 505 priced tradable
|
||||
symbols plus 4,149 priced rank-only symbols.
|
||||
|
||||
## Capacity is economically free
|
||||
|
||||
The clean capacity treatment is `cap15_incumbent`: it changes no sizing or
|
||||
admission rule. Its cap never bound in any cell (maximum observed position count
|
||||
12; zero full-book skips), so it absorbed every opportunity blocked by cap 10.
|
||||
|
||||
At 0.10% per fill, split the 175 paths by whether the paired control recorded
|
||||
any `skipped_book_full`. Values below are mean paired changes in net EV per
|
||||
trade, in R:
|
||||
|
||||
| Arm | Cap never bound (n=70) | Cap did bind (n=105) |
|
||||
|---|---:|---:|
|
||||
| `cap15_incumbent` | +0.0000 | +0.0018 |
|
||||
| `cash_unbounded` | +0.0714 | +0.0077 |
|
||||
| `cap10_weekly_top10` | -0.0246 | -0.0426 |
|
||||
|
||||
The exact zero for cap15 in the never-bound stratum is also a harness validity
|
||||
check: when the treatment cannot act, results are identical. Where it does act,
|
||||
giving the strategy every slot it requested adds only 0.0018 R/trade. The old
|
||||
519-blocked-versus-472-admitted count was true, but it did not imply that the
|
||||
blocked opportunities were economically valuable.
|
||||
|
||||
Decision: **keep the production cap at 10.** Do not remove it or raise it in the
|
||||
expectation of additional edge.
|
||||
|
||||
## The positive arm measured the risk floor
|
||||
|
||||
`cash_unbounded` combined two treatments: no count cap and a 0.5% minimum
|
||||
effective initial-risk fraction. Its EV effect is roughly nine times larger in
|
||||
the 70 paths where the control cap never bound, so capacity cannot explain the
|
||||
improvement.
|
||||
|
||||
Within that never-bound stratum:
|
||||
|
||||
| Measure | Control | `cash_unbounded` |
|
||||
|---|---:|---:|
|
||||
| Mean trades | 75.7 | 69.9 |
|
||||
| Mean cash | 27.8% | 28.2% |
|
||||
| Mean gross exposure | 72.2% | 71.8% |
|
||||
| Mean hold | 15.4 sessions | 15.6 sessions |
|
||||
| Mean EV | +0.328 R | +0.399 R |
|
||||
| Mean profit factor | 1.60 | 1.75 |
|
||||
|
||||
The floor removes about 8% of fills while leaving exposure and holding time
|
||||
nearly unchanged. This is selection, not general de-risking: candidates that
|
||||
available sizing compresses below half the intended risk are worse on average.
|
||||
The report records repeated reject attempts, not the rejected candidates'
|
||||
ranks, so whether the effect is rank-mediated remains unknown.
|
||||
|
||||
Next research: one single-variable A/B, `cap10_incumbent` versus cap 10 with
|
||||
`min_initial_risk_fraction=0.005`, with every other rule unchanged. Do not call
|
||||
the current `cash_unbounded` result causal evidence for that floor until this
|
||||
confound-free comparison is run.
|
||||
|
||||
## Weekly replacement hurts
|
||||
|
||||
Median paired deltas read zero because enough cohorts are inert. The distribution
|
||||
is not neutral:
|
||||
|
||||
| Protocol | Mean ΔEV | P25 ΔEV | Identical paths |
|
||||
|---|---:|---:|---:|
|
||||
| Empty book | -0.0360 R | -0.0817 R | 27/78 (34.6%) |
|
||||
| Warm book | -0.0348 R | -0.1582 R | 14/97 (14.4%) |
|
||||
|
||||
The arm made 2,170 replacements and 529 same-symbol re-entries within ten
|
||||
sessions, so 24% of replacements were associated with short-horizon churn.
|
||||
|
||||
Decision: **reject weekly top-10 replacement.** Future reports should show mean
|
||||
paired effects and identical-path fractions beside medians whenever treatments
|
||||
are inert in a material share of cohorts.
|
||||
|
||||
## Warm dispersion was mostly structurally degenerate
|
||||
|
||||
For six of seven anchors, control EV IQR is numerical zero (approximately
|
||||
`1e-16`) and Calmar IQR is exactly zero. The displayed ratio `1.000` is therefore
|
||||
mostly the implementation's zero-over-zero convention, not evidence of equal
|
||||
nonzero dispersion.
|
||||
|
||||
Two mechanics cause convergence: sizing and notional limits are fractions of
|
||||
equity, making R and ratio metrics scale-invariant; and the 30-session maximum
|
||||
hold is shorter than the 63-session minimum seed offset, allowing initial books
|
||||
to wash out before the anchor.
|
||||
|
||||
The exception is 2023. Control measurement-start positions vary from 6 to 9,
|
||||
EV IQR is 0.0274 R, and Calmar IQR is 0.2675. The protocol therefore carries
|
||||
state correctly, but its chosen offsets usually erase the initialization effect
|
||||
it was intended to measure.
|
||||
|
||||
Future initialization studies should use seed offsets shorter than maximum hold,
|
||||
approximately 5–25 sessions. The current empty-book cohorts remain the primary
|
||||
start-date evidence, but they necessarily mix initialization with market regime.
|
||||
|
||||
## Final decisions
|
||||
|
||||
1. ~~Keep cap 10; its measured opportunity cost is negligible.~~ **REVERSED —
|
||||
see the correction below.**
|
||||
2. Reject weekly rank replacement. *(Stands.)*
|
||||
3. Do not interpret the `cash_unbounded` improvement as a capacity effect.
|
||||
*(Stands — and it is not a floor effect worth having either; see below.)*
|
||||
4. ~~Run only the focused cap-10 effective-risk-floor A/B next.~~ **REVERSED —
|
||||
that A/B is answered and negative; do not run it.**
|
||||
5. Report means, inert fractions, and absolute dispersion beside medians and
|
||||
ratios in future sparse-treatment studies. *(Stands, and see below — the
|
||||
metric itself matters as much as the summary statistic.)*
|
||||
|
||||
## Correction 2026-08-05: EV per trade was the wrong lens
|
||||
|
||||
Everything above judged the arms on **mean paired net EV per trade**. That is the
|
||||
wrong metric for any treatment that changes how many trades the book takes.
|
||||
Capacity does not change trade *quality*; it changes trade *count*. A flat EV/trade
|
||||
delta therefore does not mean "no benefit" — it means the blocked entries were
|
||||
**just as good** as the taken ones, so refusing them cost their entire
|
||||
contribution to return. Re-running the same paired comparison on CAGR inverts two
|
||||
conclusions.
|
||||
|
||||
### Capacity: raise the cap (reverses decision 1)
|
||||
|
||||
`cap15_incumbent` versus `cap10_incumbent`, paired, all 175 paths, 0.10% per fill:
|
||||
|
||||
| Metric | Mean Δ | Worse / better |
|
||||
|---|---:|---:|
|
||||
| Trades | +1.00 | **0 / 76** (never fewer) |
|
||||
| **CAGR pp** | **+1.075** | 2 / 51 |
|
||||
| Total return pp | +1.079 | 1 / 51 |
|
||||
| Max drawdown pp | +0.007 | 1 / 2 |
|
||||
| Calmar | +0.062 | **1 / 51** |
|
||||
| Sharpe | +0.022 | 10 / 28 |
|
||||
| Net EV R/trade | +0.001 | 47 / 29 |
|
||||
|
||||
Restricted to the 105 paths where the cap actually bound: **+1.791pp CAGR**.
|
||||
|
||||
The honest tail: exactly one path was materially hurt — `empty-2023-04`, CAGR
|
||||
87.2 → 81.2 (−6.0pp), drawdown 13.0 → 14.3, from two extra trades. Second-worst
|
||||
was −0.1pp. The best paths (+6.6/+6.7/+6.9pp) came with *identical* drawdown. Best
|
||||
and worst magnitudes are symmetric at roughly ±6pp, but the frequency is 51:1.
|
||||
|
||||
Blocked count is not lost value in either direction: `empty-2021-05` had **244**
|
||||
blocked entries under cap 10, and relieving every one of them moved CAGR by
|
||||
−0.1pp.
|
||||
|
||||
**Shipped:** `SIM_MAX_POSITIONS` and `shadow_book_service.DEFAULT_CAPACITY` raised
|
||||
10 → 15. Fifteen is headroom, not a target — cap15 peaked at 12 with zero
|
||||
full-book skips, so cash plus the 20% notional cap is the real ceiling and
|
||||
15/20/None are the same experiment.
|
||||
|
||||
### Effective-risk floor: closed negative (reverses decision 4)
|
||||
|
||||
The floor A/B does not need running — this study already contains it.
|
||||
`cap15_incumbent` (peak 12, zero blocked, no floor) and `cash_unbounded` (peak 12,
|
||||
floor) have the same effective capacity and differ essentially only by
|
||||
`min_initial_risk_fraction`. Paired, n=175, 0.10% per fill, floor minus no-floor:
|
||||
|
||||
| Metric | Mean Δ | Worse / better |
|
||||
|---|---:|---:|
|
||||
| Net EV R/trade | **+0.032** | 53 / 121 |
|
||||
| Profit factor | **+0.073** | 46 / 128 |
|
||||
| Trades | **−11.4** | **174 / 0** (never adds one) |
|
||||
| **CAGR pp** | **−0.753** | 105 / 68 |
|
||||
| Total return pp | −0.765 | 105 / 68 |
|
||||
| Sharpe | −0.047 | 108 / 65 |
|
||||
| Calmar | −0.051 | 103 / 71 |
|
||||
| Max drawdown pp | +0.333 (worse) | — |
|
||||
|
||||
The same trap, mirrored: the floor raises per-trade quality *precisely by deleting
|
||||
trades*, and the deleted trades were net positive contributors. The frozen
|
||||
specification in [effective-risk-floor-ab.md](effective-risk-floor-ab.md) would
|
||||
have passed it on paired EV and shipped a change costing 0.75pp of CAGR.
|
||||
|
||||
Genuinely open, low priority: 0.005 clearly over-cuts, but the sizing code's real
|
||||
floor is a **$1** minimum, which is no floor at all. Whether something near 0.001
|
||||
strips true dust without cutting real trades is untested, and only worth revisiting
|
||||
if live broker order minimums force it.
|
||||
|
||||
### Start-date sensitivity is real but not a capacity artifact
|
||||
|
||||
Within-year spread of EV across monthly start dates is ~0.672 R and is
|
||||
*identical* for `cap10` (0.672), `cap15` (0.672) and `cash_unbounded` (0.677). It
|
||||
is small-sample noise — roughly 84 trades per 252-session window drawn from a
|
||||
fat-tailed R distribution gives an EV standard error near 0.15–0.25 R — not a
|
||||
queueing artifact. No construction policy reduces it.
|
||||
|
||||
### Rule for future studies
|
||||
|
||||
Choose the metric from the treatment's mechanism before reading any table. If a
|
||||
treatment changes trade count, CAGR and total return are the decision metrics and
|
||||
EV per trade is a diagnostic. The generated report's headline tables lead with
|
||||
ΔEV net R, which is what made this error easy to make twice.
|
||||
@@ -0,0 +1,169 @@
|
||||
# Portfolio-capacity bracket — frozen specification
|
||||
|
||||
Date frozen: 2026-08-05
|
||||
Branch: research/portfolio-capacity-rebalancing
|
||||
Runner: scripts/run_portfolio_construction_matrix.py
|
||||
|
||||
## Question and motivation
|
||||
|
||||
The daily Phase A production control (a0_control: close fill, 30-session
|
||||
maximum hold, 1% fixed-fractional risk, no correlation or volatility overlay)
|
||||
recorded 472 trades and 519 otherwise qualified entries rejected because the
|
||||
ten-position book was full. The blocked share is 519 / (519 + 472) = 52.4%.
|
||||
The book is therefore materially arrival-order constrained.
|
||||
|
||||
This supersedes the older statement that the ten-slot cap never bound. That
|
||||
statement came from a shorter, weekly, pre-gate-reset replay and is not evidence
|
||||
about the current daily strategy.
|
||||
|
||||
The study brackets the value of capacity before tuning replacement details. It
|
||||
does not contain a formal promotion rule or automatically change production.
|
||||
Because the current ~505-name production membership is projected backward,
|
||||
paired arm-versus-control differences are the primary evidence. Absolute
|
||||
profitability is descriptive and survivorship-biased.
|
||||
|
||||
Implementation correction: the first completed v1 artifact at commit `23fe39f`
|
||||
incorrectly allowed the snapshot's broad rank-only universe to submit trades.
|
||||
That artifact is invalid, is removed from the branch, and must not be used for
|
||||
strategy conclusions. Runner v2 fixes the construction/ranking partition below.
|
||||
|
||||
## Frozen arms
|
||||
|
||||
1. **cap10_incumbent:** exact production-style cap-10 control, no displacement.
|
||||
2. **cash_unbounded:** no position-count cap; cash/no leverage and the existing
|
||||
20% per-position notional ceiling remain. Reject an entry if actual initial
|
||||
stop-risk after cash/notional sizing is below 0.5% of marked equity.
|
||||
3. **cap10_weekly_top10:** on the final trading session of each ISO week, rank
|
||||
holdings plus fresh same-day qualified entrants and retain the top ten.
|
||||
4. **cap15_incumbent:** cap 15, no displacement.
|
||||
|
||||
All arms use the frozen Phase A control configuration: daily candidate replay,
|
||||
live-like full-universe residual-momentum/low-volatility 80/20 rank, activation
|
||||
threshold 80, normal gate-reset re-entry, close fill, 3×ATR trail, 30-session
|
||||
maximum hold, 1% risk, and costs of 0.10% and 0.20% per fill.
|
||||
|
||||
Every priced symbol contributes to the daily cross-sectional rank. Only symbols
|
||||
not listed in the snapshot's `research_rank_only` side table may submit trade
|
||||
setups to any arm. The resulting construction universe must contain 450-600
|
||||
symbols (expected approximately 505); validation fails outside that frozen
|
||||
guardrail or when the side table references unknown ticker symbols.
|
||||
|
||||
The daily replay uses zero outcome horizon: setup and rank observations continue
|
||||
through the snapshot's last session because portfolio simulation, unlike outcome
|
||||
grading, does not require 30 future bars.
|
||||
|
||||
Control-parity note: a direct main-versus-branch comparison found identical
|
||||
total return, CAGR, maximum drawdown, and Sharpe. The branch intentionally
|
||||
changes only the first calendar year's `yearly_returns` convention: it starts
|
||||
from initial capital rather than equity after the first session, so day-one
|
||||
entry costs are now charged to year one. Older reports can therefore show a
|
||||
different first-year contextual return without a strategy-performance
|
||||
regression. New trade-detail and measurement-start fields are additive.
|
||||
|
||||
### Weekly-selection mechanics
|
||||
|
||||
- Ordinary exits run before entries/rebalancing.
|
||||
- Open slots may still fill from daily qualified entries during the week.
|
||||
- On the final ISO-week session, current holdings and that day's fresh qualified
|
||||
entrants use the full-universe strategy_rank for that same date.
|
||||
- Stored entry-day rank is never used.
|
||||
- Holdings with missing current rank/data are protected and consume a slot;
|
||||
entrants missing rank are ineligible.
|
||||
- Incumbents win exact rank ties; symbol is the deterministic final tie-breaker.
|
||||
- Rebalance exits pay costs and bypass cooldown/post-stop state.
|
||||
- Report entrant-pool sizes, replacements, turnover, and same-symbol re-entry
|
||||
within 5/10/20 sessions.
|
||||
|
||||
## Frozen cohorts
|
||||
|
||||
research.sqlite is expected to cover 2016-01-04 through 2026-07-17. Residual
|
||||
momentum requires 252 benchmark sessions. Empty-book starts additionally require
|
||||
504 prior scoring sessions and 252 forward measurement sessions.
|
||||
|
||||
- **Empty book:** first eligible session of each month, approximately January
|
||||
2019 through July 2025; start with no positions and measure 252 sessions.
|
||||
- **Warm book:** first session of each year 2019–2025 is the measurement anchor.
|
||||
Seed the portfolio on the first session of every ISO week falling 63–126
|
||||
trading sessions before the anchor, carry all positions and gate-reset state
|
||||
forward, and measure the same 252-session anchor window.
|
||||
|
||||
Warm portfolio returns reset to marked equity immediately before the anchor
|
||||
session. P&L after the anchor from carried positions belongs to portfolio
|
||||
returns, while trade EV includes only entries on or after the anchor. Remaining
|
||||
positions liquidate at the last measurement close with costs.
|
||||
|
||||
The validate-only mode must print realized cohort counts and fail unless both
|
||||
protocols contain the seven annual clusters 2019–2025 and every warm anchor has
|
||||
at least 12 seeds. It must also print ranking, rank-only, and tradable symbol
|
||||
counts plus the raw, removed, and retained qualified-long counts.
|
||||
|
||||
## Reporting
|
||||
|
||||
Primary reported measures:
|
||||
|
||||
- net EV per trade in R, with costs and actual initial stop-risk dollars;
|
||||
- Calmar (CAGR / max drawdown);
|
||||
- profit factor on net trade R;
|
||||
- Gain-to-Pain (sum of all monthly returns / absolute sum of negative months);
|
||||
- Sortino using daily returns and zero target.
|
||||
|
||||
Also report total return/CAGR, maximum drawdown, Sharpe, win rate, time
|
||||
underwater, exposure, cash, average/peak positions, sessions at capacity,
|
||||
turnover, costs, qualified/admitted/blocked opportunities, and minimum-risk
|
||||
rejections.
|
||||
|
||||
For each arm/protocol/cost/metric, pair identical paths with cap10_incumbent,
|
||||
take the median paired delta within each start year or annual anchor, show all
|
||||
seven cluster values, and headline their median.
|
||||
|
||||
Initialization dispersion is reported separately for EV and Calmar: calculate
|
||||
the seed-path IQR within each warm anchor, divide by the paired control IQR, show
|
||||
all seven ratios, and headline their median. Do not combine them into a composite.
|
||||
|
||||
For context only, run a deterministic 10,000-replicate cluster bootstrap over
|
||||
the seven paired annual summaries and report the central 90% percentile interval
|
||||
for median EV and Calmar deltas and warm IQR ratios. These intervals are not
|
||||
promotion gates, independent-population confidence claims, or formal inference.
|
||||
|
||||
## Reproducibility and execution
|
||||
|
||||
Candidate replay/ranks cache under reports/.cache; each matrix cell checkpoints
|
||||
atomically and resume verifies a fingerprint over the implementation commit,
|
||||
this specification hash, snapshot SHA-256, cache key, arm definitions, costs,
|
||||
and cohort manifest. An authoritative run refuses a dirty worktree.
|
||||
|
||||
The existing v1 candidate/rank cache is intentionally reusable: its
|
||||
full-universe current-day ranks are correct. Runner v2 derives a fingerprinted
|
||||
construction view by removing qualified rows whose symbols are rank-only. V2
|
||||
uses a versioned checkpoint directory, so invalid v1 portfolio cells are never
|
||||
resumed and the expensive daily rank replay does not need to run again.
|
||||
|
||||
The loader reads only ticker ID/symbol and the OHLCV columns used by replay, so
|
||||
snapshots created before SEC metadata added `tickers.cik`, `tickers.sic`, and
|
||||
`tickers.sic_description` remain valid. Do not migrate or alter the research
|
||||
snapshot: its original SHA-256 is part of the run fingerprint.
|
||||
|
||||
macOS environment setup from the repository root (zsh):
|
||||
|
||||
python3 -m venv .venv
|
||||
./.venv/bin/python -m pip install -e '.[dev]'
|
||||
|
||||
Preflight:
|
||||
|
||||
./.venv/bin/python scripts/run_portfolio_construction_matrix.py \
|
||||
backtest_snapshots/research.sqlite \
|
||||
--run-id prod505-capacity-bracket-daily-v1 \
|
||||
--workers auto \
|
||||
--resume \
|
||||
--validate-only
|
||||
|
||||
Authoritative run:
|
||||
|
||||
./.venv/bin/python scripts/run_portfolio_construction_matrix.py \
|
||||
backtest_snapshots/research.sqlite \
|
||||
--run-id prod505-capacity-bracket-daily-v1 \
|
||||
--workers auto \
|
||||
--resume
|
||||
|
||||
Commit only the compact final JSON and Markdown reports. Raw curves, trades,
|
||||
candidate caches, and checkpoints remain ignored.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,114 @@
|
||||
# Focused daily portfolio-capacity matrix
|
||||
|
||||
Generated: 2026-08-05T19:25:17.150472+00:00
|
||||
|
||||
## Question
|
||||
|
||||
The current daily Phase A control admitted 472 trades and rejected 519 qualified opportunities because the ten-slot book was full. This run brackets the economic cost of that binding constraint; it has no formal promotion gate.
|
||||
|
||||
> Universe caveat: today's production membership is projected backward. Use paired arm-versus-control differences, not absolute profitability, for construction conclusions.
|
||||
|
||||
## Validated universes
|
||||
|
||||
- Tradable setup symbols with prices: 505.
|
||||
- Rank-only symbols with prices: 4149.
|
||||
- Full ranking symbols with prices: 4654.
|
||||
- Tradable qualified longs: 6118.
|
||||
- Rank-only qualified rows removed: 136286.
|
||||
|
||||
## Paired annual medians
|
||||
|
||||
### Empty Book — 0.10% per fill
|
||||
|
||||
| Arm | ΔEV net R | 90% context | ΔCalmar | 90% context |
|
||||
|---|---:|---:|---:|---:|
|
||||
| cap10_incumbent | 0.000 | [0.000, 0.000] | 0.000 | [0.000, 0.000] |
|
||||
| cash_unbounded | 0.044 | [-0.011, 0.060] | 0.030 | [-0.030, 0.120] |
|
||||
| cap10_weekly_top10 | 0.000 | [-0.091, 0.000] | 0.000 | [-0.260, 0.000] |
|
||||
| cap15_incumbent | 0.000 | [0.000, 0.011] | 0.000 | [0.000, 0.130] |
|
||||
|
||||
| Arm | ΔPF | ΔGain-to-Pain | ΔSortino | ΔCAGR pp | ΔMaxDD pp |
|
||||
|---|---:|---:|---:|---:|---:|
|
||||
| cap10_incumbent | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
|
||||
| cash_unbounded | 0.079 | 0.047 | -0.013 | 1.350 | 0.000 |
|
||||
| cap10_weekly_top10 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
|
||||
| cap15_incumbent | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
|
||||
|
||||
### Warm Book — 0.10% per fill
|
||||
|
||||
| Arm | ΔEV net R | 90% context | ΔCalmar | 90% context |
|
||||
|---|---:|---:|---:|---:|
|
||||
| cap10_incumbent | 0.000 | [0.000, 0.000] | 0.000 | [0.000, 0.000] |
|
||||
| cash_unbounded | 0.034 | [-0.014, 0.100] | 0.050 | [-0.160, 0.250] |
|
||||
| cap10_weekly_top10 | 0.000 | [-0.158, 0.065] | 0.000 | [-0.200, 0.330] |
|
||||
| cap15_incumbent | 0.000 | [-0.006, 0.000] | 0.000 | [0.000, 0.180] |
|
||||
|
||||
| Arm | ΔPF | ΔGain-to-Pain | ΔSortino | ΔCAGR pp | ΔMaxDD pp |
|
||||
|---|---:|---:|---:|---:|---:|
|
||||
| cap10_incumbent | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
|
||||
| cash_unbounded | 0.062 | 0.085 | -0.004 | 2.200 | 0.400 |
|
||||
| cap10_weekly_top10 | 0.000 | 0.012 | 0.018 | 0.300 | 0.000 |
|
||||
| cap15_incumbent | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
|
||||
|
||||
### Empty Book — 0.20% per fill
|
||||
|
||||
| Arm | ΔEV net R | 90% context | ΔCalmar | 90% context |
|
||||
|---|---:|---:|---:|---:|
|
||||
| cap10_incumbent | 0.000 | [0.000, 0.000] | 0.000 | [0.000, 0.000] |
|
||||
| cash_unbounded | 0.041 | [-0.010, 0.052] | 0.030 | [-0.015, 0.100] |
|
||||
| cap10_weekly_top10 | 0.000 | [-0.090, 0.000] | 0.000 | [-0.260, 0.000] |
|
||||
| cap15_incumbent | 0.000 | [0.000, 0.010] | 0.000 | [0.000, 0.110] |
|
||||
|
||||
| Arm | ΔPF | ΔGain-to-Pain | ΔSortino | ΔCAGR pp | ΔMaxDD pp |
|
||||
|---|---:|---:|---:|---:|---:|
|
||||
| cap10_incumbent | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
|
||||
| cash_unbounded | 0.066 | 0.035 | -0.014 | 0.900 | 0.000 |
|
||||
| cap10_weekly_top10 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
|
||||
| cap15_incumbent | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
|
||||
|
||||
### Warm Book — 0.20% per fill
|
||||
|
||||
| Arm | ΔEV net R | 90% context | ΔCalmar | 90% context |
|
||||
|---|---:|---:|---:|---:|
|
||||
| cap10_incumbent | 0.000 | [0.000, 0.000] | 0.000 | [0.000, 0.000] |
|
||||
| cash_unbounded | 0.034 | [-0.022, 0.102] | 0.040 | [-0.130, 0.230] |
|
||||
| cap10_weekly_top10 | 0.000 | [-0.158, 0.065] | 0.000 | [-0.190, 0.310] |
|
||||
| cap15_incumbent | 0.000 | [-0.006, 0.000] | 0.000 | [0.000, 0.170] |
|
||||
|
||||
| Arm | ΔPF | ΔGain-to-Pain | ΔSortino | ΔCAGR pp | ΔMaxDD pp |
|
||||
|---|---:|---:|---:|---:|---:|
|
||||
| cap10_incumbent | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
|
||||
| cash_unbounded | 0.060 | 0.083 | -0.003 | 2.100 | 0.300 |
|
||||
| cap10_weekly_top10 | 0.000 | 0.017 | 0.020 | 0.300 | 0.000 |
|
||||
| cap15_incumbent | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
|
||||
|
||||
## Warm-seed initialization dispersion
|
||||
|
||||
| Arm | Cost/fill | Median EV IQR ratio | Median Calmar IQR ratio |
|
||||
|---|---:|---:|---:|
|
||||
| cap10_incumbent | 0.10% | 1.000 | 1.000 |
|
||||
| cash_unbounded | 0.10% | 1.000 | 1.000 |
|
||||
| cap10_weekly_top10 | 0.10% | 1.000 | 1.000 |
|
||||
| cap15_incumbent | 0.10% | 1.000 | 1.000 |
|
||||
| cap10_incumbent | 0.20% | 1.000 | 1.000 |
|
||||
| cash_unbounded | 0.20% | 1.000 | 1.000 |
|
||||
| cap10_weekly_top10 | 0.20% | 1.000 | 1.000 |
|
||||
| cap15_incumbent | 0.20% | 1.000 | 1.000 |
|
||||
|
||||
## Capacity and operations — 0.10% per fill
|
||||
|
||||
| Arm | Median trades | Median blocked | Median positions | Peak | Turnover | Min-risk rejects |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| cap10_incumbent | 76.0 | 21.6% | 4.98 | 10 | 26.36 | 0 |
|
||||
| cash_unbounded | 74.0 | 0.0% | 4.82 | 12 | 26.76 | 85517 |
|
||||
| cap10_weekly_top10 | 88.0 | 18.1% | 5.13 | 10 | 28.44 | 0 |
|
||||
| cap15_incumbent | 79.0 | 0.0% | 5.15 | 12 | 27.32 | 0 |
|
||||
|
||||
## Weekly-ranking opportunity set
|
||||
|
||||
- Median fresh entrant pool: 0.0.
|
||||
- Median zero-entrant fraction: 0.558.
|
||||
- Replacements across reported paths: 2170.
|
||||
- Same-symbol re-entries within 10 sessions: 529.
|
||||
|
||||
Bootstrap intervals above resample seven annual summaries and are descriptive context only. They are not gates or independent-population confidence claims.
|
||||
@@ -0,0 +1,84 @@
|
||||
'''Shared production-style historical ranking helpers for research runners.'''
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date
|
||||
|
||||
|
||||
def _period_percentiles(
|
||||
observations: list[dict], value_key: str
|
||||
) -> dict[tuple[str, str], float]:
|
||||
'''Rank one deterministic ticker observation per historical period.'''
|
||||
by_period: dict[tuple, list[dict]] = {}
|
||||
seen: set[tuple[str, str]] = set()
|
||||
for row in observations:
|
||||
identity = (str(row['symbol']), str(row['date']))
|
||||
if identity in seen:
|
||||
raise ValueError(f'Duplicate universe rank observation: {identity}')
|
||||
seen.add(identity)
|
||||
if row.get(value_key) is None:
|
||||
continue
|
||||
period = tuple(row['ranking_period'])
|
||||
by_period.setdefault(period, []).append(row)
|
||||
|
||||
result: dict[tuple[str, str], float] = {}
|
||||
for group in by_period.values():
|
||||
ordered = sorted(
|
||||
group,
|
||||
key=lambda row: (float(row[value_key]), str(row['symbol'])),
|
||||
)
|
||||
denominator = len(ordered) - 1
|
||||
for rank, row in enumerate(ordered):
|
||||
result[(str(row['symbol']), str(row['date']))] = round(
|
||||
rank / denominator * 100.0 if denominator > 0 else 100.0,
|
||||
2,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def _live_universe_rank_map(
|
||||
observations: list[dict],
|
||||
benchmark_closes: dict[date, float],
|
||||
momentum_weight: float,
|
||||
) -> dict[tuple[str, str], dict[str, float | None]]:
|
||||
'''Historical equivalent of production compute_activation_ranks.
|
||||
|
||||
Every ticker contributes at most once per session. Residual momentum starts
|
||||
only once 252 benchmark closes were point-in-time available; earlier dates
|
||||
use the same raw-momentum fallback as production.
|
||||
'''
|
||||
identities = [(str(row['symbol']), str(row['date'])) for row in observations]
|
||||
if len(identities) != len(set(identities)):
|
||||
raise ValueError('Universe ranking requires one observation per ticker/date')
|
||||
|
||||
raw_pct = _period_percentiles(observations, 'momentum')
|
||||
residual_pct = _period_percentiles(observations, 'residual_momentum')
|
||||
vol_pct = _period_percentiles(observations, 'vol_6m')
|
||||
benchmark_ords = sorted(value.toordinal() for value in benchmark_closes)
|
||||
residual_start_ord = benchmark_ords[251] if len(benchmark_ords) >= 252 else None
|
||||
|
||||
ranks: dict[tuple[str, str], dict[str, float | None]] = {}
|
||||
for row in observations:
|
||||
identity = (str(row['symbol']), str(row['date']))
|
||||
asof_ord = date.fromisoformat(identity[1]).toordinal()
|
||||
momentum_pct = (
|
||||
residual_pct.get(identity)
|
||||
if residual_start_ord is not None and asof_ord >= residual_start_ord
|
||||
else raw_pct.get(identity)
|
||||
)
|
||||
volatility_pct = vol_pct.get(identity)
|
||||
strategy_rank = (
|
||||
round(
|
||||
momentum_pct * momentum_weight
|
||||
+ volatility_pct * (1.0 - momentum_weight),
|
||||
2,
|
||||
)
|
||||
if momentum_pct is not None and volatility_pct is not None
|
||||
else momentum_pct
|
||||
)
|
||||
ranks[identity] = {
|
||||
'momentum_percentile': momentum_pct,
|
||||
'volatility_percentile': volatility_pct,
|
||||
'strategy_rank': strategy_rank,
|
||||
}
|
||||
return ranks
|
||||
@@ -29,6 +29,11 @@ ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from scripts.research_rankings import ( # noqa: E402
|
||||
_live_universe_rank_map,
|
||||
_period_percentiles,
|
||||
)
|
||||
|
||||
POLICY_NAMES = (
|
||||
"immediate",
|
||||
"next_session",
|
||||
@@ -107,85 +112,6 @@ def _default_output_path() -> Path:
|
||||
return Path("reports") / f"daily-reentry-matrix-{stamp}.json"
|
||||
|
||||
|
||||
def _period_percentiles(
|
||||
observations: list[dict], value_key: str
|
||||
) -> dict[tuple[str, str], float]:
|
||||
"""Production-style percentiles, one deterministic symbol row per period."""
|
||||
by_period: dict[tuple, list[dict]] = {}
|
||||
seen: set[tuple[str, str]] = set()
|
||||
for row in observations:
|
||||
identity = (str(row["symbol"]), str(row["date"]))
|
||||
if identity in seen:
|
||||
raise ValueError(f"Duplicate universe rank observation: {identity}")
|
||||
seen.add(identity)
|
||||
if row.get(value_key) is None:
|
||||
continue
|
||||
period = tuple(row["ranking_period"])
|
||||
by_period.setdefault(period, []).append(row)
|
||||
|
||||
result: dict[tuple[str, str], float] = {}
|
||||
for group in by_period.values():
|
||||
ordered = sorted(
|
||||
group,
|
||||
key=lambda row: (float(row[value_key]), str(row["symbol"])),
|
||||
)
|
||||
denominator = len(ordered) - 1
|
||||
for rank, row in enumerate(ordered):
|
||||
result[(str(row["symbol"]), str(row["date"]))] = round(
|
||||
rank / denominator * 100.0 if denominator > 0 else 100.0,
|
||||
2,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def _live_universe_rank_map(
|
||||
observations: list[dict],
|
||||
benchmark_closes: dict[date, float],
|
||||
momentum_weight: float,
|
||||
) -> dict[tuple[str, str], dict[str, float | None]]:
|
||||
"""Historical equivalent of ``compute_activation_ranks``.
|
||||
|
||||
Every ticker contributes at most once per session. Residual momentum starts
|
||||
only once 252 benchmark closes were point-in-time available; earlier dates
|
||||
use the same raw-momentum fallback as production.
|
||||
"""
|
||||
identities = [(str(row["symbol"]), str(row["date"])) for row in observations]
|
||||
if len(identities) != len(set(identities)):
|
||||
raise ValueError("Universe ranking requires one observation per ticker/date")
|
||||
|
||||
raw_pct = _period_percentiles(observations, "momentum")
|
||||
residual_pct = _period_percentiles(observations, "residual_momentum")
|
||||
vol_pct = _period_percentiles(observations, "vol_6m")
|
||||
benchmark_ords = sorted(value.toordinal() for value in benchmark_closes)
|
||||
residual_start_ord = benchmark_ords[251] if len(benchmark_ords) >= 252 else None
|
||||
|
||||
ranks: dict[tuple[str, str], dict[str, float | None]] = {}
|
||||
for row in observations:
|
||||
identity = (str(row["symbol"]), str(row["date"]))
|
||||
asof_ord = date.fromisoformat(identity[1]).toordinal()
|
||||
momentum_pct = (
|
||||
residual_pct.get(identity)
|
||||
if residual_start_ord is not None and asof_ord >= residual_start_ord
|
||||
else raw_pct.get(identity)
|
||||
)
|
||||
volatility_pct = vol_pct.get(identity)
|
||||
strategy_rank = (
|
||||
round(
|
||||
momentum_pct * momentum_weight
|
||||
+ volatility_pct * (1.0 - momentum_weight),
|
||||
2,
|
||||
)
|
||||
if momentum_pct is not None and volatility_pct is not None
|
||||
else momentum_pct
|
||||
)
|
||||
ranks[identity] = {
|
||||
"momentum_percentile": momentum_pct,
|
||||
"volatility_percentile": volatility_pct,
|
||||
"strategy_rank": strategy_rank,
|
||||
}
|
||||
return ranks
|
||||
|
||||
|
||||
class PrecomputedDailyEngine:
|
||||
"""Exact date/symbol lookup over the already-ranked production gate."""
|
||||
|
||||
|
||||
@@ -55,6 +55,11 @@ ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from scripts.research_rankings import ( # noqa: E402
|
||||
_live_universe_rank_map,
|
||||
_period_percentiles,
|
||||
)
|
||||
|
||||
# Must match Phase A cache when reusing research-cands.pkl
|
||||
CACHE_VERSION = "research-matrix-v1-daily-prod"
|
||||
|
||||
@@ -104,66 +109,6 @@ def _parse_args() -> argparse.Namespace:
|
||||
return p.parse_args()
|
||||
|
||||
|
||||
def _period_percentiles(
|
||||
observations: list[dict], value_key: str
|
||||
) -> dict[tuple[str, str], float]:
|
||||
by_period: dict[tuple, list[dict]] = {}
|
||||
for row in observations:
|
||||
if row.get(value_key) is None:
|
||||
continue
|
||||
period = tuple(row["ranking_period"])
|
||||
by_period.setdefault(period, []).append(row)
|
||||
result: dict[tuple[str, str], float] = {}
|
||||
for group in by_period.values():
|
||||
ordered = sorted(
|
||||
group, key=lambda row: (float(row[value_key]), str(row["symbol"]))
|
||||
)
|
||||
denominator = len(ordered) - 1
|
||||
for rank, row in enumerate(ordered):
|
||||
result[(str(row["symbol"]), str(row["date"]))] = round(
|
||||
rank / denominator * 100.0 if denominator > 0 else 100.0,
|
||||
2,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def _live_universe_rank_map(
|
||||
observations: list[dict],
|
||||
benchmark_closes: dict[date, float],
|
||||
momentum_weight: float,
|
||||
) -> dict[tuple[str, str], dict[str, float | None]]:
|
||||
raw_pct = _period_percentiles(observations, "momentum")
|
||||
residual_pct = _period_percentiles(observations, "residual_momentum")
|
||||
vol_pct = _period_percentiles(observations, "vol_6m")
|
||||
benchmark_ords = sorted(value.toordinal() for value in benchmark_closes)
|
||||
residual_start_ord = benchmark_ords[251] if len(benchmark_ords) >= 252 else None
|
||||
ranks: dict[tuple[str, str], dict[str, float | None]] = {}
|
||||
for row in observations:
|
||||
identity = (str(row["symbol"]), str(row["date"]))
|
||||
asof_ord = date.fromisoformat(identity[1]).toordinal()
|
||||
momentum_pct = (
|
||||
residual_pct.get(identity)
|
||||
if residual_start_ord is not None and asof_ord >= residual_start_ord
|
||||
else raw_pct.get(identity)
|
||||
)
|
||||
volatility_pct = vol_pct.get(identity)
|
||||
strategy_rank = (
|
||||
round(
|
||||
momentum_pct * momentum_weight
|
||||
+ volatility_pct * (1.0 - momentum_weight),
|
||||
2,
|
||||
)
|
||||
if momentum_pct is not None and volatility_pct is not None
|
||||
else momentum_pct
|
||||
)
|
||||
ranks[identity] = {
|
||||
"momentum_percentile": momentum_pct,
|
||||
"volatility_percentile": volatility_pct,
|
||||
"strategy_rank": strategy_rank,
|
||||
}
|
||||
return ranks
|
||||
|
||||
|
||||
def _window(arm: dict, name: str) -> dict | None:
|
||||
for row in arm.get("windows") or []:
|
||||
if row.get("window") == name:
|
||||
|
||||
@@ -68,6 +68,11 @@ ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from scripts.research_rankings import ( # noqa: E402
|
||||
_live_universe_rank_map,
|
||||
_period_percentiles,
|
||||
)
|
||||
|
||||
CACHE_VERSION = "research-matrix-v1-daily-prod"
|
||||
|
||||
# Pre-registered arm catalogue (order is report order). Control is a0.
|
||||
@@ -210,66 +215,6 @@ def _sqlite_url(path: Path) -> str:
|
||||
return f"sqlite+aiosqlite:///{path.resolve().as_posix()}"
|
||||
|
||||
|
||||
def _period_percentiles(
|
||||
observations: list[dict], value_key: str
|
||||
) -> dict[tuple[str, str], float]:
|
||||
by_period: dict[tuple, list[dict]] = {}
|
||||
for row in observations:
|
||||
if row.get(value_key) is None:
|
||||
continue
|
||||
period = tuple(row["ranking_period"])
|
||||
by_period.setdefault(period, []).append(row)
|
||||
result: dict[tuple[str, str], float] = {}
|
||||
for group in by_period.values():
|
||||
ordered = sorted(
|
||||
group, key=lambda row: (float(row[value_key]), str(row["symbol"]))
|
||||
)
|
||||
denominator = len(ordered) - 1
|
||||
for rank, row in enumerate(ordered):
|
||||
result[(str(row["symbol"]), str(row["date"]))] = round(
|
||||
rank / denominator * 100.0 if denominator > 0 else 100.0,
|
||||
2,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def _live_universe_rank_map(
|
||||
observations: list[dict],
|
||||
benchmark_closes: dict[date, float],
|
||||
momentum_weight: float,
|
||||
) -> dict[tuple[str, str], dict[str, float | None]]:
|
||||
raw_pct = _period_percentiles(observations, "momentum")
|
||||
residual_pct = _period_percentiles(observations, "residual_momentum")
|
||||
vol_pct = _period_percentiles(observations, "vol_6m")
|
||||
benchmark_ords = sorted(value.toordinal() for value in benchmark_closes)
|
||||
residual_start_ord = benchmark_ords[251] if len(benchmark_ords) >= 252 else None
|
||||
ranks: dict[tuple[str, str], dict[str, float | None]] = {}
|
||||
for row in observations:
|
||||
identity = (str(row["symbol"]), str(row["date"]))
|
||||
asof_ord = date.fromisoformat(identity[1]).toordinal()
|
||||
momentum_pct = (
|
||||
residual_pct.get(identity)
|
||||
if residual_start_ord is not None and asof_ord >= residual_start_ord
|
||||
else raw_pct.get(identity)
|
||||
)
|
||||
volatility_pct = vol_pct.get(identity)
|
||||
strategy_rank = (
|
||||
round(
|
||||
momentum_pct * momentum_weight
|
||||
+ volatility_pct * (1.0 - momentum_weight),
|
||||
2,
|
||||
)
|
||||
if momentum_pct is not None and volatility_pct is not None
|
||||
else momentum_pct
|
||||
)
|
||||
ranks[identity] = {
|
||||
"momentum_percentile": momentum_pct,
|
||||
"volatility_percentile": volatility_pct,
|
||||
"strategy_rank": strategy_rank,
|
||||
}
|
||||
return ranks
|
||||
|
||||
|
||||
def _parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(
|
||||
description=__doc__,
|
||||
|
||||
Reference in New Issue
Block a user