signal-platform/docs/research/fundamentals-weight-backtest.md

Point-in-time fundamentals weight backtest

Status: pre-registered local research. Running it does not change production.

Question

Does reordering already-qualified long setups with SEC fundamentals improve the production book's risk-adjusted return? The qualification gate, execution model, position sizing, capacity, costs, ATR trail, and post-stop re-entry policy remain unchanged. This isolates the incremental value of fundamentals as a ranking overlay.

Registered experiment

The control is the production 80/20 residual-momentum / volatility rank. The runner tests three fundamental composites at weights 10%, 20%, 30%, and 40%:

• Quality: operating margin, FCF margin, low net-debt/EBITDA, and low dilution. At least two inputs must exist.
• Growth: revenue growth and diluted-EPS growth. At least one must exist.
• Balanced: equal weight to the quality and growth sub-scores. Both must exist.

Each raw metric is ranked favorably from 0 to 100 across the CIK-deduplicated tracked universe. Missing composite scores are neutral at 50. The formula is:

final rank = (1 - weight) * production rank + weight * fundamental rank

There are 13 registered portfolio trials including the control. That complete count is used by the Deflated Sharpe calculation.

Historical P/E and FCF yield are excluded. Stored Alpaca bars are split-adjusted, while filing-time EPS and shares are not guaranteed to use today's split basis; mixing them without point-in-time split factors can manufacture valuation moves. Earnings surprise is also excluded because the completed Dolt SUE study already failed its promotion bar for this strategy.

Point-in-time rule

Only SEC rows accepted before midnight America/New_York at the start of a signal date are visible. This is conservative relative to the daily pre-market SEC import and prevents same-day filings or later amendments from leaking backward. The derivation then selects the newest visible accession per fiscal period.

Default windows are fixed before the first run:

• Train: entry date before 2024-01-01
• Validation: 2024-01-01 through 2024-12-31
• Test: entry date on or after 2025-01-01

Do not move these boundaries after seeing results. The test window is used only to check the single arm chosen from train and validation. Reports expose all registered rows for auditability and correct multiple-testing accounting.

1. Create the portable snapshot

Run this wherever the production PostgreSQL connection is already configured. The exporter copies prices, the safe strategy settings, SEC snapshots, and Dolt earnings rows. It does not copy credentials or unrelated system settings.

Windows PowerShell:

.venv\Scripts\python.exe scripts\create_backtest_snapshot.py `
  --output backtest_snapshots\fundamentals-backtest.sqlite `
  --force

Linux production host:

.venv/bin/python scripts/create_backtest_snapshot.py \
  --output backtest_snapshots/fundamentals-backtest.sqlite \
  --force

Copy only the SQLite file to the MacBook. scp, a local network share, or an encrypted USB drive are all fine. Do not copy .env.

2. Prepare the MacBook

Use the same Git commit as the machine that created the report. From the repo:

python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e '.[dev]'
chmod +x scripts/run_fundamentals_macbook.sh

Put the snapshot at backtest_snapshots/fundamentals-backtest.sqlite, or pass a different path to the launcher.

3. Run it

./scripts/run_fundamentals_macbook.sh \
  backtest_snapshots/fundamentals-backtest.sqlite

The launcher defaults to logical CPU count minus one. Override it if the laptop gets too warm or memory pressure rises:

WORKERS=8 ./scripts/run_fundamentals_macbook.sh \
  backtest_snapshots/fundamentals-backtest.sqlite

The first run builds two caches under reports/.cache: production candidate replay and point-in-time fundamental scores. If interrupted, rerun the same command; valid caches are reused. Cache keys include snapshot size and mtime, so a new snapshot triggers a rebuild.

4. Bring the result back

The final line names one ZIP such as:

reports/fundamentals-overlay-20260723-180000.zip

That ZIP contains:

• the complete JSON report and reproducibility metadata;
• a readable Markdown summary;
• portfolio-arm CSV;
• factor-IC CSV;
• full-period trade CSV for every arm, allowing winner-concentration checks;
• this registered protocol.

Copy the ZIP into this workspace or attach it in the conversation. The snapshot itself is not needed for the first evaluation unless a result looks inconsistent.

Evaluation order

1. Data coverage and the accepted-at range.
2. Individual factor IC: sign, magnitude, consistency, and cross-section size.
3. Composite IC in train, validation, and test.
4. Development selection made without the test window.
5. Test Sharpe, CAGR, drawdown, yearly returns, trial-corrected DSR, and trade overlap versus control.
6. Sensitivity to a few dominant winners and whether the effect is economically large enough to justify production complexity.

The mechanical development bar requires train and validation Sharpe not below control and validation drawdown no more than two percentage points worse. A test pass is still research evidence, not automatic deployment.

Known limitation

The snapshot contains today's tracked tickers, not historical constituent membership or delisted names. This creates survivorship bias. The same biased universe is used for control and overlays, so the local comparison is useful, but its absolute Sharpe or CAGR must not be presented as an unbiased live expectation. Forward paper performance remains the true out-of-sample check.