signal-platform/docs/research/fundamentals-weight-backtest.md

# Point-in-time fundamentals weight backtest

Status: initial experiment completed; split-safe follow-up registered locally.
Running either protocol does not change production.

## Question

Does reordering already-qualified long setups with SEC fundamentals improve the
production book's risk-adjusted return? The qualification gate, execution model,
position sizing, capacity, costs, ATR trail, and post-stop re-entry policy remain
unchanged. This isolates the incremental value of fundamentals as a ranking
overlay.

## Completed initial experiment

The control is the production 80/20 residual-momentum / volatility rank. The
runner tests three fundamental composites at weights 10%, 20%, 30%, and 40%:

- Quality: operating margin, FCF margin, low net-debt/EBITDA, and low dilution.
  At least two inputs must exist.
- Growth: revenue growth and diluted-EPS growth. At least one must exist.
- Balanced: equal weight to the quality and growth sub-scores. Both must exist.

Each raw metric is ranked favorably from 0 to 100 across the CIK-deduplicated
tracked universe. Missing composite scores are neutral at 50. The formula is:

`final rank = (1 - weight) * production rank + weight * fundamental rank`

There are 13 registered portfolio trials including the control. That complete
count is used by the Deflated Sharpe calculation.

No overlay passed the registered train and validation requirements. Review also
found that filing-time diluted EPS and shares are not guaranteed to use the same
split basis across periods. That makes EPS growth and share-count change unsafe
for historical ranking without point-in-time split factors. The initial result
remains an auditable rejection of its registered arms, but it is not evidence
that split-safe fundamentals have no value.

## Registered split-safe follow-up

Run with `--protocol split-safe`. This is a smaller sensitivity experiment:

- Quality: operating margin, FCF margin, and low net-debt/EBITDA. At least two
  inputs must exist.
- Growth: revenue growth only.
- Balanced: equal weight to the quality and growth sub-scores. Both must exist.
- Overlay weights: 5%, 10%, and 15%.

Diluted-EPS growth and share-count change are excluded completely: they are not
ranked, do not enter composites, and do not appear in factor-IC output. Historical
P/E and FCF yield remain excluded for the same split-basis reason. Earnings
surprise remains excluded because the completed Dolt SUE study failed its
promotion bar for this strategy.

There are 10 registered portfolio trials including the control. The split-safe
report uses 10 in its Deflated Sharpe calculation. It has a separate score cache
and `fundamentals-splitsafe-*` output prefix, so it cannot be confused with or
silently reuse the initial experiment's scores.

The test window has already been inspected during the initial experiment. Keep
the original date boundaries and development selection discipline, but treat the
follow-up as sensitivity evidence. Any promotion still requires forward paper
evidence.

## Point-in-time rule

Only SEC rows accepted before midnight America/New_York at the start of a signal
date are visible. This is conservative relative to the daily pre-market SEC
import and prevents same-day filings or later amendments from leaking backward.
The derivation then selects the newest visible accession per fiscal period.

Default windows are fixed before the first run:

- Train: entry date before 2024-01-01
- Validation: 2024-01-01 through 2024-12-31
- Test: entry date on or after 2025-01-01

Do not move these boundaries after seeing results. The test window is used only
to check the single arm chosen from train and validation. Reports expose all
registered rows for auditability and correct multiple-testing accounting.

## 1. Create the portable snapshot

Run this wherever the production PostgreSQL connection is already configured.
The exporter copies prices, the safe strategy settings, SEC snapshots, and Dolt
earnings rows. It does not copy credentials or unrelated system settings.

Windows PowerShell:

```powershell
.venv\Scripts\python.exe scripts\create_backtest_snapshot.py `
  --output backtest_snapshots\fundamentals-backtest.sqlite `
  --force
```

Linux production host:

```bash
.venv/bin/python scripts/create_backtest_snapshot.py \
  --output backtest_snapshots/fundamentals-backtest.sqlite \
  --force
```

Copy only the SQLite file to the MacBook. `scp`, a local network share, or an
encrypted USB drive are all fine. Do not copy `.env`.

## 2. Prepare the MacBook

Use the same Git commit as the machine that created the report. From the repo:

```bash
python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e '.[dev]'
chmod +x scripts/run_fundamentals_macbook.sh
```

Put the snapshot at `backtest_snapshots/fundamentals-backtest.sqlite`, or pass a
different path to the launcher.

## 3. Run it

The launcher now defaults to the registered `split-safe` follow-up:

```bash
./scripts/run_fundamentals_macbook.sh \
  backtest_snapshots/fundamentals-backtest.sqlite
```

The launcher defaults to logical CPU count minus one. Override it if the laptop
gets too warm or memory pressure rises:

```bash
WORKERS=8 ./scripts/run_fundamentals_macbook.sh \
  backtest_snapshots/fundamentals-backtest.sqlite
```

To reproduce the completed initial matrix instead, opt in explicitly:

```bash
PROTOCOL=original ./scripts/run_fundamentals_macbook.sh \
  backtest_snapshots/fundamentals-backtest.sqlite
```

The first run builds two caches under `reports/.cache`: production candidate
replay and protocol-specific point-in-time fundamental scores. If interrupted,
rerun the same command; valid caches are reused. Cache keys include the protocol,
snapshot size, and mtime, so neither a protocol switch nor a new snapshot can
reuse incompatible scores.

## 4. Bring the result back

The final line names one ZIP such as:

`reports/fundamentals-splitsafe-20260723-180000.zip`

That ZIP contains:

- the complete JSON report and reproducibility metadata;
- a readable Markdown summary;
- portfolio-arm CSV;
- factor-IC CSV;
- full-period trade CSV for every arm, allowing winner-concentration checks;
- this registered protocol.

Copy the ZIP into this workspace or attach it in the conversation. The snapshot
itself is not needed for the first evaluation unless a result looks inconsistent.

## Evaluation order

1. Data coverage and the accepted-at range.
2. Individual factor IC: sign, magnitude, consistency, and cross-section size.
3. Composite IC in train, validation, and test.
4. Development selection made without the test window.
5. Test Sharpe, CAGR, drawdown, yearly returns, trial-corrected DSR, and trade
   overlap versus control.
6. Sensitivity to a few dominant winners and whether the effect is economically
   large enough to justify production complexity.

The mechanical development bar requires train and validation Sharpe not below
control and validation drawdown no more than two percentage points worse. A test
pass is still research evidence, not automatic deployment.

## Known limitation

The snapshot contains today's tracked tickers, not historical constituent
membership or delisted names. This creates survivorship bias. The same biased
universe is used for control and overlays, so the local comparison is useful,
but its absolute Sharpe or CAGR must not be presented as an unbiased live
expectation. Forward paper performance remains the true out-of-sample check.