signal-platform/README.md

# Signal Dashboard

Investing-signal platform for NASDAQ stocks. Surfaces the best trading opportunities through weighted multi-dimensional scoring — technical indicators, support/resistance quality, sentiment, fundamentals, and momentum — with asymmetric risk:reward scanning.

**Philosophy:** Don't predict price. Find the path of least resistance, key S/R zones, and asymmetric R:R setups.

## How It Works

Scheduled pipelines turn raw prices into a ranked, gated list of tradeable setups. Everything downstream of OHLCV is recomputed from stored data, so each refresh is cheap and idempotent. Job timing is cron-based and configurable in **Admin → Jobs** (default timezone Europe/Berlin).

### Daily Load — the full refresh

Once a day (default 07:00). Steps run **in dependency order**, each consuming the previous step's fresh output:

1. **OHLCV** — fetch the latest daily bars for every tracked ticker (Alpaca); new tickers backfill ~5 years.
2. **Sentiment** — fetch sentiment for the names that matter and are stale (> 5 days): top-pick feeders (momentum leaders with a tradeable long setup), the watchlist, and open paper trades, plus a top-N-by-composite discovery net. Runs *before* the scan so the scan sees fresh sentiment.
3. **R:R Scan** — recompute S/R zones, the 5-dimension scores and long/short setups (ATR stops, S/R targets) for every ticker, and attach each ticker's 12‑1 momentum percentile.
4. **Outcome Eval** — resolve setups that hit target/stop or expired (default 30 trading days) and close paper trades that hit a level.
5. **Market Regime** — recompute the regime index (breadth/trend).
6. **Regime Monitor** — observational early-warning snapshot (VIX, credit spreads via FRED); feeds nothing else.

A failing step is logged; the pipeline continues with the next.

### Intraday — light refresh

Hourly across the US session (Mon–Fri): only **OHLCV → Outcome Eval**, to keep prices current and close paper trades intraday. No scan/sentiment — the dashboard recomputes live R:R from the latest price, so fresh prices are enough.

### Other jobs

Fundamentals (weekly, early Monday) · Alerts (hourly, Telegram) · Backtest (weekly) · Ticker-universe sync (daily). Deep history backfill and event study are manual-only (Admin → Jobs).

### From score to "top pick"

1. **Composite score** — technical, S/R-quality, sentiment, fundamental and momentum sub-scores (0–100) combine into a weighted composite (weights configurable; missing dimensions re-normalize).
2. **Setups** — the scanner builds long/short setups with ATR stops and S/R targets, then adds a confidence score, conflict flags and a target reach-probability.
3. **Activation gate** — a setup *qualifies* only if it clears the R:R and confidence floors **and** ranks in the top momentum percentile of the universe (the validated edge is long-only momentum).
4. **Top pick** — the highest-momentum qualified setup; highlighted on the Dashboard and labelled on the ticker page.

## Key Use Cases

- **Find today's best long setup.** On the **Dashboard**, the *Top Setups* table lists qualified setups ranked by momentum with the #1 flagged "Top pick". Each row opens the ticker page for the chart, scores, S/R targets and entry/stop.
- **Track a trade you took.** Mark a setup as a **paper trade**: it's marked-to-market against the latest close, auto-closed on stop/target, and its sentiment stays fresh while open. *Signals → Track Record* shows the realized edge.

## Stack

| Layer | Tech |
|---|---|
| Backend | Python 3.12+, FastAPI, Uvicorn, async SQLAlchemy, Alembic |
| Database | PostgreSQL (asyncpg) |
| Scheduler | APScheduler — daily & intraday pipelines, fundamentals, alerts, regime, backtest |
| Frontend | React 18, TypeScript, Vite 5 |
| Styling | Tailwind CSS 3 with custom glassmorphism design system |
| State | TanStack React Query v5 (server), Zustand (client/auth) |
| Charts | Canvas 2D candlestick chart with S/R overlays |
| Routing | React Router v6 (SPA) |
| HTTP | Axios with JWT interceptor |
| Data providers | Alpaca (OHLCV); OpenAI / Gemini / DeepSeek / xAI (sentiment, pluggable); Fundamentals chain: FMP → Finnhub → Alpha Vantage; FRED (regime); Telegram (alerts) |

## Features

### Backend
- Ticker registry with full cascade delete
- Universe bootstrap for `sp500`, `nasdaq100`, `nasdaq_all` via admin endpoint
- OHLCV price storage with upsert and validation
- Technical indicators: ADX, EMA, RSI, ATR, Volume Profile, Pivot Points, EMA Cross
- Support/Resistance detection with strength scoring and merge-within-tolerance
- Sentiment analysis with time-decay weighted scoring
- Fundamental data tracking (P/E, revenue growth, earnings surprise, market cap)
- 5-dimension scoring engine (technical, S/R quality, sentiment, fundamental, momentum) with configurable weights
- Risk:Reward scanner — long and short setups, ATR-based stops, S/R-based targets, configurable R:R threshold (default 1.5:1)
- Activation gate — qualifies setups on a momentum-percentile floor plus R:R/confidence (validated long-only edge); ranks the rest by expected value
- Recommendation layer — directional confidence, conflict detection, per-target reach-probability
- Paper trading — take a setup, mark-to-market vs. latest close, auto-close on stop/target, realized track record + outcome evaluation
- Market-regime index + FRED early-warning monitor (VIX, credit spreads); weekly backtest + manual event study
- Telegram alerts (e.g. regime-quadrant changes)
- User-curated watchlist (cap: 20), enriched with composite score, R:R and S/R summary
- JWT auth with admin role, configurable registration, user access control
- Cron-scheduled pipelines (admin-configurable) with per-job enable/disable and live status monitoring
- Admin panel: user management, data cleanup, job control, system settings

### Frontend
- Glassmorphism UI with frosted glass panels, gradient text, ambient glow effects, mesh gradient background
- Interactive candlestick chart (Canvas 2D) with hover tooltips showing OHLCV values
- Support/Resistance level overlays on chart (top 6 by strength, dashed lines with labels)
- Data freshness bar showing availability and recency of each data source
- Watchlist with composite scores, R:R ratios, and S/R summaries
- Ticker detail page: chart, scores, sentiment breakdown, fundamentals, technical indicators, S/R table
- Rankings table with configurable dimension weights
- Trade scanner showing detected R:R setups
- Admin page: user management, job status with live indicators, enable/disable toggles, data cleanup, system settings
- Protected routes with JWT auth, admin-only sections
- Responsive layout with mobile navigation
- Toast notifications for async operations

## Pages

| Route | Page | Access |
|---|---|---|
| `/login` | Login | Public |
| `/register` | Register | Public (when enabled) |
| `/` | Dashboard — top setups, open trades, regime (default) | Authenticated |
| `/market` | Market — watchlist + rankings tabs | Authenticated |
| `/signals` | Signals — scanner + track record tabs | Authenticated |
| `/regime` | Market Regime | Authenticated |
| `/ticker/:symbol` | Ticker Detail | Authenticated |
| `/admin` | Admin Panel | Admin only |

Legacy routes redirect: `/watchlist` → `/market`, `/rankings` → `/market?tab=rankings`, `/scanner` → `/signals`, `/performance` → `/signals?tab=track`.

## API Endpoints

All under `/api/v1/`. Interactive docs at `/docs` (Swagger) and `/redoc`.

| Group | Endpoints |
|---|---|
| Health | `GET /health` |
| Auth | `POST /auth/register`, `POST /auth/login` |
| Tickers | `POST /tickers`, `GET /tickers`, `DELETE /tickers/{symbol}` |
| OHLCV | `POST /ohlcv`, `GET /ohlcv/{symbol}` |
| Ingestion | `POST /ingestion/fetch/{symbol}` |
| Indicators | `GET /indicators/{symbol}/{type}`, `GET /indicators/{symbol}/ema-cross` |
| S/R Levels | `GET /sr-levels/{symbol}` |
| Sentiment | `GET /sentiment/{symbol}` |
| Fundamentals | `GET /fundamentals/{symbol}` |
| Scores | `GET /scores/{symbol}`, `GET /rankings`, `PUT /scores/weights` |
| Trades | `GET /trades`, `GET /trades/{symbol}`, `GET /trades/{symbol}/history`, `GET /trades/activation`, `GET /trades/performance` |
| Paper Trades | `GET /paper-trades`, `POST /paper-trades`, `POST /paper-trades/{id}/close` |
| Market / Regime | `GET /market/regime`, `GET /regime/monitor`, `GET/PUT /regime/config`, `GET /regime/history`, `GET /regime/event-study`, `GET/PUT /regime/fundamentals`, `GET /backtest/report` |
| Jobs | `GET /jobs/running` |
| Watchlist | `GET /watchlist`, `POST /watchlist/{symbol}`, `DELETE /watchlist/{symbol}` |
| Admin | `GET /admin/users`, `POST /admin/users`, `PUT /admin/users/{id}/access`, `PUT /admin/users/{id}/password`, `PUT /admin/settings/registration`, `GET /admin/settings`, `PUT /admin/settings/{key}`, `GET/PUT /admin/settings/recommendations`, `GET/PUT /admin/settings/ticker-universe`, `POST /admin/tickers/bootstrap`, `POST /admin/data/cleanup`, `GET /admin/jobs`, `POST /admin/jobs/{name}/trigger`, `PUT /admin/jobs/{name}/toggle`, `GET /admin/pipeline/readiness` |

## Development Setup

### Prerequisites

- Python 3.12+
- PostgreSQL (via Homebrew on macOS: `brew install postgresql@17`)
- Node.js 18+ and npm

### Backend Setup

```bash
# Create and activate virtual environment
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

# Configure environment
cp .env.example .env
# Edit .env with your values (see Environment Variables below)

# Start PostgreSQL and create database
brew services start postgresql@17
createdb stock_data_backend
createuser stock_backend

# Run migrations
alembic upgrade head

# Start the backend
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
```

A default `admin`/`admin` account is created on first startup. Open http://localhost:8000/docs for Swagger UI.

### Frontend Setup

```bash
cd frontend
npm install
npm run dev
```

Open http://localhost:5173 for the Signal Dashboard. The Vite dev server proxies `/api/v1/` requests to the backend at `http://127.0.0.1:8000`.

### Frontend Build

```bash
cd frontend
npm run build     # TypeScript check + production build → frontend/dist/
npm run preview   # Preview the production build locally
```

### Tests

```bash
# Backend tests (in-memory SQLite — no PostgreSQL needed)
pytest tests/ -v

# Frontend tests
cd frontend
npm test
```

## Environment Variables

Configure in `.env` (copy from `.env.example`):

| Variable | Required | Default | Description |
|---|---|---|---|
| `DATABASE_URL` | Yes | — | PostgreSQL connection string (`postgresql+asyncpg://...`) |
| `JWT_SECRET` | Yes | — | Random secret for JWT signing |
| `JWT_EXPIRY_MINUTES` | No | `60` | JWT token expiry |
| `ALPACA_API_KEY` | For OHLCV | — | Alpaca Markets API key |
| `ALPACA_API_SECRET` | For OHLCV | — | Alpaca Markets API secret |
| `GEMINI_API_KEY` | For sentiment | — | Google Gemini API key |
| `GEMINI_MODEL` | No | `gemini-2.0-flash` | Gemini model name |
| `OPENAI_API_KEY` | For sentiment (OpenAI path) | — | OpenAI API key |
| `OPENAI_MODEL` | No | `gpt-4o-mini` | OpenAI model name |
| `OPENAI_SENTIMENT_BATCH_SIZE` | No | `5` | Micro-batch size for sentiment collector |
| `FMP_API_KEY` | Optional (fundamentals) | — | Financial Modeling Prep API key (first provider in chain) |
| `FINNHUB_API_KEY` | Optional (fundamentals) | — | Finnhub API key (fallback provider) |
| `ALPHA_VANTAGE_API_KEY` | Optional (fundamentals) | — | Alpha Vantage API key (fallback provider) |
| `FRED_API_KEY` | Optional (regime) | — | FRED key for the regime monitor (VIX, credit spreads) |
| `TELEGRAM_BOT_TOKEN` | Optional (alerts) | — | Telegram bot token for alerts (can also be set in Admin) |
| `TELEGRAM_CHAT_ID` | Optional (alerts) | — | Telegram chat id for alerts |
| `DATA_COLLECTOR_FREQUENCY` | No | `daily` | OHLCV collection schedule (legacy — see note below) |
| `SENTIMENT_POLL_INTERVAL_MINUTES` | No | `30` | Sentiment polling interval |
| `FUNDAMENTAL_FETCH_FREQUENCY` | No | `weekly` | Fundamentals fetch cadence |
| `RR_SCAN_FREQUENCY` | No | `daily` | R:R scanner schedule |
| `FUNDAMENTAL_RATE_LIMIT_RETRIES` | No | `3` | Retries per ticker on fundamentals rate-limit |
| `FUNDAMENTAL_RATE_LIMIT_BACKOFF_SECONDS` | No | `15` | Base backoff seconds for fundamentals retry (exponential) |
| `DEFAULT_WATCHLIST_AUTO_SIZE` | No | `10` | Auto-watchlist size |
| `DEFAULT_RR_THRESHOLD` | No | `1.5` | Minimum R:R ratio for setups |
| `DB_POOL_SIZE` | No | `5` | Database connection pool size |
| `LOG_LEVEL` | No | `INFO` | Logging level |

> **Note:** Pipeline timing (daily / intraday / fundamentals cron, timezone) is configured at runtime in **Admin → Jobs** and stored in the DB — the `*_FREQUENCY` env vars are legacy fallbacks for the few jobs still on interval triggers (alerts, universe sync).

## Production Deployment (Debian 12)

### 1. Install dependencies

```bash
sudo apt update && sudo apt install -y python3.12 python3.12-venv postgresql nginx nodejs npm
```

### 2. Create service user

```bash
sudo useradd -r -s /usr/sbin/nologin stockdata
```

### 3. Deploy application

```bash
sudo mkdir -p /opt/stock-data-backend
# Copy project files to /opt/stock-data-backend
cd /opt/stock-data-backend
python3.12 -m venv .venv
source .venv/bin/activate
pip install .
```

### 4. Configure

```bash
sudo cp .env.example /opt/stock-data-backend/.env
sudo chown stockdata:stockdata /opt/stock-data-backend/.env
# Edit .env with production values (strong JWT_SECRET, real API keys, etc.)
```

### 5. Database

```bash
DB_NAME=stock_data_backend DB_USER=stock_backend DB_PASS=strong_password ./deploy/setup_db.sh
```

### 6. Build frontend

```bash
cd frontend
npm ci
npm run build
```

### 7. Systemd service

```bash
sudo cp deploy/stock-data-backend.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now stock-data-backend
```

### 8. Nginx reverse proxy

```bash
sudo cp deploy/nginx.conf /etc/nginx/sites-available/stock-data-backend
sudo ln -s /etc/nginx/sites-available/stock-data-backend /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
```

Nginx serves the frontend static files from `frontend/dist/` and proxies `/api/v1/` to the backend.

### 9. SSL (recommended)

```bash
sudo apt install certbot python3-certbot-nginx
sudo certbot --nginx -d signal.thiessen.io
```

### Verify

```bash
curl https://signal.thiessen.io/api/v1/health
```

## Project Structure

```
app/
├── main.py              # FastAPI app, lifespan, router wiring
├── config.py            # Pydantic settings from .env
├── database.py          # Async SQLAlchemy engine + session
├── dependencies.py      # DI: DB session, auth guards
├── exceptions.py        # Exception hierarchy
├── middleware.py         # Global error handler → JSON envelope
├── cache.py             # LRU cache with per-ticker invalidation
├── scheduler.py         # APScheduler job definitions
├── models/              # SQLAlchemy ORM models
├── schemas/             # Pydantic request/response schemas
├── services/            # Business logic layer
├── providers/           # External data provider integrations
└── routers/             # FastAPI route handlers

frontend/
├── index.html           # SPA entry point
├── vite.config.ts       # Vite config with API proxy
├── tailwind.config.ts   # Tailwind + glassmorphism theme
├── package.json
└── src/
    ├── App.tsx          # Route definitions
    ├── main.tsx         # React entry point
    ├── api/             # Axios API client modules (one per resource)
    ├── components/
    │   ├── admin/       # User table, job controls, settings, data cleanup
    │   ├── auth/        # Protected route wrapper
    │   ├── charts/      # Canvas candlestick chart
    │   ├── layout/      # App shell, sidebar, mobile nav
    │   ├── rankings/    # Rankings table, weights form
    │   ├── scanner/     # Trade table
    │   ├── ticker/      # Sentiment panel, fundamentals, indicators, S/R overlay
    │   ├── ui/          # Badge, toast, skeleton, score card, confirm dialog
    │   └── watchlist/   # Watchlist table, add ticker form
    ├── hooks/           # React Query hooks (one per resource)
    ├── lib/             # Types, formatting utilities
    ├── pages/           # Page components (Login, Register, Dashboard, Market, Signals, Regime, Ticker, Admin)
    ├── stores/          # Zustand auth store
    └── styles/          # Global CSS with glassmorphism classes

deploy/
├── nginx.conf           # Reverse proxy + static file serving
├── setup_db.sh          # Idempotent DB setup script
└── stock-data-backend.service  # systemd unit

tests/
├── conftest.py          # Fixtures, strategies, test DB
├── unit/                # Unit tests
└── property/            # Property-based tests (Hypothesis)
```