# Implementation Plan: Dashboard Enhancements

## Overview

Incremental implementation of four dashboard enhancements: sentiment drill-down, fundamentals drill-down, chart zoom/crosshair, and S/R zone clustering. Each feature area is built backend-first (model → service → schema → router) then frontend, with tests alongside implementation. All changes are additive to existing components.

## Tasks

- [x] 1. Sentiment drill-down — backend
  - [x] 1.1 Add `reasoning` and `citations_json` columns to `SentimentScore` model and create Alembic migration
    - Add `reasoning: Mapped[str] = mapped_column(Text, nullable=False, default="")` and `citations_json: Mapped[str] = mapped_column(Text, nullable=False, default="[]")` to `app/models/sentiment.py`
    - Create Alembic migration with `server_default` so existing rows are backfilled
    - _Requirements: 1.1, 1.2, 1.3_

  - [x] 1.2 Update `OpenAISentimentProvider` to extract reasoning and citations from OpenAI response
    - Add `reasoning` and `citations` fields to the `SentimentData` dataclass
    - Extract `reasoning` from the parsed JSON response body
    - Iterate `response.output` items for `url_citation` annotations, collect `{"url": ..., "title": ...}` dicts
    - Return empty citations list when no annotations exist (no error)
    - _Requirements: 1.1, 1.2, 1.4_

  - [x] 1.3 Update `sentiment_service.store_sentiment()` to persist reasoning and citations
    - Accept `reasoning` and `citations` parameters
    - Serialize citations to JSON string before storing
    - _Requirements: 1.3_

  - [x] 1.4 Update sentiment schema and router to include reasoning and citations in API response
    - Add `CitationItem` model and `reasoning`/`citations` fields to `SentimentScoreResult` in `app/schemas/sentiment.py`
    - Deserialize `citations_json` when building the response, catch `JSONDecodeError` and default to `[]`
    - _Requirements: 1.3_

  - [ ]* 1.5 Write property tests for sentiment reasoning and citations extraction
    - **Property 1: Sentiment reasoning extraction** — Generate random JSON with reasoning fields, verify extraction
    - **Validates: Requirements 1.1**
    - **Property 2: Sentiment citations extraction** — Generate mock OpenAI responses with 0–10 annotations, verify citations list
    - **Validates: Requirements 1.2, 1.4**

  - [ ]* 1.6 Write property test for sentiment data round-trip
    - **Property 3: Sentiment data round-trip** — Generate random reasoning + citations, store, retrieve via test client, compare
    - **Validates: Requirements 1.3**

- [x] 2. Sentiment drill-down — frontend
  - [x] 2.1 Add `CitationItem`, `reasoning`, and `citations` fields to `SentimentScore` type in `frontend/src/lib/types.ts`
    - _Requirements: 1.3, 2.2, 2.3_

  - [x] 2.2 Add expandable detail section to `SentimentPanel`
    - Add `useState<boolean>(false)` for expand/collapse toggle
    - Render chevron toggle button below summary metrics
    - When expanded: show reasoning text (or "No reasoning available" placeholder if empty) and citations as clickable `<a>` links
    - When collapsed: hide reasoning and citations, keep summary metrics visible
    - Default to collapsed state on initial render
    - Preserve glassmorphism UI style
    - _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5_

  - [ ]* 2.3 Write property tests for SentimentPanel drill-down
    - **Property 4: Expanded sentiment detail displays all data** — Generate random reasoning and citations, render expanded, verify DOM content
    - **Validates: Requirements 2.2, 2.3**
    - **Property 5: Sentiment detail collapse hides content** — Expand then collapse, verify summary visible and detail hidden
    - **Validates: Requirements 2.4**

- [x] 3. Checkpoint — Sentiment drill-down complete
  - Ensure all tests pass, ask the user if questions arise.

- [x] 4. Fundamentals drill-down — backend
  - [x] 4.1 Add `unavailable_fields_json` column to fundamental model and create Alembic migration
    - Add `unavailable_fields_json: Mapped[str] = mapped_column(Text, nullable=False, default="{}")` to `app/models/fundamental.py`
    - Add column to the same Alembic migration as sentiment columns (or a new one if migration 1.1 is already applied), with `server_default='{}'`
    - _Requirements: 3.1, 3.2_

  - [x] 4.2 Update `FMPFundamentalProvider` to record 402 reasons in `unavailable_fields`
    - Add `unavailable_fields` field to `FundamentalData` dataclass
    - In `_fetch_json_optional`, when HTTP 402 is received, map endpoint to field name: `ratios-ttm` → `pe_ratio`, `financial-growth` → `revenue_growth`, `earnings` → `earnings_surprise`
    - Record `"requires paid plan"` as the reason for each affected field
    - _Requirements: 3.1_

  - [x] 4.3 Update `fundamental_service` to persist `unavailable_fields`
    - Serialize `unavailable_fields` dict to JSON string before storing
    - _Requirements: 3.2_

  - [x] 4.4 Update fundamentals schema and router to include `unavailable_fields` in API response
    - Add `unavailable_fields: dict[str, str] = {}` to `FundamentalResponse` in `app/schemas/fundamental.py`
    - Deserialize `unavailable_fields_json` when building the response, catch `JSONDecodeError` and default to `{}`
    - _Requirements: 3.2_

  - [ ]* 4.5 Write property tests for FMP 402 reason recording and round-trip
    - **Property 6: FMP 402 reason recording** — Generate random 402/200 combinations for 3 endpoints, verify unavailable_fields mapping
    - **Validates: Requirements 3.1**
    - **Property 7: Fundamentals unavailable_fields round-trip** — Generate random dicts, store, retrieve, compare
    - **Validates: Requirements 3.2**

- [x] 5. Fundamentals drill-down — frontend
  - [x] 5.1 Add `unavailable_fields` to `FundamentalResponse` type in `frontend/src/lib/types.ts`
    - _Requirements: 3.2, 3.3_

  - [x] 5.2 Update `FundamentalsPanel` to show unavailability reasons and expandable detail section
    - When a metric is null and `unavailable_fields[field_name]` exists, display reason text in amber instead of "—"
    - When a metric is null and no reason exists, display "—"
    - Add expand/collapse toggle below summary metrics (default collapsed)
    - When expanded: show data source name ("FMP"), fetch timestamp, and list of unavailable fields with reasons
    - When collapsed: hide detail, keep summary metrics visible
    - Preserve glassmorphism UI style
    - _Requirements: 3.3, 3.4, 4.1, 4.2, 4.3, 4.4_

  - [ ]* 5.3 Write property tests for FundamentalsPanel display logic
    - **Property 8: Null field display depends on reason existence** — Generate random FundamentalResponse with various null/reason combos, verify displayed text
    - **Validates: Requirements 3.3, 3.4**
    - **Property 9: Fundamentals expanded detail content** — Generate random response, expand, verify source/timestamp/reasons in DOM
    - **Validates: Requirements 4.2**

- [x] 6. Checkpoint — Fundamentals drill-down complete
  - Ensure all tests pass, ask the user if questions arise.

- [x] 7. S/R zone clustering — backend
  - [x] 7.1 Implement `cluster_sr_zones()` function in `app/services/sr_service.py`
    - Sort levels by price ascending
    - Greedy merge: walk sorted levels, merge if within tolerance % of current cluster midpoint
    - Compute zone: low, high, midpoint, strength (sum capped at 100), level_count
    - Tag zone type: "support" if midpoint < current_price, else "resistance"
    - Sort by strength descending
    - If `max_zones` set, return top N; if 0 or negative, return empty list
    - Handle empty input by returning empty list
    - _Requirements: 7.1, 7.2, 7.3, 7.4, 8.2_

  - [x] 7.2 Add `SRZoneResult` schema and update `SRLevelResponse` in `app/schemas/sr_level.py`
    - Add `SRZoneResult` model with `low`, `high`, `midpoint`, `strength`, `type`, `level_count`
    - Add `zones: list[SRZoneResult] = []` to `SRLevelResponse`
    - _Requirements: 7.2, 9.1_

  - [x] 7.3 Update S/R router to accept `max_zones` parameter and return zones
    - Add `max_zones: int = 6` query parameter to the S/R levels endpoint
    - Call `cluster_sr_zones()` with fetched levels and current price
    - Include zones in the response
    - _Requirements: 8.1, 8.3_

  - [ ]* 7.4 Write property tests for S/R zone clustering
    - **Property 14: Clustering merges nearby levels** — Generate random level sets and tolerances, verify no two zones have midpoints within tolerance
    - **Validates: Requirements 7.2**
    - **Property 15: Zone strength is capped sum** — Generate random level sets, cluster, verify strength = min(100, sum)
    - **Validates: Requirements 7.3**
    - **Property 16: Zone type tagging** — Generate random zones and current prices, verify support/resistance tagging
    - **Validates: Requirements 7.4**
    - **Property 17: Zone filtering returns top N by strength** — Generate random zone sets and limits, verify top-N selection
    - **Validates: Requirements 8.2**

- [x] 8. Checkpoint — S/R clustering backend complete
  - Ensure all tests pass, ask the user if questions arise.

- [x] 9. Chart enhancements — zoom and pan
  - [x] 9.1 Add `SRZone` and `SRLevelResponse.zones` types to `frontend/src/lib/types.ts`
    - _Requirements: 9.1_

  - [x] 9.2 Implement zoom (mouse wheel) on `CandlestickChart`
    - Add `visibleRange: { start: number, end: number }` state initialized to full dataset
    - Add `onWheel` handler: positive delta narrows range (zoom in), negative widens (zoom out)
    - Clamp visible range to min 10 bars, max full dataset length
    - Disable zoom if dataset has fewer than 10 bars
    - Slice data by `visibleRange` for rendering
    - Debounce zoom via `requestAnimationFrame`
    - _Requirements: 5.1, 5.2, 5.3, 5.5_

  - [x] 9.3 Implement pan (click-drag) on `CandlestickChart`
    - Add `isPanning` and `panStartX` state
    - `onMouseDown` starts pan, `onMouseMove` shifts visible range proportionally, `onMouseUp` ends pan
    - Pan only active when zoomed in (visible range < full dataset)
    - Clamp range to dataset bounds
    - _Requirements: 5.4_

  - [x] 9.4 Implement crosshair overlay on `CandlestickChart`
    - Add `crosshair: { x: number, y: number } | null` state
    - `onMouseMove` updates crosshair position
    - Draw vertical line at cursor x spanning full chart height
    - Draw horizontal line at cursor y spanning full chart width
    - Display price label on y-axis at horizontal line position
    - Display date label on x-axis at vertical line position
    - `onMouseLeave` clears crosshair
    - _Requirements: 6.1, 6.2, 6.3, 6.4, 6.5_

  - [ ]* 9.5 Write property tests for chart zoom/pan invariants
    - **Property 10: Zoom adjusts visible range proportionally** — Generate random datasets and wheel deltas, verify range narrows/widens
    - **Validates: Requirements 5.1, 5.2, 5.3**
    - **Property 11: Pan shifts visible range** — Generate random ranges and drag deltas, verify shift without width change
    - **Validates: Requirements 5.4**
    - **Property 12: Zoom range invariant** — Generate random zoom/pan sequences, verify range bounds always valid
    - **Validates: Requirements 5.5**
    - **Property 13: Coordinate-to-value mapping** — Generate random chart configs, verify yToPrice and xToBarIndex mappings
    - **Validates: Requirements 6.3, 6.4**

- [x] 10. S/R zone rendering on chart
  - [x] 10.1 Update `CandlestickChart` to accept `zones` prop and render shaded zone rectangles
    - Accept `zones?: SRZone[]` prop
    - Render each zone as a semi-transparent rectangle spanning low→high price range across full chart width
    - Use green shading (rgba) for support zones, red shading for resistance zones
    - Draw zones behind candlestick bodies (render zones first, then candles)
    - Display label with midpoint price and strength score for each zone
    - Re-render zones correctly at every zoom level using current price scale
    - _Requirements: 9.1, 9.2, 9.3, 9.4, 9.5_

  - [x] 10.2 Update `SROverlay` and `TickerDetailPage` to pass zones to `CandlestickChart`
    - Update `useTickerDetail` hook or `SROverlay` to extract zones from the S/R API response
    - Pass zones array to `CandlestickChart` component
    - Default to max 6 zones (handled by backend `max_zones=6` default)
    - _Requirements: 8.3, 9.1_

  - [x] 10.3 Ensure S/R overlays re-render correctly at all zoom levels
    - Verify zone rectangles reposition when zoom/pan changes the visible price scale
    - Handle single-level zones (low == high) as thin 2px-height rectangles
    - _Requirements: 5.6, 9.5_

- [x] 11. Final checkpoint — All features integrated
  - Ensure all tests pass, ask the user if questions arise.

## Notes

- Tasks marked with `*` are optional and can be skipped for faster MVP
- Each task references specific requirements for traceability
- Checkpoints ensure incremental validation after each feature area
- Property tests validate universal correctness properties from the design document
- The Alembic migration for sentiment and fundamentals columns should ideally be a single migration file
- S/R zones are computed on-the-fly (not persisted), so no additional migration is needed for zones