major update

.kiro/specs/dashboard-enhancements/tasks.md

@@ -0,0 +1,219 @@
+# Implementation Plan: Dashboard Enhancements
+
+## Overview
+
+Incremental implementation of four dashboard enhancements: sentiment drill-down, fundamentals drill-down, chart zoom/crosshair, and S/R zone clustering. Each feature area is built backend-first (model → service → schema → router) then frontend, with tests alongside implementation. All changes are additive to existing components.
+
+## Tasks
+
+- [x] 1. Sentiment drill-down — backend
+  - [x] 1.1 Add `reasoning` and `citations_json` columns to `SentimentScore` model and create Alembic migration
+    - Add `reasoning: Mapped[str] = mapped_column(Text, nullable=False, default="")` and `citations_json: Mapped[str] = mapped_column(Text, nullable=False, default="[]")` to `app/models/sentiment.py`
+    - Create Alembic migration with `server_default` so existing rows are backfilled
+    - _Requirements: 1.1, 1.2, 1.3_
+
+  - [x] 1.2 Update `OpenAISentimentProvider` to extract reasoning and citations from OpenAI response
+    - Add `reasoning` and `citations` fields to the `SentimentData` dataclass
+    - Extract `reasoning` from the parsed JSON response body
+    - Iterate `response.output` items for `url_citation` annotations, collect `{"url": ..., "title": ...}` dicts
+    - Return empty citations list when no annotations exist (no error)
+    - _Requirements: 1.1, 1.2, 1.4_
+
+  - [x] 1.3 Update `sentiment_service.store_sentiment()` to persist reasoning and citations
+    - Accept `reasoning` and `citations` parameters
+    - Serialize citations to JSON string before storing
+    - _Requirements: 1.3_
+
+  - [x] 1.4 Update sentiment schema and router to include reasoning and citations in API response
+    - Add `CitationItem` model and `reasoning`/`citations` fields to `SentimentScoreResult` in `app/schemas/sentiment.py`
+    - Deserialize `citations_json` when building the response, catch `JSONDecodeError` and default to `[]`
+    - _Requirements: 1.3_
+
+  - [ ]* 1.5 Write property tests for sentiment reasoning and citations extraction
+    - **Property 1: Sentiment reasoning extraction** — Generate random JSON with reasoning fields, verify extraction
+    - **Validates: Requirements 1.1**
+    - **Property 2: Sentiment citations extraction** — Generate mock OpenAI responses with 0–10 annotations, verify citations list
+    - **Validates: Requirements 1.2, 1.4**
+
+  - [ ]* 1.6 Write property test for sentiment data round-trip
+    - **Property 3: Sentiment data round-trip** — Generate random reasoning + citations, store, retrieve via test client, compare
+    - **Validates: Requirements 1.3**
+
+- [x] 2. Sentiment drill-down — frontend
+  - [x] 2.1 Add `CitationItem`, `reasoning`, and `citations` fields to `SentimentScore` type in `frontend/src/lib/types.ts`
+    - _Requirements: 1.3, 2.2, 2.3_
+
+  - [x] 2.2 Add expandable detail section to `SentimentPanel`
+    - Add `useState<boolean>(false)` for expand/collapse toggle
+    - Render chevron toggle button below summary metrics
+    - When expanded: show reasoning text (or "No reasoning available" placeholder if empty) and citations as clickable `<a>` links
+    - When collapsed: hide reasoning and citations, keep summary metrics visible
+    - Default to collapsed state on initial render
+    - Preserve glassmorphism UI style
+    - _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5_
+
+  - [ ]* 2.3 Write property tests for SentimentPanel drill-down
+    - **Property 4: Expanded sentiment detail displays all data** — Generate random reasoning and citations, render expanded, verify DOM content
+    - **Validates: Requirements 2.2, 2.3**
+    - **Property 5: Sentiment detail collapse hides content** — Expand then collapse, verify summary visible and detail hidden
+    - **Validates: Requirements 2.4**
+
+- [x] 3. Checkpoint — Sentiment drill-down complete
+  - Ensure all tests pass, ask the user if questions arise.
+
+- [x] 4. Fundamentals drill-down — backend
+  - [x] 4.1 Add `unavailable_fields_json` column to fundamental model and create Alembic migration
+    - Add `unavailable_fields_json: Mapped[str] = mapped_column(Text, nullable=False, default="{}")` to `app/models/fundamental.py`
+    - Add column to the same Alembic migration as sentiment columns (or a new one if migration 1.1 is already applied), with `server_default='{}'`
+    - _Requirements: 3.1, 3.2_
+
+  - [x] 4.2 Update `FMPFundamentalProvider` to record 402 reasons in `unavailable_fields`
+    - Add `unavailable_fields` field to `FundamentalData` dataclass
+    - In `_fetch_json_optional`, when HTTP 402 is received, map endpoint to field name: `ratios-ttm` → `pe_ratio`, `financial-growth` → `revenue_growth`, `earnings` → `earnings_surprise`
+    - Record `"requires paid plan"` as the reason for each affected field
+    - _Requirements: 3.1_
+
+  - [x] 4.3 Update `fundamental_service` to persist `unavailable_fields`
+    - Serialize `unavailable_fields` dict to JSON string before storing
+    - _Requirements: 3.2_
+
+  - [x] 4.4 Update fundamentals schema and router to include `unavailable_fields` in API response
+    - Add `unavailable_fields: dict[str, str] = {}` to `FundamentalResponse` in `app/schemas/fundamental.py`
+    - Deserialize `unavailable_fields_json` when building the response, catch `JSONDecodeError` and default to `{}`
+    - _Requirements: 3.2_
+
+  - [ ]* 4.5 Write property tests for FMP 402 reason recording and round-trip
+    - **Property 6: FMP 402 reason recording** — Generate random 402/200 combinations for 3 endpoints, verify unavailable_fields mapping
+    - **Validates: Requirements 3.1**
+    - **Property 7: Fundamentals unavailable_fields round-trip** — Generate random dicts, store, retrieve, compare
+    - **Validates: Requirements 3.2**
+
+- [x] 5. Fundamentals drill-down — frontend
+  - [x] 5.1 Add `unavailable_fields` to `FundamentalResponse` type in `frontend/src/lib/types.ts`
+    - _Requirements: 3.2, 3.3_
+
+  - [x] 5.2 Update `FundamentalsPanel` to show unavailability reasons and expandable detail section
+    - When a metric is null and `unavailable_fields[field_name]` exists, display reason text in amber instead of "—"
+    - When a metric is null and no reason exists, display "—"
+    - Add expand/collapse toggle below summary metrics (default collapsed)
+    - When expanded: show data source name ("FMP"), fetch timestamp, and list of unavailable fields with reasons
+    - When collapsed: hide detail, keep summary metrics visible
+    - Preserve glassmorphism UI style
+    - _Requirements: 3.3, 3.4, 4.1, 4.2, 4.3, 4.4_
+
+  - [ ]* 5.3 Write property tests for FundamentalsPanel display logic
+    - **Property 8: Null field display depends on reason existence** — Generate random FundamentalResponse with various null/reason combos, verify displayed text
+    - **Validates: Requirements 3.3, 3.4**
+    - **Property 9: Fundamentals expanded detail content** — Generate random response, expand, verify source/timestamp/reasons in DOM
+    - **Validates: Requirements 4.2**
+
+- [x] 6. Checkpoint — Fundamentals drill-down complete
+  - Ensure all tests pass, ask the user if questions arise.
+
+- [x] 7. S/R zone clustering — backend
+  - [x] 7.1 Implement `cluster_sr_zones()` function in `app/services/sr_service.py`
+    - Sort levels by price ascending
+    - Greedy merge: walk sorted levels, merge if within tolerance % of current cluster midpoint
+    - Compute zone: low, high, midpoint, strength (sum capped at 100), level_count
+    - Tag zone type: "support" if midpoint < current_price, else "resistance"
+    - Sort by strength descending
+    - If `max_zones` set, return top N; if 0 or negative, return empty list
+    - Handle empty input by returning empty list
+    - _Requirements: 7.1, 7.2, 7.3, 7.4, 8.2_
+
+  - [x] 7.2 Add `SRZoneResult` schema and update `SRLevelResponse` in `app/schemas/sr_level.py`
+    - Add `SRZoneResult` model with `low`, `high`, `midpoint`, `strength`, `type`, `level_count`
+    - Add `zones: list[SRZoneResult] = []` to `SRLevelResponse`
+    - _Requirements: 7.2, 9.1_
+
+  - [x] 7.3 Update S/R router to accept `max_zones` parameter and return zones
+    - Add `max_zones: int = 6` query parameter to the S/R levels endpoint
+    - Call `cluster_sr_zones()` with fetched levels and current price
+    - Include zones in the response
+    - _Requirements: 8.1, 8.3_
+
+  - [ ]* 7.4 Write property tests for S/R zone clustering
+    - **Property 14: Clustering merges nearby levels** — Generate random level sets and tolerances, verify no two zones have midpoints within tolerance
+    - **Validates: Requirements 7.2**
+    - **Property 15: Zone strength is capped sum** — Generate random level sets, cluster, verify strength = min(100, sum)
+    - **Validates: Requirements 7.3**
+    - **Property 16: Zone type tagging** — Generate random zones and current prices, verify support/resistance tagging
+    - **Validates: Requirements 7.4**
+    - **Property 17: Zone filtering returns top N by strength** — Generate random zone sets and limits, verify top-N selection
+    - **Validates: Requirements 8.2**
+
+- [x] 8. Checkpoint — S/R clustering backend complete
+  - Ensure all tests pass, ask the user if questions arise.
+
+- [x] 9. Chart enhancements — zoom and pan
+  - [x] 9.1 Add `SRZone` and `SRLevelResponse.zones` types to `frontend/src/lib/types.ts`
+    - _Requirements: 9.1_
+
+  - [x] 9.2 Implement zoom (mouse wheel) on `CandlestickChart`
+    - Add `visibleRange: { start: number, end: number }` state initialized to full dataset
+    - Add `onWheel` handler: positive delta narrows range (zoom in), negative widens (zoom out)
+    - Clamp visible range to min 10 bars, max full dataset length
+    - Disable zoom if dataset has fewer than 10 bars
+    - Slice data by `visibleRange` for rendering
+    - Debounce zoom via `requestAnimationFrame`
+    - _Requirements: 5.1, 5.2, 5.3, 5.5_
+
+  - [x] 9.3 Implement pan (click-drag) on `CandlestickChart`
+    - Add `isPanning` and `panStartX` state
+    - `onMouseDown` starts pan, `onMouseMove` shifts visible range proportionally, `onMouseUp` ends pan
+    - Pan only active when zoomed in (visible range < full dataset)
+    - Clamp range to dataset bounds
+    - _Requirements: 5.4_
+
+  - [x] 9.4 Implement crosshair overlay on `CandlestickChart`
+    - Add `crosshair: { x: number, y: number } | null` state
+    - `onMouseMove` updates crosshair position
+    - Draw vertical line at cursor x spanning full chart height
+    - Draw horizontal line at cursor y spanning full chart width
+    - Display price label on y-axis at horizontal line position
+    - Display date label on x-axis at vertical line position
+    - `onMouseLeave` clears crosshair
+    - _Requirements: 6.1, 6.2, 6.3, 6.4, 6.5_
+
+  - [ ]* 9.5 Write property tests for chart zoom/pan invariants
+    - **Property 10: Zoom adjusts visible range proportionally** — Generate random datasets and wheel deltas, verify range narrows/widens
+    - **Validates: Requirements 5.1, 5.2, 5.3**
+    - **Property 11: Pan shifts visible range** — Generate random ranges and drag deltas, verify shift without width change
+    - **Validates: Requirements 5.4**
+    - **Property 12: Zoom range invariant** — Generate random zoom/pan sequences, verify range bounds always valid
+    - **Validates: Requirements 5.5**
+    - **Property 13: Coordinate-to-value mapping** — Generate random chart configs, verify yToPrice and xToBarIndex mappings
+    - **Validates: Requirements 6.3, 6.4**
+
+- [x] 10. S/R zone rendering on chart
+  - [x] 10.1 Update `CandlestickChart` to accept `zones` prop and render shaded zone rectangles
+    - Accept `zones?: SRZone[]` prop
+    - Render each zone as a semi-transparent rectangle spanning low→high price range across full chart width
+    - Use green shading (rgba) for support zones, red shading for resistance zones
+    - Draw zones behind candlestick bodies (render zones first, then candles)
+    - Display label with midpoint price and strength score for each zone
+    - Re-render zones correctly at every zoom level using current price scale
+    - _Requirements: 9.1, 9.2, 9.3, 9.4, 9.5_
+
+  - [x] 10.2 Update `SROverlay` and `TickerDetailPage` to pass zones to `CandlestickChart`
+    - Update `useTickerDetail` hook or `SROverlay` to extract zones from the S/R API response
+    - Pass zones array to `CandlestickChart` component
+    - Default to max 6 zones (handled by backend `max_zones=6` default)
+    - _Requirements: 8.3, 9.1_
+
+  - [x] 10.3 Ensure S/R overlays re-render correctly at all zoom levels
+    - Verify zone rectangles reposition when zoom/pan changes the visible price scale
+    - Handle single-level zones (low == high) as thin 2px-height rectangles
+    - _Requirements: 5.6, 9.5_
+
+- [x] 11. Final checkpoint — All features integrated
+  - Ensure all tests pass, ask the user if questions arise.
+
+## Notes
+
+- Tasks marked with `*` are optional and can be skipped for faster MVP
+- Each task references specific requirements for traceability
+- Checkpoints ensure incremental validation after each feature area
+- Property tests validate universal correctness properties from the design document
+- The Alembic migration for sentiment and fundamentals columns should ideally be a single migration file
+- S/R zones are computed on-the-fly (not persisted), so no additional migration is needed for zones