major update

2026-02-27 16:08:09 +01:00
parent 61ab24490d
commit 181cfe6588
71 changed files with 7647 additions and 281 deletions
--- a/.kiro/specs/dashboard-enhancements/.config.kiro
+++ b/.kiro/specs/dashboard-enhancements/.config.kiro
@@ -0,0 +1 @@
+{"specId": "9b39d94f-51e1-42d3-bacc-68eb3961f2b1", "workflowType": "requirements-first", "specType": "feature"}
--- a/.kiro/specs/dashboard-enhancements/design.md
+++ b/.kiro/specs/dashboard-enhancements/design.md
@@ -0,0 +1,502 @@
+# Design Document — Dashboard Enhancements
+
+## Overview
+
+This design covers four enhancements to the TickerDetailPage dashboard:
+
+1. **Sentiment drill-down** — Store OpenAI reasoning text and web search citations in the DB; expose via API; render in an expandable detail section within SentimentPanel.
+2. **Fundamentals drill-down** — Track which FMP endpoints returned 402 (paid-plan-required) and surface those reasons in the API and an expandable detail section within FundamentalsPanel.
+3. **TradingView-style chart** — Add mouse-wheel zoom, click-drag pan, and a crosshair overlay with price/date labels to the existing canvas-based CandlestickChart.
+4. **S/R clustering** — Cluster nearby S/R levels into zones with aggregated strength, filter to top N, and render as shaded rectangles instead of dashed lines.
+
+All changes are additive to existing components and preserve the glassmorphism UI style.
+
+## Architecture
+
+```mermaid
+graph TD
+    subgraph Backend
+        OAI[OpenAI Responses API] -->|reasoning + annotations| SP[OpenAISentimentProvider]
+        SP -->|SentimentData + reasoning + citations| SS[sentiment_service]
+        SS -->|persist| DB[(PostgreSQL)]
+
+        FMP[FMP Stable API] -->|402 metadata| FP[FMPFundamentalProvider]
+        FP -->|FundamentalData + unavailable_fields| FS[fundamental_service]
+        FS -->|persist| DB
+
+        DB -->|query| SRS[sr_service]
+        SRS -->|cluster_sr_zones| SRS
+        SRS -->|SRZone list| SRAPI[/sr-levels endpoint/]
+    end
+
+    subgraph Frontend
+        SRAPI -->|zones JSON| Chart[CandlestickChart]
+        Chart -->|canvas render| ZR[Zone rectangles + crosshair + zoom/pan]
+
+        SS -->|API| SentAPI[/sentiment endpoint/]
+        SentAPI -->|reasoning + citations| SPan[SentimentPanel]
+        SPan -->|expand/collapse| DD1[Detail Section]
+
+        FS -->|API| FundAPI[/fundamentals endpoint/]
+        FundAPI -->|unavailable_fields| FPan[FundamentalsPanel]
+        FPan -->|expand/collapse| DD2[Detail Section]
+    end
+```
+
+The changes touch four layers:
+- **Provider layer** — Extract additional data from external API responses (OpenAI annotations, FMP 402 reasons).
+- **Service layer** — Store new fields, add zone clustering logic.
+- **API/Schema layer** — Extend response schemas with new fields.
+- **Frontend components** — Add interactive chart features and expandable detail sections.
+
+## Components and Interfaces
+
+### 1. Sentiment Provider Changes (`app/providers/openai_sentiment.py`)
+
+The `SentimentData` DTO gains two optional fields:
+
+```python
+@dataclass(frozen=True, slots=True)
+class SentimentData:
+    ticker: str
+    classification: str
+    confidence: int
+    source: str
+    timestamp: datetime
+    reasoning: str = ""                          # NEW
+    citations: list[dict[str, str]] = field(default_factory=list)  # NEW: [{"url": ..., "title": ...}]
+```
+
+The provider already parses `reasoning` from the JSON response but discards it. The change:
+- Return `reasoning` from the parsed JSON in the `SentimentData`.
+- Iterate `response.output` items looking for `type == "web_search_call"` output items, then extract URL annotations from the subsequent message content blocks that have `annotations` with `type == "url_citation"`. Each annotation yields `{"url": annotation.url, "title": annotation.title}`.
+- If no annotations exist, return an empty list (no error).
+
+### 2. Sentiment DB Model Changes (`app/models/sentiment.py`)
+
+Add two columns to `SentimentScore`:
+
+```python
+reasoning: Mapped[str] = mapped_column(Text, nullable=False, default="")
+citations_json: Mapped[str] = mapped_column(Text, nullable=False, default="[]")
+```
+
+Citations are stored as a JSON-encoded string (list of `{url, title}` dicts). This avoids a separate table for a simple list of links.
+
+Alembic migration adds these two columns with defaults so existing rows are unaffected.
+
+### 3. Sentiment Service Changes (`app/services/sentiment_service.py`)
+
+`store_sentiment()` gains `reasoning: str` and `citations: list[dict]` parameters. It serializes citations to JSON and stores both fields.
+
+### 4. Sentiment Schema Changes (`app/schemas/sentiment.py`)
+
+```python
+class CitationItem(BaseModel):
+    url: str
+    title: str
+
+class SentimentScoreResult(BaseModel):
+    id: int
+    classification: Literal["bullish", "bearish", "neutral"]
+    confidence: int = Field(ge=0, le=100)
+    source: str
+    timestamp: datetime
+    reasoning: str = ""                    # NEW
+    citations: list[CitationItem] = []     # NEW
+```
+
+### 5. FMP Provider Changes (`app/providers/fmp.py`)
+
+`FundamentalData` DTO gains an `unavailable_fields` dict:
+
+```python
+@dataclass(frozen=True, slots=True)
+class FundamentalData:
+    ticker: str
+    pe_ratio: float | None
+    revenue_growth: float | None
+    earnings_surprise: float | None
+    market_cap: float | None
+    fetched_at: datetime
+    unavailable_fields: dict[str, str] = field(default_factory=dict)  # NEW: {"pe_ratio": "requires paid plan", ...}
+```
+
+In `_fetch_json_optional`, when a 402 is received, the provider records which fields map to that endpoint. The mapping:
+- `ratios-ttm` → `pe_ratio`
+- `financial-growth` → `revenue_growth`
+- `earnings` → `earnings_surprise`
+
+After all fetches, any field that is `None` AND whose endpoint returned 402 gets an entry in `unavailable_fields`.
+
+### 6. Fundamentals DB Model Changes (`app/models/fundamental.py`)
+
+Add one column:
+
+```python
+unavailable_fields_json: Mapped[str] = mapped_column(Text, nullable=False, default="{}")
+```
+
+Stored as JSON-encoded `{"field_name": "reason"}` dict.
+
+### 7. Fundamentals Schema Changes (`app/schemas/fundamental.py`)
+
+```python
+class FundamentalResponse(BaseModel):
+    symbol: str
+    pe_ratio: float | None = None
+    revenue_growth: float | None = None
+    earnings_surprise: float | None = None
+    market_cap: float | None = None
+    fetched_at: datetime | None = None
+    unavailable_fields: dict[str, str] = {}  # NEW
+```
+
+### 8. S/R Zone Clustering (`app/services/sr_service.py`)
+
+New function `cluster_sr_zones()`:
+
+```python
+def cluster_sr_zones(
+    levels: list[dict],
+    current_price: float,
+    tolerance: float = 0.02,  # 2% default clustering tolerance
+    max_zones: int | None = None,
+) -> list[dict]:
+    """Cluster nearby S/R levels into zones.
+    
+    Returns list of zone dicts:
+    {
+        "low": float,
+        "high": float,
+        "midpoint": float,
+        "strength": int,  # sum of constituent strengths, capped at 100
+        "type": "support" | "resistance",
+        "level_count": int,
+    }
+    """
+```
+
+Algorithm:
+1. Sort levels by `price_level` ascending.
+2. Greedy merge: walk sorted levels; if the next level is within `tolerance` (percentage of the current cluster midpoint) of the current cluster, merge it in. Otherwise, start a new cluster.
+3. For each cluster: `low` = min price, `high` = max price, `midpoint` = (low + high) / 2, `strength` = sum of constituent strengths capped at 100.
+4. Tag each zone as `"support"` if midpoint < current_price, else `"resistance"`.
+5. Sort by strength descending.
+6. If `max_zones` is set, return only the top N.
+
+### 9. S/R Schema Changes (`app/schemas/sr_level.py`)
+
+```python
+class SRZoneResult(BaseModel):
+    low: float
+    high: float
+    midpoint: float
+    strength: int = Field(ge=0, le=100)
+    type: Literal["support", "resistance"]
+    level_count: int
+
+class SRLevelResponse(BaseModel):
+    symbol: str
+    levels: list[SRLevelResult]
+    zones: list[SRZoneResult] = []  # NEW
+    count: int
+```
+
+### 10. S/R Router Changes (`app/routers/sr_levels.py`)
+
+Add `max_zones` query parameter (default 6). After fetching levels, call `cluster_sr_zones()` and include zones in the response.
+
+### 11. CandlestickChart Enhancements (`frontend/src/components/charts/CandlestickChart.tsx`)
+
+State additions:
+- `visibleRange: { start: number, end: number }` — indices into the data array for the currently visible window.
+- `isPanning: boolean`, `panStartX: number` — for drag-to-pan.
+- `crosshair: { x: number, y: number } | null` — cursor position for crosshair rendering.
+
+New event handlers:
+- `onWheel` — Adjust `visibleRange` (zoom in = narrow range, zoom out = widen range). Clamp to min 10 bars, max full dataset.
+- `onMouseDown` / `onMouseMove` / `onMouseUp` — When zoomed in, click-drag pans the visible range left/right.
+- `onMouseMove` (extended) — Track cursor position for crosshair. Draw vertical + horizontal lines and axis labels.
+- `onMouseLeave` — Clear crosshair state.
+
+The `draw()` function changes:
+- Use `data.slice(visibleRange.start, visibleRange.end)` instead of full `data`.
+- After drawing candles, if `crosshair` is set, draw crosshair lines and labels.
+- Replace S/R dashed lines with shaded zone rectangles when `zones` prop is provided.
+
+New prop: `zones?: SRZone[]` (from the API response).
+
+### 12. SentimentPanel Drill-Down (`frontend/src/components/ticker/SentimentPanel.tsx`)
+
+Add `useState<boolean>(false)` for expand/collapse. When expanded, render:
+- Reasoning text in a `<p>` block.
+- Citations as a list of `<a>` links with title and URL.
+
+Toggle button uses a chevron icon below the summary metrics.
+
+### 13. FundamentalsPanel Drill-Down (`frontend/src/components/ticker/FundamentalsPanel.tsx`)
+
+Add `useState<boolean>(false)` for expand/collapse. Changes:
+- When a metric is `null` and `unavailable_fields[field_name]` exists, show the reason text (e.g., "Requires paid plan") in amber instead of "—".
+- When expanded, show data source name ("FMP"), fetch timestamp, and a list of unavailable fields with reasons.
+
+### 14. Frontend Type Updates (`frontend/src/lib/types.ts`)
+
+```typescript
+// Sentiment additions
+export interface CitationItem {
+  url: string;
+  title: string;
+}
+
+export interface SentimentScore {
+  id: number;
+  classification: 'bullish' | 'bearish' | 'neutral';
+  confidence: number;
+  source: string;
+  timestamp: string;
+  reasoning: string;       // NEW
+  citations: CitationItem[]; // NEW
+}
+
+// Fundamentals additions
+export interface FundamentalResponse {
+  symbol: string;
+  pe_ratio: number | null;
+  revenue_growth: number | null;
+  earnings_surprise: number | null;
+  market_cap: number | null;
+  fetched_at: string | null;
+  unavailable_fields: Record<string, string>; // NEW
+}
+
+// S/R Zone
+export interface SRZone {
+  low: number;
+  high: number;
+  midpoint: number;
+  strength: number;
+  type: 'support' | 'resistance';
+  level_count: number;
+}
+
+export interface SRLevelResponse {
+  symbol: string;
+  levels: SRLevel[];
+  zones: SRZone[];  // NEW
+  count: number;
+}
+```
+
+## Data Models
+
+### Database Schema Changes
+
+#### `sentiment_scores` table — new columns
+
+| Column | Type | Default | Description |
+|--------|------|---------|-------------|
+| `reasoning` | TEXT | `""` | AI reasoning text from OpenAI response |
+| `citations_json` | TEXT | `"[]"` | JSON array of `{url, title}` citation objects |
+
+#### `fundamental_data` table — new column
+
+| Column | Type | Default | Description |
+|--------|------|---------|-------------|
+| `unavailable_fields_json` | TEXT | `"{}"` | JSON dict of `{field_name: reason}` for missing data |
+
+No new tables are needed. The S/R zones are computed on-the-fly from existing `sr_levels` rows — they are not persisted.
+
+### Alembic Migration
+
+A single migration file adds the three new columns with server defaults so existing rows are populated automatically:
+
+```python
+op.add_column('sentiment_scores', sa.Column('reasoning', sa.Text(), server_default='', nullable=False))
+op.add_column('sentiment_scores', sa.Column('citations_json', sa.Text(), server_default='[]', nullable=False))
+op.add_column('fundamental_data', sa.Column('unavailable_fields_json', sa.Text(), server_default='{}', nullable=False))
+```
+
+
+## Correctness Properties
+
+*A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.*
+
+### Property 1: Sentiment reasoning extraction
+
+*For any* valid OpenAI Responses API response containing a JSON body with a `reasoning` field, the `OpenAISentimentProvider.fetch_sentiment()` method should return a `SentimentData` whose `reasoning` field equals the reasoning string from the parsed JSON.
+
+**Validates: Requirements 1.1**
+
+### Property 2: Sentiment citations extraction
+
+*For any* valid OpenAI Responses API response containing zero or more `url_citation` annotations across its output items, the `OpenAISentimentProvider.fetch_sentiment()` method should return a `SentimentData` whose `citations` list contains exactly the URLs and titles from those annotations, in order. When no annotations exist, the citations list should be empty (no error raised).
+
+**Validates: Requirements 1.2, 1.4**
+
+### Property 3: Sentiment data round-trip
+
+*For any* sentiment record with arbitrary reasoning text and citations list, storing it via `store_sentiment()` and then retrieving it via the `/sentiment/{symbol}` API endpoint should return a response where the latest score's `reasoning` and `citations` fields match the originally stored values.
+
+**Validates: Requirements 1.3**
+
+### Property 4: Expanded sentiment detail displays all data
+
+*For any* `SentimentScore` with non-empty reasoning and a list of citations, when the SentimentPanel detail section is expanded, the rendered output should contain the reasoning text and every citation's title and URL as clickable links.
+
+**Validates: Requirements 2.2, 2.3**
+
+### Property 5: Sentiment detail collapse hides content
+
+*For any* SentimentPanel state where the detail section is expanded, collapsing it should result in the reasoning text and citations being hidden from the DOM while the summary metrics (classification, confidence, dimension score, source count) remain visible.
+
+**Validates: Requirements 2.4**
+
+### Property 6: FMP 402 reason recording
+
+*For any* subset of supplementary FMP endpoints (ratios-ttm, financial-growth, earnings) that return HTTP 402, the `FMPFundamentalProvider.fetch_fundamentals()` method should return a `FundamentalData` whose `unavailable_fields` dict contains an entry for each corresponding metric field name with the reason "requires paid plan".
+
+**Validates: Requirements 3.1**
+
+### Property 7: Fundamentals unavailable_fields round-trip
+
+*For any* fundamental data record with an arbitrary `unavailable_fields` dict, storing it via `store_fundamental()` and retrieving it via the `/fundamentals/{symbol}` API endpoint should return a response whose `unavailable_fields` matches the originally stored dict.
+
+**Validates: Requirements 3.2**
+
+### Property 8: Null field display depends on reason existence
+
+*For any* `FundamentalResponse` where a metric field is null, the FundamentalsPanel should display the reason text from `unavailable_fields` (if present for that field) or a dash character "—" (if no reason exists for that field).
+
+**Validates: Requirements 3.3, 3.4**
+
+### Property 9: Fundamentals expanded detail content
+
+*For any* `FundamentalResponse` with a fetch timestamp and unavailable fields, when the FundamentalsPanel detail section is expanded, the rendered output should contain the data source name, the formatted fetch timestamp, and each unavailable field's name and reason.
+
+**Validates: Requirements 4.2**
+
+### Property 10: Zoom adjusts visible range proportionally
+
+*For any* dataset of length N (N ≥ 10) and any current visible range [start, end], applying a positive wheel delta (zoom in) should produce a new range that is strictly narrower (fewer bars), and applying a negative wheel delta (zoom out) should produce a new range that is strictly wider (more bars), unless already at the limit.
+
+**Validates: Requirements 5.1, 5.2, 5.3**
+
+### Property 11: Pan shifts visible range
+
+*For any* dataset and any visible range that does not cover the full dataset, a horizontal drag of Δx pixels should shift the visible range start and end indices by a proportional amount in the corresponding direction, without changing the range width.
+
+**Validates: Requirements 5.4**
+
+### Property 12: Zoom range invariant
+
+*For any* sequence of zoom and pan operations on a dataset of length N, the visible range should always satisfy: `end - start >= 10` AND `end - start <= N` AND `start >= 0` AND `end <= N`.
+
+**Validates: Requirements 5.5**
+
+### Property 13: Coordinate-to-value mapping
+
+*For any* chart configuration with a visible price range [lo, hi] and visible data slice, the `yToPrice` function should map any y-coordinate within the chart area to a price within [lo, hi], and the `xToBarIndex` function should map any x-coordinate within the chart area to a valid index within the visible data slice.
+
+**Validates: Requirements 6.3, 6.4**
+
+### Property 14: Clustering merges nearby levels
+
+*For any* set of S/R levels and a clustering tolerance T, after calling `cluster_sr_zones()`, no two distinct zones should have midpoints within T percent of each other. Equivalently, all input levels that are within T percent of each other must end up in the same zone.
+
+**Validates: Requirements 7.2**
+
+### Property 15: Zone strength is capped sum
+
+*For any* SR zone produced by `cluster_sr_zones()`, the zone's strength should equal `min(100, sum(constituent_level_strengths))`.
+
+**Validates: Requirements 7.3**
+
+### Property 16: Zone type tagging
+
+*For any* SR zone and current price, the zone's type should be `"support"` if the zone midpoint is less than the current price, and `"resistance"` otherwise.
+
+**Validates: Requirements 7.4**
+
+### Property 17: Zone filtering returns top N by strength
+
+*For any* set of SR zones and a limit N, `cluster_sr_zones(..., max_zones=N)` should return at most N zones, and those zones should be the N zones with the highest strength scores from the full unfiltered set.
+
+**Validates: Requirements 8.2**
+
+## Error Handling
+
+### Backend
+
+| Scenario | Handling |
+|----------|----------|
+| OpenAI response has no `reasoning` field in JSON | Default to empty string `""` — no error |
+| OpenAI response has no `url_citation` annotations | Return empty citations list — no error |
+| OpenAI response JSON parse failure | Existing `ProviderError` handling unchanged |
+| FMP endpoint returns 402 | Record in `unavailable_fields`, return `None` for that metric — no error |
+| FMP profile endpoint fails | Existing `ProviderError` propagation unchanged |
+| `citations_json` column contains invalid JSON | Catch `json.JSONDecodeError` in schema serialization, default to `[]` |
+| `unavailable_fields_json` column contains invalid JSON | Catch `json.JSONDecodeError`, default to `{}` |
+| `cluster_sr_zones()` receives empty levels list | Return empty zones list |
+| `max_zones` is 0 or negative | Return empty zones list |
+
+### Frontend
+
+| Scenario | Handling |
+|----------|----------|
+| `reasoning` is empty string | Detail section shows "No reasoning available" placeholder |
+| `citations` is empty array | Detail section omits citations subsection |
+| `unavailable_fields` is empty object | All null metrics show "—" as before |
+| Chart data has fewer than 10 bars | Disable zoom (show all bars, no zoom controls) |
+| Wheel event fires rapidly | Debounce zoom recalculation to 1 frame via `requestAnimationFrame` |
+| Zone `low` equals `high` (single-level zone) | Render as a thin line (minimum 2px height rectangle) |
+
+## Testing Strategy
+
+### Property-Based Testing
+
+Library: **Hypothesis** (Python backend), **fast-check** (TypeScript frontend)
+
+Each property test runs a minimum of 100 iterations. Each test is tagged with a comment referencing the design property:
+
+```
+# Feature: dashboard-enhancements, Property 14: Clustering merges nearby levels
+```
+
+Backend property tests (Hypothesis):
+- **Property 1**: Generate random JSON strings with reasoning fields → verify extraction.
+- **Property 2**: Generate mock OpenAI response objects with 0–10 annotations → verify citations list.
+- **Property 3**: Generate random reasoning + citations → store → retrieve via test client → compare.
+- **Property 6**: Generate random 402/200 combinations for 3 endpoints → verify unavailable_fields mapping.
+- **Property 7**: Generate random unavailable_fields dicts → store → retrieve → compare.
+- **Property 10–12**: Generate random datasets (10–500 bars) and zoom/pan sequences → verify range invariants.
+- **Property 13**: Generate random chart dimensions and price ranges → verify coordinate mapping round-trips.
+- **Property 14**: Generate random level sets (1–50 levels, prices 1–1000, strengths 1–100) and tolerances (0.5%–5%) → verify no two zones should have been merged.
+- **Property 15**: Generate random level sets → cluster → verify each zone's strength = min(100, sum).
+- **Property 16**: Generate random zones and current prices → verify type tagging.
+- **Property 17**: Generate random zone sets and limits → verify top-N selection.
+
+Frontend property tests (fast-check):
+- **Property 4**: Generate random reasoning strings and citation lists → render SentimentPanel expanded → verify DOM content.
+- **Property 5**: Generate random sentiment data → expand then collapse → verify summary visible, detail hidden.
+- **Property 8**: Generate random FundamentalResponse with various null/non-null + reason combinations → verify displayed text.
+- **Property 9**: Generate random FundamentalResponse → expand → verify source, timestamp, reasons in DOM.
+
+### Unit Tests
+
+Unit tests cover specific examples and edge cases:
+- Sentiment provider with a real-shaped OpenAI response fixture (example for 1.1, 1.2).
+- Sentiment provider with no annotations (edge case for 1.4).
+- FMP provider with all-402 responses (edge case for 3.1).
+- FMP provider with mixed 200/402 responses (example for 3.1).
+- SentimentPanel default collapsed state (example for 2.5).
+- FundamentalsPanel default collapsed state (example for 4.4).
+- Chart with exactly 10 bars — zoom in should be blocked (edge case for 5.5).
+- Chart with 1 bar — zoom disabled entirely (edge case).
+- Crosshair removed on mouse leave (example for 6.5).
+- `cluster_sr_zones()` with empty input (edge case).
+- `cluster_sr_zones()` with all levels at the same price (edge case).
+- `cluster_sr_zones()` with levels exactly at tolerance boundary (edge case).
+- Default max_zones = 6 in the dashboard (example for 8.3).
+- Zone with single constituent level (edge case — low == high).
--- a/.kiro/specs/dashboard-enhancements/requirements.md
+++ b/.kiro/specs/dashboard-enhancements/requirements.md
@@ -0,0 +1,124 @@
+# Requirements Document
+
+## Introduction
+
+This specification covers four dashboard enhancements for the stock signal platform: sentiment drill-down with OpenAI response details, fundamentals drill-down with missing-data transparency, TradingView-style chart improvements (zoom, crosshair), and S/R level clustering into filterable shaded zones. All changes target the existing TickerDetailPage and its child components, preserving the glassmorphism UI style.
+
+## Glossary
+
+- **Dashboard**: The TickerDetailPage in the React frontend that displays ticker data, charts, scores, sentiment, and fundamentals.
+- **Sentiment_Panel**: The SentimentPanel component that displays classification, confidence, dimension score, and source count for a ticker.
+- **Fundamentals_Panel**: The FundamentalsPanel component that displays P/E Ratio, Revenue Growth, Earnings Surprise, and Market Cap for a ticker.
+- **Chart_Component**: The CandlestickChart canvas-based component that renders OHLCV candlesticks and S/R level overlays.
+- **SR_Service**: The backend service (sr_service.py) that detects, scores, merges, and tags support/resistance levels from OHLCV data.
+- **Sentiment_Provider**: The OpenAISentimentProvider that calls the OpenAI Responses API with web_search_preview to produce sentiment classifications.
+- **FMP_Provider**: The FMPFundamentalProvider that fetches fundamental data from Financial Modeling Prep stable endpoints.
+- **SR_Zone**: A price range representing a cluster of nearby S/R levels, displayed as a shaded area on the chart instead of individual lines.
+- **Detail_Section**: A collapsible/expandable UI region within a panel that reveals additional information on user interaction.
+- **Data_Availability_Indicator**: A visual element within the Fundamentals_Panel that communicates which data fields are unavailable and the reason.
+- **Crosshair**: A vertical and horizontal line overlay on the Chart_Component that tracks the cursor position and displays corresponding price and date values.
+
+## Requirements
+
+### Requirement 1: Sentiment Detail Storage
+
+**User Story:** As a developer, I want the backend to store the full OpenAI response details (reasoning text, web search citations, and annotations) alongside the sentiment classification, so that the frontend can display drill-down information.
+
+#### Acceptance Criteria
+
+1. WHEN the Sentiment_Provider receives a response from the OpenAI Responses API, THE Sentiment_Provider SHALL extract and return the reasoning text from the parsed JSON response.
+2. WHEN the Sentiment_Provider receives a response containing web_search_preview output items, THE Sentiment_Provider SHALL extract and return the list of source URLs and titles from the search result annotations.
+3. THE sentiment API endpoint SHALL include the reasoning text and citations list in the response payload for each sentiment score.
+4. IF the OpenAI response contains no annotations or citations, THEN THE Sentiment_Provider SHALL return an empty citations list without raising an error.
+
+### Requirement 2: Sentiment Drill-Down UI
+
+**User Story:** As a user, I want to drill into the sentiment analysis to see the AI reasoning and source citations, so that I can evaluate the quality of the sentiment classification.
+
+#### Acceptance Criteria
+
+1. THE Sentiment_Panel SHALL display a clickable expand/collapse toggle below the summary metrics.
+2. WHEN the user expands the Detail_Section, THE Sentiment_Panel SHALL display the reasoning text from the latest sentiment score.
+3. WHEN the user expands the Detail_Section and citations are available, THE Sentiment_Panel SHALL display each citation as a clickable link showing the source title and URL.
+4. WHEN the user collapses the Detail_Section, THE Sentiment_Panel SHALL hide the reasoning and citations without removing the summary metrics.
+5. THE Detail_Section SHALL default to the collapsed state on initial render.
+
+### Requirement 3: Fundamentals Data Availability Transparency
+
+**User Story:** As a user, I want to understand why certain fundamental metrics are missing for a ticker, so that I can distinguish between "data not available from provider" and "data not fetched."
+
+#### Acceptance Criteria
+
+1. WHEN the FMP_Provider receives an HTTP 402 response for a supplementary endpoint, THE FMP_Provider SHALL record the endpoint name and the reason "requires paid plan" in the response metadata.
+2. THE fundamentals API endpoint SHALL include a field listing which data fields are unavailable and the corresponding reason for each.
+3. WHEN a fundamental metric value is null and a corresponding unavailability reason exists, THE Fundamentals_Panel SHALL display the reason text (e.g., "Requires paid plan") instead of a dash character.
+4. WHEN a fundamental metric value is null and no unavailability reason exists, THE Fundamentals_Panel SHALL display a dash character as the placeholder.
+
+### Requirement 4: Fundamentals Drill-Down UI
+
+**User Story:** As a user, I want to drill into the fundamentals data to see additional detail and data source information, so that I can better assess the fundamental metrics.
+
+#### Acceptance Criteria
+
+1. THE Fundamentals_Panel SHALL display a clickable expand/collapse toggle below the summary metrics.
+2. WHEN the user expands the Detail_Section, THE Fundamentals_Panel SHALL display the data source name, the fetch timestamp, and any unavailability reasons for missing fields.
+3. WHEN the user collapses the Detail_Section, THE Fundamentals_Panel SHALL hide the detail information without removing the summary metrics.
+4. THE Detail_Section SHALL default to the collapsed state on initial render.
+
+### Requirement 5: Chart Zoom Capability
+
+**User Story:** As a user, I want to zoom in and out on the candlestick chart, so that I can examine specific time periods in detail or see the full price history.
+
+#### Acceptance Criteria
+
+1. WHEN the user scrolls the mouse wheel over the Chart_Component, THE Chart_Component SHALL zoom in or out by adjusting the visible date range.
+2. WHEN the user zooms in, THE Chart_Component SHALL increase the candle width and reduce the number of visible bars proportionally.
+3. WHEN the user zooms out, THE Chart_Component SHALL decrease the candle width and increase the number of visible bars proportionally.
+4. WHEN the chart is zoomed in, THE Chart_Component SHALL allow the user to pan left and right by clicking and dragging horizontally.
+5. THE Chart_Component SHALL constrain zoom limits so that the minimum visible range is 10 bars and the maximum visible range is the full dataset length.
+6. THE Chart_Component SHALL re-render S/R overlays correctly at every zoom level.
+
+### Requirement 6: Chart Crosshair
+
+**User Story:** As a user, I want a crosshair overlay on the chart that tracks my cursor, so that I can precisely read price and date values at any point.
+
+#### Acceptance Criteria
+
+1. WHEN the user moves the cursor over the Chart_Component, THE Chart_Component SHALL draw a vertical line at the cursor x-position spanning the full chart height.
+2. WHEN the user moves the cursor over the Chart_Component, THE Chart_Component SHALL draw a horizontal line at the cursor y-position spanning the full chart width.
+3. THE Chart_Component SHALL display the corresponding price value as a label on the y-axis at the horizontal crosshair position.
+4. THE Chart_Component SHALL display the corresponding date value as a label on the x-axis at the vertical crosshair position.
+5. WHEN the cursor leaves the Chart_Component, THE Chart_Component SHALL remove the crosshair lines and labels.
+
+### Requirement 7: S/R Level Clustering
+
+**User Story:** As a user, I want nearby S/R levels to be clustered into zones, so that the chart is less cluttered and I can focus on the most significant price areas.
+
+#### Acceptance Criteria
+
+1. THE SR_Service SHALL accept a configurable clustering tolerance parameter that defines the maximum price distance (as a percentage) for grouping levels into a single SR_Zone.
+2. WHEN two or more S/R levels fall within the clustering tolerance of each other, THE SR_Service SHALL merge those levels into a single SR_Zone with a price range (low bound, high bound) and an aggregated strength score.
+3. THE SR_Service SHALL compute the aggregated strength of an SR_Zone as the sum of constituent level strengths, capped at 100.
+4. THE SR_Service SHALL tag each SR_Zone as "support" or "resistance" based on the zone midpoint relative to the current price.
+
+### Requirement 8: S/R Zone Filtering
+
+**User Story:** As a user, I want to see only the strongest S/R zones on the chart, so that I can focus on the most significant price areas.
+
+#### Acceptance Criteria
+
+1. THE S/R API endpoint SHALL accept an optional parameter to limit the number of returned zones.
+2. WHEN a zone limit is specified, THE SR_Service SHALL return only the zones with the highest aggregated strength scores, up to the specified limit.
+3. THE Dashboard SHALL default to displaying a maximum of 6 SR_Zones on the chart.
+
+### Requirement 9: S/R Zone Chart Rendering
+
+**User Story:** As a user, I want S/R zones displayed as shaded areas on the chart instead of individual lines, so that I can visually identify price ranges of significance.
+
+#### Acceptance Criteria
+
+1. THE Chart_Component SHALL render each SR_Zone as a semi-transparent shaded rectangle spanning the zone price range (low bound to high bound) across the full chart width.
+2. THE Chart_Component SHALL use green shading for support zones and red shading for resistance zones.
+3. THE Chart_Component SHALL display a label for each SR_Zone showing the zone midpoint price and strength score.
+4. THE Chart_Component SHALL render SR_Zones behind the candlestick bodies so that candles remain fully visible.
+5. WHEN the chart is zoomed, THE Chart_Component SHALL re-render SR_Zones at the correct vertical positions for the current price scale.
--- a/.kiro/specs/dashboard-enhancements/tasks.md
+++ b/.kiro/specs/dashboard-enhancements/tasks.md
@@ -0,0 +1,219 @@
+# Implementation Plan: Dashboard Enhancements
+
+## Overview
+
+Incremental implementation of four dashboard enhancements: sentiment drill-down, fundamentals drill-down, chart zoom/crosshair, and S/R zone clustering. Each feature area is built backend-first (model → service → schema → router) then frontend, with tests alongside implementation. All changes are additive to existing components.
+
+## Tasks
+
+- [x] 1. Sentiment drill-down — backend
+  - [x] 1.1 Add `reasoning` and `citations_json` columns to `SentimentScore` model and create Alembic migration
+    - Add `reasoning: Mapped[str] = mapped_column(Text, nullable=False, default="")` and `citations_json: Mapped[str] = mapped_column(Text, nullable=False, default="[]")` to `app/models/sentiment.py`
+    - Create Alembic migration with `server_default` so existing rows are backfilled
+    - _Requirements: 1.1, 1.2, 1.3_
+
+  - [x] 1.2 Update `OpenAISentimentProvider` to extract reasoning and citations from OpenAI response
+    - Add `reasoning` and `citations` fields to the `SentimentData` dataclass
+    - Extract `reasoning` from the parsed JSON response body
+    - Iterate `response.output` items for `url_citation` annotations, collect `{"url": ..., "title": ...}` dicts
+    - Return empty citations list when no annotations exist (no error)
+    - _Requirements: 1.1, 1.2, 1.4_
+
+  - [x] 1.3 Update `sentiment_service.store_sentiment()` to persist reasoning and citations
+    - Accept `reasoning` and `citations` parameters
+    - Serialize citations to JSON string before storing
+    - _Requirements: 1.3_
+
+  - [x] 1.4 Update sentiment schema and router to include reasoning and citations in API response
+    - Add `CitationItem` model and `reasoning`/`citations` fields to `SentimentScoreResult` in `app/schemas/sentiment.py`
+    - Deserialize `citations_json` when building the response, catch `JSONDecodeError` and default to `[]`
+    - _Requirements: 1.3_
+
+  - [ ]* 1.5 Write property tests for sentiment reasoning and citations extraction
+    - **Property 1: Sentiment reasoning extraction** — Generate random JSON with reasoning fields, verify extraction
+    - **Validates: Requirements 1.1**
+    - **Property 2: Sentiment citations extraction** — Generate mock OpenAI responses with 0–10 annotations, verify citations list
+    - **Validates: Requirements 1.2, 1.4**
+
+  - [ ]* 1.6 Write property test for sentiment data round-trip
+    - **Property 3: Sentiment data round-trip** — Generate random reasoning + citations, store, retrieve via test client, compare
+    - **Validates: Requirements 1.3**
+
+- [x] 2. Sentiment drill-down — frontend
+  - [x] 2.1 Add `CitationItem`, `reasoning`, and `citations` fields to `SentimentScore` type in `frontend/src/lib/types.ts`
+    - _Requirements: 1.3, 2.2, 2.3_
+
+  - [x] 2.2 Add expandable detail section to `SentimentPanel`
+    - Add `useState<boolean>(false)` for expand/collapse toggle
+    - Render chevron toggle button below summary metrics
+    - When expanded: show reasoning text (or "No reasoning available" placeholder if empty) and citations as clickable `<a>` links
+    - When collapsed: hide reasoning and citations, keep summary metrics visible
+    - Default to collapsed state on initial render
+    - Preserve glassmorphism UI style
+    - _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5_
+
+  - [ ]* 2.3 Write property tests for SentimentPanel drill-down
+    - **Property 4: Expanded sentiment detail displays all data** — Generate random reasoning and citations, render expanded, verify DOM content
+    - **Validates: Requirements 2.2, 2.3**
+    - **Property 5: Sentiment detail collapse hides content** — Expand then collapse, verify summary visible and detail hidden
+    - **Validates: Requirements 2.4**
+
+- [x] 3. Checkpoint — Sentiment drill-down complete
+  - Ensure all tests pass, ask the user if questions arise.
+
+- [x] 4. Fundamentals drill-down — backend
+  - [x] 4.1 Add `unavailable_fields_json` column to fundamental model and create Alembic migration
+    - Add `unavailable_fields_json: Mapped[str] = mapped_column(Text, nullable=False, default="{}")` to `app/models/fundamental.py`
+    - Add column to the same Alembic migration as sentiment columns (or a new one if migration 1.1 is already applied), with `server_default='{}'`
+    - _Requirements: 3.1, 3.2_
+
+  - [x] 4.2 Update `FMPFundamentalProvider` to record 402 reasons in `unavailable_fields`
+    - Add `unavailable_fields` field to `FundamentalData` dataclass
+    - In `_fetch_json_optional`, when HTTP 402 is received, map endpoint to field name: `ratios-ttm` → `pe_ratio`, `financial-growth` → `revenue_growth`, `earnings` → `earnings_surprise`
+    - Record `"requires paid plan"` as the reason for each affected field
+    - _Requirements: 3.1_
+
+  - [x] 4.3 Update `fundamental_service` to persist `unavailable_fields`
+    - Serialize `unavailable_fields` dict to JSON string before storing
+    - _Requirements: 3.2_
+
+  - [x] 4.4 Update fundamentals schema and router to include `unavailable_fields` in API response
+    - Add `unavailable_fields: dict[str, str] = {}` to `FundamentalResponse` in `app/schemas/fundamental.py`
+    - Deserialize `unavailable_fields_json` when building the response, catch `JSONDecodeError` and default to `{}`
+    - _Requirements: 3.2_
+
+  - [ ]* 4.5 Write property tests for FMP 402 reason recording and round-trip
+    - **Property 6: FMP 402 reason recording** — Generate random 402/200 combinations for 3 endpoints, verify unavailable_fields mapping
+    - **Validates: Requirements 3.1**
+    - **Property 7: Fundamentals unavailable_fields round-trip** — Generate random dicts, store, retrieve, compare
+    - **Validates: Requirements 3.2**
+
+- [x] 5. Fundamentals drill-down — frontend
+  - [x] 5.1 Add `unavailable_fields` to `FundamentalResponse` type in `frontend/src/lib/types.ts`
+    - _Requirements: 3.2, 3.3_
+
+  - [x] 5.2 Update `FundamentalsPanel` to show unavailability reasons and expandable detail section
+    - When a metric is null and `unavailable_fields[field_name]` exists, display reason text in amber instead of "—"
+    - When a metric is null and no reason exists, display "—"
+    - Add expand/collapse toggle below summary metrics (default collapsed)
+    - When expanded: show data source name ("FMP"), fetch timestamp, and list of unavailable fields with reasons
+    - When collapsed: hide detail, keep summary metrics visible
+    - Preserve glassmorphism UI style
+    - _Requirements: 3.3, 3.4, 4.1, 4.2, 4.3, 4.4_
+
+  - [ ]* 5.3 Write property tests for FundamentalsPanel display logic
+    - **Property 8: Null field display depends on reason existence** — Generate random FundamentalResponse with various null/reason combos, verify displayed text
+    - **Validates: Requirements 3.3, 3.4**
+    - **Property 9: Fundamentals expanded detail content** — Generate random response, expand, verify source/timestamp/reasons in DOM
+    - **Validates: Requirements 4.2**
+
+- [x] 6. Checkpoint — Fundamentals drill-down complete
+  - Ensure all tests pass, ask the user if questions arise.
+
+- [x] 7. S/R zone clustering — backend
+  - [x] 7.1 Implement `cluster_sr_zones()` function in `app/services/sr_service.py`
+    - Sort levels by price ascending
+    - Greedy merge: walk sorted levels, merge if within tolerance % of current cluster midpoint
+    - Compute zone: low, high, midpoint, strength (sum capped at 100), level_count
+    - Tag zone type: "support" if midpoint < current_price, else "resistance"
+    - Sort by strength descending
+    - If `max_zones` set, return top N; if 0 or negative, return empty list
+    - Handle empty input by returning empty list
+    - _Requirements: 7.1, 7.2, 7.3, 7.4, 8.2_
+
+  - [x] 7.2 Add `SRZoneResult` schema and update `SRLevelResponse` in `app/schemas/sr_level.py`
+    - Add `SRZoneResult` model with `low`, `high`, `midpoint`, `strength`, `type`, `level_count`
+    - Add `zones: list[SRZoneResult] = []` to `SRLevelResponse`
+    - _Requirements: 7.2, 9.1_
+
+  - [x] 7.3 Update S/R router to accept `max_zones` parameter and return zones
+    - Add `max_zones: int = 6` query parameter to the S/R levels endpoint
+    - Call `cluster_sr_zones()` with fetched levels and current price
+    - Include zones in the response
+    - _Requirements: 8.1, 8.3_
+
+  - [ ]* 7.4 Write property tests for S/R zone clustering
+    - **Property 14: Clustering merges nearby levels** — Generate random level sets and tolerances, verify no two zones have midpoints within tolerance
+    - **Validates: Requirements 7.2**
+    - **Property 15: Zone strength is capped sum** — Generate random level sets, cluster, verify strength = min(100, sum)
+    - **Validates: Requirements 7.3**
+    - **Property 16: Zone type tagging** — Generate random zones and current prices, verify support/resistance tagging
+    - **Validates: Requirements 7.4**
+    - **Property 17: Zone filtering returns top N by strength** — Generate random zone sets and limits, verify top-N selection
+    - **Validates: Requirements 8.2**
+
+- [x] 8. Checkpoint — S/R clustering backend complete
+  - Ensure all tests pass, ask the user if questions arise.
+
+- [x] 9. Chart enhancements — zoom and pan
+  - [x] 9.1 Add `SRZone` and `SRLevelResponse.zones` types to `frontend/src/lib/types.ts`
+    - _Requirements: 9.1_
+
+  - [x] 9.2 Implement zoom (mouse wheel) on `CandlestickChart`
+    - Add `visibleRange: { start: number, end: number }` state initialized to full dataset
+    - Add `onWheel` handler: positive delta narrows range (zoom in), negative widens (zoom out)
+    - Clamp visible range to min 10 bars, max full dataset length
+    - Disable zoom if dataset has fewer than 10 bars
+    - Slice data by `visibleRange` for rendering
+    - Debounce zoom via `requestAnimationFrame`
+    - _Requirements: 5.1, 5.2, 5.3, 5.5_
+
+  - [x] 9.3 Implement pan (click-drag) on `CandlestickChart`
+    - Add `isPanning` and `panStartX` state
+    - `onMouseDown` starts pan, `onMouseMove` shifts visible range proportionally, `onMouseUp` ends pan
+    - Pan only active when zoomed in (visible range < full dataset)
+    - Clamp range to dataset bounds
+    - _Requirements: 5.4_
+
+  - [x] 9.4 Implement crosshair overlay on `CandlestickChart`
+    - Add `crosshair: { x: number, y: number } | null` state
+    - `onMouseMove` updates crosshair position
+    - Draw vertical line at cursor x spanning full chart height
+    - Draw horizontal line at cursor y spanning full chart width
+    - Display price label on y-axis at horizontal line position
+    - Display date label on x-axis at vertical line position
+    - `onMouseLeave` clears crosshair
+    - _Requirements: 6.1, 6.2, 6.3, 6.4, 6.5_
+
+  - [ ]* 9.5 Write property tests for chart zoom/pan invariants
+    - **Property 10: Zoom adjusts visible range proportionally** — Generate random datasets and wheel deltas, verify range narrows/widens
+    - **Validates: Requirements 5.1, 5.2, 5.3**
+    - **Property 11: Pan shifts visible range** — Generate random ranges and drag deltas, verify shift without width change
+    - **Validates: Requirements 5.4**
+    - **Property 12: Zoom range invariant** — Generate random zoom/pan sequences, verify range bounds always valid
+    - **Validates: Requirements 5.5**
+    - **Property 13: Coordinate-to-value mapping** — Generate random chart configs, verify yToPrice and xToBarIndex mappings
+    - **Validates: Requirements 6.3, 6.4**
+
+- [x] 10. S/R zone rendering on chart
+  - [x] 10.1 Update `CandlestickChart` to accept `zones` prop and render shaded zone rectangles
+    - Accept `zones?: SRZone[]` prop
+    - Render each zone as a semi-transparent rectangle spanning low→high price range across full chart width
+    - Use green shading (rgba) for support zones, red shading for resistance zones
+    - Draw zones behind candlestick bodies (render zones first, then candles)
+    - Display label with midpoint price and strength score for each zone
+    - Re-render zones correctly at every zoom level using current price scale
+    - _Requirements: 9.1, 9.2, 9.3, 9.4, 9.5_
+
+  - [x] 10.2 Update `SROverlay` and `TickerDetailPage` to pass zones to `CandlestickChart`
+    - Update `useTickerDetail` hook or `SROverlay` to extract zones from the S/R API response
+    - Pass zones array to `CandlestickChart` component
+    - Default to max 6 zones (handled by backend `max_zones=6` default)
+    - _Requirements: 8.3, 9.1_
+
+  - [x] 10.3 Ensure S/R overlays re-render correctly at all zoom levels
+    - Verify zone rectangles reposition when zoom/pan changes the visible price scale
+    - Handle single-level zones (low == high) as thin 2px-height rectangles
+    - _Requirements: 5.6, 9.5_
+
+- [x] 11. Final checkpoint — All features integrated
+  - Ensure all tests pass, ask the user if questions arise.
+
+## Notes
+
+- Tasks marked with `*` are optional and can be skipped for faster MVP
+- Each task references specific requirements for traceability
+- Checkpoints ensure incremental validation after each feature area
+- Property tests validate universal correctness properties from the design document
+- The Alembic migration for sentiment and fundamentals columns should ideally be a single migration file
+- S/R zones are computed on-the-fly (not persisted), so no additional migration is needed for zones
				`@@ -0,0 +1 @@`
				`{"specId": "9b39d94f-51e1-42d3-bacc-68eb3961f2b1", "workflowType": "requirements-first", "specType": "feature"}`