sentiment: LLM buy/hold/avoid + full analysis, and search-budget scoping

Richer LLM output (same grounded call, ~no extra cost):
- All providers now also return a recommendation (buy/hold/avoid) and a thorough
  reasoning paragraph; Gemini now actually captures reasoning + grounding
  citations (it was dropping them). Stored on sentiment_scores (migration 008),
  exposed in the API; display-only — NOT fed into the composite/EV.
- Ticker Sentiment panel shows an "LLM view" badge and a "Full analysis & sources"
  expander with the complete reasoning + citations.

Search-budget scoping (Gemini grounding free tier = 5000/mo):
- collect_sentiment now targets only watchlist + open paper trades + top-N by
  composite, skips tickers refreshed within sentiment_fresh_hours (72h), and caps
  per run (sentiment_max_per_run). Once the relevant set is fresh, runs spend 0
  searches until it ages out — bounding monthly usage well under the free tier.
- Widened sentiment lookback to 7d (scoring + display) so sparser collection
  still feeds the dimension score.

Deploy: alembic upgrade (sentiment_scores.recommendation). Switch provider to
Gemini Flash in Admin for the cost win (grounded, cheapest).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

app/providers/gemini_sentiment.py

@@ -30,19 +30,48 @@ if _CA_BUNDLE and Path(_CA_BUNDLE).exists():
         logger.warning("Could not patch aiohttp SSL context", exc_info=True)
 
 _SENTIMENT_PROMPT = """\
-Analyze the current market sentiment for the stock ticker {ticker}.
-Search the web for recent news articles, social media mentions, and analyst opinions.
+Search the web for the latest news, analyst ratings/opinions, and retail/social \
+discussion (e.g. Reddit, StockTwits) about the stock ticker {ticker} from roughly \
+the past 1-2 weeks.
 
-Respond ONLY with a JSON object in this exact format (no markdown, no extra text):
-{{"classification": "<bullish|bearish|neutral>", "confidence": <0-100>, "reasoning": "<brief explanation>"}}
+Assess (1) the current market sentiment and (2) whether BUYING here looks advisable now.
+
+Respond ONLY with a JSON object (no markdown, no extra text):
+{{"classification": "<bullish|bearish|neutral>", "confidence": <0-100>, "recommendation": "<buy|hold|avoid>", "reasoning": "<a thorough paragraph citing specific analyst views, news, and retail sentiment you found, and what drives the recommendation>"}}
 
 Rules:
-- classification must be exactly one of: bullish, bearish, neutral
+- classification = overall mood/tone (bullish, bearish, neutral)
+- recommendation = actionable view on buying now (buy, hold, avoid)
 - confidence must be an integer from 0 to 100
-- reasoning should be a brief one-sentence explanation
+- reasoning should be several sentences citing specific, recent findings
 """
 
 VALID_CLASSIFICATIONS = {"bullish", "bearish", "neutral"}
+VALID_RECOMMENDATIONS = {"buy", "hold", "avoid"}
+
+
+def _parse_recommendation(value: object) -> str | None:
+    v = str(value or "").strip().lower()
+    return v if v in VALID_RECOMMENDATIONS else None
+
+
+def _extract_citations(response: object) -> list[dict[str, str]]:
+    """Pull source URLs/titles from Gemini's grounding metadata."""
+    citations: list[dict[str, str]] = []
+    try:
+        candidates = getattr(response, "candidates", None) or []
+        for cand in candidates:
+            meta = getattr(cand, "grounding_metadata", None)
+            for chunk in (getattr(meta, "grounding_chunks", None) or []):
+                web = getattr(chunk, "web", None)
+                if web is not None:
+                    citations.append({
+                        "url": getattr(web, "uri", "") or "",
+                        "title": getattr(web, "title", "") or "",
+                    })
+    except Exception:
+        pass
+    return citations
 
 
 class GeminiSentimentProvider:
@@ -90,6 +119,9 @@ class GeminiSentimentProvider:
                 confidence=confidence,
                 source="gemini",
                 timestamp=datetime.now(timezone.utc),
+                reasoning=reasoning,
+                citations=_extract_citations(response),
+                recommendation=_parse_recommendation(parsed.get("recommendation")),
             )
 
         except json.JSONDecodeError as exc:

app/providers/openai_compatible_sentiment.py

@@ -28,18 +28,26 @@ _CA_BUNDLE = os.environ.get("SSL_CERT_FILE", "")
 
 _SENTIMENT_PROMPT = """\
 Assess the CURRENT market sentiment for the stock ticker {ticker} based on your \
-knowledge of the company, its sector, and recent developments you are aware of.
+knowledge of the company, its sector, and recent developments you are aware of, \
+and whether BUYING here looks advisable.
 
-Respond ONLY with a JSON object in this exact format (no markdown, no extra text):
-{{"classification": "<bullish|bearish|neutral>", "confidence": <0-100>, "reasoning": "<brief explanation>"}}
+Respond ONLY with a JSON object (no markdown, no extra text):
+{{"classification": "<bullish|bearish|neutral>", "confidence": <0-100>, "recommendation": "<buy|hold|avoid>", "reasoning": "<a thorough explanation of the drivers>"}}
 
 Rules:
 - classification must be exactly one of: bullish, bearish, neutral
+- recommendation must be exactly one of: buy, hold, avoid
 - confidence must be an integer from 0 to 100
-- reasoning should be a brief one-sentence explanation
+- reasoning should be several sentences
 """
 
 VALID_CLASSIFICATIONS = {"bullish", "bearish", "neutral"}
+VALID_RECOMMENDATIONS = {"buy", "hold", "avoid"}
+
+
+def _parse_recommendation(value: object) -> str | None:
+    v = str(value or "").strip().lower()
+    return v if v in VALID_RECOMMENDATIONS else None
 
 
 def _clean_json_text(raw: str) -> str:
@@ -116,6 +124,7 @@ class OpenAICompatibleSentimentProvider:
                 source=self._source,
                 timestamp=datetime.now(timezone.utc),
                 reasoning=reasoning,
+                recommendation=_parse_recommendation(parsed.get("recommendation")),
             )
 
         except json.JSONDecodeError as exc:

app/providers/openai_sentiment.py

@@ -19,39 +19,48 @@ logger = logging.getLogger(__name__)
 _CA_BUNDLE = os.environ.get("SSL_CERT_FILE", "")
 
 _SENTIMENT_PROMPT = """\
-Search the web for the LATEST news, analyst opinions, and market developments \
-about the stock ticker {ticker} from the past 24-48 hours.
+Search the web for the latest news, analyst ratings/opinions, and retail/social \
+discussion (e.g. Reddit, StockTwits) about the stock ticker {ticker} from roughly \
+the past 1-2 weeks.
 
-Based on your web search findings, analyze the CURRENT market sentiment.
+Assess (1) the current market sentiment and (2) whether BUYING here looks advisable now.
 
-Respond ONLY with a JSON object in this exact format (no markdown, no extra text):
-{{"classification": "<bullish|bearish|neutral>", "confidence": <0-100>, "reasoning": "<brief explanation citing recent news>"}}
+Respond ONLY with a JSON object (no markdown, no extra text):
+{{"classification": "<bullish|bearish|neutral>", "confidence": <0-100>, "recommendation": "<buy|hold|avoid>", "reasoning": "<a thorough paragraph citing specific analyst views, news, and retail sentiment you found, and what drives the recommendation>"}}
 
 Rules:
-- classification must be exactly one of: bullish, bearish, neutral
+- classification = overall mood/tone of the coverage (bullish, bearish, neutral)
+- recommendation = actionable view on buying at the current price (buy, hold, avoid)
 - confidence must be an integer from 0 to 100
-- reasoning should cite specific recent news or events you found
+- reasoning should be several sentences citing specific, recent findings
 """
 
 _SENTIMENT_BATCH_PROMPT = """\
-Search the web for the LATEST news, analyst opinions, and market developments \
-about each stock ticker from the past 24-48 hours.
+Search the web for the latest news, analyst ratings/opinions, and retail/social \
+discussion about each stock ticker from roughly the past 1-2 weeks.
 
 Tickers:
 {tickers_csv}
 
 Respond ONLY with a JSON array (no markdown, no extra text), one object per ticker:
-[{{"ticker":"AAPL","classification":"bullish|bearish|neutral","confidence":0-100,"reasoning":"brief explanation"}}]
+[{{"ticker":"AAPL","classification":"bullish|bearish|neutral","confidence":0-100,"recommendation":"buy|hold|avoid","reasoning":"thorough explanation citing findings"}}]
 
 Rules:
-- Include every ticker exactly once
-- ticker must be uppercase symbol
+- Include every ticker exactly once; ticker must be the uppercase symbol
 - classification must be exactly one of: bullish, bearish, neutral
+- recommendation must be exactly one of: buy, hold, avoid
 - confidence must be an integer from 0 to 100
 - reasoning should cite specific recent news or events you found
 """
 
 VALID_CLASSIFICATIONS = {"bullish", "bearish", "neutral"}
+VALID_RECOMMENDATIONS = {"buy", "hold", "avoid"}
+
+
+def parse_recommendation(value: object) -> str | None:
+    """Normalise a recommendation to buy/hold/avoid, or None if absent/invalid."""
+    v = str(value or "").strip().lower()
+    return v if v in VALID_RECOMMENDATIONS else None
 
 
 class OpenAISentimentProvider:
@@ -135,6 +144,7 @@ class OpenAISentimentProvider:
             timestamp=datetime.now(timezone.utc),
             reasoning=reasoning,
             citations=citations,
+            recommendation=parse_recommendation(parsed.get("recommendation")),
         )
 
     async def fetch_sentiment(self, ticker: str) -> SentimentData:

app/providers/protocol.py

@@ -41,6 +41,7 @@ class SentimentData:
     timestamp: datetime
     reasoning: str = ""
     citations: list[dict[str, str]] = field(default_factory=list)  # [{"url": ..., "title": ...}]
+    recommendation: str | None = None  # "buy" | "hold" | "avoid" — actionable LLM view
 
 
 @dataclass(frozen=True, slots=True)