fix(F14): call keyword relevance what it is, and teach it German

The scout presented a keyword count as "Strong fit" / "Medium fit". It is
neither: it measures how many profile keywords a posting echoes, and knows
nothing about his evidence, the hard requirements, the level, the language or
the commute.

- Buckets renamed to keyword relevance, with "NOT a fit judgement" in the
  heading, and score_job documents the limit with the review's own example:
  "Senior AI Engineer, Generative AI / LLM / RAG" scores 12 against "Senior Data
  Engineer" at 3, while the second is much closer to what he can evidence.
- German job vocabulary added. Every German-language posting used to score near
  zero regardless of fit, which is the real reason RUAG, SBB, BKW, PostFinance
  and Kdo Cy all needed a _score_floor. The floor was a workaround; the missing
  terms were the defect. "Senior Dateningenieur" 1 -> 3, level with its English
  twin.
- Platform/DevOps vocabulary was simply absent: devsecops, platform engineer,
  platform automation, infrastructure as code, observability, site reliability.
  The Kdo Cy "ICT-System Engineer (DevSecOps / Platform Automation)" scored ZERO
  before this - a posting whose requirement lines match his canonical evidence
  almost line for line. It now scores 10.
- Noise is unaffected: "Office Manager:in Administration" still scores 0.

Floors were left in place so this diff stays attributable; they can likely be
reduced now that the vocabulary exists.

34/34 tests pass; validated on a live 37-company run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qej5SBvhcbGXHwLqNWiskM
This commit is contained in:
dennisthiessenandClaude Opus 5 committed 2026-09-11 19:54:17 +02:00
1 parent afd8174bbd
commit ce8619b8e3
2 files changed
+47 -10

No files matched your search

+14 -7
View File
@@ -6,11 +6,12 @@ Do not rewrite submitted application archives or invent missing evidence.
## Resume point
F01–F10 complete. **Next: F11** — dedupe scout results by requisition, not title
(`job_scout/scout.py`, inside `_process_company`).
F01–F14 complete. **Next: F15** — typed dates and retained contact/compensation metadata
(`job_scout/scout.py`: the Taleo parser writes a deadline into `posted`, and the Prospective
path's unit/Lohnklasse/contact are folded into `description`, which `_process_company` discards).
Read this file and inspect `git log`/`git diff` before resuming; do not repeat completed work.
Checkpoints: `afdef7e` (F01+F02) · `fc5f3e4` (F03–F06) · `ce5b7f5` (F07) · plus F08–F10.
Checkpoints: `afdef7e` (F01+F02) · `fc5f3e4` (F03–F06) · `ce5b7f5` (F07) · `3519b2f` (F08–F10) · `afd8174` (F11–F13) · plus F14.
## User rulings — 2026-09-11
@@ -35,10 +36,10 @@ Checkpoints: `afdef7e` (F01+F02) · `fc5f3e4` (F03–F06) · `ce5b7f5` (F07) ·
| F08 | DONE | `evidence_schema` + per-claim `evidence` block on all 26; 8 dates recovered from existing text, 18 written `unrecorded` rather than guessed; coverage reported as an advisory hint, never a gate |
| F09 | DONE | `12+ years` -> `over ten years` with the reasoning inline; `cv_template.tex` banner-marked as the academic legacy path |
| F10 | DONE | US figures renamed `*_orientation_only`; the Bavarian formula runs foreign-INTO-German, so inverting it is not a recognised conversion. German grades stay canonical |
| F11 | TODO | Dedupe by requisition, not title |
| F12 | TODO | Structured country/remote eligibility |
| F13 | TODO | Shared decision identity resolver |
| F14 | TODO | Search relevance distinct from fit; German terminology |
| F11 | DONE | Requisition-keyed; same-req locations merge; shared titles warn instead of deleting. **Live run recovered 57 roles (758→815)**, including a second RUAG *Software Engineer C5I* req that was invisible |
| F12 | DONE | Whole-word place matching (Bernau no longer reads as Bern); ~25 Swiss towns incl. Thun and Zimmerwald; Danish cities; pan-regional scope no longer beats an explicit onsite/one-country restriction. All 14 review cases correct |
| F13 | DONE | Identifying query params kept in the key; read and write paths share `lookup_decision`. **Live run recovered 7 reposted RUAG decisions**, labelled RELATED rather than applied as identical |
| F14 | DONE | Buckets renamed to keyword relevance with an explicit 'NOT a fit judgement'; German titles and platform/DevOps vocabulary added. Kdo Cy *ICT-System Engineer (DevSecOps/Platform Automation)* went 0→10; *Senior Dateningenieur* 1→3, level with its English twin |
| F15 | TODO | Typed dates and retained contact/compensation metadata |
| F16 | TODO | Run-scoped reports, transactional persistence and scan coverage |
| F17 | TODO | Reproducible new builds and safe historical archive classification |
@@ -83,6 +84,12 @@ mid-remediation — it would destroy the ability to attribute a regression to a
- F08: `unrecorded` is deliberately a real value. Failing the build on it would push a future
session to invent dates to get a green run.
- F10: no stale `final_grade_us_4pt` references remain anywhere in the repo.
- F11–F14 validated on a live 37-company run: 815 matches vs 758 the day before, the delta being
requisitions the title dedupe had been deleting. Seven employers reported shared-title collisions
that are now kept and flagged (Grafana 5, Axpo 6, AWS 3, RUAG 2, …).
- F14 note: the score floor on RUAG/SBB/BKW/PostFinance/Kdo Cy was a workaround for missing German
vocabulary. The vocabulary is now the fix; the floors can probably be reduced later, but they were
left in place to keep this diff attributable.
- Not re-run this session: the 45-document LaTeX build sweep and the visual sampling.
## Completion rules
+33 -3
View File
@@ -93,6 +93,23 @@ GLOBAL_SCOPE_KEYWORDS = [
]
POSITIVE_KEYWORDS = {
# German/Swiss job titles (added 2026-09-11, F14). Without these, every German-language
# posting scored near zero regardless of fit, which is why RUAG, SBB, BKW, PostFinance
# and Kdo Cy all needed a _score_floor to stay visible at all. The floor is a workaround;
# these terms are the actual fix. Weights mirror their English counterparts.
# Platform/DevOps vocabulary that was simply absent (F14). The Kdo Cy
# "ICT-System Engineer (DevSecOps / Platform Automation)" scored ZERO before this -
# a posting whose every requirement line matches his canonical evidence.
"devsecops": 3, "platform engineer": 3, "platform automation": 3, "platform": 1,
"automation": 1, "infrastructure as code": 2, "observability": 2,
"site reliability": 3, "system engineer": 1, "ict": 1,
"dateningenieur": 2, "data engineer (m/w/d)": 2, "systemingenieur": 2,
"software-entwickler": 2, "softwareentwickler": 2, "entwickler": 1,
"plattform": 2, "plattform-engineer": 3, "datenplattform": 3,
"automatisierung": 2, "betrieb": 1, "datenanalyse": 2, "datenmodellierung": 2,
"schnittstellen": 1, "datenqualitaet": 2, "datenqualität": 2,
"künstliche intelligenz": 3, "kuenstliche intelligenz": 3, "maschinelles lernen": 3,
"cloud-plattform": 2, "containerisierung": 2, "informatiker": 1,
"genai": 3, "generative ai": 3, "llm": 3, "large language model": 3,
"applied ai": 3, "applied ml": 3, "ai engineer": 3, "ml engineer": 3,
"mlops": 3, "ai platform": 3, "ml platform": 3,
@@ -1819,6 +1836,15 @@ def _kw_in(kw, text):
def score_job(job, title_only=False):
"""Keyword RELEVANCE for search ranking - explicitly not a fit score (F14).
It measures how many profile keywords a posting echoes. It does not know his evidence,
the hard requirements, the level, the language or the commute, and a higher number must
never be read as a better application. Compare: "Senior AI Engineer, Generative AI / LLM
/ RAG" scores 12 against "Senior Data Engineer" at 3, while the second is much closer to
what he can actually evidence. German titles score near zero regardless of fit, which is
why pre-filtered boards carry a _score_floor.
"""
# Title carries the real signal; the JD body is full of company boilerplate (every
# Kraken post mentions crypto/blockchain/trading, every cloud post mentions python).
# So title matches score at full weight and body-only matches at half (min 1) — enough
@@ -2130,10 +2156,14 @@ def write_report(path, results, errors, new_only, include_weak, stats=None, tota
lines.append(f"_{n_decided} role(s) already in the decision log ({shown}; "
f"see state/decisions.json)._")
buckets = [("Strong fit (score >= 6)", strong),
("Medium fit (score 2-5)", medium)]
# F14: this number is SEARCH RELEVANCE - how many profile keywords the posting echoes.
# It is not a fit assessment and must never be presented as one: "Senior AI Engineer"
# with GenAI/LLM/RAG scores 12 while "Senior Data Engineer" scores 3, although the
# second sits far closer to his actual evidence. Fit is decided on the full JD.
buckets = [("High keyword relevance (score >= 6) - NOT a fit judgement", strong),
("Medium keyword relevance (score 2-5)", medium)]
if include_weak:
buckets.append(("Weak / noise (score < 2)", weak))
buckets.append(("Low keyword relevance (score < 2) - German titles land here", weak))
for bucket_name, bucket in buckets:
shown = [r for r in bucket