From ce8619b8e3f10ee45bed22e4e7346a6bad9bf89b Mon Sep 17 00:00:00 2001 From: Dennis Thiessen Date: Fri, 11 Sep 2026 19:54:17 +0200 Subject: [PATCH] fix(F14): call keyword relevance what it is, and teach it German The scout presented a keyword count as "Strong fit" / "Medium fit". It is neither: it measures how many profile keywords a posting echoes, and knows nothing about his evidence, the hard requirements, the level, the language or the commute. - Buckets renamed to keyword relevance, with "NOT a fit judgement" in the heading, and score_job documents the limit with the review's own example: "Senior AI Engineer, Generative AI / LLM / RAG" scores 12 against "Senior Data Engineer" at 3, while the second is much closer to what he can evidence. - German job vocabulary added. Every German-language posting used to score near zero regardless of fit, which is the real reason RUAG, SBB, BKW, PostFinance and Kdo Cy all needed a _score_floor. The floor was a workaround; the missing terms were the defect. "Senior Dateningenieur" 1 -> 3, level with its English twin. - Platform/DevOps vocabulary was simply absent: devsecops, platform engineer, platform automation, infrastructure as code, observability, site reliability. The Kdo Cy "ICT-System Engineer (DevSecOps / Platform Automation)" scored ZERO before this - a posting whose requirement lines match his canonical evidence almost line for line. It now scores 10. - Noise is unaffected: "Office Manager:in Administration" still scores 0. Floors were left in place so this diff stays attributable; they can likely be reduced now that the vocabulary exists. 34/34 tests pass; validated on a live 37-company run. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Qej5SBvhcbGXHwLqNWiskM --- FIX_PROGRESS.md | 21 ++++++++++++++------- job_scout/scout.py | 36 +++++++++++++++++++++++++++++++++--- 2 files changed, 47 insertions(+), 10 deletions(-) diff --git a/FIX_PROGRESS.md b/FIX_PROGRESS.md index 5f7c1f7..3f6ed34 100644 --- a/FIX_PROGRESS.md +++ b/FIX_PROGRESS.md @@ -6,11 +6,12 @@ Do not rewrite submitted application archives or invent missing evidence. ## Resume point -F01–F10 complete. **Next: F11** — dedupe scout results by requisition, not title -(`job_scout/scout.py`, inside `_process_company`). +F01–F14 complete. **Next: F15** — typed dates and retained contact/compensation metadata +(`job_scout/scout.py`: the Taleo parser writes a deadline into `posted`, and the Prospective +path's unit/Lohnklasse/contact are folded into `description`, which `_process_company` discards). Read this file and inspect `git log`/`git diff` before resuming; do not repeat completed work. -Checkpoints: `afdef7e` (F01+F02) · `fc5f3e4` (F03–F06) · `ce5b7f5` (F07) · plus F08–F10. +Checkpoints: `afdef7e` (F01+F02) · `fc5f3e4` (F03–F06) · `ce5b7f5` (F07) · `3519b2f` (F08–F10) · `afd8174` (F11–F13) · plus F14. ## User rulings — 2026-09-11 @@ -35,10 +36,10 @@ Checkpoints: `afdef7e` (F01+F02) · `fc5f3e4` (F03–F06) · `ce5b7f5` (F07) · | F08 | DONE | `evidence_schema` + per-claim `evidence` block on all 26; 8 dates recovered from existing text, 18 written `unrecorded` rather than guessed; coverage reported as an advisory hint, never a gate | | F09 | DONE | `12+ years` -> `over ten years` with the reasoning inline; `cv_template.tex` banner-marked as the academic legacy path | | F10 | DONE | US figures renamed `*_orientation_only`; the Bavarian formula runs foreign-INTO-German, so inverting it is not a recognised conversion. German grades stay canonical | -| F11 | TODO | Dedupe by requisition, not title | -| F12 | TODO | Structured country/remote eligibility | -| F13 | TODO | Shared decision identity resolver | -| F14 | TODO | Search relevance distinct from fit; German terminology | +| F11 | DONE | Requisition-keyed; same-req locations merge; shared titles warn instead of deleting. **Live run recovered 57 roles (758→815)**, including a second RUAG *Software Engineer C5I* req that was invisible | +| F12 | DONE | Whole-word place matching (Bernau no longer reads as Bern); ~25 Swiss towns incl. Thun and Zimmerwald; Danish cities; pan-regional scope no longer beats an explicit onsite/one-country restriction. All 14 review cases correct | +| F13 | DONE | Identifying query params kept in the key; read and write paths share `lookup_decision`. **Live run recovered 7 reposted RUAG decisions**, labelled RELATED rather than applied as identical | +| F14 | DONE | Buckets renamed to keyword relevance with an explicit 'NOT a fit judgement'; German titles and platform/DevOps vocabulary added. Kdo Cy *ICT-System Engineer (DevSecOps/Platform Automation)* went 0→10; *Senior Dateningenieur* 1→3, level with its English twin | | F15 | TODO | Typed dates and retained contact/compensation metadata | | F16 | TODO | Run-scoped reports, transactional persistence and scan coverage | | F17 | TODO | Reproducible new builds and safe historical archive classification | @@ -83,6 +84,12 @@ mid-remediation — it would destroy the ability to attribute a regression to a - F08: `unrecorded` is deliberately a real value. Failing the build on it would push a future session to invent dates to get a green run. - F10: no stale `final_grade_us_4pt` references remain anywhere in the repo. +- F11–F14 validated on a live 37-company run: 815 matches vs 758 the day before, the delta being + requisitions the title dedupe had been deleting. Seven employers reported shared-title collisions + that are now kept and flagged (Grafana 5, Axpo 6, AWS 3, RUAG 2, …). +- F14 note: the score floor on RUAG/SBB/BKW/PostFinance/Kdo Cy was a workaround for missing German + vocabulary. The vocabulary is now the fix; the floors can probably be reduced later, but they were + left in place to keep this diff attributable. - Not re-run this session: the 45-document LaTeX build sweep and the visual sampling. ## Completion rules diff --git a/job_scout/scout.py b/job_scout/scout.py index db6c9a3..606cbe3 100644 --- a/job_scout/scout.py +++ b/job_scout/scout.py @@ -93,6 +93,23 @@ GLOBAL_SCOPE_KEYWORDS = [ ] POSITIVE_KEYWORDS = { + # German/Swiss job titles (added 2026-09-11, F14). Without these, every German-language + # posting scored near zero regardless of fit, which is why RUAG, SBB, BKW, PostFinance + # and Kdo Cy all needed a _score_floor to stay visible at all. The floor is a workaround; + # these terms are the actual fix. Weights mirror their English counterparts. + # Platform/DevOps vocabulary that was simply absent (F14). The Kdo Cy + # "ICT-System Engineer (DevSecOps / Platform Automation)" scored ZERO before this - + # a posting whose every requirement line matches his canonical evidence. + "devsecops": 3, "platform engineer": 3, "platform automation": 3, "platform": 1, + "automation": 1, "infrastructure as code": 2, "observability": 2, + "site reliability": 3, "system engineer": 1, "ict": 1, + "dateningenieur": 2, "data engineer (m/w/d)": 2, "systemingenieur": 2, + "software-entwickler": 2, "softwareentwickler": 2, "entwickler": 1, + "plattform": 2, "plattform-engineer": 3, "datenplattform": 3, + "automatisierung": 2, "betrieb": 1, "datenanalyse": 2, "datenmodellierung": 2, + "schnittstellen": 1, "datenqualitaet": 2, "datenqualität": 2, + "künstliche intelligenz": 3, "kuenstliche intelligenz": 3, "maschinelles lernen": 3, + "cloud-plattform": 2, "containerisierung": 2, "informatiker": 1, "genai": 3, "generative ai": 3, "llm": 3, "large language model": 3, "applied ai": 3, "applied ml": 3, "ai engineer": 3, "ml engineer": 3, "mlops": 3, "ai platform": 3, "ml platform": 3, @@ -1819,6 +1836,15 @@ def _kw_in(kw, text): def score_job(job, title_only=False): + """Keyword RELEVANCE for search ranking - explicitly not a fit score (F14). + + It measures how many profile keywords a posting echoes. It does not know his evidence, + the hard requirements, the level, the language or the commute, and a higher number must + never be read as a better application. Compare: "Senior AI Engineer, Generative AI / LLM + / RAG" scores 12 against "Senior Data Engineer" at 3, while the second is much closer to + what he can actually evidence. German titles score near zero regardless of fit, which is + why pre-filtered boards carry a _score_floor. + """ # Title carries the real signal; the JD body is full of company boilerplate (every # Kraken post mentions crypto/blockchain/trading, every cloud post mentions python). # So title matches score at full weight and body-only matches at half (min 1) — enough @@ -2130,10 +2156,14 @@ def write_report(path, results, errors, new_only, include_weak, stats=None, tota lines.append(f"_{n_decided} role(s) already in the decision log ({shown}; " f"see state/decisions.json)._") - buckets = [("Strong fit (score >= 6)", strong), - ("Medium fit (score 2-5)", medium)] + # F14: this number is SEARCH RELEVANCE - how many profile keywords the posting echoes. + # It is not a fit assessment and must never be presented as one: "Senior AI Engineer" + # with GenAI/LLM/RAG scores 12 while "Senior Data Engineer" scores 3, although the + # second sits far closer to his actual evidence. Fit is decided on the full JD. + buckets = [("High keyword relevance (score >= 6) - NOT a fit judgement", strong), + ("Medium keyword relevance (score 2-5)", medium)] if include_weak: - buckets.append(("Weak / noise (score < 2)", weak)) + buckets.append(("Low keyword relevance (score < 2) - German titles land here", weak)) for bucket_name, bucket in buckets: shown = [r for r in bucket