diff --git a/.gitignore b/.gitignore index 3d775e4..5dc9745 100644 --- a/.gitignore +++ b/.gitignore @@ -30,6 +30,7 @@ job_scout/state/* !job_scout/state/decisions.json !job_scout/state/application_cohort.json !job_scout/state/applications.json +!job_scout/state/employer_reputation.json !job_scout/state/history/ job_scout/state/history/* !job_scout/state/history/application_status_before_2026-09-11.json diff --git a/CLAUDE.md b/CLAUDE.md index cd05354..beb0a6e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -183,6 +183,8 @@ _See `config.md` for user-specific corrections. Add verified errors here as you | Date | Correction | Files fixed | |------|-----------|-------------| +| 2026-10-21 | **Software design and architecture was under-recorded: the seventh canonical gap of the recurring shape, surfaced by the Google *Staff SWE, Shopping Graph* minimum *"3 years of experience with software design and architecture."*** Before this, the only design evidence was SW-7 (current, undated, 2026-10-03) and a *contribution* at Generali (GN-3), so the line read as unproven. User-confirmed: **software design and architecture were part of his work at Bosch (ingestion and inference scripts, 2020-2022) and at Swisscom (data products)**, and both count toward the year requirement; together they exceed three years. Recorded on BS-1, BS-2 and SW-7, with `designed` added to the BS-1/BS-2 allowed verbs. **Scope kept tight:** at Bosch, "designed" covers the ingestion scripts and the inference integration only, not the data services in general (the first patch over-extended it and was trimmed the same session). Design never means an architect title, design authority or shared-platform architecture. The Swisscom start date stays undated. **Lesson, the same as 10-08 and 10-14: when a JD has a years-of-X minimum that the canonical file states without a duration, ask for the duration before scoring.** | `claims.json` (BS-1, BS-2, SW-7); `config.md` (data-product design row); `knowledge_base/extractions/2026-10-21_software_design_architecture_clarification.md` | +| 2026-10-21 | **FIXED 2026-10-22 (Microsoft links); the EMEA leak below is still OPEN.** Fix: `fetch_pcsx` now emits `apply.careers.microsoft.com?jobId=`, a `?jobId=` pattern precedes the `/job/` one in `_JOB_ID_PATTERNS`, and `_HOST_ALIASES` maps the new host to the old so `_decision_key` is unchanged. Verified: all 7 logged Microsoft decisions still resolve (applied 200043897 included), and a new-form link opened the real posting with the job number on the page. Reports before 2026-10-22 still carry the dead form. Original finding: **every Microsoft link the scout emits is dead, in every report**, found in the 2026-10-20 triage. `fetch_pcsx` builds `jobs.careers.microsoft.com/global/en/job/`, which now redirects to a generic `apply.careers.microsoft.com/careers?...&pid=1970393557041551` search page. Verified on 200060867 and on the **pending application 200043897** (Principal FDE). The application itself is unaffected; only the link is. The working form is `https://apply.careers.microsoft.com/careers/job/`, the API's `positionUrl`. **Deliberately NOT patched yet:** 6 logged Microsoft decisions (including 200043897) are keyed on `displayJobId` through the `careers\.microsoft\.com /job/(\d+)` pattern. Swapping the URL would re-key them on the pcsx id and orphan them, the same failure as the 09-04 RUAG/BIS de-linking. **Safe fixes:** (a) keep `url` and add a working `apply_url` field to the row, or (b) emit the new URL with the display id carried in it, plus a host-normalised key pattern. Either way, verify all 6 rows still resolve through `find_decision_url` afterwards. Same triage, also OPEN: an **"EMEA" region suffix lets country-fenced roles through the location filter** (GitLab FR/DE/NL/ES/UK, Grafana ×5, Fivetran Serbia: 7 of 109 new rows), which breaks the docstring rule against multi-country allowlists that omit CH. | not yet fixed: `scout.py` (`fetch_pcsx`, location filter); evidence in `reports/2026-10-20.triage.md` | | 2026-10-15 | **Two scout adapters were reporting wrong data without failing; both were found by the 2026-10-14 triage.** (a) **BIS:** the RSS adapter stamped `default_location: Basel, Switzerland` on every vacancy, so **jr100491 (Hong Kong, BIS Asia office)** passed the CH filter and reached a JD read. The feed carries no location, but each vacancy page has a static *Geographical location* field. `fetch_rss` now reads it per item (`location_page_regex`), falls back to a *"based in ..."* sentence in the feed description, and only then uses `default_location`, printing a warning when it does. Verified: Basel reqs still pass, and jr100491 now reads *8 Finance Street, Central, Hong Kong* and is excluded. (b) **Confluent:** IBM closed the acquisition on 2026-03-17, and the Ashby board went from 17 jobs (10-08) to `{"jobs":[]}` (10-14). careers.confluent.io sits behind a Vercel bot checkpoint. The roles are now IBM requisitions, served by the open IBM careers search API (`www-api.ibm.com/search/api/v2`, POST, `size` capped at 100, pages with `from`). New `ibm` adapter; Confluent keeps its id, so seen-state and decisions still link. First run: 84 roles, 0 in CH/Europe-remote scope. **HashiCorp was dropped on 2026-06-06 for having "no clean public ATS API" after IBM absorbed it. That API exists, so HashiCorp can come back through the same adapter if wanted.** **Rule: when an adapter must assume a field, the assumption has to be visible. A silent default looks exactly like real data.** | `scout.py` (`fetch_rss`, `_rss_item_location`, `fetch_ibm`, BIS and Confluent config) | | 2026-10-14 | **Spotfire was the sixth evidenced-but-unregistered skill, and junior mentoring was unrecorded - both surfaced by the BIS Basel jr100498 *Senior/Lead Data & Analytics Engineer* (MED IT) triage.** (a) **TIBCO Spotfire** sat in the BS-5 and BS-8 scopes (co-owned environment, technical Application Owner, C# extensions, TAF 2022, spatial wafer-data map charts) but not in the `skills` array, so the JD line *"using database reporting and business intelligence tools"* had no citable skill; registered `production-historical`, `allowed` on user instruction. It does not license Power BI, Tableau or Qlik. (b) The JD requires *"Experience providing general supervision and task-specific guidance to junior colleagues"*, and the only canonical evidence was BS-7 (students, one thesis as Zweitpruefer). User-confirmed: he **onboarded and mentored junior developers at Bosch and at Swisscom** - recorded as **BS-9** and **SW-10**, one per employer. The Swisscom phase (ART B2B or ART DPT) is not recorded, so write it in past tense. Neither is people management. (c) **SDMX: user-confirmed never used**; registered `forbidden` so the scanner catches it. It is an accountability line in the JD, not a qualification. (d) The same audit found **Teradata, Oracle Database and Hadoop/Impala** evidenced only in the SW-2/BS-2/BS-8 scopes and absent from the `skills` array, although every past skills line listed them; registered `production-historical` from those scopes (Oracle explicitly does not establish DBA work). **Lesson, same as 10-08: when a JD has a supervision or mentoring line, ask about junior colleagues before falling back on BS-7. Student supervision is weaker evidence and the stronger evidence was simply unrecorded.** | `claims.json` (BS-9, SW-10; skills TIBCO Spotfire, SDMX, Teradata, Oracle Database, Hadoop / Impala); `knowledge_base/extractions/2026-10-14_mentoring_spotfire_sdmx_clarification.md` | | 2026-10-08 | **Three more canonical gaps of the recurring shape, surfaced by the Kongsberg Seatex *Lead Engineer, DevOps and Test Infrastructure* (Trondheim) package.** (a) **Jenkins and GitLab CI/CD** were evidenced in claim scopes (GN-1, FC-1, BS-6, SW-3) but absent from the `skills` array, so no skills line could cite them; registered as `production-historical`, with GitLab keeping SW-3's "used, not built" limit. (b) **Embedded coursework** was unrecorded: user-confirmed an Embedded Systems course (C/C++ and VHDL with an ARM platform) and Robotics during his studies, and **no embedded work since** - recorded as **ED-1**, coursework-only, degree attribution not yet given. (c) **Current unit testing** was unrecorded: he writes unit tests today and knows pytest - recorded as **SW-9**; whether pytest is the Swisscom framework is not confirmed. The JD's required line *"Experience working with embedded SW and Linux"* stays a **half Gap** (hard-gate FAIL); the user overrode it knowingly. **Lesson, again: for any DevOps/test JD ask about current testing practice and about academic hardware work before scoring - neither lives in `claims.json` by default.** | `claims.json` (ED-1, SW-9, 7 skills); `knowledge_base/extractions/2026-10-08_embedded_coursework_testing_clarification.md`; Kongsberg package | diff --git a/job_scout/scout.py b/job_scout/scout.py index f1cd2cf..970339f 100644 --- a/job_scout/scout.py +++ b/job_scout/scout.py @@ -12,19 +12,29 @@ Location policy (default for most companies): - NOT multi-country allowlists that omit CH/DK/Europe (e.g. "UK | Brazil | Ireland | ...") Company overrides via args["_location_policy"]: - - "norway_only" — Equinor: Norway only - - "dk_no" — NATO-style: Denmark + Norway only (no CH — no NATO seat in CH) + - "norway_only" — Norway only, Oslo included + - "norway_no_oslo" — Norway with a location outside the Oslo area (Equinor, Telenor, Nordic, + DNV, Arm, Microsoft Norway, NAV register; user preference 2026-10-21) + - "dk_no" — NATO-style: Denmark + Norway only (no CH — no NATO seat in CH) + +Norway rows also get a language/citizenship check from the posting's requirement sentence +(only "Norwegian required" or "Norwegian citizenship required" filters a role; clearance is +noted, never a gate), and every row gets an employer-reputation verdict from +state/employer_reputation.json (Vizrt is excluded by user decision). Usage: py scout.py # Pull all configured companies (strong + medium only) py scout.py --only=nvidia # Pull a single company by id + py scout.py --region=no # Only the Norway boards (or --region=ch for Switzerland + + # Europe-remote). Reports always list regions separately. py scout.py --new-only # Report only jobs not seen before py scout.py --include-weak # Include weak/noise bucket (default hidden) py scout.py --hide-decided # Drop roles already in the decision log (undecided-only view) py scout.py --decide "" [note...] # Record a decision and exit # status is free-text: shortlist | skip | applied | paused | ... -State : state/seen_jobs.json · state/decisions.json · state/last_scrape.json +State : state/seen_jobs.json · state/decisions.json · state/last_scrape.json · + state/employer_reputation.json (curated, tracked) · state/nav_detail_cache.json Output: reports/YYYY-MM-DD.md (+ .json dump of every eligible role) To add a company: append to COMPANIES with one of the existing adapter types. A few sites @@ -79,6 +89,17 @@ NO_LOCATION_KEYWORDS = [ "norway", "oslo", "stavanger", "bergen", "trondheim", "norge", ] +# Oslo and its commuter belt. The user wants Norway roles in Trondheim, Bergen or similar +# and explicitly not in Oslo (2026-10-08, re-stated 2026-10-21), so the norway_no_oslo policy +# drops a role whose every Norwegian location is one of these. A role listing Oslo AND +# Trondheim stays in. Høvik (DNV HQ), Fornebu (Telenor HQ) and Kjeller (FFI) are Oslo area. +OSLO_AREA_PLACES = [ + "oslo", "fornebu", "lysaker", "bærum", "baerum", "sandvika", "asker", "høvik", "hovik", + "stabekk", "skøyen", "skoyen", "nydalen", "lillestrøm", "lillestrom", "kjeller", + "skedsmo", "lørenskog", "lorenskog", "oppegård", "kolbotn", "nordre follo", "ski", + "jessheim", "ullensaker", "gardermoen", "billingstad", "kolsås", "kolsas", "rælingen", +] + US_ONLY_PATTERNS = [ "remote - us", "remote, us", "remote-us", "us remote", "us-remote", "remote-friendly us", "remote (us)", "united states - remote", @@ -112,6 +133,16 @@ POSITIVE_KEYWORDS = { "schnittstellen": 1, "datenqualitaet": 2, "datenqualität": 2, "künstliche intelligenz": 3, "kuenstliche intelligenz": 3, "maschinelles lernen": 3, "cloud-plattform": 2, "containerisierung": 2, "informatiker": 1, + # Norwegian titles (added 2026-10-22, same reason as the German block above): without them + # every Norwegian-titled NAV posting scored 0, so it sat in the hidden weak bucket and its + # language requirement was never checked - exactly the "norsk og/eller engelsk" ads the + # user wants surfaced. Compounds are listed explicitly: word-boundary matching does not + # find "utvikler" inside "plattformutvikler". + "utvikler": 1, "systemutvikler": 2, "programvareutvikler": 2, "backendutvikler": 2, + "plattformutvikler": 3, "plattformingeniør": 3, "dataingeniør": 2, "dataingenior": 2, + "dataplattform": 3, "skyplattform": 2, "maskinlæring": 3, "kunstig intelligens": 3, + "ki-ingeniør": 3, "automatisering": 2, "testautomatisering": 2, "datamodellering": 2, + "driftsingeniør": 1, "genai": 3, "generative ai": 3, "llm": 3, "large language model": 3, "applied ai": 3, "applied ml": 3, "ai engineer": 3, "ml engineer": 3, "mlops": 3, "ai platform": 3, "ml platform": 3, @@ -313,7 +344,8 @@ COMPANIES = [ "host": "equinor.wd3.myworkdayjobs.com", "tenant": "equinor", "site": "EQNR", - "_location_policy": "norway_only", + "_location_policy": "norway_no_oslo", + "nav_aliases": ["Equinor"], # Norway-only filter already scopes the board; titles are often generic/energy # ops and score 0 under the English eng keyword list — keep them visible. "_score_floor": 2, @@ -350,7 +382,8 @@ COMPANIES = [ ("telenor", "Telenor Norge", "teamtailor", { "slug": "telenornorway", "default_location": "Oslo/Trondheim, Norway", - "_location_policy": "norway_only", + "_location_policy": "norway_no_oslo", + "nav_aliases": ["Telenor"], "_score_floor": 2, }), # NATO civilian vacancies (Oracle Taleo). Public marketing page is @@ -367,6 +400,58 @@ COMPANIES = [ # Staff Officer / NATO-body titles score unevenly under eng keywords — keep # pre-filtered NO/DK roles visible (same pattern as Equinor/SBB). "_score_floor": 2, + "nav_aliases": ["NATO"], + }), + # --- Norway outside Oslo (added 2026-10-22, user request) ----------------------------- + # The user wants a move to Trondheim, Bergen or similar, not Oslo. NAV covers most + # Norwegian employers; these boards are where NAV was measured thin or empty + # (2026-10-22: Nordic 0, DNV 2, Equinor 2 vs ~10, NATO 0). Every Norway row gets the + # language/citizenship check and a reputation verdict in the report. + # Nordic Semiconductor (Trondheim HQ) — Teamtailor on a custom domain; 29 roles worldwide + # on 2026-10-22, the schema.org address gives the real city. + ("nordic", "Nordic Semiconductor", "teamtailor", { + "base_url": "https://careers.nordicsemi.com", + "default_location": "", + "_location_policy": "norway_no_oslo", + "nav_aliases": ["Nordic Semiconductor"], + }), + # DNV — Algolia-backed jobs.dnv.com (credentials read from the page each run). HQ Høvik is + # Oslo area and filtered; Trondheim/Bergen/Stavanger offices stay. + ("dnv", "DNV", "algolia_jobs", { + "page_url": "https://jobs.dnv.com/job-search", + "_location_policy": "norway_no_oslo", + "nav_aliases": ["DNV"], + }), + # Arm (Trondheim, ex-Falanx: Mali GPU drivers, compilers, hardware) — Radancy front end + # over iCIMS. See fetch_radancy for the multi-location limitation. + ("arm", "Arm", "radancy", { + "base_url": "https://careers.arm.com", + "keywords": ["Trondheim", "Norway"], + "_location_policy": "norway_no_oslo", + "nav_aliases": ["Arm Norway", "ARM Norway"], + }), + # Microsoft Development Center Norway (Oslo, Trondheim, Tromsø) — same PCSX API as the + # Swiss entry. NOTE: a cold application here stacks on the pending Principal FDE + # (200043897) until that resolves. + ("microsoft_no", "Microsoft (Norway)", "pcsx", { + "domain": "microsoft.com", + "location": "Norway", + "_location_policy": "norway_no_oslo", + "nav_aliases": ["Microsoft"], + }), + # NAV national job register — the wide net for everything else (Kongsberg, Forsvaret, + # Nammo, SINTEF, Kitron, Andøya, the long tail). Keyword queries on a request budget; + # see fetch_nav for the rate-limit behaviour. Rows carry the real employer name. + ("nav", "NAV register (Norway)", "nav", { + "queries": ["data engineer", "dataingeniør", "devops", "platform engineer", + "software engineer", "systemutvikler", "python", "machine learning", + "data platform", "cloud engineer", "site reliability", "backend", + "test automation", "utvikler"], + "search_budget": 14, + "pages_per_query": 1, + "detail_budget": 25, + "pace_s": 3.0, + "_location_policy": "norway_no_oslo", }), # International org — BIS (Basel), commutable from Bern, salary net of Swiss tax. # Low-volume RSS feed; no title filter (Innovation Hub roles can be oddly titled). @@ -769,7 +854,16 @@ def fetch_pcsx(args): jobs, start = [], 0 while True: url = f"{base}?domain={domain}&query=&location={urllib.parse.quote(location)}&start={start}&num=50" - data = http_get_json(url, headers={"Referer": f"https://apply.careers.microsoft.com/careers?location={urllib.parse.quote(location)}"}) + # The PCSX API rate-limits after a handful of calls (429 seen 2026-10-22), and two + # entries (CH, Norway) now query it in parallel. Back off and retry before failing. + for attempt in range(3): + try: + data = http_get_json(url, headers={"Referer": f"https://apply.careers.microsoft.com/careers?location={urllib.parse.quote(location)}"}) + break + except urllib.error.HTTPError as e: + if e.code != 429 or attempt == 2: + raise + time.sleep(10 * (attempt + 1)) payload = data.get("data") or {} positions = payload.get("positions", []) or [] for p in positions: @@ -778,9 +872,19 @@ def fetch_pcsx(args): "id": str(p.get("id")), "title": p.get("name", ""), "location": " | ".join(locs), - "url": f"https://jobs.careers.microsoft.com/global/en/job/{p.get('displayJobId') or p.get('id')}", + # The old jobs.careers.microsoft.com/global/en/job/ form redirects + # to a generic search page (verified 2026-10-20, including for 200043897). The + # working form is the API's positionUrl on apply.careers.microsoft.com. The + # display id rides along as ?jobId= so the decision log keeps keying on it. + "url": (f"https://apply.careers.microsoft.com" + f"{p.get('positionUrl') or '/careers/job/' + str(p.get('id'))}" + f"?jobId={p.get('displayJobId') or p.get('id')}"), "posted": p.get("postedTs", ""), "description": (p.get("description") or "")[:2000], + # Norway entry: language check on the full description, before truncation. + **({"lang_check": classify_language_requirement( + _html_to_text(p.get("description") or ""))} + if args.get("_location_policy", "").startswith("norway") else {}), }) # The API serves ~10 rows per call regardless of `num`, so a short page is NOT # end-of-results — page until a call comes back empty or `count` is reached. @@ -951,14 +1055,21 @@ def fetch_teamtailor(args): parts.append(bit) if parts: location = " | ".join(parts) - jobs.append({ + job = { "id": it.get("id") or it.get("url", ""), "title": it.get("title", ""), "location": location, "url": it.get("url", ""), "posted": it.get("date_published", ""), "description": re.sub(r"<[^>]+>", " ", it.get("content_html", ""))[:2500], - }) + } + # Norway boards: check the language requirement on the FULL ad. The stored + # description is truncated for scoring and usually cuts the requirement list off + # (Telenor's "kommunisere godt ... på norsk og engelsk" read UNCLEAR before this). + if args.get("_location_policy", "").startswith("norway"): + job["lang_check"] = classify_language_requirement( + _html_to_text(it.get("content_html", ""))) + jobs.append(job) nxt = data.get("next_url") if not items or not nxt or nxt == url: break @@ -1842,7 +1953,313 @@ def fetch_ibm(args): return jobs +# --- Language / citizenship requirement check (Norway) ----------------------------------- +# Added 2026-10-22 on user instruction. The language a posting is WRITTEN in, and even NAV's +# "Arbeidsspråk" tag, is not the requirement: Norwegian-language ads often accept English +# ("norsk og/eller engelsk"), and English ads sometimes demand Norwegian. He has A2 +# Norwegian, so only an explicit Norwegian requirement is a fail. Clearance is NOT a gate +# (user, 2026-10-22: EU/NATO citizen, expects to be clearable); only an explicit Norwegian- +# citizenship requirement is. Every verdict carries its evidence snippet so it can be checked. +_LANG_ENGLISH_OK = [re.compile(p, re.I) for p in ( + r"norsk\s+og\s*/\s*eller\s+engelsk", r"engelsk\s+og\s*/\s*eller\s+norsk", + r"norsk\s+eller\s+engelsk", r"engelsk\s+eller\s+norsk", + r"skandinavisk\w*\s+(?:språk\s+)?(?:og\s*/\s*)?eller\s+engelsk", + r"engelsk\s+(?:og\s*/\s*)?eller\s+(?:et\s+)?skandinavisk", + r"english\s+(?:and\s*/\s*)?or\s+(?:a\s+)?(?:scandinavian|norwegian)", + r"(?:norwegian|scandinavian)\s+(?:and\s*/\s*)?or\s+english", + r"(?:working|company|corporate|official|team(?:'s)?)\s+language\s+is\s+english", + r"arbeidsspråk(?:et)?\s+(?:er|vil\s+være)\s+engelsk", + r"norwegian\s+(?:language\s+)?(?:skills?\s+)?(?:is|are)\s+(?:not\s+(?:a\s+)?(?:required|requirement|necessary)" + r"|(?:a|an)\s+(?:plus|advantage|bonus|asset)|beneficial|nice\s+to\s+have)", + r"norsk\w*\s+(?:er\s+(?:en\s+)?(?:fordel|ønskelig|et\s+pluss)|(?:er\s+)?ikke\s+(?:et\s+)?krav)", + r"no\s+norwegian\s+(?:is\s+)?(?:required|needed|necessary)", +)] +_LANG_NORWEGIAN_REQUIRED = [re.compile(p, re.I) for p in ( + r"(?:flytende|god|gode|meget\s+gode?|svært\s+gode?|utmerkede?|beherske\w*)\s+(?:\w+\s+){0,4}norsk", + r"norsk\s+(?:muntlig\s+og\s+skriftlig|skriftlig\s+og\s+muntlig)", + r"(?:må|krav\s+om|kreves|forutsetter)\s+(?:\w+\s+){0,6}norsk", + r"norsk\w*\s+(?:og|samt)\s+engelsk", + r"skandinavisk\w*\s+språk", + r"fluent\s+(?:in\s+)?(?:norwegian|a\s+scandinavian)", + r"(?:proficien\w+|fluency|good\s+command)\s+(?:in\s+|of\s+)?(?:norwegian|a\s+scandinavian)", + r"norwegian\s+(?:language\s+)?(?:is\s+)?(?:required|mandatory|a\s+requirement|essential)", + r"(?:must|need\s+to|required\s+to)\s+(?:\w+\s+){0,4}(?:norwegian|scandinavian)", +)] +# A "Norwegian" that belongs to clearance or citizenship wording is not a language demand. +_LANG_FALSE_CONTEXT = re.compile( + r"clearance|klarering|statsborger|citizen|myndighet|authorit|armed\s+forces|forsvar", re.I) +_CITIZENSHIP_NO = re.compile(r"norsk\s+statsborger\w*|norwegian\s+citizen\w*", re.I) +_CITIZENSHIP_NEGATED = re.compile(r"(?:ikke|not|no)\s+(?:\w+\s+){0,4}$", re.I) +_CLEARANCE = re.compile(r"sikkerhetsklar\w*|security\s+clearance|klareres|klarering", re.I) + + +def _snippet(text, match, width=90): + start = max(0, match.start() - width // 2) + return re.sub(r"\s+", " ", text[start:match.end() + width // 2]).strip() + + +def classify_language_requirement(text, work_language_tags=None): + """Return {lang, evidence, citizenship_required, clearance} for a Norway posting. + + lang is one of: english_ok, norwegian_required, unclear. An explicit English-acceptable + phrase wins over a Norwegian one ("gode kunnskaper i norsk og/eller engelsk" is English + OK). With no requirement sentence, NAV's tag decides only when it is English-only. + """ + text = text or "" + tags = [t.lower() for t in (work_language_tags or [])] + result = {"lang": "unclear", "evidence": "", "citizenship_required": False, + "clearance": False} + for pattern in _LANG_ENGLISH_OK: + match = pattern.search(text) + if match: + result.update(lang="english_ok", evidence=_snippet(text, match)) + break + else: + for pattern in _LANG_NORWEGIAN_REQUIRED: + for match in pattern.finditer(text): + window = text[match.start():match.end() + 40] + if _LANG_FALSE_CONTEXT.search(window): + continue + result.update(lang="norwegian_required", evidence=_snippet(text, match)) + break + if result["lang"] == "norwegian_required": + break + else: + if tags and all(t.startswith("engelsk") or t == "english" for t in tags): + result.update(lang="english_ok", evidence="NAV work-language tag: English only") + elif tags: + result["evidence"] = f"no requirement sentence found; NAV tag: {', '.join(tags)}" + else: + result["evidence"] = "no requirement sentence found" + for match in _CITIZENSHIP_NO.finditer(text): + if not _CITIZENSHIP_NEGATED.search(text[max(0, match.start() - 40):match.start()]): + result["citizenship_required"] = True + result["citizenship_evidence"] = _snippet(text, match) + break + clearance = _CLEARANCE.search(text) + if clearance: + result["clearance"] = True + return result + + +# --- NAV national job register (Norway) ---------------------------------------------------- +# arbeidsplassen.nav.no carries most Norwegian postings, including employers with no public +# ATS API (Kongsberg, Forsvaret, Nammo, SINTEF, Kitron). Measured 2026-10-22 it is THIN for +# Equinor (2 vs ~10 on its own board), DNV (2), Nordic Semiconductor (0) and NATO (0), which +# therefore keep dedicated adapters; NAV rows that duplicate them are dropped in _main. +# RATE LIMIT: the search API answered 429 for ~10 minutes, with no Retry-After header, after +# about 20 quick requests (2026-10-21/22). The adapter therefore spends a fixed request +# budget per run, paces requests, and on a 429 stops and returns what it has with a warning +# instead of failing the run. Per-posting text checks are cached so later runs only fetch +# new postings. +NAV_SEARCH_URL = "https://arbeidsplassen.nav.no/stillinger/api/search" +NAV_AD_URL = "https://arbeidsplassen.nav.no/stillinger/stilling/{uuid}" +NAV_DETAIL_CACHE = ROOT / "state" / "nav_detail_cache.json" + + +class _RateLimited(RuntimeError): + pass + + +def _nav_get(url, pace_s): + time.sleep(pace_s) + req = urllib.request.Request(url, headers={"User-Agent": USER_AGENT}) + try: + with urllib.request.urlopen(req, timeout=40, context=_ssl_context()) as resp: + return resp.read().decode("utf-8", "replace") + except urllib.error.HTTPError as e: + if e.code == 429: + raise _RateLimited(url) from e + raise + + +def _html_to_text(raw): + text = re.sub(r"|", " ", raw, flags=re.S | re.I) + return re.sub(r"\s+", " ", html.unescape(re.sub(r"<[^>]+>", " ", text))).strip() + + +def _nav_job(source): + """Map one NAV search hit (_source) to a scout job dict, or None if not usable.""" + if source.get("status") != "ACTIVE" or not source.get("uuid"): + return None + props = source.get("properties") or {} + cities = [] + for place in source.get("locationList") or []: + city = (place.get("city") or place.get("municipal") or "").strip() + if city: + label = f"{city.title()}, Norway" + if label not in cities: + cities.append(label) + summary = ((source.get("generatedSearchMetadata") or {}).get("shortSummary") or "") + tags = " ".join(props.get("searchtagsai") or []) + employer = source.get("businessName") or (source.get("employer") or {}).get("name") or "" + due = props.get("applicationdue") or "" + return { + "id": source["uuid"], + "title": source.get("title") or "", + "employer": employer.strip(), + "location": " | ".join(cities) or "Norway", + "country": "NO", + "url": NAV_AD_URL.format(uuid=source["uuid"]), + "posted": (source.get("published") or "")[:10], + "deadline": due if re.match(r"\d{4}-\d{2}-\d{2}", due) else "", + "description": f"{summary} {tags}".strip(), + "work_language": props.get("workLanguage") or [], + } + + +def fetch_nav(args): + """NAV register search: keyword queries, Norway outside Oslo, language-checked.""" + warnings = args.setdefault("_run_warnings", []) + warnings.clear() + pace = args.get("pace_s", 3.0) + budget = args.get("search_budget", 14) + detail_budget = args.get("detail_budget", 25) + jobs, used = {}, 0 + try: + for query in args["queries"]: + for page in range(args.get("pages_per_query", 1)): + if used >= budget: + break + url = NAV_SEARCH_URL + "?" + urllib.parse.urlencode( + {"q": query, "size": 100, "from": page * 100}) + data = json.loads(_nav_get(url, pace)) + used += 1 + hits = (data.get("hits") or {}).get("hits") or [] + for hit in hits: + job = _nav_job(hit.get("_source") or {}) + if job and job["id"] not in jobs: + jobs[job["id"]] = job + if len(hits) < 100: + break + except _RateLimited: + warnings.append(f"NAV rate limit (HTTP 429) after {used} search request(s); this run " + f"is PARTIAL ({len(jobs)} postings). It usually clears within hours.") + + # Text check: cached verdicts first, then fetch the full ad for relevant postings outside + # Oslo until the detail budget runs out. Irrelevant postings never cost a request. + try: + cache = json.loads(NAV_DETAIL_CACHE.read_text(encoding="utf-8")) + except (FileNotFoundError, ValueError): + cache = {} + fetched = pending = 0 + blocked = any("rate limit" in w for w in warnings) + for job in jobs.values(): + cached = cache.get(job["id"]) + if cached: + job["lang_check"] = cached + continue + relevant = score_job(job)[0] >= 2 and location_eligibility( + job["location"], policy="norway_no_oslo", country="NO")[0] + if relevant and not blocked and fetched < detail_budget: + try: + text = _html_to_text(_nav_get(job["url"], pace)) + fetched += 1 + verdict = classify_language_requirement(text, job["work_language"]) + verdict["checked"] = datetime.now(timezone.utc).strftime("%Y-%m-%d") + cache[job["id"]] = verdict + job["lang_check"] = verdict + continue + except _RateLimited: + blocked = True + warnings.append("NAV rate limit hit during posting-text checks; remaining " + "postings use the work-language tag only.") + except (urllib.error.URLError, ValueError) as e: + warnings.append(f"NAV posting {job['id']}: text fetch failed ({e!r})") + if relevant: + pending += 1 + verdict = classify_language_requirement(job["description"], job["work_language"]) + verdict["provisional"] = True + job["lang_check"] = verdict + if pending: + warnings.append(f"{pending} relevant NAV posting(s) have no full-text language check yet " + f"(budget {detail_budget}/run); they are marked provisional.") + _atomic_write_json(NAV_DETAIL_CACHE, cache) + return list(jobs.values()) + + +def fetch_radancy(args): + """Radancy (TalentBrew) careers sites, e.g. careers.arm.com in front of iCIMS. + + The /search-jobs/results endpoint returns JSON whose `results` field is the rendered + result-list HTML; each card has the job link, data-job-id and a `location` span. One + keyword search per term in args["keywords"]. LIMITATION: a multi-location role whose card + shows another city first (e.g. Shanghai) is dropped by the location policy even when + Trondheim is one of its locations - the card does not list them all. + """ + base = args["base_url"].rstrip("/") + jobs, seen_ids = [], set() + for keyword in args.get("keywords", [""]): + query = urllib.parse.urlencode({ + "ActiveFacetID": 0, "CurrentPage": 1, "RecordsPerPage": 200, "Keywords": keyword, + "SearchType": 5, "SearchResultsModuleName": "Search Results", + "SearchFiltersModuleName": "Search Filters"}) + data = http_get_json(f"{base}/search-jobs/results?{query}", + headers={"X-Requested-With": "XMLHttpRequest"}) + results = data.get("results") or "" + for card in re.findall(r"]*job-card.*?", results, re.S): + link = re.search(r'href="(/job/[^"]+)"[^>]*data-job-id="(\d+)"[^>]*>(.*?)', card, re.S) + if not link or link.group(2) in seen_ids: + continue + seen_ids.add(link.group(2)) + loc = re.search(r'class="location"[^>]*>(.*?)', card, re.S) + intro = re.search(r'class="job-card__intro"[^>]*>(.*?)', card, re.S) + jobs.append({ + "id": link.group(2), + "title": _html_to_text(link.group(3)), + "location": _html_to_text(loc.group(1)) if loc else "", + "url": base + link.group(1), + "description": _html_to_text(intro.group(1)) if intro else "", + }) + return jobs + + +def fetch_algolia_jobs(args): + """Careers sites that search an Algolia index in the browser (DNV: jobs.dnv.com). + + The app id, search-only key and index name are public, embedded in the search page as + window.AG_ID / AG_KEY / AG_INDEX. They are read fresh on every run rather than hardcoded, + so a rotated key does not silently break the adapter. DNV's index held 237 jobs worldwide + on 2026-10-22, so one 1000-hit query covers the whole board. + """ + page = urllib.request.Request(args["page_url"], headers={ + "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/126.0"}) + with urllib.request.urlopen(page, timeout=30, context=_ssl_context()) as resp: + raw = resp.read().decode("utf-8", "replace") + app = re.search(r'window\.AG_ID\s*=\s*"([^"]+)"', raw) + key = re.search(r'window\.AG_KEY\s*=\s*"([^"]+)"', raw) + index = re.search(r'window\.AG_INDEX\s*=\s*(\{[^;]+\})', raw) + if not (app and key and index): + raise RuntimeError("Algolia config (AG_ID/AG_KEY/AG_INDEX) not found on the search page") + index_name = json.loads(index.group(1)).get(args.get("index_key", "jobs")) + data = http_get_json( + f"https://{app.group(1)}-dsn.algolia.net/1/indexes/{index_name}/query", + headers={"X-Algolia-Application-Id": app.group(1), "X-Algolia-API-Key": key.group(1)}, + data={"params": "query=&hitsPerPage=1000"}, method="POST") + origin = re.match(r"https?://[^/]+", args["page_url"]).group(0) + jobs = [] + for hit in data.get("hits") or []: + countries = hit.get("country") or [] + if isinstance(countries, str): + countries = re.findall(r"[A-Za-z][A-Za-z .'-]+", countries) + location = hit.get("display_location") or "" + if countries and not any(c.lower() in location.lower() for c in countries): + location = f"{location} | {', '.join(countries)}".strip(" |") + path = hit.get("jd_url") or "" + jobs.append({ + "id": str(hit.get("objectID") or path), + "title": hit.get("title") or "", + "location": location, + "url": path if path.startswith("http") else origin + path, + "unit": hit.get("business_unit") or "", + }) + return jobs + + ADAPTERS = { + "nav": fetch_nav, + "radancy": fetch_radancy, + "algolia_jobs": fetch_algolia_jobs, "ibm": fetch_ibm, "workday": fetch_workday, "ashby": fetch_ashby, @@ -1893,6 +2310,28 @@ COUNTRY_NAMES = { } +def _has_non_oslo_norway_place(loc_text): + """True unless every location segment that can be placed is in the Oslo area. + + Segments are split on | ; / and newlines, so "Trondheim, Norway | Oslo, Norway" keeps the + role and "Oslo, Norway, 0010 Oslo" drops it. A bare "Norway" or an unknown town counts as + outside Oslo: we cannot tell, and dropping it would hide real roles silently. + """ + # Only segments that name a Norwegian place count. Workday location strings carry the job + # slug ("/job/Oslo-Norway/Senior-Data-Engineer_JR123"), and treating the slug as a place + # would wrongly keep every Oslo-only Workday role. + segments = [s.strip().lower() for s in re.split(r"[|;/\n]", loc_text or "") if s.strip()] + placed = False + for seg in segments: + if not _place_match(seg, NO_LOCATION_KEYWORDS + OSLO_AREA_PLACES): + continue + if _place_match(seg, OSLO_AREA_PLACES): + placed = True + continue + return True + return not placed + + def location_eligibility(loc_text, policy="default", country=None): """Decide whether a posting's location is in-scope. @@ -1907,8 +2346,10 @@ def location_eligibility(loc_text, policy="default", country=None): (e.g. Kraken "UK | Brazil | Ireland | ..." with no Switzerland/Europe) Overrides (args["_location_policy"]): - - norway_only — Equinor: Norway only - - dk_no — NATO-style: Denmark + Norway only (no CH) + - norway_only — Norway only, Oslo included + - norway_no_oslo — Norway with at least one location outside the Oslo area (Equinor, + Telenor, NAV and the other Norway boards; user preference) + - dk_no — NATO-style: Denmark + Norway only (no CH) """ if not loc_text: return False, False, False, False @@ -1945,6 +2386,9 @@ def location_eligibility(loc_text, policy="default", country=None): if policy == "norway_only": eligible = in_no and not is_us_only return eligible, False, False, eligible + if policy == "norway_no_oslo": + eligible = in_no and not is_us_only and _has_non_oslo_norway_place(loc_text) + return eligible, False, False, eligible if policy == "dk_no": # NATO etc.: Denmark + Norway only — CH onsite is out (no NATO seat in CH). eligible = (in_dk or in_no) and not is_us_only @@ -2035,10 +2479,15 @@ def save_seen(seen): # Board-specific req-ID patterns, tried in order against the URL path+query. Every key is # scoped by host (see _decision_key), so an ID only ever has to be unique within one board # and cross-board collisions are impossible by construction. +_HOST_ALIASES = {"apply.careers.microsoft.com": "jobs.careers.microsoft.com"} + _JOB_ID_PATTERNS = [ # Amazon: the slug is truncated to a fixed width and that width has changed between runs, # so the same req yields different URLs. This is the drift that motivated the whole helper. (r"amazon\.jobs$", r"/jobs/(\d+)"), + # New-form links carry the display id as ?jobId= (the /job/ path holds the internal + # pcsx id), so this must come before the /job/ pattern. Old-form links fall through to it. + (r"careers\.microsoft\.com$", r"[?&]jobId=(\d+)"), (r"careers\.microsoft\.com$", r"/job/(\d+)"), (r"google\.com$", r"/results/(\d+)-"), # Workday: trailing _REQID on the job slug (Roche 202608-120791-1, Novartis REQ-…, NVIDIA JR…). @@ -2093,6 +2542,9 @@ def _decision_key(url): host = (parts.netloc or "").lower().split(":")[0] if host.startswith("www."): host = host[4:] + # One board, two hostnames: keep decisions recorded under the old Microsoft link form + # linked to rows carrying the new one (2026-10-22 link fix). + host = _HOST_ALIASES.get(host, host) target = parts.path + (f"?{parts.query}" if parts.query else "") for host_pat, id_pat in _JOB_ID_PATTERNS: @@ -2249,6 +2701,143 @@ def _parse_posted(s): return None +# --- Employer reputation (user requirement 2026-10-22) ----------------------------------- +# "The company should have a good reputation for working there." Glassdoor blocks direct +# access (HTTP 403), so ratings are LOOKUPS recorded in state/employer_reputation.json with +# date and source, not live data. Every report row shows the verdict; an employer nobody has +# looked up is UNCHECKED and must be looked up before a role there is recommended. +REPUTATION_FILE = ROOT / "state" / "employer_reputation.json" +REPUTATION_STALE_DAYS = 365 + + +def load_reputation(): + try: + return json.loads(REPUTATION_FILE.read_text(encoding="utf-8")) + except (FileNotFoundError, ValueError): + return {"thresholds": {}, "employers": {}} + + +def _norm_name(name): + name = re.sub(r"\b(?:as|asa|ab|ag|gmbh|inc|ltd|plc|sa|nv|bv)\b\.?", " ", (name or "").lower()) + return re.sub(r"[^a-z0-9æøå]+", " ", name).strip() + + +def reputation_for(name, registry): + """Registry entry plus computed verdict for an employer name, or an UNCHECKED stub.""" + norm = _norm_name(name) + thresholds = registry.get("thresholds") or {} + def hit(alias): + alias = _norm_name(alias) + return bool(alias) and bool( + re.search(r"(? REPUTATION_STALE_DAYS: + return "stale" + except ValueError: + pass + rating = (entry.get("glassdoor") or {}).get("rating") + reviews = (entry.get("glassdoor") or {}).get("reviews") or 0 + if rating is None: + return "ok" if entry.get("secondary") else "unchecked" + good_min = thresholds.get("good_min_rating", 3.8) + min_reviews = thresholds.get("good_min_reviews", 50) + ok_min = thresholds.get("ok_min_rating", 3.5) + if rating >= good_min and reviews >= min_reviews: + return "good" + if rating >= good_min and entry.get("secondary"): + return "ok" + if rating >= ok_min: + return "ok" + return "avoid" + + +def _reputation_line(rep): + verdict = rep.get("verdict", "unchecked").upper() + if verdict == "UNCHECKED": + return "UNCHECKED - look up before recommending (state/employer_reputation.json)" + gd = rep.get("glassdoor") or {} + bits = [] + if gd.get("rating") is not None: + reviews = f", {gd['reviews']:,} reviews" if gd.get("reviews") else "" + scope = f", {gd['scope']}" if gd.get("scope") else "" + bits.append(f"Glassdoor {gd['rating']}/5{reviews}{scope}") + if rep.get("secondary"): + bits.append(rep["secondary"]) + if rep.get("checked"): + bits.append(f"checked {rep['checked']}") + note = f" - {rep['note']}" if rep.get("note") else "" + return f"{verdict} ({'; '.join(bits)}){note}" + + +def _norm_title(title): + return re.sub(r"[^a-z0-9æøå]+", " ", (title or "").lower()).strip() + + +def drop_nav_duplicates(results, companies): + """Drop NAV rows that repeat a posting a dedicated board already returned. + + Matching is employer alias (args["nav_aliases"]) plus normalized title. A NAV row for an + employer with a dedicated board but a different title is kept: it may be a role the board + does not list. Returns (kept_rows, dropped_count). + """ + alias_by_display = {display: [_norm_name(a) for a in (args.get("nav_aliases") or [])] + for _, display, _, args in companies} + dedicated = {} + for row in results: + if row.get("source"): + continue + for alias in alias_by_display.get(row["company"], []): + dedicated.setdefault(alias, set()).add(_norm_title(row["title"])) + kept, dropped = [], 0 + for row in results: + if row.get("source"): + employer = _norm_name(row["company"]) + if any(re.search(r"(?= 6] - medium = [r for r in results if 2 <= r["score"] < 6] - weak = [r for r in results if r["score"] < 2] + # Excluded employers (user: never Vizrt again) are hidden; Norway roles that fail the + # language or citizenship check go to a compact list at the end instead of the buckets, + # so a misread requirement sentence can still be spotted and overridden. + excluded = [r for r in results if (r.get("reputation") or {}).get("verdict") == "excluded"] + gated = [r for r in results if r not in excluded and _lang_gated(r)] + hidden_ids = {id(r) for r in excluded + gated} + visible = [r for r in results if id(r) not in hidden_ids] + if excluded: + names = sorted({r["company"] for r in excluded}) + lines.append(f"_{len(excluded)} role(s) hidden from excluded employers: {', '.join(names)}._") + n_unchecked = sum(1 for r in visible if r["score"] >= 2 and + (r.get("reputation") or {}).get("verdict") in (None, "unchecked", "stale")) + if n_unchecked: + lines.append(f"_{n_unchecked} shown role(s) are at employers with no current reputation " + f"check - look them up before recommending._") - if not include_weak and weak: - lines.append(f"\n_Hiding {len(weak)} weak/noise roles (score < 2). Use --include-weak to show._") + weak_all = [r for r in visible if r["score"] < 2] + if not include_weak and weak_all: + lines.append(f"\n_Hiding {len(weak_all)} weak/noise roles (score < 2). Use --include-weak to show._") n_decided = sum(1 for r in results if is_same_decision( lookup_decision(decisions, _raw_decisions(), r["url"]))) if n_decided: @@ -2306,59 +2908,100 @@ def write_report(path, results, errors, new_only, include_weak, stats=None, tota lines.append(f"_{n_decided} role(s) already in the decision log ({shown}; " f"see state/decisions.json)._") - # F14: this number is SEARCH RELEVANCE - how many profile keywords the posting echoes. - # It is not a fit assessment and must never be presented as one: "Senior AI Engineer" - # with GenAI/LLM/RAG scores 12 while "Senior Data Engineer" scores 3, although the - # second sits far closer to his actual evidence. Fit is decided on the full JD. - buckets = [("High keyword relevance (score >= 6) - NOT a fit judgement", strong), - ("Medium keyword relevance (score 2-5)", medium)] - if include_weak: - buckets.append(("Low keyword relevance (score < 2) - German titles land here", weak)) + def render_row(r): + d = lookup_decision(decisions, _raw_decisions(), r["url"]) + new_tag = " [NEW]" if r["is_new"] else "" + decided_tag = f" — 🗂 {d['decision'].upper()}" if d else "" + loc_tag = ("CH" if r["in_ch"] else + "Remote" if r["remote"] else + "Relocation" if r.get("relocation") else "?") + out = [f"### [{r['score']}] {r['company']} - {r['title']}{new_tag}{decided_tag}", + f"- Location: {r['location']} *({loc_tag})*"] + if r.get("source"): + out.append(f"- Source: {r['source']}") + if r.get("lang"): + out.append(f"- Language: {_lang_line(r['lang'])}") + if r["lang"].get("clearance"): + out.append("- Security clearance mentioned (not a gate; runs after an offer)") + if r.get("reputation"): + out.append(f"- Reputation: {_reputation_line(r['reputation'])}") + if r.get("posted"): + out.append(f"- Posted: {r['posted']}") + if r.get("deadline"): + out.append(f"- **Bewerbungsfrist: {r['deadline']}**") + if r.get("updated_at"): + out.append(f"- Updated: {r['updated_at']}") + if r.get("unit"): + out.append(f"- Einheit: {r['unit']}") + if r.get("compensation"): + out.append(f"- Vergütung: {r['compensation']}") + if r.get("contact"): + # Channel is the rationed variable: a published, reachable name means a cold + # submit is a choice rather than the only option. + out.append(f"- **Ansprechperson: {r['contact']}**") + out.append(f"- URL: {r['url']}") + if d: + note = f" — {d['note']}" if d.get("note") else "" + if d.get("matched_by") == "slug": + out.append( + f"- 🗂 RELATED decision (same slug, **new requisition id** — a repost can " + f"differ, so re-read before reusing): **{d['decision']}**{note} " + f"({d.get('date','')}) · matched `{d.get('matched_url','')}`") + else: + out.append(f"- 🗂 Decision: **{d['decision']}**{note} ({d.get('date','')})") + if r["pos"]: + out.append(f"- Positive: {', '.join(r['pos'])}") + if r["neg"]: + out.append(f"- Negative: {', '.join(r['neg'])}") + out.append("") + return out - for bucket_name, bucket in buckets: - shown = [r for r in bucket - if not (hide_decided and - is_same_decision(lookup_decision(decisions, _raw_decisions(), r["url"])))] - if not shown: + # User preference 2026-10-22: Norway and Switzerland are separate searches and are always + # listed separately. Europe-remote and Denmark roles get their own section so they are + # never mixed into either. + for region in REGION_ORDER: + region_rows = [r for r in visible if r.get("region") == region] + region_gated = [r for r in gated if r.get("region") == region] + if not region_rows and not region_gated: continue - lines.append(f"\n## {bucket_name} - {len(shown)} role(s)\n") - for r in shown: - d = lookup_decision(decisions, _raw_decisions(), r["url"]) - new_tag = " [NEW]" if r["is_new"] else "" - decided_tag = f" — 🗂 {d['decision'].upper()}" if d else "" - loc_tag = ("CH" if r["in_ch"] else - "Remote" if r["remote"] else - "Relocation" if r.get("relocation") else "?") - lines.append(f"### [{r['score']}] {r['company']} - {r['title']}{new_tag}{decided_tag}") - lines.append(f"- Location: {r['location']} *({loc_tag})*") - if r.get("posted"): - lines.append(f"- Posted: {r['posted']}") - if r.get("deadline"): - lines.append(f"- **Bewerbungsfrist: {r['deadline']}**") - if r.get("updated_at"): - lines.append(f"- Updated: {r['updated_at']}") - if r.get("unit"): - lines.append(f"- Einheit: {r['unit']}") - if r.get("compensation"): - lines.append(f"- Vergütung: {r['compensation']}") - if r.get("contact"): - # Channel is the rationed variable: a published, reachable name means a cold - # submit is a choice rather than the only option. - lines.append(f"- **Ansprechperson: {r['contact']}**") - lines.append(f"- URL: {r['url']}") - if d: - note = f" — {d['note']}" if d.get("note") else "" - if d.get("matched_by") == "slug": - lines.append( - f"- 🗂 RELATED decision (same slug, **new requisition id** — a repost can " - f"differ, so re-read before reusing): **{d['decision']}**{note} " - f"({d.get('date','')}) · matched `{d.get('matched_url','')}`") - else: - lines.append(f"- 🗂 Decision: **{d['decision']}**{note} ({d.get('date','')})") - if r["pos"]: - lines.append(f"- Positive: {', '.join(r['pos'])}") - if r["neg"]: - lines.append(f"- Negative: {', '.join(r['neg'])}") + n_shown = sum(1 for r in region_rows if r["score"] >= 2 or include_weak) + lines.append(f"\n# {region} - {n_shown} role(s) shown, {len(region_rows)} eligible\n") + + # F14: this number is SEARCH RELEVANCE - how many profile keywords the posting + # echoes. It is not a fit assessment and must never be presented as one: "Senior AI + # Engineer" with GenAI/LLM/RAG scores 12 while "Senior Data Engineer" scores 3, + # although the second sits far closer to his actual evidence. Fit is decided on the + # full JD. + buckets = [("High keyword relevance (score >= 6) - NOT a fit judgement", + [r for r in region_rows if r["score"] >= 6]), + ("Medium keyword relevance (score 2-5)", + [r for r in region_rows if 2 <= r["score"] < 6])] + if include_weak: + buckets.append(("Low keyword relevance (score < 2) - German/Norwegian titles land here", + [r for r in region_rows if r["score"] < 2])) + for bucket_name, bucket in buckets: + shown = [r for r in bucket + if not (hide_decided and + is_same_decision(lookup_decision(decisions, _raw_decisions(), r["url"])))] + if not shown: + continue + lines.append(f"\n## {region}: {bucket_name} - {len(shown)} role(s)\n") + for r in shown: + lines.extend(render_row(r)) + + if region_gated: + relevant_gated = [r for r in region_gated if r["score"] >= 2 or include_weak] + lines.append(f"\n## {region}: filtered out by language/citizenship - " + f"{len(region_gated)} ({len(relevant_gated)} relevant shown)\n") + lines.append("Verdicts come from the posting's requirement sentence. If one is " + "misread, the role is still in the JSON dump.\n") + for r in relevant_gated: + lang = r.get("lang") or {} + why = ("Norwegian citizenship required: " + lang.get("citizenship_evidence", "") + if lang.get("citizenship_required") + else "Norwegian required: " + lang.get("evidence", "")) + lines.append(f"- [{r['score']}] {r['company']} - {r['title']} ({r['location']}) - " + f"{why[:180]} - {r['url']}") lines.append("") if MANUAL_CHECK: @@ -2487,8 +3130,27 @@ def _process_company(cid, display, args, jobs, seen, today, last_scrape): score = floor if score >= 2: match += 1 + # Aggregator rows (NAV) carry the real employer; the board name becomes the source. + employer = (j.get("employer") or "").strip() + lang = j.get("lang_check") + if lang is None and (policy.startswith("norway") or j.get("country") == "NO"): + lang = classify_language_requirement(j.get("description") or "", + j.get("work_language")) + if not j.get("description"): + lang["evidence"] = "no posting text from this board; read the JD" + if (policy.startswith("norway") or j.get("country") == "NO" or + (policy == "dk_no" and _place_match((j.get("location") or "").lower(), + NO_LOCATION_KEYWORDS))): + region = "Norway" + elif in_ch: + region = "Switzerland" + else: + region = "Europe-remote / Denmark" row = { - "company": display, "company_id": cid, + "company": employer or display, "company_id": cid, + "source": display if employer else "", + "region": region, + "lang": lang, "title": j["title"], "location": j["location"], "url": j["url"], "posted": j.get("posted", ""), "updated_at": j.get("updated_at", ""), @@ -2546,8 +3208,15 @@ def _main(): print(f"Recorded: {status} — {url}", file=sys.stderr) return - only, new_only, include_weak, hide_decided = None, False, False, False + only, new_only, include_weak, hide_decided, region = None, False, False, False, None for arg in sys.argv[1:]: + if arg.startswith("--region="): + region = arg.split("=", 1)[1].strip().lower() + if region not in ("ch", "no"): + print("Usage: --region=ch (Switzerland + Europe-remote boards) or " + "--region=no (Norway boards)", file=sys.stderr) + sys.exit(2) + continue if arg in ("--help", "-h"): print(__doc__) return @@ -2561,7 +3230,8 @@ def _main(): only = arg.split("=", 1)[1] targets = [(cid, display, adapter, args) for cid, display, adapter, args in COMPANIES - if not only or cid == only] + if (not only or cid == only) + and (not region or _company_region(args) == region)] if only and not targets: known = ", ".join(c[0] for c in COMPANIES) print(f"Unknown --only={only!r}. Known ids: {known}", file=sys.stderr) @@ -2621,6 +3291,8 @@ def _main(): results, stat, warnings = _process_company( cid, display, args, jobs, seen, today, last_scrape ) + # Adapters that degrade instead of failing (NAV on a rate limit) leave notes here. + warnings = [(display, w) for w in (args.get("_run_warnings") or [])] + warnings stat["secs"] = secs for w in warnings: errors.append(w) @@ -2642,6 +3314,14 @@ def _main(): total_secs = time.perf_counter() - run_start + all_results, n_dup = drop_nav_duplicates(all_results, targets) + if n_dup: + errors.append(("NAV register", f"{n_dup} posting(s) dropped as duplicates of a " + f"dedicated employer board")) + registry = load_reputation() + for r in all_results: + r["reputation"] = reputation_for(r["company"], registry) + if new_only: all_results = [r for r in all_results if r["is_new"]] @@ -2654,6 +3334,8 @@ def _main(): scope = [] if only: scope.append(f"only-{only}") + if region: + scope.append(f"region-{region}") if new_only: scope.append("new-only") if hide_decided: @@ -2677,6 +3359,12 @@ def _main(): "date_provenance": r.get("date_provenance", ""), "unit": r.get("unit", ""), "compensation": r.get("compensation", ""), "contact": r.get("contact", ""), "is_new": r["is_new"], + "region": r.get("region", ""), + "source": r.get("source", ""), + "language": r.get("lang"), + "reputation": {k: r["reputation"].get(k) for k in + ("verdict", "glassdoor", "secondary", "checked", "note") + if r["reputation"].get(k) is not None}, "decision": lookup_decision(dec_index, decisions, r["url"]), "keyword_signal": {"score": r["score"], "pos": r["pos"], "neg": r["neg"]}, "loc_flags": { @@ -2709,16 +3397,19 @@ def _main(): # dk_no — NATO (Denmark + Norway only; no CH) # # Automated (COMPANIES above): -# workday nvidia, novartis, equinor (norway_only) +# nav nav (NAV national register, Norway outside Oslo; added 2026-10-22) +# radancy arm (Trondheim) +# algolia_jobs dnv +# workday nvidia, novartis, equinor (norway_no_oslo) # ashby kraken, openai, snowflake # ibm confluent (IBM careers search API since 2026-10-15; Ashby board empty) # greenhouse anthropic, gitlab, grafana, databricks, datadog, elastic, fivetran -# pcsx microsoft +# pcsx microsoft, microsoft_no # smartrecruiters metgroup, cern # rss bis # amazon amazon (Amazon + AWS share one board) # phenom roche (Cisco still uses the playwright path) -# teamtailor telenor, axpo (custom domain via base_url) +# teamtailor telenor, axpo, nordic (custom domain via base_url) # getro (no company configured — adapter kept for future VC talent networks) # onlyfy bitcoin_suisse # lever palantir, quantco diff --git a/job_scout/state/employer_reputation.json b/job_scout/state/employer_reputation.json new file mode 100644 index 0000000..ecb42da --- /dev/null +++ b/job_scout/state/employer_reputation.json @@ -0,0 +1,100 @@ +{ + "description": "Employer reputation lookups for job_scout (user requirement 2026-10-22: only recommend roles at employers with a good reputation as a place to work). Glassdoor blocks direct access (HTTP 403), so every figure here is a dated lookup from search results, not live data. Verdicts are computed by scout.reputation_verdict() from the thresholds below unless verdict_override is set. Entries older than 365 days show as STALE and must be re-checked. An employer missing from this file is UNCHECKED: look it up before recommending a role there.", + "thresholds": { + "good_min_rating": 3.8, + "good_min_reviews": 50, + "ok_min_rating": 3.5, + "rule": "good: Glassdoor >= 3.8 with >= 50 reviews. ok: 3.5-3.79, or >= 3.8 on fewer than 50 reviews backed by a second source, or no Glassdoor page but a solid second source. avoid: below 3.5 unless the user overrides. excluded: user decision.", + "approved": "user, 2026-10-22" + }, + "employers": { + "Vizrt": { + "aliases": [], + "verdict_override": "excluded", + "note": "User decision 2026-10-22: worked there (Bergen, 2017-18) and does not want to work there again. Roles are hidden from reports.", + "checked": "2026-10-22" + }, + "Arm": { + "aliases": ["Arm Norway", "ARM Norway"], + "glassdoor": {"rating": 4.5, "reviews": 2633, "scope": "company-wide"}, + "note": "Trondheim-specific reviews are mixed: recent ones positive, a 2023 one criticised local middle management for engineers relocating from abroad.", + "sources": ["https://static.glassdoor.at/Reviews/Arm-Reviews-E7834_P800.htm", "https://static-pc.glassdoor.fr/Reviews/Employee-Review-Arm-E7834-RVW80607582.htm"], + "checked": "2026-10-22" + }, + "Nordic Semiconductor": { + "aliases": ["Nordic Semi"], + "glassdoor": {"rating": 4.0, "reviews": 103, "recommend_pct": 83, "scope": "company-wide"}, + "secondary": "TrueUp 82% recommend (100+ reviews)", + "sources": ["https://www.glassdoor.com/Overview/X-EI_IE681660.htm", "https://www.trueup.io/co/nordic-semi"], + "checked": "2026-10-22" + }, + "Equinor": { + "aliases": [], + "glassdoor": {"rating": 4.3, "reviews": 769, "recommend_pct": 87, "scope": "company-wide"}, + "secondary": "Indeed 4.0 (299); Universum: top student employer in engineering/business/technology 2024", + "sources": ["https://www.glassdoor.co.uk/Reviews/Equinor-Reviews-E10408.htm", "https://uk.indeed.com/cmp/Equinor/reviews"], + "checked": "2026-10-22" + }, + "Kongsberg": { + "aliases": ["Kongsberg Gruppen", "Kongsberg Maritime", "Kongsberg Digital", "Kongsberg Defence", "Kongsberg Discovery", "Kongsberg Satellite Services", "KSAT", "Kongsberg Norcontrol", "Kongsberg Seatex"], + "not_aliases": ["Kongsberg Automotive"], + "glassdoor": {"rating": null, "reviews": null}, + "secondary": "No Glassdoor page found; Indeed 3.8 (101 reviews, group); Universum 2025 professionals: 3rd in IT, 2nd in Engineering", + "note": "Ratings vary by division; Kongsberg Maritime JobStreet sample is small (7). Kongsberg Automotive is an unrelated company and is NOT covered by this entry.", + "sources": ["https://uk.indeed.com/cmp/Kongsberg-1", "https://www.kongsberg.com/news/stories/2025/11/Professionals-place-KONGSBERG-among-Norways-top-employers-in-Universum/"], + "checked": "2026-10-22" + }, + "DNV": { + "aliases": ["DNV AS"], + "glassdoor": {"rating": 4.2, "reviews": 94, "recommend_pct": 82, "scope": "company-wide (CA/NZ snapshot; UK page 3.9 on 47)"}, + "secondary": "Indeed 3.8 (444 reviews, Feb 2026)", + "note": "Praised for flexibility and people; common complaints are pay and slow progression.", + "sources": ["https://www.glassdoor.ca/Reviews/DNV-Reviews-E6648554.htm", "https://uk.indeed.com/cmp/Dnv/reviews"], + "checked": "2026-10-22" + }, + "SINTEF": { + "aliases": [], + "glassdoor": {"rating": 4.0, "reviews": 62, "recommend_pct": 81, "scope": "company-wide"}, + "note": "Autonomy and good people; salary on the lower end, some bureaucracy for project leads.", + "sources": ["https://www.glassdoor.co.in/Reviews/SINTEF-Reviews-E458950.htm"], + "checked": "2026-10-22" + }, + "Telenor": { + "aliases": ["Telenor Norge", "Telenor ASA"], + "glassdoor": {"rating": 3.8, "reviews": 1305, "recommend_pct": 65, "scope": "company-wide, not Norway-specific"}, + "secondary": "Indeed 4.1 (311)", + "note": "Exactly at the threshold. Good work-life balance; complaints about limited growth and frequently changing strategy.", + "sources": ["https://www.glassdoor.com/Reviews/Telenor-Reviews-E11282.htm"], + "checked": "2026-10-22" + }, + "Microsoft": { + "aliases": ["Microsoft Norge", "Microsoft Development Center Norway"], + "glassdoor": {"rating": 4.0, "reviews": 54474, "recommend_pct": 76, "scope": "company-wide"}, + "sources": ["https://static-pc.glassdoor.fr/Reviews/Microsoft-Reviews-E1651.htm"], + "checked": "2026-10-22" + }, + "NATO": { + "aliases": ["NATO JWC", "Joint Warfare Centre"], + "glassdoor": {"rating": 4.0, "reviews": 240, "recommend_pct": 93, "scope": "all NATO bodies"}, + "note": "Pay, benefits, international setting; heavy workload, bureaucracy, rigid military structure. Reviews do not separate civilian staff.", + "sources": ["https://www.glassdoor.com/Reviews/NATO-Reviews-E22846.htm"], + "checked": "2026-10-22" + }, + "Nammo": { + "aliases": [], + "glassdoor": {"rating": 3.9, "reviews": 21, "recommend_pct": 56, "scope": "company-wide"}, + "secondary": "Indeed 3.2 (87 reviews)", + "verdict_override": "avoid", + "note": "Thin and mixed: Glassdoor 3.9 on only 21 reviews with 56% recommending; the larger Indeed sample is 3.2, below the 3.5 line. A 2026 review alleges production pressure compromising safety. User may override.", + "sources": ["https://www.glassdoor.co.nz/Reviews/Nammo-Reviews-E4444878.htm", "https://www.indeed.com/cmp/Nammo"], + "checked": "2026-10-22" + }, + "Kitron": { + "aliases": [], + "glassdoor": {"rating": 3.1, "reviews": 15, "recommend_pct": 34, "scope": "company-wide"}, + "note": "Below threshold on a thin sample: limited growth, pay, unclear strategy.", + "sources": ["https://www.glassdoor.co.uk/Reviews/Kitron-Reviews-E311133.htm"], + "checked": "2026-10-22" + } + } +}