dennisthiessen/claude-resume-kit
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
3a9c2faf44c68cb60277d9c02ba736caa28d8c0d
claude-resume-kit/job_scout/scout.py
T
dennisthiessenandClaude Opus 5.5 ff12f16545 fix(scout): Apple blind to CH for 2 months, BIS behind Cloudflare, NAV rate limit, EMEA leak; reputation for every board 
Apple (the big one)
- The location=switzerland-CHE slug stopped resolving; Apple silently applied
  a US location, so every run from at least 2026-08-31 returned the same 21
  US/intern cards and 0 Swiss roles. Now location code CHEC (from
  /api/v1/refData/postlocation), result list item as the card (location
  readable), pagination, settle delay, longer wait and per-element timeout.
  Verified twice: 24 Swiss roles each run.

BIS
- bis.org went behind a Cloudflare challenge (403 even headless). Moved to the
  Workday tenant bis.wd3.myworkdayjobs.com/External, which also carries the
  office. Host alias + lower-cased jr-id keep all 9 bis.org#jr decisions
  linked (applied jr100498 included).

NAV
- Search API 429s after ~9 requests and stays blocked for hours: discovery
  capped at 8 searches (4s pacing). Posting text now comes from NAV's official
  third-party feed (pam-stilling-feed, public rotating token), with a small,
  slow website-page fallback for ads the feed lacks (~2/3). Verified live on
  15 real ads; fallback path tested on fixtures.

Location filter
- EMEA leak fixed: a country allowlist without CH/DK plus a trailing region
  tag is no longer Europe-remote (27 rows). Swiss/Danish segments in
  multi-location strings are evaluated per segment. Regression over all
  2026-10-22 rows: only the fenced rows changed.

Reputation
- Every board now has a dated verdict; kununu added as a primary source next
  to Glassdoor (the one with more reviews decides; user tip). Second-source
  scores now count (a low one means avoid). AVOID: BIS (Glassdoor 2.9),
  BFH (kununu 3.3), Kdo Cy (federal administration kununu 3.4).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 10:44:45 +02:00

3645 lines
183 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""Job scout for Dennis's quarterly target companies.
Pulls latest openings from companies via public ATS APIs (Workday/Ashby/Greenhouse/
SmartRecruiters/Lever/Eightfold/RSS) and, for JS-rendered careers sites, a headless-browser
(playwright) adapter. Filters by location policy, scores fit against profile keywords,
tracks which job IDs we've already seen, writes a markdown report.
Location policy (default for most companies):
- Switzerland onsite/hybrid (CH)
- Denmark onsite/reloc
- CH-remote and Europe/EMEA/global remote (work-from CH or DK is realistic)
- NOT multi-country allowlists that omit CH/DK/Europe (e.g. "UK | Brazil | Ireland | ...")
Company overrides via args["_location_policy"]:
- "norway_only" — Norway only, Oslo included
- "norway_no_oslo" — Norway with a location outside the Oslo area (Equinor, Telenor, Nordic,
DNV, Arm, Microsoft Norway, NAV register; user preference 2026-10-21)
- "dk_no" — NATO-style: Denmark + Norway only (no CH — no NATO seat in CH)
Norway rows also get a language/citizenship check from the posting's requirement sentence
(only "Norwegian required" or "Norwegian citizenship required" filters a role; clearance is
noted, never a gate), and every row gets an employer-reputation verdict from
state/employer_reputation.json (Vizrt is excluded by user decision).
Usage:
py scout.py # Pull all configured companies (strong + medium only)
py scout.py --only=nvidia # Pull a single company by id
py scout.py --region=no # Only the Norway boards (or --region=ch for Switzerland +
# Europe-remote). Reports always list regions separately.
py scout.py --new-only # Report only jobs not seen before
py scout.py --include-weak # Include weak/noise bucket (default hidden)
py scout.py --hide-decided # Drop roles already in the decision log (undecided-only view)
py scout.py --decide "<url>" <status> [note...] # Record a decision and exit
# status is free-text: shortlist | skip | applied | paused | ...
State : state/seen_jobs.json · state/decisions.json · state/last_scrape.json ·
state/employer_reputation.json (curated, tracked) · state/nav_detail_cache.json
Output: reports/YYYY-MM-DD.md (+ .json dump of every eligible role)
To add a company: append to COMPANIES with one of the existing adapter types. A few sites
resist scraping even headless and stay in MANUAL_CHECK (surfaced as a report checklist).
See the adapter-coverage notes at the bottom for the current automated/manual split.
"""
import html
import json
import os
import re
import ssl
import sys
import tempfile
from contextlib import contextmanager
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from functools import lru_cache
import urllib.error
import urllib.parse
import urllib.request
from datetime import datetime, timezone
from pathlib import Path
ROOT = Path(__file__).parent
STATE_FILE = ROOT / "state" / "seen_jobs.json"
DECISIONS_FILE = ROOT / "state" / "decisions.json"
LAST_SCRAPE_FILE = ROOT / "state" / "last_scrape.json"
REPORTS_DIR = ROOT / "reports"
USER_AGENT = "Mozilla/5.0 (compatible; job-scout/0.1)"
API_FETCH_WORKERS = 8
CH_LOCATION_KEYWORDS = [
"switzerland", "zurich", "zürich", "basel", "bern", "geneva", "genf",
"lausanne", "zug", "rüschlikon", "stäfa", "schweiz", "suisse",
# Added 2026-09-11 (F12). These were being rejected outright when the adapter gave no
# country: Zimmerwald is the Kdo Cy site he applied to, Thun is where he lives.
"thun", "zimmerwald", "baar", "olten", "winterthur", "luzern", "lucerne",
"st. gallen", "sankt gallen", "aarau", "baden", "emmen", "duebendorf", "dübendorf",
"frauenfeld", "biel", "bienne", "solothurn", "chur", "sion", "neuchâtel", "neuchatel",
"fribourg", "freiburg", "lugano", "bellinzona", "kloten", "wallisellen", "opfikon",
"lupfig", "villigen", "rotkreuz", "allschwil", "kaiseraugst", "payerne",
]
DK_LOCATION_KEYWORDS = [
"denmark", "copenhagen", "danmark", "københavn", "koebenhavn",
# Added 2026-09-11 (F12): the two other Danish cities that actually hire here.
"aarhus", "århus", "odense", "aalborg", "ålborg", "lyngby", "roskilde",
]
NO_LOCATION_KEYWORDS = [
"norway", "oslo", "stavanger", "bergen", "trondheim", "norge",
]
# Oslo and its commuter belt. The user wants Norway roles in Trondheim, Bergen or similar
# and explicitly not in Oslo (2026-10-08, re-stated 2026-10-21), so the norway_no_oslo policy
# drops a role whose every Norwegian location is one of these. A role listing Oslo AND
# Trondheim stays in. Høvik (DNV HQ), Fornebu (Telenor HQ) and Kjeller (FFI) are Oslo area.
OSLO_AREA_PLACES = [
"oslo", "fornebu", "lysaker", "bærum", "baerum", "sandvika", "asker", "høvik", "hovik",
"stabekk", "skøyen", "skoyen", "nydalen", "lillestrøm", "lillestrom", "kjeller",
"skedsmo", "lørenskog", "lorenskog", "oppegård", "kolbotn", "nordre follo", "ski",
"jessheim", "ullensaker", "gardermoen", "billingstad", "kolsås", "kolsas", "rælingen",
]
US_ONLY_PATTERNS = [
"remote - us", "remote, us", "remote-us", "us remote", "us-remote",
"remote-friendly us", "remote (us)", "united states - remote",
"remote, united states",
]
# Pan-regional remote scope: work-from CH or DK is realistic.
EUROPE_SCOPE_KEYWORDS = [
"europe", "emea", "european", "eu-wide", "eu wide", "remote europe",
"europe remote", "remote - europe", "remote, europe",
]
GLOBAL_SCOPE_KEYWORDS = [
"global", "worldwide", "work from anywhere", "anywhere in the world",
]
POSITIVE_KEYWORDS = {
# German/Swiss job titles (added 2026-09-11, F14). Without these, every German-language
# posting scored near zero regardless of fit, which is why RUAG, SBB, BKW, PostFinance
# and Kdo Cy all needed a _score_floor to stay visible at all. The floor is a workaround;
# these terms are the actual fix. Weights mirror their English counterparts.
# Platform/DevOps vocabulary that was simply absent (F14). The Kdo Cy
# "ICT-System Engineer (DevSecOps / Platform Automation)" scored ZERO before this -
# a posting whose every requirement line matches his canonical evidence.
"devsecops": 3, "platform engineer": 3, "platform automation": 3, "platform": 1,
"automation": 1, "infrastructure as code": 2, "observability": 2,
"site reliability": 3, "system engineer": 1, "ict": 1,
"dateningenieur": 2, "data engineer (m/w/d)": 2, "systemingenieur": 2,
"software-entwickler": 2, "softwareentwickler": 2, "entwickler": 1,
"plattform": 2, "plattform-engineer": 3, "datenplattform": 3,
"automatisierung": 2, "betrieb": 1, "datenanalyse": 2, "datenmodellierung": 2,
"schnittstellen": 1, "datenqualitaet": 2, "datenqualität": 2,
"künstliche intelligenz": 3, "kuenstliche intelligenz": 3, "maschinelles lernen": 3,
"cloud-plattform": 2, "containerisierung": 2, "informatiker": 1,
# Norwegian titles (added 2026-10-22, same reason as the German block above): without them
# every Norwegian-titled NAV posting scored 0, so it sat in the hidden weak bucket and its
# language requirement was never checked - exactly the "norsk og/eller engelsk" ads the
# user wants surfaced. Compounds are listed explicitly: word-boundary matching does not
# find "utvikler" inside "plattformutvikler".
"utvikler": 1, "systemutvikler": 2, "programvareutvikler": 2, "backendutvikler": 2,
"plattformutvikler": 3, "plattformingeniør": 3, "dataingeniør": 2, "dataingenior": 2,
"dataplattform": 3, "skyplattform": 2, "maskinlæring": 3, "kunstig intelligens": 3,
"ki-ingeniør": 3, "automatisering": 2, "testautomatisering": 2, "datamodellering": 2,
"driftsingeniør": 1,
"genai": 3, "generative ai": 3, "llm": 3, "large language model": 3,
"applied ai": 3, "applied ml": 3, "ai engineer": 3, "ml engineer": 3,
"mlops": 3, "ai platform": 3, "ml platform": 3,
"python": 2, "java": 2, "data engineer": 2, "data engineering": 2,
# "data scientist" scored modestly (medium, not strong) — secondary to his data-eng/
# platform thesis, but the targeted band at boutiques like QuantCo (see target memory).
"data scientist": 2, "data science": 2,
"solutions architect": 2, "platform engineer": 2,
"ai infrastructure": 2, "inference": 2, "rag": 2, "agentic": 2,
# Spelled-out "Artificial Intelligence" (int'l orgs like BIS write it out instead of
# "AI") and analytics-engineer variants — was missing entirely, dropped a genuinely
# on-thesis BIS role (Senior Data & Analytics Engineer - AI) into the noise bucket.
"artificial intelligence": 3, "analytics engineer": 2, "data analytics": 2,
"kubernetes": 1, "docker": 1, "etl": 1, "pipeline": 1,
# Core CV lane — DevOps / data-platform / cloud (was scoring 0; surfaced only via "senior")
"data platform": 3, "platform engineering": 2, "devops": 2,
"sre": 2, "site reliability": 2, "cloud engineer": 2, "cloud": 1,
"software engineer": 1,
# Technical-architect pivot targets (cloud/data/platform = build on his stack; rank above
# bare "solutions architect" pre-sales). Generic "architect" catches the long tail.
"cloud architect": 3, "data architect": 3, "platform architect": 3,
"enterprise architect": 2, "architect": 1,
"crypto": 2, "blockchain": 2, "web3": 2, "solidity": 3,
# Trading / quant-finance — explicit user interest (energy/finance/crypto trading)
"trading": 2, "trader": 2, "quant": 2, "quantitative": 2,
"market data": 2, "low latency": 2, "low-latency": 2, "fix protocol": 2,
"brokerage": 2, "commodity": 1, "execution": 1,
# "solutions architect" (plural) already scored above; add singular + adjacent stack
"solution architect": 2, "c#": 1, ".net": 1,
# Forward-deployed / field / resident-architect lane — vendor "deploy-to-customer" roles
# where Dennis's multi-country ramp is the differentiator (see user_international_mobility).
# Travel-from-Bern fits his mobility appetite; no relocation required.
"forward deployed": 3, "forward-deployed": 3, "field engineer": 3,
"resident architect": 3, "resident engineer": 3, "resident solutions architect": 3,
"customer engineer": 2,
"senior": 1, "staff": 1, "lead": 1, "principal": 1,
}
NEGATIVE_KEYWORDS = {
"cuda": -3, "kernel driver": -3, "gpu programming": -3,
"compiler engineer": -3, "pytorch internals": -3, "jax internals": -3,
"rdma": -2, "infiniband": -2, "nccl": -3, "hpc cluster": -2,
"frontend": -3, "front-end": -3, "react native": -3,
"ios engineer": -3, "android engineer": -3, "mobile engineer": -3,
"ui engineer": -2, "ux engineer": -2,
"verilog": -3, "vhdl": -3, "asic": -3, "rtl design": -3,
"physical design": -3, "silicon": -2,
"expert c++": -2, "5+ years c++": -2, "deep c++": -2,
"intern": -5, "internship": -5, "graduate program": -3, "junior": -3,
}
# Title prefilter for high-volume boards (all-remote tech orgs + commodity traders that
# post mostly non-tech roles). Only keep titles containing one of these specific role
# phrases — kept tight so "Sales Engineer"/"Staff Accountant"/"Data Privacy Counsel"
# don't leak in. Matched as case-insensitive substrings against the title only.
# Inclusion allowlist. Applied ONLY to boards too large to score in full (>~200 roles:
# Databricks, Snowflake, Datadog, Elastic, Fivetran, Louis Dreyfus, Palantir). Everywhere else
# use NOISE_TITLE_EXCLUDE — see the design note there for why fail-open is the default.
ENG_TITLE_FILTER = [
"data engineer", "data engineering", "data platform", "platform engineer",
"data infrastructure", "data architect", "analytics engineer",
"mlops", "ml engineer", "ml platform", "machine learning engineer",
"site reliability", "sre", "backend engineer", "back-end engineer",
"devops engineer", "cloud engineer", "software engineer", "infrastructure engineer",
"kafka", "streaming", "big data", "quantitative developer", "quant developer",
# Forward-deployed / field / resident lane (vendor boards) — see user_international_mobility.
# "resident" alone catches Resident Solutions Architect/Engineer without opening the gate to
# all pre-sales SAs (the overscoring trap); "customer engineer" is Google's field-eng term.
"forward deployed", "forward-deployed", "field engineer", "resident", "customer engineer",
# Added 2026-08-18 after the CH boards were title-filtered: without these, real near-misses
# were dropped (BFH "Wissenschaftliche Mitarbeit Data-Driven Government", Novartis "Director
# & Group Head (AI-Systems & Scale)"). Deliberately NOT adding bare "ai" — it substring-matches
# Maintenance/Training/Chair and floods every board.
"artificial intelligence", "ai-systems", "ai platform", "ai engineer", "ai architect",
"data scientist", "data-driven", "data science",
]
# Generic non-tech title exclusions, shared by every board that uses exclusion-mode filtering.
#
# DESIGN NOTE (2026-08-18). Boards are filtered one of two ways:
# * inclusion (`_title_filter`) — an allowlist, used ONLY on boards too large to score in
# full (Databricks ~800, OpenAI ~730, Palantir ~300). It fails CLOSED: a great-fit role
# with a title nobody anticipated is dropped at fetch time and never reaches the scorer,
# so it appears in no report and in no JSON dump. That is an acceptable trade only where
# volume forces it.
# * exclusion (`_title_exclude`) — this list. It fails OPEN: everything survives unless it
# is *clearly* not an engineering role, and the score + the weak/noise bucket decide what
# surfaces. Prefer this. The final call belongs to the reviewer, not to a keyword gate.
#
# Keep these terms unambiguous. Anything that could plausibly attach to a technical role
# (analyst, manager, specialist, consultant, architect, lead) must NOT go in here.
NOISE_TITLE_EXCLUDE = [
# Facilities / retail / hospitality / admin
"hausdienst", "reinigung", "empfang", "hauswart", "chef de partie", "koch", "küche",
"restaurant", "catering", "fahrer", "logistik mitarbeiter", "lagerist", "verkauf",
"verkäufer", "retail", "barista", "security guard", "sicherheitsdienst",
# Care / health / teaching-of-non-tech
"physiotherapie", "pflege", "pflegefach", "hebamme", "ergotherapie", "psychologie",
"medical representative", "nurse", "arzt", "ärztin", "dentist",
# Arts / music / sport
"violine", "klavier", "musik", "dozierende*r violine", "sport",
# Back-office
"fundraising", "buchhaltung", "accountant", "accounting", "payroll", "steuer",
"recruiter", "talent acquisition", "human resources", "personalwesen",
"legal counsel", "rechtsanwalt", "rechtsreferendar", "notar", "jurist",
"kommunikation", "public relations", "übersetzer", "translator",
# Early-career / non-role listings
"lehrstelle", "praktikum", "praktikant", "internship", "intern ", "trainee",
"apprenti", "ausbildung", "schnupper", "where we're hiring", "talent community",
# Non-software engineering trades
"elektroplaner", "elektroinstallat", "sanitär", "heizung", "maler", "schreiner",
"hochspannung", "wasserbau", "strassenbau", "holzbau", "bauingenieur", "bauleiter",
]
# id, display, adapter, adapter_args
COMPANIES = [
("nvidia", "NVIDIA", "workday", {
"host": "nvidia.wd5.myworkdayjobs.com",
"tenant": "nvidia",
"site": "NVIDIAExternalCareerSite",
"search_text": "Switzerland",
}),
("kraken", "Kraken", "ashby", {"slug": "kraken.com"}),
("openai", "OpenAI", "ashby", {"slug": "openai"}),
("anthropic", "Anthropic", "greenhouse", {"board": "anthropic"}),
("novartis", "Novartis", "workday", {
"host": "novartis.wd3.myworkdayjobs.com",
"tenant": "novartis",
"site": "Novartis_Careers",
"search_text": "Switzerland",
"_title_exclude": NOISE_TITLE_EXCLUDE,
}),
# PCSX (Eightfold) — Microsoft has a public position search endpoint
("microsoft", "Microsoft", "pcsx", {
"domain": "microsoft.com",
"location": "Switzerland",
}),
# --- Data-infra US tech (his exact stack; mostly all-remote — title-filtered to eng/data) ---
# Dropped: ClickHouse (Glassdoor 3.3, 36% recommend, toxic-culture flag — 2026-05).
# Dropped: HashiCorp — acquired by IBM (closed 2025); greenhouse/ashby/lever boards all 404,
# roles folded into IBM's careers (no clean public ATS API). 2026-06-06.
# Confluent: acquired by IBM (closed 2026-03-17). The Ashby board went empty (17 jobs on
# 2026-10-08, 0 on 2026-10-14); roles are now IBM requisitions. Moved to the `ibm`
# adapter 2026-10-15. Same id, so seen-state and decisions stay keyed to "confluent".
("confluent", "Confluent (IBM)", "ibm", {"query": "confluent", "_title_exclude": NOISE_TITLE_EXCLUDE}),
("gitlab", "GitLab", "greenhouse", {"board": "gitlab", "_title_exclude": NOISE_TITLE_EXCLUDE}),
("grafana", "Grafana Labs","greenhouse",{"board": "grafanalabs", "_title_exclude": NOISE_TITLE_EXCLUDE}),
# Added 2026-06-06 (Tier A/B data-infra). Databricks/Snowflake/Datadog have Zürich offices
# (Swiss-scale comp, clears bar); Elastic/Fivetran are remote-EU (verify CH-equiv comp —
# may be geo-banded below 180k, like Grafana). All title-filtered (boards are 160-760 roles).
("databricks","Databricks","greenhouse", {"board": "databricks", "_title_filter": ENG_TITLE_FILTER}), # Zürich SWE + remote-EU; DK via default location policy
("snowflake", "Snowflake", "ashby", {"slug": "snowflake", "_title_filter": ENG_TITLE_FILTER}), # Zürich "Observe" observability SWE roles
("datadog", "Datadog", "greenhouse", {"board": "datadog", "_title_filter": ENG_TITLE_FILTER}), # Zürich branch + remote-EU
("elastic", "Elastic", "greenhouse", {"board": "elastic", "_title_filter": ENG_TITLE_FILTER}), # remote-first; ELK = his stack
# dbt Labs -> Fivetran: the "dbtlabsinc" board 404'd from 2026-08-18. getdbt.com/careers now
# redirects to a page serving Fivetran's Greenhouse listings (gh_jid links) — the two merged, so
# the dbt roles live on board "fivetran". EMEA presence is Dublin/London/Serbia, no CH office, so
# most roles fall out on the location policy; kept for remote-EU analytics-eng reqs. 2026-08-18.
("fivetran", "Fivetran (ex-dbt Labs)", "greenhouse", {"board": "fivetran", "_title_filter": ENG_TITLE_FILTER}),
# --- Energy / commodity trading (SmartRecruiters; title-filtered to tech roles) ---
# Dropped: Vitol (Glassdoor 3.5, 55% recommend, grueling-hours/toxic flag — 2026-05).
# Dropped: Sygnum (Glassdoor 3.4, 51% recommend, comp 2.3/5 — below 180k bar — 2026-05).
# Axpo (Baden/Zurich) — Teamtailor on a custom domain; no <slug>.teamtailor.com host
# resolves, so this uses base_url. Added 2026-08-18: memory names Axpo/Alpiq as the
# energy-trading targets but only MET was ever configured, and MET+LDC have produced
# 0 CH-eligible roles across 1,464 scrapes. ~400 roles, German-speaking region (which is
# what rules out the Geneva traders), incl. a "Forward Deployed AI Engineer" at add time.
("axpo", "Axpo", "teamtailor", {
"base_url": "https://careers.axpo.com",
"default_location": "Switzerland",
"_title_exclude": NOISE_TITLE_EXCLUDE,
}),
# CERN (Geneva) — probed 2026-08-18: SmartRecruiters board "CERN", 66 live postings, all
# Geneva. Real engineering exists (Deep Learning Developer, ML Engineer for Next Generation
# Triggers, Full-stack SWE, Data Storage R&D Engineer, Software Engineer) and it is an
# English-working international organisation in CH — the BIS Basel lane.
# TWO STANDING CAVEATS, do not forget them when scoring a CERN role:
# 1. Geneva is ~1h50 each way from Bern — not viable for a 2-3 day hybrid without
# relocation, which the user has ruled out. Practical constraints will score low.
# 2. CERN pay grades sit below the CHF 180k bar.
# Added anyway because the adapter is free (already implemented) and a fully-remote or
# exceptional CERN req would otherwise never surface.
("cern", "CERN (Geneva)", "smartrecruiters", {
"company": "CERN",
"_title_exclude": NOISE_TITLE_EXCLUDE,
}),
("metgroup", "MET Group", "smartrecruiters", {"company": "METGroup", "_title_exclude": NOISE_TITLE_EXCLUDE}),
# Dropped: Louis Dreyfus — 2026-08-18. 300 roles, every one Brazil/India/Bulgaria/Asia
# operations (its Data Modeler reqs are in Bangalore). Geneva HQ but zero Swiss
# postings in any run. MET Group is retained by contrast: once exclusion-mode
# filtering replaced the allowlist, MET showed real Baar/Zug roles (Wholesale
# Specialist, Lead Trade Process Analyst, LNG Financial Trader) — its earlier
# "0 eligible across 1,464 scrapes" was a filter artifact, not an empty board.
# No data/platform eng reqs there yet, but it is the live CH commodity-trading lane.
# Equinor (Workday) — Norway energy major; lived/worked in NO before. Outlier location
# policy: Norway only (not CH / Europe-remote). Small board (~15); no title filter.
("equinor", "Equinor", "workday", {
"host": "equinor.wd3.myworkdayjobs.com",
"tenant": "equinor",
"site": "EQNR",
"_location_policy": "norway_no_oslo",
"nav_aliases": ["Equinor"],
# Norway-only filter already scopes the board; titles are often generic/energy
# ops and score 0 under the English eng keyword list — keep them visible.
"_score_floor": 2,
}),
# Telenor Norge (Teamtailor) — Nordic telco; Swisscom is the direct domain analogue
# and Dennis worked in Norway before (Vizrt, Bergen). NOTE: the old Workday tenant
# telenorgroup.wd3.myworkdayjobs.com is dead (HTTP 422 on /wday/cxs, 2026-07-29) —
# Telenor migrated to Teamtailor. telenornorway is the real Norway board (redirects
# to careers.telenor.no).
# TRAP — slug telenorgroup is NOT a "near-empty group-HQ board" as this comment used
# to claim (corrected 2026-08-27): it is a claimed board titled "Sandbox: Telenor
# Shared Services" serving 18 phantom listings whose newest is 2024-11-11, including
# plausible-looking "Lead Software Engineer" / "Frontend Developer - Oslo" roles. None
# of its titles hit the DEMO_TITLES fingerprint in fetch_teamtailor(), so that guard
# would NOT catch it — do not widen DEMO_TITLES to cover this. That guard is for
# *unclaimed* slugs served Teamtailor's demo seed set; this is a claimed board serving
# years-stale data, a different failure mode (a newest-item age threshold is the fix,
# if one is ever wanted). Never point this entry at telenorgroup.
# Sibling Telenor boards probed 2026-08-27, all rejected, do not re-probe:
# telenorcyberdefence (3 roles, newest 2025-10-06 — dormant), telenordenmark (4 roles:
# retail sales, praktikant, legal student — zero engineering, and it would run under
# the default CH/DK policy rather than norway_only, so it is pure noise),
# telenoramp / telenormaritime / telenorsatellite / telenorsverige / telenorkystradio /
# telenorbusiness (all HTTP 404 — no such board).
# THROUGHPUT (measured 2026-08-27): the live board carries 4-5 roles at a time and is
# mostly Norwegian-language ops/security. Exactly one relevant role has surfaced —
# Platform Engineer, Core Automation (job 7829423, posted 2026-06-01, score 5), first
# reported 2026-08-18 and HTTP 410 Gone by 2026-08-27. The feed did NOT silently drop
# it; it genuinely closed. Expect roughly one tailorable role per quarter here.
# CAVEAT: most Telenor Norge postings require Norwegian ("norsk og engelsk") and are
# Oslo/Trondheim onsite-hybrid — verify both per role before tailoring.
# Norwegian-language titles/descriptions score ~0 under the English eng keyword
# list, so keep a score floor (same pattern as Equinor).
("telenor", "Telenor Norge", "teamtailor", {
"slug": "telenornorway",
"default_location": "Oslo/Trondheim, Norway",
"_location_policy": "norway_no_oslo",
"nav_aliases": ["Telenor"],
"_score_floor": 2,
}),
# NATO civilian vacancies (Oracle Taleo). Public marketing page is
# https://www.nato.int/en/work-with-us/careers/vacancies — jobs actually live on
# nato.taleo.net career section 2. Location policy dk_no: Denmark + Norway only
# (no NATO seat in CH). Server-side location taxonomy IDs filter the board before
# parse; dk_no is a second safety net on the location string.
# Taleo option values (jobsearch.ftl dropdown): Norway=14470035151, Denmark=16270035151.
("nato", "NATO", "taleo", {
"host": "nato.taleo.net",
"career_section": "2",
"location_ids": ["14470035151", "16270035151"], # Norway, Denmark
"_location_policy": "dk_no",
# Staff Officer / NATO-body titles score unevenly under eng keywords — keep
# pre-filtered NO/DK roles visible (same pattern as Equinor/SBB).
"_score_floor": 2,
"nav_aliases": ["NATO"],
}),
# --- Norway outside Oslo (added 2026-10-22, user request) -----------------------------
# The user wants a move to Trondheim, Bergen or similar, not Oslo. NAV covers most
# Norwegian employers; these boards are where NAV was measured thin or empty
# (2026-10-22: Nordic 0, DNV 2, Equinor 2 vs ~10, NATO 0). Every Norway row gets the
# language/citizenship check and a reputation verdict in the report.
# Nordic Semiconductor (Trondheim HQ) — Teamtailor on a custom domain; 29 roles worldwide
# on 2026-10-22, the schema.org address gives the real city.
("nordic", "Nordic Semiconductor", "teamtailor", {
"base_url": "https://careers.nordicsemi.com",
"default_location": "",
"_location_policy": "norway_no_oslo",
"nav_aliases": ["Nordic Semiconductor"],
}),
# DNV — Algolia-backed jobs.dnv.com (credentials read from the page each run). HQ Høvik is
# Oslo area and filtered; Trondheim/Bergen/Stavanger offices stay.
("dnv", "DNV", "algolia_jobs", {
"page_url": "https://jobs.dnv.com/job-search",
"_location_policy": "norway_no_oslo",
"nav_aliases": ["DNV"],
}),
# Arm (Trondheim, ex-Falanx: Mali GPU drivers, compilers, hardware) — Radancy front end
# over iCIMS. See fetch_radancy for the multi-location limitation.
("arm", "Arm", "radancy", {
"base_url": "https://careers.arm.com",
"keywords": ["Trondheim", "Norway"],
"_location_policy": "norway_no_oslo",
"nav_aliases": ["Arm Norway", "ARM Norway"],
}),
# Microsoft Development Center Norway (Oslo, Trondheim, Tromsø) — same PCSX API as the
# Swiss entry. NOTE: a cold application here stacks on the pending Principal FDE
# (200043897) until that resolves.
("microsoft_no", "Microsoft (Norway)", "pcsx", {
"domain": "microsoft.com",
"location": "Norway",
"_location_policy": "norway_no_oslo",
"nav_aliases": ["Microsoft"],
}),
# NAV national job register — the wide net for everything else (Kongsberg, Forsvaret,
# Nammo, SINTEF, Kitron, Andøya, the long tail). Keyword queries on a request budget;
# see fetch_nav for the rate-limit behaviour. Rows carry the real employer name.
("nav", "NAV register (Norway)", "nav", {
# Ordered by expected yield: only the first `search_budget` run. Measured 2026-10-22:
# 9 searches at 3s spacing triggered a 429 that lasted hours, so stay at 8 and pace 4s.
"queries": ["data engineer", "devops", "software engineer", "systemutvikler",
"python", "platform engineer", "dataingeniør", "utvikler",
"machine learning", "test automation", "site reliability", "backend"],
"search_budget": 8,
"pages_per_query": 1,
"pace_s": 4.0,
# Text checks go to the official feed host (no 429 seen at 0.5s), so the budget can
# be generous; results are cached, so later runs only fetch new postings.
"detail_budget": 60,
"feed_pace_s": 0.5,
# Website-page fallback for ads the public feed lacks (10 of 15 on 2026-10-22).
# Same host as search, so few and slow; skipped entirely after a search 429.
"page_fallback_budget": 6,
"page_pace_s": 8.0,
"_location_policy": "norway_no_oslo",
}),
# International org — BIS (Basel), commutable from Bern, salary net of Swiss tax.
# Low-volume RSS feed; no title filter (Innovation Hub roles can be oddly titled).
# BIS moved from the bis.org RSS feed to its Workday tenant on 2026-10-22: bis.org went
# behind a Cloudflare challenge that answers 403 even to a headless browser, while
# bis.wd3.myworkdayjobs.com/External serves the same vacancies WITH their office
# ("BIS Headquarters Basel" vs "BIS Representative Office for Asia and the Pacific"), so
# the per-vacancy location lookup the RSS adapter needed is no longer necessary.
# Decision keys stay on bis.org#jr…: see _HOST_ALIASES and the bis.org patterns.
("bis", "BIS (Basel)","workday", {
"host": "bis.wd3.myworkdayjobs.com",
"tenant": "bis",
"site": "External",
"nav_aliases": ["BIS"],
}),
# Dropped: Coinbase Ventures web3 talent network (Getro collection 1625) — 2026-08-18.
# Never carried Coinbase's own roles, only portfolio companies. Across ~3 months
# it surfaced 3 distinct CH-eligible roles, ALL from Ashby (recruiting software,
# ruled out 2026-07-28) and all decided "skip". With Ashby excluded the board has
# returned 0 jobs on every run since, reported as a false error each time.
# fetch_getro is retained for a future VC talent network worth tracking.
# Bitcoin Suisse (Zug) uses the onlyfy.jobs ATS. No title filter — small crypto
# firm, only a handful of CH roles; let scoring rank them (CH filter does the rest).
("bitcoin_suisse", "Bitcoin Suisse", "onlyfy", {"slug": "bitcoin-suisse"}),
# Headless-browser scrapers — slower (3-15s per company) but covers JS-rendered sites.
# Google actively bot-detects; the STEALTH_JS init script (applied to every context)
# is what makes its job list render. Cards are <li> with a "Learn more about <title>"
# aria-label link; location lives in the card text (captured via blob mode).
("google", "Google", "playwright", {
"url": "https://www.google.com/about/careers/applications/jobs/results/?location=Switzerland",
"wait_for": "a[href*='jobs/results/'][aria-label*='Learn more']",
"card": "li:has(a[aria-label*='Learn more about'])",
"title_sel": "a[aria-label*='Learn more about']",
"title_sel_attr": "aria-label",
"title_strip_prefix": "Learn more about ",
"link_sel": "a[href*='jobs/results/']",
"link_attr": "href",
"url_prefix": "https://www.google.com/about/careers/applications/",
"default_location": "",
"scroll_count": 5,
"use_inner_text_as_blob": True,
"cookie_accept": ["button:has-text('Accept all')", "button:has-text('Reject all')"],
# Results are paged 20 at a time and "page" is 1-indexed — without this only the
# first 20 of ~48 Zürich roles were ever scraped.
"page_param": "page",
"page_param_start": 2,
"max_pages": 6,
}),
# Amazon + AWS share one board. Added 2026-08-18: AWS is the cloud the evidence base
# actually supports (Swisscom migration + data products), whereas claims.json marks GCP
# output-forbidden — so an AWS-native Zurich FDE req is the Google FDE lane without the
# cloud gap. 32 CH roles at time of adding, incl. "Senior Forward Deployed Engineer, AWS
# Forward Deployed Engineering" (Zurich).
# AWS names its delivery/field org differently from every other board, so the shared
# filter caught only 1 of 32 CH roles. These four terms are added narrowly and ONLY here:
# "solutions architect" is the classic overscoring trap (see job_scout_overscoring_findings),
# but on the AWS board it is the delivery-side title attached to migration/data work, which
# is exactly SW-1. Read the JD before trusting the score on any of these.
("amazon", "Amazon / AWS", "amazon", {
"countries": ["CHE"],
"_title_exclude": NOISE_TITLE_EXCLUDE,
}),
("apple", "Apple", "playwright", {
# Location code is CHEC (postLocation-CHEC, from /api/v1/refData/postlocation). The old
# "switzerland-CHE" slug stopped resolving: Apple silently applied a US location
# ("Chesterfield, Missouri") instead, so every run from at least 2026-08-31 to
# 2026-10-22 returned the same 21 US/intern cards and 0 Swiss roles. Fixed 2026-10-22;
# the CHEC filter shows 25 Swiss roles, Zurich engineering included. If this board
# ever returns only non-Swiss cards again, re-check the code via that endpoint.
"url": "https://jobs.apple.com/en-us/search?location=switzerland-CHEC",
"wait_for": "a[href*='/en-us/details/']",
"wait_timeout_ms": 45000,
"settle_ms": 6000,
"element_timeout_ms": 10000,
# 20 results per page, 1-indexed (25 Swiss roles on 2026-10-22 = 2 pages). Each Apple
# page is slow (an empty last page costs the full wait), so cap at 2 extra pages:
# up to 60 roles. Raise it if the Swiss board ever grows past that.
"page_param": "page",
"page_param_start": 2,
"max_pages": 2,
# The card is the result list item, not the title link: the link alone carries no
# location, so 26 of 28 Swiss rows read location-less and were dropped (2026-10-22).
# The item text reads "<title> <team> <date> Location Zurich Actions".
"card": "li.rc-accordion-item",
"title_sel": "a[href*='/en-us/details/']",
"link_sel": "a[href*='/en-us/details/']",
"url_prefix": "https://jobs.apple.com",
# NO default_location. Apple's own postLocation-CHE filter leaks global reqs whose
# postLocationId is postLocation-USA ("Various Locations within United States"), and
# a forced "Switzerland" default relabelled every one of them as CH-eligible — 84
# phantom CH rows over 4 runs, all internships/retail, which then read as "Apple is a
# bad fit" in the decision log. Read the real location off the card instead; when
# Apple has no Swiss reqs the honest answer is 0. Verified 2026-08-18.
"use_inner_text_as_blob": True,
"scroll_count": 5,
}),
# Meta job links are /profile/job_details/<id>; title + location are in the link text.
# NOT broken despite returning ~1 role: verified 2026-08-18 that metacareers itself
# reports "1 Items" for the Zurich office filter. The board is genuinely near-empty;
# don't "fix" this scraper without first checking the live item count on the page.
("meta", "Meta", "playwright", {
"url": "https://www.metacareers.com/jobs?offices[0]=Zurich%2C%20Switzerland",
"wait_for": "a[href*='/profile/job_details/']",
"card": "a[href*='/profile/job_details/']",
"title_attr": "text",
"link_attr": "href",
"url_prefix": "https://www.metacareers.com",
"default_location": "Zurich, Switzerland",
"scroll_count": 5,
"use_inner_text_as_blob": True,
}),
# Roche: was a playwright scrape of the search-results page until 2026-08-18. That page's
# ?locationsearch=Switzerland filter silently fails — the page state reports "no-results"
# and the scrape harvested *recommendation-widget* cards instead, returning Shanghai,
# Kyiv, Bogota and Mannheim and zero Swiss roles for months while looking healthy. The
# underlying Phenom refineSearch endpoint filters correctly (134 CH roles), so hit it
# directly. Title-filtered: Roche CH is overwhelmingly lab/pharma/apprenticeship reqs.
("roche", "Roche", "phenom", {
"url": "https://careers.roche.com/widgets",
"ref_num": "ROCHGLOBAL",
"page_id": "page11-ds",
"country_facet": "Switzerland",
"_title_exclude": NOISE_TITLE_EXCLUDE,
}),
# Cisco (PhenomPeople, new careers.cisco.com domain). Keyword search surfaces CH roles.
("cisco", "Cisco", "playwright", {
"url": "https://careers.cisco.com/global/en/search-results?keywords=Switzerland",
"wait_for": "a[href*='/job/'], div[role='listitem']",
"card": "div[role='listitem']:has(a[href*='/job/'])",
"title_sel": "a[href*='/job/']",
"link_sel": "a[href*='/job/']",
"link_attr": "href",
"url_prefix": "https://careers.cisco.com",
"default_location": "Switzerland",
"cookie_accept": ["#onetrust-accept-btn-handler"],
"scroll_count": 5,
"use_inner_text_as_blob": True,
}),
# --- Zürich/Zug high-comp additions (2026-05-31 list review) ---
# Palantir (Lever). Verified: 221 postings on the public board. It's US/London-heavy, so
# Swiss/Schwyz roles are rare but self-surface when posted (the location filter drops the
# US/London bulk). No title filter: his target titles (Forward Deployed Software Engineer,
# Deployment Strategist) aren't in ENG_TITLE_FILTER, so filtering would hide them.
("palantir", "Palantir", "lever", {"slug": "palantir"}),
# QuantCo (Lever — note the trailing-hyphen slug "quantco-"). ~16 roles, most tagged
# "Europe" (hybrid); QuantCo's continental hub is Zürich, so Europe-scope in
# location_eligibility surfaces them. No title filter: target band is DS/Quant/AI/Cloud
# (ENG_TITLE_FILTER would drop some); interns/frontend caught by NEGATIVE_KEYWORDS.
("quantco", "QuantCo", "lever", {"slug": "quantco-"}),
# --- Bern/Thun local tier — WLB & proximity exception (comp bar relaxed; 2026-06-01) ---
# Wired after live endpoint discovery. ⚠️ German citizen: RUAG classified work may require
# Swiss citizenship — verify per-role before tailoring (see project_target_companies).
# Swissgrid (Aarau): Magnolia CMS JSON endpoint (verified). placeOfWork is a bare city
# (Aarau/Prilly/...), so loc_suffix tags it Switzerland for the CH filter. No title filter
# (small board ~13 roles; lets Data Scientist / Applied-ML roles surface).
# Swiss federal job portal (jobs.admin.ch) via the Prospective API, filtered to
# Kommando Cyber. Added 2026-09-10 on user request: the Kdo Cy DevOps Engineer III req
# was found and applied to MANUALLY (rejected 2026-09-10) because nothing here was
# automated, and 15 Kdo Cy roles were open at the time this was wired up.
#
# `units` matches substrings against the sub-unit name. Other VBS units in the same
# defence-tech lane, one line away if wanted (counts as of 2026-09-10):
# "armasuisse" (5) · "Bundesamt für Cybersicherheit" (1) · "Nachrichtendienst" (10)
# "Kommando Ausbildung" (19) · "Kommando Operationen" (50) · "Armeestab" (6)
# Drop `units` entirely to scan all 463 federal roles and let the scorer filter.
#
# Pay: rows carry a `lohnklasse`. The official EPA table (maxima, 2026) is recorded in
# memory `reference_bund_lohnklassen` — LK 18 120'923 · 22 142'832 · 24 158'954 · 26 175'617,
# with BPV Anhang 1 splitting Standardplan (LK 1-23) from Kaderplan (LK 24-38). It is
# CALIBRATION ONLY: the user ruled 2026-09-11 that Lohnklasse must NOT be a screening gate,
# since Ortszuschlag, Familienzulagen and benefits make LK 24 and below workable.
# _score_floor: Kdo Cy titles are German administrative ("ICT-System Engineer",
# "ICT-Architekt/-in III") and score unevenly under English keywords — same reason
# NATO and SBB carry a floor. Keep pre-filtered roles from this unit visible.
("kdocy", "Schweizer Armee - Kdo Cy", "prospective", {
"medium_id": "1000624",
"units": ["Kommando Cyber"],
"lang": "de",
"_score_floor": 2,
# The floor lifts every German-titled row to 2, which would also float the
# apprenticeship intake. These terms are unambiguous on this board.
"_title_exclude": ["lernende", "lehrgang", "praktikant", "praktikum", "ausbildung zum",
"ausbildung zur", "sportlehre", "musiklehre"],
}),
("swissgrid", "Swissgrid (Aarau)", "json", {
"url": "https://www.swissgrid.ch/.rest/cloud/component-data?path=%2Fswissgrid%2Fen%2Fhome%2Fcareer%2Fjobs%2Fmain%2Fjoblist_transferred_11",
"jobs_key": "jobs",
"field_title": "title", "field_location": "placeOfWork",
"field_url": "descriptionUrl", "field_date": "onlineSince",
"loc_suffix": " Switzerland",
"desc_keys": ["department", "typeOfEmployment", "entryLevel"],
"_title_exclude": NOISE_TITLE_EXCLUDE,
}),
# RUAG (Thun/Bern/Emmen). Jobs render on the portal as anchors to jobs.ruag.ch; the first
# line of each anchor is the title. All sites are Swiss, so default_location=Switzerland
# passes the CH filter. ENG_TITLE_FILTER cuts the apprenticeship/Lehrstelle bulk.
# Drupal portal: 20 jobs/page, server-rendered, paginated via ?page=N (0-indexed). The
# first page is apprenticeship-heavy; eng roles (DevOps/Data/Cloud) are on later pages,
# so we page through until a page adds nothing new (~5-6 pages).
("ruag", "RUAG (Thun/Bern)", "playwright", {
"url": "https://www.ruag.ch/en/working-us/job-portal",
"wait_for": "a[href*='/offene-stellen/']",
"card": "a[href*='/offene-stellen/']",
"title_attr": "text",
"link_attr": "href",
"default_location": "Switzerland",
"scroll_count": 1,
"page_param": "page",
"max_pages": 10,
"_title_exclude": NOISE_TITLE_EXCLUDE,
}),
# SBB (company.sbb.ch — the correct host; company-jobs.sbb.ch was wrong). AEM job filter
# served as a flat JSON list; the fetch_sbb adapter replicates the user's IT + Bern-region
# filter. German/generic titles, so _score_floor keeps the pre-filtered results visible.
# ⚠️ DE-citizen limits may apply to some SBB security/critical-infra roles.
("sbb", "SBB", "sbb", {
"topic": "IT / Telekommunikation",
"region": "Bern Mittelland",
"_score_floor": 2,
"_title_exclude": NOISE_TITLE_EXCLUDE,
}),
# BKW Group (jobs.bkw.com — the real ATS host). PMS structured-data API; ~600 roles
# group-wide, so fetch_bkw keeps only Berufsfeld categories Informatik/Trading/Finanzen
# (IT/data + energy-trading, incl. the flagged Energiehandel roles). German/generic
# titles, so _score_floor keeps the pre-filtered set visible.
("bkw", "BKW (Bern)", "bkw", {"_score_floor": 2,
"_title_exclude": NOISE_TITLE_EXCLUDE}),
# PostFinance (Bern). The careers site renders a small, client-side paginated board;
# scrape all pages through its stable next-page control. No title filter: the board is
# low-volume, and the scorer keeps unrelated banking/customer-service roles out of the
# default report while retaining locally relevant engineering variants.
("postfinance", "PostFinance (Bern)", "playwright", {
"url": "https://jobs.postfinance.ch/PostFinance/search?locale=de_DE",
"wait_for": "a[href*='/PostFinance/job/']",
"card": "a[href*='/PostFinance/job/']",
"title_attr": "text",
"link_attr": "href",
"default_location": "Switzerland",
"use_inner_text_as_blob": True,
"next_button": "#pfch-pagination-next",
"max_pages": 10,
"_title_exclude": NOISE_TITLE_EXCLUDE,
}),
# BFH (Bern University of Applied Sciences). Re-added 2026-07-14: the jobs.bfh.ch domain
# itself is a broken/stub SPA shell (renders "Career Center project template", nothing
# else — this is what looked like a 403 previously), but the actual listing widget is
# embedded and fully renders on the www.bfh.ch careers page. Card = one <div> per posting
# with 3 li.teaser-text-list-item fields (department, location, employment type); location
# is the 2nd. Low-volume (12 open reqs), single page. German titles score 0 under the
# English-keyword scorer (same problem as SBB/BKW) — _score_floor keeps them visible
# instead of hidden as "noise", e.g. "Professur Data-Driven Government",
# "Co-Leitung ... digitale Souveränität".
("bfh", "BFH (Bern)", "playwright", {
"url": "https://www.bfh.ch/en/about-bfh/careers/jobs/",
"wait_for": ".teas-prospective-job-teaser",
"card": ".teas-prospective-job-teaser",
"title_sel": "h2.teaser-title",
"location_sel": "li.teaser-text-list-item:nth-of-type(2)",
"link_sel": "a",
"default_location": "Switzerland",
"_score_floor": 2,
"_title_exclude": NOISE_TITLE_EXCLUDE,
}),
]
# Companies that resist scraping — clickable checklist in the report.
# (BFH re-automated 2026-07-14 on www.bfh.ch. Dialectic dropped — crypto covered elsewhere.)
MANUAL_CHECK = [
# Oracle (Tier C, requested 2026-06-06). Oracle Recruiting Cloud (ORC) resists clean
# scraping: careers.oracle.com renders job tiles that are NOT anchors, so the playwright
# selector pattern fails. The ORC REST endpoint works and returns data, but reliable CH
# filtering needs the Switzerland geography node id, which the facet-expand call 400s on.
# Wireable later as a `json` adapter once that geographyId is resolved. For now, manual:
# https://eeho.fa.us2.oraclecloud.com/hcmRestApi/resources/latest/recruitingCEJobRequisitions
# ?onlyData=true&expand=requisitionList.workLocation
# &finder=findReqs;siteNumber=CX_45001,limit=200,sortBy=POSTING_DATES_DESC
# (then client-filter requisitionList[].PrimaryLocation for Switzerland/Zürich)
("Oracle", "ORC SPA resists scraping; REST endpoint known but needs CH geographyId (see code comment). Check Switzerland tech roles manually.",
"https://careers.oracle.com/en/sites/jobsearch/jobs?location=Switzerland"),
# NATO NCIA (Brussels/The Hague/Mons) — probed 2026-08-18. NCIA is NATO's technology arm and
# a SEPARATE pipeline from the `nato` adapter above, which only returns JWC Stavanger reqs
# (Taleo). nato.int sits behind Cloudflare and returns "Sorry, you have been blocked" to a
# headless browser, so there is no automated path. Manual only.
("NATO NCIA", "Cloudflare-blocked to headless browsers. NCIA is the tech arm and is separate from the JWC Stavanger reqs the `nato` adapter returns — check it manually for AI/data roles.",
"https://www.ncia.nato.int/careers.html"),
# Bundeswehr Reserve (Reserveoffizier ausserhalb des Wehrdienstes). Probed 2026-09-10.
# The portal is SAP E-Recruiting (sap-client 300); the data sits behind an OData service
# at /erece/unreg/ (entity set `Stellensuche_Set`, `SearchCategory eq '0022'` = Reserve,
# ~79 postings). It is NOT wired up, deliberately:
# * the portal actively blocks automation — a headless browser gets an HTTP 404
# "Bundeswehr Fehlerseite" on every path, and filtered OData GETs are 404'd by the WAF;
# the only way through was spoofing the UA and masking navigator.webdriver.
# * that is an access control the operator deliberately set. Routinely circumventing it
# from a scheduled scraper is a different posture from jobs.admin.ch, which serves an
# open, unauthenticated API with no countermeasures. Not a call to make silently.
# BETTER CHANNEL, operator-sanctioned: the portal has a Suchauftrag (job alert) —
# `AttSuchauftragSet` in the OData metadata. It e-mails new matching Reserve postings.
# Zero maintenance, no evasion. Set it up once instead of scraping.
# NOTE on Laufbahn: most IT-titled reserve postings are Feldwebel (NCO) roles. As a
# Leutnant d.R. only the Offizier ones apply — filter on that before reading titles.
("Bundeswehr Reserve", "SAP E-Recruiting behind bot detection; NOT scraped by choice (see code comment). Use the portal's own Suchauftrag/job alert. Filter: Art der Stelle = Stellenangebote Reserve. Watch for Offizier (not Feldwebel) IT roles.",
"https://bewerbung.bundeswehr-karriere.de/erece/portal/index.html"),
]
@lru_cache(maxsize=1)
def _ssl_context():
"""Prefer certifi's CA bundle over the OS trust store: on Windows, urllib's default
context pulls trust anchors from the system cert store, which has been observed to
reject valid Workday-hosted certs as expired (stale cached root) even though the
live chain verifies fine elsewhere. Falls back to the default context if certifi
isn't installed (pip install -r requirements.txt covers it)."""
try:
import certifi
return ssl.create_default_context(cafile=certifi.where())
except ImportError:
return ssl.create_default_context()
def http_get_json(url, headers=None, data=None, method="GET"):
headers = headers or {}
headers.setdefault("User-Agent", USER_AGENT)
headers.setdefault("Accept", "application/json")
if data is not None and isinstance(data, dict):
data = json.dumps(data).encode("utf-8")
headers.setdefault("Content-Type", "application/json")
req = urllib.request.Request(url, data=data, headers=headers, method=method)
with urllib.request.urlopen(req, timeout=30, context=_ssl_context()) as resp:
return json.loads(resp.read().decode("utf-8"))
def fetch_workday(args):
host, site, tenant = args["host"], args["site"], args["tenant"]
search_text = args.get("search_text", "")
url = f"https://{host}/wday/cxs/{tenant}/{site}/jobs"
jobs, offset = [], 0
while True:
data = http_get_json(url, method="POST", data={
"appliedFacets": {}, "limit": 20, "offset": offset,
"searchText": search_text,
})
postings = data.get("jobPostings", [])
for p in postings:
ext = p.get("externalPath", "")
jid = (p.get("bulletFields") or [ext])[0] if p.get("bulletFields") else ext
jobs.append({
"id": jid,
"title": p.get("title", ""),
"location": p.get("locationsText", "") + " " + ext,
# externalPath is site-relative ("/job/<loc>/<slug>_<req>"), so it needs the
# locale + site prefix. Without it every Workday URL 404s — verified 2026-09-04
# on NVIDIA, Novartis and Equinor (4/4 short form 404, long form 200).
"url": f"https://{host}/en-US/{site}{ext}",
"posted": p.get("postedOn", ""),
"description": "",
})
total = data.get("total", 0)
offset += len(postings)
if not postings or offset >= total:
break
return jobs
def fetch_ashby(args):
slug = args["slug"]
url = f"https://api.ashbyhq.com/posting-api/job-board/{slug}?includeCompensation=true"
data = http_get_json(url)
jobs = []
for j in data.get("jobs", []):
secs = j.get("secondaryLocations", []) or []
sec_names = [s.get("location", "") if isinstance(s, dict) else str(s) for s in secs]
loc_blob = " | ".join([j.get("location", "") or ""] + sec_names)
jobs.append({
"id": j.get("id"),
"title": j.get("title", ""),
"location": loc_blob,
"url": j.get("jobUrl"),
"posted": j.get("publishedAt", ""),
"description": (j.get("descriptionPlain") or "")[:2500],
"department": j.get("department", ""),
})
return jobs
def fetch_greenhouse(args):
# Title-filtered boards score title_only — skip heavy JD HTML (content=true) for those.
board = args["board"]
want_content = not args.get("_title_filter")
url = f"https://boards-api.greenhouse.io/v1/boards/{board}/jobs"
if want_content:
url += "?content=true"
data = http_get_json(url)
jobs = []
for j in data.get("jobs", []):
loc = (j.get("location") or {}).get("name", "")
offices = j.get("offices") or []
office_names = " | ".join(o.get("name", "") for o in offices if isinstance(o, dict))
loc_blob = " ".join(x for x in [loc, office_names] if x)
desc = ""
if want_content:
desc = j.get("content", "") or ""
desc = re.sub(r"<[^>]+>", " ", desc)
desc = re.sub(r"\s+", " ", desc).strip()
jobs.append({
"id": str(j.get("id")),
"title": j.get("title", ""),
"location": loc_blob,
"url": j.get("absolute_url"),
"posted": j.get("published_at", ""),
"updated_at": j.get("updated_at", ""),
"date_provenance": "Greenhouse updated_at is an update, not a publication date",
"description": desc[:2500],
})
return jobs
def fetch_pcsx(args):
"""Eightfold PCSX search API. Microsoft uses apply.careers.microsoft.com.
The same endpoint pattern is used by other PCS-hosted boards."""
domain = args["domain"]
location = args.get("location", "")
base = "https://apply.careers.microsoft.com/api/pcsx/search"
jobs, start = [], 0
while True:
url = f"{base}?domain={domain}&query=&location={urllib.parse.quote(location)}&start={start}&num=50"
# The PCSX API rate-limits after a handful of calls (429 seen 2026-10-22), and two
# entries (CH, Norway) now query it in parallel. Back off and retry before failing.
for attempt in range(3):
try:
data = http_get_json(url, headers={"Referer": f"https://apply.careers.microsoft.com/careers?location={urllib.parse.quote(location)}"})
break
except urllib.error.HTTPError as e:
if e.code != 429 or attempt == 2:
raise
time.sleep(10 * (attempt + 1))
payload = data.get("data") or {}
positions = payload.get("positions", []) or []
for p in positions:
locs = p.get("locations") or []
jobs.append({
"id": str(p.get("id")),
"title": p.get("name", ""),
"location": " | ".join(locs),
# The old jobs.careers.microsoft.com/global/en/job/<displayJobId> form redirects
# to a generic search page (verified 2026-10-20, including for 200043897). The
# working form is the API's positionUrl on apply.careers.microsoft.com. The
# display id rides along as ?jobId= so the decision log keeps keying on it.
"url": (f"https://apply.careers.microsoft.com"
f"{p.get('positionUrl') or '/careers/job/' + str(p.get('id'))}"
f"?jobId={p.get('displayJobId') or p.get('id')}"),
"posted": p.get("postedTs", ""),
"description": (p.get("description") or "")[:2000],
# Norway entry: language check on the full description, before truncation.
**({"lang_check": classify_language_requirement(
_html_to_text(p.get("description") or ""))}
if args.get("_location_policy", "").startswith("norway") else {}),
})
# The API serves ~10 rows per call regardless of `num`, so a short page is NOT
# end-of-results — page until a call comes back empty or `count` is reached.
# (The old `len(positions) < 50` break capped Microsoft at its first 10 CH roles
# and hid the entire Zürich Principal-FDE cluster sitting on page 2.)
if not positions:
break
start += len(positions)
total = payload.get("count")
if isinstance(total, int) and start >= total:
break
if start >= 500:
break
return jobs
def fetch_smartrecruiters(args):
"""SmartRecruiters public postings API. Used by many EU energy/commodity firms."""
company = args["company"]
base = f"https://api.smartrecruiters.com/v1/companies/{company}/postings"
jobs, offset = [], 0
while True:
data = http_get_json(f"{base}?limit=100&offset={offset}")
content = data.get("content", []) or []
for p in content:
loc = p.get("location") or {}
country = (loc.get("country") or "").upper()
parts = [loc.get("fullLocation") or loc.get("city") or "", country]
if loc.get("remote"):
parts.append("Remote")
if loc.get("hybrid"):
parts.append("Hybrid")
loc_str = " ".join(x for x in parts if x)
dept = (p.get("department") or {}).get("label", "") if isinstance(p.get("department"), dict) else ""
func = (p.get("function") or {}).get("label", "") if isinstance(p.get("function"), dict) else ""
jobs.append({
"id": str(p.get("id")),
"title": p.get("name", ""),
"location": loc_str,
"country": country,
"url": f"https://jobs.smartrecruiters.com/{company}/{p.get('id')}",
"posted": p.get("releasedDate", ""),
"description": " ".join(filter(None, [dept, func])),
})
total = data.get("totalFound", 0)
offset += len(content)
if not content or offset >= total or offset >= 300:
break
return jobs
def _rss_item_location(link, description, args):
"""Per-item location for feeds that carry none.
Order: the item's own page via `location_page_regex` (first group), then a
"based in <place>" sentence in the feed description, then `default_location`.
Added 2026-10-15: stamping every BIS vacancy with default_location labelled a Hong Kong
req (jr100491) as Basel, so it passed the CH location filter and reached the triage.
"""
pattern = args.get("location_page_regex")
if pattern and link:
try:
req = urllib.request.Request(link, headers={"User-Agent": USER_AGENT})
with urllib.request.urlopen(req, timeout=30, context=_ssl_context()) as resp:
page = resp.read().decode("utf-8", errors="replace")
match = re.search(pattern, page, re.I | re.S)
if match and match.group(1).strip():
return html.unescape(match.group(1).strip())
except (urllib.error.URLError, OSError, ValueError):
pass
match = re.search(r"\bbased in ([A-Z][^.;:]{2,60})", description)
if match:
return match.group(1).strip()
print(f" ! {link or '?'}: no location on page or in feed, using default_location",
file=sys.stderr)
return args.get("default_location", "")
def fetch_rss(args):
"""Generic RSS/RDF feed parser. BIS publishes vacancies as RSS 1.0 (RDF), whose
<item> elements live in the http://purl.org/rss/1.0/ namespace. Falls back to plain
RSS 2.0 <item> elements. Location isn't in the feed; see _rss_item_location."""
import xml.etree.ElementTree as ET
req = urllib.request.Request(args["url"], headers={"User-Agent": USER_AGENT})
with urllib.request.urlopen(req, timeout=30, context=_ssl_context()) as resp:
root = ET.fromstring(resp.read())
ns = {"rss1": "http://purl.org/rss/1.0/", "dc": "http://purl.org/dc/elements/1.1/"}
items = root.findall(".//rss1:item", ns) or root.findall(".//item")
jobs = []
for it in items:
def field(tag, namespaced=True):
el = it.find(f"rss1:{tag}", ns) if namespaced else it.find(tag)
if el is None and namespaced:
el = it.find(tag)
return (el.text or "").strip() if el is not None and el.text else ""
link = field("link")
description = re.sub(r"<[^>]+>", " ", field("description"))
jobs.append({
"id": link or field("title"),
"title": field("title"),
"location": _rss_item_location(link, description, args),
"url": link,
"posted": (it.findtext("dc:date", default="", namespaces=ns) or field("date")),
"description": description[:1500],
})
return jobs
# schema.org JobPosting addresses carry ISO alpha-2 country codes; the location policy
# keyword lists are full names. Only the countries the policies actually test for.
_ISO2_COUNTRY = {
"CH": "Switzerland", "NO": "Norway", "DK": "Denmark", "DE": "Germany",
"AT": "Austria", "FR": "France", "IT": "Italy", "ES": "Spain", "PL": "Poland",
"NL": "Netherlands", "BE": "Belgium", "SE": "Sweden", "FI": "Finland",
"GB": "United Kingdom", "UK": "United Kingdom", "IE": "Ireland", "US": "United States",
"PT": "Portugal", "CZ": "Czechia", "RO": "Romania", "RS": "Serbia", "TR": "Turkey",
}
def fetch_teamtailor(args):
"""Teamtailor public JSON Feed (`https://<slug>.teamtailor.com/jobs.json`).
Used by Telenor (`slug`) and Axpo (`base_url`, a Teamtailor board on a custom domain —
careers.axpo.com — which no <slug>.teamtailor.com host resolves to). The public /jobs
HTML page paginates and under-reports; the feed is authoritative. The feed carries no
location field, so default_location is required (Teamtailor boards are per-country).
Descriptions are frequently Norwegian — see the _score_floor note on the company.
The feed caps at 100 items and links the next page via `next_url`; follow it, or large
boards silently truncate (Axpo is ~400 roles, i.e. 4 pages).
TRAP: an unclaimed slug returns Teamtailor's *demo* board (HTTP 200, plausible JSON)
instead of 404. Verified 2026-07-29: `ksat` and `akerbp` both served the seed set
below though neither company uses Teamtailor. Filtered here so phantom roles never
reach a report — if a real board is ever dropped by this, widen the fingerprint."""
DEMO_TITLES = {"sales development manager", "team lead - csm", "social media manager",
"customer success manager", "key account manager", "ux designer"}
if args.get("base_url"):
url = args["base_url"].rstrip("/") + "/jobs.json"
else:
url = f"https://{args['slug']}.teamtailor.com/jobs.json"
jobs = []
for _ in range(args.get("max_pages", 10)):
req = urllib.request.Request(
url, headers={"User-Agent": USER_AGENT, "Accept": "application/json"})
with urllib.request.urlopen(req, timeout=30, context=_ssl_context()) as resp:
data = json.loads(resp.read().decode("utf-8", "replace"))
items = data.get("items", []) or []
for it in items:
# Prefer the schema.org JobPosting payload's real address over default_location.
# Axpo is a pan-European trader (Madrid, Milan, Warsaw, Germany, France as well as
# Baden/Zurich), so defaulting every role to the board's home country would mark
# ~380 non-Swiss roles CH-eligible — the same class of bug as Apple's forced
# "Switzerland" default. Telenor's feed has no _jobposting and falls back cleanly.
location = args.get("default_location", "")
places = ((it.get("_jobposting") or {}).get("jobLocation") or [])
if isinstance(places, dict):
places = [places]
parts = []
for place in places:
addr = (place or {}).get("address") or {}
# schema.org gives ISO alpha-2 ("CH", "NO"); the location policy matches on
# full country names, so expand or the role reads as location-unknown.
country = _ISO2_COUNTRY.get((addr.get("addressCountry") or "").upper(),
addr.get("addressCountry"))
bit = ", ".join(x for x in (addr.get("addressLocality"), country) if x)
if bit and bit not in parts:
parts.append(bit)
if parts:
location = " | ".join(parts)
job = {
"id": it.get("id") or it.get("url", ""),
"title": it.get("title", ""),
"location": location,
"url": it.get("url", ""),
"posted": it.get("date_published", ""),
"description": re.sub(r"<[^>]+>", " ", it.get("content_html", ""))[:2500],
}
# Norway boards: check the language requirement on the FULL ad. The stored
# description is truncated for scoring and usually cuts the requirement list off
# (Telenor's "kommunisere godt ... på norsk og engelsk" read UNCLEAR before this).
if args.get("_location_policy", "").startswith("norway"):
job["lang_check"] = classify_language_requirement(
_html_to_text(it.get("content_html", "")))
jobs.append(job)
nxt = data.get("next_url")
if not items or not nxt or nxt == url:
break
url = nxt
titles = {j["title"].strip().lower() for j in jobs}
if jobs and len(titles & DEMO_TITLES) >= 4:
raise RuntimeError(
f"teamtailor slug '{args['slug']}' served the demo board, not a real one "
f"({len(jobs)} placeholder ads) — the company likely uses a different ATS")
return jobs
def fetch_getro(args):
"""Getro network job-board search API (POST JSON). Powers VC portfolio talent
networks. NO company in COMPANIES currently uses this adapter — the Coinbase Ventures
network was dropped 2026-08-18 (see the note there). Kept because it generalises to any
Getro collection id. Note such boards list PORTFOLIO companies' roles, never the VC's
own openings, so an org filter is usually needed. Server-side filters:
searchable_locations and job_functions. Org name is folded into the title since this
is a multi-company board."""
collection = args["collection"]
url = f"https://api.getro.com/api/v2/collections/{collection}/search/jobs"
filters = {}
if args.get("locations"):
filters["searchable_locations"] = args["locations"]
if args.get("job_functions"):
filters["job_functions"] = args["job_functions"]
jobs, page = [], 0
excluded_orgs = {name.casefold() for name in args.get("_exclude_orgs", [])}
while True:
data = http_get_json(url, method="POST", data={
"hitsPerPage": 100, "page": page, "query": "", "filters": filters,
})
res = data.get("results", {}) or {}
batch = res.get("jobs", []) or []
for j in batch:
org = (j.get("organization") or {}).get("name", "")
if org.casefold() in excluded_orgs:
continue
locs = j.get("searchable_locations") or j.get("locations") or []
loc_str = " | ".join(locs) if isinstance(locs, list) else str(locs)
ts = j.get("created_at")
posted = ""
if isinstance(ts, (int, float)):
posted = datetime.fromtimestamp(ts, tz=timezone.utc).strftime("%Y-%m-%d")
title = j.get("title", "")
jobs.append({
"id": str(j.get("id")),
"title": f"{title} @ {org}" if org else title,
"location": loc_str,
"url": j.get("url", ""),
"posted": posted,
"description": " ".join(filter(None, [org] + (j.get("skills") or []))),
})
total = res.get("count", 0)
page += 1
if not batch or len(jobs) >= total or page >= 10:
break
return jobs
def fetch_amazon(args):
"""amazon.jobs public search JSON. Covers Amazon + AWS, which share one board.
`normalized_country_code[]` is the filter that actually works; the plain `country[]`
and `loc_query` params are ignored and silently return the global (US-heavy) set —
verified 2026-08-18, where `country[]=CHE` returned 6,837 mostly-Seattle hits against
32 for the normalized form."""
base = "https://www.amazon.jobs/en/search.json"
headers = {"User-Agent": ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36")}
jobs, offset, page_size = [], 0, 100
while True:
qs = urllib.parse.urlencode({
"radius": "100mi", "offset": offset, "result_limit": page_size,
"sort": "recent", "base_query": args.get("base_query", ""),
})
countries = "".join(f"&normalized_country_code[]={urllib.parse.quote(c)}"
for c in args.get("countries", ["CHE"]))
data = http_get_json(f"{base}?{qs}{countries}", headers=headers)
batch = data.get("jobs", []) or []
for j in batch:
jobs.append({
"id": str(j.get("id_icims") or j.get("id") or ""),
"title": j.get("title", ""),
"location": j.get("normalized_location") or j.get("location") or "",
"url": "https://www.amazon.jobs" + (j.get("job_path") or ""),
"posted": j.get("posted_date", ""),
"description": (j.get("description_short") or j.get("description") or "")[:2000],
})
total = data.get("hits", 0)
offset += page_size
if not batch or offset >= total or offset >= args.get("max_results", 500):
break
return jobs
def fetch_phenom(args):
"""Phenom People careers search (POST /widgets with ddoKey=refineSearch). Used by Roche.
The public search-results page is a JS shell whose location filter silently fails: a
scrape of `?locationsearch=Switzerland` returns *recommendation-widget* cards (Shanghai,
Kyiv, Bogota) while the page state reports `no-results`. Verified 2026-08-18 — this is
why Roche contributed 0 Swiss roles for months while looking healthy. The underlying
refineSearch endpoint filters correctly (134 CH roles), so query it directly.
`country_facet` is the value for the `country` facet (e.g. "Switzerland"). `ref_num` and
`page_id` are tenant constants visible in any /widgets POST from the careers site."""
url = args["url"]
ref_num, page_id = args["ref_num"], args.get("page_id", "page11-ds")
lang = args.get("lang", "en_global")
page_size = args.get("page_size", 100)
headers = {
"Origin": "{0.scheme}://{0.netloc}".format(urllib.parse.urlsplit(url)),
"Referer": url,
# Phenom 403s the default library UA.
"User-Agent": ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"),
}
jobs, offset = [], 0
while True:
data = http_get_json(url, method="POST", headers=headers, data={
"lang": lang, "deviceType": "desktop", "country": "global",
"pageName": "search-results", "ddoKey": "refineSearch",
"sortBy": "", "subsearch": "", "from": offset, "jobs": True, "counts": True,
"all_fields": ["country", "state", "city", "category", "type"],
"size": page_size, "clearAll": False, "jdsource": "facets",
"isSliderEnable": False, "pageId": page_id, "siteType": "external",
"keywords": "", "global": True, "locationData": {},
"selected_fields": {"country": [args["country_facet"]]},
})
res = data.get("refineSearch", {}) or {}
batch = (res.get("data", {}) or {}).get("jobs", []) or []
for j in batch:
jobs.append({
"id": str(j.get("jobId") or j.get("jobSeqNo") or ""),
"title": j.get("title", ""),
"location": j.get("location") or j.get("cityStateCountry") or "",
"url": j.get("applyUrl", ""),
"posted": (j.get("postedDate") or "")[:10],
"description": (j.get("descriptionTeaser") or "")[:2000],
})
total = res.get("totalHits", 0)
offset += page_size
if not batch or offset >= total or offset >= args.get("max_results", 600):
break
return jobs
def fetch_onlyfy(args):
"""onlyfy.jobs board (XING E-Recruiting / ex-Prinzip), used by Bitcoin Suisse. The
candidate/job/ajax_list endpoint returns an HTML fragment listing every posting; each
card carries a <a href="/job/ID">title</a> and a location cell flagged by an
icon-map-marker. Titles and locations appear in document order, one of each per card,
so we extract both lists and zip them. No JSON API and no headless browser needed."""
import html as _html
slug = args["slug"]
base = f"https://{slug}.onlyfy.jobs"
url = (f"{base}/candidate/job/ajax_list"
f"?display_length=100&page=1&sort=date&sort_dir=DESC&search=")
req = urllib.request.Request(url, headers={
"User-Agent": USER_AGENT, "X-Requested-With": "XMLHttpRequest",
})
with urllib.request.urlopen(req, timeout=30, context=_ssl_context()) as resp:
page = resp.read().decode("utf-8", "replace")
titles = re.findall(r'<a href="(/job/[a-z0-9]+)">(.*?)</a>', page, re.S)
locs = re.findall(r'icon-map-marker[^>]*></i>\s*([^<]+)', page)
if titles and locs and len(titles) != len(locs):
# Misaligned regex pairs would invent wrong locations; keep titles, drop bad locs.
print(f" onlyfy: title/loc count mismatch ({len(titles)} vs {len(locs)}); "
f"using titles only", file=sys.stderr)
locs = [""] * len(titles)
jobs = []
for (href, raw_title), raw_loc in zip(titles, locs if locs else [""] * len(titles)):
title = _html.unescape(re.sub(r"<[^>]+>", "", raw_title)).strip()
loc = _html.unescape(raw_loc).strip() if raw_loc else ""
jobs.append({
"id": href.rsplit("/", 1)[-1],
"title": title,
"location": loc,
"url": base + href,
"posted": "",
"description": loc,
})
return jobs
def fetch_lever(args):
"""Lever public postings API. Palantir uses this. The board is US/London-heavy;
Swiss/Zurich (Schwyz hub) roles are rare on it but will surface here when posted —
location filtering downstream drops the US/London bulk. categories.allLocations
captures multi-location postings; createdAt is epoch-ms."""
slug = args["slug"]
data = http_get_json(f"https://api.lever.co/v0/postings/{slug}?mode=json")
jobs = []
for j in data:
cats = j.get("categories") or {}
all_locs = cats.get("allLocations") or []
loc_blob = " | ".join(x for x in ([cats.get("location") or ""] + [str(a) for a in all_locs]) if x)
ts = j.get("createdAt")
posted = ""
if isinstance(ts, (int, float)):
posted = datetime.fromtimestamp(ts / 1000, tz=timezone.utc).strftime("%Y-%m-%d")
jobs.append({
"id": j.get("id"),
"title": j.get("text", ""),
"location": loc_blob,
"url": j.get("hostedUrl"),
"posted": posted,
"description": (j.get("descriptionPlain") or "")[:2500],
})
return jobs
def _parse_taleo_jobsearch_html(html, host, career_section):
"""Extract jobs from a Taleo jobsearch.ftl HTML response.
Taleo does not expose a reliable public JSON search API for this board (REST
returns careerSectionUnAvailable). Jobs are embedded in the page as
api.fillInterface arrays and as a pipe-delimited initialHistory field.
"""
import html as _html
jobs, seen = [], set()
detail_base = f"https://{host}/careersection/{career_section}/jobdetail.ftl"
# Quoted fillInterface form (already-unescaped dates in modern Taleo builds).
# Observed record shape (NATO 2026-07):
# '<id>','<title>','<id>','<title>', five more '<id>',
# '<jobNo>','<Country-City>','false','','','','',
# '<deadline>','<NATO body>','<grade>'
pat = re.compile(
r"'(\d+)','((?:[^'\\]|\\.)+)',"
r"'\1','\2',"
r"(?:'\1',){5}"
r"'(\d+)','((?:[^'\\]|\\.)+)',"
r"'false','','','','',"
r"'((?:[^'\\]|\\.)*)','((?:[^'\\]|\\.)*)','((?:[^'\\]|\\.)*)'"
)
for m in pat.finditer(html):
job_no = m.group(3)
if job_no in seen:
continue
seen.add(job_no)
def _unq(s):
return _html.unescape(s.replace("\\'", "'").replace("\\:", ":"))
title = _unq(m.group(2))
loc = _unq(m.group(4)).replace("-", ", ")
deadline = _unq(m.group(5))
body = _unq(m.group(6))
grade = _unq(m.group(7))
# List view exposes an application DEADLINE only, never a posted-on date. It used to
# be stored in `posted`, which made a role closing soon look freshly published -
# backwards for a freshness signal (F15). It now goes to `deadline`, and `posted`
# stays empty because this board genuinely does not publish one.
deadline_short = deadline.split(",")[0].strip() if deadline else ""
jobs.append({
"id": job_no,
"title": title,
"location": loc,
"url": f"{detail_base}?job={job_no}&lang=en",
"posted": "",
"deadline": deadline_short,
"date_provenance": "taleo list view exposes a closing date only; no publication date",
"description": f"{body} {grade}".strip()[:500],
})
if jobs:
return jobs
# Fallback: pipe-delimited initialHistory hidden field
m = re.search(
r'(?:name|id)="initialHistory"[^>]*value="([^"]*)"', html, re.I
) or re.search(
r'value="([^"]*)"[^>]*(?:name|id)="initialHistory"', html, re.I
)
if not m:
return []
hist = urllib.parse.unquote(m.group(1).replace("%5C:", ":").replace("\\:", ":"))
parts = hist.split("!|!")
for i, p in enumerate(parts):
if not re.fullmatch(r"\d{5,6}", p) or i + 1 >= len(parts):
continue
loc_raw = parts[i + 1]
if not re.search(
r"Norway|Denmark|Belgium|Germany|Italy|France|United|Luxembourg|"
r"Netherlands|Poland|Spain|Portugal|Canada|Turkey|Türkiye",
loc_raw, re.I,
):
continue
job_no = p
if job_no in seen:
continue
title = ""
for j in range(i - 1, max(-1, i - 12), -1):
cand = parts[j]
if (cand and not re.fullmatch(r"\d+", cand)
and cand not in ("false", "true", "Apply")
and len(cand) > 8):
title = cand
break
if not title:
continue
seen.add(job_no)
deadline = parts[i + 6] if i + 6 < len(parts) else ""
body = parts[i + 7] if i + 7 < len(parts) else ""
grade = parts[i + 8] if i + 8 < len(parts) else ""
deadline_short = deadline.split(",")[0].strip() if deadline else ""
jobs.append({
"id": job_no,
"title": _html.unescape(title),
"location": loc_raw.replace("-", ", "),
"url": f"{detail_base}?job={job_no}&lang=en",
"posted": deadline_short,
"description": f"{body} {grade}".strip()[:500],
})
return jobs
def fetch_taleo(args):
"""Oracle Taleo careersection board (used by NATO civilian vacancies).
The marketing page https://www.nato.int/en/work-with-us/careers/vacancies is an
AEM shell; the live board is Taleo. Public REST search is unavailable, so we
GET jobsearch.ftl (optionally filtered by Taleo location taxonomy IDs from the
board's location dropdown) and parse embedded job records.
Args: host (default nato.taleo.net), career_section (default "2"),
location_ids (optional list of Taleo option values — e.g. Norway/Denmark).
"""
host = args.get("host", "nato.taleo.net")
section = str(args.get("career_section", "2"))
location_ids = args.get("location_ids")
# One request per location id (Taleo query param takes a single id); merge by job no.
# If no location_ids, one unfiltered first-page fetch.
fetches = location_ids if location_ids else [None]
by_id = {}
for loc_id in fetches:
url = f"https://{host}/careersection/{section}/jobsearch.ftl?lang=en"
if loc_id:
url += f"&location={urllib.parse.quote(str(loc_id))}"
req = urllib.request.Request(url, headers={
"User-Agent": USER_AGENT,
"Accept": "text/html,application/xhtml+xml",
})
with urllib.request.urlopen(req, timeout=60, context=_ssl_context()) as resp:
html = resp.read().decode("utf-8", "replace")
for j in _parse_taleo_jobsearch_html(html, host, section):
by_id[j["id"]] = j
return list(by_id.values())
def _prospective_unit(attrs):
"""Sub-unit name from a Prospective attributes dict.
The key is `verwaltungseinheit_<id>` where <id> is DEPARTMENT-SPECIFIC — VBS uses
1083398, EFD uses 1083364. Never hardcode one: match the prefix, or the adapter breaks
the moment it is pointed at another department.
"""
for key, val in attrs.items():
if key.startswith("verwaltungseinheit_") and val:
return " / ".join(str(v) for v in val)
return ""
def fetch_prospective(args):
"""Prospective (ohws.prospective.ch) public jobs API.
Verified use: jobs.admin.ch — the Swiss federal job portal (medium 1000624, 463 roles).
jobs.admin.ch itself is an S3-hosted SPA that returns S3 `AccessDenied` for any direct
path fetch, headless browser included, so scraping the site is a dead end. The SPA reads
this API, which is public, unauthenticated, fast and far richer than the rendered page.
Why this board is worth special handling: unlike every other source in the scout, each
row carries a **Lohnklasse** (the admin.ch analogue of SBB's Anforderungsniveau — a
direct read on the comp bar), a hard **application deadline**, and a **named contact**,
which turns a cold submit there into a choice rather than a necessity. It also exposes
the real Aufgaben/Anforderungen text, so scoring runs against the JD body instead of the
title — the failure mode `job_scout_overscoring_findings` records.
Args: medium_id (required), units (list of substrings matched against the sub-unit name;
empty/absent = no unit filter), lang (default "de"), page_size (default 200).
"""
medium = args["medium_id"]
units = [u.lower() for u in args.get("units", [])]
lang = args.get("lang", "de")
page_size = int(args.get("page_size", 200))
base = f"https://ohws.prospective.ch/public/v1/medium/{medium}/jobs"
# Page defensively: one request that happens to fit today is not a contract.
raw, offset, total = [], 0, None
for _ in range(50):
data = http_get_json(f"{base}?lang={lang}&offset={offset}&limit={page_size}")
batch = data.get("jobs") or []
if total is None:
total = data.get("total")
if not batch:
break
raw.extend(batch)
offset += len(batch)
if total is not None and offset >= total:
break
jobs = []
for j in raw:
attrs = j.get("attributes") or {}
unit = _prospective_unit(attrs)
if units and not any(u in unit.lower() for u in units):
continue
szas = j.get("szas") or {}
loc = html.unescape(" / ".join(attrs.get("arbeitsort") or []) or "Schweiz")
lohnklasse = " / ".join(attrs.get("lohnklasse") or [])
deadline = (j.get("end_date") or "")[:10]
# Fold the scoreable JD body plus the fields that drive real decisions (unit,
# Lohnklasse, deadline, contact) into description so the scorer sees them too.
body = " ".join(str(szas.get(k, "")) for k in
("sza_role", "sza_tasks", "sza_requirements", "sza_field_of_activity"))
body = html.unescape(re.sub(r"<[^>]+>", " ", body))
desc = " | ".join(x for x in (
unit,
f"Lohnklasse {lohnklasse}" if lohnklasse else "",
f"Bewerbungsfrist {deadline}" if deadline else "",
str(szas.get("sza_contact", "")),
body,
) if x)
jobs.append({
"id": str(j.get("id") or j.get("viewkey") or ""),
"title": html.unescape((j.get("title") or "").strip()),
# Suffix so the CH location filter matches city-only values like "Zimmerwald".
"location": f"{loc}, Switzerland" if args.get("loc_suffix", True) else loc,
"url": (j.get("links") or {}).get("directlink", ""),
"posted": (j.get("start_date") or "")[:10],
"deadline": deadline,
"date_provenance": "prospective start_date = publication; end_date = application deadline",
# Carried as fields, not just folded into description: these decide level, pay and
# whether a cold submit is even necessary (F15).
"unit": unit,
"compensation": f"Lohnklasse {lohnklasse}" if lohnklasse else "",
"contact": re.sub(r"\s+", " ", html.unescape(
re.sub(r"<[^>]+>", " ", str(szas.get("sza_contact", ""))))).strip()[:300],
"description": re.sub(r"\s+", " ", desc)[:1200],
})
return jobs
def fetch_json(args):
"""Generic JSON jobs API with configurable field names, for employer sites that expose
a clean public endpoint. Verified use: Swissgrid (Magnolia CMS
/.rest/cloud/component-data — {config, jobs:[...], filters}). Field names vary by site,
so they're configurable: field_title/field_location/field_url/field_date. loc_suffix
appends e.g. ' Switzerland' so the CH location filter matches city-only values such as
"Aarau"/"Prilly" (not every Swiss town is in CH_LOCATION_KEYWORDS). desc_keys fold extra
fields (department, employment type) into the description for keyword scoring.
Args: url, jobs_key (default "jobs"), field_* (defaults title/location/url/date),
url_prefix, loc_suffix, desc_keys."""
data = http_get_json(args["url"])
arr = data.get(args.get("jobs_key", "jobs"), []) if isinstance(data, dict) else (data or [])
ft, fl = args.get("field_title", "title"), args.get("field_location", "location")
fu, fd = args.get("field_url", "url"), args.get("field_date", "date")
prefix, suffix = args.get("url_prefix", ""), args.get("loc_suffix", "")
desc_keys = args.get("desc_keys", [])
jobs = []
for j in arr:
url = j.get(fu, "") or ""
if url and not url.startswith("http") and prefix:
url = prefix.rstrip("/") + "/" + url.lstrip("/")
loc = (j.get(fl, "") or "").strip() + suffix
desc = " ".join(str(j.get(k)) for k in desc_keys if j.get(k))
jobs.append({
"id": str(j.get("id") or url),
"title": j.get(ft, ""),
"location": loc,
"url": url,
"posted": j.get(fd, "") or "",
"description": desc[:500],
})
return jobs
def fetch_sbb(args):
"""SBB (company.sbb.ch) AEM job filter. The whole board is served as a flat JSON list
at .../jobfilter.results.json (~145 roles); the website filters client-side via each
job's numbered `attributes`: '20'=Berufsfeld/topic, '110'=region, '100'=city,
'links.directlink'=the jobs.sbb.ch URL. We replicate the user's IT + Bern-region filter
so only commutable IT roles surface. Titles are German/generic (Application Engineer,
Network Security Engineer, OT Architekt) and won't match ENG_TITLE_FILTER or the keyword
scorer, so this company is given a _score_floor in COMPANIES to keep its pre-filtered
results visible. topic/region are configurable substrings."""
url = args.get("url", ("https://company.sbb.ch/content/internet/corporate/de/"
"jobs-karriere/jobs/job-suche/jcr:content/parmain/"
"jobfilter.results.json"))
topic = args.get("topic", "IT / Telekommunikation")
region = args.get("region", "Bern Mittelland")
data = http_get_json(url)
arr = data if isinstance(data, list) else (data.get("results") or data.get("jobs") or [])
jobs = []
for j in arr:
a = j.get("attributes", {}) or {}
blob = " ".join(str(x) for v in a.values() for x in (v if isinstance(v, list) else [v]))
if topic and topic not in blob:
continue
if region and region not in blob:
continue
region_v = " ".join(a.get("110", []) or [])
city_v = " ".join(a.get("100", []) or [])
field_v = " ".join(a.get("20", []) or [])
jobs.append({
"id": str(j.get("id") or j.get("viewkey") or ""),
"title": j.get("title", ""),
"location": f"{city_v} {region_v} Schweiz".strip(),
"url": (j.get("links") or {}).get("directlink", ""),
"posted": j.get("start_date", "") or "",
"description": (field_v + " " + (j.get("text", "") or ""))[:400],
})
return jobs
def fetch_bkw(args):
"""BKW Group (jobs.bkw.com) PMS structured-data API. The whole-group board is ~600 roles
dominated by building-tech / electrical / civil-engineering trades; we keep only the
Berufsfeld categories relevant to the user (Informatik / Trading / Finanzen), which
surfaces IT/data plus the energy-trading roles (Quant Risk Modeller, Solution Architect
Energiehandel, Energy Derivatives/Market-Risk analysts). locations[].address gives
city/country. Pre-filtered + German/generic titles, so paired with a _score_floor in
COMPANIES. The category allowlist is configurable."""
url = args.get("url", ("https://jobs.bkw.com/_api/v1/structureddata?"
"configFromContentElement=82381&language=de-ch"))
allow = [c.lower() for c in args.get("categories", ["Informatik", "Trading", "Finanzen"])]
data = http_get_json(url)
arr = data if isinstance(data, list) else []
if not arr and isinstance(data, dict):
for v in data.values():
if isinstance(v, list) and v and isinstance(v[0], dict) and "title" in v[0]:
arr = v
break
jobs = []
for j in arr:
if j.get("type") and j.get("type") != "jobs":
continue
cats = [c.get("title", "") for c in (j.get("relations", {}) or {}).get("Berufsfeld", []) or []]
if allow and not any(any(a in c.lower() for a in allow) for c in cats):
continue
locs = j.get("locations") or []
addr = (locs[0].get("address") if locs and isinstance(locs[0], dict) else {}) or {}
loc = " ".join(x for x in [addr.get("city", ""), addr.get("country", "")] if x) or "Schweiz"
jobs.append({
"id": str(j.get("id") or j.get("url") or ""),
"title": j.get("title", ""),
"location": loc,
"url": j.get("url", ""),
"posted": "",
"description": " ".join(cats + [j.get("subtitle", "") or ""])[:300],
})
return jobs
# Injected before page scripts run, to mask the most common headless-detection signals.
# Required for Google; harmless for the other sites.
STEALTH_JS = """
Object.defineProperty(navigator, 'webdriver', {get: () => undefined});
window.chrome = {runtime: {}, loadTimes: () => {}, csi: () => {}, app: {}};
Object.defineProperty(navigator, 'plugins', {get: () => [1, 2, 3, 4, 5]});
Object.defineProperty(navigator, 'languages', {get: () => ['en-US', 'en', 'de']});
const _q = navigator.permissions && navigator.permissions.query;
if (_q) {
navigator.permissions.query = (p) => p && p.name === 'notifications'
? Promise.resolve({state: Notification.permission}) : _q(p);
}
"""
_playwright_singleton = {"pw": None, "browser": None}
def _get_browser():
"""Lazy-init a single shared headless browser. Saves ~3s per company."""
if _playwright_singleton["browser"] is not None:
return _playwright_singleton["browser"]
try:
from playwright.sync_api import sync_playwright
except ImportError as e:
raise RuntimeError("playwright not installed - run: pip install -r requirements.txt") from e
pw = sync_playwright().start()
browser = pw.chromium.launch(
headless=True,
args=["--disable-blink-features=AutomationControlled"],
)
_playwright_singleton["pw"] = pw
_playwright_singleton["browser"] = browser
return browser
def _absolutize(href, prefix):
"""Join a possibly-relative href with the configured prefix."""
if not href or href.startswith("http"):
return href
cleaned = href.lstrip("./").lstrip("/")
if not prefix:
return href
return prefix.rstrip("/") + "/" + cleaned
def _strip_query_param(url, param):
"""Drop a single query parameter from a URL, preserving the rest.
Job-card hrefs inherit the listing page's query string, so with query-param
pagination the same posting would get a different URL (and therefore a different
id) depending on which page it happened to land on — making every reshuffle look
like a brand-new job and orphaning its decision-log entry.
"""
if not url or f"{param}=" not in url:
return url
head, _, query = url.partition("?")
if not query:
return url
kept = [kv for kv in query.split("&") if kv and not kv.startswith(f"{param}=")]
return f"{head}?{'&'.join(kept)}" if kept else head
def _extract_location_from_blob(full, default=""):
"""Pull a short location line out of a job-card text blob.
Google/Meta/Roche-style cards often dump Material-icon chrome into inner_text
(e.g. 'corporate_fare' / 'place' / 'Zürich, Switzerland' / 'bar_chart'). Prefer the
line after a 'place'/'location' label, else the first short line that looks like a
place, else the configured default — never the first 300 chars of the whole card.
"""
if not full:
return default
lines = [ln.strip() for ln in full.splitlines() if ln.strip()]
skip = {
"place", "location", "corporate_fare", "bar_chart", "work", "schedule",
"google", "meta", "apple", "roche", "cisco",
}
for i, ln in enumerate(lines):
if ln.lower() in ("place", "location") and i + 1 < len(lines):
cand = lines[i + 1].strip()
if cand and cand.lower() not in skip and len(cand) < 120:
return cand[:200]
place_hints = (
CH_LOCATION_KEYWORDS + DK_LOCATION_KEYWORDS + NO_LOCATION_KEYWORDS
+ list(EUROPE_SCOPE_KEYWORDS) + list(GLOBAL_SCOPE_KEYWORDS)
+ ["remote", "hybrid", "onsite", "on-site"]
)
for ln in lines:
low = ln.lower()
if len(ln) > 120 or low in skip:
continue
if any(k in low for k in place_hints):
return ln[:200]
return default
def _close_browser():
if _playwright_singleton["browser"]:
try:
_playwright_singleton["browser"].close()
except Exception:
pass
if _playwright_singleton["pw"]:
try:
_playwright_singleton["pw"].stop()
except Exception:
pass
def fetch_playwright(args):
"""Generic headless-browser scraper. See COMPANIES entries for selector args."""
browser = _get_browser()
ctx = browser.new_context(
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
locale="en-US",
viewport={"width": 1366, "height": 768},
)
ctx.add_init_script(STEALTH_JS)
page = ctx.new_page()
# Per-element timeout for slow boards: Apple took 876s once and 25s the next time on the
# same day (2026-10-22); a stalled card read should fail fast, not hold the whole run.
if args.get("element_timeout_ms"):
page.set_default_timeout(args["element_timeout_ms"])
jobs = []
seen_ids = set()
def scrape_current():
"""Extract cards from the currently-loaded page; append new ones to `jobs`.
Returns the count of newly-added (not-yet-seen) cards so a pagination loop can
stop once a page contributes nothing new."""
wait_for = args.get("wait_for")
if wait_for:
try:
# Per-board timeout: Apple's search page sometimes takes well over 15s to
# render its result cards, and a miss here returns 0 jobs (2026-10-22: 0 vs 21
# two days earlier, selectors unchanged).
page.wait_for_selector(wait_for, timeout=args.get("wait_timeout_ms", 15000))
# Boards that render results progressively need a pause after the FIRST card
# appears, or the scrape catches a partial list (Apple: 15 vs 24, 2026-10-22).
if args.get("settle_ms"):
page.wait_for_timeout(args["settle_ms"])
except Exception:
page.wait_for_timeout(4000)
# Scroll a few times to trigger any lazy-loaded results
for _ in range(args.get("scroll_count", 3)):
try:
page.mouse.wheel(0, 4000)
page.wait_for_timeout(700)
except Exception:
break
cards = page.locator(args["card"])
n = min(cards.count(), args.get("max_cards", 150))
added = 0
for i in range(n):
card = cards.nth(i)
try:
title = ""
if args.get("title_attr") == "text":
title = (card.inner_text() or "").strip().split("\n", 1)[0][:200]
elif args.get("title_attr"):
title = (card.get_attribute(args["title_attr"]) or "").strip()
elif args.get("title_sel"):
t = card.locator(args["title_sel"]).first
if t.count():
# Read either an attribute (e.g. aria-label) or the inner text
if args.get("title_sel_attr"):
title = (t.get_attribute(args["title_sel_attr"]) or "").strip()
else:
title = (t.inner_text() or "").strip()
if args.get("title_strip_prefix") and title.startswith(args["title_strip_prefix"]):
title = title[len(args["title_strip_prefix"]):].strip()
if not title:
title = (card.inner_text() or "").strip().split("\n", 1)[0][:200]
location = args.get("default_location", "")
if args.get("location_sel"):
lsel = card.locator(args["location_sel"]).first
if lsel.count():
location = (lsel.inner_text() or location).strip()
link_el = card if not args.get("link_sel") else card.locator(args["link_sel"]).first
href = (link_el.get_attribute(args.get("link_attr", "href")) or "") if link_el.count() else ""
href = _absolutize(href, args.get("url_prefix", ""))
if args.get("page_param"):
href = _strip_query_param(href, args["page_param"])
if not title:
continue
jid = href or f"{page.url}#{i}"
if jid in seen_ids:
continue
seen_ids.add(jid)
added += 1
description = ""
if args.get("use_inner_text_as_blob"):
# Full card text for keyword scoring; location via structured extract.
full = (card.inner_text() or "")
description = full[:2000]
extracted = _extract_location_from_blob(
full, default=args.get("default_location", "")
)
# Prefer a real location line over a default that is empty or a
# polluted leftover; keep default_location when extract finds nothing.
if extracted:
location = extracted
elif not location:
location = args.get("default_location", "")
jobs.append({
"id": jid,
"title": title,
"location": location,
"url": href or page.url,
"posted": "",
"description": description,
})
except Exception:
continue
return added
try:
page.goto(args["url"], timeout=45000, wait_until="domcontentloaded")
# Optional cookie banner acceptance (once, on the first page)
for sel in args.get("cookie_accept", []) or []:
try:
btn = page.locator(sel).first
if btn.is_visible(timeout=2000):
btn.click()
page.wait_for_timeout(500)
except Exception:
pass
# Optional query-param pagination (e.g. Drupal "?page=N"). The base URL is the first
# page (already loaded); fetch successive pages until one adds no new cards.
# `page_param_start` is the value the SECOND page takes: 1 for 0-indexed boards
# (Drupal), 2 for 1-indexed ones (Google, where "?page=1" is just page one again).
page_param = args.get("page_param")
if page_param:
base = args["url"]
joiner = "&" if "?" in base else "?"
second = args.get("page_param_start", 1)
for p in range(args.get("max_pages", 8)):
if p > 0:
page.goto(f"{base}{joiner}{page_param}={second + p - 1}", timeout=45000,
wait_until="domcontentloaded")
added = scrape_current()
if p > 0 and added == 0:
break
else:
# Optional in-page pagination for client-side job boards. This is distinct from
# query-param pagination above: the URL does not change when moving between pages.
next_button = args.get("next_button")
if next_button:
for _ in range(args.get("max_pages", 8)):
added = scrape_current()
button = page.locator(next_button)
if button.count() != 1:
break
try:
if not button.is_visible() or button.is_disabled():
break
button.click()
page.wait_for_timeout(args.get("next_wait_ms", 800))
except Exception:
break
# A page with no unseen cards signals a loop or exhausted pagination.
if added == 0:
break
else:
scrape_current()
finally:
ctx.close()
return jobs
IBM_SEARCH_API = "https://www-api.ibm.com/search/api/v2"
def fetch_ibm(args):
"""IBM careers search: the open JSON endpoint behind ibm.com/careers/search.
Added 2026-10-15 for Confluent. IBM closed the acquisition on 2026-03-17, and the
Confluent Ashby board (`jobs.ashbyhq.com/confluent`) returns no jobs. Its roles are now
IBM requisitions, mostly titled "... - Confluent". careers.confluent.io sits behind a
Vercel bot checkpoint, so this API is the clean path.
`query` is a keyword search over title, description and body, as the site's own search
box sends it. The API rejects `size` above 100, so pages are fetched with `from`.
Location = city (unless "Multiple Cities" / "No City") + country + work mode, e.g.
"United States - Remote" or "Zurich, CH, Switzerland - Hybrid".
"""
body = {
"appId": "careers", "scopes": ["careers2"], "lang": "zz", "size": 100,
"query": {"bool": {"must": [{"simple_query_string": {
"query": args["query"],
"fields": ["keywords^1", "body^1", "url^2", "description^2",
"h1s_content^2", "title^3", "field_text_01"],
}}]}},
"sort": [{"_score": "desc"}],
"_source": ["title", "url", "description",
"field_keyword_05", "field_keyword_17", "field_keyword_19"],
}
jobs, offset = [], 0
while True:
body["from"] = offset
req = urllib.request.Request(
IBM_SEARCH_API, data=json.dumps(body).encode("utf-8"), method="POST",
headers={"Content-Type": "application/json", "User-Agent": USER_AGENT})
with urllib.request.urlopen(req, timeout=30, context=_ssl_context()) as resp:
data = json.load(resp)
hits = data.get("hits", {}).get("hits", [])
for hit in hits:
src = hit.get("_source", {})
city = (src.get("field_keyword_19") or "").strip()
country = src.get("field_keyword_05") or ""
if isinstance(country, list):
country = ", ".join(country)
place = ", ".join(p for p in (
"" if city.lower().startswith(("multiple cities", "no city")) else city,
country.strip()) if p)
mode = (src.get("field_keyword_17") or "").strip()
url = src.get("url", "")
job_id = urllib.parse.parse_qs(urllib.parse.urlsplit(url).query).get("jobId", [""])[0]
jobs.append({
"id": job_id or url,
"title": (src.get("title") or "").strip(),
"location": f"{place} - {mode}" if mode else place,
"url": url,
"posted": "",
"description": (src.get("description") or "")[:1500],
})
total = data.get("hits", {}).get("total", {}).get("value", 0)
offset += len(hits)
if not hits or offset >= total or offset >= 1000:
return jobs
# --- Language / citizenship requirement check (Norway) -----------------------------------
# Added 2026-10-22 on user instruction. The language a posting is WRITTEN in, and even NAV's
# "Arbeidsspråk" tag, is not the requirement: Norwegian-language ads often accept English
# ("norsk og/eller engelsk"), and English ads sometimes demand Norwegian. He has A2
# Norwegian, so only an explicit Norwegian requirement is a fail. Clearance is NOT a gate
# (user, 2026-10-22: EU/NATO citizen, expects to be clearable); only an explicit Norwegian-
# citizenship requirement is. Every verdict carries its evidence snippet so it can be checked.
_LANG_ENGLISH_OK = [re.compile(p, re.I) for p in (
# The optional hyphen covers compounds: "gode norsk- eller engelskkunnskaper".
r"norsk-?\s+og\s*/\s*eller\s+engelsk", r"engelsk-?\s+og\s*/\s*eller\s+norsk",
r"norsk-?\s+eller\s+engelsk", r"engelsk-?\s+eller\s+norsk",
r"skandinavisk\w*-?\s+(?:språk\s+)?(?:og\s*/\s*)?eller\s+engelsk",
r"engelsk\s+(?:og\s*/\s*)?eller\s+(?:et\s+)?skandinavisk",
r"english\s+(?:and\s*/\s*)?or\s+(?:a\s+)?(?:scandinavian|norwegian)",
r"(?:norwegian|scandinavian)\s+(?:and\s*/\s*)?or\s+english",
r"(?:working|company|corporate|official|team(?:'s)?)\s+language\s+is\s+english",
r"arbeidsspråk(?:et)?\s+(?:er|vil\s+være)\s+engelsk",
r"norwegian\s+(?:language\s+)?(?:skills?\s+)?(?:is|are)\s+(?:not\s+(?:a\s+)?(?:required|requirement|necessary)"
r"|(?:a|an)\s+(?:plus|advantage|bonus|asset)|beneficial|nice\s+to\s+have)",
r"norsk\w*\s+(?:er\s+(?:en\s+)?(?:fordel|ønskelig|et\s+pluss)|(?:er\s+)?ikke\s+(?:et\s+)?krav)",
r"no\s+norwegian\s+(?:is\s+)?(?:required|needed|necessary)",
)]
_LANG_NORWEGIAN_REQUIRED = [re.compile(p, re.I) for p in (
r"(?:flytende|god|gode|meget\s+gode?|svært\s+gode?|utmerkede?|beherske\w*)\s+(?:\w+\s+){0,4}norsk",
r"norsk\s+(?:muntlig\s+og\s+skriftlig|skriftlig\s+og\s+muntlig)",
r"(?:må|krav\s+om|kreves|forutsetter)\s+(?:\w+\s+){0,6}norsk",
r"norsk\w*\s+(?:og|samt)\s+engelsk",
r"skandinavisk\w*\s+språk",
r"fluent\s+(?:in\s+)?(?:norwegian|a\s+scandinavian)",
r"(?:proficien\w+|fluency|good\s+command)\s+(?:in\s+|of\s+)?(?:norwegian|a\s+scandinavian)",
r"norwegian\s+(?:language\s+)?(?:is\s+)?(?:required|mandatory|a\s+requirement|essential)",
r"(?:must|need\s+to|required\s+to)\s+(?:\w+\s+){0,4}(?:norwegian|scandinavian)",
)]
# An ad that states only an English requirement ("communication and presentation skills in
# English", Nordic Semiconductor 2026-10-22). Checked only AFTER no Norwegian requirement was
# found, so "fluent English and Norwegian" still reads as Norwegian required.
_LANG_ENGLISH_STATED = [re.compile(p, re.I) for p in (
r"(?:communication|presentation|written|spoken|verbal|language)\s+(?:and\s+\w+\s+)?skills?\s+in\s+english",
r"(?:fluent|fluency|proficien\w*|excellent|good|strong)\s+(?:\w+\s+){0,2}(?:in\s+)?english",
r"english\s+(?:is\s+)?(?:required|a\s+must|mandatory)",
r"(?:gode|god|flytende|svært\s+gode)\s+(?:\w+\s+){0,3}engelsk",
)]
# A "Norwegian" that belongs to clearance or citizenship wording is not a language demand.
_LANG_FALSE_CONTEXT = re.compile(
r"clearance|klarering|statsborger|citizen|myndighet|authorit|armed\s+forces|forsvar", re.I)
_CITIZENSHIP_NO = re.compile(r"norsk\s+statsborger\w*|norwegian\s+citizen\w*", re.I)
_CITIZENSHIP_NEGATED = re.compile(r"(?:ikke|not|no)\s+(?:\w+\s+){0,4}$", re.I)
_CLEARANCE = re.compile(r"sikkerhetsklar\w*|security\s+clearance|klareres|klarering", re.I)
def _snippet(text, match, width=90):
start = max(0, match.start() - width // 2)
return re.sub(r"\s+", " ", text[start:match.end() + width // 2]).strip()
def classify_language_requirement(text, work_language_tags=None):
"""Return {lang, evidence, citizenship_required, clearance} for a Norway posting.
lang is one of: english_ok, norwegian_required, unclear. An explicit English-acceptable
phrase wins over a Norwegian one ("gode kunnskaper i norsk og/eller engelsk" is English
OK). With no requirement sentence, NAV's tag decides only when it is English-only.
"""
text = text or ""
tags = [t.lower() for t in (work_language_tags or [])]
result = {"lang": "unclear", "evidence": "", "citizenship_required": False,
"clearance": False}
for pattern in _LANG_ENGLISH_OK:
match = pattern.search(text)
if match:
result.update(lang="english_ok", evidence=_snippet(text, match))
break
else:
for pattern in _LANG_NORWEGIAN_REQUIRED:
for match in pattern.finditer(text):
window = text[match.start():match.end() + 40]
if _LANG_FALSE_CONTEXT.search(window):
continue
result.update(lang="norwegian_required", evidence=_snippet(text, match))
break
if result["lang"] == "norwegian_required":
break
else:
stated = next((m for p in _LANG_ENGLISH_STATED if (m := p.search(text))), None)
if stated:
result.update(lang="english_ok",
evidence=_snippet(text, stated) + " (English stated, no Norwegian requirement)")
elif tags and all(t.startswith("engelsk") or t == "english" for t in tags):
result.update(lang="english_ok", evidence="NAV work-language tag: English only")
elif tags:
result["evidence"] = f"no requirement sentence found; NAV tag: {', '.join(tags)}"
else:
result["evidence"] = "no requirement sentence found"
for match in _CITIZENSHIP_NO.finditer(text):
if not _CITIZENSHIP_NEGATED.search(text[max(0, match.start() - 40):match.start()]):
result["citizenship_required"] = True
result["citizenship_evidence"] = _snippet(text, match)
break
clearance = _CLEARANCE.search(text)
if clearance:
result["clearance"] = True
return result
# --- NAV national job register (Norway) ----------------------------------------------------
# arbeidsplassen.nav.no carries most Norwegian postings, including employers with no public
# ATS API (Kongsberg, Forsvaret, Nammo, SINTEF, Kitron). Measured 2026-10-22 it is THIN for
# Equinor (2 vs ~10 on its own board), DNV (2), Nordic Semiconductor (0) and NATO (0), which
# therefore keep dedicated adapters; NAV rows that duplicate them are dropped in _main.
# RATE LIMIT: the search API answered 429 for ~10 minutes, with no Retry-After header, after
# about 20 quick requests (2026-10-21/22). The adapter therefore spends a fixed request
# budget per run, paces requests, and on a 429 stops and returns what it has with a warning
# instead of failing the run. Per-posting text checks are cached so later runs only fetch
# new postings.
NAV_SEARCH_URL = "https://arbeidsplassen.nav.no/stillinger/api/search"
NAV_AD_URL = "https://arbeidsplassen.nav.no/stillinger/stilling/{uuid}"
NAV_DETAIL_CACHE = ROOT / "state" / "nav_detail_cache.json"
# Full ad text comes from NAV's OFFICIAL third-party feed, not the website. Measured
# 2026-10-22: the website search API 429s after ~9 requests at 3s spacing and the block then
# lasts hours, while the feed host served 12 requests at 0.5s without a 429. The feed entry
# endpoint takes the same uuid the search returns and returns clean ad HTML (no page chrome).
# Its bearer token is public but rotates (exp ~1 month), so it is fetched fresh each run.
# The feed itself cannot replace search for discovery: it pages chronologically from 2023 and
# ?last=true only shows changes from now on, not the ads already open.
NAV_FEED_TOKEN_URL = "https://pam-stilling-feed.nav.no/api/publicToken"
NAV_FEED_ENTRY_URL = "https://pam-stilling-feed.nav.no/api/v1/feedentry/{uuid}"
def _nav_feed_token():
req = urllib.request.Request(NAV_FEED_TOKEN_URL, headers={"User-Agent": USER_AGENT})
with urllib.request.urlopen(req, timeout=30, context=_ssl_context()) as resp:
body = resp.read().decode("utf-8", "replace")
match = re.search(r"eyJ[\w-]+\.[\w-]+\.[\w-]+", body)
if not match:
raise RuntimeError("NAV feed public token not found")
return match.group(0)
def _nav_ad_text(uuid, token, pace_s):
"""Full ad text for one posting via the official feed; raises _RateLimited on 429."""
time.sleep(pace_s)
req = urllib.request.Request(NAV_FEED_ENTRY_URL.format(uuid=uuid), headers={
"User-Agent": USER_AGENT, "Accept": "application/json",
"Authorization": f"Bearer {token}"})
try:
with urllib.request.urlopen(req, timeout=40, context=_ssl_context()) as resp:
entry = json.loads(resp.read().decode("utf-8", "replace"))
except urllib.error.HTTPError as e:
if e.code == 429:
raise _RateLimited(uuid) from e
raise
ad = entry.get("ad_content") or {}
return _html_to_text(f"{ad.get('title', '')} {ad.get('description') or ''}")
def _nav_page_text(job, pace_s):
"""Fallback: the arbeidsplassen ad page. Measured 2026-10-22 the public feed returned 404
for 10 of 15 relevant postings (ads it does not republish), so without this most Norway
ads would never get a language check. The page is on the rate-limited website host, so it
runs on its own small budget, slowly, and only when the search itself did not hit a 429."""
return _html_to_text(_nav_get(job["url"], pace_s))
class _RateLimited(RuntimeError):
pass
def _nav_get(url, pace_s):
time.sleep(pace_s)
req = urllib.request.Request(url, headers={"User-Agent": USER_AGENT})
try:
with urllib.request.urlopen(req, timeout=40, context=_ssl_context()) as resp:
return resp.read().decode("utf-8", "replace")
except urllib.error.HTTPError as e:
if e.code == 429:
raise _RateLimited(url) from e
raise
def _html_to_text(raw):
text = re.sub(r"<script.*?</script>|<style.*?</style>", " ", raw, flags=re.S | re.I)
return re.sub(r"\s+", " ", html.unescape(re.sub(r"<[^>]+>", " ", text))).strip()
def _nav_job(source):
"""Map one NAV search hit (_source) to a scout job dict, or None if not usable."""
if source.get("status") != "ACTIVE" or not source.get("uuid"):
return None
props = source.get("properties") or {}
cities = []
for place in source.get("locationList") or []:
city = (place.get("city") or place.get("municipal") or "").strip()
if city:
label = f"{city.title()}, Norway"
if label not in cities:
cities.append(label)
summary = ((source.get("generatedSearchMetadata") or {}).get("shortSummary") or "")
tags = " ".join(props.get("searchtagsai") or [])
employer = source.get("businessName") or (source.get("employer") or {}).get("name") or ""
due = props.get("applicationdue") or ""
return {
"id": source["uuid"],
"title": source.get("title") or "",
"employer": employer.strip(),
"location": " | ".join(cities) or "Norway",
"country": "NO",
"url": NAV_AD_URL.format(uuid=source["uuid"]),
"posted": (source.get("published") or "")[:10],
"deadline": due if re.match(r"\d{4}-\d{2}-\d{2}", due) else "",
"description": f"{summary} {tags}".strip(),
"work_language": props.get("workLanguage") or [],
}
def fetch_nav(args):
"""NAV register search: keyword queries, Norway outside Oslo, language-checked."""
warnings = args.setdefault("_run_warnings", [])
warnings.clear()
pace = args.get("pace_s", 3.0)
budget = args.get("search_budget", 14)
detail_budget = args.get("detail_budget", 25)
jobs, used = {}, 0
try:
for query in args["queries"]:
for page in range(args.get("pages_per_query", 1)):
if used >= budget:
break
url = NAV_SEARCH_URL + "?" + urllib.parse.urlencode(
{"q": query, "size": 100, "from": page * 100})
data = json.loads(_nav_get(url, pace))
used += 1
hits = (data.get("hits") or {}).get("hits") or []
for hit in hits:
job = _nav_job(hit.get("_source") or {})
if job and job["id"] not in jobs:
jobs[job["id"]] = job
if len(hits) < 100:
break
except _RateLimited:
warnings.append(f"NAV rate limit (HTTP 429) after {used} search request(s); this run "
f"is PARTIAL ({len(jobs)} postings). It usually clears within hours.")
# Text check: cached verdicts first, then fetch the full ad for relevant postings outside
# Oslo until the detail budget runs out. Irrelevant postings never cost a request.
try:
cache = json.loads(NAV_DETAIL_CACHE.read_text(encoding="utf-8"))
except (FileNotFoundError, ValueError):
cache = {}
fetched = pending = page_fetched = 0
# The text checks use the official feed host, so a rate limit on the search host does not
# block them; only a 429 or a token failure on the feed itself does. Ads missing from the
# feed fall back to the website page, which shares the search host's rate limit.
blocked, token = False, None
page_ok = not any("rate limit" in w for w in warnings)
page_budget = args.get("page_fallback_budget", 6)
for job in jobs.values():
cached = cache.get(job["id"])
if cached:
job["lang_check"] = cached
continue
relevant = score_job(job)[0] >= 2 and location_eligibility(
job["location"], policy="norway_no_oslo", country="NO")[0]
if relevant and not blocked and fetched < detail_budget:
try:
if token is None:
token = _nav_feed_token()
try:
text = _nav_ad_text(job["id"], token, args.get("feed_pace_s", 0.5))
except urllib.error.HTTPError as e:
if e.code != 404 or not page_ok or page_fetched >= page_budget:
raise
try:
text = _nav_page_text(job, args.get("page_pace_s", 8.0))
page_fetched += 1
except _RateLimited:
page_ok = False
warnings.append("NAV website rate limit during page fallback; ads the "
"public feed lacks stay provisional this run.")
raise urllib.error.URLError("page fallback rate-limited")
fetched += 1
verdict = classify_language_requirement(text, job["work_language"])
verdict["checked"] = datetime.now(timezone.utc).strftime("%Y-%m-%d")
cache[job["id"]] = verdict
job["lang_check"] = verdict
continue
except _RateLimited:
blocked = True
warnings.append("NAV feed rate limit hit during posting-text checks; remaining "
"postings use the work-language tag only.")
except RuntimeError as e:
blocked = True
warnings.append(f"NAV feed token unavailable ({e}); posting-text checks skipped "
f"this run.")
except urllib.error.HTTPError as e:
if e.code != 404:
warnings.append(f"NAV posting {job['id']}: text fetch failed ({e!r})")
# 404 = not in the public feed and no page budget left: counted as pending.
except (urllib.error.URLError, ValueError) as e:
if "page fallback rate-limited" not in str(e):
warnings.append(f"NAV posting {job['id']}: text fetch failed ({e!r})")
if relevant:
pending += 1
verdict = classify_language_requirement(job["description"], job["work_language"])
verdict["provisional"] = True
job["lang_check"] = verdict
if pending:
warnings.append(f"{pending} relevant NAV posting(s) have no full-text language check yet "
f"(checked {fetched} this run, {page_fetched} via the website fallback; "
f"budgets {detail_budget} feed / {page_budget} page); they are marked "
f"provisional and get checked on later runs as the cache fills.")
_atomic_write_json(NAV_DETAIL_CACHE, cache)
return list(jobs.values())
def fetch_radancy(args):
"""Radancy (TalentBrew) careers sites, e.g. careers.arm.com in front of iCIMS.
The /search-jobs/results endpoint returns JSON whose `results` field is the rendered
result-list HTML; each card has the job link, data-job-id and a `location` span. One
keyword search per term in args["keywords"]. LIMITATION: a multi-location role whose card
shows another city first (e.g. Shanghai) is dropped by the location policy even when
Trondheim is one of its locations - the card does not list them all.
"""
base = args["base_url"].rstrip("/")
jobs, seen_ids = [], set()
for keyword in args.get("keywords", [""]):
query = urllib.parse.urlencode({
"ActiveFacetID": 0, "CurrentPage": 1, "RecordsPerPage": 200, "Keywords": keyword,
"SearchType": 5, "SearchResultsModuleName": "Search Results",
"SearchFiltersModuleName": "Search Filters"})
data = http_get_json(f"{base}/search-jobs/results?{query}",
headers={"X-Requested-With": "XMLHttpRequest"})
results = data.get("results") or ""
for card in re.findall(r"<li[^>]*job-card.*?</li>", results, re.S):
link = re.search(r'href="(/job/[^"]+)"[^>]*data-job-id="(\d+)"[^>]*>(.*?)</a>', card, re.S)
if not link or link.group(2) in seen_ids:
continue
seen_ids.add(link.group(2))
loc = re.search(r'class="location"[^>]*>(.*?)</span>', card, re.S)
intro = re.search(r'class="job-card__intro"[^>]*>(.*?)</span>', card, re.S)
jobs.append({
"id": link.group(2),
"title": _html_to_text(link.group(3)),
"location": _html_to_text(loc.group(1)) if loc else "",
"url": base + link.group(1),
"description": _html_to_text(intro.group(1)) if intro else "",
})
return jobs
def fetch_algolia_jobs(args):
"""Careers sites that search an Algolia index in the browser (DNV: jobs.dnv.com).
The app id, search-only key and index name are public, embedded in the search page as
window.AG_ID / AG_KEY / AG_INDEX. They are read fresh on every run rather than hardcoded,
so a rotated key does not silently break the adapter. DNV's index held 237 jobs worldwide
on 2026-10-22, so one 1000-hit query covers the whole board.
"""
page = urllib.request.Request(args["page_url"], headers={
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/126.0"})
with urllib.request.urlopen(page, timeout=30, context=_ssl_context()) as resp:
raw = resp.read().decode("utf-8", "replace")
app = re.search(r'window\.AG_ID\s*=\s*"([^"]+)"', raw)
key = re.search(r'window\.AG_KEY\s*=\s*"([^"]+)"', raw)
index = re.search(r'window\.AG_INDEX\s*=\s*(\{[^;]+\})', raw)
if not (app and key and index):
raise RuntimeError("Algolia config (AG_ID/AG_KEY/AG_INDEX) not found on the search page")
index_name = json.loads(index.group(1)).get(args.get("index_key", "jobs"))
data = http_get_json(
f"https://{app.group(1)}-dsn.algolia.net/1/indexes/{index_name}/query",
headers={"X-Algolia-Application-Id": app.group(1), "X-Algolia-API-Key": key.group(1)},
data={"params": "query=&hitsPerPage=1000"}, method="POST")
origin = re.match(r"https?://[^/]+", args["page_url"]).group(0)
jobs = []
for hit in data.get("hits") or []:
countries = hit.get("country") or []
if isinstance(countries, str):
countries = re.findall(r"[A-Za-z][A-Za-z .'-]+", countries)
location = hit.get("display_location") or ""
if countries and not any(c.lower() in location.lower() for c in countries):
location = f"{location} | {', '.join(countries)}".strip(" |")
path = hit.get("jd_url") or ""
jobs.append({
"id": str(hit.get("objectID") or path),
"title": hit.get("title") or "",
"location": location,
"url": path if path.startswith("http") else origin + path,
"unit": hit.get("business_unit") or "",
})
return jobs
ADAPTERS = {
"nav": fetch_nav,
"radancy": fetch_radancy,
"algolia_jobs": fetch_algolia_jobs,
"ibm": fetch_ibm,
"workday": fetch_workday,
"ashby": fetch_ashby,
"greenhouse": fetch_greenhouse,
"pcsx": fetch_pcsx,
"smartrecruiters": fetch_smartrecruiters,
"rss": fetch_rss,
"teamtailor": fetch_teamtailor,
"getro": fetch_getro,
"amazon": fetch_amazon,
"phenom": fetch_phenom,
"onlyfy": fetch_onlyfy,
"lever": fetch_lever,
"taleo": fetch_taleo,
"json": fetch_json,
"prospective": fetch_prospective,
"sbb": fetch_sbb,
"bkw": fetch_bkw,
"playwright": fetch_playwright,
}
# An explicit onsite marker or a single-country restriction beats a stray pan-regional word.
_ONSITE = re.compile(r"\b(?:on-?site|in-?office|vor ort|presence required|100% office)\b", re.I)
_COUNTRY_ONLY = re.compile(
r"\b(?:only|nur|exclusively|must (?:be|reside) in|residents? of|based in)\b", re.I)
def _place_match(low, keywords):
"""Whole-word place matching.
Substring matching accepted "Bernau bei Berlin" as Bern, which put German roles into a
Swiss-only result set. Hyphens and slashes count as boundaries so "Bern/Zurich" still hits.
"""
for keyword in keywords:
if re.search(r"(?<![\w])" + re.escape(keyword) + r"(?![\w])", low):
return True
return False
COUNTRY_NAMES = {
"CH": ("switzerland", "schweiz", "suisse", "svizzera", "ch"),
"DK": ("denmark", "danmark", "dk"), "NO": ("norway", "norge", "no"),
"DE": ("germany", "deutschland", "de"), "AT": ("austria", "österreich", "at"),
"FR": ("france", "frankreich", "fr"), "IT": ("italy", "italien", "it"),
"GB": ("united kingdom", "great britain", "uk", "gb"),
"US": ("united states", "usa", "us"), "NL": ("netherlands", "niederlande", "nl"),
# Added 2026-10-22 for the EMEA-leak fix: countries that appear in remote allowlists.
# Full names only - two-letter codes ("es", "se", "in", "be") would match ordinary words.
"IE": ("ireland", "republic of ireland"), "ES": ("spain",), "SE": ("sweden",),
"FI": ("finland",), "PL": ("poland",), "PT": ("portugal",), "BE": ("belgium",),
"RS": ("serbia",), "CZ": ("czech republic", "czechia"), "IL": ("israel",),
"CA": ("canada",), "IN": ("india",), "RO": ("romania",), "GR": ("greece",),
"HU": ("hungary",), "EE": ("estonia",), "LT": ("lithuania",), "LV": ("latvia",),
"UA": ("ukraine",), "TR": ("turkey", "türkiye"), "AE": ("united arab emirates",),
"SG": ("singapore",), "AU": ("australia",), "JP": ("japan",), "BR": ("brazil",),
}
# A real pan-European remote scope, as opposed to an ATS region tag appended to a country
# list ("Germany (Remote) EMEA", "Remote, France; ...; Remote, United Kingdom EMEA").
_GENUINE_EUROPE_SCOPE = re.compile(
r"remote[\s,(\-–:]+(?:in\s+)?(?:europe|emea|the\s+eu)\b|(?:europe|emea)[\s,\-–]+remote"
r"|anywhere\s+in\s+(?:europe|emea|the\s+world)|europe-wide|eu-wide|pan-european"
r"|worldwide|work\s+from\s+anywhere|remote\s*[-–]\s*global", re.I)
def _segment_names_place(loc_text, code, town_keywords):
"""True if one location segment names this country, or one of its towns without naming
any other country. Multi-location strings ("PL-Poland-Remote | CH-Zurich-MSO") otherwise
lose the Swiss/Danish segment as soon as another country is recognised, while a single
segment like "Baden, Germany" must not count as Swiss Baden."""
for seg in re.split(r"[|;\n]", (loc_text or "").lower()):
others = {c for c, names in COUNTRY_NAMES.items() if c != code and
any(_place_match(seg, [n]) for n in names if len(n) > 2)}
own = [n for n in COUNTRY_NAMES.get(code, ()) if len(n) > 2]
if _place_match(seg, own) or (_place_match(seg, town_keywords) and not others):
return True
return False
def _has_non_oslo_norway_place(loc_text):
"""True unless every location segment that can be placed is in the Oslo area.
Segments are split on | ; / and newlines, so "Trondheim, Norway | Oslo, Norway" keeps the
role and "Oslo, Norway, 0010 Oslo" drops it. A bare "Norway" or an unknown town counts as
outside Oslo: we cannot tell, and dropping it would hide real roles silently.
"""
# Only segments that name a Norwegian place count. Workday location strings carry the job
# slug ("/job/Oslo-Norway/Senior-Data-Engineer_JR123"), and treating the slug as a place
# would wrongly keep every Oslo-only Workday role.
segments = [s.strip().lower() for s in re.split(r"[|;/\n]", loc_text or "") if s.strip()]
placed = False
for seg in segments:
if not _place_match(seg, NO_LOCATION_KEYWORDS + OSLO_AREA_PLACES):
continue
if _place_match(seg, OSLO_AREA_PLACES):
placed = True
continue
return True
return not placed
def location_eligibility(loc_text, policy="default", country=None):
"""Decide whether a posting's location is in-scope.
Returns (eligible, in_ch, is_remote, is_relocation).
Default policy (most companies):
- CH onsite/hybrid
- Denmark onsite/reloc
- Europe/EMEA/global remote scope (work-from CH or DK is realistic)
- CH-remote (Switzerland + remote wording)
- Multi-country allowlists WITHOUT CH/DK/Europe/global scope are NOT eligible
(e.g. Kraken "UK | Brazil | Ireland | ..." with no Switzerland/Europe)
Overrides (args["_location_policy"]):
- norway_only — Norway only, Oslo included
- norway_no_oslo — Norway with at least one location outside the Oslo area (Equinor,
Telenor, NAV and the other Norway boards; user preference)
- dk_no — NATO-style: Denmark + Norway only (no CH)
"""
if not loc_text:
return False, False, False, False
low = loc_text.lower()
# An adapter's country code is stronger than an ambiguous town name. In free text,
# recognize country names; short codes count only as comma/slash-separated fields.
countries = {country.upper()} if country else {
code for code, names in COUNTRY_NAMES.items() if any(
_place_match(low, [name]) if len(name) > 2 else
bool(re.search(r"(?:^|[,/])\s*" + name + r"\s*(?:$|[,/])", low)) for name in names)}
explicit_country = bool(countries)
excluded = set()
for code in countries:
for name in COUNTRY_NAMES.get(code, (code.lower(),)):
token = r"\b" + re.escape(name) + r"\b"
if re.search(r"(?:excluding|except|not in|außer|ausser|nicht in)\s+" + token, low) or re.search(token + r"\s+(?:excluded|not eligible|ausgeschlossen)", low):
excluded.add(code)
countries -= excluded
# With an adapter-supplied country, trust it. With countries parsed from free text, also
# check segment by segment so a Swiss/Danish segment survives beside another country.
if country:
in_ch, in_dk = "CH" in countries, "DK" in countries
elif explicit_country:
in_ch = "CH" in countries or _segment_names_place(loc_text, "CH", CH_LOCATION_KEYWORDS)
in_dk = "DK" in countries or _segment_names_place(loc_text, "DK", DK_LOCATION_KEYWORDS)
else:
in_ch = _place_match(low, CH_LOCATION_KEYWORDS)
in_dk = _place_match(low, DK_LOCATION_KEYWORDS)
in_no = "NO" in countries if explicit_country else _place_match(low, NO_LOCATION_KEYWORDS)
is_us_only = any(p in low for p in US_ONLY_PATTERNS) and not in_ch
has_remote = bool(re.search(r"\bremote\b", low)) or "work from anywhere" in low
europe_scope = any(k in low for k in EUROPE_SCOPE_KEYWORDS)
global_scope = any(k in low for k in GLOBAL_SCOPE_KEYWORDS) and not is_us_only
# Pan-regional scope counts even without the word "remote" (e.g. QuantCo "Europe"),
# but NOT when the posting also says onsite or restricts residence to one country
# (F12: "Europe" beside "London / Onsite" used to read as remote-eligible).
restricted = "CH" in excluded or bool(_ONSITE.search(low)) or (
bool(_COUNTRY_ONLY.search(low)) and not (in_ch or in_dk))
# EMEA leak (found 2026-10-20, fixed 2026-10-22): a country allowlist without CH/DK plus a
# trailing region tag is NOT pan-European scope. Seven such rows (GitLab FR/DE/NL/ES/UK,
# Grafana DE/IE/ES/SE/UK, Fivetran Serbia/Ireland) were passing as Europe-remote.
fenced = (explicit_country and not (in_ch or in_dk)
and not _GENUINE_EUROPE_SCOPE.search(low))
is_europe_or_global = ((europe_scope or global_scope) and not is_us_only and not restricted
and not fenced)
is_ch_remote = in_ch and has_remote
if policy == "norway_only":
eligible = in_no and not is_us_only
return eligible, False, False, eligible
if policy == "norway_no_oslo":
eligible = in_no and not is_us_only and _has_non_oslo_norway_place(loc_text)
return eligible, False, False, eligible
if policy == "dk_no":
# NATO etc.: Denmark + Norway only — CH onsite is out (no NATO seat in CH).
eligible = (in_dk or in_no) and not is_us_only
return eligible, False, False, eligible
# default
is_relocation = in_dk
is_remote = is_europe_or_global or is_ch_remote
eligible = (in_ch or in_dk or is_europe_or_global) and not (is_us_only and not in_ch)
return eligible, in_ch, is_remote, is_relocation
@lru_cache(maxsize=512)
def _kw_pattern(kw):
"""Word-boundary regex for a keyword. Plain substring matching produced false hits
('rag' inside 'sto[rag]e'/'tet[rag]on', 'intern' inside 'inte[rnal]'); we instead
require the keyword not be flanked by alphanumerics. Keywords that begin/end on a
non-word char (c#, .net, c++) skip that side's guard so they still match."""
esc = re.escape(kw.strip())
left = r"(?<![a-z0-9])" if kw.strip()[:1].isalnum() else ""
right = r"(?![a-z0-9])" if kw.strip()[-1:].isalnum() else ""
return re.compile(left + esc + right)
def _kw_in(kw, text):
return bool(_kw_pattern(kw).search(text))
def score_job(job, title_only=False):
"""Keyword RELEVANCE for search ranking - explicitly not a fit score (F14).
It measures how many profile keywords a posting echoes. It does not know his evidence,
the hard requirements, the level, the language or the commute, and a higher number must
never be read as a better application. Compare: "Senior AI Engineer, Generative AI / LLM
/ RAG" scores 12 against "Senior Data Engineer" at 3, while the second is much closer to
what he can actually evidence. German titles score near zero regardless of fit, which is
why pre-filtered boards carry a _score_floor.
"""
# Title carries the real signal; the JD body is full of company boilerplate (every
# Kraken post mentions crypto/blockchain/trading, every cloud post mentions python).
# So title matches score at full weight and body-only matches at half (min 1) — enough
# to surface a role without letting boilerplate inflate it. Negatives count fully
# wherever they appear (a disqualifier in the body still disqualifies). Title-filtered
# boards pass title_only=True and skip body scoring entirely.
title = (job.get("title") or "").lower()
desc = "" if title_only else (job.get("description") or "").lower()
score, pos, neg = 0, [], []
for kw, w in POSITIVE_KEYWORDS.items():
if _kw_in(kw, title):
score += w
pos.append(kw)
elif desc and _kw_in(kw, desc):
score += max(1, w // 2)
pos.append(kw)
for kw, w in NEGATIVE_KEYWORDS.items():
if _kw_in(kw, title) or (desc and _kw_in(kw, desc)):
score += w
neg.append(kw)
return score, pos, neg
def _atomic_write_json(path, data):
"""Write JSON via temp + os.replace so a crash mid-write cannot truncate state files."""
path.parent.mkdir(parents=True, exist_ok=True)
# Unique temp name (F16): a fixed ".tmp" let two concurrent writers share one temp file
# and os.replace the wrong content into place.
descriptor, tmp_name = tempfile.mkstemp(dir=path.parent, prefix=path.name + ".", suffix=".tmp")
tmp = Path(tmp_name)
try:
with os.fdopen(descriptor, "w", encoding="utf-8") as stream:
stream.write(json.dumps(data, indent=2, ensure_ascii=False))
os.replace(tmp, path)
except BaseException:
tmp.unlink(missing_ok=True)
raise
def load_seen():
if STATE_FILE.exists():
return json.loads(STATE_FILE.read_text(encoding="utf-8"))
return {}
def save_seen(seen):
_atomic_write_json(STATE_FILE, seen)
# Board-specific req-ID patterns, tried in order against the URL path+query. Every key is
# scoped by host (see _decision_key), so an ID only ever has to be unique within one board
# and cross-board collisions are impossible by construction.
_HOST_ALIASES = {"apply.careers.microsoft.com": "jobs.careers.microsoft.com",
# BIS Workday tenant (2026-10-22) keeps the bis.org#jr… decision keys.
"bis.wd3.myworkdayjobs.com": "bis.org"}
_JOB_ID_PATTERNS = [
# Amazon: the slug is truncated to a fixed width and that width has changed between runs,
# so the same req yields different URLs. This is the drift that motivated the whole helper.
(r"amazon\.jobs$", r"/jobs/(\d+)"),
# New-form links carry the display id as ?jobId= (the /job/<n> path holds the internal
# pcsx id), so this must come before the /job/ pattern. Old-form links fall through to it.
(r"careers\.microsoft\.com$", r"[?&]jobId=(\d+)"),
(r"careers\.microsoft\.com$", r"/job/(\d+)"),
(r"google\.com$", r"/results/(\d+)-"),
# Workday: trailing _REQID on the job slug (Roche 202608-120791-1, Novartis REQ-…, NVIDIA JR…).
# Also collapses the /apply suffix some rows carry and others don't.
(r"myworkdayjobs\.com$", r"_([A-Za-z]*[\d-]+\d)(?:/apply)?/?$"),
# BIS migrated its vacancy URLs from /careers/vacancies/jrNNNNNN.htm to /vacancy/jrNNNNNN,
# orphaning every decision recorded under the old scheme — including the `applied` row for
# jr100429. The jr-id is stable across both. (Verified 2026-09-04.)
(r"bis\.org$", r"/(jr\d+)"),
# BIS Workday URLs end "..._JR100498" or "..._JR100499-1"; same jr-id, lower-cased in
# _decision_key so it matches the bis.org keys.
(r"bis\.org$", r"(?i)_(jr\d+)"),
(r"smartrecruiters\.com$", r"/(\d{6,})"),
(r"greenhouse\.io$", r"/jobs/(\d+)"),
(r"careers\.cisco\.com$", r"/job/(\d+)"),
# PostFinance publishes one req per locale (…/74421-fr_FR, …/74372-de_DE) — same job.
(r"jobs\.postfinance\.ch$", r"/(\d+)-[a-z]{2}_[A-Z]{2}/?$"),
# Apple: same req appears with ?team=… and with a /locationPicker suffix.
(r"jobs\.apple\.com$", r"/details/([\d-]+)"),
(r"successfactors\.eu$", r"[?&]jobId=(\d+)"),
(r"taleo\.net$", r"[?&]job=(\d+)"),
(r"bis\.org$", r"/vacancies/(jr\d+)"),
(r"metacareers\.com$", r"/job_details/(\d+)"),
(r"finn\.no$", r"[?&]finnkode=(\d+)"),
(r"linkedin\.com$", r"/jobs/view/.*-(\d+)/?$"),
(r"careers\.roche\.com$", r"/job/([\w-]+)/"),
# Greenhouse-backed careers pages that carry the req in a query param (Databricks,
# Fivetran, Datadog, Elastic — Elastic repeats gh_jid twice; the first match wins).
(None, r"[?&]gh_jid=(\d+)"),
# Teamtailor (Axpo, Telenor) — /jobs/<id>-<slug>.
(None, r"/jobs/(\d+)-"),
# Ashby / Lever / Recruitee-style boards (RUAG, BKW, BFH, SBB) — trailing UUID.
(None, r"/([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-"
r"[0-9a-f]{4}-[0-9a-f]{12})/?$"),
]
def _decision_key(url):
"""Stable per-job identity for the decision log, resilient to URL drift.
Boards rewrite their own URLs: Amazon truncates the slug to a width that has changed
between runs, PostFinance publishes one URL per locale, Apple appends ?team= or
/locationPicker, Workday adds /apply. Keying decisions on the raw URL therefore loses
them — an already-skipped role silently resurfaces as undecided (this is what happened
to the AWS Principal Delivery Consultant req between the 2026-08-18 and 08-21 runs).
Returns "<host>#<req-id>" when a board pattern matches, else "<host>#<normalized path>".
Host-scoping keeps two boards that use the same numbering scheme from colliding, so a
wrong match can never mark a genuinely undecided role as decided at a *different* company.
"""
try:
parts = urllib.parse.urlsplit(url.strip())
except ValueError:
return url.strip()
host = (parts.netloc or "").lower().split(":")[0]
if host.startswith("www."):
host = host[4:]
# One board, two hostnames: keep decisions recorded under the old Microsoft link form
# linked to rows carrying the new one (2026-10-22 link fix).
host = _HOST_ALIASES.get(host, host)
target = parts.path + (f"?{parts.query}" if parts.query else "")
for host_pat, id_pat in _JOB_ID_PATTERNS:
if host_pat and not re.search(host_pat, host):
continue
m = re.search(id_pat, target)
if m:
ident = m.group(1).lower() if host == "bis.org" else m.group(1)
return f"{host}#{ident}"
# No known ID shape: fall back to the path with cosmetic variation stripped. The QUERY
# is kept (F13): dropping it made /vacancy?job=1 and /vacancy?job=2 the same key, so one
# decision silently covered two different requisitions.
path = parts.path.rstrip("/")
if path.endswith("/apply"):
path = path[: -len("/apply")]
query = urllib.parse.parse_qs(parts.query)
# Only identifying parameters; tracking noise must not fragment the key.
keep = {k: v for k, v in query.items()
if k.lower() in {"job", "jobid", "id", "req", "reqid", "requisition", "gh_jid", "vacancy"}}
suffix = ""
if keep:
suffix = "?" + "&".join(f"{k}={sorted(v)[0]}" for k, v in sorted(keep.items()))
return f"{host}#{path.lower()}{suffix}"
def load_decisions():
"""Decision log keyed by job URL: {url: {company, title, decision, note, date}}.
Decisions persist across runs so we don't re-evaluate roles we've already judged
(shortlist / skip / applied / paused / rejected — free-text, not enforced).
The file stays URL-keyed — it is hand-edited and the URL is the useful thing to read.
Lookups go through index_decisions() instead, which is drift-proof."""
if DECISIONS_FILE.exists():
return json.loads(DECISIONS_FILE.read_text(encoding="utf-8"))
return {}
def index_decisions(decisions):
"""Build {stable key -> entry} for lookup. On collision the newest decision wins."""
idx = {}
for url, entry in decisions.items():
key = _decision_key(url)
prev = idx.get(key)
if prev is None or str(entry.get("date", "")) >= str(prev.get("date", "")):
idx[key] = entry
return idx
_RUAG_SLUG = re.compile(r"^https://jobs\.ruag\.ch/offene-stellen/([^/]+)/[0-9a-f-]{36}/?$")
def _ruag_slug(url):
m = _RUAG_SLUG.match(url.strip())
return m.group(1) if m else None
def find_decision_url(decisions, url):
"""Existing log URL for the same job, if one is already recorded under a drifted URL."""
key = _decision_key(url)
for existing in decisions:
if _decision_key(existing) == key:
return existing
# RUAG regenerates the UUID in /offene-stellen/<slug>/<uuid> when a req is reposted, so the
# exact key misses and an already-judged role resurfaces as undecided — this is what happened
# to every RUAG rule-out between the 2026-08-31 and 09-04 runs. The slug survives the repost,
# but it is NOT unique: RUAG had two live reqs sharing
# `flugzeugmechanikerin-technician-mechanical-f-a-18` on 2026-09-04. So the slug is only a
# fallback, and only when it is unambiguous in the log — never let one skip decide two reqs.
slug = _ruag_slug(url)
if slug:
matches = [e for e in decisions if _ruag_slug(e) == slug]
if len(matches) == 1:
return matches[0]
return None
@lru_cache(maxsize=1)
def _raw_decisions():
"""Raw decision log for the report path's slug fallback (cached per run)."""
return load_decisions()
def lookup_decision(dec_index, decisions, url):
"""Resolve a decision for a report row, using the SAME identity rules as the write path.
The 2026-09-04 fix taught `find_decision_url` (used by --decide) that RUAG regenerates the
UUID on every repost, but the REPORT path kept doing a bare `_decision_key` lookup. Result:
a reposted RUAG requisition came back as undecided although the decision was in the log -
which is exactly what happened to `Lead-Architekt Network- und Richtfunk` on 2026-09-10,
days after 63 RUAG rule-outs had been written.
A slug match is reported as RELATED, not as the same requisition: a repost can differ
materially, so an old rule-out must not be applied to it automatically.
"""
hit = dec_index.get(_decision_key(url))
if hit is not None:
return hit
existing = find_decision_url(decisions, url)
if existing is None:
return None
related = dict(decisions[existing])
related["matched_by"] = "slug"
related["matched_url"] = existing
return related
def is_same_decision(decision):
return decision is not None and decision.get("matched_by") != "slug"
def save_decisions(decisions):
_atomic_write_json(DECISIONS_FILE, decisions)
def load_last_scrape():
"""Per-company last successful scraped counts — used to flag sudden 0-job deaths."""
if LAST_SCRAPE_FILE.exists():
try:
return json.loads(LAST_SCRAPE_FILE.read_text(encoding="utf-8"))
except (json.JSONDecodeError, OSError):
return {}
return {}
def save_last_scrape(data):
_atomic_write_json(LAST_SCRAPE_FILE, data)
def _parse_posted(s):
"""Best-effort parse of an adapter's `posted` field into a date, across the mix of
formats the boards use (ISO 8601 incl. trailing Z, YYYY-MM-DD, DD.MM.YYYY). Returns None
for unparseable values (e.g. Workday's relative "Posted 5 Days Ago", or empty)."""
if not s or not isinstance(s, str):
return None
s = s.strip()
try:
return datetime.fromisoformat(s.replace("Z", "+00:00")).date()
except ValueError:
pass
for fmt in ("%Y-%m-%d", "%d.%m.%Y", "%Y/%m/%d", "%d/%m/%Y", "%d-%b-%Y", "%d-%B-%Y"):
try:
# %b/%B need full token (e.g. 09-Aug-2026), not a fixed 10-char slice
token = s.split(",")[0].strip()
return datetime.strptime(token if "%b" in fmt or "%B" in fmt else s[:10], fmt).date()
except ValueError:
pass
m = re.search(r"\d{4}-\d{2}-\d{2}", s)
if m:
try:
return datetime.strptime(m.group(0), "%Y-%m-%d").date()
except ValueError:
pass
return None
# --- Employer reputation (user requirement 2026-10-22) -----------------------------------
# "The company should have a good reputation for working there." Glassdoor blocks direct
# access (HTTP 403), so ratings are LOOKUPS recorded in state/employer_reputation.json with
# date and source, not live data. Every report row shows the verdict; an employer nobody has
# looked up is UNCHECKED and must be looked up before a role there is recommended.
REPUTATION_FILE = ROOT / "state" / "employer_reputation.json"
REPUTATION_STALE_DAYS = 365
def load_reputation():
try:
return json.loads(REPUTATION_FILE.read_text(encoding="utf-8"))
except (FileNotFoundError, ValueError):
return {"thresholds": {}, "employers": {}}
def _norm_name(name):
name = re.sub(r"\b(?:as|asa|ab|ag|gmbh|inc|ltd|plc|sa|nv|bv)\b\.?", " ", (name or "").lower())
return re.sub(r"[^a-z0-9æøå]+", " ", name).strip()
def reputation_for(name, registry):
"""Registry entry plus computed verdict for an employer name, or an UNCHECKED stub."""
norm = _norm_name(name)
thresholds = registry.get("thresholds") or {}
def hit(alias):
alias = _norm_name(alias)
return bool(alias) and bool(
re.search(r"(?<![a-z0-9])" + re.escape(alias) + r"(?![a-z0-9])", norm))
for key, entry in (registry.get("employers") or {}).items():
# Same-name strangers (Kongsberg Automotive is not KONGSBERG) are listed explicitly.
if any(hit(a) for a in entry.get("not_aliases") or []):
continue
if any(hit(a) for a in [key] + list(entry.get("aliases") or [])):
return dict(entry, key=key, verdict=reputation_verdict(entry, thresholds))
return {"key": None, "verdict": "unchecked"}
def reputation_verdict(entry, thresholds):
"""good / ok / avoid / excluded / unchecked / stale, from the user-approved thresholds."""
if entry.get("verdict_override"):
return entry["verdict_override"]
checked = entry.get("checked")
try:
if checked and (datetime.now() - datetime.strptime(checked, "%Y-%m-%d")).days > REPUTATION_STALE_DAYS:
return "stale"
except ValueError:
pass
# Glassdoor and kununu are both primary sources; the one with more reviews decides. For
# Swiss employers kununu is often far better populated (user tip, 2026-10-22: SBB 1,808
# kununu reviews vs no usable Glassdoor figure; RUAG 523 vs a thin Indeed 3.1).
primaries = [s for s in (entry.get("glassdoor"), entry.get("kununu"))
if s and s.get("rating") is not None]
primary = max(primaries, key=lambda s: s.get("reviews") or 0) if primaries else {}
rating = primary.get("rating")
reviews = primary.get("reviews") or 0
good_min = thresholds.get("good_min_rating", 3.8)
min_reviews = thresholds.get("good_min_reviews", 50)
ok_min = thresholds.get("ok_min_rating", 3.5)
# A second source counts only if it is solid: when it carries a score (Indeed, kununu),
# a score below the ok line makes the verdict avoid, not ok (RUAG: Indeed CH 3.1).
secondary_rating = entry.get("secondary_rating")
if rating is None:
if secondary_rating is not None:
return "ok" if secondary_rating >= ok_min else "avoid"
return "ok" if entry.get("secondary") else "unchecked"
if rating >= good_min and reviews >= min_reviews:
return "good"
if rating >= good_min and secondary_rating is not None and secondary_rating < ok_min:
return "avoid"
if rating >= good_min and entry.get("secondary"):
return "ok"
if rating >= ok_min:
return "ok"
return "avoid"
def _reputation_line(rep):
verdict = rep.get("verdict", "unchecked").upper()
if verdict == "UNCHECKED":
return "UNCHECKED - look up before recommending (state/employer_reputation.json)"
bits = []
for label, src in (("Glassdoor", rep.get("glassdoor") or {}), ("kununu", rep.get("kununu") or {})):
if src.get("rating") is not None:
reviews = f", {src['reviews']:,} reviews" if src.get("reviews") else ""
scope = f", {src['scope']}" if src.get("scope") else ""
bits.append(f"{label} {src['rating']}/5{reviews}{scope}")
if rep.get("secondary"):
bits.append(rep["secondary"])
if rep.get("checked"):
bits.append(f"checked {rep['checked']}")
note = f" - {rep['note']}" if rep.get("note") else ""
return f"{verdict} ({'; '.join(bits)}){note}"
def _norm_title(title):
return re.sub(r"[^a-z0-9æøå]+", " ", (title or "").lower()).strip()
def drop_nav_duplicates(results, companies):
"""Drop NAV rows that repeat a posting a dedicated board already returned.
Matching is employer alias (args["nav_aliases"]) plus normalized title. A NAV row for an
employer with a dedicated board but a different title is kept: it may be a role the board
does not list. Returns (kept_rows, dropped_count).
"""
alias_by_display = {display: [_norm_name(a) for a in (args.get("nav_aliases") or [])]
for _, display, _, args in companies}
dedicated = {}
for row in results:
if row.get("source"):
continue
for alias in alias_by_display.get(row["company"], []):
dedicated.setdefault(alias, set()).add(_norm_title(row["title"]))
kept, dropped = [], 0
for row in results:
if row.get("source"):
employer = _norm_name(row["company"])
if any(re.search(r"(?<![a-z0-9])" + re.escape(alias) + r"(?![a-z0-9])", employer)
and _norm_title(row["title"]) in titles
for alias, titles in dedicated.items()):
dropped += 1
continue
kept.append(row)
return kept, dropped
REGION_ORDER = ["Switzerland", "Norway", "Europe-remote / Denmark"]
def _company_region(args):
"""Search region a board belongs to, for --region runs: 'no' or 'ch'."""
policy = args.get("_location_policy", "default")
return "no" if policy.startswith("norway") or policy == "dk_no" else "ch"
def _lang_gated(row):
lang = row.get("lang") or {}
return lang.get("lang") == "norwegian_required" or bool(lang.get("citizenship_required"))
def _lang_line(lang):
label = {"english_ok": "ENGLISH OK", "norwegian_required": "NORWEGIAN REQUIRED",
"unclear": "UNCLEAR"}.get(lang.get("lang"), "UNCLEAR")
prov = " (provisional: tag/summary only)" if lang.get("provisional") else ""
evidence = f" - \"{lang['evidence'][:160]}\"" if lang.get("evidence") else ""
return f"{label}{prov}{evidence}"
def write_stats_table(stats, total_secs):
"""Render the per-company scan stats as a markdown table (+ a totals row)."""
out = ["## Scan stats\n",
"| Company | Scraped | CH/Remote | Match ≥2 | Newest posting | Time (s) |",
"|---|--:|--:|--:|:--|--:|"]
t_scraped = t_elig = t_match = 0
newest_all = None
for s in stats:
name = s["company"] + (" ⚠️" if s.get("error") else "")
newest = s["newest"].isoformat() if s["newest"] else "—"
out.append(f"| {name} | {s['scraped']:,} | {s['eligible']:,} | "
f"{s['match']:,} | {newest} | {s['secs']:.1f} |")
t_scraped += s["scraped"]; t_elig += s["eligible"]; t_match += s["match"]
if s["newest"] and (newest_all is None or s["newest"] > newest_all):
newest_all = s["newest"]
out.append(f"| **Total ({len(stats)})** | **{t_scraped:,}** | **{t_elig:,}** | "
f"**{t_match:,}** | **{newest_all.isoformat() if newest_all else '—'}** | "
f"**{total_secs:.1f}** |")
out.append("")
return out
def write_report(path, results, errors, new_only, include_weak, stats=None, total_secs=0.0,
decisions=None, hide_decided=False):
# `decisions` here is the stable-key index from index_decisions(), not the raw
# URL-keyed log — job URLs drift between runs, the keys do not.
decisions = decisions or {}
today = datetime.now().strftime("%Y-%m-%d")
n_new = sum(1 for r in results if r["is_new"])
n_match = sum(1 for r in results if r["score"] >= 2)
lines = [
f"# Job scout report {today}{' (new only)' if new_only else ''}\n",
f"Automated coverage: **{len(COMPANIES)}** companies. Manual checks: **{len(MANUAL_CHECK)}**.",
f"Eligible (CH/remote): **{len(results)}** · interest matches (score ≥ 2): "
f"**{n_match}** · **{n_new}** new since last run\n",
]
if stats:
lines += write_stats_table(stats, total_secs)
if errors:
lines.append("## Errors\n")
for company, err in errors:
lines.append(f"- **{company}**: {err}")
lines.append("")
# Excluded employers (user: never Vizrt again) are hidden; Norway roles that fail the
# language or citizenship check go to a compact list at the end instead of the buckets,
# so a misread requirement sentence can still be spotted and overridden.
excluded = [r for r in results if (r.get("reputation") or {}).get("verdict") == "excluded"]
gated = [r for r in results if r not in excluded and _lang_gated(r)]
hidden_ids = {id(r) for r in excluded + gated}
visible = [r for r in results if id(r) not in hidden_ids]
if excluded:
names = sorted({r["company"] for r in excluded})
lines.append(f"_{len(excluded)} role(s) hidden from excluded employers: {', '.join(names)}._")
n_unchecked = sum(1 for r in visible if r["score"] >= 2 and
(r.get("reputation") or {}).get("verdict") in (None, "unchecked", "stale"))
if n_unchecked:
lines.append(f"_{n_unchecked} shown role(s) are at employers with no current reputation "
f"check - look them up before recommending._")
weak_all = [r for r in visible if r["score"] < 2]
if not include_weak and weak_all:
lines.append(f"\n_Hiding {len(weak_all)} weak/noise roles (score < 2). Use --include-weak to show._")
n_decided = sum(1 for r in results if is_same_decision(
lookup_decision(decisions, _raw_decisions(), r["url"])))
if n_decided:
shown = "hidden" if hide_decided else "tagged inline"
lines.append(f"_{n_decided} role(s) already in the decision log ({shown}; "
f"see state/decisions.json)._")
def render_row(r):
d = lookup_decision(decisions, _raw_decisions(), r["url"])
new_tag = " [NEW]" if r["is_new"] else ""
decided_tag = f" — 🗂 {d['decision'].upper()}" if d else ""
loc_tag = ("CH" if r["in_ch"] else
"Remote" if r["remote"] else
"Relocation" if r.get("relocation") else "?")
out = [f"### [{r['score']}] {r['company']} - {r['title']}{new_tag}{decided_tag}",
f"- Location: {r['location']} *({loc_tag})*"]
if r.get("source"):
out.append(f"- Source: {r['source']}")
if r.get("lang"):
out.append(f"- Language: {_lang_line(r['lang'])}")
if r["lang"].get("clearance"):
out.append("- Security clearance mentioned (not a gate; runs after an offer)")
if r.get("reputation"):
out.append(f"- Reputation: {_reputation_line(r['reputation'])}")
if r.get("posted"):
out.append(f"- Posted: {r['posted']}")
if r.get("deadline"):
out.append(f"- **Bewerbungsfrist: {r['deadline']}**")
if r.get("updated_at"):
out.append(f"- Updated: {r['updated_at']}")
if r.get("unit"):
out.append(f"- Einheit: {r['unit']}")
if r.get("compensation"):
out.append(f"- Vergütung: {r['compensation']}")
if r.get("contact"):
# Channel is the rationed variable: a published, reachable name means a cold
# submit is a choice rather than the only option.
out.append(f"- **Ansprechperson: {r['contact']}**")
out.append(f"- URL: {r['url']}")
if d:
note = f" — {d['note']}" if d.get("note") else ""
if d.get("matched_by") == "slug":
out.append(
f"- 🗂 RELATED decision (same slug, **new requisition id** — a repost can "
f"differ, so re-read before reusing): **{d['decision']}**{note} "
f"({d.get('date','')}) · matched `{d.get('matched_url','')}`")
else:
out.append(f"- 🗂 Decision: **{d['decision']}**{note} ({d.get('date','')})")
if r["pos"]:
out.append(f"- Positive: {', '.join(r['pos'])}")
if r["neg"]:
out.append(f"- Negative: {', '.join(r['neg'])}")
out.append("")
return out
# User preference 2026-10-22: Norway and Switzerland are separate searches and are always
# listed separately. Europe-remote and Denmark roles get their own section so they are
# never mixed into either.
for region in REGION_ORDER:
region_rows = [r for r in visible if r.get("region") == region]
region_gated = [r for r in gated if r.get("region") == region]
if not region_rows and not region_gated:
continue
n_shown = sum(1 for r in region_rows if r["score"] >= 2 or include_weak)
lines.append(f"\n# {region} - {n_shown} role(s) shown, {len(region_rows)} eligible\n")
# F14: this number is SEARCH RELEVANCE - how many profile keywords the posting
# echoes. It is not a fit assessment and must never be presented as one: "Senior AI
# Engineer" with GenAI/LLM/RAG scores 12 while "Senior Data Engineer" scores 3,
# although the second sits far closer to his actual evidence. Fit is decided on the
# full JD.
buckets = [("High keyword relevance (score >= 6) - NOT a fit judgement",
[r for r in region_rows if r["score"] >= 6]),
("Medium keyword relevance (score 2-5)",
[r for r in region_rows if 2 <= r["score"] < 6])]
if include_weak:
buckets.append(("Low keyword relevance (score < 2) - German/Norwegian titles land here",
[r for r in region_rows if r["score"] < 2]))
for bucket_name, bucket in buckets:
shown = [r for r in bucket
if not (hide_decided and
is_same_decision(lookup_decision(decisions, _raw_decisions(), r["url"])))]
if not shown:
continue
lines.append(f"\n## {region}: {bucket_name} - {len(shown)} role(s)\n")
for r in shown:
lines.extend(render_row(r))
if region_gated:
relevant_gated = [r for r in region_gated if r["score"] >= 2 or include_weak]
lines.append(f"\n## {region}: filtered out by language/citizenship - "
f"{len(region_gated)} ({len(relevant_gated)} relevant shown)\n")
lines.append("Verdicts come from the posting's requirement sentence. If one is "
"misread, the role is still in the JSON dump.\n")
for r in relevant_gated:
lang = r.get("lang") or {}
why = ("Norwegian citizenship required: " + lang.get("citizenship_evidence", "")
if lang.get("citizenship_required")
else "Norwegian required: " + lang.get("evidence", ""))
lines.append(f"- [{r['score']}] {r['company']} - {r['title']} ({r['location']}) - "
f"{why[:180]} - {r['url']}")
lines.append("")
if MANUAL_CHECK:
lines.append("\n## Manual check (companies without scrapable APIs)\n")
lines.append("These use Cloudflare-protected sites, custom GraphQL APIs, or JS-rendered SPAs.")
lines.append("Open each link, scan for new postings since your last quarterly review:\n")
for name, note, url in MANUAL_CHECK:
lines.append(f"- [ ] **{name}** — {note}: <{url}>")
lines.append("")
path.write_text("\n".join(lines), encoding="utf-8")
def _fetch_company(cid, display, adapter, args):
"""Fetch one company; returns (cid, display, adapter, args, jobs|None, error|None, secs)."""
t0 = time.perf_counter()
try:
jobs = ADAPTERS[adapter](args)
return cid, display, adapter, args, jobs, None, time.perf_counter() - t0
except Exception as e:
kind = "unexpected" if not isinstance(
e, (urllib.error.URLError, urllib.error.HTTPError, ValueError, RuntimeError)
) else "error"
msg = f"{kind}: {e!r}" if kind == "unexpected" else repr(e)
return cid, display, adapter, args, None, msg, time.perf_counter() - t0
def _requisition_key(cid, job):
"""Stable per-requisition identity within one employer.
Prefers a board requisition id parsed out of the URL (the same patterns the decision log
uses), then the adapter's own id, then the URL. Employer-scoped so two boards that number
requisitions the same way can never collide.
"""
url = (job.get("url") or "").strip()
if url:
for host_pat, id_pat in _JOB_ID_PATTERNS:
try:
parts = urllib.parse.urlsplit(url)
except ValueError:
break
host = (parts.netloc or "").lower()
if host_pat and not re.search(host_pat, host):
continue
match = re.search(id_pat, parts.path + (f"?{parts.query}" if parts.query else ""))
if match:
return f"{cid}#{match.group(1)}"
identifier = job.get("id")
return f"{cid}#{identifier}" if identifier else f"{cid}#{url or job.get('title', '')}"
def _process_company(cid, display, args, jobs, seen, today, last_scrape):
"""Filter, score, and update seen for one company's job list.
Returns (results, stat_dict, warnings) where warnings is a list of (display, msg).
"""
warnings = []
req_seen: dict[str, dict] = {}
title_collisions: dict[str, int] = {}
scraped = len(jobs)
prev = (last_scrape.get(cid) or {}).get("scraped", 0)
if scraped == 0:
if prev and prev > 0:
warnings.append((
display,
f"0 jobs returned (was {prev} last run — possible dead board/slug/selectors)",
))
else:
warnings.append((
display,
"0 jobs returned (verify board slug/selectors if this is unexpected)",
))
# Inclusion allowlist (fails closed — only for boards too large to score in full).
title_filter = args.get("_title_filter")
if title_filter:
jobs = [j for j in jobs
if any(_kw_in(k, (j.get("title") or "").lower()) for k in title_filter)]
# Exclusion denylist (fails open — preferred; see NOISE_TITLE_EXCLUDE).
title_exclude = args.get("_title_exclude")
if title_exclude:
jobs = [j for j in jobs
if not any(_kw_in(k, (j.get("title") or "").lower()) for k in title_exclude)]
dates = [d for j in jobs if (d := _parse_posted(j.get("posted")))]
newest = max(dates) if dates else None
policy = args.get("_location_policy", "default")
company_seen = seen.setdefault(cid, {})
title_seen = set()
results = []
eligible = match = 0
for j in jobs:
jid = str(j.get("id") or j.get("url"))
ok, in_ch, is_remote, is_relocation = location_eligibility(
j.get("location", ""), policy=policy, country=j.get("country")
)
if not ok:
continue
# Requisition identity is the primary key (F11). Title alone is NOT identity: two
# different reqs called "Senior Data Engineer" in Bern and Zurich used to collapse
# into one, and different teams hiring the same title vanished silently.
req_key = _requisition_key(cid, j)
if req_key in req_seen:
# Same requisition seen again - the case the old rule existed for, a role
# republished once per location. Merge the location instead of dropping it.
first = req_seen[req_key]
extra = (j.get("location") or "").strip()
if extra and extra.lower() not in first["location"].lower():
first["location"] = f"{first['location']} / {extra}"
continue
norm_title = re.sub(r"\s+", " ", (j.get("title") or "").split("|")[0]).strip().lower()
if norm_title in title_seen:
# Same title, DIFFERENT requisition. Keep it and flag it; removing it here is
# what lost real roles.
title_collisions.setdefault(norm_title, 0)
title_collisions[norm_title] += 1
title_seen.add(norm_title)
eligible += 1
is_new = jid not in company_seen
score, pos, neg = score_job(j, title_only=bool(title_filter))
# Pre-filtered boards (e.g. SBB) with German/generic titles: _score_floor keeps
# already-relevant results out of the hidden weak bucket.
floor = args.get("_score_floor")
if floor is not None and score < floor:
score = floor
if score >= 2:
match += 1
# Aggregator rows (NAV) carry the real employer; the board name becomes the source.
employer = (j.get("employer") or "").strip()
lang = j.get("lang_check")
if lang is None and (policy.startswith("norway") or j.get("country") == "NO"):
lang = classify_language_requirement(j.get("description") or "",
j.get("work_language"))
if not j.get("description"):
lang["evidence"] = "no posting text from this board; read the JD"
if (policy.startswith("norway") or j.get("country") == "NO" or
(policy == "dk_no" and _place_match((j.get("location") or "").lower(),
NO_LOCATION_KEYWORDS))):
region = "Norway"
elif in_ch:
region = "Switzerland"
else:
region = "Europe-remote / Denmark"
row = {
"company": employer or display, "company_id": cid,
"source": display if employer else "",
"region": region,
"lang": lang,
"title": j["title"], "location": j["location"],
"url": j["url"], "posted": j.get("posted", ""),
"updated_at": j.get("updated_at", ""),
"deadline": j.get("deadline", ""),
"date_provenance": j.get("date_provenance", ""),
"unit": j.get("unit", ""),
"compensation": j.get("compensation", ""),
"contact": j.get("contact", ""),
"score": score, "pos": pos, "neg": neg,
"in_ch": in_ch, "remote": is_remote, "relocation": is_relocation,
"is_new": is_new,
}
results.append(row)
req_seen[req_key] = row
if is_new:
company_seen[jid] = {"title": j["title"], "first_seen": today}
if title_collisions:
detail = ", ".join(f"{title!r} x{count + 1}" for title, count in sorted(title_collisions.items())[:4])
warnings.append((
display,
f"{len(title_collisions)} title(s) shared by different requisitions, all KEPT "
f"(check for real duplicates): {detail}"))
stat = {
"company": display, "scraped": scraped, "eligible": eligible,
"match": match, "newest": newest, "error": False,
}
return results, stat, warnings
def _main():
today = datetime.now(timezone.utc).strftime("%Y-%m-%d")
# Record a decision and exit: --decide "<url>" <status> [note words...]
if "--decide" in sys.argv:
rest = sys.argv[sys.argv.index("--decide") + 1:]
if len(rest) < 2:
print('Usage: --decide "<url>" <status> [note...]', file=sys.stderr)
return
url, status, note = rest[0], rest[1], " ".join(rest[2:])
decisions = load_decisions()
# Update in place if this job is already logged under a drifted URL (truncated
# slug, other locale, /apply suffix) rather than adding a second row for it.
existing = find_decision_url(decisions, url)
prev = decisions.pop(existing, {}) if existing else {}
# Preserve any company/title already stored (filled when a report run tagged the URL).
decisions[url] = {
"company": prev.get("company", ""),
"title": prev.get("title", ""),
"decision": status, "note": note, "date": today,
}
save_decisions(decisions)
if existing and existing != url:
print(f"(replaced earlier URL for the same job: {existing})", file=sys.stderr)
print(f"Recorded: {status} — {url}", file=sys.stderr)
return
only, new_only, include_weak, hide_decided, region = None, False, False, False, None
for arg in sys.argv[1:]:
if arg.startswith("--region="):
region = arg.split("=", 1)[1].strip().lower()
if region not in ("ch", "no"):
print("Usage: --region=ch (Switzerland + Europe-remote boards) or "
"--region=no (Norway boards)", file=sys.stderr)
sys.exit(2)
continue
if arg in ("--help", "-h"):
print(__doc__)
return
if arg == "--new-only":
new_only = True
elif arg == "--include-weak":
include_weak = True
elif arg == "--hide-decided":
hide_decided = True
elif arg.startswith("--only="):
only = arg.split("=", 1)[1]
targets = [(cid, display, adapter, args) for cid, display, adapter, args in COMPANIES
if (not only or cid == only)
and (not region or _company_region(args) == region)]
if only and not targets:
known = ", ".join(c[0] for c in COMPANIES)
print(f"Unknown --only={only!r}. Known ids: {known}", file=sys.stderr)
sys.exit(2)
seen = load_seen()
decisions = load_decisions()
dec_index = index_decisions(decisions)
last_scrape = load_last_scrape()
all_results, errors, stats = [], [], []
new_last_scrape = dict(last_scrape)
run_start = time.perf_counter()
# API adapters in parallel; playwright serial (shared browser is not thread-safe).
api_targets = [t for t in targets if t[2] != "playwright"]
pw_targets = [t for t in targets if t[2] == "playwright"]
# Preserve COMPANIES order in stats/report via ordered result map.
fetch_results = {}
try:
if api_targets:
print(f"Fetching {len(api_targets)} API boards "
f"({API_FETCH_WORKERS} workers)...", file=sys.stderr)
with ThreadPoolExecutor(max_workers=API_FETCH_WORKERS) as pool:
futs = {
pool.submit(_fetch_company, cid, display, adapter, args): cid
for cid, display, adapter, args in api_targets
}
for fut in as_completed(futs):
cid, display, adapter, args, jobs, err, secs = fut.result()
print(f" {display}: "
f"{'ERROR' if err else f'{len(jobs)} jobs'} ({secs:.1f}s)",
file=sys.stderr)
fetch_results[cid] = (display, args, jobs, err, secs)
for cid, display, adapter, args in pw_targets:
print(f"Fetching {display} (playwright)...", file=sys.stderr)
_, display, _, args, jobs, err, secs = _fetch_company(
cid, display, adapter, args
)
print(f" {display}: "
f"{'ERROR' if err else f'{len(jobs)} jobs'} ({secs:.1f}s)",
file=sys.stderr)
fetch_results[cid] = (display, args, jobs, err, secs)
# Process in COMPANIES order for stable stats tables.
for cid, display, adapter, args in targets:
display, args, jobs, err, secs = fetch_results[cid]
if err is not None:
errors.append((display, err))
stats.append({
"company": display, "scraped": 0, "eligible": 0, "match": 0,
"newest": None, "secs": secs, "error": True,
})
continue
results, stat, warnings = _process_company(
cid, display, args, jobs, seen, today, last_scrape
)
# Adapters that degrade instead of failing (NAV on a rate limit) leave notes here.
warnings = [(display, w) for w in (args.get("_run_warnings") or [])] + warnings
stat["secs"] = secs
for w in warnings:
errors.append(w)
# Soft warning only when 0 jobs — still mark row with ⚠️ if was previously >0
prev = (last_scrape.get(cid) or {}).get("scraped", 0)
if stat["scraped"] == 0 and prev > 0:
stat["error"] = True
stats.append(stat)
all_results.extend(results)
new_last_scrape[cid] = {
"scraped": stat["scraped"], "date": today, "display": display,
}
finally:
# Seen-state is NOT committed here (F16). It used to be, which meant a failed report
# write left roles marked as seen and a `--new-only` retry hid exactly the hits that
# were never delivered. It is committed after the report is on disk.
save_last_scrape(new_last_scrape)
_close_browser()
total_secs = time.perf_counter() - run_start
all_results, n_dup = drop_nav_duplicates(all_results, targets)
if n_dup:
errors.append(("NAV register", f"{n_dup} posting(s) dropped as duplicates of a "
f"dedicated employer board"))
registry = load_reputation()
for r in all_results:
r["reputation"] = reputation_for(r["company"], registry)
if new_only:
all_results = [r for r in all_results if r["is_new"]]
all_results.sort(key=lambda r: (-r["score"], r["company"], r["title"]))
REPORTS_DIR.mkdir(parents=True, exist_ok=True)
# Run scope in the filename (F16). A partial run used to overwrite the full daily report:
# `--only=kdocy` replaced a 37-company report with seven rows, and the full run had to be
# repeated by hand. Only an unrestricted run owns the plain `<date>` name.
scope = []
if only:
scope.append(f"only-{only}")
if region:
scope.append(f"region-{region}")
if new_only:
scope.append("new-only")
if hide_decided:
scope.append("undecided")
stem = today if not scope else f"{today}.{'.'.join(scope)}"
if scope:
print(f"Restricted run: writing {stem}.md/.json, leaving the full daily report intact.",
file=sys.stderr)
report_path = REPORTS_DIR / f"{stem}.md"
write_report(report_path, all_results, errors, new_only, include_weak,
stats=stats, total_secs=total_secs,
decisions=dec_index, hide_decided=hide_decided)
# Plain data dump of every eligible role, unfiltered by keyword score — fit judgment
# for these is done in conversation against the profile, not by the keyword scorer.
json_path = REPORTS_DIR / f"{stem}.json"
dump = [{
"company": r["company"], "title": r["title"], "location": r["location"],
"url": r["url"], "posted": r["posted"], "deadline": r.get("deadline", ""),
"updated_at": r.get("updated_at", ""),
"date_provenance": r.get("date_provenance", ""), "unit": r.get("unit", ""),
"compensation": r.get("compensation", ""), "contact": r.get("contact", ""),
"is_new": r["is_new"],
"region": r.get("region", ""),
"source": r.get("source", ""),
"language": r.get("lang"),
"reputation": {k: r["reputation"].get(k) for k in
("verdict", "glassdoor", "kununu", "secondary", "checked", "note")
if r["reputation"].get(k) is not None},
"decision": lookup_decision(dec_index, decisions, r["url"]),
"keyword_signal": {"score": r["score"], "pos": r["pos"], "neg": r["neg"]},
"loc_flags": {
"in_ch": r["in_ch"], "remote": r["remote"], "relocation": r["relocation"],
},
} for r in all_results]
_atomic_write_json(json_path, dump)
# Report and dump are on disk, so the run genuinely delivered: now the roles may be
# marked seen. A restricted run must not mark anything seen at all - it did not look at
# the whole board, and doing so would hide those roles from the next full run.
# All returned matches have been delivered in the JSON, even for new-only or
# one-company runs. Unvisited companies remain unchanged in the loaded seen map.
save_seen(seen)
n_new = sum(1 for r in all_results if r["is_new"])
print(f"\nReport written: {report_path}", file=sys.stderr)
print(f"Total matches: {len(all_results)} ({n_new} new) | "
f"scanned {len(stats)} companies in {total_secs:.1f}s", file=sys.stderr)
if errors:
print(f"Errors/warnings: {len(errors)} - see report", file=sys.stderr)
# === Adapter coverage (refreshed 2026-07-18) ==================================
# 34 companies automated across 14 adapter types; 1 in MANUAL_CHECK (Oracle).
#
# Location policy:
# default — CH + Denmark + Europe/EMEA/global remote (work-from CH or DK)
# norway_only — Equinor
# dk_no — NATO (Denmark + Norway only; no CH)
#
# Automated (COMPANIES above):
# nav nav (NAV national register, Norway outside Oslo; added 2026-10-22)
# radancy arm (Trondheim)
# algolia_jobs dnv
# workday nvidia, novartis, equinor (norway_no_oslo)
# ashby kraken, openai, snowflake
# ibm confluent (IBM careers search API since 2026-10-15; Ashby board empty)
# greenhouse anthropic, gitlab, grafana, databricks, datadog, elastic, fivetran
# pcsx microsoft, microsoft_no
# smartrecruiters metgroup, cern
# rss bis
# amazon amazon (Amazon + AWS share one board)
# phenom roche (Cisco still uses the playwright path)
# teamtailor telenor, axpo, nordic (custom domain via base_url)
# getro (no company configured — adapter kept for future VC talent networks)
# onlyfy bitcoin_suisse
# lever palantir, quantco
# taleo nato (dk_no; nato.taleo.net §2 — marketing page is nato.int/vacancies)
# json swissgrid
# sbb sbb
# bkw bkw
# playwright google, apple, meta, cisco, ruag, postfinance, bfh
#
# MANUAL_CHECK: Oracle (ORC needs CH geographyId).
# ==============================================================================
@contextmanager
def writer_lock():
path = ROOT / "state" / "scout.lock"
path.parent.mkdir(parents=True, exist_ok=True)
try:
descriptor = os.open(path, os.O_CREAT | os.O_EXCL | os.O_WRONLY)
except FileExistsError:
raise SystemExit("Another scout writer is active. If a prior run crashed, verify it has stopped before removing state/scout.lock.")
try:
with os.fdopen(descriptor, "w") as stream:
stream.write(str(os.getpid()))
yield
finally:
path.unlink(missing_ok=True)
def main():
if any(arg in sys.argv for arg in ("--help", "-h")):
return _main()
with writer_lock():
_raw_decisions.cache_clear()
return _main()
if __name__ == "__main__":
main()
Reference in new issue View Git Blame Copy Permalink