fix(scout): Apple blind to CH for 2 months, BIS behind Cloudflare, NAV rate limit, EMEA leak; reputation for every board
Apple (the big one) - The location=switzerland-CHE slug stopped resolving; Apple silently applied a US location, so every run from at least 2026-08-31 returned the same 21 US/intern cards and 0 Swiss roles. Now location code CHEC (from /api/v1/refData/postlocation), result list item as the card (location readable), pagination, settle delay, longer wait and per-element timeout. Verified twice: 24 Swiss roles each run. BIS - bis.org went behind a Cloudflare challenge (403 even headless). Moved to the Workday tenant bis.wd3.myworkdayjobs.com/External, which also carries the office. Host alias + lower-cased jr-id keep all 9 bis.org#jr decisions linked (applied jr100498 included). NAV - Search API 429s after ~9 requests and stays blocked for hours: discovery capped at 8 searches (4s pacing). Posting text now comes from NAV's official third-party feed (pam-stilling-feed, public rotating token), with a small, slow website-page fallback for ads the feed lacks (~2/3). Verified live on 15 real ads; fallback path tested on fixtures. Location filter - EMEA leak fixed: a country allowlist without CH/DK plus a trailing region tag is no longer Europe-remote (27 rows). Swiss/Danish segments in multi-location strings are evaluated per segment. Regression over all 2026-10-22 rows: only the fenced rows changed. Reputation - Every board now has a dated verdict; kununu added as a primary source next to Glassdoor (the one with more reviews decides; user tip). Second-source scores now count (a low one means avoid). AVOID: BIS (Glassdoor 2.9), BFH (kununu 3.3), Kdo Cy (federal administration kununu 3.4). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
eee608a213
commit
ff12f16545
3 files changed
+920
-77
No files matched your search
+222
-42
@@ -443,24 +443,37 @@ COMPANIES = [
|
||||
# Nammo, SINTEF, Kitron, Andøya, the long tail). Keyword queries on a request budget;
|
||||
# see fetch_nav for the rate-limit behaviour. Rows carry the real employer name.
|
||||
("nav", "NAV register (Norway)", "nav", {
|
||||
"queries": ["data engineer", "dataingeniør", "devops", "platform engineer",
|
||||
"software engineer", "systemutvikler", "python", "machine learning",
|
||||
"data platform", "cloud engineer", "site reliability", "backend",
|
||||
"test automation", "utvikler"],
|
||||
"search_budget": 14,
|
||||
# Ordered by expected yield: only the first `search_budget` run. Measured 2026-10-22:
|
||||
# 9 searches at 3s spacing triggered a 429 that lasted hours, so stay at 8 and pace 4s.
|
||||
"queries": ["data engineer", "devops", "software engineer", "systemutvikler",
|
||||
"python", "platform engineer", "dataingeniør", "utvikler",
|
||||
"machine learning", "test automation", "site reliability", "backend"],
|
||||
"search_budget": 8,
|
||||
"pages_per_query": 1,
|
||||
"detail_budget": 25,
|
||||
"pace_s": 3.0,
|
||||
"pace_s": 4.0,
|
||||
# Text checks go to the official feed host (no 429 seen at 0.5s), so the budget can
|
||||
# be generous; results are cached, so later runs only fetch new postings.
|
||||
"detail_budget": 60,
|
||||
"feed_pace_s": 0.5,
|
||||
# Website-page fallback for ads the public feed lacks (10 of 15 on 2026-10-22).
|
||||
# Same host as search, so few and slow; skipped entirely after a search 429.
|
||||
"page_fallback_budget": 6,
|
||||
"page_pace_s": 8.0,
|
||||
"_location_policy": "norway_no_oslo",
|
||||
}),
|
||||
# International org — BIS (Basel), commutable from Bern, salary net of Swiss tax.
|
||||
# Low-volume RSS feed; no title filter (Innovation Hub roles can be oddly titled).
|
||||
("bis", "BIS (Basel)","rss", {
|
||||
"url": "https://www.bis.org/doclist/vacancies.rss",
|
||||
# Each vacancy page has a static "Geographical location" sidebar field. BIS also
|
||||
# hires for Hong Kong, Mexico City and other offices; see _rss_item_location.
|
||||
"location_page_regex": r"Geographical location</h6>\s*<div[^>]*>\s*<p>([^<]+)</p>",
|
||||
"default_location": "Basel, Switzerland",
|
||||
# BIS moved from the bis.org RSS feed to its Workday tenant on 2026-10-22: bis.org went
|
||||
# behind a Cloudflare challenge that answers 403 even to a headless browser, while
|
||||
# bis.wd3.myworkdayjobs.com/External serves the same vacancies WITH their office
|
||||
# ("BIS Headquarters Basel" vs "BIS Representative Office for Asia and the Pacific"), so
|
||||
# the per-vacancy location lookup the RSS adapter needed is no longer necessary.
|
||||
# Decision keys stay on bis.org#jr…: see _HOST_ALIASES and the bis.org patterns.
|
||||
("bis", "BIS (Basel)","workday", {
|
||||
"host": "bis.wd3.myworkdayjobs.com",
|
||||
"tenant": "bis",
|
||||
"site": "External",
|
||||
"nav_aliases": ["BIS"],
|
||||
}),
|
||||
# Dropped: Coinbase Ventures web3 talent network (Getro collection 1625) — 2026-08-18.
|
||||
# Never carried Coinbase's own roles, only portfolio companies. Across ~3 months
|
||||
@@ -510,11 +523,29 @@ COMPANIES = [
|
||||
"_title_exclude": NOISE_TITLE_EXCLUDE,
|
||||
}),
|
||||
("apple", "Apple", "playwright", {
|
||||
"url": "https://jobs.apple.com/en-us/search?location=switzerland-CHE",
|
||||
# Location code is CHEC (postLocation-CHEC, from /api/v1/refData/postlocation). The old
|
||||
# "switzerland-CHE" slug stopped resolving: Apple silently applied a US location
|
||||
# ("Chesterfield, Missouri") instead, so every run from at least 2026-08-31 to
|
||||
# 2026-10-22 returned the same 21 US/intern cards and 0 Swiss roles. Fixed 2026-10-22;
|
||||
# the CHEC filter shows 25 Swiss roles, Zurich engineering included. If this board
|
||||
# ever returns only non-Swiss cards again, re-check the code via that endpoint.
|
||||
"url": "https://jobs.apple.com/en-us/search?location=switzerland-CHEC",
|
||||
"wait_for": "a[href*='/en-us/details/']",
|
||||
"card": "a[href*='/en-us/details/']",
|
||||
"title_attr": "text",
|
||||
"link_attr": "href",
|
||||
"wait_timeout_ms": 45000,
|
||||
"settle_ms": 6000,
|
||||
"element_timeout_ms": 10000,
|
||||
# 20 results per page, 1-indexed (25 Swiss roles on 2026-10-22 = 2 pages). Each Apple
|
||||
# page is slow (an empty last page costs the full wait), so cap at 2 extra pages:
|
||||
# up to 60 roles. Raise it if the Swiss board ever grows past that.
|
||||
"page_param": "page",
|
||||
"page_param_start": 2,
|
||||
"max_pages": 2,
|
||||
# The card is the result list item, not the title link: the link alone carries no
|
||||
# location, so 26 of 28 Swiss rows read location-less and were dropped (2026-10-22).
|
||||
# The item text reads "<title> <team> <date> Location Zurich Actions".
|
||||
"card": "li.rc-accordion-item",
|
||||
"title_sel": "a[href*='/en-us/details/']",
|
||||
"link_sel": "a[href*='/en-us/details/']",
|
||||
"url_prefix": "https://jobs.apple.com",
|
||||
# NO default_location. Apple's own postLocation-CHE filter leaks global reqs whose
|
||||
# postLocationId is postLocation-USA ("Various Locations within United States"), and
|
||||
@@ -1746,6 +1777,10 @@ def fetch_playwright(args):
|
||||
)
|
||||
ctx.add_init_script(STEALTH_JS)
|
||||
page = ctx.new_page()
|
||||
# Per-element timeout for slow boards: Apple took 876s once and 25s the next time on the
|
||||
# same day (2026-10-22); a stalled card read should fail fast, not hold the whole run.
|
||||
if args.get("element_timeout_ms"):
|
||||
page.set_default_timeout(args["element_timeout_ms"])
|
||||
jobs = []
|
||||
seen_ids = set()
|
||||
|
||||
@@ -1756,7 +1791,14 @@ def fetch_playwright(args):
|
||||
wait_for = args.get("wait_for")
|
||||
if wait_for:
|
||||
try:
|
||||
page.wait_for_selector(wait_for, timeout=15000)
|
||||
# Per-board timeout: Apple's search page sometimes takes well over 15s to
|
||||
# render its result cards, and a miss here returns 0 jobs (2026-10-22: 0 vs 21
|
||||
# two days earlier, selectors unchanged).
|
||||
page.wait_for_selector(wait_for, timeout=args.get("wait_timeout_ms", 15000))
|
||||
# Boards that render results progressively need a pause after the FIRST card
|
||||
# appears, or the scrape catches a partial list (Apple: 15 vs 24, 2026-10-22).
|
||||
if args.get("settle_ms"):
|
||||
page.wait_for_timeout(args["settle_ms"])
|
||||
except Exception:
|
||||
page.wait_for_timeout(4000)
|
||||
# Scroll a few times to trigger any lazy-loaded results
|
||||
@@ -1961,9 +2003,10 @@ def fetch_ibm(args):
|
||||
# (user, 2026-10-22: EU/NATO citizen, expects to be clearable); only an explicit Norwegian-
|
||||
# citizenship requirement is. Every verdict carries its evidence snippet so it can be checked.
|
||||
_LANG_ENGLISH_OK = [re.compile(p, re.I) for p in (
|
||||
r"norsk\s+og\s*/\s*eller\s+engelsk", r"engelsk\s+og\s*/\s*eller\s+norsk",
|
||||
r"norsk\s+eller\s+engelsk", r"engelsk\s+eller\s+norsk",
|
||||
r"skandinavisk\w*\s+(?:språk\s+)?(?:og\s*/\s*)?eller\s+engelsk",
|
||||
# The optional hyphen covers compounds: "gode norsk- eller engelskkunnskaper".
|
||||
r"norsk-?\s+og\s*/\s*eller\s+engelsk", r"engelsk-?\s+og\s*/\s*eller\s+norsk",
|
||||
r"norsk-?\s+eller\s+engelsk", r"engelsk-?\s+eller\s+norsk",
|
||||
r"skandinavisk\w*-?\s+(?:språk\s+)?(?:og\s*/\s*)?eller\s+engelsk",
|
||||
r"engelsk\s+(?:og\s*/\s*)?eller\s+(?:et\s+)?skandinavisk",
|
||||
r"english\s+(?:and\s*/\s*)?or\s+(?:a\s+)?(?:scandinavian|norwegian)",
|
||||
r"(?:norwegian|scandinavian)\s+(?:and\s*/\s*)?or\s+english",
|
||||
@@ -2068,6 +2111,50 @@ def classify_language_requirement(text, work_language_tags=None):
|
||||
NAV_SEARCH_URL = "https://arbeidsplassen.nav.no/stillinger/api/search"
|
||||
NAV_AD_URL = "https://arbeidsplassen.nav.no/stillinger/stilling/{uuid}"
|
||||
NAV_DETAIL_CACHE = ROOT / "state" / "nav_detail_cache.json"
|
||||
# Full ad text comes from NAV's OFFICIAL third-party feed, not the website. Measured
|
||||
# 2026-10-22: the website search API 429s after ~9 requests at 3s spacing and the block then
|
||||
# lasts hours, while the feed host served 12 requests at 0.5s without a 429. The feed entry
|
||||
# endpoint takes the same uuid the search returns and returns clean ad HTML (no page chrome).
|
||||
# Its bearer token is public but rotates (exp ~1 month), so it is fetched fresh each run.
|
||||
# The feed itself cannot replace search for discovery: it pages chronologically from 2023 and
|
||||
# ?last=true only shows changes from now on, not the ads already open.
|
||||
NAV_FEED_TOKEN_URL = "https://pam-stilling-feed.nav.no/api/publicToken"
|
||||
NAV_FEED_ENTRY_URL = "https://pam-stilling-feed.nav.no/api/v1/feedentry/{uuid}"
|
||||
|
||||
|
||||
def _nav_feed_token():
|
||||
req = urllib.request.Request(NAV_FEED_TOKEN_URL, headers={"User-Agent": USER_AGENT})
|
||||
with urllib.request.urlopen(req, timeout=30, context=_ssl_context()) as resp:
|
||||
body = resp.read().decode("utf-8", "replace")
|
||||
match = re.search(r"eyJ[\w-]+\.[\w-]+\.[\w-]+", body)
|
||||
if not match:
|
||||
raise RuntimeError("NAV feed public token not found")
|
||||
return match.group(0)
|
||||
|
||||
|
||||
def _nav_ad_text(uuid, token, pace_s):
|
||||
"""Full ad text for one posting via the official feed; raises _RateLimited on 429."""
|
||||
time.sleep(pace_s)
|
||||
req = urllib.request.Request(NAV_FEED_ENTRY_URL.format(uuid=uuid), headers={
|
||||
"User-Agent": USER_AGENT, "Accept": "application/json",
|
||||
"Authorization": f"Bearer {token}"})
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=40, context=_ssl_context()) as resp:
|
||||
entry = json.loads(resp.read().decode("utf-8", "replace"))
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 429:
|
||||
raise _RateLimited(uuid) from e
|
||||
raise
|
||||
ad = entry.get("ad_content") or {}
|
||||
return _html_to_text(f"{ad.get('title', '')} {ad.get('description') or ''}")
|
||||
|
||||
|
||||
def _nav_page_text(job, pace_s):
|
||||
"""Fallback: the arbeidsplassen ad page. Measured 2026-10-22 the public feed returned 404
|
||||
for 10 of 15 relevant postings (ads it does not republish), so without this most Norway
|
||||
ads would never get a language check. The page is on the rate-limited website host, so it
|
||||
runs on its own small budget, slowly, and only when the search itself did not hit a 429."""
|
||||
return _html_to_text(_nav_get(job["url"], pace_s))
|
||||
|
||||
|
||||
class _RateLimited(RuntimeError):
|
||||
@@ -2155,8 +2242,13 @@ def fetch_nav(args):
|
||||
cache = json.loads(NAV_DETAIL_CACHE.read_text(encoding="utf-8"))
|
||||
except (FileNotFoundError, ValueError):
|
||||
cache = {}
|
||||
fetched = pending = 0
|
||||
blocked = any("rate limit" in w for w in warnings)
|
||||
fetched = pending = page_fetched = 0
|
||||
# The text checks use the official feed host, so a rate limit on the search host does not
|
||||
# block them; only a 429 or a token failure on the feed itself does. Ads missing from the
|
||||
# feed fall back to the website page, which shares the search host's rate limit.
|
||||
blocked, token = False, None
|
||||
page_ok = not any("rate limit" in w for w in warnings)
|
||||
page_budget = args.get("page_fallback_budget", 6)
|
||||
for job in jobs.values():
|
||||
cached = cache.get(job["id"])
|
||||
if cached:
|
||||
@@ -2166,7 +2258,21 @@ def fetch_nav(args):
|
||||
job["location"], policy="norway_no_oslo", country="NO")[0]
|
||||
if relevant and not blocked and fetched < detail_budget:
|
||||
try:
|
||||
text = _html_to_text(_nav_get(job["url"], pace))
|
||||
if token is None:
|
||||
token = _nav_feed_token()
|
||||
try:
|
||||
text = _nav_ad_text(job["id"], token, args.get("feed_pace_s", 0.5))
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code != 404 or not page_ok or page_fetched >= page_budget:
|
||||
raise
|
||||
try:
|
||||
text = _nav_page_text(job, args.get("page_pace_s", 8.0))
|
||||
page_fetched += 1
|
||||
except _RateLimited:
|
||||
page_ok = False
|
||||
warnings.append("NAV website rate limit during page fallback; ads the "
|
||||
"public feed lacks stay provisional this run.")
|
||||
raise urllib.error.URLError("page fallback rate-limited")
|
||||
fetched += 1
|
||||
verdict = classify_language_requirement(text, job["work_language"])
|
||||
verdict["checked"] = datetime.now(timezone.utc).strftime("%Y-%m-%d")
|
||||
@@ -2175,10 +2281,19 @@ def fetch_nav(args):
|
||||
continue
|
||||
except _RateLimited:
|
||||
blocked = True
|
||||
warnings.append("NAV rate limit hit during posting-text checks; remaining "
|
||||
warnings.append("NAV feed rate limit hit during posting-text checks; remaining "
|
||||
"postings use the work-language tag only.")
|
||||
except RuntimeError as e:
|
||||
blocked = True
|
||||
warnings.append(f"NAV feed token unavailable ({e}); posting-text checks skipped "
|
||||
f"this run.")
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code != 404:
|
||||
warnings.append(f"NAV posting {job['id']}: text fetch failed ({e!r})")
|
||||
# 404 = not in the public feed and no page budget left: counted as pending.
|
||||
except (urllib.error.URLError, ValueError) as e:
|
||||
warnings.append(f"NAV posting {job['id']}: text fetch failed ({e!r})")
|
||||
if "page fallback rate-limited" not in str(e):
|
||||
warnings.append(f"NAV posting {job['id']}: text fetch failed ({e!r})")
|
||||
if relevant:
|
||||
pending += 1
|
||||
verdict = classify_language_requirement(job["description"], job["work_language"])
|
||||
@@ -2186,7 +2301,9 @@ def fetch_nav(args):
|
||||
job["lang_check"] = verdict
|
||||
if pending:
|
||||
warnings.append(f"{pending} relevant NAV posting(s) have no full-text language check yet "
|
||||
f"(budget {detail_budget}/run); they are marked provisional.")
|
||||
f"(checked {fetched} this run, {page_fetched} via the website fallback; "
|
||||
f"budgets {detail_budget} feed / {page_budget} page); they are marked "
|
||||
f"provisional and get checked on later runs as the cache fills.")
|
||||
_atomic_write_json(NAV_DETAIL_CACHE, cache)
|
||||
return list(jobs.values())
|
||||
|
||||
@@ -2320,8 +2437,38 @@ COUNTRY_NAMES = {
|
||||
"FR": ("france", "frankreich", "fr"), "IT": ("italy", "italien", "it"),
|
||||
"GB": ("united kingdom", "great britain", "uk", "gb"),
|
||||
"US": ("united states", "usa", "us"), "NL": ("netherlands", "niederlande", "nl"),
|
||||
# Added 2026-10-22 for the EMEA-leak fix: countries that appear in remote allowlists.
|
||||
# Full names only - two-letter codes ("es", "se", "in", "be") would match ordinary words.
|
||||
"IE": ("ireland", "republic of ireland"), "ES": ("spain",), "SE": ("sweden",),
|
||||
"FI": ("finland",), "PL": ("poland",), "PT": ("portugal",), "BE": ("belgium",),
|
||||
"RS": ("serbia",), "CZ": ("czech republic", "czechia"), "IL": ("israel",),
|
||||
"CA": ("canada",), "IN": ("india",), "RO": ("romania",), "GR": ("greece",),
|
||||
"HU": ("hungary",), "EE": ("estonia",), "LT": ("lithuania",), "LV": ("latvia",),
|
||||
"UA": ("ukraine",), "TR": ("turkey", "türkiye"), "AE": ("united arab emirates",),
|
||||
"SG": ("singapore",), "AU": ("australia",), "JP": ("japan",), "BR": ("brazil",),
|
||||
}
|
||||
|
||||
# A real pan-European remote scope, as opposed to an ATS region tag appended to a country
|
||||
# list ("Germany (Remote) EMEA", "Remote, France; ...; Remote, United Kingdom EMEA").
|
||||
_GENUINE_EUROPE_SCOPE = re.compile(
|
||||
r"remote[\s,(\-–:]+(?:in\s+)?(?:europe|emea|the\s+eu)\b|(?:europe|emea)[\s,\-–]+remote"
|
||||
r"|anywhere\s+in\s+(?:europe|emea|the\s+world)|europe-wide|eu-wide|pan-european"
|
||||
r"|worldwide|work\s+from\s+anywhere|remote\s*[-–]\s*global", re.I)
|
||||
|
||||
|
||||
def _segment_names_place(loc_text, code, town_keywords):
|
||||
"""True if one location segment names this country, or one of its towns without naming
|
||||
any other country. Multi-location strings ("PL-Poland-Remote | CH-Zurich-MSO") otherwise
|
||||
lose the Swiss/Danish segment as soon as another country is recognised, while a single
|
||||
segment like "Baden, Germany" must not count as Swiss Baden."""
|
||||
for seg in re.split(r"[|;\n]", (loc_text or "").lower()):
|
||||
others = {c for c, names in COUNTRY_NAMES.items() if c != code and
|
||||
any(_place_match(seg, [n]) for n in names if len(n) > 2)}
|
||||
own = [n for n in COUNTRY_NAMES.get(code, ()) if len(n) > 2]
|
||||
if _place_match(seg, own) or (_place_match(seg, town_keywords) and not others):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _has_non_oslo_norway_place(loc_text):
|
||||
"""True unless every location segment that can be placed is in the Oslo area.
|
||||
@@ -2381,8 +2528,16 @@ def location_eligibility(loc_text, policy="default", country=None):
|
||||
if re.search(r"(?:excluding|except|not in|außer|ausser|nicht in)\s+" + token, low) or re.search(token + r"\s+(?:excluded|not eligible|ausgeschlossen)", low):
|
||||
excluded.add(code)
|
||||
countries -= excluded
|
||||
in_ch = "CH" in countries if explicit_country else _place_match(low, CH_LOCATION_KEYWORDS)
|
||||
in_dk = "DK" in countries if explicit_country else _place_match(low, DK_LOCATION_KEYWORDS)
|
||||
# With an adapter-supplied country, trust it. With countries parsed from free text, also
|
||||
# check segment by segment so a Swiss/Danish segment survives beside another country.
|
||||
if country:
|
||||
in_ch, in_dk = "CH" in countries, "DK" in countries
|
||||
elif explicit_country:
|
||||
in_ch = "CH" in countries or _segment_names_place(loc_text, "CH", CH_LOCATION_KEYWORDS)
|
||||
in_dk = "DK" in countries or _segment_names_place(loc_text, "DK", DK_LOCATION_KEYWORDS)
|
||||
else:
|
||||
in_ch = _place_match(low, CH_LOCATION_KEYWORDS)
|
||||
in_dk = _place_match(low, DK_LOCATION_KEYWORDS)
|
||||
in_no = "NO" in countries if explicit_country else _place_match(low, NO_LOCATION_KEYWORDS)
|
||||
is_us_only = any(p in low for p in US_ONLY_PATTERNS) and not in_ch
|
||||
has_remote = bool(re.search(r"\bremote\b", low)) or "work from anywhere" in low
|
||||
@@ -2393,7 +2548,13 @@ def location_eligibility(loc_text, policy="default", country=None):
|
||||
# (F12: "Europe" beside "London / Onsite" used to read as remote-eligible).
|
||||
restricted = "CH" in excluded or bool(_ONSITE.search(low)) or (
|
||||
bool(_COUNTRY_ONLY.search(low)) and not (in_ch or in_dk))
|
||||
is_europe_or_global = (europe_scope or global_scope) and not is_us_only and not restricted
|
||||
# EMEA leak (found 2026-10-20, fixed 2026-10-22): a country allowlist without CH/DK plus a
|
||||
# trailing region tag is NOT pan-European scope. Seven such rows (GitLab FR/DE/NL/ES/UK,
|
||||
# Grafana DE/IE/ES/SE/UK, Fivetran Serbia/Ireland) were passing as Europe-remote.
|
||||
fenced = (explicit_country and not (in_ch or in_dk)
|
||||
and not _GENUINE_EUROPE_SCOPE.search(low))
|
||||
is_europe_or_global = ((europe_scope or global_scope) and not is_us_only and not restricted
|
||||
and not fenced)
|
||||
is_ch_remote = in_ch and has_remote
|
||||
|
||||
if policy == "norway_only":
|
||||
@@ -2492,7 +2653,9 @@ def save_seen(seen):
|
||||
# Board-specific req-ID patterns, tried in order against the URL path+query. Every key is
|
||||
# scoped by host (see _decision_key), so an ID only ever has to be unique within one board
|
||||
# and cross-board collisions are impossible by construction.
|
||||
_HOST_ALIASES = {"apply.careers.microsoft.com": "jobs.careers.microsoft.com"}
|
||||
_HOST_ALIASES = {"apply.careers.microsoft.com": "jobs.careers.microsoft.com",
|
||||
# BIS Workday tenant (2026-10-22) keeps the bis.org#jr… decision keys.
|
||||
"bis.wd3.myworkdayjobs.com": "bis.org"}
|
||||
|
||||
_JOB_ID_PATTERNS = [
|
||||
# Amazon: the slug is truncated to a fixed width and that width has changed between runs,
|
||||
@@ -2510,6 +2673,9 @@ _JOB_ID_PATTERNS = [
|
||||
# orphaning every decision recorded under the old scheme — including the `applied` row for
|
||||
# jr100429. The jr-id is stable across both. (Verified 2026-09-04.)
|
||||
(r"bis\.org$", r"/(jr\d+)"),
|
||||
# BIS Workday URLs end "..._JR100498" or "..._JR100499-1"; same jr-id, lower-cased in
|
||||
# _decision_key so it matches the bis.org keys.
|
||||
(r"bis\.org$", r"(?i)_(jr\d+)"),
|
||||
(r"smartrecruiters\.com$", r"/(\d{6,})"),
|
||||
(r"greenhouse\.io$", r"/jobs/(\d+)"),
|
||||
(r"careers\.cisco\.com$", r"/job/(\d+)"),
|
||||
@@ -2565,7 +2731,8 @@ def _decision_key(url):
|
||||
continue
|
||||
m = re.search(id_pat, target)
|
||||
if m:
|
||||
return f"{host}#{m.group(1)}"
|
||||
ident = m.group(1).lower() if host == "bis.org" else m.group(1)
|
||||
return f"{host}#{ident}"
|
||||
|
||||
# No known ID shape: fall back to the path with cosmetic variation stripped. The QUERY
|
||||
# is kept (F13): dropping it made /vacancy?job=1 and /vacancy?job=2 the same key, so one
|
||||
@@ -2763,15 +2930,28 @@ def reputation_verdict(entry, thresholds):
|
||||
return "stale"
|
||||
except ValueError:
|
||||
pass
|
||||
rating = (entry.get("glassdoor") or {}).get("rating")
|
||||
reviews = (entry.get("glassdoor") or {}).get("reviews") or 0
|
||||
if rating is None:
|
||||
return "ok" if entry.get("secondary") else "unchecked"
|
||||
# Glassdoor and kununu are both primary sources; the one with more reviews decides. For
|
||||
# Swiss employers kununu is often far better populated (user tip, 2026-10-22: SBB 1,808
|
||||
# kununu reviews vs no usable Glassdoor figure; RUAG 523 vs a thin Indeed 3.1).
|
||||
primaries = [s for s in (entry.get("glassdoor"), entry.get("kununu"))
|
||||
if s and s.get("rating") is not None]
|
||||
primary = max(primaries, key=lambda s: s.get("reviews") or 0) if primaries else {}
|
||||
rating = primary.get("rating")
|
||||
reviews = primary.get("reviews") or 0
|
||||
good_min = thresholds.get("good_min_rating", 3.8)
|
||||
min_reviews = thresholds.get("good_min_reviews", 50)
|
||||
ok_min = thresholds.get("ok_min_rating", 3.5)
|
||||
# A second source counts only if it is solid: when it carries a score (Indeed, kununu),
|
||||
# a score below the ok line makes the verdict avoid, not ok (RUAG: Indeed CH 3.1).
|
||||
secondary_rating = entry.get("secondary_rating")
|
||||
if rating is None:
|
||||
if secondary_rating is not None:
|
||||
return "ok" if secondary_rating >= ok_min else "avoid"
|
||||
return "ok" if entry.get("secondary") else "unchecked"
|
||||
if rating >= good_min and reviews >= min_reviews:
|
||||
return "good"
|
||||
if rating >= good_min and secondary_rating is not None and secondary_rating < ok_min:
|
||||
return "avoid"
|
||||
if rating >= good_min and entry.get("secondary"):
|
||||
return "ok"
|
||||
if rating >= ok_min:
|
||||
@@ -2783,12 +2963,12 @@ def _reputation_line(rep):
|
||||
verdict = rep.get("verdict", "unchecked").upper()
|
||||
if verdict == "UNCHECKED":
|
||||
return "UNCHECKED - look up before recommending (state/employer_reputation.json)"
|
||||
gd = rep.get("glassdoor") or {}
|
||||
bits = []
|
||||
if gd.get("rating") is not None:
|
||||
reviews = f", {gd['reviews']:,} reviews" if gd.get("reviews") else ""
|
||||
scope = f", {gd['scope']}" if gd.get("scope") else ""
|
||||
bits.append(f"Glassdoor {gd['rating']}/5{reviews}{scope}")
|
||||
for label, src in (("Glassdoor", rep.get("glassdoor") or {}), ("kununu", rep.get("kununu") or {})):
|
||||
if src.get("rating") is not None:
|
||||
reviews = f", {src['reviews']:,} reviews" if src.get("reviews") else ""
|
||||
scope = f", {src['scope']}" if src.get("scope") else ""
|
||||
bits.append(f"{label} {src['rating']}/5{reviews}{scope}")
|
||||
if rep.get("secondary"):
|
||||
bits.append(rep["secondary"])
|
||||
if rep.get("checked"):
|
||||
@@ -3376,7 +3556,7 @@ def _main():
|
||||
"source": r.get("source", ""),
|
||||
"language": r.get("lang"),
|
||||
"reputation": {k: r["reputation"].get(k) for k in
|
||||
("verdict", "glassdoor", "secondary", "checked", "note")
|
||||
("verdict", "glassdoor", "kununu", "secondary", "checked", "note")
|
||||
if r["reputation"].get(k) is not None},
|
||||
"decision": lookup_decision(dec_index, decisions, r["url"]),
|
||||
"keyword_signal": {"score": r["score"], "pos": r["pos"], "neg": r["neg"]},
|
||||
|
||||
Reference in new issue
Block a user