diff --git a/CLAUDE.md b/CLAUDE.md index 49b8726..cd05354 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -183,6 +183,7 @@ _See `config.md` for user-specific corrections. Add verified errors here as you | Date | Correction | Files fixed | |------|-----------|-------------| +| 2026-10-15 | **Two scout adapters were reporting wrong data without failing; both were found by the 2026-10-14 triage.** (a) **BIS:** the RSS adapter stamped `default_location: Basel, Switzerland` on every vacancy, so **jr100491 (Hong Kong, BIS Asia office)** passed the CH filter and reached a JD read. The feed carries no location, but each vacancy page has a static *Geographical location* field. `fetch_rss` now reads it per item (`location_page_regex`), falls back to a *"based in ..."* sentence in the feed description, and only then uses `default_location`, printing a warning when it does. Verified: Basel reqs still pass, and jr100491 now reads *8 Finance Street, Central, Hong Kong* and is excluded. (b) **Confluent:** IBM closed the acquisition on 2026-03-17, and the Ashby board went from 17 jobs (10-08) to `{"jobs":[]}` (10-14). careers.confluent.io sits behind a Vercel bot checkpoint. The roles are now IBM requisitions, served by the open IBM careers search API (`www-api.ibm.com/search/api/v2`, POST, `size` capped at 100, pages with `from`). New `ibm` adapter; Confluent keeps its id, so seen-state and decisions still link. First run: 84 roles, 0 in CH/Europe-remote scope. **HashiCorp was dropped on 2026-06-06 for having "no clean public ATS API" after IBM absorbed it. That API exists, so HashiCorp can come back through the same adapter if wanted.** **Rule: when an adapter must assume a field, the assumption has to be visible. A silent default looks exactly like real data.** | `scout.py` (`fetch_rss`, `_rss_item_location`, `fetch_ibm`, BIS and Confluent config) | | 2026-10-14 | **Spotfire was the sixth evidenced-but-unregistered skill, and junior mentoring was unrecorded - both surfaced by the BIS Basel jr100498 *Senior/Lead Data & Analytics Engineer* (MED IT) triage.** (a) **TIBCO Spotfire** sat in the BS-5 and BS-8 scopes (co-owned environment, technical Application Owner, C# extensions, TAF 2022, spatial wafer-data map charts) but not in the `skills` array, so the JD line *"using database reporting and business intelligence tools"* had no citable skill; registered `production-historical`, `allowed` on user instruction. It does not license Power BI, Tableau or Qlik. (b) The JD requires *"Experience providing general supervision and task-specific guidance to junior colleagues"*, and the only canonical evidence was BS-7 (students, one thesis as Zweitpruefer). User-confirmed: he **onboarded and mentored junior developers at Bosch and at Swisscom** - recorded as **BS-9** and **SW-10**, one per employer. The Swisscom phase (ART B2B or ART DPT) is not recorded, so write it in past tense. Neither is people management. (c) **SDMX: user-confirmed never used**; registered `forbidden` so the scanner catches it. It is an accountability line in the JD, not a qualification. (d) The same audit found **Teradata, Oracle Database and Hadoop/Impala** evidenced only in the SW-2/BS-2/BS-8 scopes and absent from the `skills` array, although every past skills line listed them; registered `production-historical` from those scopes (Oracle explicitly does not establish DBA work). **Lesson, same as 10-08: when a JD has a supervision or mentoring line, ask about junior colleagues before falling back on BS-7. Student supervision is weaker evidence and the stronger evidence was simply unrecorded.** | `claims.json` (BS-9, SW-10; skills TIBCO Spotfire, SDMX, Teradata, Oracle Database, Hadoop / Impala); `knowledge_base/extractions/2026-10-14_mentoring_spotfire_sdmx_clarification.md` | | 2026-10-08 | **Three more canonical gaps of the recurring shape, surfaced by the Kongsberg Seatex *Lead Engineer, DevOps and Test Infrastructure* (Trondheim) package.** (a) **Jenkins and GitLab CI/CD** were evidenced in claim scopes (GN-1, FC-1, BS-6, SW-3) but absent from the `skills` array, so no skills line could cite them; registered as `production-historical`, with GitLab keeping SW-3's "used, not built" limit. (b) **Embedded coursework** was unrecorded: user-confirmed an Embedded Systems course (C/C++ and VHDL with an ARM platform) and Robotics during his studies, and **no embedded work since** - recorded as **ED-1**, coursework-only, degree attribution not yet given. (c) **Current unit testing** was unrecorded: he writes unit tests today and knows pytest - recorded as **SW-9**; whether pytest is the Swisscom framework is not confirmed. The JD's required line *"Experience working with embedded SW and Linux"* stays a **half Gap** (hard-gate FAIL); the user overrode it knowingly. **Lesson, again: for any DevOps/test JD ask about current testing practice and about academic hardware work before scoring - neither lives in `claims.json` by default.** | `claims.json` (ED-1, SW-9, 7 skills); `knowledge_base/extractions/2026-10-08_embedded_coursework_testing_clarification.md`; Kongsberg package | | 2026-09-15 | **Swisscom was recorded as one undifferentiated current role, so past work read as present.** The first Geo Platform build said *"Own the ETL pipeline components"* and *"Owns ... Teradata"* in the summary - wrong. User-supplied: the tenure has two phases. **ART B2B** (first part, PAST): the legacy Teradata DWH - Fulfillment ETL component ownership (SW-2), Python on Kubernetes with GitLab CI/CD (SW-3), dashboards and analyses (SW-4), Kafka. **ART DPT, Team Offering** (CURRENT): migration and implementation of AWS data products on AWS, ODP-Flow, PySpark/Python, Glue, Iceberg, Athena, Airflow (SW-1, SW-6, SW-7). **Kafka and Kubernetes downgraded to `production-historical`**; SW-2/SW-3/SW-4 allowed verbs are now past tense; Apache Iceberg added as a canonical skill. Official internal title recorded: `Data, Analytics & AI Engineer IV` (III before Apr 2025), per the Oct 2025 Zwischenzeugnis. **The switch date is deliberately not recorded (user: not relevant) - never date the two units.** Any already-submitted package that presents Swisscom Kubernetes, Kafka or Teradata ownership as current stays as submitted history, but expect the question in interviews. **Lesson: the Zwischenzeugnis gives the whole tenure as one duty list, and a duty list says nothing about which duties are current - ask which phase each belongs to.** Same session, same shape: the Zeugnis's *"automation of CI/CD processes in GitLab"* overstated his part - the user clarified he **worked with** automated GitLab CI/CD rather than building it, and SW-3 now forbids claiming the automation. | `claims.json` (SWISSCOM org/title history; SW-1/2/3/4/6/7; Kafka, Kubernetes, Apache Iceberg; SW-3 CI/CD scope); `experience_swisscom.md` SW-3; Geo Platform resume | diff --git a/job_scout/scout.py b/job_scout/scout.py index 998a0e5..f1cd2cf 100644 --- a/job_scout/scout.py +++ b/job_scout/scout.py @@ -254,7 +254,10 @@ COMPANIES = [ # Dropped: ClickHouse (Glassdoor 3.3, 36% recommend, toxic-culture flag — 2026-05). # Dropped: HashiCorp — acquired by IBM (closed 2025); greenhouse/ashby/lever boards all 404, # roles folded into IBM's careers (no clean public ATS API). 2026-06-06. - ("confluent", "Confluent", "ashby", {"slug": "confluent", "_title_exclude": NOISE_TITLE_EXCLUDE}), + # Confluent: acquired by IBM (closed 2026-03-17). The Ashby board went empty (17 jobs on + # 2026-10-08, 0 on 2026-10-14); roles are now IBM requisitions. Moved to the `ibm` + # adapter 2026-10-15. Same id, so seen-state and decisions stay keyed to "confluent". + ("confluent", "Confluent (IBM)", "ibm", {"query": "confluent", "_title_exclude": NOISE_TITLE_EXCLUDE}), ("gitlab", "GitLab", "greenhouse", {"board": "gitlab", "_title_exclude": NOISE_TITLE_EXCLUDE}), ("grafana", "Grafana Labs","greenhouse",{"board": "grafanalabs", "_title_exclude": NOISE_TITLE_EXCLUDE}), # Added 2026-06-06 (Tier A/B data-infra). Databricks/Snowflake/Datadog have Zürich offices @@ -369,6 +372,9 @@ COMPANIES = [ # Low-volume RSS feed; no title filter (Innovation Hub roles can be oddly titled). ("bis", "BIS (Basel)","rss", { "url": "https://www.bis.org/doclist/vacancies.rss", + # Each vacancy page has a static "Geographical location" sidebar field. BIS also + # hires for Hong Kong, Mexico City and other offices; see _rss_item_location. + "location_page_regex": r"Geographical location\s*]*>\s*

([^<]+)

", "default_location": "Basel, Switzerland", }), # Dropped: Coinbase Ventures web3 talent network (Getro collection 1625) — 2026-08-18. @@ -826,10 +832,37 @@ def fetch_smartrecruiters(args): return jobs +def _rss_item_location(link, description, args): + """Per-item location for feeds that carry none. + + Order: the item's own page via `location_page_regex` (first group), then a + "based in " sentence in the feed description, then `default_location`. + Added 2026-10-15: stamping every BIS vacancy with default_location labelled a Hong Kong + req (jr100491) as Basel, so it passed the CH location filter and reached the triage. + """ + pattern = args.get("location_page_regex") + if pattern and link: + try: + req = urllib.request.Request(link, headers={"User-Agent": USER_AGENT}) + with urllib.request.urlopen(req, timeout=30, context=_ssl_context()) as resp: + page = resp.read().decode("utf-8", errors="replace") + match = re.search(pattern, page, re.I | re.S) + if match and match.group(1).strip(): + return html.unescape(match.group(1).strip()) + except (urllib.error.URLError, OSError, ValueError): + pass + match = re.search(r"\bbased in ([A-Z][^.;:]{2,60})", description) + if match: + return match.group(1).strip() + print(f" ! {link or '?'}: no location on page or in feed, using default_location", + file=sys.stderr) + return args.get("default_location", "") + + def fetch_rss(args): """Generic RSS/RDF feed parser. BIS publishes vacancies as RSS 1.0 (RDF), whose elements live in the http://purl.org/rss/1.0/ namespace. Falls back to plain - RSS 2.0 elements. Location isn't in the feed, so default_location is required.""" + RSS 2.0 elements. Location isn't in the feed; see _rss_item_location.""" import xml.etree.ElementTree as ET req = urllib.request.Request(args["url"], headers={"User-Agent": USER_AGENT}) with urllib.request.urlopen(req, timeout=30, context=_ssl_context()) as resp: @@ -844,13 +877,14 @@ def fetch_rss(args): el = it.find(tag) return (el.text or "").strip() if el is not None and el.text else "" link = field("link") + description = re.sub(r"<[^>]+>", " ", field("description")) jobs.append({ "id": link or field("title"), "title": field("title"), - "location": args.get("default_location", ""), + "location": _rss_item_location(link, description, args), "url": link, "posted": (it.findtext("dc:date", default="", namespaces=ns) or field("date")), - "description": re.sub(r"<[^>]+>", " ", field("description"))[:1500], + "description": description[:1500], }) return jobs @@ -1746,7 +1780,70 @@ def fetch_playwright(args): return jobs +IBM_SEARCH_API = "https://www-api.ibm.com/search/api/v2" + + +def fetch_ibm(args): + """IBM careers search: the open JSON endpoint behind ibm.com/careers/search. + + Added 2026-10-15 for Confluent. IBM closed the acquisition on 2026-03-17, and the + Confluent Ashby board (`jobs.ashbyhq.com/confluent`) returns no jobs. Its roles are now + IBM requisitions, mostly titled "... - Confluent". careers.confluent.io sits behind a + Vercel bot checkpoint, so this API is the clean path. + + `query` is a keyword search over title, description and body, as the site's own search + box sends it. The API rejects `size` above 100, so pages are fetched with `from`. + Location = city (unless "Multiple Cities" / "No City") + country + work mode, e.g. + "United States - Remote" or "Zurich, CH, Switzerland - Hybrid". + """ + body = { + "appId": "careers", "scopes": ["careers2"], "lang": "zz", "size": 100, + "query": {"bool": {"must": [{"simple_query_string": { + "query": args["query"], + "fields": ["keywords^1", "body^1", "url^2", "description^2", + "h1s_content^2", "title^3", "field_text_01"], + }}]}}, + "sort": [{"_score": "desc"}], + "_source": ["title", "url", "description", + "field_keyword_05", "field_keyword_17", "field_keyword_19"], + } + jobs, offset = [], 0 + while True: + body["from"] = offset + req = urllib.request.Request( + IBM_SEARCH_API, data=json.dumps(body).encode("utf-8"), method="POST", + headers={"Content-Type": "application/json", "User-Agent": USER_AGENT}) + with urllib.request.urlopen(req, timeout=30, context=_ssl_context()) as resp: + data = json.load(resp) + hits = data.get("hits", {}).get("hits", []) + for hit in hits: + src = hit.get("_source", {}) + city = (src.get("field_keyword_19") or "").strip() + country = src.get("field_keyword_05") or "" + if isinstance(country, list): + country = ", ".join(country) + place = ", ".join(p for p in ( + "" if city.lower().startswith(("multiple cities", "no city")) else city, + country.strip()) if p) + mode = (src.get("field_keyword_17") or "").strip() + url = src.get("url", "") + job_id = urllib.parse.parse_qs(urllib.parse.urlsplit(url).query).get("jobId", [""])[0] + jobs.append({ + "id": job_id or url, + "title": (src.get("title") or "").strip(), + "location": f"{place} - {mode}" if mode else place, + "url": url, + "posted": "", + "description": (src.get("description") or "")[:1500], + }) + total = data.get("hits", {}).get("total", {}).get("value", 0) + offset += len(hits) + if not hits or offset >= total or offset >= 1000: + return jobs + + ADAPTERS = { + "ibm": fetch_ibm, "workday": fetch_workday, "ashby": fetch_ashby, "greenhouse": fetch_greenhouse, @@ -2613,7 +2710,8 @@ def _main(): # # Automated (COMPANIES above): # workday nvidia, novartis, equinor (norway_only) -# ashby kraken, openai, confluent, snowflake +# ashby kraken, openai, snowflake +# ibm confluent (IBM careers search API since 2026-10-15; Ashby board empty) # greenhouse anthropic, gitlab, grafana, databricks, datadog, elastic, fivetran # pcsx microsoft # smartrecruiters metgroup, cern