fix(scout): repair Roche/Apple, add Amazon+Axpo, make title filtering fail-open
Scraper fixes: - Roche: new fetch_phenom adapter (Phenom refineSearch). The old playwright scrape of ?locationsearch=Switzerland harvested recommendation-widget cards (Shanghai, Kyiv, Bogota) while the page reported no-results. 0 -> 88 CH-eligible roles. - Apple: dropped default_location "Switzerland", which relabelled US "Various Locations" postings as Swiss (84 phantom CH rows over 4 runs). Now honestly 0. - Meta: NOT broken — metacareers reports "1 Items" for Zurich. Comment added so it is not "fixed" again. New boards: - Amazon/AWS (fetch_amazon): 32 CH roles incl. a Zurich AWS FDE req and a Bern ProServe Cloud Architect. AWS is the evidenced cloud; claims.json forbids GCP. - Axpo (teamtailor via base_url + pagination): 461 roles, opens the energy lane. Locations read from schema.org jobLocation with ISO alpha-2 expanded, so Madrid/Milan/Warsaw roles are not marked Swiss. Telenor benefits too. Title filtering now has two explicit modes. Inclusion allowlists fail closed and hide unanticipated good-fit roles, so they are now used only where volume forces it (>~200 roles). Everything else uses the shared, board-agnostic NOISE_TITLE_EXCLUDE, which fails open and leaves the final call to the scorer and the reviewer. Palantir stays unfiltered per its existing documented rationale. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,6 +1,10 @@
|
||||
# Scout roster review — 2026-08-18
|
||||
|
||||
Analysis only. No changes made to `COMPANIES`. Every claim below was verified by probing the
|
||||
> **Status: implemented 2026-08-18.** Sections 1, 2 and 5 have been acted on; see
|
||||
> "What was actually done" at the bottom, which also records two findings in this
|
||||
> document that turned out to be **wrong** once the fixes were attempted.
|
||||
|
||||
Originally analysis only. Every claim below was verified by probing the
|
||||
live endpoint; unverified candidates are listed separately as *probe-worthy*, not as recommendations.
|
||||
|
||||
Method: aggregated the scan-stats tables from the four most recent full runs (2026-07-14, 07-27,
|
||||
@@ -134,3 +138,68 @@ rules out the Geneva traders.
|
||||
4. **Add title filters** to BKW, SBB, BFH, PostFinance, Swissgrid, Novartis, Palantir.
|
||||
5. **Decide on MET/LDC and the Norway three** — a judgment call, not a bug.
|
||||
6. Probe CERN, Swiss Post, NATO NCIA, SIX.
|
||||
|
||||
|
||||
---
|
||||
|
||||
# What was actually done (2026-08-18)
|
||||
|
||||
Implementing the recommendations disproved two of them. Both corrections are recorded here
|
||||
rather than quietly edited above, because the original claims were used to justify the work.
|
||||
|
||||
## Correction 1 — Meta is NOT broken
|
||||
|
||||
Section 1 listed Meta as returning garbage. It does not. `metacareers.com` itself reports
|
||||
**"1 Items"** for the Zurich office filter — the board is genuinely near-empty and the scraper
|
||||
reports it accurately. No change made beyond a comment recording the verification, so it is
|
||||
not "fixed" again later. Roche and Apple were real bugs; Meta was not.
|
||||
|
||||
## Correction 2 — inclusion filters were the wrong instrument
|
||||
|
||||
Section 2 recommended adding `_title_filter` allowlists to seven boards. That was implemented
|
||||
and then **reversed**, on the objection that an allowlist *fails closed*: a strong-fit role
|
||||
with a title nobody anticipated is dropped at fetch time, never scored, and appears in no
|
||||
report and no JSON dump. The final call should sit with the scorer and the reviewer, not a
|
||||
keyword gate.
|
||||
|
||||
Filtering is now two explicit modes:
|
||||
|
||||
| Mode | Behaviour | Where |
|
||||
|---|---|---|
|
||||
| `_title_exclude` (**default**) | Fails **open** — drops only unambiguous non-tech titles (`hausdienst`, `physiotherapie`, `violine`, `lehrstelle`, `legal counsel`…). Everything else is scored. | 14 boards |
|
||||
| `_title_filter` | Fails **closed** — allowlist, used only where volume makes full scoring impractical (>~200 roles) | Databricks, Snowflake, Datadog, Elastic, Fivetran, Louis Dreyfus |
|
||||
|
||||
`NOISE_TITLE_EXCLUDE` is one shared, board-agnostic list. Terms that could plausibly attach to
|
||||
a technical role (analyst, manager, specialist, consultant, architect, lead) are deliberately
|
||||
excluded from it.
|
||||
|
||||
Also corrected: BKW's 61/75 match rate was blamed on the missing filter. The real cause is
|
||||
`_score_floor: 2`, set deliberately because the English keyword scorer cannot read German
|
||||
titles. The floor was left in place.
|
||||
|
||||
## Fixes shipped
|
||||
|
||||
- **Roche** — new `fetch_phenom` adapter (Phenom `refineSearch`). Was 40 scraped / 0 Swiss;
|
||||
now **133 scraped, 88 CH-eligible, 25 matches**.
|
||||
- **Apple** — removed `default_location: "Switzerland"`, which was relabelling US "Various
|
||||
Locations" postings as Swiss. Now honestly reports **0 CH-eligible**.
|
||||
- **Amazon / AWS** — new `fetch_amazon` adapter. **32 CH roles, 29 eligible, 13 matches**,
|
||||
including the Zurich AWS FDE req and a Senior ProServe Cloud Architect in **Bern**.
|
||||
- **Axpo** — `fetch_teamtailor` extended for custom domains and pagination. 461 roles.
|
||||
Locations now come from the feed's schema.org `jobLocation`, with ISO alpha-2 expanded to
|
||||
full country names — without that, `"Burgdorf, CH"` failed the CH keyword match, and a
|
||||
forced `default_location` would have marked ~380 Madrid/Milan/Warsaw roles as Swiss (the
|
||||
Apple bug again). Telenor gained real locations from the same change.
|
||||
- **Filter modes** as described above, plus `artificial intelligence`, `data scientist`,
|
||||
`data-driven`, `ai engineer`, `ai platform`, `ai architect`, `ai-systems`, `data science`
|
||||
added to the shared allowlist. Bare `"ai"` deliberately omitted — it substring-matches
|
||||
*Maintenance*, *Training*, *Chair*.
|
||||
- **Palantir left unfiltered** — its entry already documents why (its target titles, e.g.
|
||||
"Deployment Strategist", are not in the allowlist). The original recommendation to filter it
|
||||
would have hidden exactly the roles worth seeing.
|
||||
|
||||
## Still open (deliberately not done)
|
||||
|
||||
- **Section 4 judgment calls**: MET Group + Louis Dreyfus, and the three Norway companies.
|
||||
These are preference decisions, not bugs.
|
||||
- **Section 7 probes**: CERN, Swiss Post, NATO NCIA, SIX Group.
|
||||
|
||||
Reference in New Issue
Block a user