Day 2 of 7 - AISChallenge · AI Automation Society

Day 2 of 7 - AISChallenge

I scraped 23 unique contract + remote/hybrid roles across Jobserve + Indeed UK Sorted by relevance to a new GCP Solutions Architect cert holder with a software engineering background.

What surprised me

The decision tree in CLAUDE.md said "have a URL → scrape", so scraping Jobserve seemed obvious. But Jobserve uses session-based search IDs scraping the URL with query parameters returned 31,531 completely unfiltered results because the site ignores URL params without a valid browser session.

The fix wasn't to inspect the raw HTML to find the actual form selectors (#JobSearch_Keywords, #JobSearch_IncludeRemoteWorking), then submit the form programmatically. That filtered down to 1,270 relevant results instantly.

The lesson: the right tool wasn't obvious from the URL alone it took reading what the site was doing (session vs. stateless) before choosing between scrape and scrape + interact.

1 comment