Day 2 - Scraping New Articles - #AISChallenge

Managed to build the scraper after some minor trouble. After installing the Firecrawl MCP, Claude code did not recognize it even after multiple reloads. After looking for solutions, I managed to fix it simply by going back and forth with Claude.

Once the MCP server is recognized, I tested it by scraping various websites. The best one of all is that I managed to scrape AI news articles from news.google.com over the last 30 days. As per my prompt, I got a csv file with article headline, source, date of publishing, article summary, and url. Sharing the screenshot here.

During the scraping process, I noticed that Claude, at first, managed to scrape only 7 articles. However, it self-corrected its scraping model and returned me with 44 articles published in the last 30 days.

Looking forward to Day 3 tomorrow.

1 comment