I almost paid $29/mo to scrape X. Then this.
I wanted AI to find me content ideas automatically. Surface what's trending in my niche, score it, tell me what to create next. So I built a system. Convex database for storage, scoring engine for ranking tweets by engagement velocity, and Claude Code to evaluate which signals actually matter for my audience. The full pipeline. The one piece I was missing: getting the tweets in. I tried 4 different scraping services. One returned random tweets. One returned fake data. One ignored my filters and scraped 10,000 tweets in 5 minutes, burning $32 in credits before I caught it. I was about to sign up for their $29/month plan when I stopped and asked: can Claude Code help me find a better way? Turns out X has built-in advanced search operators that most people never use: - min_faves:10 - only tweets with 10+ likes (kills the noise) - min_faves:100 - only viral posts (competitive research) - -is:retweet - original content only - filter:links - posts sharing actual resources - filter:media - posts with screenshots or demos - lang:en - English only Combine them like this: "your niche keyword" min_faves:10 -is:retweet lang:en That one query gives you every high-engagement original post in your niche. No scraping service needed. I also run an "outlier" search with no engagement filter: "claude code" (built OR shipped OR "just launched") filter:links -is:retweet lang:en This catches new tools and announcements the moment they drop. I found Garry Tan's open-source Claude Code skill pack this way, minutes after he posted it. Zero likes at the time. Now the AI part: Claude Code takes these results, scores each tweet on relevance to my audience, and promotes the top 3 into my content pipeline as discovery notes. Every morning I see "here are today's best signals" without opening X. You craft the search strategy. AI helps evaluate and sort. The $29/month scraping service would have given me the same tweets wrapped in a database. X search plus Claude Code does it better, for free.