Is anyone in here a high-level expert with data architecture and data science?
Quick background - I'm the Founder of a SaaS company and we're trying to find a consultant to help us with a large data project.
We run a platform that pulls product data from hundreds of different sources every day. The problem is, every source looks different, so it takes a lot of time and custom work to get them all working together. We need to rebuild our system so there’s one simple way to bring in and organize data, no matter where it comes from. We also want tools to automatically flag when a feed breaks, goes stale, or changes without warning. The big picture goal is to make adding new sources fast and easy, cut down on how much senior dev time is needed, and keep the whole system running smoothly so we can focus on growing instead of constantly fixing things.
If anyone has recs, it would be greatly appreciated!