Background job runner for iPaaS data pipelines

The moment that triggers this problem: a data analyst kicks off a large mapping job inside their iPaaS tool, the browser tab freezes, and they have no idea if it's running, stuck, or silently failed. They end up re-running the flow 'numerous times' just to confirm a new field is available. There's no feedback, no queue visibility, no way to walk away and come back to a completed result.

This gap persists because iPaaS vendors are incentivised to keep you inside their UI — background async processing that lets you close the tab and check results later removes session time and makes their product feel less 'sticky'. Their business model and product metrics reward engagement, not efficient exit. So the async infrastructure stays weak on purpose, or at least is never prioritised.

What actually breaks for users: the 2-step sampling process described in the complaints is synchronous and blocks the entire interface. Loading gigabyte-plus files causes the UI to get stuck and 'fail to load the entire data sheets'. The output step at the end of a Trifacta-style flow can take as long as the preparation itself, with no progress visibility. Users sit watching a spinner.

This is a business and not a feature because the need recurs every time someone processes a large file — it's not a setup problem, it's a daily workflow tax. A buyer pays because the alternative is analysts babysitting browser tabs, running jobs twice, and losing hours per week to uncertainty. The recurrence is baked into the job.

What to build

Build a lightweight job orchestration layer that intercepts long-running iPaaS export and mapping jobs via API, executes them asynchronously in a managed worker queue, and sends the analyst a Slack or email notification with a download link when the output is ready — so they never need to watch a progress bar.

Where to start

Start with one specific iPaaS vendor that exposes a documented job API and has a large base of frustrated users complaining about this exact issue in public forums — win that community before expanding to others.

The hard part

Getting reliable API access to trigger and poll job state across multiple iPaaS vendors without official async support — you'll hit rate limits, undocumented endpoints, and session authentication walls that break the handoff.

How it makes money

Usage-based pricing per job-minute of compute time processed through the queue, with a free tier for jobs under 5 minutes to drive adoption among individual analysts before landing team or company contracts.