Skills-driven agentic data engineering

Problem

Every new dataset on the platform went through the same manual loop: read the ticket, find the source, write the ingestion, write the dbt model, write the tests, open the PR, wait. Senior engineers were spending most of their week on work that followed a pattern they could describe in a paragraph.

Approach

• Encoded the team's conventions (model layering, naming, test coverage, PR shape) as versioned skills that any agent run loads before touching a repo.
• Built a branch-triggered orchestrator on GitHub Actions: an issue label spins up a Claude-powered sub-agent per repo (ingestion, transformation, quality) that opens PRs against the right codebase.
• Wired the agents to Snowflake, dbt, and GitHub through MCP servers so they work from live schema and lineage rather than stale docs.
• Kept a human gate at PR review. The agent proposes, the engineer decides. No agent merges to main.

Impact

• Roughly 2.5× faster time-to-production for analytics work, as presented at dbt Summit 2026.
• Targeting an 80% reduction in manual pipeline development across multi-repository dbt and ingestion workflows.
• Skills became the team's living style guide: onboarding engineers read the same files the agents do.

Why this way

Skills over tools. Loading every MCP tool into context is expensive and brittle; progressive disclosure through skills keeps the context small and makes the agent's behavior auditable. The workflow is only trusted because every decision it makes is traceable to a file a human wrote.

Links

• Visit dbt Summit session page