The dawn of the Data Intelligence Engineer | MotherDuck
AI eliminated MotherDuck’s lead data engineer’s backlog, but left him busier than ever making sure agents’ answers are right.
3 comments
Yeah, I think the shift from writing code to verifying the results is a big one. At LinkedIn, it felt like we spent 20% of our effort figuring out which metrics we needed for executive reporting and 80% building the Airflow pipelines to produce them.
Now we can try more metrics and iterate much faster. As more people start answering their own questions with AI, I think we’ll spend more time agreeing on what’s worth measuring and checking that the numbers mean what people think they mean. Also (at least in the messy middle) there probably will be a lot more "wrong" analytics unless there is a verification/audit layer.
Sure, accuracy is important, but the easier it gets to build anything the more I’d also pay for someone who knows what to build. Give an agent clean data and a good definition of churn and you can get churn by subscription tier in minutes. That’s only useful if churn by subscription tier will actually answer the question of “why”.
I get why the whole industry focused (and still focuses) on the building part (“analytics engineering”). Getting the grain right and seeing the dbt tests go green is satisfying and reasonably straightforward. You can spend a week on that, close the ticket, and chalk it up as a win even if the request was literally just somebody’s guess at what they needed from the data. LLMs now let us do that at 100x speed.
If I’m thinking about putting a team on retention for a quarter. OK, churn is worse on the basic plan. How many of those customers could we keep with a change we could ship? What would it cost? If we’d need an experiment to get a decent answer, I’d want someone to spot that before we spend a bunch of tokens on dashboards that are accurate but not useful.
I had a front-row seat to AI entering into data workflows, especially managing a data team for the majority of 2026. November 2025, it was lightly used. By April, no one was writing a single line of SQL across the team (including myself - jarring after 12 years of becoming a SQL expert!)
Conversations quickly became "what jobs will data people still have in the next 2-3 years?" both where I worked and with people I knew in the industry elsewhere. This article sums up well the most obvious landing spot: making the data and infra for AI systems to answer questions for the business. It's absolutely more engineering focused too - data modeling matters in the age of AI (and looks different than data modeling for people!).
Soon to be gone are the days of analysts hearing a question and looking into how to answer it.
Soon to be here are the days of data people producing reliable datasets for companies to use to self-serve decisions.
Self-serve was always a distant Holy Grail in data, and it's suddenly at the doorstep.