1. Why AI runs on data engineering

A trained model is a compressed function of its training data, so a model can never be better than the data pipeline that fed it. Every AI system is a data system first and a model second.

The first principle

Supervised learning estimates a mapping from inputs to outputs using samples. The model has no access to the world, only to the samples you hand it. If those samples are biased, stale, duplicated, mislabeled, or secretly contain the answer, the model faithfully learns exactly that.

This is why practitioners often get larger gains from fixing data than from changing architectures. The idea has a name, data-centric AI: hold the model fixed and systematically improve the dataset.

Where data engineering shows up in AI work

• Training data: collect, clean, deduplicate, label, split and version the examples a model learns from.
• Features: compute the same model inputs identically during training and at prediction time.
• Evaluation: keep frozen, versioned test sets that never leak into training.
• Retrieval (RAG): ingest, parse, chunk, embed and index documents, then keep the index fresh.
• Inference logs: capture prompts, inputs, outputs, latencies and user feedback.
• Feedback loops: turn those logs into the next round of evaluation and training data.
• Monitoring: detect when production inputs drift away from what the model was trained on.

The loop at the end matters most. A production AI system generates its own future training data, so a bad logging decision today becomes a bad model next quarter.

[embedded content: AI data lifecycle · 8 stages, one feedback loop]

Read it clockwise from Sources: the bottom row runs right to left, and logging closes the loop back to the top.

Why AI data is harder than reporting data

Classic data engineering grew up serving dashboards (business intelligence, BI). AI adds five pressures that BI rarely has.

1. Correctness is statistical. A dashboard with a wrong number looks wrong. A training set with 3% mislabeled rows trains fine and quietly produces a worse model.
2. Time travel. A feature must be computed as of the moment a prediction would have been made, not as of today. Getting this wrong is called leakage.
3. Unstructured data at volume. Text, images, audio, PDFs and embeddings do not fit neatly into tables, and are measured in terabytes.
4. Reproducibility. Months later you must rebuild the exact dataset that produced model v3, to debug it, audit it, or compare against it.
5. Legal weight. Licensing, consent and GDPR erasure requests apply to data that has already been baked into a model.

Silent failures data engineering exists to prevent

Failure	What you see	What actually happened	Covered in
Leakage	Great offline metrics, poor production results	Training features contained information from the future	12
Training-serving skew	Model degrades right after deployment	Features computed differently in training and serving code	12
Eval contamination	Benchmark scores look excellent	Test examples, or near-duplicates, were in the training set	13
Schema drift	Pipeline runs green, model gets worse	An upstream team renamed a field or changed its units	11
Stale index	Chatbot answers with outdated facts	Retrieval index was not refreshed when sources changed	14
Duplicate data	Model memorises and overfits	The same documents were ingested many times	13
GPU starvation	Training is slow and expensive	Data loading cannot keep up with the accelerators	15

Who does what

Role	Owns	Typical question
Data engineer	Ingestion, storage, pipelines, platform	Is the data arriving, complete and on time?
Analytics engineer	Modeled tables in the warehouse, often with dbt	What does “active customer” mean, in SQL?
ML engineer	Training pipelines, features, model serving	Are training and serving features identical?
MLOps / platform engineer	Infrastructure, orchestration, registries, monitoring	Can anyone reproduce and deploy this model?
AI engineer	LLM applications, RAG, evaluation, agents	Is retrieval returning the right context, and how do we measure it?

The boundaries blur in small teams. As an AI engineer you need enough data engineering to ask the platform the right questions, and to build the pipeline yourself when no platform exists.

Questions to ask

• Where did this data come from, and who is allowed to change it?
• What could make it wrong without anything crashing?
• Can I rebuild the exact dataset that trained the current model?
• How fresh does this need to be, and what does each step of freshness cost?
• When the upstream source changes, who finds out first: us, or our users?