How do you use AI in data engineering?

Data engineers use AI to draft SQL, dbt models and pipeline code, to explain unfamiliar queries and schemas, and to write tests and documentation. Many also build pipelines that feed AI features, such as embedding documents for search. The core skill is checking AI-written SQL and pipeline code for the things it often gets wrong: joins, grain, nulls, time zones and idempotency.

Get started - freeLive cloud workspace in your browser · nothing to install

Writing SQL and pipeline code faster

AI can draft a query, a dbt model or a load script in seconds. Give it the real schema and the grain of each table. Then check the result against the data: row counts, totals, and a few rows by hand.

What to check in AI-written data code

A join that multiplies rows, a GROUP BY at the wrong grain, NULLs compared with = instead of IS NULL, time zones dropped, a load that double counts when it runs twice, and a bare except that drops bad rows silently. These look correct in a quick read.

Tests and documentation

Ask AI to propose dbt tests (unique, not_null, relationships, accepted values) and column descriptions, then keep the ones that match how the data really behaves.

Pipelines for AI features

More data teams now prepare data for AI: cleaning and chunking documents, generating embeddings, and keeping vector stores in sync with source data. The same pipeline skills apply - incremental loads, validation and monitoring.

Real Data work to put in your portfolio

SQL Join Returning Duplicate Rows? Find the Missing Join ConditionA SQL join that returns 100,000 rows for 1,000 orders is a cross join - find the missing join condition,...How to Debug a SQL Query COUNT and GROUP BY That Double-CountsLearn why a SQL query COUNT GROUP BY returns totals twice as high as they should be, how a one-to-many JOIN...How to Make an ETL Pipeline Idempotent (PostgreSQL Upsert)A plain INSERT ETL pipeline doubles row counts on every re-run. Fix it with a unique index and INSERT ON...

Related questions

Will AI replace data engineers?

AI speeds up writing code, but someone still has to design pipelines, decide what correct data means and catch where generated code is wrong. Those skills matter more, not less.

Should I use AI to write SQL?

Yes, as a first draft. Verify every query against the data with row counts and totals before you trust it.

What data engineering skills matter more now that AI writes code?

Data modeling, understanding grain and joins, debugging pipelines and data quality. These are where AI output most often needs a person to catch problems.

Do real production work, free. Live cloud workspaces in your browser - every project you complete lands on your portfolio.

Get started - free
Get started - free