The data behind the AI
Every AI system you can build is capped by the data underneath it. Meet Harbor Lane's messy reality, and the foundation you'll build in five weeks.
If you've taken our engineering track, you've built impressive things for Harbor Lane, the specialty coffee retailer: a triage assistant, a document Q&A engine, a resolution agent, maybe an analytics agent that answers questions in plain English. Every one of those systems has the same silent dependency: the data it reads. A RAG system over stale docs confidently cites last year's policy. An analytics agent over duplicated customers reports revenue that's wrong by 8%. An agent that acts on the wrong customer record doesn't just answer wrong — it does wrong.
This course is about that dependency. Not 'data engineering' in the abstract — the specific, load-bearing disciplines that decide whether AI built on your data can be trusted: modeling (shape the data so questions have one right answer), pipelines (move it reliably, every day), quality (catch breakage before the AI serves it), entity resolution (one customer = one record), and a semantic layer (metric definitions the whole company — and its AI agents — share).
Harbor Lane's actual situation
Harbor Lane sells online and in three physical stores. That innocent sentence hides everything this course teaches:
- Two customer systems. The web store has
web_customers(clean-ish, email required). The point-of-sale system haspos_customers(names typed by cashiers, phones in three formats, email optional). Thousands of people exist in both — as two unrelated rows. - Daily order exports. Each morning a job drops
orders_YYYY-MM-DD.csv. Some days it's late. One day it double-ran. Nobody noticed for a week. - No agreed definitions. Finance computes revenue net of refunds; marketing counts gross. Both present 'revenue' to the CEO. The numbers differ. The meeting is not fun.
You will fix all of it, hands-on. By the end you'll have a real warehouse file on your laptop — modeled, loaded idempotently, quality-gated, with resolved golden customers and a metrics layer that compiles definitions to SQL. Then the AI courses' systems have something worth standing on.
The toolchain: DuckDB + Python
The labs use DuckDB — a real analytical SQL database that installs with pip install duckdb and lives in a single file. No cloud account, no cluster, no cost. Everything you build translates line-for-line to BigQuery, Snowflake, Databricks, or Postgres: the SQL dialect shifts slightly; the disciplines — grain, idempotency, tests, survivorship, governed metrics — transfer exactly. That's deliberate: we teach the part that survives a vendor change.
This course is the spine of the Data & Analytics track. It pairs directly with two Edova products: Vigil (data observability — Module 3 is Vigil built by hand) and Meld (entity resolution — Module 4 is Meld built by hand). And the Conversational Analytics Agent course builds its agent on the same kind of governed semantic layer you'll build in Module 5 (that course uses its own company and metrics file — you share the pattern, not the artifact).
You need basic SQL (SELECT, JOIN, GROUP BY) and enough Python to read and lightly edit a 60-line script (change a number, comment a line) — every script is provided complete. If you have AI Foundations or equivalent working experience, you're ready. The labs build one artifact across all six modules — do them in order.