From invites to meetups
Wayfriend wants to know whether friends who plan to meet actually spend time together. Clean the raw events, model meetups, measure success and plan the next day of work.
Level 1: Newcomer
0 points
Sign in to save your progress across devices.
Learn SQL in depth, enough Python to move and check data, a cloud warehouse, tested transformations in dbt and an orchestrator that runs it all on schedule. Every exercise is checked, and the pathway ends in pipeline projects, a timed assessment across the tools and the interview.
About 53 hours of lessons, from the estimate on each lesson.
Nothing done yet. Progress saves in this browser as you finish lessons, and in your account when you sign in.
Take the steps in this order. Each step is a set of units from one course, not the whole course: the units this role uses. Open a unit to start its first lesson.
SQL is the language of the job. Beyond querying, joins and window functions, a data engineer creates and changes tables, designs schemas, changes data safely inside transactions and reads a query plan when something is slow.
Design tables with constraints, choose indexes for a workload and test transaction boundaries in PostgreSQL. Read execution plans and practice safe schema changes before a service depends on the database.
Python moves data between systems and checks it on the way. You learn the language from values and collections to functions, then read and write files and formats, and write code that fails clearly instead of quietly.
At many companies, analytics data ends up in a cloud warehouse. You learn your way around Snowflake, query it the Snowflake way, flatten JSON and arrays to the right grain, change data safely and load files incrementally.
dbt turns raw tables into tested models people can trust. You set up a project, declare sources, build and test models, write analytics models, add data quality tests and documentation, process incrementally and keep history with snapshots.
A pipeline has to run on its own, in order, and tell you when it fails. You define assets and their dependencies, schedule jobs, split work into partitions for backfills, and add checks that catch stale or wrong data.
Define task dependencies, inspect runs and logs, and choose schedules before a pipeline needs retries or backfills.
Inspect topic partitions, keyed events and committed consumer offsets. Explain how a failure or rebalance changes the work left to process.
Each project starts from a fictional company's brief and moves between tools the way the job does: what you build in one stage is where the next one starts. Every brief is free to read.
Wayfriend wants to know whether friends who plan to meet actually spend time together. Clean the raw events, model meetups, measure success and plan the next day of work.
Denver tested reminder notifications before planned meetups. Decide whether Wayfriend should send them in every city.
Copperline's on-time report is a spreadsheet someone rebuilds every Monday. Rebuild it on the warehouse: one current row per shipment in Snowflake, a tested dbt model, and a Tableau view operations can trust.
You've joined Wayfriend's two-person data team. Build the meetup pipeline, answer the growth team's first question, then keep it running.
When you finish the units, test yourself across the tools of the job at once: 12 practical tasks in 50 minutes, each in the tool it is about and checked automatically. The clock keeps running if you close the page.
| Tool | Tasks | Minutes |
|---|---|---|
| SQL | 3 | 11 |
| Python | 3 | 13 |
| Snowflake SQL | 3 | 11 |
| dbt | 3 | 13 |
The assessment is free. The result is practice feedback in the Interview Hub's report format, not a hiring decision. Retakes draw different tasks where the question pools allow.
The Interview Hub runs the data engineer interview round by round, at a fictional employer: you answer out loud or in writing, the interviewer follows up, and the report quotes your answers.
A data engineer builds and runs the pipelines that bring data from the systems a company uses into a warehouse, cleaned, modeled and on time. Analysts, dashboards and models depend on that data being right. The work is part building, part keeping things running: a source changes a column, a load runs twice, a backfill is needed for last month. Good data engineers make those events boring.
SQL comes first and goes deepest, because every later tool runs on it. Python follows for the work SQL can’t do. The warehouse is where both meet. dbt adds structure and tests on top of the warehouse, and the orchestrator runs everything in the right order at the right time.
The Wayfriend projects are a data team’s first months. You clean raw events, model meetups and measure success in “From invites to meetups”, then answer a product question about reminders from the data you built. Copperline’s on-time report moves from a spreadsheet into Snowflake, a tested dbt model and a view operations can trust. Each gives you a pipeline you can explain from source to dashboard. The assessment then shows which tool to practice before you apply.
For database reports, practice SQL query optimization: read the plan and preserve the result as you rewrite the query. The SQL cheat sheet covers the query syntax.
No. Many data engineers start as analysts who wrote more and more SQL, others as software engineers. You need solid SQL, working Python and an understanding of how data moves and breaks. This pathway practices those, step by step.
The ideas carry across warehouses: grain, safe changes, incremental loads and cost. If your target jobs name another platform, the BigQuery course and the Databricks course teach the same work in those tools.
The lessons on this pathway add up to about 53 hours. SQL is the longest step on purpose: the interviews test it hardest. We don't promise how long getting hired takes.
The first unit of every course is free, with no account. One Course opens one course in full; All Access opens the whole pathway. The pricing page shows every plan and its price.