PySpark interview: medium

Opening your lesson

A plausible total can still answer the wrong question. Before coding an interview solution, state the grain of each input and of the requested result. A join can multiply records, an aggregate can collapse them and a window can preserve them while adding context.

This set asks you to separate those behaviors. You will find customers absent from an order feed, compare row counts with known-value counts and reason about ties in ranking. The tasks use the course's fictional retail tables so you can inspect the source records in the catalog. Your explanation should say why a row is kept or removed and why a denominator includes or excludes it. Performance suggestions come after that reasoning. Broadcasting a table or caching a result will not repair an incorrect relationship between the records.

c=spark.table('main.bronze.customers').select('id','name').withColumnRenamed('id','customer_id')
o=spark.table('main.bronze.orders').select('customer_id')
display(c.join(o,'customer_id','left_anti').select('customer_id','name'))

The customer-side anti join keeps absent order matches; semi keeps customers who have orders.

Choose your character

Your Work Gym account