Some work fans out over a list you only know at run time: today's depots, the files that arrived, the parcels still open. Dynamic task mapping makes one task instance per item when the run reaches the task:
@task
def depots(ds=None):
return bramble.depots_reporting(ds)
@task
def count(depot, ds=None):
return len(bramble.scans(depot, ds))
total(count.expand(depot=depots()))
count.expand(depot=...) takes a list (or a dict, or another task's output) and creates count[0], count[1] and so on: one map index each. The Grid shows them as one task with a count; click it to see each copy and its log. A task that receives the mapped output, like total, gets all the results as a list, in map index order.
Why not a for loop? A loop runs when the file is parsed, so it builds tasks from whatever list the parser saw, the same for every run. Mapping uses the list each run produces. If that list is empty, the mapped task is skipped, and so are tasks after it under the default trigger rule. Airflow caps a map at 1,024 copies by default (max_map_length).