Networks drop, APIs time out, databases restart. Retries turn those blips into a short delay instead of a failed night: @task(retries=2, retry_delay=timedelta(minutes=1)) gives a task up to three tries, one minute apart. Between tries the task instance is up_for_retry (yellow in the Grid), and each try has its own log.
retry_exponential_backoff=True doubles the wait after each failure, up to max_retry_delay, which is kinder to a struggling service. To give every task the same policy, put it in the Dag's default_args: default_args={"retries": 2, "retry_delay": timedelta(minutes=1)}. Airflow's own default is no retries.
Some failures will never fix themselves: a rejected password, a missing column, a bug. Retrying them only delays the alert. Raise AirflowFailException (from airflow.exceptions) and the task fails at once, whatever its retries.
Retry the timeouts, but stop at once on a rejected key.