Schema evolution and constraints

Opening your lesson

Schema enforcement rejects writes that do not fit a table's schema. Schema evolution lets a deliberate write add compatible columns, such as a new source label. In the DataFrame writer, option('mergeSchema','true') opts into supported evolution for that write. It does not mean that every incompatible type should be silently accepted.

NOT NULL rejects a missing value in a required field. A CHECK constraint rejects rows for which its condition is false or unknown, so include the intended NULL policy in the expression. Generated columns derive their values from other fields and keep that derivation in the table definition. These rules protect a write boundary; they do not replace an ingestion review. Aster Supply still needs to decide whether a bad record should stop a batch or move to a separate review output before it reaches a constrained table.

CREATE TABLE main.silver.versioned(id INT NOT NULL,amount DECIMAL(10,2)) USING DELTA; INSERT INTO main.silver.versioned SELECT id,amount FROM main.bronze.orders;

Order ids are required. Customer ids contain a missing value and do not identify the orders.

Choose your character

Your Work Gym account