A reference stores the identity of a related record. Dot-walking follows that reference to a field on the related record. Caller > Department compares the caller's department, while Assignment group compares the group that owns the incident. These answer different questions even when a department and support group have similar names.
Use the field selector to follow one relationship at a time. The stored caller identity remains unchanged when you filter on a department. A blank reference has no related department, so it doesn't match a specific department value. Read the full breadcrumb after adding the condition. For workplace handoffs, include Active when the request is about open work, because a caller's department alone can also select resolved incidents.