WHERE filters rows before they're grouped. HAVING filters groups after the aggregates are worked out, so it can use COUNT, SUM and the rest:
SELECT customer_id, COUNT(*) AS orders
FROM orders
GROUP BY customer_id
HAVING COUNT(*) >= 3;
Writing WHERE COUNT(*) >= 3 is one of the most common SQL errors. The database works through a query in this order: FROM, WHERE, GROUP BY, HAVING, SELECT, ORDER BY, LIMIT. At the WHERE step there are no groups yet, so there's nothing to count.
You can use both in one query: WHERE to drop rows you don't want counted, then HAVING to drop groups that don't qualify. A condition on a plain column, like status = 'delivered', belongs in WHERE. A condition on an aggregate belongs in HAVING.