4xx is the client error class. It means the request itself has a problem, covering things like a wrong URL (404), a syntax error (400), insufficient permissions (401, 403), and too many requests (429). Sending the same request again usually gives the same result, so you have to fix the request.
5xx is the server error class. It means the request was valid but the server failed to process it, including internal exceptions (500), a bad upstream (502), overload or maintenance (503), and gateway timeouts (504). These are not problems the client can fix, so you need to check the server logs and backend state.
In terms of debugging order, when you see a 4xx it is quicker to first check the request URL, headers, body, and authentication, and when you see a 5xx to check the server logs, upstream, and resource usage. 502, 503, and 504 are usually related to backend problems behind a reverse proxy.