Server Error (5xx): what is it and how to find it?
What is a 5xx Server Error?
A server returns a 5xx server error when it can't successfully complete a client's request.
5xx means any HTTP status code that starts with
5, so among others, this covers the
A server error (5xx) is often referred to as a “5xx server error.” These mean one and the same thing, and we’ll use them interchangeably throughout this article.
Why you should care about 5xx server errors
5xx server errors present a big issue, because they directly harm your visitor’s experience. Imagine having an eCommerce store, and your visitors keep receiving these server errors. Do you think they’ll keep trying and eventually purchase something?
No, they’ll go next door and buy from the competition.
And what about search engines like Google? If they receive
5xx server errors just occasionally, then you probably don’t have much to fear. But if the
5xx server errors are persistent, then it’s likely you’ll see a steady decline in organic traffic.
In this article we’ll cover why
5xx server errors are bad for SEO, what the most common
5xx server errors are, how to find out if your site’s server returns them, what causes this, and how to fix them.
Why every server error (5xx) hurts your SEO performance
We already know that
5xx server errors make for a very poor user experience. But they’re also bad news for your SEO performance.
- When search engine crawlers (“crawlers” from here on out) consistently receive
5xxserver errors while crawling your site, they’ll want to avoid adding to the performance issues your site already has, so they’ll dial back their crawl efforts.Essentially this means that they’re lowering your site’s . This results in your content getting (re)indexed more slowly.
Don’t forget: if your content isn’t indexed, it doesn’t rank.
- When crawlers keep receiving these
5xxserver errors for your URLs, why would they grant your troubled pages high search results?Any visitors they send your way will likely have a poor user experience. So, when the
5xxserver errors persist, Google will start “demoting” your URLs – assigning lower positions or even completely removing them from its search engine result pages (SERPs).
- When pages aren’t available to crawlers, they won’t be able to (re)index them and rank them. They won’t know what the content is about, and these pages won’t surface in Google’s search results.
The bottom line: you could have the best content in the world, and the strongest links to your site, but you’ll have a really hard time ranking if your site is consistently suffering from
5xx server errors.
The most common 5xx server errors
The most common
5xx server errors are:
- 500 – Internal Server Error: the server had issues processing the request; this could have any one of a thousand reasons, often related to technical issues such as bugs or software incompatibilities.
- 502 – Bad Gateway: the request cannot be fulfilled successfully, because an upstream service (a service you’re expecting a response from) did not respond correctly. The upstream service can be running on the same machine or another machine, and it typically consists of a CGI component such as PHP and an application component such as a WordPress CMS.
- 503 – Service Unavailable: the server is temporarily unavailable, and will be available again later. Often the
503server error is used when performing maintenance or when the server is too busy.
- 504 – Gateway Timeout: this status code is closely related to the
502server error. In the 504 server error, the server – while operating as a gateway or proxy – didn’t receive a response from another server in time. As we mentioned above, keep in mind that your web server is just a gateway too. What if your back-end takes too long to respond to your web server?
- 509 – Bandwidth Limit Exceeded: your website has exceeded the server’s assigned bandwidth limits and thus is no longer available to the public. This frequently happens on shared hosting environments when they experience a sudden influx of visitors and/or crawlers. While this server error is frequently used, it’s not specified in any RFCs ( ).
- 524 – A timeout occurred: the Cloudflare CDN invented this server error and shows it when the origin server – your site – doesn’t respond in time. If the CDN doesn’t get a response, they can’t provide visitors and crawlers with one either. This results in the following error screen, which you may be familiar with:
How to find 5xx server errors
24/7 cloud-based monitoring
If your website is vital to your business, you need to monitor it nonstop for issues such as these
5xx server errors. You want to be alerted in case of trouble. That’s exactly what the ContentKing platform does. Just plug in your site, the monitoring starts, and you’ll be alerted through Slack and/or email when something goes down:
Find 5xx server errors in GSC’s Index Coverage report
Find 5xx server errors in GSC’s Crawl Stats report
Google Search Console’s Crawl Stats report shows you statistics about Google’s crawling history on your website. It lists what file types are requested most, what purpose their crawl request had (refresh content, or discover new content), and – this is what we’re interested in most – what “crawl response” they received.
- Log on to
- At the top left, choose the right property.
- Then scroll all the way down, and click on
⚙️ Settingsin the left sidebar:
4. On the Settings screen, click
5. Look under
Crawl requests breakdown >
By response to find
Server error (5XX):
6. Click on
Server error (5XX) to jump to the overview of affected URLs. Interestingly, it’s very possible that this report will show different URLs than the Index Coverage report does.
Find 5xx server errors in your server log files
And last but not least, you can go straight to the source: a log of all the requests that were made and what the server’s response was – your server’s log files.
While it can be notoriously difficult to get your hands on server log files, they are very valuable, because they provide a full picture of all the
5xx server errors your server has returned.
At the end of the day, the Index Coverage and Crawl Stats reports are a digest, and they are limited to the 5xx server errors Google received.
What causes 5xx server errors?
There are thousands of reasons why your server might return a
5xx server error;
5xx server errors vary widely within their class and – while there are RFC standards and the like – it’s up to developers’ discretion to choose which 5xx server error is most applicable when a certain error occurs.
Additionally, it’s important to understand these server errors can occur on different levels, such as:
- CDN (e.g. Cloudflare or Akamai)
- Web server (e.g. Apache or nginx)
- Programming language (e.g. PHP or .NET)
- Application (e.g. WordPress or Magento)
- Plugin (e.g. a page speed optimization or contact form plugin)
Below we’ll cover some common examples of what can cause
5xx server errors, but this is by no means an exhaustive list.
Code bugs and updates gone wrong
The most common reason for
500 – Internal Server Error to occur is because of code bugs. The client makes a request, and while the server is processing that request, a bug causes some issue, and the server can’t successfully meet the request as a result.
Another common reason for the 500 server error is when running updates has gone wrong. For instance, when a new version of your CMS comes out, and you upgrade it, but some of your plugins aren’t compatible.
So provide your developers with leads about where to start debugging. Explain in as much detail as possible what went wrong, and where it went wrong.
The back-end has died
Your server will probably respond with a
502 – Bad Gateway or
504 – Gateway Timeout error if its back-end (such as the CGI component that runs the PHP powering your website) has crashed. Your server relies on its back-end to serve a page – and when the back-end doesn’t respond, it can’t produce anything.
An underpowered or misconfigured hosting platform
When you’ve got an underpowered or misconfigured hosting platform, and your server can’t handle all of the requests anymore, it’s likely to return the
503 – Service Unavailable response.
You could be a victim of your own success because your PR campaign has drawn a pile of visitors to your site. Or maybe crawlers have gone rogue, continuously hammering your site.
There’s no “one size fits all” solution to this issue. It’s best to discuss this with your developers and hosting provider. The right solution depends on your specific situation, and oftentimes includes a combination of sub-solutions. To name a few:
- Upgrade your hosting platform (beef up your server’s specifications).
- Enable caching.
- Start using a Content Delivery Network (CDN).
- Improve code performance.
Note that a crawler situation may eventually lead to the
509 – Bandwidth Limit Exceeded server error.
Tackle 5xx server errors head on
You’re bound to encounter
5xx server errors at some point. What matters most is that you quickly find out about them, and that you deal with them swiftly. Don’t let
5xx server errors hurt your SEO performance.
Monitor your website around the clock and leverage Google Search Console’s tools and your server log files to investigate and debug these issues. Keep an open mind as to the reason why you’re getting these
5xx server errors. There are literally thousands of possible reasons out there, and they can happen on various levels in your architecture.