For Linux hosting and WP Ready services, we have implemented measures against overloading our servers with bots that overload our clusters. These measures include blocking certain user agents based on set rules and limits. One such restricted bot is
facebookexternalhit/1.1 (
more information), which can cause problems loading Facebook page previews
How does the bot restriction work?
We have introduced a limit of 30 queries per domain per minute. If this limit is exceeded, the server returns a 429 (Too Many Requests) error. The implementation of robot limits is important for the stability and performance of our servers and is designed to protect resources from excessive load.
Specific issue with Facebook
facebookexternalhit is a robot that crawls web pages and caches them for possible sharing by users. This robot ensures that the correct thumbnails, captions, descriptions and other meta data are available when content is shared on Facebook.
Unfortunately, this bot is designed in such a way that if it encounters a 429 error, it caches it. This means that if a user then shares a page on Facebook to which this limit has been applied, an error will appear instead of the expected preview. In this case, customers often complain that the sharing is not working properly and assume that the problem is on our hosting side.
Problem solution
The error is actually on the side of Facebook and their robot
facebookexternalhit. The problem can be solved with a troubleshooting tool called
Facebook Sharing Debugger. Paste the URL of the problematic page into the Sharing Debugger and let it analyze it. This tool will reload the content and update the cache on the Facebook side.
While using the Debugger, you can also check what errors are detected on the page and possibly make further adjustments to optimize the preview.
If you have further questions or encounter other issues, feel free to contact our
customer support.