Yes, I know. Bots can be a real pain. They can skew your analytics. They can purchase those Foo Fighters tickets before you (that’s a story for another day). And worse, they can inject viruses, commit DDoS attacks and make your site slower.
But some bots are good, like…
Graphic from Incapsula.
So with good bots and bad bots comprising 52% of total internet traffic in 2016, it’s important to monitor their activity on your site for three reasons:
Bot traffic can create a high load on your site’s servers, slowing down server-side response times. This results in delays for your end users, especially in the geographic regions where an influx of bot traffic is present.
One of our retail customers had a large amount of bot traffic coming from Washington, Oregon and California.
The amount and frequency of bot traffic slowed down their back-end systems. This impacted bot traffic performance by 3 seconds and real-user performance by 2 seconds on the west coast.
Our Real User Monitoring technology calculated that the 2 second real-user slow down cost the retailer $15.3K per day before the bots were stopped at the firewall.
This could be happening on your site right now. Avoid performance pains and achieve faster performance and higher revenue by doing these things:
In 2010, Google announced that site speed is considered a factor in search rankings.
In their patent for Using Resource Load Times in Ranking Search Results, Google states:
"A search result for a resource having a short load time relative to resources having longer load times can be promoted in a presentation order, and search results for the resources having longer load times can be demoted."
With Blue Triangle, you can monitor the experience of specific bots on your site.
The image below shows the path Google bots took and their experience on the home page:
Here are a few ways you can improve search engine bot experience and your SEO:
Reducing images down to 200KB or less can dramatically decrease load time. You can even use tools like TinyPNG to remove unnecessary data from images. JPEG is your best bet, as all browsers support it.
Bot traffic could be impacting your revenue right now. Contact us, we're happy to help.