Anomaly Detection for Automating Site Monitoring

Internet complexity impacts sites 

When it comes to the web, the name of the game is innovation. Networking protocols, web availability, and even hardware for spreading the Internet from ISPs has changed drastically from the beginning to now. With the evolution of smaller and smaller computer chips, the kinds of devices accessing the Internet today are more diverse than ever. 

 Websites have evolved alongside, from the elegant simplicity of the first binary code to the robust coding languages used today, in order to keep up. But regardless of how robust the language is, perfect website operation for all users is unattainable. 

Where complexity can go wrong...and right

Features of a website that may work well for someone on a laptop might be deal-breakers to a person on a mobile, not to mention that users on mobile and tablet devices all have different screen sizes, chipsets, operating systems, and browsers. Developing a website that will work well for all users is very difficult to do. 

 A lot can go wrong during a page load, too. Site servers, routing protocols, service providers, and DNS servers are all potential points of failure during a request for a page load. A site can be unreachable through no fault of your own, so perfect operation under all conditions is impossible.  

 However, new advancements in web technology has led to other technological development that can help make monitoring easier and speed up response to issues with website operation. Things like artificial intelligence and machine learning can be used to automate data identification. That includes data related to website operation – and there is no better example of this than Anomaly Detection.  

What is Anomaly Detection? 

Anomaly Detection implements machine learning to detect issues. Using both Real User and Synthetic Monitoring data, Anomaly Detection constantly checks for real and potential hang-ups in performance, JavaScript Errors, and vendor service or tags.  

 Blue Triangle’s Anomaly Detection can pinpoint issues on your site and show you the cause of the problem. We are continually refining detection methods and developing new methods for detecting anomalies, too.

Traditional site performance monitoring requires things like performance alerts, outage alerts, and user feedback. When the issue is only on a cart page for some users, for instance, the issue may go undetected for some time, and when a problem is detected, it will require digging to identify which users it affects and why.  

By using Anomaly Detection to monitor data, you can manage your site and keep it running smoothly in a way that takes a fraction of the time. Anomaly Detection replaces the need to comb through your site for errors or wait for user feedback.

How does it help? 

Time is one of the most important things on your site. If there is a problem, how long does it persist? How long does it take to find it? To fix it? Anomaly Detection saves time in a couple of ways. 

Anomaly Detection saves time by automating the alerting process. There are upwards of 10,000 elements on most sites, not including pages and vendor services or tags. Configuring alerts on even a subset of this data would take days or maybe weeks, not to mention that multiple things can go wrong – the First Byte time might be high, or perhaps Onload, or maybe JavaScript errors start happening with increasing frequency. 

A manual configuration of alerts on that many elements would take an overwhelming amount of resources that would be better spent elsewhere. Real User data is gathered for both Desktop and Mobile devices as well, so you can drill down into different types of user experiences. This is especially valuable to smaller businesses that manage their own websites without an operations team.

Through alerting and notifying about potential problems on the site as a whole, Anomaly Detection also saves the time it would normally take to find specific issues. A site anomaly will let you know what changed and where. Importantly, it will also let you know how significant an anomaly is, allowing you to narrow your focus to the most pressing current issues on your site. 

What sets Anomaly Detection apart? 

Anomaly Detection prevents anomaly detection during regular periods of low traffic, like overnight hours or maintenance windows. When a potential issue is found, it will also rate the severity of the issue by significance, pushing the most significant issues to the top spot in your notifications. 

Here is how the machine learning works: 

• Statistical averaging creates a trend of the same hour over the previous four weeks to create a baseline 
• The data is logged by time of day as well as the day of the week so irregularities are detected by time of day and day of week 
• When an issue is found by comparing to the baseline, the machine learning algorithms rate the severity by the amount of change from baseline by each metric 

This results in periods of low traffic being compared to similar periods of low traffic, and prevents things like daytime traffic on a weekday being compared to daytime traffic on a weekend. Because of this, Anomaly Detection generates only valuable alerts, making it a powerful reporting tool for any business.   

As the issues are found and reported, they are color coded based on severity, with the most pressing disruptions to your site colored in red and brought to the top of the list. Users can then easily prioritize fixing the most important problems first and work their way down to less severe issues. 

Using Anomaly Detection to optimize monitoring

With a single click, users can instantly drill down into the issues to find the source of the problem. Depending on where the issue is found, you will be able to see domain and tag performance trended, performance on different pages and different devices in both synthetic and real user data, and also JavaScript error reporting.  

Drilling down into the issues lets you utilize the power of these reporting pages, and the diversity of analysis lets you pinpoint the exact problem and take steps to fix it. Not only that, but by saving time in identifying issues, you can fix any problems affecting your bottom line faster.