Preparing for the Holidays - Part I: Connecting Your Load Testing with Revenue Metrics

Is the holiday season really an eCommerce retailer’s most wonderful time of the year? It should be for those focused on website revenue generation. After all, online retail sales are expected to increase by 14.4 percent from last year. But for DevOps, the season may not be as joyful. From Black Friday through Cyber Monday when eCommerce traffic is at its peak, it can be a bit scary for those chosen few. 

Regardless of size, eCommerce sites must be prepared for the holiday onslaught. While Nike.com had trouble processing orders during last year’s Black Friday, Best Buy had to force a shut-down of its website mid-morning for nearly 2.5 hours that same day. The electronics retailer claimed that a surge in web traffic on mobile devices was to blame for the site’s outage. Anyone can be a victim of glitches and crashes. Even Walmart.com went down on Thanksgiving Day 2013. Early preparation for the holidays is key for eCommerce retailers. It’s best to begin projects this summer to make sure your site is ready for scale and will yield the best conversion rates.

The first web performance evaluation that must be completed is Load Testing.

What is Load Testing?

Load testing is under the umbrella of Performance Testing, and for eCommerce sites, it is a process that evaluates the maximum amount of traffic a site can handle without crashing or experiencing significant decreases in performance. Load tests are typically run multiple times until page load time stays consistent between unload (normal traffic) and load (high traffic). This takes a lot of work on the DevOps end. Performance bottlenecks must be identified, pages must be optimized, and additional servers may have to be added to keep web performance on par.
There are two types of load tests and both should be conducted by eRetailers in preparation for the Holiday season:

Endurance Tests

Also known as Soak testing, this type of load testing evaluates a site’s ability to handle a constant significant traffic flow for extended periods of time. It is important that a site performs just as good, if not better, at the end of the test as it does in the beginning. Endurance tests are vital for eCommerce sites because of the longevity of the holiday season.

Volume Tests

This type evaluates site performance when it is subject to extremely high traffic. It can pinpoint performance bottlenecks and is an excellent tool for DevOps. Sites need to be ready for traffic spikes on top online shopping days like Cyber Monday, Green Monday, and Black Friday.

Taking Advantage of Load Testing

Best Buy’s site crashed because the amount of shoppers on mobile devices triggered a technology glitch in the system. With mobile devices accounting for 41 percent of Cyber Monday site traffic in 2014, eCommerce sites must not only recognize and test the impact of high-traffic desktop users, but also the impact of mobile user surges.

Beyond just performing load tests to make sure your site can handle high traffic volume from various devices, a close secondary goal should be finding out how much your site slows down at peak traffic and then compare that to your Conversion Rate vs. Page Speed Curve (see below).

Don’t overlook the impact that slower site speeds will have on your conversion rates. By conducting a load test, you can figure out with a reasonable margin of accuracy how much revenue you will lose as a result of the slower web page response times. Conversely, monitoring a load test with the right tool can tell you your revenue opportunity page-by-page on your eCommerce site.

Where Blue Triangle Fits In

Blue Triangle analyze your load test results and predict the likelihood for shoppers to buy on your site so you know what pages should matter to your business and what pages do not have an effect on conversions. Get ready for the holidays and visit our website to see how Blue Triangle can help you realize your full revenue potential.