3 Ways to Validate Your CDN's Effectiveness

It's been proven over and over again that online sales suffer when shoppers have a slow website experience. So in order to improve web performance, appropriate CDN investments need to be made. These can get really pricey and it is oftentimes very difficult to calculate their effectiveness. 

You've got to keep your CDN accountable, and I'm going to show you 3 ways to do it.

1: Run an A/B Test

This one's a no-brainer. Simply divert half of your traffic to a CDN-treated version of your site, and the other half to a non-treated version. Then measure the results.

One of our clients wanted a crystal clear business case to justify the ongoing cost of their CDN. Their performance team conducted an A/B test, setting a cookie to divert half of their traffic to CDN-treated pages and the other half to non-treated pages. The customer used Blue Triangle to measure both traffic segments simultaneously and report on the differences in conversion rate, revenue, and performance to justify their CDN investment with upper management.

Results: By using Blue Triangle to measure their A/B test, the retailer was able to fully validate their investment. The retailer could have generated an extra $21,600 if all traffic was diverted to the CDN-treated site. The company discovered that it took them only 4 months to recoup their yearly CDN investment.

The graph below shows the correlation between page load time and conversion rates. The shaded portions of the graph represent the site’s Product Detail Page traffic volume with and without CDN implementation, and the two lines represent the site’s conversion rate curves with and without CDN implementation.

Clearly, running and properly measuring an A/B test provides you with a very detailed analysis on your CDN effectiveness. If there is not much change in page response time (page speed) and especially sales, consider reevaluating your investment.

The data below speaks for itself:

2: Measure the Success of Your CDN's Features

Chances are, your CDN probably has a variety of features/treatments that you can regularly configure. Akamai for example, offers some pretty cool ones like Adaptive Image Compression, Dynamic Page Caching, and Page Prefetching. Treatments like these usually have various "dials" that can be turned to optimize your site's performance. If they're used correctly and continually monitored for their effectiveness, you can provide very good validation of your CDN expense. On the other hand, if these treatments don't improve performance and are not easily customizable to your site, you should reconsider the depth of your CDN investment.

For example, a large national retailer generating over $2 billion in online sales annually was looking to reduce their site’s full page load time by using Akamai’s Adaptive Image Compression (AIC), but needed to determine what the optimal compression levels were. As the AIC treatments occurred in production, Blue Triangle measured the relationship that page speed had on conversion rates for specific pages.

Results: The retailer found that while performance improved significantly with image compression (confirming the feature's capabilities), conversion rates dropped because the compression rate was too high. From here, they used Blue Triangle’s exclusive analysis to continually monitor and fine-tune Akamai’s AIC optimizations on a page-by-page level to support revenue improvements rather than just performance improvements. Once you make your website's performance a business conversation, you can better validate your CDN's effectiveness and keep it accountable

3: Predict the Revenue Impact of Downgrading Your CDN

If you're considering changing your CDN provider to a cheaper (and probably slower) one or looking to downgrade your service plan, it is important that you calculate how much revenue you will lose if your site will have a decline in performance. From here, you can calculate if it is worth disinvesting - providing the ultimate validation test for your CDN.

Blue Triangle's lost revenue calculator can predict the amount of revenue you will lose if you have a decline in performance. If this site was 0.5 seconds slower on Cyber Monday for example, they would have $410K in unrealized revenue.

Next Steps

First, realize that you are not alone. One the most difficult tasks for any CDN customer is validating it and quantifying its ROI, but it can be done. I hope that this post has provided you with some quality insight into calculating your CDN's effectiveness.

At Blue Triangle, we have helped many online businesses validate their CDN. Blue Triangle's Real User Monitoring can help.