Use Web Analytics to Size and Prioritize Problems

This morning I was doing some last minute checks on an A/B split test. I ran the new page through different browsers to make sure my code didn’t break anything. It was then I noticed an error. I checked the original page and saw the same problem. The next question was determining how many people are impacted by this bug. To answer this, I used my web analytics program.

The reason I wanted to see how many people were impacted was because it helps the development team prioritize their work. Seriously do you want to keep your team fixing stuff that affects only 4 people? You probably don’t, unless one of those people writes your paycheck or is a key customer.

Using CrossBrowserTesting.com, I saw we had an issue with Microsoft Internet Explorer 6 and images. In the screen snap below, you can see the backgrounds on some images show through instead of being transparent.

Example of IE6 browser issue with image backgrounds
IE6 IMAGE BACKGROUNDS

Drilling Down With Analytics

On the surface, I wasn’t too bothered as I didn’t think the site had a lot of IE 6 users. It’s an older browser, and I figured most people have updated. But, I still wanted to be sure so decided to use Google Analytics to tell me how many visits came from that browser in the past 30 days. Although the number might be small, it could also be they were more active users and perhaps had higher conversion rates.

This particular site uses Google Analytics, but any decent web analytics package should give you info by browser version.

How to find number of site visits by browser version,

  1. Log into your Google Analytics account
  2. From the STANDARD REPORTS section on the left, select Technology and Browser & OS
Browser & OS menu item in Google Analytics
BROSWER & OS BREAKOUT

Your right side should refresh and list a line for each browser. At this point, I only know that the site has had 7112 visits in the past 30 days from people with Internet Explorer.

Summary of visits by browser
IE VISITS
  1. To see version details, I’ll click Internet Explorer.
Stats for IE6 visitors
IE6 VISITS

At a glance, I can see the site had 55 visits from people using this browser, and they tend to visit fewer pages. I’ll add this information to the bug ticket so the developer can factor this into the prioritization.

In this case, the problem was extremely small, but I wouldn’t have known that without drilling down into the web analytics. I’ve used this method before on bigger issues like Safari and Flash. Sometimes, drilling down into the details can give you a totally different perspective and set of numbers. And more often than not, it makes me ask new questions.