Why You Should Measure All of Your Goals in an A/B Test

Google Website Optimiser is suitable for reporting on the statistics of one specific goal for your average A/B test, however it’s highly likely that any tests you’re running will affect other conversions you’ve set up on your website. The guide below will show you how and why to view all of your goals from each test variation page.

Make sure you have your goals set up in Google Analytics. If you don’t, I advise you to read Anna’s excellent post first.  Please note you must have been tracking your goals during the same time as you were running your test as Google Analytics will not show conversions retrospectively if they were not set up beforehand.

The first part is to set up custom Advanced Segments for your Website Optimiser test pages (original page and each page variation in your A/B test). Follow the steps below to do this. Please note that we are using the new Google Analytics version 5 throughout this post.

1. Click on the Advanced Segments drop down in the top left of your profile, then click on the button that says + New Custom Segment.
2. On the Advanced Segments view add the Page dimension from the drop down by scrolling or searching using the text field.
3. Set the condition to Containing and enter the page URL in the field to the right. For example if your original page or variation is located at http://www.mywebsite/category/productA.html then you would enter /category/productA.html in the box. Tip: if you have entered the correct URL you should get an auto complete prompt.  Make sure you name your custom advanced segment with the page name and state if it’s the variation or original. Click on the Save Segment to finish the set up.
4. Once you have created a custom advanced segment for one of the pages, repeat steps 1 and 2 for all the other page variations in your test.

Let’s now set up a Custom Report to view all of the data in a nice table that we can analyse and make some informed decisions as to the overall performance of the Website Optimiser test.

1. Click on the Custom Reports tab in the top navigation, then click the +New Custom Report button.
2. On the Create Custom Report page enter the Report Names as ‘Goals by Pages’ or something else suitable and name the Report Content as ‘Results’.
3. Now let’s add our Metrics (the blue boxes) to create the column headings for our table of results. Click on the first ‘+ add metric’ blue box to bring up your options. You can either scroll down through the drop down options or use the text field to find the Metrics we want (the latter is much quicker!). The first one we want is Visits, then we will add all of our individual goal completions (Goal1 Completions, Goal2 Completions, Goal3 Completions, etc), then the metric Goal Completions and Goal Conversion Rate so that we can see a total of the goals and their conversion rate for the number of visits each page received.
4. In the (green) Dimension Drilldowns section we want to add the dimension named Page. This will list all of our pages in the table allowing us to see the impact on each of the variations in our test.
5. Click Save

Once it is saved you can then view the custom report. The table at the top gives you a great overview of the statistics we have requested and the main table at the bottom will display the statistics for each page on your website that is affected by the goals.

Now what we want to do is quickly view the results for our Website Optimiser test pages in the top table and also explore their impact on the conversions made on other pages in the lower table. To do this, just enable the two Advanced Segments we created earlier alongside the All Visits segment if you would like to see a comparison to all visits to your website. I don’t recommend spending time analysing the page level data in the lower table but it can be useful if your Website Optimiser test is related to other specific product or service pages, so you can see its influence on those as well.

The Google Analytics analysis provides a solid overview of the impact your test is having on all of your goal conversions as opposed to just one. By just using a single goal as your measurement you may be negatively affecting many of your other ones, so always review all of your goal conversions once your Google Website Optimiser test has been completed.

Image Source

Vacant Goal via BigStock

Share this post