MD23: How to Create an AB testing Campaign With @ChrisGoward

Podcast: Play in new window | Download

Subscribe: iTunes | Android | RSS

AB Testing with Chris Goward from WiderFunnel. You can’t go wrong with Chris so keep reading/listening:

When online entrepreneurs and companies want to increase the revenue that their websites produce, the typical actions is trying to increase traffic, which is actually not always the best idea.

The truth is, it’s much easier (and affordable) to increase your conversion rates than to increase your traffic. This is why I focus so much on displaying different versions of my blog to different types of visitors.

I run tests all the time, and you should to (if you are at a stage in your business where it makes sense.

What You Will Learn About AB Testing

• Where to Start Your AB Testing
• How to Define Your Goals
• The Most Important Elements You Should be Testing
• How to Plan Your Campaign
• Tools And Resources For AB Tetsing

Resources Mentioned

9 Steps For Becoming an Optimization Champion

The PIE Framework

The LIFT Model

Smartduu For Personalization

Transcript

Hello everyone, welcome back to the show. Thank you so much for listening. And today we’re going to talk about AB testing with Chris Goward from WiderFunnel.com. Now, the reason that you might want to implement this strategy in your business, in your online business, your blog or your landing pages is because sometimes you set up goals and you’re not getting the results that you expected. And I get it. But instead of quitting, you could run AB test campaigns and figure out what’s not working. Perhaps you’re not using the right wording. Perhaps you’re not using the right colors, images or call to actions. So instead of just you know, killing the whole campaign and your whole strategy, run test. Run different test for your audience to figure out what’s not working and get your business going. Now before we get to the interview with Chris, I want to let you know that we are running another contest for you to win a lifetime premium membership to smartduu.com, the first and only provider of WordPress themes with website personalization features where you can display different versions of your website to different visitors based on their preferences, characteristics, geographical location etc. So if you want to participate heading over to WiderFunnel.com/LP/MoneyDiver and you’ll be on your way to win a lifetime premium membership to Smartduu. So without further due let’s get dive into the interview with Chris. So Chris, welcome to the show. Thank you so much for being here.

Chris Goward: Thanks Borja, great to be here.

Borja: Yeah. So for anyone who might not know who you are, would you mind telling us a little what is your – what was your journey towards becoming an optimizer so to speak and how did you end up creating your company?

Chris Goward: Sure. So I’m Chris Goward. I founded Wider Funnel in 2007 and before that I had taken a pretty winding journey to get there. I’d started in biochemistry back in the day and ended up going into web design. Taught myself web design back in the mid 90’s and started designing websites and building those for e-commerce website and really in the early, very early days of the web. And then went into marketing to understand how to persuade people and how persuasion works. That was sort of the root I took from web. That’s – I was always fascinated with business and wanted to understand how to make it better. I went into ad agencies and learned how campaigns work and how marketing persuasion happens. But more importantly, in that process really understood how agencies, the best agencies can get great results for clients but most agencies don’t. Most agencies are self-serving and are trying to maximize their billings and not interested in results. So I thought there had to be better way. And so what I did is combined the understanding of the scientific method, that was my background and I worked a lot in direct response, direct mail, off-line testing and I thought that must be able to be applied online to websites, nobody was doing that. They were just trying to back then in the early 2000’s trying to recreate TV with flash websites. They were terrible. And so we started Wider Funnel with the belief that we should be able to make improvements, dramatic improvements and test and prove that the new ideas actually worked better. And the side effect of that is that we’ve been able to test thousands of tests on hundreds of websites and actually identify the patterns and create frameworks that create reliable improvements and allow people to test even better. So that’s where we’ve focused at Wider Funnel since 2007. We’ve worked with Fortune 500 and all kinds of mid-size companies as well that have a lot of traffic on their websites.

Borja: So you had a hypothesis for each test that you would do and then you would say ”Right, this my hypothesis, I’m going to prove that it works.” How would you base and how would you define what that hypothesis should be?

Chris Goward: Right. So that’s where we’ve really focused a lot of our attention at Wider Funnel is on creating the frameworks and systems and processes that allow us to create better hypothesis and better results. And so we have frameworks, for example the lift model is the most popular conversion optimization framework anywhere. And essentially it looks at a high level at the 6 conversion factors that impact every website’s conversion rate. Your website right now is being impacted by these 6 conversion factors. And so by putting ourselves into the mind frame of the website’s customers and prospects, we can identify these 6 factors using framework thinking to look at the barriers and then create hypothesis that actually solve those barriers. Rather than just being – well, a lot of people are doing even if they’re testing, it’s just randomly testing ideas like “oh, maybe we should make the button bigger or maybe we should try adding some social proof or maybe a smiling person, a picture would help, right?” Those are just sort of random ideas but they may not actually solve any problems on the website. And so by starting with identifying the barriers, we create better hypothesis that get better results faster and more reliably and more importantly lead to insights about those costumers.

Borja: So when you said you get yourself in the mindset of your visitors, how do you define what the person is expecting to get? Do you use some sort of research or do you have like a pattern for, yeah for doing research and finding it out?

Chris Goward: Yeah, there’s two components to understanding the mind of the prospect. One is in research that happens before testing. So that maybe qualitative, quantitative research, maybe understanding that our clients already have about their costumers. They may have done focus groups or primary studies to understand them, to get in their mind frame. Or maybe looking at their analytics to understand how people are responding and what they’re – where they’re running into barriers. And if there isn’t a good research available then of course it’s conducting primary research. Surveys, user testing, focus groups – I don’t use focus groups that often but user testing and gathering that voice of the customer. Really understanding how they speak about the product and the service and the brand. And then so that’s one component, the primary qualitative research. And then the other component is taking potential hypothesis about the costumer, testing those hypothesis and then matching, looking at the results. Analyzing the results based on different variables and the variations to be tested to generate potential quantitative feedback about the costumers, ehy they’re responding. And that involves testing the value proposition and how it relates to that particular target audience and then matching that with the qualitative data that we already have to create even more powerful hypothesis on an ongoing bases. So we don’t actually know the customer until we’ve tested whether our hypothesizes are actually true. And then we can look for those ongoing patterns and when they start to become predictive, when we start to see some insights that then predict accurately test results; now we know we’ve got some robust insights and real understanding about the costumer.

Borja: Right. And tell us a little about your book.

Chris Goward: Yeah. So my book is called “You Should Test That” and it was published by Wiley a little while ago and in it what I’ve tried to do and I think what we’ve accomplished is to take a mix of strategy and tactics. So it’s very strategic in helping a high-traffic web marketer to understand how to test from a structural perspective, how to create processes and frameworks and strategies that work, how to use framework thinking but also very tactical, in a bunch of ideas to test, it really outlines the lift model in detail with those six factors plus all of the 27 sub-factors and with case studies. There’s 15 case studies in detail in there showing exactly the types of tests that we’ve run as well. So people come out of it getting inspiration as well as a lot of ideas to test and then there’s also the last chapter is about how to become a marketing optimization champion within an organization. So it’s very practical as well on how to get cultural buy-in and how to create movement in an organization, how to get things done on a practical level which is where a lot of people are sitting in their desks, trying to get things done.

Borja: Tell me about that last chapter which has – I think it has a very interesting name.

Chris Goward: Yeah. So it’s called how to become… I can’t remember the title exactly but Your Conversion Optimization Champion, yeah, the marketing champion in your organization. And so we talk about the 8 steps to become a marketing optimization champion. And actually there’s a blog post about is as well on Wider Funnel’s blog. If you Google what is it? 8 steps optimization champion Wider Funnel, there’s a blog post. But anyway, it outlines it in Chapter 13 in much more detail about the tips for how to create organizational buy-in momentum, how to get things done within an organization. You know, lot of our clients that are conversion champions are sitting in the organization knowing that this is an important strategy but they need to get senior level buy-in. They know that that’s important. They need to get the teams working together. They need to move this and make this a priority for decision making. Sometimes there’s cultural resistance to that to actually testing, to prove that ideas work. And a lot of organizations they’d rather just make gut feeling based decisions and follow the opinions of the – what we call the HiPPOs; or the highest paid person’s opinion. And to actually change that to being a data-driven organization where ideas are tested is much more powerful but a hard transition to make sometimes.

Borja: Okay, so let’s say we want to start our first AB test. What is the first step that you would take or that you would recommend someone to take?

Chris Goward: Well, yeah. That’s a good, great question because getting started can sometimes seem daunting. It’s a complex process, conversion optimization. Now, there are different ways of doing it. It’s easy technically and philosophically to get started with testing. You can just install the tool and make some changes and run a test. But usually that type of approach won’t get great results. What’s much more effective is starting by creating a system and a process that you know is reliable because often the first couple of tests might not give great results, it might take a long time to get results. You might not be testing the right things and so it’s pretty random just starting that way. So it’s important to start by building, identifying the frameworks that you need to use. To answer the important questions you’ll have to ask in creating that system. So for example, you’re going to have to answer the question where should I test. Well, you might have thousands of pages on your website. How do you know where to test? You need a framework to help answer that question, not just – don’t just Google like “top places to test on your website” because that’s irrelevant. Any tips you get are going to totally ignore your context as a website. So we’ve developed frameworks like the pie framework. And you can Google Wider Funnel pie framework and that’s available online. It’s free but that’s a framework that helps you to prioritize where to test based on the potential importance and ease of the customer touch points. And then you’re going to need to answer the question what should I test, right? So now you know where to test, what do you do there? What kind of hypothesis do you test? Well, again, if you go on and just look for top things to test and get a bunch of tips, it probably doesn’t apply to your situation. So use something like the lift model to prioritize and there are different frameworks that you can use. The important point is to define the process and define the way of thinking and answering those questions. And that’s – that involves framework thinking and looking for the right frameworks.

Borja: Right. And you have several frameworks I guess.

Chris Goward: Yes. We’ve developed a lot of frameworks that help with that. A lot of them we’ve published and they’re on the blog and they’re in the book and they’re available online. We have some others that we use internally here. But – and the process may be varied with each organization. But there’s research showing that companies that are doing conversion optimization – there’s a study by eConsultancy actually – companies that have a structured process for their optimization are twice as likely to report large increases and sales from their program. So developing that structured process however it’s formatted is the first step you should take. And we publish a lot of information about how we do it at Wider Funnel, too. It’s called the Wider Funnel system and it starts with strategy and then goes through an iterative 7 steps testing process. And then of course that’s available on our website as well but however it’s structured, there’s a variety of ways of thinking about it, the process is key.

Borja: Okay, tell me a little about that 7 step testing process.

Chris Goward: Sure. Yeah. So if you go to essentially what it is, is it involves a process of starting with hypothesis, starting with understanding the customer, creating hypothesis and then creating a test plan. We call it a Funnel Experiment Map. That’s where you take the hypothesis from your analysis of the page, right? So start with a lift analysis, create hypothesis from that analysis, create a test plan, the documents and make sure that everyone is in agreement on how the test is going to run and then flash out that test plan into actual variations that match the hypothesis. So the hypothesis need to be something that can be tested, so step 4 is doing the design and copy, creating these variations. Step 5 is actually implementing tests technically, step 6 is launching and monitoring the test to make sure that it’s running properly. And then the most important step is step 7 which is analyzing the results. And often that’s skipped where people get a winning result, they get excited, they implement it, they’re happy, they’ve got a winner, maybe a conversion rate lift but they’ve missed a whole bunch of important insights that you can gain from looking at the differences between the variations and asking why. Why did one perform better than the other? Because then that leads into potential insights that can lead into better hypothesis in the next round of testing. So it’s a continued cycle through those 7 steps.

Borja: Yes because I mean you always have to improve upon what you have. So yeah, you have to create a cycle of testing, optimizing, improving. You can never stop. So when do you start analyzing what your goals should be because I’m guessing you might have an overall understanding of what the goals of your website should be. But how do you identify and define what those goals should actually be?

Chris Goward: Yeah. So during the first phase of an engagement with one of our clients we’ll spend time while understanding the costumer, also understanding the company and what their goals are. And that’s a very important step. It’s a good question because if you’re testing and tracking the wrong goals at a technical level, you can actually optimize very fast in the wrong direction and hurt business results. So it’s important to know what are the technical goals that actually represent revenue. So that when you’re testing you’re actually optimizing to maximize revenue and not just indicators that may be biased.

Borja: So what are some different sort of conversion objectives that you use typically?

Chris Goward: Well, yes, that depends on the type of business. If it’s e-commerce often it’s very clear that it should be sales and so sales conversion rate and average order value, so total revenue per visitor. Those types of metrics are what we’ll look at. If it’s a lead generation it will be lead generation and conversion rate but also quality of the lead. So we’ll want to look at the post lead conversion quality scores. If it’s affiliate marketing it’ll be of course revenue per visitor, it will be the actual affiliate dollars as close as we can track to, it’s a revenue there. And in some businesses it will be, it may be something different depending on how they generate revenue. For publishing sites it sometimes maybe eyeballs, maybe clicks, maybe views but in most cases we want to track to producing lift in direct revenue. That’s when you have the – we have the most fun and know that we’re really impacting the business.

Borja: Right. So tell me a little about micro-conversions.

Chris Goward: Micro-conversions? Yeah. So that’s related to making sure that you’re tracking the right thing. And so micro-conversion is typically defined as an incremental conversion that leads toward the ultimate goal conversion that generates revenue. So in an e-commerce site a micro-conversion might be adding a product to a shopping cart, right? That gives a purchase indication but it doesn’t actually generate revenue. And a lot of people are optimizing for micro-conversions and the reason is that there’s usually a lot higher volume, so you can get test results faster, right? You might get 10 add to carts for every checkout completion. So maybe you’ll get a much greater volume, you can run more tests, get better results and keep going. The problem is that micro-conversion don’t always directly correlate to ultimate conversions. So we’ve often seen when we’re tracking both that we can lift the micro-conversions with a variation but actually have a corresponding drop in completed conversions. And so we never want to track micro-conversions for conversion optimization. Now, by all means you want to look at that for analytics that gives hypothesis but for conversion optimization use different goals than for analytics. That’s an important distinction that a lot of people don’t really understand. Is that for analytics you want to look at everything. Look at how people are using the website, get a rich understanding but conversion optimization needs to be focused on what drives revenue.

Borja: Of course. Because then you’ll be chasing – otherwise you’ll be chasing your tale…

Chris Goward: Exactly.

Borja: … finding no results at all and end up with real frustration there. So I guess something that people could find difficult would be to define what elements on their website should they start testing. Is there a sort of priority for elements like buttons or colors or headlines?

Chris Goward: Not really. And there are a lot of elements that tend to be tested more often. But the elements that need to be tested depend on where the barriers are in your website. So it depends. The 6 conversion factors are where you start to understand the elements. So you want to look at the value proposition. Is there anything on the page or missing from the page that is communicating value proposition? Is there something that’s hurting credibility as part of value proposition? You want to look at the relevance. Is there something on the page that’s not relevant to the message. Well, then that’s something that should be tested, maybe being removed or changed, right? Is there something unclarity that’s making the communication unclear or that could be added to increase clarity or changed? Well, that would be an element to look at. And then there’s urgency, distraction and anxiety that also need to be looked at. So if there’s any element that’s affecting any of those 6 factors that’s where you want to focus attention. Now, to be fair there are some areas that end up being tested more often because they have a greater impact in most cases. Usually that’s related to the fold. And there’s a lot of debate of the fold and where it actually sits now with so many different screen sizes but the most important thing to understand about that is that what a visitor sees immediately on the page has often the greatest impact on conversions because it communicates the immediate message. And that’s all about clarity and relevance. And then further down the transaction there might be other elements that are very important that are actually transactional like the form fields and the buttons and credibility indicators, all of those types of things. So there are some typical patterns but really where to focus is very customized to the business in terms of what are the barriers for those prospects in taking action.

Borja: Well and when it comes to, when it comes to – okay, so you know, assuming you know what elements you’re going to be testing on what should you base your tests, meaning what should you look for in your audience – sorry, in your visitors? Is it – do you look at cultural background or demographics to define how these alternate versions should be?

Chris Goward: Yeah. A rich understanding of the audience is helpful and there are many different aspects they can play a role in that. Cultural background is important, demographics, psychographics, experience with the category, the purchase cycle length, the type of decision making involvement, right? How many factors they consider. All of these things are important. And there’s often nothing that can be excluded that isn’t valuable. You have to just know what will lead to important hypothesis about the audience. And so you might look for patterns, for example if we see an IT decision making audience, we may find that there’s some similarities with – between them and house buying – house purchasers, home purchasers because it’s a complex decision cycle. It’s got longer term, there’s more factors they are looking at and so we may look at, well, here’s some hypothesis that worked for the IT decision maker, maybe we can take those and apply them to home purchasers and see if that works there, too. There may be a pattern, there may not. And so the job of a conversion strategist, an optimization strategist is sometimes messy because there aren’t strict rules. It’s all about finding those patterns, finding potential similarities across industries and looking for rich ideas that can be leveraged in whatever data we understand about this target audience.

Borja: That’s powerful because well, I mean what happened to me when I first started doing my first AB test in my blog I didn’t even – I wasn’t even testing the landing page, was that I found out that some versions that I would create would perform better to some groups of people, right? Because it turned out, it turned out that I had several groups or segments of people or buckets or however you want to call it. And the number one thing that I found out was that different colors would work for different types of people. So I would have several tests with different results and I would have like one overall result. But what if I wanted, I mean what if I wanted to also serve that little segment that responded better to the color that in general wasn’t the winner. So I went to a professor of – when I was doing my MBA, I went to a professor and I asked for some cultural – colors and culture and he pointed me to a research that demonstrated that certain cultures might react differently to different colors. So based on that I created a bunch of tests for different audiences and I had to neglect the audiences that were minorities in order to have one general, one definitive version created. And that’s what let me to create my product, my plug-in which my audience knows about, the personalization plug-in. So that I could display different versions of my sign-up form simultaneously to different segments of my visitors. Do you find any value on creating these segments?

Chris Goward: Yeah. Absolutely. Segmentation and personalization are important parts of the work that we do in optimization at Wider Funnel. We look at creating personalized content or content segmentation using three methods. One is best guess segmentation where you believe you understand something about the audience through research and you create hypothesis to back that up and then test it to validate. The second is user selected segmentation, so that might be where you have the buttons or segments clearly defined on your website that help them to self-identify and say “Oh, yes, this is the type of – I’m a small business owner,” so click on that button and get information for small businesses. Or “I’m looking for a discount carpet,” so click on that one and find the discount section. That gives a user selected segmentation method they can drive customization. Or it might be post test segmentation which is I think what you’re referring to is when you’ve done some testing and then you create variations, you test it and you look at the data and see that different variations respond – perform differently for different segments. And you can dig into the segments and find out which ones actually perform differently in different variations and then that can lead to further hypothesis to test and validate that. And the technology is available now to make that easy do. You know, you’ve got your plug-in to do that. There are various testing tools that do that sort of thing, But really, most importantly the question is how do you create hypothesis that lead to valid segments that are important enough to support. Because as you know, once you have segments now you have multiple versions of your website that you need to support and continue to operate. So you need to be careful about creating segments that are important enough.

Borja: Yeah, you have to keep track, definitely. Otherwise, I don’t know, a year from now you’ll find out that some people are getting some content that you completely forgot about. It might waste some results right there. So what are some conversion mistakes that you typically see people do?

Chris Goward: Yeah, there’s so many of them. But I think it’s important to, as I said before, have the right process in place, the right structure to creating the right hypothesis, using framework thinking, knowing how to interpret the data so that you know that you’re getting valid results. The only thing worse than not testing is tracking the wrong goals, testing inaccurately or not understanding statistical significance and so believing that you’ve learned something that you actually haven’t. You need to make sure that you use a testing tool that’s designed for AB testing, controlled testing. And that all of your tests are run concurrently, not using the before and after method or the pre and post method. But making sure that all visitors are randomly selected to be in different versions, they’re tracked accurately with a proper testing tool and that the stats are followed correctly, that you run the tests long enough, I mean all of these things. There are just so many areas that you can run into problems. And that’s why often, you know especially large organizations they don’t want to make mistakes with all of this revenue that’s flowing through their website. They’ll want to bring in experts that have the process and have made all the mistakes before, years ago and know how to do it the right way so they’ll avoid those things and get to a better result faster.

Borja: Right, perhaps some bench-marking might help avoid the mistakes that other people have done in the past.

Chris Goward: For sure.

Borja: So Chris, I try to make sure that my audience understands how important it is to know your – their audience, right? Their visitors, their target customers. And for that I ask all of my guests the same question so that they see that everyone that comes to this show understands who they’re speaking to and who they’re targeting. As a business owner who is your ideal customer and what do you do to target that person?

Chris Goward: Yeah so we have actually four different personas that we look at at Wider Funnel. And we don’t publicly share them but generally the patterns across them are that they’re a manager, a VP or owner of a high traffic website that understands the importance of data-driven decision making, they have a culture of looking to data to make the decisions, they know they need to test, they have enough traffic to test, they have the resources to be able to do it and can make an impact on their website, there’s a champion in place and that they’re interested in learning. These are people who value thought leadership and being leaders in the industry. And so that’s the profile of the type of people that we’re looking for.

Borja: Wonderful. So Chris, well, thank you so far for everything that you have shared so far. Where can people go to find out more about you and stay in touch with whatever you’re creating and whatever you do?

Chris Goward: Yeah, sure. So the best way to keep updated and to keep the best of the web on optimization strategy is at our blog, at WiderFunnel.com/blog and we publish a lot of information and case studies there. There are some new research reports coming out very soon and white papers that will be really interesting with some of the latest data. So they can sign up at WiderFunnel.com/blog and make sure they get the latest info.

Borja: Wonderful. Well, Chris, thank you so much for coming in to the show. I hope to have you again in the future.

Chris Goward: Thanks Borja, it’s been great.

Borja: Yeah. Talk to you later.

Chris Goward: Bye.

Borja: Alright guys, I hope you enjoyed that interview with Chris Goward from WiderFunnel.com. Now before you go, I do want to remind you about the contest that we’re running, WiderFunnel.com/LP/MoneyDiver. I hope you took a lot of notes, I hope you learned a lot. You can also head on over to rebelgrowth.com/episode23 for the show notes and you’ll be on the way to learning how to create your first AB testing campaign. Again guys, thank you so much for being here with me, go out and implement and keep diving.

I appreciate your interest, so thanks.