Perhaps you’ve run an A/B test before. You wanted to see which would result in more clicks: Headline A vs. Headline B. A red “Click Here!” button vs. a blue “Click Here!” button. A photo of a cat vs. a photo of a dog.
What you’ve done is an optimization test. It’s a simple form of testing — you’re tinkering with the variables to try to find the best possible combination of content.
But when I talk about testing, I’m talking about something different. A test is more than just tweaking stuff at the margins.
A good test starts with a great question.
Right now, I’m asking two really big questions at work:
1) How can we build a big, highly engaged audience through email?
2) How can we convert those readers into paying subscribers to our print or digital editions?
These are complicated questions. To get the answers, we’re going to run dozens of experiments over the coming months. We’ll test out new sign-up funnels to grow our audience; build new designs for our existing newsletters; create original content to live in our emails; launch entirely new newsletter products; and test all sorts of calls to action to see how, when, and why a newsletter subscriber might be willing to pay for access to our premium products.
But it all starts when you ask clear questions. Those questions help set the boundaries for your work, and make clear what you should be focusing on, and what you shouldn’t.
And a few months down the road, once we’ve used these tests to build out the framework to answer these questions, that’s when we’ll get into the nitty gritty of optimizing. We’ll run all sorts of little tests — button colors! subject lines! cats and dogs![1. OK, maybe not this one.]— to get to that optimal version.
But first, we have to answer these big questions.
———
That photo, “Science experiment” by Zyada, is licensed under CC BY 2.0.