There is a simple way to explain the importance of a testing plan: It's the only way to make the leap from what you thought would work to what you know will work. The information that testing generates will help you understand what's best for your customers in real-life situations and gauge how they are interacting with your brand. This is the foundation every email marketer needs for success.
In this article, Tim Underwood lays out four key steps to help you get there. Here’s a summary:
1. Plan ahead
First, compile a list of questions you need to answer:
No doubt when you brainstorm around these questions, you will come up with great ideas, and some will really push the boundaries of your strategy. However, only ask questions if you are prepared to act once you uncover the answers.
Last but not least: test one variable at a time and make absolutely certain you can isolate that variable.
Now develop a list of variables to test. Don't forget you will need to test everything twice to be certain of your results. A great strategy is to identify one stream of communications -- for example, your regular monthly newsletter -- and split it into two versions. Whichever comes out on top, use that as the control for your next test. Testing should be a continuous activity, and by using this method you can ensure your marketing is always evolving.
You need to know which success metric is the most relevant to your test; there will always be a primary and a secondary. The primary metric is the action closest to the variable, and the secondary metric as the next action after that. This is really as much as you can hope to have influence over; any action further removed from your variable has been subject to too many other factors. You can't really attribute high conversion rates to a great subject line.
Lastly, it's essential to make sure your answers are statistically significant. Even two equally sized random splits will give slightly different response rates, so it's not enough to look at the highest value and declare it the winner. You will need to do some calculations to make this decision.
4. Take action
Testing initiatives can become very involved, so don't lose sight of the goal. Once you've got your first wave of results, make sure you share them with everyone involved and take action on them. But the job doesn't end there. Monitor your results to ensure your new best practices stay fresh and are still producing the desired results. Again, a good testing plan never ends. Your customer base never stops evolving and neither should your marketing plan.
David Silverman on the Harvard Business Blog has put down some key points when revising copy for your next email in his post: 'How to Revise an Email So That People Will Read It'
1. Delete redundancies. Say it once. That's enough. If you're repetitive, the reader will stop reading and start skimming. (Like you probably just did.)
2. Use numbers and specifics instead of adverbs and adjectives. "The project is currently way behind schedule on major tasks," is not as clear as "The project is 3 weeks late delivering hamburger buns to Des Moines." (If you don't have numbers, still get rid of the adverbs and adjectives.)
3. Add missing context. Does your reader know that hamburger buns in Iowa are required for the company to collect $37 million? If you're not sure, remind them.
4. Focus on the strongest argument. Should those hamburger buns get shipped because the delay is embarrassing for the company, because it's costing children their lunch, or because it's costing the company tens of millions of dollars? Maybe all three, but one of those reasons (and it depends on your reader) will be enough to get buns on the road.
5. Delete off-topic material. The best emails say one thing and say it clearly. One-subject emails also make it easier for the recipient to file the message once they've taken action, something anyone who uses Outlook to manage tasks appreciates.
6. Seek out equivocation and remove it. "It was the best of times, it was the worst of times" works for Dickens, not status reports.
7. Kill your favorites. Is something in your text particularly pithy, amusing, or clever? Chances are, it's not. If it sticks out, it's probably a tap-dancing gorilla in boxer shorts — hilarious when you thought of it, embarrassing when it gets in your manager's inbox.
8. Delete anything written in the heat of emotion. Will this sentence show them who's been right about the hamburger buns since the beginning? Yes? Cut it.
9. Shorten. Remember the reader struggling to digest your message on the run — a BlackBerry or an iPhone gets about 40 words per screen. What looks short on your desktop monitor is an epic epistle on their mobile device.
10. Give it a day. With time, what seemed so urgent may no longer need to be said. And one less email is something everyone will thank you for.
After reading this article my colleague Skip Fidura posted a great piece on time of day testing in email marketing:
The interesting thing is that we have come full circle. In the early days of email it was a free for all but people quickly realised that email was all about getting the right message to the right people at the right time. It was the last part that caused all of the worry.
How would we as marketers know when somebody was likely to be receptive to our message? So, we assumed that all consumers care about is the weekend and applied some "logic" about their behaviour as it related to the weekend and decided Wednesday would be the best day to send.
Some of the more clever email marketers realised that if every body was sending on a Wednesday it would be too hard to achieve cut-through so they overlaid this thought with the previous thinking on the importance of the weekend and said Tuesday was better (although some argued for Thursday).
Throughout the Great Day of the Week Debate, individually we struggled to answer the question, "I have changed to the 'optimal' day and my results have gone down." Luckily as marketers we had a ready made answer to this dilemma. "Maybe our customers are different. We should test it." These individual conversations eventually lead to a groundswell until most everybody was in agreement that we should rigorously test for the optimal day of the week and hey, let's throw in time of day as well.
In this article, Chad White provide some tips for testing your subject lines. He also posted some great tips if you're looking for inspiration for your next subject lines:
Whether you're sending a promotional email or an email newsletter, your message has specific sales or marketing goals. Most commonly the main goal of your message is to get readers to register for an event or make a purchase, or to build a relationship with readers, which hopefully leads to them making a purchase. Since readers can't take the desired action within their email, you send them to a page where they can take that action, your landing page.
Too often marketers send readers to a generic, existing page on their website instead of designing a page that is specific to the audience and the goal of their email message. Worse yet, links go to a page that doesn't house the information readers are looking for, forcing them to click further to find what they're looking for and ultimately complete a transaction. Conversion rates suffer as a result. However, a properly designed landing page can greatly increase your conversion rates.
To maximize the performance of your email messages, consider creating effective landing pages following these tips as a guideline:
Define your conversion. Before you start to design your landing page, define that page’s conversion activity. For a newsletter landing page, the conversion activity is entering an email address into a form and clicking “Accept.”
Do a little research. A little research goes a long way. Figure out what your visitor is looking for and what offers work. Build a profile of your ideal visitor. Keep this person in mind when creating your landing page. Do not construct the page for anyone else—generic and broad pages are proven to fail—and keep everything “on target.” Your email message already funnels traffic to your landing page, so visitors are expecting a very targeted message. Tailor the pages to them.
Align your landing page with the main goal of your email message. A common mistake is to design a landing page that is not specific to the goal in order to appeal to a generic visitor or prospect. Just the opposite, your landing page should be highly focused. Think about your prospect and what you want him or her to do. Then design your landing page around that one goal.
Eliminate unneeded elements. Distractions kill conversions. Strip any unneeded elements from the page, such as links to other resources, or other pages on your website. The consistent navigation bar that makes sense on a website doesn't necessarily make sense on a landing page. You don't want visitors to surf your site. You want them to complete the one action on which your landing page is focused. So if you can, remove the navigation bar. Of course, don’t remove it if it is essential to the conversion process. Remember your message, and if a link has nothing to with it—chuck it!
Match the message and elements of your landing page with your e-mail message. For best results, repeat the headline from your e-mail at the top of your landing page. This gives visitors a feeling of familiarity and comfort. It tells them that they have arrived at the right place.
Keep it short and sweet. If at all possible, keep all content above the fold. If visitors have to scroll to find what they're looking for, chances are they won't find it. And if they don't find it, they won't convert.
Design for scan-ability. Just as prospects won't read every word of your email, they won't read every word of your landing page either. They will quickly scan the page, looking for further information that compels them to take an action. Make sure your headlines, sub-headings and graphics enable visitors to skim your landing page's content to quickly learn what you're offering and how it will benefit them, and most important, make sure it clearly points out how they can easily take the desired action. If your goal is sales, point readers to a page where they can place their order right there.
Provide conversion exits. Make it easy for your visitor to convert. Place conversion exits above the fold and at every scroll-and-a-half of screen space.
Important elements above the "fold"
Pay attention to the virtual fold (the bottom of the screen before scrolling). Place enough content above the fold to allow your visitor to make a decision about continuing on the site. If a visitor has to click or scroll to figure out what your site is about, the only thing they’ll click is the back button.
Lead the eye. Use typography and color to your advantage. Lead the eye along the page towards the conversion exit. Thoughtful use of whitespace, large copy and graphics can make a long page seem much shorter than it really is. Be careful though—a great image will demand a lot of eye time and if misplaced can ruin the flow of your message.
Place the important stuff (whether it’s your copy or your image) close to the middle, and never distract your user from that focal point. Avoid putting interesting material in sidebars. This pulls the eye away from the main body. If it’s interesting and valuable, keep it close to the center and use it to direct the eye.
Optimize your forms. Make the input cursor hop to the next field after a user finishes the current field. Allow the user to tab around fields. Auto-populate any fields you can.
Keep your form brief, and thoroughly pre-test it. Remember, this form may be just the first step of your sales process. You don't want to scare off prospects by asking for too much information. So remove all unneeded fields. Don't ask for city/state/province if you ask for a Zip or postal code. Focus on the essentials. If you’re asking users to register for a newsletter, ask for only an email address. You don’t need their name now. Get rid of the reset button. It’s dangerous for both the user and you.
Test, test, test. Just like you should test your email creative, we recommend that you test different elements and copy on your landing pages to learn what works best. You may want to consider web analytics software that allows you run A/B tests or use multivariate testing to test multiple items and combinations at the same time.
Go over a checklist with your design team:
Track results. If you don't know how your landing page performed, you can't tell what worked and what didn't work and you can't make necessary improvements to increase performance.
Landing pages are important to maximize the performance of your email marketing campaigns. Your email message and landing page should work together closely for best results.
In this article, Aaron Smith explains how to improve the results of your email marketing campaigns through testing.
"Meaningful testing is easier to conduct than you might think, and needn’t be disruptive to your regular production process. Effective testing doesn’t have to involve complex multivariate data analysis, or the creation of multiple versions of disparate designs. In fact, most times, simpler A/B tests will yield more effective results, and just as importantly won’t drive your production team up the walls" he says.
There are several keys to successful testing:
1. Make tests controlled and easy to understand. Simple A/B tests, comparing just one changed element against another, are the easiest to understand and act upon. If you test five different items within the same email, you may need to hire a rocket scientist to figure out the results — and good luck trying to explain the results to your boss and co-workers.
2. Test frequently and make it a consistent part of the process. Optimally, you should include a simple test in every mail, but at the very least, create a regular schedule for testing (for example, every two weeks or once a month) — and stick to it!
3. Most importantly, use the results to inform the process. Great tests won’t do you any good if you don’t act on the results. After sifting through the results, sit down with your team to review the test, and determine how the information will be used going forward.
4. Archive the results. Post the results of your tests in a document, wiki or spreadsheet that everyone on the team has access to. Not only will it help you remember the results of important tests six months after the fact, but it will also be an invaluable training aid when bringing new members onboard.
5. Don’t get complacent. We live and work in an ever-changing media world, where today’s best practices become tomorrow’s pitfalls overnight. Don’t be hesitant to run the same tests every few months. Effective subject lines, for example, are a constantly moving target, and what works well today may not be very effective in three months.
With the above key elements in mind, here are a few simple ideas for tests you can run within your own email program:
These are just a few ideas — the possibilities are truly limitless.
Source: Email Insider
I got an email yesterday about a new tool developped by the folks behind Litmus (a tool that shows you what your email will look like in different browsers). The new tool is called Fingerprint and it shows you which email clients your readers are using. I tried it out when I was sending out the newsletter for the Email Marketer's Club yesterday and I was very impressed with the results.
Basically what happens is that you get a piece of code (an image tag) to add to your email and about an hour after sending your campaign you get to see the first results.
I would advise everyone to try it out to get a better idea of which email clients you should be designing your emails for. Is Lotus Notes used by a big part of your audience? Do they use version 6 or 8? Do they use Gmail in Firefox or in Internet Explorer? Do they view your email on their iPhone...
Drop everything you're doing and check it out now! Really. I'm not kidding.
And in case you're wondering: I didn't get paid to post this. I really believe in this product and I think they are selling it way too cheap, so you want to get in and start using it before they realize what goldmine they are sitting on ;-)
Consider this situation: You run the marketing department for an e-commerce outdoor outfitter, and your team has decided on a subject line and special offer for your next campaign. The variable you're trying to settle is whether the main graphic should be a product photo or one of the company president announcing the offer. This is the perfect time to employ an A/B split to find out which works better.
Creating campaign A
First create the e-mail using product shots — in this case, one that includes an array of men's and women's shoes. Then, create a landing page that reiterates the president's sale coupon offer, in addition to using the same product imagery.
This will be your ‘A' campaign.
Creating campaign B
In order to isolate how much impact the imagery has on the recipient's actions, campaign ‘B' will be an exact replica of ‘A,' but in place of the shoes photo, use a photo of your president. The landing page includes the coupon offer and that same picture.
Randomize, send and analyze
Now it's time to send out the e-mail. If your e-mail service provider offers you the option to randomly select half of your list, use it. If not, simply cut your list in half alphabetically or by order of the opt-in date, bearing in mind that the results will be slightly skewed because of these variables.
After you've sent the mail, look at your metrics. First, check your e-mail stats to determine if your click-through rates are better for either campaign. Then, use your Web analytics application to see how visitors from campaign A behaved differently from visitors from campaign B. And although you'll want to concentrate on conversion rates, you can glean knowledge from other metrics, as well. How did the visitors' average time on site differ? Did visitors from one campaign view a vastly different number of pages than visitors from the other? Contrasting and comparing how each campaign's visitors behaved lets you draw conclusions about what worked and what didn't.
In our imaginary scenario, let's assume that you found that campaign B (featuring the picture of the president) yielded 1.5 times better purchase conversion than campaign A, which featured product shots. Your customers showed you that seeing your company president made them feel more compelled to purchase.
A/B testing may be a bit more work on the front end of an e-mail campaign, but the information and insights you gain from it can make a huge difference in your click-through rates, as well as your conversion rates. So the next time your team can't decide between two good ideas, split the difference and let your customers show you.
User experience testing is a powerful e-mail optimization technique. Unlike multivariate and A/B testing, it tells you more than what users do with your e-mail. It tells you why they do it. But there are essential best practices to consider if you want to get the most out of testing. Here are a few.
For those who lack access to a statistical software solution that will also interpret the results, A/B testing is challenging at best. Testing for statistical significance is important, but just as critical is an understanding of your tests' validity and how to use the results to improve future campaigns. To answer these needs, Bulldog Solutions offers two Excel-based A/B calculators. You can view a demo or download these calculators here.
Here's an exerpt from SubscriberMail's CEO Jordan Ayan's book "The Practical Guide to E-mail Marketing" on email testing. Use this checklist next time you go into a meeting to discuss how to improve your e-mail marketing performance.
Step 1: Ask a Question
Start the testing process by asking a question. What are you hoping for? Determine a specific goal to accomplish rather than attempt multiple goals with one blanket approach. A series of small steps can be easy to test and analyze:
Step 2: Form a Theory
Use your marketing experience and best practice knowledge to determine what aspects may make a difference in achieving the goal you've defined.
Step 3: Create the Test
Set up your test, following best practices. Remember, you don't need to prove the obvious.
To optimize opens, I am going to test (one per test):
To optimize click-throughs, I'm going to test (one per test):
To optimize conversions, I'm going to test (one per test):
Step 4: Segment the List
Choose the best list or segment to test, and split it (for that specific test):
Step 5: Measure and Analyze Results
Measure and analyze results to gain insight and prove or disprove theory. Accurately compile stats (to conversions). What does it all mean? Look beyond the numbers. Even small percentage differences can mean large gains in response rates:
Step 6: Make Changes
Commit to making at least one change in each campaign.
To reach optimum returns on your emails, invest some time in filtering your email strategy through your brand standards and your marketing goals to arrive at a selection of possible messaging approaches. Then test the variables, using a basic A/B split or any other segmentation that is important you.
Here are a few things you can test:
What offer will move your audience closer to a purchase? You may intuit the answer, or have to follow a business directive to make a particular offer, but by all means test themes and variations to create an offer the audience can't refuse. (Be careful of too much product discounting unless you want your brand to be associated with "cheap.")
2. Audience empathy
Test ways of connecting on an emotional or intellectual level with your prospects.
3. Tone and manner
While these must follow your brand standards, email is inherently a more personal medium than many other channels, so you can test some variables here.
Where the offer appears may make a difference in how it is understood and acted on.
You can test which copy approach yields the best results simply by tracking click-throughs and conversions on sent emails. To dive deeper, you can also perform other types of research. Use focus groups or surveys to test the impact and takeaway of your offers and the way they are worded. You can also do eye-tracking studies to find out how readers scan your emails. When you learn what path their eyes follow, you can maximize the effect by placing your key inducements, either copy points or visuals, along the same path, in order of importance.
There are almost always several good ways of writing and designing any advertising message. If you try to test every possible combination of variables, you'll go cross-eyed. Apply lessons learned from your marketing in other channels to your email advertising, and vice-versa, and you will find the options narrowing nicely into a powerful set of guidelines you can use creatively again and again.
Source: Email Insider
In this article, David Baker provides a simple formula to testing email creative:
1. Draw a wireframes of several templates (they vary by purpose). These are simply boxes designed for the optimal width and length. Rule of thumb is, keep promotional messages and email that is designed for early lifecycle simple and straightforward and minimize the length. Newsletters and community publications can support longer-form wireframes.
2. Assign a description to each box and bullet the things that you can potentially test within those guides. For example, if your top box of 50x 700 is for “Click to add your email address to the address book,” think about what else you could test in this area. (User Name, Promotional Message, site reminder message, or leave it out altogether). If it’s a header image or text block, then think about testing typographic treatments, background colors, blending with imagery and replacing with imagery.
3. Sit down with your designer and walk her through the wireframes and what options she has for each section. Ask her, if she had two things to test in each box, what would she test — and if you tested them and they worked, what would that do to help streamline creative next time?
4. Lastly, show her past results of emails that performed, which links performed best (in a visual format). Most email systems will give you a click map overlay report to show clicks by popularity and color code.
Do remember, you aren’t limited to the email only. The email is designed to get the receiver from the email to the landing page, so test the exchange between the two. You may be surprised at what can be left out and provide better value at the point of conversion.
Source: Email Insider
Creating successful and well-received email campaigns is every marketer's dream. Ensuring your campaign gets the best possible response should involve some measure of testing.
According to a 2005 JupiterResearch report, marketers who used testing were twice as likely to attain conversion rates of 3 percent or better. Despite studies showing that testing yields positive results for marketing campaigns, only about 40 percent of marketers do it, mainly because it’s perceived as difficult.
In this article, Silverpop offers a couple of easy tactics to get your testing efforts underway.
Step 1: What to Test
Testing is completely dependant on what you’re trying to accomplish. Is your goal to improve open rates? Perhaps it’s to improve click-through rates. Whatever your goal is, your testing efforts should center on tactics that will help you in those areas. Factors marketers typically test include:
Step 2: How to Test
Conducting your test doesn’t have to be a daunting task. Follow these tips to get good, representative results for your efforts.
Whatever factors you test, don't waste your efforts testing small differences, advises Silverpop email communications strategist Stephen Guerra.
“Concentrate on testing big changes—don’t test a blue versus green background color. Instead, focus on things like a four-column format versus a postcard format. Or, test an offer of 30-percent off versus $50 off a purchase,” he said. “Don’t focus on testing a single word change in the subject line, but rather test completely different subject lines for a more statistically relevant result.”
Step 3: How to Understand the Results
The goal of testing is not just to receive high response rates and drive conversions for a single mailing. Your goal is to learn as much as possible about your recipients and use that information in future mailings to improve relevancy for greater long-term success.
Testing can allow you to better understand individual preferences within your recipient database and directly improve relevancy. With testing, you’re able to determine which groups of customers prefer specific offers and, using that information, further your campaign goals.
It's every email sender's nightmare to launch a bug-filled campaign where everyone will see your mistakes. But, if you exercise strict quality control all along the production line, you'll reduce your potential exposure and send campaigns with confidence, even the last-minute ones.
Stefan Pollard shares this 10-Point Checklist:
I just added the following checklist to the Email Marketer's Club Wiki. It contains the things you should check before you send out your email campaigns. I'm sure I've missed stuff, so feel free to help build a thorough checklist here.
Do you think something's missing? Add it to the checklist here.
Without proper testing, there is no way to tell what your customers are actually seeing or how they are responding to your message.
In this article, Spencer Kollas tells us to start with establishing a comprehensive plan that outlines what you want to test, what you will consider a success or a failure and how you will measure this.
What can/should you test?
Source: iMedia Connection
1. Be sure to send all e-mail at the same time. You don't want the day or time of the send to influence results (unless that's what you're testing).
2. Make clear notes about what you're testing and which group is getting the "business as usual" version. This makes it easy to quickly determine a winner and implement what you've learned.
3. Wait at least 48 hours before declaring a winner. Early results aren't always accurate. Often, what's ahead after an hour doesn't maintain its momentum.
4. Make sure test groups are of significant quantity. You can do an analysis to determine statistically significant sample size and margin of error. Or you can use these rules of thumb:
Continue reading here and learn more about the most common things to test in email marketing: sender lines, subject lines, and the body of the email and the landing page.
What if you could make a slight change to one of your emails and get a 20 percent lift in opens or a 10 percent increase in sales? You would do it in an instant. But how do you know what element to change? That's where testing comes in. Email marketing makes it easy to quickly test important elements of your email-at very little or no extra cost. With testing, you can find out what factors influence the success of your email. Follow these five steps to create an effective, measurable test.
Step One: Decide What to Test
Because testing with email is so easy, it's often tempting to test many elements all at once. You should start by testing just one. Why? If you test more than one element in the same email, it is challenging (and sometimes impossible) to determine what exactly influenced the response. Here are some easy and telling tests to start with:
Fill out my online form.