How to Build a Test Plan to Improve Your Email Program's Effectiveness

Here's an exerpt from SubscriberMail's CEO Jordan Ayan's book "The Practical Guide to E-mail Marketing" on email testing. Use this checklist next time you go into a meeting to discuss how to improve your e-mail marketing performance.

Step 1: Ask a Question

Start the testing process by asking a question. What are you hoping for? Determine a specific goal to accomplish rather than attempt multiple goals with one blanket approach. A series of small steps can be easy to test and analyze:

• I'd like to have more people open my messages.
• I'd like to have more people click through to my Web site.
• I'd like to reengage with historically inactive people.
• I'd like to have people click on a specific area, topic, or action.

Step 2: Form a Theory

Use your marketing experience and best practice knowledge to determine what aspects may make a difference in achieving the goal you've defined.

• I think people may be bored with my current subject lines.
• I think that the placement of the specific content may drive more people to action.
• I think that people may not understand this is from my organization and therefore will not interact.
• I think my calls to action need to be stronger.

Step 3: Create the Test

Set up your test, following best practices. Remember, you don't need to prove the obvious.

To optimize opens, I am going to test (one per test):

• From name
• Best day to send
• Subject line
• Best time to send

To optimize click-throughs, I'm going to test (one per test):

• Creative/layout
• Subject lines
• Copy
• From name
• Calls to action

To optimize conversions, I'm going to test (one per test):

• Landing pages
• Calls to action
• Creative/layout
• Subject lines
• Copy
• From name

Step 4: Segment the List

Choose the best list or segment to test, and split it (for that specific test):

• I'm confident this list is the most appropriate to prove or disprove my theory.
• My list is only large enough to do an A/B split.
• My list is large enough that I can break it into a larger control and other smaller test segments.
• My list is large enough that I can sample a percentage of my list to test.

Step 5: Measure and Analyze Results

Measure and analyze results to gain insight and prove or disprove theory. Accurately compile stats (to conversions). What does it all mean? Look beyond the numbers. Even small percentage differences can mean large gains in response rates:

• My opens increased ___%.
• My click-throughs changed __%.
• My conversions changed __%.
• Traffic to my Web site increased __%.
• My click-throughs were more focused on specific area, topic, or action.
• My click-throughs were spread out across areas, topics, or actions.
• Sales calls increased __%.

Step 6: Make Changes

Commit to making at least one change in each campaign.

• I need to change my from name.
• I need to change my subject line.
• I need to specific words.
• I need to subject line format.
• I need to add content.
• I need to decrease content and simplify.
• I need to increase clickable areas or clicks.
• I need to highlight actionable items more.
• I need to change copy.
• I need to modify layout.

Source: ClickZ

Comments

[Tim Wilson]

This is a great summary. One note on Step 5, though, is that it can be dangerous to simply look at a raw increase/decrease. It's much better to do this as an A/B test and apply a little bit of statistical rigor.

JT Buser over at Bulldog Solutions has developed an Excel-based calculator to do just that -- free and no registration required: http://www.bulldogsolutions.com/ExcelABSplitcalculator/