Best Wishes for 2007!
The New Era of Email Marketing - Webinar

20 Questions to Ask When Evaluating an Email Marketing Agency

Paula Skaper is frequently asked what makes a good email marketing agency – and how to separate the good from the mediocre. While it’s true that anyone who has a basic understanding of HTML can code an email message, it takes a real specialist to create a message that works equally well in the 35+ possible environments, with graphics on and graphics off, with preview mode on and off. And even the most talented designer in the world won’t help if the technology is limited, or deliverability is bad. Or worse – you can’t know what’s working because the reporting is incomplete!

So she came up with a list of 20 questions to ask when evaluating an email marketing agency.

Design and Agency

1. What experience do you have in email creative and design for businesses in our industry?Email creative and design is radically different from any other type of marketing. The email designer has less time to capture interest than the web designer, and must do so under heavy restrictions and a completely different and exponentially more complex technical environment than the average website designer. The person designing your emails should be a specialist.

2. What project management process do you follow for campaign management and production?
Not as obvious a question as you might think. You’d be surprised how many agencies are weak in this area.

3. Did your last three email campaigns for clients all deploy on time?
A sure sign of project management problems – a partner without solid production management will have challenges here. So ask this question directly, then ask them to explain why those that didn’t were delayed. A good agency should be able to tell you that 2 out of 3 or better were right on time. And those that are delayed should be because of business reasons, and at the client’s request.

4. How far in advance of deployment will I see a proof?
You should see your first proof between 24 and 72 hours ahead of deployment, depending on how much input the agency has on creative. Any proof sent within 24 hours of deployment should be a confirmation message allowing you to verify that all your changes have been made and to authorize deployment.

5. What experience does your team have in publication management and direct marketing?
It’s been our experience that good email teams are passionate about publishing. They understand the cyclical nature of email marketing, they’re addicted to relevance and they respect deadlines.


6. What security measures are in place?
The answer should include information about redundant servers, firewalls, physical security of the location where the servers are housed, data management practises and encryption of logins and other shared data. Keep in mind that good security costs a bit of money – and the two things you’re most likely to compromise on when selecting a lower cost provider will be the flexibility of the tools and the security of your customer data.

7. How many messages can be delivered in an hour?
And what is average server uptime? Let’s take these one at a time. Messages/hour is a measure of bandwidth. Keep in mind that even if your list only has 2000 names on it, you’re hardly the agency’s only client. I’ve heard too many stories of companies that send an email campaign and it takes four hours to four days to get it into the hands of customers. When you lose immediacy, you lose some of the most important assets of email. The other big issue – server uptime. A good ESP will ensure uptime of at least 99%. That’s the same kind of uptime you’d expect from your online banking service. If the server is down once every two weeks, then guess what, so is your entire email marketing department. Now – if you don’t mind working on Saturday to make up for the fact that the server was down on Friday afternoon, that might not be an issue. Me, I like to have some options about how to spend my Saturdays.

8. Where are the databases hosted?
Many small agencies with proprietary tools will host data on their own servers. The upside to this solution is that it’s very often extremely affordable. The downside of course is everything we talked about in security and bandwidth. Small, local, server just might mean reduced security, lower bandwidth and less reliable uptime. Most successful small agencies work with an established partner to provide robust solutions to these problems at a pretty reasonable cost.

9. How is customer information captured?
You want to hear an answer that includes key phrases like “secure, hosted web form”, “link from your website” or “host on your website”, “surveys”, “contests”, “inquiries from your website”, “import a csv file” etc. The technology your agency uses should be flexible enough to give you plenty of opportunities to entice your customers to share more information with you, and to keep the information you have up-to-date. You may never need to use them, true. But then again, you may never need a heart surgeon either. It’s still comforting to know that if you do, the one at your local hospital has a medical degree isn’t it?

10. Are bounces and unsubscribes handled automatically?
The only acceptable answer is yes, absolutely. Anything else guarantees you’re wasting valuable human time doing an ineffective job of a boring task better handled by machine.

11. Tell me about some campaigns you’ve run that were more than “batch and blast”
And yes, say “batch and blast”. A good agency will know that you mean an email campaign that sent exactly the same message to everyone on the list. And they’ll see it as an opportunity to showcase the really cool, highly targeted campaigns they’ve run for clients. Beware of anyone who starts off talking about rich media – it’s cool, but if everyone gets the same flash video, it’s really just a fancier “batch and blast campaign”. What you want to hear are keywords like “triggered messages”, “adaptive sequence campaigns”, “conditional content”, “dynamic content insertion”, “forward to a friend”, “advanced personalization”, “multiple language delivery”, ‘a/b split testing”, “distribution groups” and yes, “targeted rich media”. Oh, and a good agency, will be able to explain those terms so that they make sense for your business.

Spam and Deliverability

12. Is email delivery success to all major ISP’s monitored in real time?
Major ISP’s doesn’t mean AOL, or Hotmail. Well it might – but it should also include the local vendors unique to your neighbourhood and those that are likely to serve your customers. If you’ve got customers in Quebec, you want to know what’s happening with Videotron. If you’re in BC, that might mean Telus or Shaw.

13. Is a Spam check tool available? Is it the only anti-spam measure taken?
Spam tools are great – they provide an immediate assessment of common triggers. But they’re hardly foolproof. Your agency partner should back up the results of automated spam check utilities with other processes and tools to ensure your messages are getting through.

14. Does the provider publish an SPF?
If they don’t know what an SPF is, thank them and end the conversation as quickly as possible. If they do, they’ll tell you. And talk about other similar programs, whether they participate and why (or why not).

15. What other deliverability initiatives are in place?
Every agency is unique, and the good ones will have systems in place that are entirely proprietary. They’ll have found unique ways to provide service to their clients that address any holes left in the processes above. Give them a chance to show off. But be prepared, this is one of those questions that’s guaranteed to include words like “proprietary” – which, by the way, is agency code for “something we do really well and we know our competition can’t do. In fact we don’t think they’ve even realized it’s an issue, so the last thing we want is for you to ask them about it!”


16. Is reporting provided in real-time via secure web login?
Hey, there’s nothing more fun than watching your open rate climb and realizing that people are actually reading, and yes, clicking through, to your campaign. Plus, if your boss is as type A as I am, he’ll really appreciate a quick email letting him know how things are progressing. Makes you look especially on the ball and informed too!

17. Is the information displayed graphically?
Face it, most of us don’t get that excited about spreadsheets. Brings back too many haunting memories of Math 12 or worse, Calculus! It’s true what they say, a picture IS worth a thousand words!

18. Can I drill down within an individual mailing to compare responses by list segment?
Wouldn’t you like to know if a particular group of your customers wasn’t all that excited about an offer? Or better, couldn’t wait to respond? This kind of information is the pot of gold at the end of the rainbow!

19. Can I obtain reports on the composition of my list based on both demographics and behavioural profiles?
If you know that 50% of your customers are in Quebec or Ontario and that region is hit with a nasty ice storm that shuts off power to the entire region, you could reduce your mailing costs by 50% simply by not mailing to those segments during the crisis. Of say your ESP advises you that AOL is having server problems and you have a campaign about to deploy. Your next step will depend a great deal on what percentage of your subscribers are AOL email addresses.

20. Is cross campaign analysis possible?
Email is a cyclical activity. One of email campaigns are rare – usually they’re tied to a larger program. And looking at that program from a variety of angles, aggregating data across campaigns and comparing activity between various emails is an important part of determining your long-term success. We like to call them trending reports, but whatever name you use the golden nugget here is that change over time is an important indicator.


Need help optimizing your email marketing results? Get in touch!
comments powered by Disqus