This week I attended a demo of Harris Connect’s email marketing tool. Harris is a popular tool for maintaining alumni communities and the email tool they offer is fairly sophisticated in regards to segmentation your audience to target a message to the audience. But where it lacks is reporting, leading to what could be statistics that could be characterized as deceiving at best. Other tools that I’ve demoes in the past are not dissimilar to Harris in this respect.
Let’s jump back a notch: It’s not enough to send out mass emails to an a audience - marketing requires that you measure and analyze results. I look at some pretty standard statistics to assess the results of an email campaign, including (but not limited to) the follow:
- Open Rate: Your open rate is the unique number of times an email has been opened, divided by the total messages that were delivered. More on why this is trickier than you might think in a second.
- Click Through Rate: Just like open rates - unique click throughs divided by the total number of messages delivered.
- Click-To-Open Rate: Divide your unique click throughs by the number of unique messages that were opened. This rate is probably a much better assessment of the success of your message, since you’re only including people who actually opened it in your pool.
All of this seems fairly easy right? Wrong. The problem with many email service providers is that they don’t give you UNIQUE opens and click throughs - they give you TOTAL opens and click throughs. And the difference is significant: Unique rates tell you how many unique users took a particular action. Total rates tell you how many times they did it. Say you delivered 100 messages. 20 unique users opened the message, but several of them opened it more than once, resulting in total opens of 50. If you calculated your open rate the correct way, using the unique number, it would be 20%. If you calculated it using the total number, it would be 50%. Quite a difference.
The same applies to click through rate and click-to-open rates. If you don’t have the unique statistics, your stats are going to be completely skewed. This is an extremely important question to ask when you are looking for an email service provider - in your reports you should be able to quickly and easily differentiate between your total numbers and your unique ones.
This is just one of many ways in which your email stats can provide deceiving results. Rob S. left a great comment on SquaredPeg detailing some other problems with tracking. Evaluating the success of your campaigns is important and it might be tempting to use the skewed stats since they can oftentimes be a bit more impressive. But they aren’t real and they aren’t going to help you do what analysis is there for - improve your future marketing efforts based on what you learned from the previous ones.
Bottom Line: Open rates have at least 15-20% degree of inaccuracy. Recording an ‘open’ can only happen if the readers email client is capable of displaying html with images, and that option is turned on.
To me, the most telling stat is clickthrough rate. And/or: “We sent this email to x number of people, and this many of them actually clicked on something.
It’s too bad though. I’d like to know with absolute certainty how many people just opened it or read it.
this is a good resource for this discussion:
https://www.campaignmonitor.com/blog/archives/2007/01/all_about_email_open_rates.html
Exactly Drew, though I will say that I do use open rates as a helpful guide to benchmark off myself. That way I can at least see if I’m making some amount of gain or loss against previous messages.
I’m amazed that email vendors are providing duplicated click-through rates. That doesn’t really help much.
Another interesting issue is that the higher the bounceback rate, the lower the open (or what we call viewed) rate will be. Theory is that more bouncebacks mean the list is old and thus there will also be many active email accounts that aren’t being used anymore (thus, got delivered, but no one is home). We call these abandoned - but really no way to calculate them.
Hi Brian
I actually think that total rates can be interesting when compared to the unique rates. For instance, I want to know that the majority of my users are clicking on a link more than once because I think that means the email worked. The problem comes in when the ESP doesn’t tell their clients how they are calculating the rates (or the client doesn’t know enough to ask!)
One of the things that really impressed me at Dartmouth was how clean they keep their email lists - all of the bounces are sucked back into the system and noted. When a user hard bounces 3 times, they automatically get sent a postcard asking them to contact us and update their address. We typically have a 98-99% deliverability rate because of it.
The following comes from an ExactTarget post but is also good to keep in mind.
The Well Chosen Average:
Let’s assume the marketer sends 5 campaigns a week, one general newsletter and four highly targeted mailings. The newsletter goes to 100,000 subscribers a week gets a 10% open rate. But the four targeted mailings each go to 100 subscribers and get a 50% open rate. What is the average open rate? It depends on how the marketer chooses to calculate that average. 42% [(10% + (4 x 50%) / 5] and 10.2% [(100,000 x 10% + (4 x (100 x 50%))) / 5] are both legitimate answers. In my experience, the first answer (42%) is easier to calculate and, thus, the way most marketers answer this question. It’s not intentionally dishonest, but it’s not the whole truth, either.
So I’ll remind everyone that if you are sharing stats with others to please include your equation.
I’m surprised to hear that some software doesn’t give you unique open rates and click rates. You can get all that and more with $10 software like iContact.
I’d say open rates are an important stat because it can let you know how well your subject line is doing. Create 2 subject lines and send each to 15% of your list. Whichever subject line gets the highest open rate, use that for the other 70% of your list.
You can’t get clicks without opens.
Hi Brad
I disagree. I think your click to open rate is the best indicator of if your subject line is working - the goal of an email isn’t just to get your user to open it. It’s to get them to take a call to action. A user may open an email that has a confusing or non-descript subject line just to see what the heck it is. What you’re shooting for is to get one that both opens the email and clicks on the link.
I really liked your blog! Keep up the good work, I have checked out a few of your posts and I like what you write.
The purpose of the subject is to get them to open the e-mail.
The purpose of the body of the e-mail is to continue the conversation of the subject line to get them to read or scan the email and then click.
Click to open ratio is an indication of how well your body text is performing.
No, the purpose of both is to get them to take a call to action. If a person is opening the email but not taking your call to action, then it was a waste of time/money. You could use a deceiving subject line to get tons of opens but very few click throughs. Does that mean the email is a success? Not remotely. The click-to-open ratio is a better indicator of if your subject line was descriptive enough so the user knew what they were getting before they opened it.
@Brian, some e-mail service providers subtract out known bounces from attempted sends, and use the lower number to calculate opens and clicks. Yet another reason metrics can be murky.
Duplicated clicks can, indeed, be useful. If a particular recipient clicked on a link to apply two or more times but didn’t follow through, it reveals something. What that something is could be debated.
Maybe it is a high level of interest. Maybe it is confusion in terms of what to do on the app page. Maybe it is a simple way to return to the app to finish it up after starting it in a previous session. Maybe it is a sign that your app account creation page needs a confirmation message, or that such confirmation messages sent through your Web server are getting snagged by spam filters… Worst case it gets you thinking about these things.
Head of Marketing , I agree that, while e-mail metrics are murky, they are valuable for comparing against previous messages, particularly multiple messages sent to the same list in a short timeframe. It is one possible insight into list fatigue.
The emphasis on measurements and their meaning very helpful.
Full disclosure first - I am in charge of application development at Harris Connect.
We are enhancing our reporting across the alumni community, so this is good time to factor in these thoughts.
A specific list of unique open info with date and time is available off of the summary report page in the tool today. So the information for unique opens is available today in the application.
The Email marketing tool from Harris can target based on demographic or activity info.
Reporting is expanding to show a view across email and the resulting activity.
Asking for unique opens and click throughs is definitely something I will be doing ongoing. Thanks for sharing. Interesting article and comments too.
I don’t agree with all your thoughts, but you do have good point of view.