Ok. I’m sure most of us know what’s important. Kyle, for one, has done an excellent job in helping us all - myself included - in understanding how to use analytics to improve our higher ed web pages. But what I’m talking about here is: what’s important to tell your VP or others? As a higher ed web professional, what’s specifically important to the work that you are doing in trying to enroll and retain students and families?
I asked the question about VP reports recently in the University Web Developer’s Ning group. It seems from people’s responses that we all tend to be in the same boat: we provide reports monthly, not knowing what their use or interpretation will be. We may explain or leave others to decide how they read the analytical tea leaves. Or, we provide analytics in response to a direct question. I’m lucky enough to have a VP who asks what my opinion of the data is and what my recommendations are. So, instead of providing monthly blanket site analytics, which tell us very little beyond year over year trends or campaign responses, I’ve started thinking about what really is important:
What are we measuring? Maybe you’re lucky enough to have a great relationship with admissions and have tagged their email campaigns with specific URLs. Perhaps you manage the university AdWords account and monitor the progress of these campaigns. Beyond this, what else do you measure? Are you tracking your international hits to see if travel abroad is worthwhile? Landing pages for campaigns are great for collecting prospect information, but do you have this information built into your web pages? How is social media traffic translating into program attendance? Not having goals, whether they be analytics goals or site goals really limits what you can measure accurately. Also, if you have no mechanisms in place to capture student or parent contact information, you’re traffic means very little. You arent converting those visits into, well, anything. Stealths may get your information and apply on their own, but what else could you be offering them to make them come out of the woods?
Why are we measuring it? Is this information for budgeting travel? Will you be able to make changes to navigation or structure based on these reports? Will you get approval for more personnel or funds to make the changes to the web that you need? Are you thinking about going all electronic in some of your print campaigns? You need to know why you are measuring things just as much as you need to know what you will measure. If you are reporting for reporting’s sake, you’re wasting your own valuable time not to mention your VP’s. Measure for trends, measure for campaigns, but whatever you do, measure for a reason. I know we tend to be cynical (who, us?) but I encourage you: be curious.
Who are we measuring it for? Is this for VP’s, directors, faculty or communication professionals? Remember the audience for whom you are collecting this information and consider if you need to translate it. I’d suggest never just handing over information, but meeting face to face to ensure people understand the nuances of their reports. I also find that certain analytics professionals consider certain aspects of reports more or less reliable/important. For instance - of course China visits will have a longer time on site if they translating the page. Of course you have a higher bounce rate overall if you are incorporating more social media deep linking strategies.
What happens after we measure it? Will you actually be able to implement changes based on your information? A monthly trend report may help you keep a pulse on your visitors, but a quarterly report may actually lead you to making changes. Will you provide a report to a VP that will reference it in a board meeting? Know what you are providing the information for and the outcomes the requesting party is seeking.
In preparation for my (first! 
)HighEdWeb NY Regional conference presentation - Analytics: What’s Really Important - I ask you to add to this conversation. How do you tackle the analytics question and how much power do you have, as the reporting party, to effect change? How much collaboration is there between offices for use of this information? Do you create reports as you see the need or do individuals request specific reports?
Great questions and food for thought about what we do with that data. Sometimes, I think they can actually do more harm than good—if we stare long enough we can see correlations and trends that may not have real meaning. And then, if we act on those Type 1 errors, we might break something that was working just fine.
I prefer the experimental approach, which you hint at. Know what you want to look at before you dive in.
Note: The automatically-created emoticon in the last paragraph is kinda strange.
what’s*
Thanks for reading, and for the edits. Apostrophes and I don’t get along.
Love the “read the tea leaves” metaphor. I’ve found that for metrics to be meaningful, they have to be granular. Great globs of data will be meaningless.
I came into my position with an interest and some experience in web analytics and started to work it into my job description.
It’s been a challenge to really integrate it into department discussions on online marketing, just to get people in different departments up to speed on how to define these metrics and how to make them relevant, but in the past two years, it’s happened, though slowly. We’ve been able to establish benchmarks for specific conversion goals for our website and we’ve made it a practice to tag all of our incoming links for email campaigns.
We’ve even experimented with tracking print campaigns affect on web traffic.
Fantastic post, Jessica. My favorite line, “measure for trends, measure for campaigns, but whatever you do, measure for a reason.” Priceless.
For your question about collaboration - I think at bigger, more decentralized schools it’s extremely to collaborate about websites much less analytics of those websites. We’ve started a users group at Penn State to try to tackle this issue. We’re trying to share best practices, information, tips about implementation, standards about reporting, etc. It’s so hard when you have, literally, hundreds of sub-sites all owned by different units.
As for reporting - like you mentioned, the audience of the report is key. Leadership gets a more high level report, mid-managers more granular, etc. Also each unit type would have there own (marketers, content owners, application owners, etc.) The specific metrics on the report depend on your objectives (tied directly to business objectives) and KPIs … and, of course, politics. For instance, for very high level leadership just show trends, not raw numbers by default. They mean nothing to that group and they become fixated on the raw number. If leadership demands those raw numbers, only then do I include them. Obviously it’s different everywhere and you just need to get a feel for what people want, their level of caring and/or knowledge, etc.
For the “effect change” question - with leadership at least - I think the trick is to monetize if at all possible. What is your *goal* worth? It doesn’t matter that we’re not an e-commerce site. We can make assumptions around what a goal is worth, typically, and then make the argument. The fallout rate is X and it is a potential opportunity loss of $X. This tends to get things done.
Remember analytics involves things outside of our analytics tool as well. Voice of customer responses go a long way. Another thing that works wonders for us is to make leadership walk through the trouble areas as the user.
Anyway, great post!
@Keidra - I agree. I did web analytics (Urchin) for several non-profits and it was a much easier feat with smaller sites and very specific, federal funded programs. Those have very obvious and serious reporting outcomes. Higher ed, being political and decentralized with varying levels of understanding makes it much harder - but a great challenge.
@Shelby - Thank you, thank you, thank you. You are my new analytics muse! The blog https://www.trendingupward.net/ has become my new obsession. I’m lucky that my school is much smaller yet still with the variety of issues you bring to light. I guess I’m finding that its as much or as little work as I’d like to put in, in a proactive sense. More so, its about educating others so that they can do this work and improve on their own.
Great comments!
I’ve been warming up to analytics of the Google persuasion. And I’m struggling with wrapping my head around all the various possibilities, recognizing that most of what can be gathered is not gold. And I certainly don’t want to send out reports that will likely be misinterpreted.
Now working with marketing vendors to understand cpc and if it’s worth our while. Also, would like to get a handle on ROI as well.
What would be useful for me is to hear about what reports, beyond the usual suspects, are particularly useful for .edus.
Jess - Thanks. I’m glad you enjoy the blog. You are absolutely right - especially in higher ed. We continue to be jacks-of-all-trades. Web analytics in higher ed is definitely an *after hours* job right now, unfortunately. When I say that, I don’t mean that we don’t dabble and answer certain questions during the day when it’s necessary. Generally, though, we continue to underestimate the importance of it, in my opinion. There should be dedicated resources. Obviously that’s wishful thinking for most schools right now.
George, I agree: we need examples of reports. I’ll work on a second post on what exactly I cover in monthly trend reports and what I intend to include in my quarterly actionable reports. If others would like to share their reports for the next post fodder you can email me at jessica@doteduguru.com.
Love the “read the tea leaves” metaphor. I’ve found that for metrics to be meaningful, they have to be granular. Great globs of data will be meaningless.
Jessica, I think this is a great article to get web maestros and marketers focused on measurement and analytics in the education community. From the perspective of those in charge of admissions, measurement of the quantity and quality of inquiries generated from the website is crucial. The goal of admissions depts is to attract as many of the best and brightest students as possible; traffic and page views are a very indirect measurement tool to determine progress towards that goal. The whole organization benefits when a school can connect the performance of its website and web-based marketing campaigns to the number of inquiries, applications, and enrollments.
Agreed. But what are those reports that help to make the connections? In GA, event tracking can be a way to do that. Another way would be to track the apply online “funnel.”
Agreed. But what are those reports. In GA, event tracking can be a way to do that. Another way would be to track the apply online “funnel.”
@Martin - believe me, I report to the VP of Enrollment and Retention so I hear you on that! Also, I’d even love to get so deep into it we can track these inquires and rate them for retention purposes. Apps are great, enrolls are great but retained students are what really matters. If we could pin point what type of student that is, how they tend to apply and reach out to us, then boy, thats gold. I know there may be high end systems out there that do this, but most of us A) cant afford it, B) cant get IT to play nice or C) cant get the buy in to get it off the ground.
@George Agree on event tracking and the apply funnel. I need to get on that!
This is a topic I’ve been looking for discussion on for a long time. The big problem I see is with the length of time the “sale” cycle runs. From the point a high school student first visits your site to actually applying, enrolling, attending, it could be multiple years, even longer if you want to factor in retention success/failure. Google Analytics (GA) just isn’t designed for that type of long term tracking (cookies get erased, new computer, etc), nor is it designed to let you track personally identifiable information (it’s actually a violation of their TOS), which you have to do if you’re measuring against actual enrollment/retention information.
I went to a conference on GA and one of the presenters talked about a way to use the GA cookie and once the prospect requests more info or somehow gives his name/email in a form, the source/medium and whatever else you want to get from the cookie can be stuffed in your school’s CRM type of system where you track students through the admission process. This method circumvents some of the GA limitations, but it also means your analytics data is spread over two systems.
The advantage is you can run a report from your CRM system (five or six years from now) and see where did the students who stayed all four years come from in the analytics, or what pages did they spend the most time on.
Obviously there are other draw backs with trying to track over such large amounts of time. One issue is the page you see in the retention report from four years ago is now completely different, or the site has been restructured. So would we need to take a snapshot of the site and archive it every six months or so? :-/
Just some thoughts,
Philip
Phillip
Good thoughts. Wish I knew the answer. Is there anything in the server logs that could be of use in tracking longer term?
Agreed. But what are those reports. In GA, event tracking can be a way to do that. Another way would be to track the apply online “funnel.”
Hi George,
I’m not aware of a way to use server logs, since most of the “fingerprint” data for a particular user will likely change over this amount of time. It seems like the best bet is to get them to give you some very basic demographic information like name, email and address, but not too much that it scares them away. At that point, you can grab that data and insert it in to your CRM along with some of the GA data.
Obviously this method wouldn’t be perfect, because you can’t track the visitor as an individual until he identifies himself, so all the stealth applicants are off the radar.
I do realize that most of our time will be spent tracking smaller goals, like requested info, registered for a campus visit, etc., but it really would be nice to have a method to capture the big picture.
Philip
@Philip - I agree. I’d love to see the bigger picture from a retention stance, but it would really require a lot of time and advanced effort from everyone on campus. Getting that kind of buy in can be tough, although not entirely off the wall given that enrollment and retention are hand in hand.
Regarding the ‘Funnel’ - I’m still on the fence about this idea. I know its an industry standard and that if you have well designed pages it works most of the time and you use the funnel reports to make tweaks. But, I think it makes a lot of assumptions. 1) that people land where you expect them to in the website, 2) that everyone follows that same path and 3) that no one else is doing outside pushes to pages in your funnel via social media, email, etc.
Yes! Tracking through the entire student life cycle. That’s definitely the holy grail. Wouldn’t it be wonderful if we had one CRM tool that our schools used across all units? Ah, we can dream.
There are scripts out there that capture campaign info attached to the information requester and dumps it into the CRM. Of course to be really effective it goes back to that one CRM tool across campus, though.
Not sure how effective traffic data is over time (years), though, because websites are ever-changing. I’m not a huge proponent of comparing date ranges that are too wide (more than a year). So many variables involved if your website changes often - especially those priority pages.
I agree with Jess on the funnel. If it’s a closed, one-way funnel, like a shopping cart, I think it can be effective. As soon as you introduce a lot of variables, though, the less effective it can be for showing fallout.
Just as Jess pointed out, a college application isn’t a clean funnel. The user can go back and forth and sometimes they don’t even need to fill out some pages or upload some info - depending on the program, etc. This makes the “funnel” a little more flexible so the data won’t be as useful (for fallout).
For tracking campaigns, getting the “thank you page” tagged is a step forward. To use the data for usability, though, tracking the funnel is essential, of course. But because of the way users go through a college application, not sure it would be an effective way to show fallout. I don’t know. I go back and forth on this, though.
My bad, Jess. I just re-read the discussion on the funnel and noticed you were discussing both regular page funnels and application funnels differently.
Page funnels are tricky for the reasons you point out. I’m also still not sold on the online application funnel (for usability fallout), though, because of the back and forth nature (and the ability to skip steps or upload depending on type of student, etc.). Obviously this is different school-to-school. Anyway, wanted to clarify. Sorry if my previous comment was confusing.
I’d love to hear others thoughts on that, though.
What’s the difference between a page funnel and application funnel?
Our online undergrad and grad “personalization” app is relatively integrated with our CRM, but I never thought about looking at a four year cycle.
I understand the usability fallout issue, but why should skipping a step be an option? Back and forth behavior is a tough one, tho’.
All I meant by “page funnel” was basically path analysis without a closed-off funnel (like a shopping cart). So, just saying how many people went from home >>> program A >>> tuition rates >>> application. Basically making up a funnel of random pages (or the path that we might think the prospect would take to get to an action). I get requests all the time for path analysis of this or that and it just makes no sense to do it if they aren’t part of a closed off funnel. That’s all I meant.
As far as the “skip an option” is concerned, I was specifically thinking of something with our graduate application. Depending on the program or type of student, they don’t have to submit certain materials or information. That may not be the case elsewhere, though.
That’s fantastic that your apps are integrated with your CRM - do you have the same CRM across your school or does it depend on the unit? Do other units that don’t handle applicants (alumni association, etc.) also use it?
As a side issue, since the end to end tracking requires the user to identify himself early on, I wonder if the new Facebook sign-on would be a nice, light-weight method of getting student email addresses and a unique user id that could be filed away in the CRM.
@Philip - I agree. I’d love to see the bigger picture from a retention stance, but it would really require a lot of time and advanced effort from everyone on campus. Getting that kind of buy in can be tough, although not entirely off the wall given that enrollment and retention are hand in hand.
Regarding the ‘Funnel’ - I’m still on the fence about this idea. I know its an industry standard and that if you have well designed pages it works most of the time and you use the funnel reports to make tweaks. But, I think it makes a lot of assumptions. 1) that people land where you expect them to in the website, 2) that everyone follows that same path and 3) that no one else is doing outside pushes to pages in your funnel via social media, email, etc.
@Shelby - Yes, I agree with you on all fronts. We do not have a CRM or a separate cookie analysis system so its all just GA. I’m focusing more on organic paths to a goal now - its the wild wild west if your web site isnt the best navigation wise! Add in dropping deep links in social media and people can start all over the place. But I’ve recently started using the ‘tags’ feature to mark where social media campaigns begin, etc so we can see the specific jumps in traffic. It really is all about being agile and making GA work for you at any given moment.
@Phillip - That’s a very interesting though RE FB and your CRM. I wish we had a CRM!