The Art of Assessing Your Data

The Art of Assessing Your Data

When it comes to numbers, there are three types of people:

People who react without any supporting data whatsoever.

You’ve probably worked with quite a few of these before.  You’re doing your designated task like a happy hamster on a wheel, when someone else reads a new article, goes to a conference, talks to a vendor, gets a random complaint, or whatever, and it’s time for you to reverse course and spin the wheels in another direction.  I used to say it’s like some crazy Jim Collins’ bus metaphor where the bus driver changes direction each time a new person gets on and says he wants to go somewhere else; the bus moves in circles.

People who ask for numbers but have no clue what they want them for.

These people collect raw data like my my kids collect broken crayons.  What do they do with the data?  They can’t possibly decipher it.  Much like an abstract work—or my kids’ artwork—they pretend it holds some deeper meaning, hold it at arms length, share it at meetings with others hoping to hell that it speaks for itself and that someone will explain it to them.

People who actually know how to get the information they need from numbers.

These people possess are rare talent: the art of assessing data.  Whether you are are talking needs assessment or Web analytics, hard numbers don’t tell you the story of what’s really going on.

Shelby Thayer, of Trending Upward, would say, “It’s all about trends. Percentages. Is our bounce rate for this landing page up or down? Is it high or low? I don’t care that we get 100,000 visits a week.  I care if that number goes way down or way up.  I need to know why that’s happening and that makes you dig deeper.”  Here’s where filtering comes into the picture.

So we have this block of raw data that we have to carve into something that tells us something meaningful.  How do we know what to cut away?  That’s where qualitative data becomes valuable.  Though you will encounter people from group #2, who hear the word “anecdotal” and completely discount the validity of qualitative data (true story), as Shelby says, “You can’t have strictly quantitative or qualitative data and really know what’s going on.  The quantitative tells you what, and the qualitative tells you why!  If you strictly rely on the quantitative, you might dump a bunch of money or assets into fixing something that really just needed to be tweaked!”

Gathering good quantitative data helps you segment to a particular audience, issue, etc.  Sometimes you can get this data from tools on your site.  On occasion, I’ve gotten an open-ended comment on a survey or heard an anecdote while conducting personal interviews that led me to filter my raw numeric data and discover new patterns I hadn’t considered.  Sometimes it’s a manual process of getting out and interacting with people.  When I was a student teacher, well over a decade ago, my student teaching supervisor called it “withitness”.  Or you could consider it management by walking around (MBWA) for your data.   I’ve had people make “off the record” comments because I have taken the time to get to know them as people first.  Later I have been able to refine my data based on what I know about these people.

When you work with raw data and apply enough filters to it based on how well you know the segment you are working with, you will have the best likeness of what’s going on.  Like Michelangelo said, “It is easy. You just chip away the stone that doesn’t look like David.”

Photo Credit: “Michelango’s David” by Robert Scarth


Tweet
Share StumbleUpon It! Del.icio.us reddit

Like this post? Be sure you've subscribed to the .eduGuru RSS feed or email to get all the latest news and articles.


anecdotes, needs assessment, qualitative data, quantitative and qualitative data, quantitative data, web analytics

Read Related Posts on .eduGuru:

  1. Are you ready for Web Analytics 2.0?
  2. Data Collection #FAIL
  3. Web Analytics Strategy: Data In Context

This post was written by:

Nikki Massaro Kauffman

Nikki Massaro Kauffman - who has written 42 posts on .eduGuru

Nikki is a multimedia specialist with Penn State's World Campus Learning Design unit, creating and editing multimedia for online courses.

Previously, she was technology training coordinator with the Penn State University Libraries, responsible for technology training offered in the Libraries' 20+ departments and 30+ library locations.  

Over the years, she's been she served as an interim associate director of instructional technology and multimedia, a programmer, a database specialist, a Microsoft Certified Master Instructor, a continuing education instructor for seniors and adults with disabilities, and a high school English and communications technology teacher.  

Her interests are in the areas where technology, training, and communication intersect.  She holds degrees in both computer science and in education.  She is also an insomniac and an extreme extrovert with an indiscriminate love of language (including expletives).


14 Responses to “The Art of Assessing Your Data”

  1. Avatar image
    Jim W Says:

    Nikki,

    Just as good decisions require both quantitative and qualitative data, they also require people of both the first and third types. It’s true that type 1 people can be annoyingly knee-jerk reactive, but they tend to be people who can “sense” things and frequently know how to follow their gut. Of course, they’re so much better off when tempered by people of type 3, but those in that category need to be able to look beyond the data once in a while and see what intuition tells them.

    As for type 2 folks, well they’re trying. They collect the data because either they think they’re *supposed to,* or because they genuinely want to understand how to use it. Some of those type 3 artists ought to jump in and give them some guidance.

    -Jim

    Reply

  2. Avatar image
    Commercial coffee machine Says:

    I agree yet disagree with Jim. I think that the type 3 people collect and rationalize data efficiently and quickly to make them appear to be a type 1 person. I know a lot of times my extremely intelligent colleagues will tell me how best to go about something instantaneously after a quick look at data. If I ask why, they already have an entire lecture on why using the new data that came in.

    Reply

  3. Avatar image
    Nikki Massaro Kauffman (author) Says:

    @Jim W: We probably do need some type ones in higher ed or things would never get done. Committee-heavy & meeting-laden as we are we’d never keep up with the pace of technology if we waited for all the data. That said…

    @Commercial Coffee Machine: You’re probably right. Some of the Type 1′s may just be creatives who are really good at reading the culture without data.

    For the type 2s, send them here for help. ;)

    Reply

  4. Avatar image
    April Says:

    I agree that all three are needed in an organization to maintain balance between knee jerk reactions and analysis paralysis. I think I’m more of a type 3 than anything as an instructional designer (we’re kinda made that way.) I don’t want to be negative, but I know from experience that really big decisions shouldn’t be made by the type ones alone. When they are, things will most likely fall apart somewhere down the line. As well-meaning and in-tune with culture as they are, they most likely didn’t have time to think the decision all the way to the end to identify ramifications when a decision is made on too little information.

    Reply

  5. Avatar image
    Nikki Massaro Kauffman (author) Says:

    Thanks, April. I’ve been driven crazy enough by the knee jerk behavior of the first type to agree that filling the upper rungs of the hierarchy would be very, very bad. Finding good type threes is rare, though. We usually end up with a mix of type ones and type twos.

    There was a great recommendation by Bob Sutton this week that explained these type twos in leadership: “the notion that managers often use talk, planning, endless study, and so on as a substitute for taking action”.

    Reply

    • Avatar image
      April Says:

      Thanks, Nikki. Good article on eadership.

      Reply

  6. Avatar image
    Advertising & Marketing Ideas Says:

    Nice article I’ve been unfortunate enough in the past to start a business with a type one individual - “People who react without any supporting data whatsoever.” It can be very frustrating, especially when the data supports one way forward but they ‘have a feeling’ that we should follow a different strategy!

    Reply

  7. Avatar image
    Nikki Massaro Kauffman (author) Says:

    @Advertising & Marketing Ideas: Eventually we learn to spot the tell-tale signs of such leadership. People who work for type ones and type twos either end up leaving, or learn to parrot the jargon and pet projects of the day to appear to have a purpose…

    Reply

  8. Avatar image
    Shelby Thayer Says:

    Nikki - great post and thanks for the mention.

    I think the trick is for each type to know their own faults and count on the other types to join in the decision. Obviously we don’t want analysis paralysis, but just going on gut alone isn’t good either.

    Always be asking why. If you do that, you have no choice but to dig deeper.

    Reply

  9. Avatar image
    Nikki Massaro Kauffman (author) Says:

    @Shelby: Good advice. A little self-knowledge goes a long way.

    Reply

  10. Avatar image
    The Agra Indian Says:

    People who react without any supporting data are the most irritating kind of people. A single person of this kind can make your life like hell. He will shout on you for nothing. People should realize the importance of numbers and analysis in their life. These are the most important permanents to take wise and good decisions in favor or the organization and to simplify the work.

    Reply

  11. Avatar image
    Crafts In India Says:

    I need to teach and asses something in 15 minutes- preferably to 16-19 year olds-
    It has to be art related and have a big emphasis on the assessment bit.

    Reply

  12. Avatar image
    Danny Says:

    Writers are under appreciated, thanks for the write-up.

    Reply

Trackbacks/Pingbacks

  1. grundyhome.com » Blog Archive » A Data-Driven Life says:

    [...] has a good post about the three different approaches to data-driven work: Youâ

Leave a Reply

Spam protection by WP Captcha-Free