The Art of Assessing Your Data

When it comes to numbers, there are three types of people:

People who react without any supporting data whatsoever.

You’ve probably worked with quite a few of these before.  You’re doing your designated task like a happy hamster on a wheel, when someone else reads a new article, goes to a conference, talks to a vendor, gets a random complaint, or whatever, and it’s time for you to reverse course and spin the wheels in another direction.  I used to say it’s like some crazy Jim Collins’ bus metaphor where the bus driver changes direction each time a new person gets on and says he wants to go somewhere else; the bus moves in circles.

People who ask for numbers but have no clue what they want them for.

These people collect raw data like my my kids collect broken crayons.  What do they do with the data?  They can’t possibly decipher it.  Much like an abstract work—or my kids’ artwork—they pretend it holds some deeper meaning, hold it at arms length, share it at meetings with others hoping to hell that it speaks for itself and that someone will explain it to them.

People who actually know how to get the information they need from numbers.

These people possess are rare talent: the art of assessing data.  Whether you are are talking needs assessment or Web analytics, hard numbers don’t tell you the story of what’s really going on.

Shelby Thayer, of Trending Upward, would say, “It’s all about trends. Percentages. Is our bounce rate for this landing page up or down? Is it high or low? I don’t care that we get 100,000 visits a week.  I care if that number goes way down or way up.  I need to know why that’s happening and that makes you dig deeper.”  Here’s where filtering comes into the picture.

So we have this block of raw data that we have to carve into something that tells us something meaningful.  How do we know what to cut away?  That’s where qualitative data becomes valuable.  Though you will encounter people from group #2, who hear the word “anecdotal” and completely discount the validity of qualitative data (true story), as Shelby says, “You can’t have strictly quantitative or qualitative data and really know what’s going on.  The quantitative tells you what, and the qualitative tells you why!  If you strictly rely on the quantitative, you might dump a bunch of money or assets into fixing something that really just needed to be tweaked!”

Gathering good quantitative data helps you segment to a particular audience, issue, etc.  Sometimes you can get this data from tools on your site.  On occasion, I’ve gotten an open-ended comment on a survey or heard an anecdote while conducting personal interviews that led me to filter my raw numeric data and discover new patterns I hadn’t considered.  Sometimes it’s a manual process of getting out and interacting with people.  When I was a student teacher, well over a decade ago, my student teaching supervisor called it “withitness”.  Or you could consider it management by walking around (MBWA) for your data.   I’ve had people make “off the record” comments because I have taken the time to get to know them as people first.  Later I have been able to refine my data based on what I know about these people.

When you work with raw data and apply enough filters to it based on how well you know the segment you are working with, you will have the best likeness of what’s going on.  Like Michelangelo said, “It is easy. You just chip away the stone that doesn’t look like David.”

Photo Credit: “Michelango’s David” by Robert Scarth

14 Responses to “The Art of Assessing Your Data”

  1. Says:

    Nikki

    Just as good decisions require both quantitative and qualitative data, they also require people of both the first and third types. It’s true that type 1 people can be annoyingly knee-jerk reactive, but they tend to be people who can “sense” things and frequently know how to follow their gut. Of course, they’re so much better off when tempered by people of type 3, but those in that category need to be able to look beyond the data once in a while and see what intuition tells them.

    As for type 2 folks, well they’re trying. They collect the data because either they think they’re *supposed to,* or because they genuinely want to understand how to use it. Some of those type 3 artists ought to jump in and give them some guidance.

    -Jim

  2. Says:

    I agree yet disagree with Jim. I think that the type 3 people collect and rationalize data efficiently and quickly to make them appear to be a type 1 person. I know a lot of times my extremely intelligent colleagues will tell me how best to go about something instantaneously after a quick look at data. If I ask why, they already have an entire lecture on why using the new data that came in.

  3. Says:

    @Jim W: We probably do need some type ones in higher ed or things would never get done. Committee-heavy & meeting-laden as we are we’d never keep up with the pace of technology if we waited for all the data. That said…

    @Commercial Coffee Machine: You’re probably right. Some of the Type 1′s may just be creatives who are really good at reading the culture without data.

    For the type 2s, send them here for help. ;)

  4. Says:

    I agree that all three are needed in an organization to maintain balance between knee jerk reactions and analysis paralysis. I think I’m more of a type 3 than anything as an instructional designer (we’re kinda made that way.) I don’t want to be negative, but I know from experience that really big decisions shouldn’t be made by the type ones alone. When they are, things will most likely fall apart somewhere down the line. As well-meaning and in-tune with culture as they are, they most likely didn’t have time to think the decision all the way to the end to identify ramifications when a decision is made on too little information.

  5. Says:

    Thanks, April. I’ve been driven crazy enough by the knee jerk behavior of the first type to agree that filling the upper rungs of the hierarchy would be very, very bad. Finding good type threes is rare, though. We usually end up with a mix of type ones and type twos.

    There was a great recommendation by Bob Sutton this week that explained these type twos in leadership: “the notion that managers often use talk, planning, endless study, and so on as a substitute for taking action”.

    • Says:

      Thanks, Nikki. Good article on eadership.

  6. Says:

    Nice article I’ve been unfortunate enough in the past to start a business with a type one individual - “People who react without any supporting data whatsoever.” It can be very frustrating, especially when the data supports one way forward but they ‘have a feeling’ that we should follow a different strategy!

  7. Says:

    @Advertising & Marketing Ideas: Eventually we learn to spot the tell-tale signs of such leadership. People who work for type ones and type twos either end up leaving, or learn to parrot the jargon and pet projects of the day to appear to have a purpose…

  8. Says:

    Technology Training- great post and thanks for the mention.

    I think the trick is for each type to know their own faults and count on the other types to join in the decision. Obviously we don’t want analysis paralysis, but just going on gut alone isn’t good either.

    Always be asking why. If you do that, you have no choice but to dig deeper.

  9. Says:

    @Shelby: Good advice. A little self-knowledge goes a long way.

  10. Says:

    People who react without any supporting data are the most irritating kind of people. A single person of this kind can make your life like hell. He will shout on you for nothing. People should realize the importance of numbers and analysis in their life. These are the most important permanents to take wise and good decisions in favor or the organization and to simplify the work.

  11. Says:

    I need to teach and asses something in 15 minutes- preferably to 16-19 year olds-
    It has to be art related and have a big emphasis on the assessment bit.

  12. Says:

    Writers are under appreciated, thanks for the write-up.

Trackbacks/Pingbacks

  1. grundyhome.com » Blog Archive » A Data-Driven Life --> says:

    [...] has a good post about the three different approaches to data-driven work: Youâ