Figuring out what those poll results mean

Last week’s poll (Where’s your institution on the Culture of Analytics Ladder?) had all the things I dislike about quickie online polls, including the fact that the respondents were self-selected, and it was open to anyone who stumbled upon it, including people with no connection to the field.

I don’t bother with polls very often, but hey, it’s my blog and I’ll poll if I want to. People DO like polls, even if we should take the results with a grain of salt.

The poll is still open, so you can go back to view up-to-date results anytime, but here’s the breakdown of the 81 responses received as of today. My thoughts about the results continue below …

These results do not distinguish between non-profits and higher-education institutions, nor between small and large organizations. We don’t know if respondents are managers or staff, or even if they understand the question. So … take this for what it’s worth.

Less than a fifth of people who viewed last week’s post opted to cast a vote. Of those who did, it does seem they were honest about their answers. I am not surprised that almost a third of respondents admitted that although they have data, no one in their organization knows how to analyze it. That is deeply unfortunate but it’s a fact.

Coming a near second are people who said that their analyst’s data insights support decisions only on an ad hoc basis. (Answer 6 includes organizations who may have contracted out for predictive model score sets, so these organizations may not even have analytics talent on staff.) And in third place are the people who said data analysis is a regular process, not ad hoc, but that the benefits are limited to just part of the organization.

What might be encouraging here is that almost three-quarters of respondents have the data they need and are somewhere along the path of becoming data-driven, even if they aren’t quite there yet. On the other hand, these are CoolData readers, so keep the bias in mind.

Then there are the eight people who claim that analytics has been wholly embraced by their organizations, from top to bottom and in all operational areas. I’d love for those people to come forward and identify themselves, because we could all learn from them. Email me!

Where’s your institution on the Culture of Analytics Ladder?

I’m laying on the couch with a bad head cold, and there’s a mix of snow and rain in the forecast. Time to curl up with my laptop and a cup of tea. I’ve got a question for you!

Not long ago I asked you to give me examples of institutions you’re aware of that are shining examples of institution-wide data-driven decision making. I was grateful for the responses, but no single institution was named twice. A few people offered an opinion about how their own organizations size up, which I found interesting.

So let’s explore that a bit more with a quick and anonymous poll: Where do you think your non-profit organization or institution fits on the Culture of Analytics Ladder? (That’s CoAL for short … but no need to be so formal. I totally made this up under the influence of cold medication.) Don’t over think it. Just pick whatever stage you feel your org or institution occupies.

The categories may seem a bit vague. If it’s any help, by “analysis” or “analytics” I am referring to the process of sifting through large quantities of data in search of patterns that lead to insights, primarily about your constituents. I am NOT referring to reporting. In fact I want you to ignore a lot of the day-to-day processes that involve data but are not really “analysis,” including: data entry, gift accounting, appeal segmentation, reporting on historical results, preparation of financials, and so on.

I am thinking more along the lines of modelling for the prediction of behaviours (which group of constituents are most likely to engage in such-and-so a behaviour?), prediction of future results (i.e., forecasting), open-ended exploration of constituent data in search of “clusters”, and and any other variety of data work that would be called on to make a decision about what to do in the future, as opposed to documenting what happened in the past. I am uncertain whether A/B split testing fits my definition of analysis, but let’s be generous and say that it does.

A couple of other pointers:

• If you work for, say, a large university advancement department and aren’t sure whether analytics is used in other departments such as student admissions or recruitment, then answer just for your department. Same thing if you work for a regional office of a large non-profit and aren’t sure about the big picture.
• If you have little or no in-house expertise, but occasionally hire a vendor to produce predictive modelling scores, then you might answer “6” — but only if those scores are actually being well used.

Here we go.