I’ve been thinking a lot lately about how an organization gets good at making decisions informed by data. Or, in other words, how to build business intelligence and analytics teams. This preoccupation started with a talk I gave a couple of months ago to a gathering of Advancement leaders from across Canada. I was asked to talk about analytics in general and how our department in particular got to where we are today. Since then, I’ve also spoken to folks from other universities on the same topic.
All this talking has been helpful for me in organizing my thoughts, and I’ve come to realize a number of things in retrospect, ways in which we might have evolved more quickly. One of these is a realization about what it means to make data and analytics an “organizing principle.”
For my talk in May I was asked to begin with an overview of analytics, so I’ll devote this post to that topic. In a future post, I will share what we learned on our journey.
Because analytics is an ever-evolving field, I avoid dictionary-like definitions for analytics. I find it more helpful to talk about what analytics “looks like” in terms of the types of work it consists of, the skill sets of the people doing the work, and the organizational structure of the team (if it’s a team).
In my mind, these concepts have resolved into a “triad of threes” … The work itself fits into three tiers, the ideal analytics practitioner is a “triple threat”, and the team is made up of three distinct teams or functions. (If what I’m presenting here is an oversimplification, at least it’s a structurally satisfying one.) What I’m talking about is fairly conventional — I’m not inventing anything — but it’s supported by my own experience.
First, the work itself. Analytics practice today works at three distinct levels: Descriptive, predictive, and prescriptive.
Descriptive analytics serves the business with information, specifically information about the past, which helps us understand current performance in relation to the past. It attempts to answer the questions, “How have we done?” and “How are we doing now?” This is the realm of reporting and a lot of what is referred to as Business Intelligence. Although this is a starting point for any analytics program, that doesn’t mean it’s easy or that it doesn’t have aspects that are advanced. KPI development, support for performance management, and ad hoc data analyses to answer specific business questions might be included in this tier.
Predictive analytics is about predicting the future. Not “the future” in general, but the behaviour of individuals. Predictive modelling is a set of techniques for ranking individuals by their likelihood to engage in some behaviour of interest (making a bequest, becoming a donor, attending an event, etc.). The business goal might be prospect identification, or focusing limited resources to save time or money.
And finally, prescriptive analytics provides advice on what action to take to influence a behaviour of interest. While predictive analytics gives us an idea who’s more likely to, say, sign up for a high-end credit card from a financial institution, prescriptive analytics suggests what types of interventions (targeting advertisements, for example) that would inspire a customer to actually do it.
Prescriptive analytics is the newest type of analytics and the most advanced — I don’t think it’s the same as A/B testing found in direct marketing — and still rare in the nonprofit and advancement sector. I’m using an example from the financial services industry for a reason: my team is just beginning to explore this type of work, and I’m not aware of anyone else doing it. (If you’re reading this in a year or two from now, the situation might be different.)
If your organization is doing a good job on reporting, business intelligence, predictive modelling, and maybe some forecasting as well — then you’re most likely doing very well in comparison with your peer institutions in terms of function.
So much for the work. What about the people?
There is a popular notion of what the ideal analytics practitioner looks like in terms of education, work experience, and skills. That person, who might be styled a Data Scientist, is what I have called a “triple threat” — he or she has extensive domain expertise (fundraising, engagement, and/or marketing), a background in computer science (adept at writing scripts in SQL, R, Python or other language to extract and transform data for analysis and advanced modelling), and mathematics (with an array of advanced statistical methods in his or her toolbox).
The problem is, such professionals are both rare and in high demand. You won’t find many of these folks working in our sector — at least not for very long. Their natural habitat is more likely to feature Big Data, not the “little data” we’ve got, and machine learning, rather than our old standbys such as multiple linear regression. I have already elaborated on these points in the blog post I link to above, Mind the data science gap. Suffice to say, we do not currently aspire to hire data scientists.
That doesn’t mean the ideal isn’t a useful model, however. When we hire, it makes sense to single out candidates with skills in one of the three areas, and who seem to have some aptitude for picking up skills in complementary areas. The strategy here is not to hire a data scientist, but to grow a reasonable facsimile of one. If you’ve got an employee who has some subject-matter knowledge, has a penchant for self-learning technical skills (on her own time perhaps), is curious about things and diving into the data, and who is a good communicator — such a person will add a lot of value in a BI role.
You can have the right people doing the right work, but they need to work in an organizational structure that promotes data-informed decision making. So, the third and final aspect: The organizational structure. There is no one perfect structure, but keeping with the theme of “three,” I think that a three-tier setup makes sense. In a large organization, each tier might be a team. In a smaller organization, each tier might be one person. (If one person is responsible for everything, this “structure” can be thought of as a way to organize or compartmentalize one’s own work.)
The first and foundational tier is the Technical Team, consisting of Advancement staff who might be responsible for building and/or maintaining a data warehouse dedicated to Advancement needs, building and maintaining materialized views and data models for use in BI software, developing complex reports and dashboards, integrating internal and external systems and platforms so that data from disparate systems can be merged or federated, and liaising with central IT.
This tier sounds very “IT”, but it’s important to recognize that it is distinct from the institution’s centralized IT department, which is responsible for maintaining hardware, servers, and the core database software itself, as well as managing the network and security.
So you’re not trying to replicate an IT shop, but you are building a team with specific technical skills. For any higher ed institution in which departments are not supported equally by central IT, having in-house expertise to integrate systems and develop data models tailored to business needs is definitely a key to success. Someone has to supply and support the data infrastructure, if central IT is too overtaxed to provide.
The next team is the Analysis Team, the people who build predictive models, define KPIs, do ad hoc analyses, and so on. This team (or person) benefits directly from the work of the Technical Team, freed from having to always extract and transform their own data. While analysis often implies exploration of the raw, unaggregated data, there’s a huge payoff in having a lot of the standard transformations (tedious and repetitive) pushed to the data warehouse level. Analysts add the most value when they’re interacting with clients to define business questions and present results, not struggling yet again with raw, transactional data that could be processed more efficiently and accurately with an ETL tool.
In my own workplace, the distinction between these two teams is something of an oversimplification, but it’s roughly analogous.
The third team is harder to define, as it may take various forms, depending on the organization. I’ve seen it referred to as the Executive Team, but a better name might be the Analytic Strategy Team or the BI Decision Team. We don’t have a name for it in my workplace, because our department doesn’t have such a group — yet. In fact, this is less a “team” than a solid business process. In any case, I’ve come to think it’s essential for data-informed decision making, and at the heart of analytics as an organizing principle.
The Analytic Strategy Team would be a cross-functional team made up of business sponsors (directors and managers of programs and units) and analysts from both the Technical and Analysis teams. In a data-driven organization, this team meets regularly to rank and prioritize analysis projects that have been submitted to the team as requests, called for by department leadership, or generated by the team members themselves. Projects rank higher for being supportive of current strategy, having a high perceived impact, having executive sponsorship, and so on.
Prioritizing is not the team’s most important role, however. As the hub of a framework for Advancement decision-making, the Analytic Strategy Team is there to ensure that when a business question is answered through analysis, there will be follow-through. The Team nails down the “why” and “how” of every analysis project: Zeroing in on the real business question that needs to be answered, drafting the general approach to answering the question, and (most critically) determining what actions will be taken if the answer is x, y, or z. Results and recommendations are channeled to a decision maker, who has agreed in advance to the definition of the business question.
Ideally, the department’s leadership team approves the ongoing analytics agenda. Having leadership sign off on the list of priorities fosters an integrated approach to making decisions as a whole department.
This team is important for focus — analysts do their best work if they can focus — but it’s even more important for driving decisions. Your team can be kept endlessly busy generating analyses, but it’s when it comes to the consequences of analyses that BI programs risk falling flat. Without the accountability implied by an agreed-on process of question, answer, and follow-through, analysts end up floating from one fishing expedition to another, generating “findings” that never get acted on, or fulfilling requests to support program managers’ foregone conclusions with “evidence.”
Of course we want to do some purely exploratory analyses without a defined outcome — but that’s not how data-informed decisions get made. As Thomas Davenport has written, “In the traditional analytics world, analysts may have lacked the ability to work closely with decision-makers to frame decisions appropriately, engage stakeholders, and structure decision processes and actions. Decision analysts in a business analytics environment need to move from back-office decision support to front-office decision consultants.”
Again I say, these observations about the “third team” are not drawn from my first-hand experience. These are things I’ve come to understand only recently. My naiveté is evident in “Score!” the book I co-authored with Peter Wylie and which was published just two years ago. What we wrote seemed to imply that all it takes is a supportive leader driving change from the top and engaged staff people with an aptitude for data work driving change from the bottom. They would somehow meet in the middle, and magic would happen. Well, we do need both of those forces, but nowadays I don’t see organizational change happening in the absence of a well-functioning business process that guides decision-making.
I’ve talked about the people, the types of work they do, and the structure of the team — all from a general perspective. In my next post, I will talk about the journey our own shop has taken towards building a BI/analytics program. Not surprisingly, the real-world program doesn’t arrive as neatly packaged as this general overview would suggest.
CoolData blog 