CoolData blog

25 May 2010

Is the Do Not Call List bogus?

Filed under: Annual Giving, External data, Phonathon, Predictor variables — Tags: — kevinmacdonell @ 9:41 am

Logo of Canada's Do Not Call registry service.

Last week I told you how I obtained a list of phone numbers from Canada’s Do Not Call List (two million phone numbers!). I matched these up with phone numbers from an alumni database in order to create a potential new predictor variable for my models. Today I reveal my rather unexpected findings.

To recap: In 2008, Canada introduced the National Do Not Call List (DNCL), which gives consumers a choice about whether to receive telemarketing calls. Anyone can add their phone numbers to the list, and telemarketing companies are forced to avoid calling those numbers. Canadian registered charities, including universities soliciting donations via calling programs, are exempt from the DNC list. However, any organization may access the list — which we did, for the purpose of research. Similar registries exist in the U.S. and around the world.

The results of my little experiment looked odd right from the beginning. When I matched up phone numbers, I discovered that a whopping 42% of living alumni with a home phone number in the area codes of interest had in fact signed up for the Do Not Call List. That seemed awfully high to me — but, oh well, I certainly didn’t lack for comparative data. Any differences between the DNC group and all other alumni were bound to be significant.

Or not! Check out these findings:

  • The two groups (DNC / not DNC) hardly differed in their age distribution. The very oldest and the very youngest alumni registered at the lowest rate (37.6% and 38.9%), but participation in the List was nearly equal across all age levels.
  • Alumni who signed up for the DNC list were slightly more likely to be donors. (Counter-intuitive, I thought.)
  • When I narrowed the definition of ‘giving’ to gifts received recently via the calling program, I found no difference in giving between the DNC and the non-DNC group.  I had expected that people who object to being called by telemarketers would also give less in response to a call from alma mater, and I was very surprised with this result. Average pledge and rate of participation were almost exactly equivalent across both groups.
  • The number of alumni who were coded ‘do not solicit by phone’ were about equal for both groups, DNC and non-DNC.
  • The number of alumni who asked not to be solicited by affinity partners (credit card, insurance, etc.) was also about equal for both groups.

The problem was not that the results were unexpected; unexpected is almost always interesting. No, the problem was that the results were impossible to interpret. The intersection of the DNC list with the alumni database was distinguished by an almost total lack of pattern or tendency. There were three possible conclusions to draw from this, one of which must be correct:

  1. The two data sets were completely unrelated due to some undiagnosed error in the analysis.
  2. The two data sets were related, but alumni draw a complete distinction between telemarketers and our student callers. They want off the calling lists of marketers, but this has nothing to do with their attitude toward alma mater and its fundraising efforts. If true, this would be good news indeed. But somehow I doubt it!
  3. The DNC list is a random data set. The near-total lack of distinguishing features strongly suggests that the DNC list is just a random sampling of the Canadian population. In other words, the list has been diluted by the mass uploading of phone numbers, despite security measures in place to prevent that from happening. If numbers are being added to the list without householders’ knowledge, the data do not represent people’s attitudes and intentions and are therefore worthless for the purpose of analysis.

Regardless of what the answer is, one thing is certain: We must never allow the DNC list to be applied to charities and nonprofits without a fight. This (possibly bogus) list will cut indiscriminately across a broad cross-section of anyone’s donor base, and a ban on calling would seriously harm any phone-based fundraising effort. Fortunately there does not seem to be any intention to extend the reach of the DNC list at present.

Getting back to the matter of finding new predictors: Every once in a while I get it in my head that the potential in our database is tapped out as far as new predictors goes. There HAVE to be other sources of data on our constituents that will provide amazing new insights into their behaviour. Sometimes going outside the database is worthwhile (survey data, for example) and sometimes  it just isn’t.

The lesson might be: Unless the data you covet relates directly to your constituents’ relationship with (or attitude towards) your institution, it may not be worth a great deal of time or money to acquire it.

Postscript: I’ve just had an opportunity to run the same lists of phone numbers against another and much larger university database. Once again, the binary variable “On the Do Not Call List” behaved like a randomly-generated number. I found that almost a third of the alumni population with phone numbers in the database is supposedly on this list, but the tiny fraction of a difference in giving behaviours between the DNC and not-DNC groups were not statistically significant.

20 May 2010

External data project: “Do Not Call” as a predictor

Filed under: External data, Phonathon, Predictor variables — Tags: , — kevinmacdonell @ 7:40 am

Here’s a little variable-creation project which shouldn’t cost much and might yield new insights into the behaviour of your database constituents, especially in connection with propensity to give over the phone: Have they registered their home phone numbers with the Do Not Call list?

A number of US states and some countries such as Canada and the United Kingdom have created these registries for  citizens keen on avoiding getting solicited at home by telemarketers. Here in Canada, all a person has to do to register is go to a website and enter their phone number(s). Commercial telemarketers are prohibited from calling any numbers on the list; violations bring stiff penalties. (If anyone can catch them.)

Charities such as my employer are exempt from the ban, so it doesn’t affect the phonathon program (although we must adhere to the stated preferences of our alumni and maintain an internal do-not-call list). But it stands to reason that there might be some connection between not wanting a call from a telemarketer, and not wanting any phone solicitation whatsoever, including from us. If being registered with the DNCL is negatively correlated with phone-solicited giving, we might gain a useful predictive variable about the people who have not already taken the step of adding themselves to our internally-maintained exclusion list.

In Canada, organizations may sign up to access this list of banned phone numbers. (Visit National DNCL page.) If they are businesses seeking to solicit Canadians by phone, they have no choice: They have to sign up and pay for the lists in order to exclude them from their calling efforts. But there’s nothing to stop exempt charities from signing up as well for the purposes of research.

So that’s what I did. In January 2009, I registered our university as a user of the Do Not Call service, and downloaded six lists of phone numbers. Each list, comprised of a single area code, cost $55. I chose the codes that captured the primary geographic areas where our alumni live. The top six area codes covered 80% of living alumni. After that it would have been a case of diminishing returns — I would have had to purchase seven more area codes to get to 90%. For this experiment, I was good with 80%.

What you get is just a list of phone numbers, but these files are huge — several megabytes in some cases, even in compressed format. If you purchase more than one file, open the smallest one in Excel to inspect the data. In order to match up these numbers with the home phone numbers in your database, you’ll need to ensure that they’re formatted in exactly the same way (i.e., no dashes, full 10 digits, whatever).

That done, now you can simply bring in the file as a new variable in your model. (If you’re using Data Desk, follow these directions for adding a new variable to an existing data set.) You may need to create a variable name, as the file you’ve downloaded might not have a column label and will use the first phone number by default. Code your matches as ‘1’ and everyone else as ‘0’, and test the results against ‘Giving’, or whatever your predicted value happens to be. Keep in mind that your list is specific to a geographic region; while you’re testing for a Do Not Call effect, you will want to exclude records outside the country or state you’re studying.

I’ve tested against both lifetime giving AND phonathon-only giving, and got some interesting results, which I’ll write about later. Give it a try.

Create a free website or blog at WordPress.com.