Is the Do Not Call List bogus?

Logo of Canada's Do Not Call registry service.

Last week I told you how I obtained a list of phone numbers from Canada’s Do Not Call List (two million phone numbers!). I matched these up with phone numbers from an alumni database in order to create a potential new predictor variable for my models. Today I reveal my rather unexpected findings.

To recap: In 2008, Canada introduced the National Do Not Call List (DNCL), which gives consumers a choice about whether to receive telemarketing calls. Anyone can add their phone numbers to the list, and telemarketing companies are forced to avoid calling those numbers. Canadian registered charities, including universities soliciting donations via calling programs, are exempt from the DNC list. However, any organization may access the list — which we did, for the purpose of research. Similar registries exist in the U.S. and around the world.

The results of my little experiment looked odd right from the beginning. When I matched up phone numbers, I discovered that a whopping 42% of living alumni with a home phone number in the area codes of interest had in fact signed up for the Do Not Call List. That seemed awfully high to me — but, oh well, I certainly didn’t lack for comparative data. Any differences between the DNC group and all other alumni were bound to be significant.

Or not! Check out these findings:

• The two groups (DNC / not DNC) hardly differed in their age distribution. The very oldest and the very youngest alumni registered at the lowest rate (37.6% and 38.9%), but participation in the List was nearly equal across all age levels.
• Alumni who signed up for the DNC list were slightly more likely to be donors. (Counter-intuitive, I thought.)
• When I narrowed the definition of ‘giving’ to gifts received recently via the calling program, I found no difference in giving between the DNC and the non-DNC group.  I had expected that people who object to being called by telemarketers would also give less in response to a call from alma mater, and I was very surprised with this result. Average pledge and rate of participation were almost exactly equivalent across both groups.
• The number of alumni who were coded ‘do not solicit by phone’ were about equal for both groups, DNC and non-DNC.
• The number of alumni who asked not to be solicited by affinity partners (credit card, insurance, etc.) was also about equal for both groups.

The problem was not that the results were unexpected; unexpected is almost always interesting. No, the problem was that the results were impossible to interpret. The intersection of the DNC list with the alumni database was distinguished by an almost total lack of pattern or tendency. There were three possible conclusions to draw from this, one of which must be correct:

1. The two data sets were completely unrelated due to some undiagnosed error in the analysis.
2. The two data sets were related, but alumni draw a complete distinction between telemarketers and our student callers. They want off the calling lists of marketers, but this has nothing to do with their attitude toward alma mater and its fundraising efforts. If true, this would be good news indeed. But somehow I doubt it!
3. The DNC list is a random data set. The near-total lack of distinguishing features strongly suggests that the DNC list is just a random sampling of the Canadian population. In other words, the list has been diluted by the mass uploading of phone numbers, despite security measures in place to prevent that from happening. If numbers are being added to the list without householders’ knowledge, the data do not represent people’s attitudes and intentions and are therefore worthless for the purpose of analysis.

Regardless of what the answer is, one thing is certain: We must never allow the DNC list to be applied to charities and nonprofits without a fight. This (possibly bogus) list will cut indiscriminately across a broad cross-section of anyone’s donor base, and a ban on calling would seriously harm any phone-based fundraising effort. Fortunately there does not seem to be any intention to extend the reach of the DNC list at present.

Getting back to the matter of finding new predictors: Every once in a while I get it in my head that the potential in our database is tapped out as far as new predictors goes. There HAVE to be other sources of data on our constituents that will provide amazing new insights into their behaviour. Sometimes going outside the database is worthwhile (survey data, for example) and sometimes  it just isn’t.

The lesson might be: Unless the data you covet relates directly to your constituents’ relationship with (or attitude towards) your institution, it may not be worth a great deal of time or money to acquire it.

Postscript: I’ve just had an opportunity to run the same lists of phone numbers against another and much larger university database. Once again, the binary variable “On the Do Not Call List” behaved like a randomly-generated number. I found that almost a third of the alumni population with phone numbers in the database is supposedly on this list, but the tiny fraction of a difference in giving behaviours between the DNC and not-DNC groups were not statistically significant.