Online surveys are useless

By Saturday, October 22, 2016 0 No tags Permalink 0

The magazine, Conde Nast Traveler, recently announced its readers’ choice awards. We read about it in the Toronto Star as it gushed this: Toronto’s Billy Bishop fourth overall among international airports, while Porter Airlines placed ninth among international airlines.


A small regional airline, with huge ongoing operational challenges, as the many, many posts on Porter Fail so vividly describe in painful detail, places ninth in the world?

And an airport with about 2,200,000 passengers per year ranks up there with Vancouver’s 20 million, or Pearson’s almost 39 million?

There just aren’t enough passengers using Porter or Billy Bishop to generate these results.

Unless… When one looks at how readers’ choice votes are cast, it becomes, maybe, a bit clearer: it’s an online survey, with all that means ‑ no statistical validity whatsoever, and with lots of potential for gaming.

Conde Nast itself admits: “Is there cheating? Yes, unfortunately. … Some listings performed beautifully, others not so much. No names here!”

Given such unreliability, why would any reputable newspaper devote any of its valuable editorial space to this story?

Good question.

Comments are closed.