USA Today announced on June 17, 2015 that SkyTrax awarded Porter Airlines the year’s best regional airline in North America, even though the year is only half over.

The U.K. travel consultantcy operates the self-appointed “world’s largest review site” and even has a gold seal to prove the company’s importance.

Also, shown below is a gold seal award from 2012. That’s the year the Advertising Standards Authority, the UK’s independent regulator of advertising across all media, upheld a complaint against SkyTrax that included five charges of misleading claims.

It appears that all that glitters is not gold

It seems that frequent business travellers, commenting in a group forum noticed a disconnect as early as 2011 when comments they made on SkyTrax surveys failed to appear in the subsequent published reviews. Either positive reviews about airlines that SkyTrax didn’t favour did not appear or negative reviews about airlines that SkyTrax did favour suffered the same fate.

Three years late, the credibility problem seemed to persist. In 2014, the Abu-Dhabi based carrier, Etihad Airways announced that it wouldn’t be bothering with the Skytrax ratings service. The airline had expressed doubts the Skytrax system.

As late as March 2015, Airline Passenger Guru a forum for airline passengers says this about a SkyTrax rating list on the world’s top airports.

“Like most ranking lists it is entertainment rather than serious and substantiated information.”

Indeed, a look at the SkyTrax reviews on Porter reinforces that sentiment. Understandably the number of 0/10 review ratings dating back to April 2014 sabotages the SkyTrax placement of Porter as the best North American regional airline for 2015.

How can a consultancy come up with a best in class when people paying to use the product disagree so vehemently?

Is there something else is at play here?