Most mainstream consumers know very little about benchmarks to care about them, and the ones who do, know that synthetic benchmark scores need to be taken with a pinch of salt at the best of times. However, that doesn’t seem to be deterring smartphone manufacturers from resorting to artificial (and often unethical) means of inflating benchmark scores.
While Samsung, OnePlus and other reputable companies have all got caught up in this controversy from time to time, AnandTech now says that Huawei, already reeling under charges of surreptitiously passing off DSLR shots as camera samples of its smartphones, is also guilty of fudging benchmark scores like many of its Chinese competitors.
The company’s Huawei P20 Pro, Honor Play and Huawei Nova 3 have been founded artificially inflating benchmark scores by whitelisting certain apps that are used to record the scores. This means the phones do not throttle when these apps are used which is not the case in regular usage.
According to the report, manufacturers often whitelist popular benchmarking applications on their smartphones to prevent thermal throttling while running the CPU at its full clock speed, allowing these apps to artificially extract peak performance that’d otherwise be impossible to attain in real-world usage.
As a result, the performance numbers turn out to be unnaturally inflated, and while that may be good for marketing, it reduces power efficiency drastically as the chip is pushed beyond its standard operating frequencies.
To illustrate its point, the report cited the example of Huawei’s recently-released gaming phone, Honor Play, which apparently yielded drastically different results with the benchmark detection mode turned on and off.
When confronted with evidence at the sidelines of the recently-concluded IFA trade show in Berlin, the President of Software at Huawei’s Consumer Business Group, Dr. Wang Chenglu, reportedly was relatively nonchalant about what seems to be serious allegations of false advertising.
According to AnandTech, Dr. Wang altogether dismissed the importance of benchmarks in evaluating a smartphone, saying that these apps in no way represent real-world use. He also apparently said that the company is working with industry partners for over a year to come up with tests that most closely mimic real-world use cases.
However, till such time as a new set of benchmarks become standardized, the company seemed to be in no mood to move away from its deceptive practices. According to him such misleading marketing is becoming ‘common practice’ in China, where ‘other manufacturers also mislead with their numbers’. While he might not have said it in as many words, AnandTech believes what the combative Huawei exec meant that day was, ‘others do the same testing, get high scores, and Huawei cannot stay silent’.
It will be interesting how this whole thing plays out in the future, but with the leading players in the industry unwilling to take a backward step, it will be better if all the smartphone-makers were to follow in the footsteps of companies like Meizu, which declares at the outset that all benchmarks will be run on a special high-performance mode.
