MediaTek Caught Cheating in Benchmarks; Denies Allegations

Most smartphone buyers know too little about performance benchmarks to care about them, and the ones who do, know that synthetic benchmark scores need to be taken with a grain of salt at the best of times. However, that doesn’t seem to be deterring tech companies from resorting to artificial (and often unethical) means of inflating benchmark scores.

The problem was thus far believed to have been limited to smartphone vendors, with Samsung, OnePlus and Huawei being some of the prominent companies to have been caught up in this controversy from time to time. However, AnandTech now claims that chipmaker MediaTek is also guilty of fudging benchmark scores of some of its smartphone SoCs.

The blog says it started its investigation when it found that the European version of the Oppo Reno 3 Pro with the older Helio P95 chipset was returning significantly higher benchmark scores than what was expected from a Cortex-A75 class SoC. However, what set the alarm bells ringing was the Chinese version of the Reno3 with the newer and supposedly faster Dimensity 1000L chipset, performing much worse in those same benchmark tests.

As it turns out, MediaTek was specifically preventing the benchmarking apps from activating the chip’s thermal throttling mechanism so as to return higher scores than they would in real-world scenarios. As per the report, the phone’s power_whitelist_cfg.xml file had a list of popular applications, including some of the aforementioned benchmark apps, with various power management tweaks applied to them.

Some of the common benchmarking apps found on the list include the likes of PCMark, GeekBench, AnTuTu, 3DBench and Quadrant alongside a few Chinese benchmark apps. The report further says that similar configurations were found on a whole bunch of other MediaTek devices, including the Vivo S1 with the Helio P65, the Xiaomi (Redmi) Note 8 Pro with the Helio G90, the Realme C3 with the Helio G70 and the Sony XAI with the Helio P20, among others.

MediaTek has denied all allegations of cheating, saying that its power management tweaks enable benchmark apps to better represent the hardware capabilities of their SoCs. In a statement to AnandTech, the company said: “MediaTek follows accepted industry standards and is confident that benchmarking tests accurately represent the capabilities of our chipsets … We believe that showcasing the full capabilities of a chipset in benchmarking tests is in line with the practices of other companies and gives consumers an accurate picture of device performance”.