![]() ![]() The charts in the PC Perspective article don't support this thesis, as they show the difference between Intel and AMD decreased in Geekbench 5. There is a significant error in the table in the article that may have lead the author to conclude Geekbench 5 is biased against AMD. I'm also surprised by the claims that Geekbench 5 is biased towards Intel and Apple, and against AMD and Android, given the references in the article. For example, the Machine Learning workload uses 32-bit floats, not 64-bit doubles. We strive to make Geekbench a reliable cross-platform benchmark, and we appreciate criticism regarding Geekbench, but the article gets several things wrong about Geekbench 5. Hi! I'm the founder of Primate Labs and one of the Geekbench developers. While designed to be used across any computing platform, it has commonly been used as one of the key benchmarks for mobile devices, including smartphones, tablets, and mobile PCs. In addition to CPU performance, Geekbench has typically included tests for graphics, memory, battery life and other system functions. Tirias Research would still recommend using it as part of a suite of benchmarks, but we have some concerns about the overall value of the new benchmark and recommend questioning anyone just quoting the Geekbench 5 scores. While there does seem to be some significant improvements in Geekbench 5, it appears to be less of system-level benchmark than before and even appears to have added some biases that did not exist in previous versions. PC Perspective did a similar comparison both versions of the benchmark on a pair Intel and AMD-based PCs and the differences are rather drastic - showing a much higher relative performance of the Intel-based platforms than the AMD platforms. The Apple OSs and Intel processors appear to have benefitted most from the benchmark changes, much of which is likely attributed to the elimination of the memory and battery test that benefit the AMD processors and Android OS. In a review of some of the results posted for the same platforms on both Geekbench 4 and Geekbench 5, we found that the change in performance numbers was higher on the Android OS and AMD processor than on Apple’s operating systems or Intel processors (the table provides a sample of these results). The baseline for Geekbench 5 is 1,000 for an Intel Core i3-8100 four-core/four-thread Coffee Lake processor. The baseline for Geekbench 4 was a score of 4,000 for an Intel Core i7-6600U dual-core/four-thread Skylake processor. Unfortunately, there is no correlation in the performance numbers from one version to the other and with a change in the baseline processor, the results on all platforms will be significantly lower. ![]() This change also penalizes the machine learning solutions in mobile platforms. In fact, it goes against the trend to reduce the precision level while maintaining a high-level of accuracy. In addition to these changes, the machine learning test is using just 32-bit floating point, rather than the more common levels of precision, such as 8-bit or 16-bit Integer for inference, 16-bit floating point, or the more recent bfloat16 for training. However, the hope is that benchmarks will get better over time. And every so often you catch companies cheating by using a corrupted benchmark. Companies look for idiosyncrasies in the benchmark that can be exploited or ways to optimize a platform to generate a higher score. What makes developing the perfect benchmark even more difficult is that every company wants to demonstrate the highest scores, which leads to the processing of “gaming” benchmarks. We have always maintained that no benchmark is a perfect representation of every application or workload, which is why we always recommend using multiple benchmarks for evaluating any compute platform, whether it’s a smartphone, a PC, or a server. However, a few of the changes have us at Tirias Research scratching our heads and questioning the value of some of those changes. The changes to the benchmark were made to address some limitations with the previous generation and to generally make the benchmark a better system-level benchmark. The new version touts some significant changes that drastically change the scores compared with the older Geekbench 4 version. Primate Labs recently released a new version: Geekbench 5. Among the many available, Geekbench is popular as a cross-platform benchmark that tests both single-core and multi-core CPU performance using simulated workloads similar to common applications. The processor design community can’t live without benchmarks, and yet it often maintains a love-hate relationship with them. ![]()
0 Comments
Leave a Reply. |