Formerly of Engadget, Forbes and BBC, Sharif Sakr is presently a consultant for XPRIZE and business development director at GameBench. You can follow him on Twitter @shotsheriff.
This blog post was inspired by a short but interesting chat with a game developer I met at GameBench’s recent open( London ) event.
This developer was in the midst of optimising the performance of his game across a range of Android devices, trying to ensure that most users would experience a minimum of 30 frames per second even if they owned slightly older devices. This task, he told me, was proving to be a nightmare.
The developer was fully aware that each chip vendor offers deep-level tools designed to help with this sort of performance optimisation, but he didn’t see these tools as ideal solutions. In fact, he regarded them as part of the problem. With a bit of paraphrasing, he said something along the lines of:
If all these chip companies had worked together to create one tool, instead of duplicating each other’s efforts, they’d have created something much better than anything that’s available today.
There’s some logic in this sentiment. It’s true that vendors have historically created separate tools for their own chips, leaving game developers with a pretty brutal set of choices: either become experts at all the different tools necessary to span all the popular chipsets, or rely on tools with a higher level of abstraction (offered by the likes of Apple, Google and Unity), or just forgo optimisation altogether and hope for the best.
On the other hand, it’s easy to see why existing tools have evolved the way they have, as a response to healthy competition between commercial rivals. If chip vendors did try to cooperate to build a single tool, this tool would inevitably be built upon lowest common denominators, undermining the differentiation that is necessary to push their products (and the market as whole) forward.
Instead, a cooperative spirit tends to manifest itself in a couple of different ways: Firstly, a vendor can agree to support industry standards, like the forthcoming Vulkan API overseen by the Khronos Group. Secondly, and just as importantly, a vendor can build chips that are deliberately made open to outside inspection. This latter philosophy helps to foster a market in third-party tools, which can complement a vendor’s own tools while also being tailored to the specific needs and concerns of a certain type of user.
This sort of openness is where Imagination really shines, setting an example for the rest of the industry. My own company, GameBench, makes good use of Imagination’s PVRScope performance counters to expose underlying GPU behaviour, which we then report back to our users (including game developers, publishers, device OEMs and product reviewers) in a very simple way, requiring no specialist knowledge, no multiplicity of proprietary tools and no rooting of devices.
(In case you’re interested, we specifically rely on PVRScope’s task load counters, and we’ll soon be adding the new clock speed counter offered in version 3.5 of Imagination’s PowerVR Graphics SDK, in order to provide a clear snapshot of how much spare GPU capacity is left on a device at each moment that a game is running.)
We supplement this GPU information with other resource usage metrics (such as CPU, RAM, battery and network traffic) as well as with vital frame rate measurements that game developers (like the one I quoted earlier) tend to care about. We then correlate all these data to gameplay screenshots, so that bottlenecks can be pinpointed, and we present the whole lot in the form of easily-digestible scorecards and charts in order to speed up the QA process.
GameBench works with other vendors’ chipsets too, allowing real-world game performance to be compared across the whole range of modern Android and iOS hardware. However, Imagination stands out in the way it explicitly allows and invites GameBench to do this, without the R&D that we have been required to invest on other chipsets.
Due to this need for special engineering on other platforms, it has taken GameBench two years just to be able to analyse the many thousands of devices that are currently in the market, and we appear to be the only ones to have had the necessary time, inclination and financial backing to achieve this.
When more chip vendors are prepared to take on the reputational risk of exposing the performance limits of their GPUs, and when they all offer the same performance counters to enable like-for-like comparisons, then more third-party tools like GameBench will spring up to make life easier for different types of users. The end result should be at least as good as if vendors had directly cooperated to make a single unified tool, and it will very probably be better.