Imagination is of course at GDC this week, and this year, with a nod to the current excitement around augmented reality, we’re having some fun with gnomes and emoji. We’ve given our face detection demo a (virtual) face-lift – turning the faces of visitors to our booth into gnomes!
So what exactly is going on here? Well, we’re taking a live camera feed, then per-frame and in real time we’re detecting the regions of that feed which contain faces. We’re then rendering garden gnome hats and facial hair on top of those regions.
Why? Well, why not?
On the technical side, the demo is running on a Google Nexus Player (featuring our PowerVR G6430 GPU), which has been modified to support passing a live webcam feed through to OpenCL for GPU-based face detection, with the results rendered to the screen using OpenGL ES. The algorithm doesn’t use neural networks, which have become popular in recent years. Instead, the basis of the detector in the demo is a local binary pattern (LBP) classifier, which examines the neighbourhood of individual pixels and gives a result based on thresholding the difference between the centre pixel and each of its neighbours.
In this setup, the LBPs are the weak classifier, and the outputs of groups of those weak classifiers are combined into cascades of strong classifiers. The strong classifier decides whether the combined evaluation of the LBPs should be rejected or if it is indeed a face. Alongside this core algorithm are some other heuristics such as accurately extracting regions containing skin colours, and handling rotated faces. Click on the link if you’re interested in a deep-dive on traditional computer vision on PowerVR, including a look at face detection.
Our neural-net-based face detection demo showcases not only face detection but identification, where previously registered faces are remembered after disappearing from view and coming back into view. Both of these demos run on the GPU, but by using our new PowerVR 2NX neural-network accelerator, face detection can be off-loaded to dedicated hardware which has been tuned for that kind of workload, enabling much greater performance, while freeing up the GPU for other work (perhaps for rendering physically-based gnomes or emoji). Read this blog post for more detail on the benefits of the dedicated neural network accelerator.
So if you’re at GDC this week come find us at booth 102 and see how you look after gnomeification! Why not get someone to take a pic and let us know on Twitter using #IMG_GDC18. We look forward to seeing your new look!