Deep dive: Hardware IP for computer vision [part 2]

Share on linkedin
Share on twitter
Share on facebook
Share on reddit
Share on digg
Share on email

Modern mobile application processors are highly heterogeneous, combing a variety of different hardware components optimized for different tasks. As shown in the figure below, a processor designed for vision might include an Image Signal Processor (ISP) for acquiring image sensor data, a vector processor such as a GPU for efficient data-parallel operation on pixels and feature vectors, and a CPU for implementing higher-level heuristics and decision-making. A sensor-fusion device might further combine vision processing with complementary technologies such as radar, ultrasonic, sonar and LIDAR, as well as input from compasses, accelerometers and wheel encoders.

11-Components of a modern vision processorComponents of a modern vision processor

Imagination provides all of the hardware components needed to build efficient vision products today, including:

Image Signal Processor

The PowerVR Raptor ISP can perform many common image pre-processing tasks such as noise reduction and normalizing colour and gamma values. When combined with a PowerVR GPU, areas of an image can be de-warped on-the-fly into on-chip local memory for onward GPU compute processing.

Vector Processor (GPU)

PowerVR GPUs are high-performance vector processors that can efficiently execute many data-parallel computations at low power. As explained in a previous article, convolution filters can be highly optimized for parallel execution with local memory re-use, as well as other tasks such as constructing an image pyramid or integral image. Baidu’s engineers applied similar techniques to offload a convolution neural network from the CPU to PowerVR GPU in a handset using a MediaTek MT6595 based SoC, improving matching performance while also increasing battery life of the handset from a couple of hours to a full day’s use.

Accelerators

As well as supporting efficient vector processing, PowerVR GPUs provide several hardware accelerators that can be leveraged by vision algorithms. The Texture Processing Unit (TPU) supports efficient sampling of image pixels from memory combined with optional interpolation, for example to prevent aliasing during image registration. The colour-space conversion unit supports on-the-fly conversion of sampled pixels for cases where luminosity data alone is insufficient, for example when detecting skin colour in a face or searching for colours in road signs.

Some products support hardware acceleration of more specific vision tasks, for example the OMAP4470 processor from Texas Instruments includes a hardware face detector. Vadaro chose this processor to create their retail analytics camera, utilizing the face detection hardware to identify shoppers. The resulting coordinates are then fed into an SVM classifier that identifies customer profiles such as age and gender, and which is optimized by offloading compute operations such as dot products to the PowerVR GPU. Similarly, some automotive devices contain hardware to accelerate common tasks such as HOG detection. In general the decision to commit a specific task to hardware is usually taken after considering its likely use across all users of the device and also whether the algorithm may benefit from being adapted in the near future, for example as a result of improvements in computer vision research.

Hopefully this article has provided a short overview of the on-chip processing blocks that are relevant to computer vision applications.

Further reading

Computer vision is what we’ll continue to focus on for the next section of our heterogeneous compute series; stay tuned for an overview of how you can use OpenCL on PowerVR for face detection.

Here is a menu to help you navigate through every article published in this heterogeneous compute series:

 

Please let us know if you have any feedback on the materials published on the blog and leave a comment on what you’d like to see next. Make sure you also follow us on Twitter (@ImaginationTech, @GPUCompute and @PowerVRInsider) for more news and announcements from Imagination.

Alex Kelley

Alex Kelley

Alex Kelley has over 20-years of sales, marketing, and general management experience in 3D computer graphics and has worked in the USA and several countries in Asia. Since joining Imagination Technologies Alex has launched the Visualizer brand, which has most recently brought a photorealistic virtual camera to SketchUp transforming the way people view 3D models. Alex was a Vice President at Caustic Graphics, a start up acquired by Imagination, and before that held Vice President roles at Autodesk and Alias. Alex is fluent in Japanese, and holds a B.S. and M.S. degree in Computer Science from Arizona State University.

1 thought on “Deep dive: Hardware IP for computer vision [part 2]”

Please leave a comment below

Comment policy: We love comments and appreciate the time that readers spend to share ideas and give feedback. However, all comments are manually moderated and those deemed to be spam or solely promotional will be deleted. We respect your privacy and will not publish your personal details.

Blog Contact

If you have any enquiries regarding any of our blog posts, please contact:

United Kingdom

benny.har-even@imgtec.com
Tel: +44 (0)1923 260 511

Search by Tag

Search by Author

Related blog articles

PowerVR Hidden Surface Removal

Sorting Objects and Geometry on PowerVR Hardware

It can be tough trying to figure out the best way to optimise your applications for the hardware during development. Luckily, we’re here to help and – this is the first of an ongoing series of blog posts highlighting useful tips and tricks from our brand-new documentation website.

Stay up-to-date with Imagination

Sign up to receive the latest news and product updates from Imagination straight to your inbox.

  • This field is for validation purposes and should be left unchanged.

Subscribe to our newsletter

  • This field is for validation purposes and should be left unchanged.