During the Embedded Technology 2013 conference and exhibition, Imagination is taking the opportunity to launch its new PowerVR Series2 ‘Raptor’ architecture for an image signal processor (ISP). The ISP pipeline takes raw camera data from a CMOS sensor and turns it into a high quality photo or movie. The PowerVR ‘Raptor’ ISP architecture is based on a highly scalable and power efficient design which can be deployed into many application areas. The initial architectural family announcement will be followed up in due course with core IP family members aimed at specific markets.
‘Raptor’ is the final piece in the multimedia IP puzzle which allows Imagination to cover every aspect of visual computing: GPU, VPU, display and ISP. Using PowerVR components, it is now possible to perform content creation and acquisition, compression and encode for transmission, decode, composition and display. And the system created this way can do much more, as we will see.
The ‘Raptor’ feature set is comprehensive in the sense that it interfaces directly to a CMOS sensor – no need for a stacked ISP on the sensor chip itself – and it includes all of the functions needed to prepare images or streaming videos for insertion into a real-time encoder or display to the viewfinder. In fact, it will do those things simultaneously, therefore reducing the need for memory traffic and eliminating the need for CPU interaction with those basic functions. Included in the pipeline hardware are a set of powerful, configurable image processing blocks to handle the “three A’s” (Auto focus, Auto Exposure, Auto White Balance) in hardware but controlled by the CPU for maximum flexibility.
The fundamental reason that the ISP must be so capable is that user expectations in, say, the mobile handset market are that the camera must perform at least as well as a dedicated point and shoot digital still camera for both still images and videos. This is from a lens and sensor system which is highly constrained by comparison, so the ISP must correct for all of the defects introduced by these limitations. ‘Raptor’ achieves this by including tunable blocks to correct for various lens aberrations, for variations in illumination across the sensor surface, to remove noise introduced by very high density sensors in low light and to remove bad pixels. Only then is the raw image turned into an RGB image and tuned for the characteristics of the sensor before being sent off for display and/or encode.
All of these operations are done in the most configurable way possible in order to interface to the widest selection of CMOS sensors.
The job of the ISP is evolving rapidly away from being a simple image or video capture device; forward looking features of the ‘Raptor’ architecture include the ability to interact with other powerful vision and video processors in the multimedia subsystem. One example of this is a dedicated interface to a PowerVR video encoder in order to implement Zero MemoryTM low latency video encode for live video conferencing; another is the production of image statistics, or hints, which can be used for a range of imaging and vision enhancements.
Computational photography using the GPU for heavy duty processing is already within the range of the abilities of ‘Raptor’, and forward looking applications include vision applications using multi-camera arrays (‘Raptor’ can simultaneously support multiple sensors with widely different characteristics) as well as higher pixel depth applications for industrial and automotive use. In its most cost-efficient form, ‘Raptor’ supports 10-bit pixel depths which matches the 10-bit encode and decode capability already offered by PowerVR video processors. It can be configured to support much higher bit depths (up to 16) when needed.
‘Raptor’ is a critical component of a PowerVR vision IP system. By integrating graphics, display, video processors and the ISP in a complete multimedia subsystem, system designers can create synergies among the components – a classic example of when the whole is much more than the sum of its parts. Combined with Imagination’s deep and extensive experience with hardware platforms using multiple operating systems, this will make it possible for OEMs to create standout multimedia systems offering premium functions which are immediately visible to the user.
For more news and announcements on our PowerVR visual IP, including GPU, VPU, display and ISP cores, make sure you follow us on Twitter (@ImaginationTech and @PowerVRInsider) and keep coming back to our blog.