Creating Unreal Engine 360° panoramas the easy way with ray tracing

360° VR panorama videos have been trending on YouTube since the viewing feature was introduced in 2015. Even without a VR headset they offer a uniquely immersive experience. What I’m referring to are mostly live action movies shot with a growing list of 360° video capable cameras. These panoramas are actually two 360×180 equirectangular 2D images stitched together and projected onto a conceptual 3D sphere.

They are often labelled as 3D or VR but it’s not really accurate. Virtual reality (VR) is currently described as a computer-generated world with some level of interactivity. However, with live action panoramas you are a passive viewer rather than an interactive participant. The ambiguity is compounded as VR headsets are the optimal way of viewing these 360° videos.

Some also refer to 360° videos as 3D since you can look all around, either by spinning the sphere or moving a mobile device. They should not be confused with 3D content shown on theatrical screenings or 3D capable televisions, which are viewed with special glasses. Monoscopic panoramas provide no depth information. Stereoscopic panoramas do provide some depth perception but must be viewed with a VR headset.

However, in both cases panoramas add much entertainment value by giving each viewer the potential for a unique experience. The possible applications range from computer-generated animated feature films, to architectural fly-by renderings of yet to be built structures.

So that’s a long preamble to this without the words ray tracing. This post is about creating 360° animated panoramas the easy way using the Unreal Engine and Imagination’s Wizard GR6500 ray tracing hardware. But why is creating these panoramas so much easier with ray tracing?

Creating panoramas with a rasterizer

The Unreal Engine is considered to be a rasterizer. The difference between a rasterizer and a ray tracer is that with the former the geometry dictates which screen pixel is affected, but with the latter, the pixel decides which geometry affects its colour. Due to the limitations of a rasterizer, there are currently two ways to create 360° panorama stills or video sequences with the Unreal Engine:

  1. Render six sides of a cube into a cubemap by rotating a 90° field of view camera in a central location of the scene. The cubemap image is then converted to an equirectangular spherical 360° panorama with some well-defined projection methods essentially, projecting or shrink wrapping the cube onto a sphere. This method is fast but can result in stretching artefacts at the cube corners. An interesting discussion on this method can be found on the Unreal Engine answerhub.
  2. Render many sub panels of the full 360° by rotating a camera with a fixed field of view in a central location. You then stitch each panel together to create the 360° panorama. This method is more accurate but can be very time consuming depending on the required final resolution of the panorama. A detailed description of the process using render “slices” can be found on the Unreal Engine blog.

 

Creating panoramas with a ray tracer

With Imagination’s implementation of ray tracing in the Unreal Engine the panoramas are created in one single render. Using a lens shader, which applies a spherical projection establishing the camera direction for each pixel, we can create a complete 360° flat panorama image. Of course, not only do you get a 360° image or video in one easy step, you also get physically correct reflections, refractions and shadows. This could also be applied in creating stereoscopic panoramas by appropriately shifting the camera direction for each eye.

In early March at GDC 2017 in San Francisco, Imagination demonstrated the implementation of ray tracing in the Unreal Engine running in real-time on our Wizard GR6500 development hardware platform. The demo, running with the Vulkan graphic back-end, enables you to toggle between several modes, including raster, ray traced and panorama. You can read a full, in-depth discussion of the making of the demo and the challenges involved in this blog post. If you’d like to find out more please feel free to get in touch.

The Chairs and Table project is a well-recognised tutorial for learning Unreal. Below are two movies that illustrate the creation of 360° panoramas with our ray tracing technology.

This movie is the flat panorama before it is wrapped on a sphere:


This movie is the panorama with the player in 360° mode
:

Leave a Comment

Search by Tag

Search for posts by tag.

Search by Author

Search for posts by one of our authors.

Featured posts
Popular posts

Blog Contact

If you have any enquiries regarding any of our blog posts, please contact:

United Kingdom

benny.har-even@imgtec.com
Tel: +44 (0)1923 260 511

Related blog articles

What is PowerVR Automotive? Register NOW to hear our webinar.

The automotive industry is going through many changes and that is having a huge impact on the semiconductor IP industry. The vehicle will move from being predominantly mechanical to primarily a computer on wheels enabling a future of self-driving cars,

Image-based lighting

PowerVR Tools and SDK 2018 Release 2 now available

Here’s an early Christmas present for graphics developers – the release of the latest version of our PowerVR Tools and SDK! The headline features for this release include some exciting new examples demonstrating new techniques in our SDK, and some very

on stage in China

PVRIC4 a hit at ICCAD 2018 in China

Imagination’s PVRIC4 image compression tech garnered plenty of attention at the recent ICCAD China 2018 symposium, which took place on 29th and 30th November at the Zhuhai International Convention & Exhibition Centre, China. The annual event focusses on integrated circuit

The ultimate embedded GPUs for the latest applications

Introducing PowerVR Series9XEP, Series9XMP, and Series9XTP As Benjamin Franklin once said, only three things in life are certain: death, taxes and the ongoing rapid advancement of GPUs for embedded applications*. Proving his point, this week, Imagination has once again pushed

Stay up-to-date with Imagination

Sign up to receive the latest news and product updates from Imagination straight to your inbox.

  • This field is for validation purposes and should be left unchanged.