Creating Unreal Engine 360° panoramas the easy way with ray tracing

Share on linkedin
Share on twitter
Share on facebook
Share on reddit
Share on digg
Share on email

360° VR panorama videos have been trending on YouTube since the viewing feature was introduced in 2015. Even without a VR headset they offer a uniquely immersive experience. What I’m referring to are mostly live action movies shot with a growing list of 360° video capable cameras. These panoramas are actually two 360×180 equirectangular 2D images stitched together and projected onto a conceptual 3D sphere.

They are often labelled as 3D or VR but it’s not really accurate. Virtual reality (VR) is currently described as a computer-generated world with some level of interactivity. However, with live action panoramas you are a passive viewer rather than an interactive participant. The ambiguity is compounded as VR headsets are the optimal way of viewing these 360° videos.

Some also refer to 360° videos as 3D since you can look all around, either by spinning the sphere or moving a mobile device. They should not be confused with 3D content shown on theatrical screenings or 3D capable televisions, which are viewed with special glasses. Monoscopic panoramas provide no depth information. Stereoscopic panoramas do provide some depth perception but must be viewed with a VR headset.

However, in both cases panoramas add much entertainment value by giving each viewer the potential for a unique experience. The possible applications range from computer-generated animated feature films, to architectural fly-by renderings of yet to be built structures.

So that’s a long preamble to this without the words ray tracing. This post is about creating 360° animated panoramas the easy way using the Unreal Engine and Imagination’s Wizard GR6500 ray tracing hardware. But why is creating these panoramas so much easier with ray tracing?

Creating panoramas with a rasterizer

The Unreal Engine is considered to be a rasterizer. The difference between a rasterizer and a ray tracer is that with the former the geometry dictates which screen pixel is affected, but with the latter, the pixel decides which geometry affects its colour. Due to the limitations of a rasterizer, there are currently two ways to create 360° panorama stills or video sequences with the Unreal Engine:

  1. Render six sides of a cube into a cubemap by rotating a 90° field of view camera in a central location of the scene. The cubemap image is then converted to an equirectangular spherical 360° panorama with some well-defined projection methods essentially, projecting or shrink wrapping the cube onto a sphere. This method is fast but can result in stretching artefacts at the cube corners. An interesting discussion on this method can be found on the Unreal Engine answerhub.
  2. Render many sub panels of the full 360° by rotating a camera with a fixed field of view in a central location. You then stitch each panel together to create the 360° panorama. This method is more accurate but can be very time consuming depending on the required final resolution of the panorama. A detailed description of the process using render “slices” can be found on the Unreal Engine blog.

 

Creating panoramas with a ray tracer

With Imagination’s implementation of ray tracing in the Unreal Engine the panoramas are created in one single render. Using a lens shader, which applies a spherical projection establishing the camera direction for each pixel, we can create a complete 360° flat panorama image. Of course, not only do you get a 360° image or video in one easy step, you also get physically correct reflections, refractions and shadows. This could also be applied in creating stereoscopic panoramas by appropriately shifting the camera direction for each eye.

In early March at GDC 2017 in San Francisco, Imagination demonstrated the implementation of ray tracing in the Unreal Engine running in real-time on our Wizard GR6500 development hardware platform. The demo, running with the Vulkan graphic back-end, enables you to toggle between several modes, including raster, ray traced and panorama. You can read a full, in-depth discussion of the making of the demo and the challenges involved in this blog post. If you’d like to find out more please feel free to get in touch.

The Chairs and Table project is a well-recognised tutorial for learning Unreal. Below are two movies that illustrate the creation of 360° panoramas with our ray tracing technology.

This movie is the flat panorama before it is wrapped on a sphere:


This movie is the panorama with the player in 360° mode
:

Will Anielewicz

Will Anielewicz

Will has been playing with computers since 1967. After completing an Honors Computer Science degree at York University in 1974, he was a multi-discipline Masters candidate combining Fine Arts, Computer Science and Philosophy. The mandate was to create a computer graphics piano that could be used in live concert performances. In 1976 Will had one of the world's first exhibitions of computer graphic art, at a Toronto Canada art gallery. In 1982 Will was hired as the first employee of Alias Research the creator of Maya. Since then, Will has worked on 14 feature films (such as "Phantom Menace") and several award-winning commercials. Will now works for Imagination Technologies in the PowerVR ray tracing division as a senior software engineer. His goal is to continue to push the envelope of computer-generated imagery particularly related to computer vision and augmented reality.

Please leave a comment below

Comment policy: We love comments and appreciate the time that readers spend to share ideas and give feedback. However, all comments are manually moderated and those deemed to be spam or solely promotional will be deleted. We respect your privacy and will not publish your personal details.

Blog Contact

If you have any enquiries regarding any of our blog posts, please contact:

United Kingdom

benny.har-even@imgtec.com
Tel: +44 (0)1923 260 511

Search by Tag

Search by Author

Related blog articles

PowerVR Hidden Surface Removal

Sorting Objects and Geometry on PowerVR Hardware

It can be tough trying to figure out the best way to optimise your applications for the hardware during development. Luckily, we’re here to help and – this is the first of an ongoing series of blog posts highlighting useful tips and tricks from our brand-new documentation website.

Stay up-to-date with Imagination

Sign up to receive the latest news and product updates from Imagination straight to your inbox.

  • This field is for validation purposes and should be left unchanged.

PowerVR RayTracing is available now for licensing. Download our white paper on this revolutionary graphics technology.

Subscribe to our newsletter

  • This field is for validation purposes and should be left unchanged.