Creating Unreal Engine 360° panoramas the easy way with ray tracing

Share on linkedin
Share on twitter
Share on facebook
Share on google

360° VR panorama videos have been trending on YouTube since the viewing feature was introduced in 2015. Even without a VR headset they offer a uniquely immersive experience. What I’m referring to are mostly live action movies shot with a growing list of 360° video capable cameras. These panoramas are actually two 360×180 equirectangular 2D images stitched together and projected onto a conceptual 3D sphere.

They are often labelled as 3D or VR but it’s not really accurate. Virtual reality (VR) is currently described as a computer-generated world with some level of interactivity. However, with live action panoramas you are a passive viewer rather than an interactive participant. The ambiguity is compounded as VR headsets are the optimal way of viewing these 360° videos.

Some also refer to 360° videos as 3D since you can look all around, either by spinning the sphere or moving a mobile device. They should not be confused with 3D content shown on theatrical screenings or 3D capable televisions, which are viewed with special glasses. Monoscopic panoramas provide no depth information. Stereoscopic panoramas do provide some depth perception but must be viewed with a VR headset.

However, in both cases panoramas add much entertainment value by giving each viewer the potential for a unique experience. The possible applications range from computer-generated animated feature films, to architectural fly-by renderings of yet to be built structures.

So that’s a long preamble to this without the words ray tracing. This post is about creating 360° animated panoramas the easy way using the Unreal Engine and Imagination’s Wizard GR6500 ray tracing hardware. But why is creating these panoramas so much easier with ray tracing?

Creating panoramas with a rasterizer

The Unreal Engine is considered to be a rasterizer. The difference between a rasterizer and a ray tracer is that with the former the geometry dictates which screen pixel is affected, but with the latter, the pixel decides which geometry affects its colour. Due to the limitations of a rasterizer, there are currently two ways to create 360° panorama stills or video sequences with the Unreal Engine:

  1. Render six sides of a cube into a cubemap by rotating a 90° field of view camera in a central location of the scene. The cubemap image is then converted to an equirectangular spherical 360° panorama with some well-defined projection methods essentially, projecting or shrink wrapping the cube onto a sphere. This method is fast but can result in stretching artefacts at the cube corners. An interesting discussion on this method can be found on the Unreal Engine answerhub.
  2. Render many sub panels of the full 360° by rotating a camera with a fixed field of view in a central location. You then stitch each panel together to create the 360° panorama. This method is more accurate but can be very time consuming depending on the required final resolution of the panorama. A detailed description of the process using render “slices” can be found on the Unreal Engine blog.

 

Creating panoramas with a ray tracer

With Imagination’s implementation of ray tracing in the Unreal Engine the panoramas are created in one single render. Using a lens shader, which applies a spherical projection establishing the camera direction for each pixel, we can create a complete 360° flat panorama image. Of course, not only do you get a 360° image or video in one easy step, you also get physically correct reflections, refractions and shadows. This could also be applied in creating stereoscopic panoramas by appropriately shifting the camera direction for each eye.

In early March at GDC 2017 in San Francisco, Imagination demonstrated the implementation of ray tracing in the Unreal Engine running in real-time on our Wizard GR6500 development hardware platform. The demo, running with the Vulkan graphic back-end, enables you to toggle between several modes, including raster, ray traced and panorama. You can read a full, in-depth discussion of the making of the demo and the challenges involved in this blog post. If you’d like to find out more please feel free to get in touch.

The Chairs and Table project is a well-recognised tutorial for learning Unreal. Below are two movies that illustrate the creation of 360° panoramas with our ray tracing technology.

This movie is the flat panorama before it is wrapped on a sphere:


This movie is the panorama with the player in 360° mode
:

Please leave a comment below

Comment policy: We love comments and appreciate the time that readers spend to share ideas and give feedback. However, all comments are manually moderated and those deemed to be spam or solely promotional will be deleted. We respect your privacy and will not publish your personal details.

Search by Tag

Search for posts by tag.

Search by Author

Search for posts by one of our authors.

Featured posts
Popular posts

Blog Contact

If you have any enquiries regarding any of our blog posts, please contact:

United Kingdom

benny.har-even@imgtec.com
Tel: +44 (0)1923 260 511

Related blog articles

Product and event round-up from the experts in GPU and AI

It’s certainly been a busy few months for Imagination. Towards the latter end of last year, we released a raft of new products and initiatives, and a new CEO took the helm giving us real momentum for 2019. At the

How AI is conducting the future of music technology

“We tend to think of technological advances as destroying what’s gone before, but that doesn’t usually happen. This could lead to a different way of making music.” – Jarvis Cocker, former Pulp frontman, solo artist, writer and broadcaster In recent

Why you should join Imagination at Embedded World 2019

Our technology is focussed entirely on offering SoC manufacturers low power, high-performance options for building groundbreaking products in a range of markets, from automotive to smart devices such as smart speakers to the latest smartphones. Embedded World is one of

Stay up-to-date with Imagination

Sign up to receive the latest news and product updates from Imagination straight to your inbox.

  • This field is for validation purposes and should be left unchanged.