New Insta360 8K Live Streaming Software Used for 5G VR Live Stream of NFL Game

by Jakub HanApril 10^th, 2020

A team from Verizon, Radiant Images and Flight School Studio pulled off the world’s first 8K 360-degree live broadcast of an NFL game,  at this year’s NFL Pro Bowl. The feed was integrated into ESPN’s live broadcast and projected into a dome 200 miles away in Miami. The team used Verizon’s 5G network and two Insta360 Pro 2 cameras along with the newly launched Insta360 8K Live software. Starting today, the Insta360 8K Live Software is available to broadcasters.

8K VR Live Streaming with Insta360 Pro 2. Source: Insta360

Insta360 is undoubtedly one of the largest innovators in the world of VR 360 content capture and distribution. In the past, the company introduced innovative 360 cameras like the compact 5.7K Insta360 ONE X, the modular Insta360 ONE R, or the 11K resolution Insta360 Titan. This time, their Insta360 Pro 2 (read our article here) along with their new 8K Live Software made the first 8K VR live stream possible through 5G network. How was this achieved?

First 5G 8K VR Live Stream of an NFL Game

A team from Verizon, Radiant Images and Flight School Studio pulled off the world’s first 8K 360-degree live broadcast of an NFL game. The feed was integrated into ESPN’s live broadcast and projected into a dome 200 miles away in Miami. Michael Mansouri, the co-founder of Radiant Images, explained their workflow.

1.  VR live streaming the game with a multi-cam setup

The team used two Insta360 Pro 2 cameras to film the game – one as a goal post cam and one mounted on the Skycam. The low weight of the Insta360 Pro 2 camera – 1.5 kg (3.3 lbs) – made it possible to hang it on the Skycam along with the ESPN’s traditional TV camera. For the entirety of the game, the Insta360 Pro 2 captured 360-degree 8K footage.

2.  Stitching 360 live streaming camera feeds with Insta360 8K Live

With the new Insta360 8K Live software, six video streams from the camera’s six lenses can be stitched in real-time in 8K quality. A dynamic stitching algorithm analyzes content in real-time to optimize stitch quality and placement, ensuring the feeds are merged seamlessly. The software outputs a 2D HDR VR live stream in 30 fps, using RTSP/RTMP/RTMPS protocols and H.265 HEVC codec. It also enables simultaneous recording during a live stream to save footage at full quality for future playback or post-processing.

View from the Insta360 Pro 2 in the endzone. Source: Insta360

During the Pro Bowl broadcast, the team connected the Skycam-mounted Insta360 Pro 2 with a Gigabit Ethernet cable, then converted it to fiber to connect it to their remote server PC about 1,000 feet away. The server PC’s were equipped with two RTX 2080 Ti graphics cards to provide enough processing power. With the help of two Blackmagic 8K Pro DeckLink cards and software, the signal was converted into a standard SDI output for broadcasting.

3.  Choosing camera feeds to broadcast with the switcher control panel

Using a Blackmagic ATEM Constellation 8K Switcher, the team then selected the optimum footage to broadcast in real-time from the stitched SDI video feeds from each of the two Insta360 Pro 2 cameras. In this case, the team only had two Insta360 Pro 2 feeds to monitor, but it is, of course, possible to integrate more feeds at this stage, including both 360 and standard video.

Insta360 Pro 2 on the Skycam. Source: Insta360

At this stage, it is also possible to choose whether to reframe the 360 footage or broadcast the original VR footage. ESPN decided to broadcast an equirectangular projection from the Pro 2, giving a feel for the unlimited field of view to the fans in Miami at the Dome.

4.  Transmission to the dome with Verizon 5G

Once the team had a stitched 8K video feed, the next step was to convert the footage for transmission to the dome. A software or hardware encoder can convert an SDI feed into codecs like the H.264 or H.265 (HEVC) VR live streaming protocols, but the encoders weren’t spec’d for transmitting an 8K feed.

8K VR Live Streaming with Insta360 Pro 2. Source: Insta360

They used TouchDesigner (a live visual manipulation software) to split the 8K x 4K sphere into easily transmittable quadrants. In Miami, they received the full quality image piece-by-piece and then reconstructed the desired hemisphere for projection into the dome. To get the desired transmission speed, the team used the Verizon 5G router. With 4G, the bandwidth is limited, so with such a large amount of data, the latency would be quite high.

5. Projecting the VR live stream into the dome

Footage needs to be decoded before projection. In this case, the on-site team at the dome used a hardware decoder to convert the feed back into SDI format. They then used TouchDesigner to reconstruct a 4K x 4K hemisphere (the half of the original 8K x 4K sphere that could be mapped to the dome’s interior), which was then repositioned for projection.

8K Live Diagram during NFL Game. Source: Insta360

The Miami team confirmed the projection mapping and then projected the feed onto the dome using six projectors. The number of projectors needed varies based on the size of the dome. In the end, there was only a 3-second delay from the game to the dome, thanks to the 5G transmission.

Insta360 Live Stitching Software

Starting today, the Insta360 8K Live Software is available to broadcasters. There are many possibilities for 5G VR live streaming technology and 360 live streaming cameras. In addition to VR domes, the 360 footage can also be live-streamed to VR headsets, mobile phones, desktops and tablets.

Insta360 8K Live Stitching Software. Source: Insta360

According to Insta360, their 8K Live software is also capable to stitch together the eight individual video feeds from the 11K Insta360 Titan.

After downloading, simply install Insta360 8K Live software and connect Insta360 Pro 2 or Titan to a server PC with a router or Ethernet cable. Footage will be stitched in real-time at 8K resolution, ready to be broadcast via a cloud-connected streaming server. When live streaming in 8K, a full-quality, stitched video file can be simultaneously saved locally on the server PC for later processing and playback.

Insta360 8K Live Software – vital stats:

• 8K (7680×3840) 2D @30fps
• Protocols: RTSP, RTMP, RTMPS
• Default bitrate at 80Mbps, Max Bitrate at 120Mbps
• Output: SDI, HDMI
• Dynamic Stitching – a proprietary algorithm that uses frame-by-frame analysis to ensure a seamless stitch even during dynamic action.
• Streamlines stitching and streaming tasks with a single software.

Please mind, that the 8K live stitching requires a strong computer. Insta360 published a recommended set-up for this:

• CPU: i7 8700K 8th generation or higher
• Memory: 16GB or more
• Graphics Card: Dual GTX 2080Ti
• OS: Windows 10 or above

When I look at the current trends, which are influenced by the COVID-19 pandemic, I see a bright future for live streaming of any kind. Even after the pandemic is gone, I think live streaming will become a popular alternative to going to an event personally. 360 VR content live streaming can therefore get very popular in the future in my opinion. What do you think?

What do you think about the Insta360 8K Live Software? Have you ever tried to live-stream VR footage? Do you see a bright future for streaming 8K VR footage? Let us know in the comments underneath the article.