Editing live TV in VR – behind the scenes in our latest DIY project

---

Join us behind the scenes of a truly unique TV production, as NRK, the Norwegian Broadcasting Corporation, tests out replacing an extensive control room and dozens of screens with just four VR headsets.

(En norsk versjon av denne artikkelen finnes her)

– Are you guys on the moon, or what?

This is the immediate response of Trude Drevland, member of «Kringkastingsrådet», the Advisory Board for the Norwegian national broadcaster NRK, and former mayor of Bergen. Now, as is her manner, she bursts into the control room for the live broadcast of the Kringkastingsrådet meeting.

We’re in a small back room at the NRK. Four men are seated in a row, getting ready to produce the live TV broadcast of the meeting.

But the room isn’t bursting with screens, panels, and buttons, as you would expect when making TV. In this room, the four people are just sitting there, each with a VR headset covering their eyes.

– There are really no limits, Trude Drevland exclaims, awestruck.

Had this been a traditional TV production, the entire wall would have been covered with screens. But a control room like that is expensive, large, and difficult to move. Here, they’re sitting in a fairly naked room facing a blank wall.

It does indeed look completely ridiculous. But could this be the future of TV production?
We were there when NRK produced its first live TV broadcast using VR headsets.

First: This is how live TV is made

A TV program like «Kringkastingsrådet» requires several cameras to offer the different shots – close-ups and wide shots from different angles – that are edited together on the fly by a Producer, before reaching the viewers.

In addition, one person manages the cameras, another is overlaying graphics and text, while yet another one is working the sound mixer to make sure you can hear everyone clearly. All this is happening at the same time, and more or less simultaneously with the final signal being on its way to thousands of homes.

This model does require a lot of space and equipment, though. Here’s what it usually looks like when Technical Operations Manager (TOM) Tarald Moe Bjølseth produces TV:

So, do really all these screens and panels need to exist in the real world?

Editing TV: 👍

– Shall we try with a thumbs up? Okay? Ready, five?

Tarald Moe Bjølseth is about to attempt one of the hardest things today. Is it possible to switch from one camera to another in a live TV broadcast by just showing a thumbs up sign in the air?

– Yes! It worked! Hahaha!

The control room behind the Kringkastingsrådet meeting bursts into laughter.

The broadcast is well underway, and Bjølseth has just successfully tested the thumbs-up gesture. His thumb in the air successfully switched cameras.

– We’re not quite ready for music production yet, he admits.

They suspect that a PC downstairs, encoding the video signals, is a bit underpowered, causing some frames to freeze or stutter. These issues are invisible to viewers, but are causing minor glitches in the control room.

– Don’t forget to charge your headsets, guys!

Here’s what it looked like inside Tarald Moe Bjølseth’s VR headset as he cut the broadcast:

Dreaming of a finger snap

Tarald Moe Bjølseth is usually working as a Technical Operations Manager, but today he’s the Producer, responsible for making sure the Advisory Board meeting is broadcast clearly and smoothly from the stately “Balkongen” meeting room at NRK’s Radio House.

– What’s the point of this, really?

– We wanted to play around with the way we’re working, and the different requirements we have, says Bjølseth.

– We spend a lot of resources on technical installations. When a new piece of technology has the potential to challenge how we work, it´s important that we try it out, he says.

TV is often produced at locations where there isn’t space for a large control room. Plus, there are lots of compromises across the different roles. Sound Engineers and Graphics Operators often have to make space for Producers and Vision Mixers, or vice versa.

– In a control room, we’re sharing a lot of resources. Here, everyone can have exactly the screen sizes they want, he says.

– And you’re not tied to physical workstations.

– How much of this is serious, and how much is just for fun?

– Right now, we’re in the exploration and testing phase. We have a lot of work environments where this can give us new possibilities. We need to play with it today to figure out how to use it tomorrow, says Bjølseth.

NRKbeta’s own inventor

Jon Ståle Carlsen from NRKbeta, NRKs sandbox for technology and media, is the NRK’s only employee bearing the title “inventor”. He is constantly tinkering, building and assembling quirky TV technology for NRK, and over the years, he has been involved in several of NRK’s Slow TV broadcasts. These broadcasts have often required significant technological innovations.

Carlsen explains that NRKbeta bought an Apple Vision Pro headset from the US in March, as they aren’t available on the Norwegian market yet.

– I’ve been tinkering with it and pondering what it could be used for, he says.

One possibility was as a TV control room.

– I created an interface – a website – that functions as the ‘multiviewer’, the control surface for the video mixer. Then I built a control layer on top, with each camera as a button. That was fun, he says.

NRK Produksjon, the production department responsible for the technical production of TV shows, had purchased the same type of VR headset as well, and wanted to try something similar, but inside an app. Carlsen had hadn’t built an app, but rather a website that could be viewed in a browser inside the headset.

Testing more functions

The Producer explains that they’re experimenting with various ways to control the production within the headsets:

• Voice control: So far, it performs poorly in Norwegian, but decently in English.
• Eye tracking and pointing: These features are built into the VR headsets. So, you can just look at the camera you want to select or point at it.
• Snapping: A snap of the fingers is the traditional command in the control room, signalling when to cut to the prepared shot. But this has been tricky to achieve.

– Snapping your fingers is great fun. The challenge is that everyone is snapping differently. And your thumb ends up on top of the index finger, Bjølseth explains.

When the thumb touches the index finger, the headset can mistake this gesture for a pinch instead of a snap.

In a Vision Pro headset, pinching is like a mouse click. So, creating a snap function that is closely resembling pinching isn’t the best idea. It hasn’t worked well so far.

Here producer Tarald shows how the app works (only in norwegian):

Magnus Tviberg from the app development company Shortcut developed the app and confirms that snapping is challenging.

– We wanted you to just snap, and it would cut. The headset tracks hands incredibly well and knows exactly where every joint is at any given time. But we’ve been experimenting with different gestures, he says.

Debriefing

Hours go by in the “Balkongen” meeting room without any major issues. The Kringkastingsrådet covers discussions about NRK’s coverage of Ukraine, cultural policy, and new TV concepts. Everything is produced as usual.

But it might be the first time in history that someone has cut a professional live TV broadcast using VR headsets. No one can say for sure, but no one here has seen anyone else doing it either.

– It went well, says producer Bjølseth.

Sound engineer Torbjørn Råen admits there were some issues with audio and video delays, but the controls worked perfectly.

– Did you chicken out on trying the harder stuff? Voice control and snapping and such? You didn’t do much of that?

– We didn’t try voice control much. The combination of a noisy room and language switching between Norwegian and English made it challenging. But eye tracking and thumbs up worked well, says Bjølseth.

The group concludes that gestures – movements and hand signals – is the most fun. But physical panels and buttons still provide the quickest and most precise cuts.

– And my neck and back are starting to get a bit sore, Bjølseth says after three or four hours of wearing the headset.

– Where would you like to use this setup next?

– It’s not quite ready for the road yet. But maybe a smaller sports production? A football match?

Because we can

– What’s the actual point of all this, inventor Carlsen?

– Part of it is because we can. But also to avoid setting up walls of monitors. You free yourself from the physical video mixer and audio mixer, and consequently from the room. In the future, you could do this from home.

– Will the Evening News be produced like this soon?

– No, maybe not the Evening News. But, for example, during the World Rapid and Blitz Chess Championships, we have people in both New York and Oslo. You could imagine each wearing a headset and making the show together.

– This won’t be used for the cup finals or big music shows. But for smaller productions and things happening outside the studio, it could develop into something. But I don’t think we’re getting rid of control rooms anytime soon.

Here’s what the final broadcast of the Kringkastingsrådet looked like.