Beyond video to interactive, personalised content: BBC is experimenting with rebuilding its iPlayer in WebAssembly
(2021/05/20)
- Reference: 1621502113
- News link: https://www.theregister.co.uk/2021/05/20/bbc_iplayer_webassembly/
- Source link:
QCON The BBC is researching a rebuild of its iPlayer catch-up service client in WebAssembly.
Speaking at the [1]QCon Plus developer conference under way online this week, BBC R&D software engineer Tim Pearce said: "We've used WebAssembly to... build an experimental version of iPlayer which can playback future experiences, and I'll also introduce our plans to use WebAssembly outside the browser throughout our technology stack for delivering universal access to audiences."
Pearce said that "universal access" is a part of the BBC's duty as a public service broadcaster and means ensuring that "every audience member can access every experience, regardless of what device they have at home... they might have an old smartphone or smart TV."
[2]
[3]
[4]
Despite these extensive plans, Pearce said that it is "very early stage research in applying WebAssembly to assault the challenges that we face. However, we believe WebAssembly will be an integral part of the BBC broadcast chain in the future."
Part of the challenge is that the BBC intends to move beyond mere video to a thing it calls [5]object-based media , which "allows the content of programmes to change according to the requirements of each individual audience member." It is called object-based because it attaches semantic tags to elements within the content. Features might include replacing a speaking presenter with a signing one, automatic editing to make programmes shorter or longer, personalising content such as focusing on a favourite football team, as well as some features that may disturb privacy advocates such as weather forecasting that "knows where I'm planning to go in the next week."
"We have created a variety of content experiences which demonstrate the advantages of object-based media, including augmented and virtual reality, branching narrative stories and multi-device immersive audio and video. Many of these future experiences can be viewed on the BBC Taster website, but not yet via iPlayer," said Pearce.
[6]
Currently, Pearce said, iPlayer has "multiple codebases for different platforms, iOS, Android, responsive web, smart TVs, and many more legacy codebases... the client application is written in completely different platform-specific technologies and programming languages."
[7]
A diagram showing how content compiled to Wasm could be hosted by a universal iPlayer (pic: BBC)
A "single service player" (SSP) would be capable of both traditional video and more game-like experiences. "We've implemented an object based media storage engine which can playback... interactive experiences previously only available as a web application," said Pearce. "And we've started experimenting with remote streaming, running high-end graphical experiences in the cloud, sending user inputs across the network, and streaming the resulting scene as video." This approach would potentially avoid the need for users to have high-end GPUs.
WebAssembly allowed the BBC to take its SSP written in C and compile for the web, via the [8]Emscripten toolchain, which lets developers add WebAssembly as a target using the LLVM compiler project. The SSP renders on the web in "multiple ways," Pearce said, using browser capabilities such as media source extensions, the JavaScript Fetch API, WebSockets, WebGL, and Canvas2D. The project also uses SDL (Simple DirectMedia Layer) for access to multimedia APIs, which is supported by Emscripten.
"The Emscripten toolchain generates JavaScript glue code to bootstrap our WebAssembly module and give us access to these web API's," he explained. "One of the advantages in using WebAssembly is that the SSP looks identical across all browsers and is identical to the native application, because all of the user interface components, layout and business logic is in the WebAssembly module." This avoids differences in things like CSS implementations between browsers.
The BBC researchers then decided to investigate using WebAssembly outside the browser. "We thought, could we package our launcher, application and individual content experiences as WebAssembly modules that could not just work on the web but in native applications?" said Pearce. The solution they found is to "embed a WebAssembly runtime in a new C++ application." Specifically, he said, "we are using the Wasmtime runtime from the Bytecode Alliance." This is written in Rust and more details are [9]here .
[10]
Why WebAssembly rather than one of the myriad other ways to do cross-platform code? One aspect of this is that the BBC has chosen to do its own UI, bypassing the need for a cross-platform GUI framework. "We want to be able to target WebAssembly as a universal binary. The key benefit is the sandboxing that it provides," said Pearce. He sees this as a route to a standard for content which can be hosted in iPlayer. "There are standards for broadcasts like the framerate and what codec it's in, and things like that, and we're seeing WebAssembly as a way of achieving that for interactive content," he said. "The initial design of WebAssembly does mean that it lends itself very nicely to being that kind of plug-in model."
Pearce noted that the BBC already has a [11]Global Experience Language (GEL) which is intended to provide a design standard for content authors. He also referenced a [12]post from earlier this year which gives further details on iPlayer in WebAssembly. ®
Get our [13]Tech Resources
[1] https://plus.qconferences.com/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YKYzOu4O1miRcRxhYjAAtAAAAM4&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YKYzOu4O1miRcRxhYjAAtAAAAM4&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YKYzOu4O1miRcRxhYjAAtAAAAM4&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.bbc.co.uk/rd/object-based-media
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YKYzOu4O1miRcRxhYjAAtAAAAM4&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://regmedia.co.uk/2021/05/19/wasm-bbc.png
[8] https://emscripten.org/
[9] https://wasmtime.dev/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YKYzOu4O1miRcRxhYjAAtAAAAM4&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://www.bbc.co.uk/gel
[12] https://medium.com/bbc-design-engineering/building-a-webassembly-runtime-for-bbc-iplayer-and-enhanced-audience-experiences-7087455808ef
[13] https://whitepapers.theregister.com/
Speaking at the [1]QCon Plus developer conference under way online this week, BBC R&D software engineer Tim Pearce said: "We've used WebAssembly to... build an experimental version of iPlayer which can playback future experiences, and I'll also introduce our plans to use WebAssembly outside the browser throughout our technology stack for delivering universal access to audiences."
Pearce said that "universal access" is a part of the BBC's duty as a public service broadcaster and means ensuring that "every audience member can access every experience, regardless of what device they have at home... they might have an old smartphone or smart TV."
[2]
[3]
[4]
Despite these extensive plans, Pearce said that it is "very early stage research in applying WebAssembly to assault the challenges that we face. However, we believe WebAssembly will be an integral part of the BBC broadcast chain in the future."
Part of the challenge is that the BBC intends to move beyond mere video to a thing it calls [5]object-based media , which "allows the content of programmes to change according to the requirements of each individual audience member." It is called object-based because it attaches semantic tags to elements within the content. Features might include replacing a speaking presenter with a signing one, automatic editing to make programmes shorter or longer, personalising content such as focusing on a favourite football team, as well as some features that may disturb privacy advocates such as weather forecasting that "knows where I'm planning to go in the next week."
"We have created a variety of content experiences which demonstrate the advantages of object-based media, including augmented and virtual reality, branching narrative stories and multi-device immersive audio and video. Many of these future experiences can be viewed on the BBC Taster website, but not yet via iPlayer," said Pearce.
[6]
Currently, Pearce said, iPlayer has "multiple codebases for different platforms, iOS, Android, responsive web, smart TVs, and many more legacy codebases... the client application is written in completely different platform-specific technologies and programming languages."
[7]
A diagram showing how content compiled to Wasm could be hosted by a universal iPlayer (pic: BBC)
A "single service player" (SSP) would be capable of both traditional video and more game-like experiences. "We've implemented an object based media storage engine which can playback... interactive experiences previously only available as a web application," said Pearce. "And we've started experimenting with remote streaming, running high-end graphical experiences in the cloud, sending user inputs across the network, and streaming the resulting scene as video." This approach would potentially avoid the need for users to have high-end GPUs.
WebAssembly allowed the BBC to take its SSP written in C and compile for the web, via the [8]Emscripten toolchain, which lets developers add WebAssembly as a target using the LLVM compiler project. The SSP renders on the web in "multiple ways," Pearce said, using browser capabilities such as media source extensions, the JavaScript Fetch API, WebSockets, WebGL, and Canvas2D. The project also uses SDL (Simple DirectMedia Layer) for access to multimedia APIs, which is supported by Emscripten.
"The Emscripten toolchain generates JavaScript glue code to bootstrap our WebAssembly module and give us access to these web API's," he explained. "One of the advantages in using WebAssembly is that the SSP looks identical across all browsers and is identical to the native application, because all of the user interface components, layout and business logic is in the WebAssembly module." This avoids differences in things like CSS implementations between browsers.
The BBC researchers then decided to investigate using WebAssembly outside the browser. "We thought, could we package our launcher, application and individual content experiences as WebAssembly modules that could not just work on the web but in native applications?" said Pearce. The solution they found is to "embed a WebAssembly runtime in a new C++ application." Specifically, he said, "we are using the Wasmtime runtime from the Bytecode Alliance." This is written in Rust and more details are [9]here .
[10]
Why WebAssembly rather than one of the myriad other ways to do cross-platform code? One aspect of this is that the BBC has chosen to do its own UI, bypassing the need for a cross-platform GUI framework. "We want to be able to target WebAssembly as a universal binary. The key benefit is the sandboxing that it provides," said Pearce. He sees this as a route to a standard for content which can be hosted in iPlayer. "There are standards for broadcasts like the framerate and what codec it's in, and things like that, and we're seeing WebAssembly as a way of achieving that for interactive content," he said. "The initial design of WebAssembly does mean that it lends itself very nicely to being that kind of plug-in model."
Pearce noted that the BBC already has a [11]Global Experience Language (GEL) which is intended to provide a design standard for content authors. He also referenced a [12]post from earlier this year which gives further details on iPlayer in WebAssembly. ®
Get our [13]Tech Resources
[1] https://plus.qconferences.com/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YKYzOu4O1miRcRxhYjAAtAAAAM4&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YKYzOu4O1miRcRxhYjAAtAAAAM4&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YKYzOu4O1miRcRxhYjAAtAAAAM4&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.bbc.co.uk/rd/object-based-media
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YKYzOu4O1miRcRxhYjAAtAAAAM4&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://regmedia.co.uk/2021/05/19/wasm-bbc.png
[8] https://emscripten.org/
[9] https://wasmtime.dev/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YKYzOu4O1miRcRxhYjAAtAAAAM4&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://www.bbc.co.uk/gel
[12] https://medium.com/bbc-design-engineering/building-a-webassembly-runtime-for-bbc-iplayer-and-enhanced-audience-experiences-7087455808ef
[13] https://whitepapers.theregister.com/
Fuck the BBC - fuck iPlayer
Anonymous Coward
Recently noticed they had ruined the iPlayer interface on my LG TV
No way to see when a programme was first shown from the app. The reply top the logged complaint I made to the BBC was "can't you use a web browser instead ?"
So fuck you and your "improvements". That's probably what it took for me to decide not to pay the licence fee next year. That and the fact they have become the broadcasting arm of the Tory party.
Re: Fuck the BBC - fuck iPlayer
Swordfish1
I legally stopped paying the Licence Fee last year- not missed Live TV, or Iplayer whatsoever. Watch Netflix and Amazon Prime instead, and catchup on the commercial channels, which you can legally watch without a TV Licence
Object-based media
Pascal Monett
As the old curmudgeon that I am, I initially reacted to this by thinking "that's another load of intellectual masturbation about how to stream a video".
But, as I read further into the article, I started to get an inkling of a notion that that could actually be an interesting development.
I don't yet know how, and I don't expect this pie-in-the-sky, perfect description to actually come into being, but I am now interested in finding out what they're going to deliver.
Of course, I'll have to use a VPN to check it out. The BBC doesn't like being viewed from outside the UK.
WOT??
" playback future experiences "
Wioll haven be? (thanks Douglas Adams).
Someone should teach Auntie grammar (let alone semantics) I know they abandoned their language support services decades back but this is taking things all the way into Humpty Dumpty land and meaning will shortly cease to exist.