COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

WebRTC Media in the Cloud

Chad Hart

IIT RTC Conference 2014

Why do you need a media server?

Multi-party conferencing Transcoding Interworking Recording Stream processing Person-to-machine COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

2

What do you care more about?

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

3

Bandwidth is not always ubiquitous or free

Client-side Server-side

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

4

CPU & processing power is expensive

Client view Provider View

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

5

Multi-PARTY! Video Conferencing

COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

6 Image source: http://thewrestlinglifeupdate.tumblr.com/

Easy & cheap approach to multi-party: Mesh

Full Mesh

Works for a few parties No server cost Lowest latency COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

7 7

Mesh does not scale for many video conferees

Full Mesh

Clients get overloaded Encode costs more than decode Limited uplink bandwidth Inconsistent performance across participants COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

8 8

Traditional Telephony Approach: MCU

Multipoint Control Unit (MCU)

Good at manipulating all media Transcoding Transizing Transrating Interworking Server-side CPU intensive Client can request different conference mixes Usually client friendly Downsizing & mixing can reduce bandwidth Fewer client streams to process COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

9

MCU

9

Video UX: Traditional method

MCU

1 3 2 4

Mixed stream 10 COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

3G Device

1 3 2 4

Video UX: Traditional method

MCU

1 3 2 4

Active Talker Media Server Chooses Layout Mixed stream 11 COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

3G Device

1 3 2

Modern UX with MCU mixing, HTML5 & JavaScript

MCU

Layout

App Server 1 3 2 4

Mixed stream

Client app

HTML5 Canvas Javascript COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

12

2

Browser "Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum." Application-defined UX Benefits: Application defined UX User specific & customizable layout Lower client BW & CPU vs. mesh

Newer approach: SFU

Selective Forwarding Unit (SFU) routing

Clients send one & receive many Client can instruct SFU which streams to send High throughput Can be lots of downlink bandwidth Low latency COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

13

SFU

13

New WebRTC approach: Simulcast

Selective Forwarding Unit (SFU) with Simulcast

Clients send multiple streams to SFU one high-bit rate one or more lower-bit Client directs SFU which streams to receive Reduces bandwidth vs. SFU Simulcast in WebRTC coming COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

14

SFU

14 High bitrate Low bitrate

Future Approach with VP9 – SVC?

Selective Forwarding Unit (SFU) with SVC

Clients send layered stream to SFU Varying bitrates – time, size, quality SFU directs who gets what Coming to WebRTC eventually Layered bitrates: High Medium Low COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

15

SFU

15

Typical media server network model App Server App Server App Server Load Balancer Load Balancer Media Server Media Server Media Server Client Client COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

Client Client 16 Client Client

Pseudo Call-flow with a media server AS Call signaling Media control LB MS1 status MS1 MS2 status MS2 Media control Media 17 COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

Client

Pseudo Call-flow – traditional fail-over with RE-INVITE AS Media control LB MS1 status MS1 MS2 status MS2 Media control + call states Call signaling – REINVITE with new addresses Media Client 18 COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

Pseudo Call-flow – fail-over the cloud way AS Media control LB MS1 status MS1 MS2 status MS2 New container Memcache & Virtual IP Address Media Client 19 COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

When you are serious about scale 20 Declarative Concurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

ICE, STUN, and TURN save the day Use of Public IP’s mitigates need for TURN servers Open source TURN servers do reasonably well Running the TURN server co resident with the media server works well when you need it Image source: webrtcHacks natfirewall-problem/ http://webrtchacks.com/an-intro-to-webrtcs COMPANY CONFIDENTIAL © COPYRIGHT 2014 DIALOGIC INC. ALL RIGHTS RESERVED.

21