WebRTC is a new front in the long war for an open and unencumbered web. Imagine it was easy to add video chat and peer-to-peer data sharing to your web application. That's the vision of WebRTC. Want to try it out? WebRTC is available now in Google Chrome, Safari, Firefox and Opera, on desktop and mobile. Quick start Haven't got time to read this article, or just want code? Try out some of the that exercise the WebRTC JavaScript APIs. Try out our troubleshooting page. Alternatively, jump straight into our : a step-by-step guide that explains how to build a complete video chat app, including a simple signaling server. A very short history of WebRTC One of the last major challenges for the web is to enable human communication via voice and video: Real Time Communication, RTC for short. RTC should be as natural in a web application as entering text in a text input. Without it, we're limited in our ability to innovate and develop new ways for people to interact. Historically, RTC has been corporate and complex, requiring expensive audio and video technologies to be licensed or developed in house. Integrating RTC technology with existing content, data and services has been difficult and time consuming, particularly on the web. Gmail video chat became popular in 2008, and in 2011 Google introduced Hangouts, which use the Google Talk service as did Gmail. Google bought GIPS, a company which had developed many components required for RTC, such as codecs and echo cancellation techniques. Google open sourced the technologies developed by GIPS and engaged with relevant standards bodies at the IETF and W3C to ensure industry consensus. In May 2011, Ericsson built. WebRTC implemented open standards for real-time, plugin-free video, audio and data communication. Those included Skype, Facebook and Google Hangouts. It's often difficult to persuade people to install plugins in the first place! The guiding principles of the WebRTC project are that its APIs should be open source, free, standardized, built into web browsers and more efficient than existing technologies. Where are we now? WebRTC is used in various apps like WhatsApp, Facebook Messenger, appear. WebRTC has also been integrated with and native apps. RTCPeerConnection: There's an ultra-simple demo at and a fully functional video chat application at. This app uses , a JavaScript shim, maintained Google with help from the , to abstracts away browser differences and spec changes. RTCDataChannel: Check out one of the data channel demos at to see this in action. Our shows how to use all three APIs to build a simple application for video chat and file sharing. There is detailed discussion of the network and signaling aspects of WebRTC. MediaStream aka getUserMedia The represents synchronized streams of media. For example, a stream taken from camera and microphone input has synchronized video and audio tracks. Don't confuse MediaStreamTrack with the element, which is something. Each MediaStream has an input, which might be a MediaStream generated by getUserMedia , and an output, which might be passed to a video element or an RTCPeerConnection. The getUserMedia method takes a parameter, and returns a Promise that resolves to a MediaStream object. Each MediaStream has a label, such as'Xk7EuLhsuHKbnjLWkW4yYGNJJ8ONsgwHBvLQ'. An array of MediaStreamTracks is returned by the getAudioTracks and getVideoTracks methods. For the example, stream. Each MediaStreamTrack has a kind 'video' or 'audio' , and a label something like 'FaceTime HD Camera Built-in ' , and represents one or more channels of either audio or video. In this case, there is only one video track and no audio, but it is easy to imagine use cases where there are more: for example, a chat application that gets streams from the front camera, rear camera, microphone, and a 'screenshared' application. A MediaStream can be attached to a video element by setting the. Previously this was done by setting the src attribute to an object URL created with URL. The MediaStreamTrack is actively using the camera, which takes resources and keeps the camera open and camera light on. When you are no longer using a track make sure to call track. Web Audio not supported. Thereafter the user is not asked for permission for camera or microphone access. Permission only has to be granted once for getUserMedia. First time around, an Allow button is displayed in the browser's. HTTP access for getUserMedia was deprecated by Chrome at the end of 2015 due to it being classified as a. The intention is potentially to enable a MediaStream for any streaming data source, not just a camera or microphone. This would enable streaming from disc, or from arbitrary data sources such as sensors or other inputs. Constraints can be used to set values for video resolution for getUserMedia. This also allows such as aspect ratio, facing mode front or back camera , frame rate, height and width, along with an. There's an example at. One gotcha: getUserMedia constraints may affect the available configurations of a shared resource. For example if a camera was opened in 640x480 mode by one tab, another tab will not be able to use constraints to open it in a higher-resolution mode since it can only be opened in one mode. Note that this is an implementation detail: it would be possible to let the second tab reopen the camera in a higher resolution mode and use video processing to downscale the video track to 640x480 for the first tab, but this has not been implemented. Setting a disallowed constraint value gives a DOMException, or an OverconstrainedError if for example a resolution requested is not available. To see this in action, try the demo at. Screen and tab capture Chrome apps also make it possible to share a live 'video' of a single browser tab or the entire desktop via and APIs. There's a demo and more information in the HTML5 Rocks Update article. A few years old, but still interesting. It's also possible to use screen capture as a MediaStream source in Chrome using the experimental chromeMediaSource constraint, as in. Note that screen capture requires HTTPS and should only be used for development due to it being enabled via a command line flag as explaind in this discuss-webrtc. Signaling: session control, network and media information WebRTC uses RTCPeerConnection to communicate streaming data between browsers aka peers , but also needs a mechanism to coordinate communication and to send control messages, a process known as signaling. Signaling methods and protocols are not specified by WebRTC: signaling is not part of the RTCPeerConnection API. Instead, WebRTC app developers can choose whatever messaging protocol they prefer, such as SIP or XMPP, and any appropriate duplex two-way communication channel. The example uses XHR and the Channel API as the signaling mechanism. The we built uses running on a. The exchange of information via signaling must have completed successfully before peer-to-peer streaming can begin. For example, imagine Alice wants to communicate with Bob. Here's a code sample from the , which shows the signaling process in action. The code assumes the existence of some signaling mechanism, created in the createSignalingChannel method. Also note that on Chrome and Opera, RTCPeerConnection is currently prefixed. The expression 'finding candidates' refers to the process of finding network interfaces and ports using the. WebRTC clients known as peers, aka Alice and Bob also need to ascertain and exchange local and remote audio and video media information, such as resolution and codec capabilities. The return from this of this is passed an RTCSessionDescription: Alice's local session description. Note that RTCPeerConnection won't start gathering candidates until setLocalDescription is called: this is codified in. The createAnswer callback is passed an RTCSessionDescription: Bob sets that as the local description and sends it to Alice. Make sure to allow the RTCPeerConnection to be garbage collected by calling close when it's no longer needed. Otherwise threads and connections are kept alive. It's possible to leak heavy resources in WebRTC! RTCSessionDescription objects are blobs that conform to the , SDP. There's an excellent animation explaining the process of signaling and streaming in for its first WebRTC implementation. JSEP architecture Once the signaling process has completed successfully, data can be streamed directly peer to peer, between the caller and callee—or if that fails, via an intermediary relay server more about that below. Streaming is the job of RTCPeerConnection. RTCPeerConnection RTCPeerConnection is the WebRTC component that handles stable and efficient communication of streaming data between peers. Below is a WebRTC architecture diagram showing the role of RTCPeerConnection. As you will notice, the green parts are complex! WebRTC architecture from From a JavaScript perspective, the main thing to understand from this diagram is that RTCPeerConnection shields web developers from the myriad complexities that lurk beneath. The shows a simplified example of WebRTC from a signaling perspective. Below are walkthroughs of two working WebRTC applications: the first is a simple example to demonstrate RTCPeerConnection; the second is a fully operational video chat client. RTCPeerConnection without servers The code below is taken from the 'single page' WebRTC demo at , which has local and remote RTCPeerConnection and local and remote video on one web page. This doesn't constitute anything very useful—caller and callee are on the same page—but it does make the workings of the RTCPeerConnection API a little clearer, since the RTCPeerConnection objects on the page can exchange data and messages directly without having to use intermediary signaling mechanisms. In this example, pc1 represents the local peer caller and pc2 represents the remote peer callee. This can be done directly in the code without using signaling, because both caller and callee are on the same page: pc1. NAT traversal, peer-to-peer networking, and the requirements for building a server app for user discovery and signaling, are beyond the scope of this article. Suffice to say that the protocol and its extension are used by the framework to enable RTCPeerConnection to cope with NAT traversal and other network vagaries. ICE is a framework for connecting peers, such as two video chat clients. Initially, ICE tries to connect peers directly, with the lowest possible latency, via UDP. In this process, STUN servers have a single task: to enable a peer behind a NAT to find out its public address and port. You can find out more about STUN and TURN from the HTML5 Rocks article. Finding connection candidates If UDP fails, ICE tries TCP. If direct connection fails—in particular, because of enterprise NAT traversal and firewalls—ICE uses an intermediary relay TURN server. In other words, ICE will first use STUN with UDP to directly connect peers and, if that fails, will fall back to a TURN relay server. The expression 'finding candidates' refers to the process of finding network interfaces and ports. WebRTC data pathways WebRTC engineer Justin Uberti provides more information about ICE, STUN and TURN in the. The presentation give examples of TURN and STUN server implementations. This app uses , a shim to insulate apps from spec changes and prefix differences. For full interop information, see. The code is deliberately verbose in its logging: check the console to understand the order of events. Below we give a detailed walk-through of the code. If you find this somewhat baffling, you may prefer our. This step-by-step guide explains how to build a complete video chat application, including a simple signaling server running on a. Network topologies WebRTC as currently implemented only supports one-to-one communication, but could be used in more complex network scenarios: for example, with multiple peers each communicating each other directly, peer-to-peer, or via a MCU , a server that can handle large numbers of participants and do selective stream forwarding, and mixing or recording of audio and video: Multipoint Control Unit topology example Many existing WebRTC apps only demonstrate communication between web browsers, but gateway servers can enable a WebRTC app running on a browser to interact with devices such as aka and with systems. In May 2012, Doubango Telecom open-sourced the , built with WebRTC and WebSocket which among other potential uses enables video calls between browsers and apps running on iOS or Android. Telephone communication without a carrier! The RTCDataChannel API enables peer-to-peer exchange of arbitrary data, with low latency and high throughput. There are several simple 'single page' demos at and our shows how to build a simple file transfer application. RTCDataChannel is available in Chrome, Safari, Firefox, Opera and Samsung Internet. The game uses the API to communicate game state: play a friend or play the bear! The innovative platform enabled file sharing via RTCDataChannel, and offered a glimpse of how WebRTC could enable peer-to-peer content distribution. For more information about RTCDataChannel, take a look at the IETF's. Security There are several ways a real-time communication application or plugin might compromise security. A full discussion of security for streaming media is out of scope for this article. For more information, see the proposed by the IETF. In conclusion The APIs and standards of WebRTC can democratize and decentralize tools for content creation and communication—for telephony, gaming, video production, music making, news gathering and many other applications. Technology doesn't get much more than this. We look forward to what JavaScript developers make of WebRTC as it becomes widely implemented. As blogger Phil Edholm , 'Potentially, WebRTC and HTML5 could enable the same transformation for real-time communications that the original browser did for information. Johnston and Daniel C. Native APIs for RTCPeerConnection are also available:.
