Understanding WebRTC

In this article, I will explore WebRTC and its important concepts

What is WebRTC

WebRTC (Web Real-Time Communication) is a technology that enables applications to have video, voice and message communications in real time. In general, it covers media capture devices and peer-to-peer connectivity.

Media capture devices includes video cameras and microphones, but also screen capturing “devices”. You can use navigator.mediaDevices.getUserMedia() to capture MediaStreams for cameras and microphones. For screen recording, you use navigator.mediaDevices.getDisplayMedia()

Peer connection

The peer-to-peer connection refers to connect two applications on to communicate using a peer-to-peer protocol. To connect apps behind firewalls, both clients need to provide an ICE Server configuration. The configuration asks for urls of A STUN and a TURN server. They are used to provide ICE candidates and help them to connect. The transferring of ICE candidates is commonly called signaling.

Signaling

The WebRTC supports communication with ICE Servers. However, to establish connection between peers, and share how they should connect, a signaling component is needed. WebRTC doesn’t include it. Use Socket.io to is an option to implement it.

Initiating peer connections

The RTCPeerConnection object handles peer connection. According to MDN, the RTCPeerConnection interface represents a WebRTC connection between the local computer and a remote peer. It provides methods to connect to a remote peer, maintain and monitor the connection, and close the connection once it’s no longer needed. When create a RTCPeerConnection object, it should contain information about the ICE servers to use.

The calling peer need to createOffer(). It creates a RTCSessionDescription object. This session description is set as the local description using setLocalDescription() and is then sent over our signaling channel to the receiving side.

The createOffer() method of the RTCPeerConnection interface initiates the creation of an SDP offer for the purpose of starting a new WebRTC connection to a remote peer. The SDP offer includes information about any MediaStreamTrack objects already attached to the WebRTC session, codec, and options supported by the browser, and any candidates already gathered by the ICE agent, for the purpose of being sent over the signaling channel to a potential peer to request a connection or to update the configuration of an existing connection.

On the receiving side, it uses the incoming offer to create a RTCSessionDescription and set it as the remote session description using setRemoteDescription(). createAnswer() is called to create an answer to the received offer. Then, set the answer as the local description using setLocalDescription() and finally sent the answer to the calling side over the signaling server.

When two peers have set both the local and remote session descriptions, it doesn’t mean the connection is ready. To finally connect to each other, gather ICE candidates at each side and its remote peer is a must.

ICE candidates

ICE is the external service to discover the possible candidates for connecting to a peer. It uses either a STUN or a TURN server.

STUN stands for Session Traversal Utilities for NAT, and is usually used indirectly in most WebRTC applications.
TURN (Traversal Using Relay NAT) is the more advanced solution that incorporates the STUN protocols.

Most WebRTC uses a TURN server to enable peer-to-peer connection. But both are supported. When creating RTCPeerConnection , ICE servers are provided in the configuration.

Trickle ICE

The “trickle ice” technique and transmit each ICE candidate to the remote peer as it gets discovered is used to reduce the setup time for the peer connectivity. You can add eventListener to handle icecandidate events.

An icecandidate event is sent to an RTCPeerConnection when an RTCIceCandidate has been identified and added to the local peer by a call to RTCPeerConnection.setLocalDescription(). The event handler should transmit the candidate to the remote peer over the signaling channel so the remote peer can add it to its set of remote candidates. This event is not cancelable and does not bubble.(From MDN)

The RTCPeerConnectionIceEvent emitted on that listener will contain candidate property that represents a new candidate that should be sent to the remote peer

/**
 * Ref: https://webrtc.org/getting-started/peer-connections
 * */
// Listen for local ICE candidates on the local RTCPeerConnection
peerConnection.addEventListener('icecandidate', (event) => {
  if (event.candidate) {
    signalingChannel.send({ 'new-ice-candidate': event.candidate });
  }
});

// Listen for remote ICE candidates and add them to the local RTCPeerConnection
signalingChannel.addEventListener('message', async (message) => {
  if (message.iceCandidate) {
    try {
      await peerConnection.addIceCandidate(message.iceCandidate);
    } catch (e) {
      console.error('Error adding received ice candidate', e);
    }
  }
});

After ICE candidates are received, a connection is established.

Remote streams

When RTCPeerConnection is connected to a remote peer, it can stream audio and video between them. You can access local stream by using navigator.mediaDevices.getUserMedia({ audio: true, video: true })

To transmit the media to the remote peer, you can add the local stream to RTCPeerConnection

localStream.getTracks().forEach((track) => {
  //peerConnection is the RTCPeerConnection
  peerConnection.addTrack(track, localStream);
});

Adding remote tracks

To receive the remote tracks that were added by the other peer, you can add an eventListener to the local RTCPeerConnection. It listens for the track event. Set the remotePeerVideo.srcObject to event.streams[0]. The remotePeerVideo is the element object that displays remote peer’s video and audio.

Reference

WebRTC API - MDN
Google WebRTC Doc

A study notes on the Google WebRTC doc and MDN