History
The web has been largely built around the so-called request/response paradigm of HTTP. A client loads up a web page and then nothing happens until the user clicks onto the next page.
Around 2005, AJAX started to make the web feel more dynamic. Still, all HTTP communication was steered by the client, which required user interaction or periodic polling to load new data from the server.
Technologies that enable the server to send data to the client in the very moment when it knows that new data is available have been around for quite some time. They go by names such as “Push” or “Comet”. One of the most common hacks to create the illusion of a server initiated connection is called long polling. With long polling, the client opens an HTTP connection to the server which keeps it open until sending response. Whenever the server actually has new data it sends the response. Long polling and the other techniques work quite well. You use them every day in applications such as GMail chat.
Protocol: Bayeux vs WebSocket
The CometD framework is an implementation of the Bayeux protocol, which allows for a multi-channel asynchronous communication stream over unreliable networks between a client and server. Implementations of this are used in many languages (JavaScript, Java, Perl …) but predominantly browser-based AJAX applications. Bayeux has the advantage that it can run in any AJAX capable browser using nothing more than the underlying HTTP communication facilities provided by the browser to achieve asynchronous/background updates as new information comes in (like Google Mail’s new mail notifications). In fact, the same protocol can be used to connect devices in other languages and where network communication may be spotty, such as mobile devices.
WebSockets is a draft standard which is sponsored by Google, Apple and others at the WhatWG working group that is standardising HTML 5. As a result, HTML 5 capable browsers (Chrome, Safari) are starting to include built-in support for the WebSocket protocol.
Both protocols aim to allow web-based AJAX applications to communicate with other services via asynchronous messaging or socket-based connections, rather than having to roll your own communications layer on top of an existing application. This allows the design of an application to focus on the component parts, and hand off messages to the communication layer for delivery. In addition, both can set up long-running connections such that events can be delivered asynchronously subsequently to the application. This is nothing new: HTTP 1.1 supported connection pipelining (the ability to keep open a connection after each request, and the ability to send multiple requests before the first was acknowledged); and other protocols like IMAP supported the IDLE command to put a connection into a hibernate state, where there is no ongoing communication but the server can push new messages at any time. Indeed, prior to either Bayeux or WebSockets, the term “HTTP Push” was used to indicate a general mechanism for long-lasting communication channel across HTTP.
Problems and Challenges
The real challenge is HOW TO MAINTAIN THE CONNECTION!
A connection that hasn’t had any data over a period of time may be considered to have died, and arbitrarily terminated, at some point in the future. To address this, the IMAP IDLE suggests that clients send a re-negotiated IMAP IDLE command every 29 minutes to avoid disconnection. With other proxies in the way of HTTP, it may be that a proxy determines that a connection is idle and drops the connection, even if the client and server agree to maintain an ongoing connection.
The other problem is resource-based; generally, browsers limit the number of concurrent HTTP connections to a single server to avoid overbearing the server (or the network link). It’s common for browsers to limit such concurrent connections to between 2 and 4 at any one time.
Both Bayeux and WebSockets attempt to avoid the resource limits by using a fall-back mechanism for long polling (in the case of Bayeux) or switching to a non-HTTP based secondary protocol instead. As a result, users of these libraries don’t generally have to worry about limitations placed on them by the browser or infrastructure.
COMETD 2.x
CometD 2 now supports both Bayeux, and Websocket (which as an alternative transport to the currently supported JSON long polling and JSONP callback polling). CometD is transparent to browsers with or without websocket support. Websocket usage will be able to give us even better throughput and latency for cometd than the already impressive results achieved with long polling.
References
Bayeux Protocol Specification
W3C The WebSocket API
Alex Russell’s original 2006 comet blog post