Marcus Cavanaugh

People are resistant to change. While today’s Comet servers and transports may be an intermediary step in the evolution toward the two-way web, we should not allow fundamental issues to remain in a protocol which may be in use for many years to come.

Comet, in its purest form, is a catalyst for two-way asynchronous communication with the browser. Two-way communication brings new opportunities for interactivity in web applications; true two-way communication would be an incredible boost to the web as a platform.

The web today lacks a reliable streaming transport for sending messages to web browsers (flash excluded), and a large part of a Comet server’s role is to establish a layer of abstraction above long-polling, iframe hacks, and the like. HTML5 and related technologies will allow web browsers to use a better underlying transport (server-sent events), but we won’t be able to throw away the long-polling and iframe hacks for a long time.

Bayeux Issues

In the “Battle of the Bayeux” Part 6, Alex Russell stated:

I couldn’t help but feel that Michael (and many others I’ve talked to) have simply missed the entire point of Bayeux, which is to specify enough structure to grow the number of interoperable Comet clients and servers while leaving enough to the implementations to allow integration of piecemeal evolution until such a point as the protocol itself is unnecessary.

If we follow that logic, we’re going to shoot ourselves in the foot. Here’s why:

1. Comet will be around for many years. Until the current generation of browsers die out, we must use ugly hacks to create a server-to-client asynchronous transport.
2. Comet will grow. The two-way web is a strong force, and the web is headed that way; asynchronous browser communication will become more prevalent, not less.
3. If Alex’s goal for Bayeux holds true, Bayeux would be implemented by many more clients and servers. The more people using a standard, the more deeply it becomes entrenched. If Bayeux becomes the de-facto Comet standard, it too will be around for a long time.
4. Once entrenched, Bayeux would be difficult to change and/or remove. We cannot expect Bayeux to both be a temporary workaround and a target for standardization and growth — the more Bayeux gets implemented, the more firmly planted it will become.

To me, Alex’s goals for Bayeux are contradictory and mutually exclusive: You shouldn’t try to become widespread and only be a temporary solution.

But Bayeux has a deeper design issue, in my opinion, and I believe its issues should be solved while we still have the opportunity to fix them.

Bayeux is Tightly Coupled.

Many web applications do not need publish-subscribe. In fact, pure publish-subscribe is often a bad idea when the server needs to send specific events to clients. Take an online card game, for instance. The server must send different cards to each player; here are a few options for handling it with Bayeux:

1. Use publish-subscribe, with the card deaer acting as a Bayeux client. Sure, we could have clients each subscribe to their own channel and have the dealer (as another Bayeux client) send events to each of the channels. But Bayeux servers do not provide methods for channels to enforce authentication or authorization — I could maliciously subscribe to another player’s channel and receive their cards!
2. Implement the card-dealing server as a Bayeux server. You could create a server which exposes the Bayeux protocol, but then you’d have to implement the individual comet transports. That’s not easy to do and certainly isn’t within the scope of your application.
3. Add a plugin to an existing Bayeux server to use the “/service/” channels. You could use the “/service/” channels to send events to clients, perhaps — but suddenly you are tied to a specific Bayeux server, and you’re not even using the publish/subscribe infrastructure of Bayeux. That’s a lot of unnecessary overhead to carry.

In the end, if you don’t use publish-subscribe, you’re stuck with its extra overhead and you must work around it, since Bayeux places pub/sub at the center of its design. (If you really want to make effective use of publish/subscribe, it’d be best to use technologies dedicated to solving that problem like AMQP, JMS, XMPP, etc. rather than a pub/sub tier built into a comet server.) I firmly believe that publish-subscribe is not suitable for all use-cases, and allowing it to be the fundamental method of communication in Bayeux is a serious design issue.

Bayeux is tightly-coupled, and if we’re striving for widespread comet adoption, it is a bad idea to promote a full-stack protocol which doesn’t properly address fundamental issues in its design. (I have only touched on publish/subscribe in this post, but other issues have been discussed at length in the “Battle of the Bayeux” series.)

Michael Carter pointed out:

It is very easy to start optimizing towards a particular use case, and for many projects that is the right and obvious approach. For a standardized protocol though, the designers must be extraordinarily careful that they create a technology that is robust enough to withstand the test of time. If they do not, then a whole new set of de-facto standards will evolve, ultimately displacing the original standard. If we tackle this problem now then we can avoid multiple years of setback.

Much has been written about this before, and will be written in the future, but I think this is true: a layered approach is a better way. Loosely-coupled is a good design paradigm. By separating publish/subscribe from the Comet transport, we can then swap out the transport abstraction as needed in the future, when we have better underlying transports. Publish/subscribe should be an abstraction above generalized Comet communication, not fused with the web-transport abstraction layer.

We should not treat Bayeux as a throwaway intermediate solution to Comet architecture. Bayeux has serious design issues which make it unsuitable for many applications, but we do need some standard for communication which abstracts away current Comet transports until we get native reliable, streaming communication in browsers.

A flawed architecture now will haunt us in the future. If Bayeux is not designed to be ready for the long haul, we need to replace or fix it before it’s too late. We must not stand by and let ourselves look back on this in five years, saying “I wish we had fixed these issues back when we had the chance.”

Comet transports will come and go; publish/subscribe isn’t always needed. But we will need two-way server-to-browser communication for the foreseeable future. Loose coupling would place a Comet communication standard in a much better position looking forward.

As an aside, I applaud and appreciate the Bayeux developers — they’ve begun to pave the way toward widespread Comet adoption. Whether solving these changes involves fixing Bayeux itself or replacing it with something new isn’t all that important, as long as these issues become resolved. While the “Battle of the Bayeux” promotes interesting debate, it’s important to not let it cause rifts in the Comet developer community. The Comet community may have differing opinions on Bayeux, but I believe it’s worth exploring the design of Bayeux to discover how to best position Comet as an up-and-coming technology.

Bayeux is far from ideal. Loose coupling and a layered architecture would provide a more future-proof foundation. Now is the time to address these issues.

We have the unique opportunity of popularizing two-way communication with web browsers. I think we owe ourselves the duty of making sure we do it properly and with a watchful eye toward the future.

Note: There are points in this article which I have not elaborated upon due to length. If you believe I have glossed over something, let me know and I will elaborate to substantiate my opinion.