October 04
In Search of Publish-Subscribe
For most of the last year I've been working on an exciting project, one major component of which requires publish-subscribe communication. Pub-sub continues to grow in popularity because it offers many benefits:
- The connections between applications are loosely coupled.
- Applications no longer need to know the addresses of other applications.
- Routing of information is determined by central configuration rather than details embedded in application code, and is easily changed.
- New applications can easily be hooked in to a system simply by creating a new subscription for them.
For all these reasons and more, we at Neudesic have been considering the best way to accomplish Publish-Subscribe in the Microsoft world. To find the best way, you have to begin by evaluating your options. These are the options we looked at:
- BizTalk Server does have a pub-sub mechanism at the core of its messaging engine, but it's not a general-purpose pub-sub communication facility that .NET applications can get at or use. Its main purpose is to connect senders and receivers with the BizTalk message box across one or more servers.
- WCF doesn't ship with any specific support for publish-subscribe, though we might expect to see something along those lines in the future. However, there is WCF PeerChannel, which creates a multi-party cloud or mesh. Any message sent to the cloud goes to all parties. Since you can set up as many clouds as you wish, you could have one cloud per topic. WCF PeerChannel has some attractive features, including low latency (fast) communication and the ability to scale up to 10s of 1000's of nodes without requiring an investment in server infrastructure. However, the information is not transacted or durable and there are no delivery assurances.
- MSMQ seemed to be leaning toward a pub-sub set of features in its past but information on this is hard to come by lately. There is MSMQ over UDP, which broadcasts messages to large numbers of recipients. In the enterprises I've worked with where this is in use, the loss rate is often considered unacceptably high. Alternatively, you could make use of MSMQ queues--since most pub-sub systems are based on queues--and use custom code to move messages between queues.
- SQL Server could be used as a message repository if you were going to build your own durable publish-subscribe system. SQL Server 2005 also offers communication features through its Service Broker including its own message queue facility. But you'd certainly be writing a lot of custom code to publish and receive messages and to move messages between queues.
- WS-Eventing is a standard that allows differnet messaging systems to subscribe to each others messages using web services as a way to receive messages. This standard describes the mechanisms of requesting and managing subscriptions, but doesn't spell out how pub-sub itself should be implemented.
In trying to determine which of the above is "best", it became clear that each possibibility possesses some unique capabilities that some enterprises might find important. For example, the speed, reliability, durability, and recovery story for each of the above varies. Moreover, enterprises vary in which technologies they have invested in, so a BizTalk solution might be most attractive for one enterprise but less attractive to another.
Our decision was to support all of these methods of publish-subscribe with an abstracted interface that can perform the same messaging capabilities across any of them. You can read about how we did this in our ESB White Paper (part1, part2). The benefits of this approach are many:
- Customers can take advantage of the technologies and products they have an existing investment in
- Customers can run several different qualities of service in parallel
- The modular driver approach makes it straightforward to graft in new technologies as they become available