REST and Transactional Web Services

There's a long way to go before distributed programming becomes as safe and as simple as non-distributed programming. But if the Web is going to meet its potential of interoperable services locking together to create seamless, smart software, we need to get there. And here's something we need: distributed transactions.

One shortcoming we've found to Amazon Web Services has been this: we can send something to S3, to SQS, and then to our database, but while we can rollback our database, we cannot rollback Amazon. There is a danger that an application operating over these distributed services can be left in an inconsistent state, and that's just no fun for users. EJB was designed to solve such problems with distributed transactions, but the cost of that was, well, EJB.

How does REST clarify this picture? REST gives us a uniform, simple way to describe web services. This makes it easy to describe one Web service call to another Web service. In fact, it makes it easy to describe any RESTful Web service call to a Web service. Yes, the meta-web service! We need only supply a verb, a URL, and maybe a few headers. Signature-based authentication makes it easy to authorize a service to make a call on another without releasing your credentials. With this, I can imagine Web service filtering or Web service tunneling: calling one service through another.

Here is the Web service I would love to see Amazon develop next: a transaction service. There are many, many ways to do this, but here's just one proposal. A client can create a transaction resource with Amazon, describing all the service calls comprising the transaction. Then, Amazon can act as a coordinator for a 2PC (or 3PC) transaction protocol. Any service that can work within such a transaction, of course, has to implement the protocol.

This has some disadvantages. For one, the client wouldn't be able to change state based on the responses of intermediary service calls. For another, it's hard to see how the client could commit the transaction synchronously (And without synchronous response, much of the ease of use of the Web is lost).

For the first problem, I can imagine another way of working this without tunneling. This involves splitting up coordinator duties in 2PC. The transaction client can herself make calls to cohorts during commit-request phase, then letting the transaction service takeover as coordinator during the commit phase.

As for the second problem, creating a long-running task is pretty easy for human clients. Just give them a callback for their browsers to keep polling. BackgrounDRB works like this. For the machine-readable Web, a callback URL to submit check transaction status changes is easy enough. Another alternative is for the service to expose a REST transaction resource which its consumers can poll.

What more is needed?

(1) Some way to describe transactional status within REST resources.

(2) Some way for applications to implement transactional undo/redo across network latencies without locking too many resources. This part is difficult. Database transactions really aren't up for being kept open for long periods of time. We may need new tools to make this happen or at least a better understanding of the design and performance issues involved.