Application developers frequently choose REST in the name of simplicity, but for real-time apps, REST introduces unnecessary complexity and performance bottlenecks.
REST based APIs, stateless computing and database-centric applications are the go-to patterns of today’s web and cloud apps. They are fantastic for building applications with unpredictable user loads. Developers don’t have to worry about infrastructure, because the abstractions address key needs: When an API call arrives, the URI of the request resolves to the current address of a server that hosts the API, and the request is delivered. If you’re using a serverless app infrastructure, you don’t even have to ensure that the front end of your app is running - it is dynamically loaded by the serverless infrastructure that hosts the service.
But REST, stateless computing and databases make it nearly impossible to build applications that can respond in real-time to changes in the real-world. And note that there are two requirements here: apps must always know the real-time state and analysis of the infrastructure, and they need to be able to react in real-time.
- REST APIs use a stateless model for updates. Each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. A client has a limited request vocabulary, namely GET/PUT/POST/DELETE. If it wants to wait for a request response it must poll the API. All this imposes per transaction delays which are fine for web apps, but problematic for data-driven applications.
- Any stateless application that is frequently accessed to deliver state updates from the real world must complete an update before the next one arrives. But many devices update their state at a very high rate - faster than a stateless application can read, update and write the new state back to the database. Here’s an example: an industrial compressor generates 70 data points per degree of rotation of a shaft. There are four shafts per device, rotating at 1,200 RPM. Lots of… mostly monotonous data. But an app needs to respond immediately if there is a trend that indicates valve failure.
- Cloud databases are typically 8 orders of magnitude slower than memory in the system that makes the database call. Putting it in human terms, if a memory operation took an hour, reading a database would take about 80 years. So the CPU is idle for a vast amount of time. An architecture that uses the billions of wasted CPU cycles close to where the data is generated, has a huge advantage.
How can a REST application be real-time if it spends most of it's time asleep, while waiting for an update from a centralized database?
Swim uses a streaming upgrade to HTTP, called WARP, that makes it easy to develop stateful, distributed, real-time applications that analyze, learn and predict from streaming data, on-the-fly.
Let’s start with an example: The traffic infrastructure of the city of Palo Alto generates about 50MB/s of data. Our goal was to predict the immediate future behavior of each traffic intersection to help vehicle routing applications choose routes that minimize transit delays. You can see a real-time visualization of the city’s free running intersections here.
To predict short-term evolution of traffic flow requires us to process traffic sensor events in real-time followed by fast inference to deliver predictions. We compared two approaches:
- A RESTful Lambda application in AWS to receive sensor events for forwarding either to a database for use in training, or into a data flow pipeline to Flink for inference at runtime, and
- Swim stateful processing at the edge, with WARP based streamed predictions from the edge to a cloud hosted API
The first cost about $6,000 per month for Lambda calls, data storage, and ML. The second runs on spare cycles of a traffic management server in Palo Alto, requiring no new hardware or bandwidth upgrades. We present the predictions to customers via an API in Azure, which uses about 0.2 CPU cores. Let’s say the difference is 3-4 orders of magnitude.
Learn More
Swim is an Open Source platform for building stateful data-driven applications that continually compute and stream their insights in real-time. You can get Swim here.