Sorting Through "Real-time" Hype, Kubernetes, and More
by Brad Johnson, on May 15, 2019 10:34:32 AM
Last post, we explored the challenges of architecting a real-time application, which are due in part to the many real-time technologies available in the open source ecosystem. Part of the problem is that real-time has become one of those kitchen-sink terms used to describe anything and everything. There’s no industry-standard definition for real-time; the common usage tends to be synonymous with fast. But real-time and fast don't mean the same thing. At Swim, we define real-time as meaning “right now" (give or take network latency). For example, consider the following use cases. If you’re automating an industrial process, you want to actuate a response as soon as a critical event triggers. This is a real-time application. If you’re only interested in learning about past machine behavior to make better decisions later, it’s not a real-time use case.
So how about “near” real-time applications? In my experience, no one has ever wanted an app to be exactly “near real-time.” Typically, “near real-time” is a compromise when it seems that actual real-time is too expensive or difficult to achieve. The reality is, developers are creative enough to manipulate available technologies to achieve whatever the desired performance is for their apps; real-time, “near” real-time, or other. As more developers adopt stateful, streaming-first technologies like Swim, we’re likely to do away with the notion of “near real-time” altogether.
On Kubernetes...and Containers...and Serverless...and Blockchain...
Kubernetes may be the only thing that’s even more popular than real-time, and maybe even less widely understood. Kubernetes (often just K8s) is seemingly everywhere. Gartner calls this “the peak of inflated expectations.” During TPOIE, technologies get buzzed about like a new pop culture trend, and are presented as a panacea for solving any technical challenge. This certainly isn’t unique to Kubernetes. This same pattern has held true for blockchain, microservices, serverless, containers...and the list goes on. But TPOIE is when you start seeing the “Do You Have a Blockchain Strategy for Your Pet Grooming Studio?” and “How The Cloud Will Help You Make a Better Cup of Espresso” articles. You know, stuff like this and this.
In the right context, each of these technologies is a powerful enabler for developers. However, in the zeal to adopt the latest and greatest, these technologies get shoehorned into projects regardless of fit. It can be difficult to sort through the hype and identify which technologies suit your particular application. Early adopters are usually familiar with the technical problem(s) a new technology like Kubernetes solves, and are therefore better equipped to utilize it. The same is true with Kubernetes. If you were already familiar with the concepts of VMs, software containerization, and other DevOps notions, then a distributed containerization system like Kubernetes can be a massive productivity boon over architecting a bespoke container management solution. Factor in a vibrant open source ecosystem, native integrations with a number of other software technologies, and rapid adoption by many key industry players and the success of Kubernetes makes total sense. So how do you determine if Kubernetes, Swim, or some other technology is right for your use case?
Solving DevOps Problems Versus App Dev Problems
Kubernetes solves a DevOps problem. That isn’t to say that only engineers with DevOps on their business card benefit from Kubernetes. New applications can be developed on top of Kubernetes, which utilize Kubernetes to simplify deployment, scaling and monitoring of applications. But Kubernetes doesn’t really help you build the “app” part of an app, it helps you with the infrastructure part. You still need to develop your application logic, setup data pipelines, and then choose and integrate servers, brokers, databases, and everything else yourself.
Swim solves an application development problem. With Swim, you don’t need to build a bespoke application architecture. You simply write your application logic and setup streaming links. If you want, you can integrate with Kafka, Cassandra, or a Hadoop database by setting up ingress or egress bridges. Swim is different from a distributed containerization system like Kubernetes. Where Kubernetes distributes processes, Swim distributes continuously consistent graphs of data and processes that run on that graph. Kubernetes provides no decoupling of logical application structure from physical compute infrastructure. Swim automatically optimizes use of hardware resources, so developers only have to focus on writing their application logic. Whereas Kubernetes makes it easier to consider the logistical aspects of application infrastructure, Swim makes it easier to reason through the logical aspects of your application, itself. Importantly, this is why Kubernetes and Swim are not mutually exclusive. Swim can help developers build streaming analytics or automation on top of Kubernetes applications, or can even provide additional real-time monitoring of KPIs provided by Kubernetes.
Like with any project, it's critical to have the right tools. If you're looking for a distributed containerization system, Kubernetes is the clear place to start. And if you're building a new real-time application or web service, make sure to check out Swim.
Swim is an Open Source platform for building distributed real-time applications. For technical resources, you can find out more about Swim here.