Why Swim is Prioritizing Polyglot Programming

by Brad Johnson, on Jul 2, 2019 3:19:54 PM

Today, the Java ecosystem remains one of the largest developer communities on planet earth. At over 9 million active developers, the Java programming language is used every day to build and run systems that shape the modern everyday experience. That’s why we ultimately decided to architect Swim in Java, and leverage the same patterns and possibilities that made Java the right choice for so many developers. We could keep ourselves busy for years building tools and writing documentation just to support Swim’s Java user community. But instead, one of our most significant efforts recently has been to improve polyglot support for the open source Swim platform. Why?

We aim for Swim to become the go-to development platform for anyone building distributed, real-time applications. As such, it’s important for us to build the Swim platform in a way that makes it easy for developers to leverage their favorite open source libraries within Swim apps, regardless of programming language. By enabling users to inject piecemeal functions (written in multiple languages), and then run them run natively in Swim, we ensure that the Swim community can continue to benefit from other innovative projects in the open source world. We’re close to being able to build full Swim apps in JavaScript, Python, Ruby, and R, in addition to the usual Java and Scala we typically use when building Swim apps.

Polyglot Programming: Updates to the Swim Platform

In order to build in support for polyglot programming, we’re making a few architectural changes to the Swim core platform. One of the more significant changes was to take advantage of the language support provided by GraalVM. If you’re not familiar with GraalVM, it’s a “a universal virtual machine for running applications written in JavaScript, Python, Ruby, R, JVM-based languages like Java, Scala, Groovy, Kotlin, Clojure, and LLVM-based languages such as C and C++.” By utilizing GraalVM, we’re making it possible to run Swim applications in a shared runtime which supports language interoperability. This will enable developers to do relatively simple things like embed a Python script into a streaming data pipeline, or enable more complex use cases like running an entire ML library within a Swim Web Agent, to continuously train on real-time data streams.

The polyglot language integration is handled in two layers within the Swim Java framework. The Swim dynamic layer deals with dynamic language bindings for Swim APIs. It’s typeclass based, allowing us to decouple the language bindings from the libraries that they bind. It also lets us inject slight API variations for different guest languages, ensuring that each API feels natural in every guest language. However, Swim’s dynamic bindings are still decoupled from any particular VM—it does not depend on GraalVM.

The actual polyglot VM integration is handled by a Swim polyglot layer. This layer injects swim-dynamic bindings into GraalVM guest languages. This new layer makes it possible to do some really cool things. You can create arbitrary expression trees, and Graal will compile them ahead of time when building a native image, JIT them when running on the Graal substrate VM, or interpret them when running on any other JVM. There may be some additional simplification possible by better leveraging GraalVM’s internal Truffle APIs (Truffle is Graal’s language implementation framework), but this is not an immediate priority for us at the moment.

A Frictionless Way to Run Stateful Lambda Functions

One significant outcome of the shift towards polyglot programming is the ability to support running stateful lambda functions, by containing them in Swim Web Agents. The notion of “stateful lambdas” may seem like an oxymoron. Aren’t lambda functions inherently stateless? For example, the whole value statement of using a service like AWS Lambda is that it’s serverless, in that it doesn’t require setting up a dedicated application server in order to run. Serverless apps (and functions) are ephemeral. They’re spun up and phased out as needed, and you never have to set up a custom application backend to make them work. That seems like a good thing.

But what if you want the function to exist continuously? This is the beauty of having a stateful lambda function. Stateful lambda functions are persistent, and have local access to application state via a Web Agent, so they can continuously transform data as it flows. By setting up arbitrary business logic (for example, triggering an alert when a predetermined threshold is met), stateful lambda functions can operate independently of a database or other central datastore. This has far-reaching consequences for edge computing, industrial applications, IoT, autonomous vehicles, gaming, AR/VR and other use cases which require real-time automation. Persistent, stateful lambda functions are a simple way for doing these real-time transformations on streaming data, without introducing the infrastructure complexity or required for monolithic or microservices-based application architectures.

In future posts, we'll highlight some real-world examples of how Swim-style stateful lambdas can work for real-time use cases. In the meantime, keep an eye out for more technical details about Swim architecture changes.

Once we have a stable release, we’ll be pushing the updated core libraries to the Swim open source project on GitHub. Exciting stuff, with lots more coming soon!

Learn More

Share your thoughts stateful applications in the comments section below and let us know what you're building using the open source Swim platform.

You can get started with Swim here and make sure to STAR us on GitHub.

Topics:Stateful ApplicationsEdge AnalyticsSWIM AIdistributed computingserverlessweb applicationsswimOSWARPstreamingnodejspythonweb agentsstreaming apijavastateful lambdaspolyglot programminglambda functions