Navigating the complexities of local development: Our journey to Tilt at Jam

Hello, I'm Arég, an engineer at Jam. Today, I’m diving into a topic that I think resonates with most software developers – the local dev environment. At Jam, like many other companies, we’ve grappled with the challenges of efficiently managing our devStack. So, my focus today is on what happens before the shipping phase, specifically the evolution of local development ergonomics.

If you're like Jam, you have a bunch of different services. It all starts with one, and maybe you have a front-end that needs to talk to it. And then a problem pops up, and you're like, “well, this is a microservice architecture, so I need to make another service for this…” And pretty soon, you may have dozens of services. Spinning them all up in your local dev environment becomes a chore, right? At Jam, we have eight primary services. But, it's not just the primary services that you're running on your local machine. They also have dependencies. 

So, in this blog, I'll delve into why and how we chose to integrate Tilt into our development workflow, detailing each step of our journey and the impact Tilt has had on our team's efficiency and morale.

Level zero: The groundwork of local development

My journey at Jam began at what I call "level zero." This is where every developer starts – setting up their local environment. At Jam, this involved detailed instructions in a README or Confluence page. We had a plethora of steps to initiate our dev stack, which was no small feat considering our stack includes multiple primary services and dependencies like PostgreSQL for data persistence, Redis for caching, and emulated Google Pub/Sub for event pushing.

Level one: The advent of dev Scripts

As the stack grows, the manual setup becomes increasingly cumbersome. This leads to "level one" – scripting the repetitive tasks. We developed shell scripts to automate the setup process. These scripts were initially a boon, reducing the time and effort for developers. However, as the script expanded (reaching over 400 lines), it became unwieldy and difficult to maintain. A large shell script is prone to errors and can become a maintenance nightmare.

Skipping level two: Why Docker Compose didn't fit

Typically, the next evolution would be "level two," using Docker Compose to manage microservices. However, at Jam, we decided to skip this step. Docker Compose, while powerful, didn't quite align with the complexity and scale of our architecture. We needed a solution that was not only scalable but also intuitive for our team to use.

Level three: Tilt

This brought us to "level three" – embracing Tilt, an open-source tool that fundamentally changed our approach to local development. Tilt allows us to declaratively describe our development environment, ensuring consistency across all developers’ local setups. This was a significant shift from our previous methods.

The power of Tilt

With Tilt, the complexity of our local development instructions was drastically reduced. What used to be an extensive setup process became as simple as running `tilt up`. This command initiates a user-friendly web UI, listing all services and their statuses. Developers can see which services are starting, running, or have encountered issues. It's a seamless and interactive experience that significantly boosts productivity.

Tiltfile: Declarative and efficient

The core of Tilt lies in its Tiltfile. Inspired by Dockerfiles, it allows us to define all our services, dependencies, and their initialization order declaratively. This file respects the dependency tree, ensuring that our complex microservices are organized and initialized correctly. It's a transparent and efficient way to handle our local dev stack, eliminating the need to reinvent the wheel for every new service we add.

Maintainable impact 

Our journey through the levels of local development ergonomics led us to Tilt, a decision that has significantly improved the way we handle our devStack at Jam. This transition added efficiency and also to the overall experience of the engineering team. 

For my fellow developers grappling with similar challenges, I hope my journey offers insights and encourages you to explore tools like Tilt. It's all about finding the balance between technical capability and simplicity in operation. I look forward to hearing your thoughts and experiences, reach out if you ever want to Jam!