How our Service Templating Engine Improves Microservice Development At Shippo

Microservices architecture has become a cornerstone for many modern development teams, providing scalability and flexibility. While the debate over the pros and cons of microservices continues, this article delves into the importance of investing in templating tools to streamline the development process. We’ve found templates can serve as a canonical example of technology use, encourage best practices, harmonize microservices, reduce time to first deployment, and keep developers happy. To fully realize these benefits, we created a library of our own that supported not just service initialization, but modification and updates as well.

Why invest in service templates?

Investing in service templates allowed us to realize several benefits, including:

• Canonical examples of technology use: For example, Kafka is new to our team relative to other messaging technologies. A template allows developers to quickly add a functional producer or consumer to their application without scouring our codebase or the internet for a reference implementation.

• Encouraging best practices: Templates make it easy to lay down unit tests, robust pre-commit checks, least-privilege access, and observability in the first draft of a service rather than hoping Proof of Concept is “productionalized” once it takes off.

• Harmonizing microservices: Templates promote the adoption of common technologies by building a golden path and establishing shared conventions that help developers understand and contribute to other teams’ services.

• Reducing time to deploy in production: Templates dramatically cut the time it takes to deploy a minimally viable service to production which makes our developers feel productive and happy.

• Coordinating non-local configuration changes: Ideally, all configurations for a service would live in a single place in a single repository. In practice, launching a service may require editing different repositories containing configurations for the Service Mesh, Kubernetes, and Infrastructure. Templates can stage PRs wherever these configurations live, reducing the burden of context switching.

What we built:

There are tools like cookiecutter and backstage that allow users to create new services or libraries from templates. Unfortunately, they don’t support modifying or updating existing projects and we’ve found that it is adding new functionality to existing services and migrating services to new standards that end up consuming the bulk of our developer’s time. For example, when we moved our CI/CD provider from CircleCI to GitHub Actions, it was invaluable to be able to swap out the configurations for these pipelines in each service with a single action.

This led us to create our own rendering tool with a CLI interface that can be invoked from any existing microservice. It allows developers to not only start a new service or library but also add datastores like DynamoDB, connectivity like new HTTP routers, gRPC servers, and Kafka producers and consumers, and update common elements of a service like CI/CD in place. We used a Python module along with Jinja templates which can be invoked from the makefile. This makes for a seamless experience when modifying Python code but we’d consider using something like a docker container for polyglot development.

Lessons we learned:

Plan for the ability to modify services from the start, not just initialize them.

Migrations are much more challenging given that they must operate on an ever-increasing variety of input services. With this in mind, we built a few principles into our template from the start that make this more tractable. First, we separate generated and hand-written code as much as possible so that the files we are editing have predictable structures. Sometimes this takes the form of generated files and modules that developers don’t edit such as the entry point for our service which controls the lifespans of the service’s components like gRPC servers and kafka consumers.

Other times, we collect handwritten configuration into a predictable place within a file such as defining the much of the configuration for our CI/CD pipeline in environment variables:

For files that contain little hand-written code, it may be reasonable to entirely replace the file and expect developers to roll back any destructive edits with their Version Control System.

Second, we invested the time to build file editing primitives and utilities which allow us to assert project and file structure and make targeted changes. For example, we have a helper function to add a new container to a deployment, which builds on a helper for appending to YAML arrays, which in turn builds on a helper to insert into a file at a particular location. These generic helpers are made available to specific file types though inheritance.

This has paid dividends as our templates grew by eliminating duplicate logic and replacing opaque Regular Expressions with clearly named methods. However, even with these tools, migrations remain more challenging to write and reason about than initializing new services and libraries. We are exploring ways to harness coding assistants powered by large language models to update services in a way that is more declarative than current migrations.

Make it simple to maintain templates

Writing and reviewing code templates can be a challenge because the template commands obscure the logic and prevent automatic IDE linting from catching errors. The first step in simplifying maintenance is to avoid adding control flow like{% if ... %} or{% for ... %} within templates when they can be replaced by utility functions that are more easily tested.

We also recommend generating template outputs for many of the common permutations of inputs. For example, our template libraries tests generate the expected output for a new service that has a DynamoDB table, a gRPC server, and a Kafka consumer then update a copy of that output checked into the repo. This means that every change that occurs in the template is mirrored by one or more changes to the generated output.

We’ve found it is often far easier to spot issues during PR review in the expected output than the templates themselves. Additionally, these generated services also contain unit tests of their own, which are run as a part of the template library's Continuous Integration tests. This ensures that the generated code is valid, which is particularly critical for interpreted languages like Python.

Proactively seek feedback from your internal customers

While a project like this might arise from a Hack Day or passion project, templates need a clear, owning team with a product mindset to thrive. This team should publish release notes and/or demo new functionality internally to inspire teams to keep their services up to date. They also need to be responsive to feedback and bug reports as tools like templates are easy to bypass or developers often find it easier to correct a mistake in the generated code rather than fix it upstream. Finally, we’ve found it necessary to balance being responsive to requests for new technologies and features with the burden of maintaining the templates. We recommend developing your own “Rule of Three” to determine what makes sense to create a template.

Investing in a microservices template is more than a development strategy; it's a commitment to efficiency, cohesion, and continuous improvement. By investing in a template-driven approach, teams can mitigate some of the complexities of microservices architecture, ensuring that their applications are not only quick to iterate but also maintainable and scalable in the long run.

Thank you to the all members of Shippo’s Developer Experience Team that have contributed to this work: Mike Lueders, Beth Richardson, Mitch Ackermann, and Manoj Pahuja.