--
Infrastructure-as-code is a proven and robust approach for managing distributed services in the cloud. It is mostly used in a cloud infrastructure context, but is also very applicable to pretty much any API-First SaaS service.
In this article we explain what value an open source Terraform provider brings to SaaS platforms and why we believe that it is a great way to improve the developer experience, that any SaaS API company should pursue.
We hope this encourages SaaS and MACH platforms that don’t offer Terraform support yet, to consider starting with it. Because not only the platform users will benefit from it but the larger ecosystem as well, especially when looking at it from a standardisation & interoperability perspective.
Lab Digital is part of the MACH Alliance and is a tech agency focussed on building cloud native digital commerce platforms. For years we’ve been building developer tools such as Terraform providers and SDKs to improve the developer experience; it is our mission to make building high quality platforms as simple and enjoyable as possible.
Cloud infrastructure platforms such as Amazon Web Services have existed for over 15 years (since 2006) and have changed the world in under a decade.
In order to manage resources in the cloud, in 2010, AWS introduced AWS CloudFormation, one of the first implementations of ‘infrastructure-as-code’; a tool that allows you to manage API configuration in declarative code, which in turn allows software and system engineers to employ industry-standard software engineering practises.
Since then, this created a whole new industry with HashiCorp’s Terraform (released in 2014) leading the charge. Terraform does not only supports AWS, but practically any cloud platform (thousands today). So it quickly became a default tool to manage resources in different clouds and spun up an ecosystem of tools that you can use to work with Terraform.
Today using infrastructure-as-code for managing resources in the cloud is the default; not using it in production settings is considered a bad practise and not suitable for production workloads. And for good reason! Using it brings a great number of benefits when working with any web service in the cloud.
Increased consistency in managing resources leads to a shorter time to market and decreases risk during releases, also providing an audit trail for compliance.
Standard SDLC (Software Delivery Life Cycle) practices such as CI/CD and Version Control can be used to manage cloud service configuration throughout its lifecycle
Using a Terraform provider prevents the need for end-users to build custom scripts, reducing the risk of manual, brittle processes
Configuration drift between environments can be eliminated, ensuring consistency across all environments
2. Increased standardisation reduces the development time spend on configuration and maintenance of resources which results in a decreased Total Costs of Ownership (TCO)
Terraform knowledge and experience is widely available and can be easily incorporated into development teams
Terraform configuration is human-readable and easily understandable, serving as documentation for your project
A large ecosystem of tools is available to support Terraform configuration management, including CI/CD tools and Terraform-CDK
3. Easier integration reduces the development time spend on integrating new resources into the existing architecture
Setting up new environments is easier and faster, increasing productivity for developers working in the cloud
Terraform can be used to manage any cloud or SaaS service, and can connect different services together using webhooks and other integrations, providing a single language/interface for managing all services.
4. Increased re-usability reduces the development time spend on creating new resources
Terraform configuration modules are reusable and easy to copy/paste from examples, reducing development time and increasing consistency
So overall, using infrastructure-as-code and Terraform brings huge improvements in terms of productivity, stability and robustness in managing the configuration of Cloud services that you use in your system. The business benefits are obvious, who doesn't want that?
API-First SaaS services almost always have configuration APIs that allow you to influence how the API behaves. Things like configuring data models, creating projects and environments, managing integrations through webhooks, etc. All of that is managed through a management API when using MACH.
What SaaS companies often offer to make this easier, is a management SDK in your preferred language, or for example a CLI application that makes it easier to perform those management actions against the SaaS platform.
While this is definitely usable, applying those tools often results in a ‘bespoke’ set of one-off scripts that is brittle and hard to maintain; using a management SDK leads to writing custom code that needs to be maintained, and using a CLI application leads to custom bash scripts that may be invoked through for example a CI/CD pipeline.
As you might have read earlier on our blog, we didn't find this approach acceptable when we started working with the commercetools API, which led to the creation of our Terraform provider for commercetools. We were kind of spoiled by having been Terraform users in the cloud space for many years, so not having that available was a couple of steps back.
Having this provider eliminated the use of custom management scripts through SDKs and streamlined our CI/CD pipelines by being able to leverage the robust ecosystem that terraform provides. No more bash scripts that invoke a management CLI.
While we are convinced that this approach is much better than leveraging SDKs and CLI applications, we don’t see much adoption across SaaS vendors to provide Terraform providers themselves. This is a missed opportunity. But perhaps some debunking needs to take place; infrastructure-as-code does seem focussed on ‘just’ infrastructure, doesn't it?
Historically, SaaS services were implemented as web-based versions of their desktop equivalents. So primarily a user interface (like Office 365) to use as an end user. And any configuration in a SaaS service like that, was done through the UI as well. APIs were not a default thing for a long time. And definitely not APIs that would cover all of the SaaS platforms functionality.
In the last years however, we’ve seen a huge adoption of the ‘API First’ movement in the SaaS (business applications) space. And because of that, APIs built like this are as mature and stable as we are used to from Cloud platforms.
Naturally, this makes the SaaS API infinitely more usable to build anything against it. And very importantly, it allows you to perform any configuration/management action against it, via an API.
Having that management functionality available through an API (and having mature/robust APIs available in a SaaS product), is the key to unlocking the Terraform Provider capabilities for any given SaaS API: as long as management and configuration actions are ultimately available via an HTTP API, it is possible to build Terraform provider support for it.
So ultimately, there isn't much difference between an API that’s build in a SaaS product, or available in a cloud suite. It is simply software, delivered as an API. Just like any cloud service in AWS, Azure, GCP or any of the other platforms.
So from Terraforms perspective, it doesn't really matter what happens behinds the scenes of an API — whether it is spinning up infrastructure, or configuring a data model in an e-commerce application. It is all the same.
When working with a SaaS API, most of the time the common method to manage resources in it, is to write custom scripts to create the resources you need. And if you want to have the script manage the resource over time, it should also include logic to update or delete existing resources.
When managing the SaaS API with Terraform, you can rely on standard Terraform mechanics for many of the operations that you need to perform over the course of a configuration’s lifecycle. Below we show what the difference is between the two approaches!
The below example shows how you can create webhooks in a real-world SAAS, using its TypeScript SDK. It also includes logic to update or delete an existing webhook, which, as you can see, you need to write yourself.
import { WebhooksApi, Webhook } from '@saas/saas-client';// Set up the API clientconst client = new WebhooksApi();client.basePath = 'https://youraccount.saas.api';// Define the webhook requestconst webhookRequest = { url: 'https://yourwebhookurl.com', method: 'POST', headers: { 'Content-Type': 'application/json', }, body: '{"example": "data"}', disabled: false, events: ['coupon.created', 'coupon.updated', 'coupon.deleted'],};// Define a function to create or update the webhookasync function createOrUpdateWebhook() { // Check if the webhook already exists const existingWebhooks = await client.getWebhooks(); const existingWebhook = existingWebhooks.find(w => w.url === webhookRequest.url); if (existingWebhook) { // Update the existing webhook existingWebhook.events.push('campaign.created'); const updatedWebhook = await client.updateWebhook(existingWebhook.id, existingWebhook); console.log(`Webhook ${updatedWebhook.id} updated`); } else { // Create a new webhook const newWebhook = await client.createWebhook(webhookRequest); console.log(`Webhook ${newWebhook.id} created`); }}// Delete the webhookasync function deleteWebhook(webhook: Webhook) { await client.deleteWebhook(webhook.id); console.log(`Webhook ${webhook.id} deleted`);}// Run the example(async () => { try { await createOrUpdateWebhook(); // Uncomment this line to test webhook deletion // await deleteWebhook(existingWebhook); } catch (e) { console.error(e); }})();
This example shows how you can use the SAAS's Typescript SDK to create, update, and delete a webhook. As you can see, this is quite some code and is also completely custom-built — so depending on the developer, the script might be structured differently.
Of course there are possibilities to use a CLI framework for this to allow for i.e. arguments, but you would then effectively rebuild Terraform :-). Furthermore, imagine having to provision resources in multiple cloud and SaaS platforms, that even might to be aware of each other… That will become a mess!
Here's an example of how managing a SAAS API webhook using a custom Terraform provider would look like, if one would exist.
provider "saas" { api_key = "yourapikey" account_id = "youraccountid"}resource "saas_webhook" "example_webhook" { url = "https://yourwebhookurl.com" method = "POST" headers = { "Content-Type" = "application/json" } body = jsonencode({ example = "data" }) disabled = false events = [ "coupon.created", "coupon.updated", "coupon.deleted", ]}
This Terraform code defines a new webhook resource in the given SAAS API. The resource includes the same properties as in the Typescript SDK example, such as the webhook URL, method, headers, body, and disabled status.
With this Terraform code, you can use the standard Terraform workflow to create, update, and delete webhooks in the API, effectively managing the resource over its entire lifecycle. This also provides the benefits of Infrastructure-as-Code, such as versioning, testing, and auditing.
Executing this configuration would be a matter of invoking terraform apply. And if resources from other cloud and SaaS services would be part of the configuration, Terraform will pick these up in one go!
As you can see, managing the resource with Terraform is much simpler than creating custom scripts for this yourself, as you can rely on Terraforms standard workflow and its declarative nature. And furthermore, you’ll have a single approach to configuration management across all of your cloud and SaaS platforms.
The only thing needed, is for the SAAS API vendor to provide a Terraform provider that implements managing their API resources in Terraform.
For us this would be the ultimate goal. Being able to build and configure large scale digital commerce systems, spanning many cloud, SaaS and bespoke services, through a single configuration management approach: Terraform (and its ecosystem).
It is not difficult to envision the benefits that has from an ‘integrator’ perspective. It would greatly mature the way we operate and build modern platforms and allow for far better reusability, integration and reusable methods/approaches when building these systems.
Building MACH, Composable and distributed systems is hard. And it is much harder if there are no common methods for an area like configuration management. This is something we need to work on as a community, as the amount of integrated services will only increase.
The increasingly higher maturity of the SaaS-API-First movement unlocks a lot of tools and practises that have been available in the cloud infrastructure space for years. SaaS vendors and particularly their power users, can greatly benefit from that.
A Terraform provider primarily makes working with the configuration side of your API a joy; it will save a lot of time initially but also over time, and makes the process robust and scalable for any team that uses it.
It removes many of the pains that users have, resulting in great efficiency gains and proper software engineering methods around configuration management. And it integrates with a wide ecosystem, including many other cloud and SaaS platforms out there.
This is a big opportunity for SaaS vendors for improving the developer experience around their product. Having the breadth of HashiCorps ecosystem available is very powerful and will take the management around your product to another level.
When we look at it from a MACH ecosystem perspective, Terraform has the potential to become a common approach for managing these systems at scale — something that we feel is desperately needed for our ecosystem.
We can imagine! Having a Terraform provider for configuration makes the lives of your power users infinitely better! ;-)
If you don’t know how to build one, there are some great resources to get started. Primarily familiarising yourself with the Golang ecosystem and walking through available tutorials should get you started quite quickly.
However, in case you or your team don’t have time for that, at Lab Digital we are able to help! We’ve implemented a number of Terraform providers for several SaaS platforms and would love to help you improve the developer experience of your platform. So feel free to reach out!