--
Today we are introducing the first version of the Terraform Provider for commercetools. This provider will allow us to apply Infrastructure-as-Code principles to commercetools deployments. This can greatly speed up deployments and make them more stable/robust at the same time.
With the Terraform Provider for commercetools, we are able to consistently manage multiple commercetools environments using Infrastructure-as-Code principles. As a digital agency, this allows us to be consistent across projects and clients, which makes us more efficient and allows for higher quality.
With this provider we are able to provision commercetools environments automatically for a large part. And even better: we can integrate them with different cloud providers in an elegant way. This is possible because commercetools offers powerful integration capabilities via their API Extensions and Subscriptions that allow you to configure synchronous and asynchronous hooks in the commercetools API (in our case for example AWS Lambda functions and SQS queues).
For example, in below (pseudo) example we configure commercetools to publish messages to an AWS SQS queue. When running below code, both AWS and commercetools will be setup and configured to talk to each other. Pretty powerful, if you consider you can use Azure or Google cloud as well. Hooking to multiple clouds in a single configuration would be just as easy!
provider "aws" { region = "eu-west-1"}provider "commercetools" { client_id = "<your client id>" client_secret = "<your client secret>" project_key = "<your project key>"}resource "aws_sqs_queue" "examplequeue" { name = "examplequeue" delay_seconds = 90 max_message_size = 2048 message_retention_seconds = 86400 receive_wait_time_seconds = 10}# IAM permissions omitted for readabilityresource "commercetools_subscription" "my-sqs-subscription" { key = "my-subscription" destination { type = "SQS" queue_url = "${aws_sqs_queue.examplequeue.id}" access_key = "${aws_iam_access_key.ct.id}" access_secret = "${aws_iam_access_key.ct.secret}" region = "eu-west-1" } changes { resource_type_ids = ["product"] } message { resource_type_id = "product" types = ["ProductPublished", "ProductCreated"] }}
When we started investigating commercetools as a technology, we immediately started thinking about how to embrace the technology as an agency — how to deploy it in a robust way that is in line with our other technologies, in a way that can be consistent across our clients.
Being a heavy Terraform user for about three years, we quickly decided using it for commercetools was the best way. We’ve used this approach for our AWS infrastructures for years, and are really happy with it. The only problem was: commercetools did not offer a Terraform provider (or a Go SDK, for that matter).
So we started to build a proof of concept to test our approach, aiming at building it ourselves. Within a day we had a working program, so this was quite successful. After showing the demo to the commercetools team it turned out that having a Terraform provider had been on the product team’s wish list for a while. Therefore the commercetools team invited us for a joint Hackathon in Berlin, to work on the provider together.
This hackathon took place last week — the Lab Digital team has been working on the provider for about a month and together with the commercetools team we polished the provider and SDK to make it releasable. It was great to work with them to get the provider launched!
Creating a Terraform provider was new for both our team as well as the commercetools team. It turned out to be a lot of fun, as it required us to learn a couple of new technologies and approaches. Among them learning a new programming language, Golang :-).
Terraform and its providers are written in Go. And as per Terraforms best practises, we tried to add as little code as possible to the provider, and leverage an Commercetools Go SDK for the heavy lifting. That SDK did not exist yet however. So we created it!
While learning the new language, creating the terraform provider and the SDK, we learnt that Go as a language really appeals to us. However, we did notice a lot of ‘copy pasting’, while creating both.
Matthias from commercetools, who works in the SDK team, introduced us to their vision for maintaining different SDK’s for different platforms, including documentation. This vision is largely built around building up all of those from an elegant single source of truth, a RAML specification for the API. This allows you to generate a lot of the clients and documentation automatically, which makes maintenance easier and results in less copy-pasting (for Golang at least).
commercetools is a next-generation software technology company that offers a true cloud commerce platform, providing the building blocks for the new digital commerce age.
So, commercetools is a new breed of e-commerce platform. And they just arrived in the Gartner Magic Quadrant for Digital Commerce as a visionary. In other words: next generation Digital Commerce aimed at large enterprises.
Having built e-commerce systems for years, we can honestly say we’re quite experienced in it. Lots of our projects have been done with Oscar Commerce, but also bespoke systems based on ‘just’ Django are common for us.
Most projects that we do involve some kind of online conversion and shopping component. Common theme in these projects is the bespoke-ness/complexity. Oscar Commerce and Django are great in supporting that in an elegant way.
In order to fulfil certain (larger) use-cases where a fully open source solution is not always the desired option, we decided to look at new technology that complements our existing portfolio of tools.
This is where commercetools comes in for us: commercetools is a Cloud Native solution that can be provisioned via web service API’s. And at the same time, it allows for the flexibility and extensibility that we are so used to with Oscar and Django. This fits perfectly with our experience with Cloud Native development, as well as Digital Commerce. And it allows for building a Terraform Provider for it, because it is API-First.
Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.
In other words: a tool for applying infrastructure-as-code practises, covering multiple (cloud) service providers. In our case we can use it to entangle commercetools with cloud providers such as Azure, AWS and Google.
Using infrastructure-as-code allows you to codify your infrastructure and perform rollouts automatically, making your infrastructure part of the common software development lifecycle. And furthermore, repeatable deployments become a possibility- this means you can setup additional environments really quick (as in hours instead of weeks).
Until now we have only used infrastructure-as-code practises and Terraform for provisioning actual cloud infrastructure, i.e. loadbalancers, virtual machines, networks, etc.
Using this approach to provision a user-facing application, in this case a Digital Commerce system, is new to us — and the market we believe.
The Headless/API-First trend that cloud vendors have been following for years allows for techniques like these. And as more and more companies are embracing it, we expect this will greatly speed up deployments of scalable, end-to-end infrastructures that span systems and use-cases. In an ideal world we are able to provision all of our required services using this method.
We expect a lot from our approach and hope that having open sourced the Provider and SDK will encourage commercetools developers and partners to contribute in making the Provider and SDK more robust and complete. We certainly are committed, but we definitely need the help from the community!
To get started with the modules or contributing to them, have a look at our GitHub repositories for both the Terraform Provider as well as the SDK. Or start with the documentation!