--
In the previous post in this series, we’ve set the scene for what’s required for building a flexible global e-commerce platform that is capable of catering to the local market needs.
In this follow-up, we will talk about how we’ve built exactly that, using a combination of infrastucture-as-code, AWS, serverless microservices and a headless e-commerce solution called commercetools. Also, we will talk about how to scale that to lots of CBU’s in a maintainable way.
We use commercetools as our e-commerce platform. Its tenancy model allows for creating per-CBU instances with multiple stores, in multiple languages and currencies.
Leveraging microservices on AWS Lambda, and hooking these to the commercetools API, allow for both customisations of commercetools and gives us endless scalability. Partly because we outsource any persistence to commercetools itself.
By wrapping the above in a Terraform-based infrastructure-as-code package, we can blueprint and maintain the entire solution from that single infrastructure-as-code template, with a configurable and customisable instance of it per CBU.
An elegant and easy to integrate ReactJS based front-end UI Kit library allows us to augment existing touchpoints with e-commerce functionality, where these library components interact directly with the commercetools API.
At the core of it all is a relatively new player in the enterprise digital commerce space: commercetools. Although they only exist for about five years, commercetools is already mentioned in the well know analyst quadrants by Gartner and Forrester. And rightfully so, we believe!
What separates commercetools from ‘the rest’ is that commercetools is not a package that you buy and host yourself, but a cloud service (SAAS) that you subscribe to. Of course other Digital Commerce cloud services exist, such as Demandware or Shopify, however what makes commercetools different is that what you get is exclusively an API. It is API-Only, or so-called ‘Headless’.
What you get is exclusively an API
This means that any interaction with the platform, both for configuration of it as well as end-users, will go through a well-modelled API platform.
This makes it the ideal candidate for us: having all commerce logic separated from the user experience means that we can re-use and integrate it into any channel and don’t have to recreate it.
And on top of that, commercetools makes sure that the platform is scaled on-demand, is secured, and has the features one may expect of a mature e-commerce system.
commercetools is a scalable platform at the core. Their tenancy model is set-up around organisations with projects in it, that you can basically consider an ‘instance’ of the platform, deployed in a given region. Each project stands completely on its own and has to be configured individually.
A project receives configuration in terms of languages, tax settings, currency and shipping settings, as well as logistics & fulfilment channels.
Furthermore, recently commercetools introduced the stores concept, which allows you to define contexts within a project that connect products, orders and customers, to the given store context. This is perfect for creating brand stores.
User access can be given at the organisation and then project level, with permissions on a per-store basis.
This model allows us to create a project per CBU — as project settings are usually consistent for a single CBU. And then we can create one store for each brand (or more, if you want). Being an API, commercetools then allows you to integrate these store endpoints into any channel you want.
Or a more structured way of explaining it;
global platform: commercetools orgnisation country A: commercetools project brand A: store within a project channel A: touchpoint via API channel B: touchpoint via API brand B: store within a project channel A: touchpoint via API channel B: touchpoint via API country B: commercetools project brand A: store within a project channel A: touchpoint via API channel B: touchpoint via API brand B: store within a project channel A: touchpoint via API channel B: touchpoint via API
Now that we know the tenancy model for commercetools that allows us to scale up, we still need to solve the local tailoring requirements.
We want CBU instances to be as consistent as possible with the ‘core’ product, but treat potential deviations in a ‘1st class citizen’, thereby maintaining compatibility with our core.
The way that we’ve approached this is the usage of microservices for each customisation that we implement in the product. This means that a customisation stands on its own, and can be turned on or of, or be replaced by something else on a per-instance basis.
As an example, let’s take the ‘different fulfilment providers per country’ requirement.
Let’s say that country A wants to work with fulfilment company FULFIL, and country B uses SHIPIT. In our model, this results in two micro services:fulfilment-service-FULFIL and fulfilment-service-SHIPIT. Both services have the same responsibility, but they implement different fulfilment providers.
Now we can ‘simply’ attach both services to either the country A or country B commercetools environment, only replacing the individual component.
The same approach can be chosen for any type of business logic, such as integrating different (or multiple) payment providers on a per-CBU basis, or building different ERP integrations.
Attaching the mentioned microservices to commercetools is possible through the usage of API Extensions or Subscriptions within the commercetools API. This basically allows you to create synchronous and asynchronous hooks into to commercetools API, for specific events that happen at different stages of the request lifecycle.
This customisation model allows for tailoring the entire commercetools API to your needs, meaning you can implement any form of business logic yourself.
Read more about commercetools’ extension model in this blogpost that they published.
Microservices and serverless are an ideal fit. So that is why we chose for AWS Lambda to power all of the extensions that we’ve built. AWS Lambda scales horizontally on-demand along with the amount of traffic, meaning that you don’t have to worry about adding new servers, high availability, fail over, or anything. You just have to provide the code that has to be executed.
Also, AWS Lambda calls are charged to you on a per-request basis. Meaning that if you don’t have any traffic (such as on your test or QA environments), you are not charged for anything.
The result is that our solution is completely serverless, which means we don’t run any server or load balancing infrastructure ourselves. Everything is handled through AWS Lambda and AWS API Gateway. This makes the platform both very scalable in terms of traffic, very reliable in terms of availability and at the same time also very cost-efficient because it only runs the resources you need.
Many services that you build require some data to be persisted. Either some user data, or data that is required to make systems talk to each other. For example storing payment reference ID’s, to make sure an order can be connected to an actual payment made through the payment service provider.
Usually you create a database for this, that stores this data. And databases are traditionally hard to scale.
Luckily, the commercetools API has ample possibilities of defining custom types and objects, which allow you to store any data structure in their API. Also it is possible to customise existing Resources in the API, so that you can tailor them to your needs.
Leveraging these capabilities in the API takes away the need for us to manage any persistence, we rather outsource this complexity to commercetools, who will manage, scale and secure it for us!
Now that we’ve established how we build the platform and tailor it to our needs, the question arises: if you perform all of these customisations, how do you scale this to potentially 10’s or 100’s of commercetools projects, that are all roughly the same and need to stay aligned with the same core product?
This is where our open source Terraform Provider for commercetools comes in!
Leveraging Infrastructure-as-Code principles has been second-nature for our company since we started four years ago. So when we started using commercetools at our company, we immediately searched for a way to apply infrastructure-as-code in this context as well, which resulted in creating the Terraform Provider. Read more about the creation and open sourcing of this provider in our blogpost.
As it turns out, this provider has been key in scaling our global e-commerce platform. Because basically it allows us to create a codified blueprint of an end-to-end e-commerce platform, that will actually be completely functional after rolling out the blueprint.
This means we can create an infrastructure-as-code template that contains the following:
The commercetools configuration for a CBU, such as stores, languages, tax & currency settings.
The required AWS hosting infrastructure for a CBU, meaning API Gateway configuration, DNS & certificates, security & logging controls, etc.
All necessary microservices for an individual CBU, which contains both the service code itself, as well as required AWS resources and commercetools configuration, such as API Extension configuration (which events to hook to) and custom data types that might be needed for the service.
In other words, everything you need in an e-commerce platform!
Furthermore, through an abstraction layer that we’ve built on top of our Terraform Provider, we gained the capability of creating a configuration for managing a commercetools organisation, with per CBU a configuration that defines an individual instance setup. That organisation config then generates the required Terraform code that defines the entire organisation infrastructure and configuration, and rolls it out in a given AWS region. Something like the below (pseudo-code, simplified).
organisation-ACME: country-A: aws_region: eu-west-1 commercetools: project: country-a-prd currency: EUR stores: - brand-store-A - brand-store-B services: fulfilment-service: fulfilment-service-FULFIL payment-service: payment-service-STRIPE country-B: aws_region: us-east-1 commercetools: project: country-b-prd currency: USD stores: - brand-store-A - brand-store-B services: fulfilment-service: fulfilment-service-SHIPIT payment-service: payment-service-ADYEN
After the Terraform-defined infrastructure has been rolled out, a fully functional e-commerce API is available, including all necessary payment, logistic and back-end integrations, as well as customisations for a CBU.
This way, we can manage all of our instances from a single infrastructure-as-code foundation, and use configuration to tailor for CBU-needs. And because of how Terraform is built, this is also the way we can maintain and update any configuration in the platform automatically.
As mentioned earlier, in our global model we wanted to support local collaborations and support through the local IT department, as well as collaborations with local agencies and vendors.
As you have read, we setup an instance of our platform per CBU in a completely separate environment — or even an existing (AWS) environment. Therefore, managing the instance stands completely on its own and makes it possible, for example, to grant local administrative access to the systems.
Furthermore, as discussed to tailor the platform we use microservices. And because the entire system is distributed in nature, microservices can live anywhere — as long as they are reachable via HTTP and declare at what points they should be called by the commercetools API.
So, for the core platform we manage several microservices, but why wouldn’t it be possible that additional extensions can be implemented by a different party than us, and hooked to the platform through our infrastructure-as-code approach?
While the above-defined approach is very scalable and flexible for the back-end of our platform, this does not result in a usable service for the end-user, yet, as it still lacks a user interface.
As addressed in our previous blogpost, we aim at reusing existing touchpoints that our client already has. Existing websites usually have a product catalog in place, but no shopping functionality just yet. So we need to add those!
Our approach to embedding e-commerce in those sites was building a so-called UI Kit that contains all necessary e-commerce components as individual components, to be placed anywhere on a given web site.
We built the components using ReactJS and implemented Storybook as well, which allowed us to create an elegant UI Library. Combined with React-Habitat we were able to create an SDK that we can embed in any website, with just a couple of lines of code.
<div data-ecom-component="shopping-cart" /></div><div data-ecom-component="checkout" /></div><div data-ecom-component="add-to-cart" data-prop-sku="1235"></div><script src="https://cdn.oursite.net/commerce.js" data-prop-theme="brandX"></script>
The SDK will search for ‘shell’ components and replaces these with e-commerce functionality. These components, including a shopping cart, the checkout and add to cart buttons, will connect directly to the commercetools API in real time, from the front-end.
The UI Kit furthermore can be translated through the usage of gettext and convenient translation tool called Transifex. And via theming capabilities we are able to quickly implement the right branding for a given touchpoint.
Also, the UI Kit library allows for deviations per country, where for example the checkout in country A might require different information than the checkout in country B.
Using this front-end driven way of incorporating commerce functionality in any website has resulted in a solution that is elegantly decoupled and is easy to integrate. Just a couple of lines of code! The components themselves can be placed anywhere on a page, under full control of the ‘site owner’. Via parameterisation we allow for hooking on events (i.e. for analytics) or change the behaviour of the component. Also we allow for overriding appearance via simple CSS.
In the next part of this series we will talk about the impact our platform has on the business, in terms of time to market, ease of integration, the total cost of ownership and results of leveraging existing channels.
In the meantime, we invite you to have a look at our blog, our website, and the roles we are currently searching for.