SWAPs: From monolith to small web applications
Web applications usually start small and loveable, but gradually become larger and larger as more people work on them and front-end technologies move on. If you’re not careful they can potentially end up as the one thing no-one wants to touch anymore and cause a lot of frustrations for agile teams and business innovators alike.
In this blog, I would like to give insight into the development challenges we encountered while working on our self-service tooling and how we solved these challenges by introducing smaller web applications.
The challenge
Most web applications and the services they provide to their users, are important channels for business processes. The most well-known web application within bol.com is of course our webshop, where customers can buy products online. A less well-known web application is the portal that we supply to our partners; other retailers who sell their products through the bol.com platform. As you can imagine these retailers need quite a lot of tools and also data to manage their shop within bol.com.
For years we were organized in a way that a couple of agile teams specifically worked on this portal together. This created many dependencies between these teams, and deployments of one specific functionality required everyone to get on board. As the amount of bol.com partners continued to increase, the business demanded more and more functionality to be introduced in the portal and at a higher pace. The overhead created by aligning all the teams working on different parts of the portal started to take its toll as projects took more time to complete and initial estimations of simple functionality increased.
Still, that is not the most important problem we tried to solve by changing our web application architecture. Bol.com itself is both a retailer and a platform. This means that as a company we share many business processes with our retailing partners. At the same rate at which we introduce new functionality for our partners to manage their shop, our internal employees feel the need for new tooling as well. In this case, we would very much like to ‘eat our own dogfood’ and know exactly how our partners and suppliers feel about the tools we’re offering them.
Figure 1: The bol.com sourcing process and self-service environments we offer to our users and customers
To summarize, we saw 3 major challenges ahead:
- Team autonomy was limited - Because everyone worked together in one large monolithic web application there we’re many dependencies to manage. While other (back-end) teams moved towards DevOps, the portal teams still had to align with each other and all stakeholder for simple releases.
- Business innovation speed was impaired - All the technical dependencies between teams and stakeholders impacted the speed with which new functionality could be introduced. Seemingly small changes required a lot of planning and more time than warranted.
- Duplicate functionality - Other user groups, which have their own portal and environment, share many of the processes with our retailing partners. This would mean that we had to implement the same functionality a couple of times and have to duplicate code across teams.
Thinking of a solution we were inspired by the microservice architecture which is already in place at bol.com for years. We asked ourselves these questions:
- ‘What if web applications could be small and easily changed?’
- ‘What if portal functionality could be used by all our user groups, regardless of the environment?’
- ‘What do we need to create the best user experience in our web applications but also work and innovate independently?’
Introducing: the SWAP
Thus we introduced Small Web Applications (aka SWAPs). SWAPs are just like any other web application, except that they are very small and work together with other SWAPs to create a user flow. So instead of having one large application which offers a lot of functionality, the same functionality is offered by many small specialized and independently scalable applications.
Figure 2: Our journey to team autonomy from Monolith to SWAPs.
SWAPs have a back-end and a front-end: The back-end deals with getting data from and to the microservice landscape in a secure, normalized and aggregated way while the front-end deals with presenting the data and/or functionality in an interactive way to the user.
Every SWAP is the same in that they share common boiler plate code to implement SSO and authorization in the back-end. Their back-ends are based on our standard microservice framework (called AXLE), therefore providing all the basic application plumbing so teams have easy access to things like development tooling, monitoring, logging and metrics.
We decided to choose a shared UX design between all our tooling environments, so it’s easy to re-purpose SWAPs for different user groups. At the same time having separate environments means that we would still be in control to deploy new functionality one user group at a time.
We chose AngularJS as our shared front-end framework because it allowed us to re-use front-end components from a component library which is shared between teams. In theory, SWAPs are technology agnostic just like microservices, but re-using components makes it easier to create a consistent user experience.
This is all great in theory, but what about reality?
It’s now almost 2 years from when we first had this idea. Together with a group of people that shared enthusiasm and commitment we started small with 1 and then 2 SWAPs in a portal environment.
The part which took the most time was getting SSO (Single Sign On) in place for all our portal environments using OAuth2. This was an absolute requirement since it would make it possible to switch between SWAPs without having to log in every time you navigated to the next page. For a smooth integration with the existing monolithic web application, this was essential.
The other challenging part was getting support from our system operations teams for getting new (virtual) hardware every time we wanted to create a new SWAP in one of our 3 portal environments. Especially since for every environment different availability and SLA requirements were set, alignment with operations was crucial. Together with our SRT (Service Reliability Team) we created a process to make it very straightforward to request hardware for new SWAPs. In the end, naming every separate SWAP is now the hardest part about requesting new hardware.
Figure 3: Deploying a single SWAP to 3 different environment to re-use code and standardize functionality between user groups.
At first the teams really had to get used to the idea and way of working, but after a while it became easier and easier to create new SWAPs. Although a bit reluctant at first, most teams were very happy after creating their first SWAP because it gave them independence and autonomy. Teams that didn’t wish to add to the existing monolith could now instead just create a SWAP for their functionality and thus cut themselves loose from being dependent on other teams and stakeholders.
When things started to take off and more and more teams added SWAPs, we needed a way to keep visual alignment even though working independently. Together with UX and Front-end engineers from all the teams who owned SWAPs, we created a virtual team to guard the shared front-end component library and things like the Menu which would be visible in every SWAP. This wasn’t always easy because of the way we are organized. Work inside of the team always gets priority above other work and thus the success of this cross-functional teams all hangs on dedication and persistence of the developers and UX analysts themselves. Luckily at bol.com we have plenty of people ready to stand up and take ownership when it is necessary.
Summary
To summarize SWAPs are a big success at bol.com. We currently have 20+ (and more being created as I write this) different SWAPs owned by 8 teams across 6 different business domains. At least half of these SWAPs are available to multiple user groups across different environments, re-using functionality and allowing us ‘to eat our own dogfood’. SWAPs give teams autonomy and allow them to add new functionality in their own pace. In addition to accelerating business innovation capacity, SWAPs made it possible to move away from an existing monolith without the pain of migration, because it could be done one SWAP at a time. SWAP architecture also brought UX and development teams closer together and forces us to be consistent in user interaction design and look & feel, which greatly improved the overall usability of our tooling.
Donuts image used in the feature is by Ferry Sitompul