Software Testing at bol.com
Testing has been in an accelerated evolution over the last decade. Due to the fact that different practices — like Continuous Integration and Delivery, DevOps, and Agile — have shifted and refined at a rapid pace.
In this blog, we would like to give a short insight on testing at bol.com, in order to create a foundation for future Techlab blogs about testing.
Testing through the years
Software development at bol.com began with 10 developers and 1 tester when deployment to production happened about once every month.
Nowadays, bol.com has more than 500 services which are being cared for by around 60 scrum teams. They all have very different characteristics, like being front-end, back-end, or big data-focused services. Hence, we use different approaches for testing; a front-end team can have a totally different approach and set of tools than a back-end team for example.
Evolution
Generally, we see testing efforts are shifting to being performed earlier in the development lifecycle, shifting towards isolated environments rather than on shared environments. This helps to release small chunks of functionality to production fast.
Teams have more and more responsibility for their services, actively assessing their quality by monitoring them on production. Testing is also an essential part of the responsibility, by assessing risks at the right place. It has become an integrated part of the software development lifecycle.
Teams have become more autonomous by setting up integration tests in dockerized environments, and more tests are integrated into the build pipeline to facilitate early feedback. Moreover, testing has become a democratized effort, rather than an effort conducted by a single person guarding the testing activity.
We are working on setting up a set of test principles, to help guide us with further improvements in the future.
Challenges
Of course, we have a number of testing challenges, like slow and unreliable system integration tests in a shared environment, due to external dependencies. We are experimenting with a Consumer Driven Contracts approach, by letting the supplier team have copies of your contract tests so that they can run them as part of their build pipeline.
Flaky front-end testing in a shared environment is another challenge. We are experimenting with cypress, a rather new and evolving front-end testing tool. We expect its new features and the possibility to mock other services in the build, to help us reduce flakiness, and enable more independent service deployments.
I have been working at bol.com about 3 years, and I’m happy to be a part of this testing evolution. I especially like the diversity in testing approaches, and the freedom to use different tools.
This blog obviously doesn’t cover all aspects of testing at bol.com. There are more interesting topics to share like contract testing, front-end testing in isolation with cypress, and performance testing. We will write about these topics in upcoming blogs, so stay tuned!