Know what you deploy – a script using git log for listing all features that go live

On 19-03-2021
Category: Blog, Development

Know what you deploy – a script using git log for listing all features that go live

Introduction

For me, even after having pushed the deploy button many times, deploying to production keeps being dreadful. Accidentally pushing an unwanted feature that sneaked in somehow is one of the many things I am afraid of. In this blog post I show you how we reduced our deployment anxiety with a script using git log for listing all the features that go live after pushing that deploy-to-pro button.

In this blog post I first introduce you to the way we work and provide you with the necessary information to understand the essence of the script. After that, I briefly show you how to use the script. Because your way-of-working might be different to ours, I explain the script in more detail so you can modify it to accommodate  your specific situation. You don’t need to be an expert in shell scripting or git to understand this blog post, but some basic knowledge on these matters definitely helps.

Our way of working

We keep track of our feature requests in jira – a web-based tool to plan, track, and release software features. We create a jira ticket for each request or user story. We develop each feature on a git feature branch having a name that starts with the jira ticket number and a short description of the feature, e.g.  SEO-4734-report-diff-acc-prod. Any commits I push to that branch are prefixed with the jira ticket number followed by a sensible description of what the commit is about.

When I believe I have finished a feature and one of my team members approves my changes, I merge my feature branch into master. This triggers a Jenkins build that tags the application version and automatically pushes my changes to a test environment. On this environment we run automated integration and performance tests before going to production.

The script for listing all features that go live in action

Prerequisites

First off, your code should be stored in a git repository, but you probably already figured that out. In order to run the script on your local machine, you need a Linux/Unix like system with git and the following lightweight command line tools installed:

  • jq – a lightweight and flexible command-line json processor
  • the_silver_searcher – a code searching tool

Both tools can be installed with brew if you like:

So far for the boring part. The reason why you need these tools will become apparent in what follows. First, it is time to see the script in action.

Running the script for listing all features that go live

Before you run the script you need to be on master (or the production branch from which you deploy) and be sure to have fetched the latest changes:

After that, you’re ready to run the script from the directory where the script file resides in. Note that the script works without any command line arguments:

The script will then ask you for the version that is on production:

You probably don’t know what the ‘axle internal’ page is. No worries, I’ll explain that later. After you entered the latest version, the script will list all the differences between the application version that is currently running on acceptance and the one that is running in production. More specifically, the script outputs the git log, links to jira tickets and feature branches that were merged into master, for example:

Pretty cool, isn’t it? You don’t have to worry if all this seems magic to you – I’ll explain everything step by step in the following section.

Breaking down the script for listing all features that go live

Ah, you are still here? Good, that probably means you’re interested to know how the script works. So, let me break it down and explain it piece by piece.

Fetch latest version from acceptance

The script starts by fetching the latest tag from the application that runs on the acceptance environment. Here at bol.com we use an in-house framework for building REST applications: Axle. Each service built with Axle has an internal page showing useful information about the application, such as the git tag denoting the service’s version. So in case you wondered what this ‘Axle internal page’ is while reading the previous section – now you know.

You can access the Axle’s internal page by visiting a url that ends with /internal. This shows the application information in human readable format. Luckily, the Axle framework also provides an endpoint on /internal/info that returns the information as json. The script uses the curl command to obtain the application info:

So, no need to parse html, we only have to parse json – and that is where the command line tool jq comes in handy. We use it to get the version field out of the raw json response:

The -r flag ensures that quotes surrounding the version string are stripped off. If you like to know more about the jq command, this might be a good read.

 What if you don’t have an endpoint that provides the version number?
In that case you could always provide the version number as input to the script, of course. However, having an endpoint that does provide this information is still something I would recommend. If you use Spring Boot 2, you can find an easy way to expose this information through an endpoint using a maven plugin here.

Determine git tags

To determine the difference between the versions that run on acceptance and production, the script also needs to know the version of the latter. Recall that we need to provide this as input to the script.

Did you just ask yourself why we do not simply fetch the production version in the exact same way as we do for acceptance? Great, that means that you have a great automate-all-the-things mindset. The reason is that the info endpoint on production is behind a login. Of course, there are still ways to read the version automatically. I leave that as an exercise to the reader. 😉 For now, the script asks its user to enter it manually:

The script compares the application on production with the one on acceptance based on the tag. In our setup the Jenkins build tool tags our code automatically after each push to master. Tags consist of the application version prefixed with the service name. The script builds up the tag as follows:

Show the git log

Comparison is done based on git tags. Now the script knows which tags to compare, showing it to the user is to run the following git command:

The no-pager argument makes git output just text which we can search on. The git log tag1..tag2  command outputs all commits being done between the two tags on the current branch – master in our particular case.

The script needs the output of this command later on. Therefore, it stores the command’s output in a variable before executing and displaying its output to the user:

 What if you don’t use git tags?
No worries, the git log command also works with commit ids. You just have to replace git log tag1..tag2 with git log commit_id1..commit_id2.

Scan the git log for jira ticket numbers

Looking at the git log, you can see one thing that is a direct consequence of our way-of-working: we prefix each commit message with a jira ticket number starting with three up to nine alphanumerical characters, a dash and a digit code, for example: SEO-4734.

Recall that the script needs the silver search command line tool to run. The script uses this tool to search for ticket numbers in the git log:

Note that a jira ticket can appear more than once in the git log, because multiple commits share the same jira ticket number. To ensure the script displays each ticket number only once, the last expression ends with the uniq command.

If you like to learn more about the ag command, you could start here. Also note that we could have used an alternative tool for the parsing, for instance awk.

Finally, the script echos each jira ticket number to the console:

Scan the git log for feature branch names

As explained before, we develop features on a feature branch. Once reviewed and approved, we push it to master. As a convention, we name our branches starting with the jira ticket number followed by a short description of the feature in kebab case. That is, dashes separate the words of the description. So, extracting and displaying the feature branch names that were merged into master from the git log is very similar to extracting the jira ticket number:

Putting it all together

That’s it. I have explained all the steps that comprise this script for listing all features that will go live. It helps me deploy to production with great(er) confidence. Hope it does the same to you. Now it is up to you. Let me put it all together for you, so you only have to copy paste and modify: