For better device profiling, delete your DDR!

Bol.com features a fully responsive online store that adjusts its look and feel to the type and capabilities of the device and browser you’re using. This enhances usability, speed and conversion. But let me guess, that’s not news to you. Don’t worry, the rest of this post will be.

To be able to adjust the information on the page itself (in the backend, in Java at bol.com) we need to know the device type server side. This then allows us, for instance, to avoid featuring certain functionality on smaller screens, decrease the amount of data served, and increase the user-perceived performance. There is an official name for this pattern: Responsive with Server Side extensions (RESS). Server side we only need a very low granular device type, like “TABLET”, “SMARTPHONE”, “DESKTOP”, et cetera.

Use the User Agent string, Luke

At the back end all we can use is the information from the HTTPS request. To be more specific: the information we can use is found in the so-called User Agent (UA). Here is an example of such UA:

Mozilla/5.0 (iPhone;CPU iPhone OS 10_0 like Mac OS X) AppleWebKit/602.1.38 (KHTML, like Gecko) Version/10.0 Mobile/14A5297c Safari/602.1

The standard solution for this task is the implementation of a repository-based engine. The application server queries the Device Description Repository (DDR) to find out what the type of the device is. Then, based upon the specific device, appropriate web content is returned to the customer and rendered on their device.

We found Apache’s freshly incubated deviceMap, an F/OSS device detection solution, tested it and used it in our production environment. The developer involved (Volkan Yazici) became an official contributor to the project.

We lived happily ever after

Well, maybe not.

After some time, we started to notice downsides: omissions and errors in the DDR decreased usability for some of our customers (e.g. they were served desktop content on their smartphones); the DDR was hardly being updated; the number of backers of this product seemed to decline rapidly; and finally, the project was abandoned officially.

As always change turned out to be the only constant in IT development. DDR’s weren’t working. We needed something else.

Rethink device profiling!

So we took a different approach. We decided to focus on what we did know.

All UA’s from our customers over a recent period are available on our big data platform. This makes it quite easy to handle the large number of different ones. My colleague IT Architect Niels Basjes (who specializes in big data) looked for patterns in the collection of UA’s. From these patterns he built a set of rules that interpret certain parts of the UA and conclude what device type should be associated. Like this:

This User Agent:

Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87

Leads to this classification:

User Agent Classification Table

After structuring the UA to a set of clearly defined fields and values, the rule engine steps in to interpret them and conclude this device is a “DESKTOP”. Spot on.

Extending the set of rules, we found that it did not take us too long to come to a point where the rule-based engine outperformed the repository-based one, even for new devices (initially unknown to DDR’s). In fact, most device types can now be found using this method.

What’s new?

• No more DDR-matching
• A declarative grammar to match UA tokens with a particular ranking scheme
• Smaller footprint
• Higher yield
• Faster device type recognition
• Easier data management

The code for our Yet Another User Agent Analyzer (YAUAA) is available from GitHub, for you to study and reuse.

Wrap-up

We updated the service within our service-oriented architecture and performed a running deploy. And the new model has been up and running ever since.

So, can you imagine deleting your DDR?