The closer you look, the more difficult it becomes

Eric Funk

7.2.2019

Translation: machine translated

Everyone knows that a functioning search function is what makes it possible to find your way around the Internet. There is also a search function on digitec and Galaxus that is used intensively.

With several hundred thousand hits per day, the search function is one of the most frequently used features on our website. We at Team Endeavour have been developing the search function for about a year now. Our aim is to provide you with convenient access to our content.

Of course, the most important thing is that the search leads you quickly to the right product from the countless available. On the other hand, we also want to show you product knowledge in the form of magazine and Community articles - so that you can inform yourself before you buy.

But how do you build a good search? What are the pitfalls? And what problems do we still need to solve? For those interested in technology, I've written this article that gives you an insight into our engineering.

Solid data is the basis

In order to be able to search something, a good database must first be created. Without it, there are no good results. This is easier said than done, as our search combines far more than just product data. Millions of pieces of content quickly come together here. As a user, you don't care, you just want to see suitable results clearly displayed in a single view. And quickly. To make this possible, we use the open source search technology Elasticsearch.

In recent months, we have put a lot of work into a new architecture in which almost all searchable content is sent to our search module in the Azure Cloud and stored in Elasticsearch. Both technologies together help us, among other things, to update price changes on the website in real time and to scale our software solution.

Scaling is becoming increasingly important as the amount of information is exploding. Since our team started working on search, the amount of data has already tripled and this process will accelerate as we expand abroad and extend our range.

Speed is everything

So much for clean data and our technological basis. The next and biggest challenge for a good search is the search query itself. It must not only show the relevant results but also deliver results quickly - really quickly. We're talking about an average of 150 milliseconds.

Every software developer knows that performance does not necessarily increase with the number of features. The complexity increases with the rules that are applied to the query. Example: Popular products should be displayed more prominently than rarely purchased products. The name should be weighted higher than a secondary feature of the product. And these are just two obvious cases.

As the number of rules for the search increases, so does the susceptibility to errors. Care must be taken when making changes so that unexpected results do not suddenly appear. Speaking of susceptibility to errors: we humans are not immune to them either. When typing, small errors creep in again and again, which should not lead to a deterioration in the search results.

We still have a lot to do

Sometimes we work on things that you can't even see. At most you can feel them. I'm talking about the remodelling of the results page - which still looks exactly the same after the remodelling. Under the bonnet, it now employs the frontend framework React. The live shopping in the online shop already uses React, the checkout and other pages will follow. We hope that the conversion will improve user flow and performance in the browser - especially on mobile devices. And, of course, a leaner code base in the long term.

At the moment, we at Team Endeavour are still in the process of mastering the complexity of the search. We are aware that we still have a lot of potential for improvement in search. One approach is machine learning (ML). Major search engines such as Google are already using machine learning to implement search suggestions, corrections and search trends. This year, we want to acquire basic knowledge of this new topic and implement an initial prototype.

Now you!

The search is an application that seems simple, but it's actually quite complex. The deeper you delve into the topic, the more complex and challenging it becomes. Are you interested in this? Or can you even support us with your knowledge of Elastic? It just so happens that we are currently looking for reinforcements. If you don't want to apply straight away - we also welcome praise and criticism.

Eric Funk

Teamleader Engineering

eric.funk@digitecgalaxus.ch

These articles might also interest you