Machine Learning in Customer Experience

Tell us the story behind the Machine Learning team at NETCONOMY. How is it set up and how do you operate?

It’s the year of 2021 and customer expectations are only growing. Customers demand tailored, personalized experiences – or they will abandon the cart very quickly. Either they will get quick and reliable customer service, or simply won’t ever pick you over your competitor again.

A modern customer is well aware of all the data processed with all the digital interactions – and therefore also expects you to utilize that data and provide a seamless experience. With so much data being processed, we’re seeing a push for the adoption of Machine Learning (ML) approaches and techniques to provide additional value which is otherwise so hard to provide.

As a strategic advisor of our customers, we at NETCONOMY also see the economic perspective of ML-empowered solutions. We believe that new technologies and concepts require dedicated resources that will focus on growing that specific field and that’s why we formed a Machine Learning team.

One of our Machine Learning experts, Vid Jelen, explains what the team focuses on and what ML means for our customers.

Machine Learning is both a buzzword, as well as a “hot” field with huge potential. There is no doubt that ML concepts can play a major role in developing solutions and platforms that manage and improve customer experience. But let’s mention some specific cases. For instance, how can organizations leverage ML concepts to enable a better recommendation system?

When it comes to customer experience, there are many reasons and ways organizations can leverage the potential of ML. Recommendation systems are for sure one of them. Trends currently show that personalization as a feature is picking up the pace and starting to matter more than ever. Let’s say you are looking for a certain piece of clothing. If I am a returning customer, there might already be some data patterns about my preferences and behavior available from previous interactions with the shop. ML-empowered solutions can help brands recognize those patterns in buying a certain product, particular color, or size. Based on those, the frontend can be adapted to personalize the experience and provide best-fitting products for me, without having to search for them manually. As the ML solution provider, we can prioritize a certain product, brand, or style in search results as well. Or place them on top of the homepage. The same goes for complimentary items and up-selling. Customer data helps in building a personalized solution and also enables continuous improvements over time. This, of course, is taking customer consent into consideration, and we make sure we comply with data regulations such as GDPR and would provide this experience to people who willingly opt-in for it.

Image recognition is another popular case when it comes to Machine Learning-empowered solutions. Why is that?

Browsing a shop with images instead of text through visual similarity searches is one of the most popular topics right now. The area of ML which enables this is called “computer vision” and it has developed many ways of extracting information from image and video data. On a high level – we, human beings, understand the unknown objects we have not seen before if they are similar to an object we already know. This does not hold true for computers, – they have to be explicitly programmed to do something; but with the techniques of computer vision, we can “teach” the computers to recognize similarities between the objects they have never seen before. For instance, people understand that a shoe is very similar to a sandal (both belong to footwear) and this is also a concept we can teach computers by creating “machine learning models”. They house the extracted high-level representations of the objects we want them to recognize. Such a model can predict – if you show it a picture of a sandal and a picture of a hat, that the sandal is part of footwear, whereas the hat is most likely not. At the moment this is a very popular use case, not only because of the visual comparison component but also because it enables text-free shopping. Or – let’s say you are looking for a specific electronic device. Maybe the brand is visible, but you don’t really know the exact product code or name. Normally, you would have to type all of that information – which takes so much time. With visual comparison, you can simply take a picture and find out the price, availability, the full specifications of the item or instead find one that is most closely related. This basically means you can browse the entire shop simply by taking a picture. In this way, we simplify and improve the overall customer experience – saving time, giving more opportunities, and offloading the customer from having to enter unnecessary information.

Tell us the story behind the Machine Learning team at NETCONOMY. How is it set up and how do you operate?

In general, ML-oriented companies these days are very multifaceted. They have data product teams and also various other data-oriented teams. These companies cover a multitude of Machine Learning domains and are either experienced, long-time runners in the ML space (like Google), or have more recently pivoted to this area with various levels of experience, and mostly focus on particular subdomains while building up their ML capabilities. Our team is part of NETCONOMY efforts to expand the ML capabilities we brought from academic into real-world, at this stage focusing on ML projects for retail. On a high level, market growth projections for Machine Learning are exponential. Needless to say, the share ML is getting not only in e-commerce but also in other fields, is growing very fast. Our team had strong management support from the start and began building up internal capacities in early 2019. We are a very small and heavily research-oriented team, which means we work on technologies that still need to be developed before they can be used in projects, both in terms of know-how and the ways to implement them in an efficient way. Our jargon and technical terms are very specific, so they still sound foreign to many people, even within the company.) Machine Learning is advancing very fast, and we need to constantly learn and iterate on new ideas, while often completely discarding the old ones. But it is also a challenge our team likes to tackle as we expand our know-how and introduce new solutions that can bring benefits to our customers.

Another interesting aspect of our work is not only that the new tools are emerging, but also new, unique problems appear – in one year, we might be solving problems we don’t even know we have today. Therefore, it is clear that we want to be a step ahead, hence we are heavily investing in training, education, and experimentation.

We also had to increase the speed with which our team could iterate on ML solutions, so we invested into an on-premise purposely-built server dedicated to ML compute workloads.

ML computations include lots of linear algebra – matrix additions, multiplications, etc. These are relatively simple processes but need to be performed over and over many times. Luckily, they can be done in parallel, so we can speed them up if we use parallel processing. This means lots of CPU cores along with RAM to house that data and for ML we can also use graphic cards, which can do millions of such operations in parallel. This drastically improves the speed of our ML workloads.

You’ve already worked on many cases that include Machine Learning-empowered solutions. Can you mention some other interesting cases?

Gladly! The field itself is huge and there are loads of technologies we’ve experimented with and there are many possibilities to implement them in projects related to customer experience. Visual similarity search was one of the use cases – already mentioned above. Another one is video content analysis – you can detect individuals and products in a video, do video summarization on a big library of videos and also detect particular scenes. Let’s say there is a commercial about certain food products. A person opens the fridge, and several products are shown. With the use of ML, you can detect these scenes and products quickly, without going through the whole video again.

Another interesting project was object detection with Microsoft HoloLens. We used the HoloLens video stream to superimpose bounding boxes of objects we could recognize with a ML model. We also worked with chatbot solutions – they provide a possibility to automate customer support and decrease much of the manual work that has to be performed there. Customers really appreciate a responsive and helpful support channel as part of their experience on a webshop.

Natural language processing (NLP) is also a quickly evolving and interesting field of ML. We’ve experimented with full-stack support solutions containing NLP-powered components. Let’s say a customer buys something and gets a receipt at the shop. This receipt could be photographed, analyzed with NLP methods, and immediately be available as data in the shop for additional benefits, e.g. for the loyalty program system. In this case, the customer would only have to take a photo of the receipt and upload it to a personal webshop account. Loyalty points would be awarded automatically, and the information could be used for creating personalized offers and discounts later on.

NETCONOMY Machine Learning Team

Last but not least – we want a personal opinion. What is it like to work in this team? How do you see things developing in the future?

There are technology-related challenges, of course – since this is a field that is changing so much. But when you start early enough and have the management support, all of that can be easily solved.

When it comes to our team – we are still a small one, in a field that is growing so fast. We kind of have this “startup mentality”, where we are encouraged to experiment and iterate fast, which might feel overwhelming at times, but is also extremely rewarding when complex implementations take on life. I also like to consider another very common term we have in parallel with a startup – growth. Over time the ideas get bigger and bolder, the customers offer more compelling and challenging problems to solve and we feel more confident in delivering more complex and innovative solutions. The future looks bright and will also be exciting – I am sure of that!