Visual Search – The Future of Retail

We live in a digital economy – and a visual one at that.

65% of the population consists of visual learners. This means that more than half of us absorb and retain information better if it’s conveyed through a picture or video. That’s because text-based information is often vague and hard to grasp. It’s no wonder, then, that images and videos are quickly turning into our primary form of communication. Emojis, memes or GIFs have replaced traditional, text-based communication in messaging apps and on social media. Posting or sending an image is often faster. And in most cases, it better communicates what we are trying to say. Words often fail us, especially in fashion. It’s difficult to put the style of a product into words. 

Our visual economy

This trend towards a visual, digital economy could explain the meteoric rise and steady popularity of visual social media. The user numbers on Pinterest and Instagram are constantly growing. And the companies themselves are at the forefront of this visual revolution. Pinterest recently raised $150m in funding to further focus on visual search after launching the function earlier this year. They are currently valued at $12.3 billion. 

shop social media images
Social media and the future of visual search.

According to Mary Meeker’s Internet Trends 2017 report, this is just the beginning of visual communication. Consider these insights from the report:

  1. Internet usage is way up and is going to continue that way
  2. The lines between ads, content, and products is blurring, and fast
  3.  eCommerce is growing at a rate of +15%, year over year

We can draw several conclusions from this information:

  1. Shoppers want distributed commerce – the ability to shop whenever, wherever
  2. Growing usage of visual social media = the language of images is universal
  3. Shopping will almost exclusively be online in the near future
visual search future of retail ecommerce pinterest
We agree with Pinterest CEO Ben Silbermann: Visual Search is the future of eCommerce.

Fashion, at its core, is visual

Digital and print fashion media spend a considerable amount of time describing products with words. But for most people, the product only becomes tangible through an accompanying visual.

Shoppers aren’t inspired by words. When they click through a blog or scroll through Instagram, it’s the image of the pink suede sneakers or the black silk top that catches their eye.

So it only makes sense that when shopping for apparel – which is also the fastest growing sector in eCommerce – that the communication between retailer and customer is also image-based. That is, with visual search.

Why the future of search is visual

Here is a common scenario for the modern consumer: Shopper A sees a picture of Rihanna on holidays on her Instagram feed. In this picture, she’s wearing a white V-neck jumpsuit. Shopper A loves Rihanna and her style, so she wants to buy the same jumpsuit. She visits her favorite online retailer and types “white jumpsuit” into the search bar. She scrolls through the results, but they aren’t good. Same thing on a second site. When she clicks through the third site, Shopper A is frustrated and no longer in the mood to shop. The result: no jumpsuit for Shopper A, and no sale for the retailers.

This would never have happened with an image search. Search by image services are basically like typing “white minimalistic jumpsuit that Rihanna wore in Portofino” into the search bar, but actually getting viable results.

Visual search is what modern shoppers expect. They already enjoy many conveniences made possible by the latest technological innovations. Visual search and recommendation software is the logical next step. 

Visual search is made possible by machine learning

Machine learning (ML), specifically deep learning, make image recognition and visual search possible.

Basically, visual search is the localization of products within an image and the placement of those products within the greater context of a shop catalogue. To achieve this, we must first build a fashion universe, also known as a feature space. This is a collection of all the products sold by a brand or retailer. The fashion universe has one simple feature: products of a similar style are close together, products of different styles are far apart.

The Fashwell fashion universe.
The Fashwell fashion universe.

The fashion universe is created by a neural network that is inspired by the structure of our brain. It consists of many different layers of neurons , each with a specific function. As they comb through the images, the neurons light up once they hit upon a specific visual feature.

fashion deep learning image recognition
Localization and Retrieval.

Which leads us to the second part of visual search: The input image is analyzed. The products are localized. Then, they’re mapped within the feature space. In our example, we input the image of Rihanna, the products – jumpsuit, sunglasses, sandals – are localized. They are then grouped with similar products in the feature space of, say, Zalando.

In order to achieve highly accurate image data analysis results, the machine requires massives amounts of training data. It must learn from other fashion images not only what the difference between a dress and a jumpsuit is, but also between two jumpsuits, like a catsuit or a romper.

How visual search works in eCommerce

An image search service, like that offered by Fashwell, can easily be integrated into an eCommerce site or app through an API.

Standard visual search works like this: Shopper A uploads an image to a specific section on the retailer’s website or app. The image is scanned and data points are extracted. The neural network identifies a white jumpsuit, but also a pair of sandals and sunglasses. Immediately, many similar jumpsuits pop up on the eCommerce site or app. Shopper A finds the item she was looking for without having to search manually. She even saves a lot of time in doing so. Impressed and satisfied, Shopper A places the jumpsuit (and the sunglasses, because they go so well with the outfit) in her shopping cart and checks out.

Of course all of this happens within seconds, and that’s exactly the point. The modern shopper doesn’t want to waste time by typing twenty different word combinations into a search engine. Scrolling through endless pages of products on multiple shopping platforms isn’t fun, either.

Image recognition providers such as Fashwell work with retailers to turn their eCommerce channels into the best possible shopping experience for their customers. As more storefronts are closing, it’s imperative for brands to stay ahead of the curve. Especially when it comes to providing the same standard of service that a customer would receive in-store.

Benefits of visual search for retailers

The cold hard facts:

According to Mary Meeker’s report, the modern shopper wants choice, easy discovery, personalization and a curated shopping experience – all at the same time. Search by image and shop by image functionalities speed up the shopping process from the moment of visual inspiration to checkout. Not only is searching with an image faster and easier, it also produces more accurate results. Additionally, visual search leads to product discovery: it recommends items that are similar to the search query or recommends products that complete the look. Next to an image recognition API, Fashwell also powers visual search chatbots that act like a personal shopper.

Case in point: Italian lingerie company Cosabella recently paired up with an AI-company to test personalized website designs for their customers. The marketing director immediately noted a 35% boost in sales. The visual search and recommendation feature at both Sketcher’s and Nieman Marcus’ shopping app also produced similar results; both companies expressed their satisfaction and continued use of visual search and recommendation in the future.

Visual search checks off all the expectations of modern shoppers: choice, ease, personalization and curation. Brands and eCommerce retailers who offer shop by image can count on increased conversions and revenue and, ultimately, customer loyalty.


We live in a visual, digital economy. Visual communication on social media and messaging apps are the new normal. Since fashion is inherently visual, it makes sense for all fashion-based communication, i.e. shopping, to also occur through an image. That is why search by image and shop by image is an integral part of the future of eCommerce and retail. It’s also what the modern shopper, who wants speedy and accurate results, is beginning to expect from brands. Image recognition software and the many eCommerce solutions it provides, ensures a faster, more accurate and more personalized shopping experience. This, in turn, increases revenue and brand loyalty for any retailer.

Visual search: it’s the future of eCommerce.

If you’re a brand, retailer or affiliate shopping network and you’re interested in using an image recognition provider, Fashwell is here to help. Find out more by getting in touch.