Best Practices

How to choose and optimize reference images for image recognition

icon Date Best Practices

icon Date Dec 18, 2020

icon author Catchoom Team

icon folder , , ,

A well-working Image Recognition experience starts with selecting the right targets – in Catchoom’s terminology, the right reference images – for your experiences. Here are some tips to help you prepare the right collection of images for our Image Recognition solutions.

How to recognize everyday objects

One of Catchoom’s specialties is that our recognition engine not only works great on flat objects, like photos or print images, but it works very well on ‘non-planar’, in other words, non-flat objects, too.

Such non-flat objects may be bottle labels, cans, bottles, other consumer packaged goods, etc. As long as they have a distinctive pattern, they can be recognized. (Read our guidance regarding what kind of images and objects can be recognized well, and which don’t.)

With Craftar, companies can let their users interact with tangible, everyday objects and link all sorts of surprising digital experiences to them. For example, scanning a bottle of wine to see reviews or check out alternatives in the online store.

What are ‘reference images’ and why do they matter?

You need to represent your objects –your items – via ‘reference images’ in order to make them recognizable. In the Craftar platform, you can group those items into so called ‘collections’.

Then, you can link those reference images to specific events, such as redirecting the user to a web page, or other content like a video link. Finally, APIs help synchronize these collections with your mobile and web apps, being able to keep your database up-to-date.

Even if Catchoom’s recognition seemed more accurate than the market average in tests, even in difficult conditions, good reference images are crucial to boost the performance and keep your users happy.

Let’s see some recommendations to help you prepare the perfect collection of images for your experiences.

Recommendations for preparing your collections of reference images

Include various sides to let users scan more naturally

In most cases, one reference image is enough, as our engine is invariant to size and rotation of the actual real-life object, and it can handle different angles to certain extent, too.

If you want to get a photo or print advertisement recognized in a magazine or print catalog, it’s enough to use the original jpeg or png image file. Even in cases of non-flat objects, say, cans or bottles, one reference image of their representative frontal face of the label tends to give good results on their own.

However, if you want to use physical objects that have different sides, such as a book, a box of cereals or other kind of consumer packaged goods, it’s wise to upload corresponding shots besides frontal image(s).

Reference images uploaded for the same item, a makeup box.
Top row: frontal images of the two most characteristic sides of the box.
Bottom row: aiding the recognition by adding additional reference images of the most typical viewpoints,
considering that this is how many of the users may see the box in real life.

Adding reference images from different sides and typical viewpoints helps the users successfully interact with the objects while minimizing the instructions given to them.

Use the original image files whenever possible.

We generally recommend to use reference images that predominantly show the actual image or object, focusing on its distinctive pattern.

Our technology is so reliable that as long as irrelevant elements of the background do not take up more than 20% of the reference image, the recognition will still work in real life. Nevertheless, it’s better to rule this problem out.

If you need to take a photo of the object, don’t forget to crop it.

Sometimes you cannot avoid taking a photo of the object to use it as a reference image.

For example, you need to take pictures of a bottle of wine to make it searchable in your online catalog. In this case, if you don’t have the original image file, it’s advised to crop the photo you take to focus on the label.


Crop the photo around the label if you need to take a picture of the CPG object.

Don’t use blurred images.

Blurry reference images negatively affect the performance of real-life recognition. If you need to take a photo of the object, pay attention to the right focus.

Similarly, if you use digital images, don’t scale it up excessively – as it might get distorted – and don’t add ‘noise’. Read more about the recommended reference image quality, dimensions and formats.

Look up best practices for your specific use cases.

While there are some general guidelines, as explained above, it is important to do your homework for your specific project.

Here you can find some further recommendations, tailored to common use cases:

Check your reference images with Craftar’s Image Quality Rating.

The Image Recognition item view inside our Craftar platform includes an orientative 5-star rating. (It is also available via our CraftAR Management API.) The score is based on whether the image has a strong visual pattern and the presence of such a pattern in the entire image. Low scores indicate few features or large textureless areas.

Left: this reference image reached a high score, as it has a distinctive pattern all over the image area.
Right: this reference image was given a lower quality value, because the distinctive pattern covers only a small area when compared to the patternless grey margin. This is why we mentioned the importance of cropping.

This reference image, showing a piece of wooden floor, has very little visual pattern.
Therefore, its rating is very low, as it’s a poor candidate for recognition.

Note that the rating does not take the repetitiveness of the pattern into account, which may also negatively affect the recognition performance. Hence, the rating feature mainly serves to rule out reference images that will surely not work well. It’s generally wise to eventually double check the recognition on the actual physical object, too, making sure that it will deliver the experience the way it is expected.

Getting rid of images that don’t work well will help you save time and effort and make your job more efficient, besides getting the most value out of our solutions.

Disclaimer: we do not have official affiliations with most of the brands and artworks presented in the article. We meant to use them for editorial purposes as examples of the possibilities of the technology. Image copyright belongs to the brands.

Related Posts in Best Practices