Tutorial: Use Cloud Image Recognition on Cordova

Dec 22, 2020

Catchoom Team

Cordova, cordova cloud image recognition, cordova Image Recognition

This section applies to the following Plugins:

Augmented Reality Cordova Plugin v2.4+
Cloud Image Recognition Cordova Plugin v2.4+
On-device Image Recognition Cordova Plugin v2.4+
CraftAR Pro Cordova Plugin v2.4+

Are you still using an older version? Previous versions of the SDKs will not receive updates anymore. If you need help transitioning to the newer versions, you can get support through our support channel.

An Image Recognition app using the Cordova Plugin can be implemented in two steps. First, you need to set up a view with the camera capture and then you can run image recognition and parse the results for each item that is recognized.

Set up the SDK in your project

Once you have set up the CraftAR Cloud Image Recognition Plugin in your project, it’s time to implement the views for the experience.

The following is an example for an app that uses CraftAR’s Cloud Image Recognition.

1. Start the camera capture in a new view

First of all, make sure the cordova.js script is included in your index.html start page.

[crayon-616a4ad2a1b3c174275936 inline="true" ]&lt;script src="cordova.js"&gt;&lt;/script&gt;

1	[crayon-616a4ad2a1b3c174275936 inline="true" ]<script src="cordova.js"></script>

[/crayon]
After that, We need to start a new view. This view will allow the Plugin to open the camera in the background and display our UI as a transparent overlay.

document.addEventListener("deviceready", function () {
    CraftARSDK.startView(null, null, {"loadUrl" : "your_search_view.html"});
}, false);

document.addEventListener("deviceready", function () {

CraftARSDK.startView(null, null, {"loadUrl" : "your_search_view.html"});

}, false);

2. Start the video capture

To be able to access the Plugin JS interface, the capture page needs to import the cordova.js script. Then you can listen for the deviceReady event and use the SDK to start the camera capture.

[crayon-616a4ad2a1b45533200852 inline="true" ]&lt;script src="cordova.js"&gt;&lt;/script&gt;

1	[crayon-616a4ad2a1b45533200852 inline="true" ]<script src="cordova.js"></script>

[/crayon]
Once the device is ready, you will have to start the capture using CraftAR SDK:

document.addEventListener("deviceready", function () {
    startCraftAR();
}, false);

function startCraftAR() {
    CraftARSDK.onStartCapture(didStartCapture);
    CraftARSDK.startCapture();
}

document.addEventListener("deviceready", function () {

startCraftAR();

}, false);

function startCraftAR() {

CraftARSDK.onStartCapture(didStartCapture);

CraftARSDK.startCapture();

}

The SDK manages the Single shot and Finder Mode searches with the frames and pictures from the camera.

When video capture is started, the function passed to onStartCapture will be called. At this moment, it is also worth setting the token of the collection that you want to search through when performing Cloud Image Recognition. Once the collection is ready, you will be able to set callbacks to handle the search results and errors. On the other hand, you will have to set the search controller, which is in charge of performing searches using images coming from the camera (either from the Finder Mode or the Single shot search). Set this to the CraftARCloudRecognition.searchController to use the Cloud Image recognition.

function didStartCapture() {
    CraftARCloudRecognition.setCollectionWithToken(“your_collection_token", function () {
        CraftARSDK.searchController = CraftARCloudRecognition.searchController;
        CraftARCloudRecognition.onSearchResults(getResults);
        CraftARCloudRecognition.onSearchError(onError);
    }, function (error) {
        alert("Error:" + error.errorMessage);
    });
}

function didStartCapture() {

CraftARCloudRecognition.setCollectionWithToken(“your_collection_token", function () {

CraftARSDK.searchController = CraftARCloudRecognition.searchController;

CraftARCloudRecognition.onSearchResults(getResults);

CraftARCloudRecognition.onSearchError(onError);

}, function (error) {

alert("Error:" + error.errorMessage);

});

}

3. Implementing the Image Recognition request and parsing the results

Option A. Use Single Shot Mode to take a single picture

You can call singleShotSearch to perform a search with a single image. This method takes a photo with the camera, sends query to CraftAR’s Cloud Image Recognition and leaves the camera capture frozen

function singleShotSearch() {
    CraftARSDK.singleShotSearch();
}

function singleShotSearch() {

CraftARSDK.singleShotSearch();

}

When the search is done, the callback set through the onSearchResults method will be called with the list of results passed. The link will be empty if anything was found.

function getResults(results) {        
    if (results.length &gt; 0) {
        // Each result has one item
         var item = results[0].item;
    } else {
        alert("Nothing found!");
    }

    // Unfreeze the video capture that the single shot search freezes by restarting the capture
    CraftARSDK.camera.restartCapture();
}

function getResults(results) {

if (results.length > 0) {

// Each result has one item

var item = results[0].item;

} else {

alert("Nothing found!");

}

// Unfreeze the video capture that the single shot search freezes by restarting the capture

CraftARSDK.camera.restartCapture();

}

Option B. Use Finder Mode for continuous scanning

You can call startFinder to start searching continuously without user intervention. This method sends queries at a controllable rate to CraftAR’s Cloud Image Recognition. For every query, the response triggers the callback set through the onSearchResults method. You could for instance call the following function once the Collection token has been set:

function finderModeSearch() {
    CraftARSDK.startFinder();
}

function finderModeSearch() {

CraftARSDK.startFinder();

}

and then the implementation of didGetSearchResults should process the results.

function getResults(results) {
    if (results.length &gt; 0) {
        CraftARSDK.stopFinder();
        // Each result has one item
        var item = results[0].item;
    } else {
        alert("Nothing found!");
    }
}