Developer Guide

Tutorial: how to add videos with transparency to Augmented Reality with CraftAR

icon Date Developer Guide

icon Date Dec 14, 2020

icon author Catchoom Team

icon folder , , ,

In this article we explain how to add videos with transparency to your AR experiences in CraftAR.

The article discusses the following sections:

Transparency in videos for Augmented Reality purposes

Videos with transparency (also known as videos with alpha channel) refer to videos or animations that have a partial or total see-through background, which allows the user to see what is behind the video.

Transparency is a cool effect when it comes to Augmented Reality experiences, since you can better integrate the video or animation with the image being augmented. Here’s an example of what it may look like:

How to format your transparent video

In order to support transparent videos across mobile platforms (Android and iOS), our AR SDKs require the videos to be produced in a particular way. You need to duplicate the video frames, where the upper half is the original frame of the video, while the lower half contains a greyscale version of the frame to represent the transparent areas. This results in a video that has twice the height of the original video.

An example showing the duplicated video frames, corresponding to the transparent video experience shown above:

The CraftAR Augmented Reality SDKs will take care of using the lower half of each frame in the video as a mask for the upper half of the same frame and display the video with transparency, with the correct height of the original video.

If you want to use transparent videos in your Augmented Reality experiences, you have different options. You can add video content with a transparency layer to your CraftAR items inside our web-based editor. Or, you can manipulate your items programmatically. We will now explain the different options.

Option 1: Use the Creator to add transparent videos manually

In order to publish your Augmented Reality video contents with transparency, you need to follow the steps below:

  1. Produce a video with transparency mask, like the example mentioned in the previous section. This link gives hints on how to do it in Adobe After Effects, but it can be done with any other video editing tool as well.
  2. Upload your video to your server or CDN and make it accessible from the Internet.
  3. Create an AR Item in your collection in the CraftAR Creator with a reference image of your choice, then add the video to the scene by clicking on the “Video” icon. Tick the checkbox “Transparent video” to indicate that the video has transparency.

  4. Once you have added the video to the media library and selected it as a content used for your AR item, you can edit the video’s position in the AR scene. When editing the scene, the CraftAR Creator already displays the video in its original height.
  5. When you are done, publish your experience.

The AR SDKs will display your content with the transparency as expected.

Option 2: Add transparent videos programmatically

Transparent videos using the Management API

If you have a process to create AR items from your backend and you want to add transparent video content, you can also indicate that the videos have transparency through the Management API by modifying the AR contents.

The AR contents are stored in CraftAR as a JSON in a specific format. For each type of content (image, video, 3d models…) there are several attributes that you can utilize in order to modify how the contents are presented in the AR experience.

In case of videos, you need to set the “transparency_mask” attribute to true in order to indicate that the video contains a transparency layer.

Check out the documentation on AR contents and the JSON format for more information. Also you can review the Management API documentation to see how to upload or modify your AR items from your backend.

Programmatically adding transparent videos in the AR SDKs

In some cases, you may want to add the AR contents programmatically from your app’s code using the SDKs. In order to indicate in the AR SDKs that a video content has a transparency layer, you just need to set the “transparency_mask”.

Note on applications in Cordova

Adding videos with transparency programmatically is not available for Cordova yet. But videos with transparency will still work for contents loaded from the CraftAR Service, either uploaded through the Management API or created with the CraftAR Creator.

Note on applications in Unity

The Unity Augmented Reality SDK does not load contents from the CraftAR Creator. The AR contents are Unity GameObjects and have to be uploaded to CraftAR through Unity as AssetBundles. If you want to use videos in apps created with Unity, you’ll need a 3rd party plugin.

Related Posts in Developer Guide