Skip to main content
Version: 7.0.1

Adding AR Overlays

In the previous section we covered how to vizualize the scan process using the BarcodeBatchBasicOverlay. In this section we will cover how to add custom AR overlays to your MatrixScan application.

There are two ways to add custom AR overlays to your application:

The first option is the easiest and recommended approach for most applications. It covers adding, removing, and animating the overlay’s views whenever needed and is also flexible enough to cover the majority of use cases.

Using BarcodeBatchAdvancedOverlay​

As mentioned above, the advanced overlay combined with its listener offers an easy way of adding augmentations to your DataCaptureView. In this guide we will add a view above each barcode showing its content.

First of all, create a new instance of BarcodeBatchAdvancedOverlay and add it to the DataCaptureView.

const overlay = BarcodeBatchAdvancedOverlay.withBarcodeBatchForView(
	barcodeBatch,
	view
);

At this point, you have two options.

note

The second way will take priority over the first one, which means that if a view for a barcode has been set using BarcodeBatchAdvancedOverlay.setViewForTrackedBarcode(), the function BarcodeBatchAdvancedOverlayListener.viewForTrackedBarcode() won’t be invoked for that specific barcode.

Using BarcodeBatchAdvancedOverlayListener

// The component must be registered: `AppRegistry.registerComponent('ARView', () => ARView)` e.g. in index.js
class ARView extends BarcodeBatchAdvancedOverlayView {
render() {
return <Text style={{backgroundColor: 'white' }}>{this.props.barcodeData}</Text>
}
}

// ...

overlay.listener = {
viewForTrackedBarcode: (overlay, trackedBarcode) => {
// Create and return the view you want to show for this tracked barcode. You can also return null, to have no view
for this barcode.
return new ARView({barcodeData: trackedBarcode.barcode.data});
},

anchorForTrackedBarcode: (overlay, trackedBarcode) => {
// As we want the view to be above the barcode, we anchor the view's center to the top-center of the barcode
quadrilateral.
// Use the function 'offsetForTrackedBarcode' below to adjust the position of the view by providing an offset.
return Anchor.TopCenter;
},

offsetForTrackedBarcode: (overlay, trackedBarcode) => {
// This is the offset that will be applied to the view.
// You can use .Fraction to give a measure relative to the view itself, the SDK will take care of transforming this
into pixel size.
// We now center horizontally and move up the view to make sure it's centered and above the barcode quadrilateral by
half of the view's height.
return new PointWithUnit(
new NumberWithUnit(0, MeasureUnit.Fraction),
new NumberWithUnit(-1, MeasureUnit.Fraction),
);
},
};

Using the setters in the overlay

The function BarcodeBatchListener.didUpdateSession() gives you access to a session, which contains all added, updated and removed tracked barcodes. From here you can create the view you want to display, and then call BarcodeBatchAdvancedOverlay.setViewForTrackedBarcode(), BarcodeBatchAdvancedOverlay.setAnchorForTrackedBarcode() and BarcodeBatchAdvancedOverlay.setOffsetForTrackedBarcode()

didUpdateSession: (barcodeBatch, session) => {
	session.addedTrackedBarcodes.map((trackedBarcode) => {
		let trackedBarcodeView = new ARView({
			barcodeData: trackedBarcode.barcode.data,
		});

		this.overlay.setViewForTrackedBarcode(trackedBarcodeView, trackedBarcode);
		this.overlay.setAnchorForTrackedBarcode(Anchor.TopCenter, trackedBarcode);
		this.overlay.setOffsetForTrackedBarcode(
			new PointWithUnit(
				new NumberWithUnit(0, MeasureUnit.Fraction),
				new NumberWithUnit(-1, MeasureUnit.Fraction)
			),
			trackedBarcode
		);
	});
};

Provide your own custom implementation​

If you do not want to use the overlay, it is also possible to add augmented reality features based on the tracking identifier and the quadrilateral coordinates that every tracked barcode has. Below are some pointers.

note

The frame coordinates from TrackedBarcode.location need to be mapped to view coordinates, using DataCaptureView.viewQuadrilateralForFrameQuadrilateral().

didUpdateSession: (barcodeBatch, session) => {
	session.removedTrackedBarcodes.map((lostTrackIdentifier) => {
		// You now know the identifier of the tracked barcode that has been lost.
		// Usually here you would remove the views associated.
	});

	session.addedTrackedBarcodes.map((trackedBarcode) => {
		// Fixed identifier for the tracked barcode.
		const trackingIdentifier = trackedBarcode.identifier;

		// Current location of the tracked barcode.
		const location = trackedBarcode.location;
		view
			.viewQuadrilateralForFrameQuadrilateral(location)
			.then((quadrilateral) => {
				// You now know the location of the tracked barcode.
			});
	});
};