Skip to main content

Adding AR Overlays

There are two ways to add advanced AR overlays to a Data Capture View:

note
  • The first way is the easiest, as it takes care of adding, removing and animating the overlay’s views whenever needed. It’s also flexible enough to cover the majority of use cases.
  • You can always handle touch events on the views you create like you normally would.

Using BarcodeTrackingAdvancedOverlay​

As mentioned above, the advanced overlay combined with its listener offers an easy way of adding augmentations to your DataCaptureView. In this guide we will add a view above each barcode showing its content.

First of all, create a new instance of BarcodeTrackingAdvancedOverlay and add it to the DataCaptureView.

BarcodeTrackingAdvancedOverlay overlay = BarcodeTrackingAdvancedOverlay.Create(barcodeTracking, dataCaptureView);

At this point, you have two options.

note

The second way will take priority over the first one, which means that if a view for a barcode has been set using BarcodeTrackingAdvancedOverlay.SetViewForTrackedBarcode(), the function IBarcodeTrackingAdvancedOverlayListener.ViewForTrackedBarcode() won’t be invoked for that specific barcode.

Using BarcodeTrackingAdvancedOverlayListener​

public View ViewForTrackedBarcode(BarcodeTrackingAdvancedOverlay overlay, TrackedBarcode trackedBarcode)
{
// Create and return the view you want to show for this tracked barcode. You can also return null, to have no view for this barcode.
UITextView textView = new UITextView(new CGRect(0, 0, 200, 50));
textView.BackgroundColor = UIColor.White;
textView.Text = trackedBarcode.Barcode.Data;
return textView;
}

public Anchor AnchorForTrackedBarcode(
BarcodeTrackingAdvancedOverlay overlay,
TrackedBarcode trackedBarcode)
{
// As we want the view to be above the barcode, we anchor the view's center to the top-center of the barcode quadrilateral.
// Use the function 'OffsetForTrackedBarcode' below to adjust the position of the view by providing an offset.
return Anchor.TopCenter;
}

public PointWithUnit OffsetForTrackedBarcode(
BarcodeTrackingAdvancedOverlay overlay,
TrackedBarcode trackedBarcode)
{
// This is the offset that will be applied to the view.
// You can use MeasureUnit.Fraction to give a measure relative to the view itself, the sdk will take care of transforming this into pixel size.
// We now center horizontally and move up the view to make sure it's centered and above the barcode quadrilateral by half of the view's height.
return new PointWithUnit(
new FloatWithUnit(0f, MeasureUnit.Fraction),
new FloatWithUnit(-1f, MeasureUnit.Fraction));
}

Using the Setters​

The function IBarcodeTrackingListener.OnSessionUpdated() gives you access to a session, which contains all added, updated and removed tracked barcodes. From here you can create the view you want to display, and then call BarcodeTrackingAdvancedOverlay.SetViewForTrackedBarcode(), BarcodeTrackingAdvancedOverlay.SetAnchorForTrackedBarcode() and BarcodeTrackingAdvancedOverlay.SetOffsetForTrackedBarcode()

public void OnSessionUpdated(BarcodeTracking barcodeTracking, BarcodeTrackingSession session, IFrameData frameData)
{
// Be careful, this method is not necessarily invoked on the main thread!
DispatchQueue.MainQueue.DispatchAsync(() =>
{
foreach (TrackedBarcode trackedBarcode in session.AddedTrackedBarcodes)
{
UITextView textView = new UITextView(new CGRect(0, 0, 200, 50));
textView.BackgroundColor = UIColor.White;
textView.Text = trackedBarcode.Barcode.Data;
overlay.SetViewForTrackedBarcode(trackedBarcode, textView);
overlay.SetAnchorForTrackedBarcode(trackedBarcode, Anchor.TopCenter);
overlay.SetOffsetForTrackedBarcode(
trackedBarcode,
new PointWithUnit(
new FloatWithUnit(0f, MeasureUnit.Fraction),
new FloatWithUnit(-1f, MeasureUnit.Fraction)
)
);
}
});
// Dispose the frame when you have finished processing it. If the frame is not properly disposed,
// different issues could arise, e.g. a frozen, non-responsive, or "severely stuttering" video feed.
frameData.Dispose();
}

Provide Custom Implementation​

If you do not want to use the overlay, it is also possible to add augmented reality features based on the tracking identifier and the quadrilateral coordinates that every tracked barcode has. Below are some pointers.

Please note that there is no need to animate the change of location, the change of position will happen frequently enough that the view will look as it is animated.

note

The frame coordinates from TrackedBarcode.Location need to be mapped to view coordinates, using DataCaptureView.MapFrameQuadrilateralToView().

public void OnSessionUpdated(BarcodeTracking mode, BarcodeTrackingSession session, IFrameData frameData)
{
// Be careful, this function is not invoked on the main thread!
DispatchQueue.MainQueue.DispatchAsync(() =>
{
foreach (int lostTrackIdentifier in session.RemovedTrackedBarcodes) {
// You now know the identifier of the tracked barcode that has been lost. Usually here you would remove the views associated.
}

foreach (TrackedBarcode trackedBarcode in session.AddedTrackedBarcodes)
{
// Fixed identifier for the tracked barcode.
int trackingIdentifier = trackedBarcode.Identifier;

// Current location of the tracked barcode.
Quadrilateral location = trackedBarcode.Location;
Quadrilateral quadrilateral = dataCaptureView.MapFrameQuadrilateralToView(location);

// You now know this new tracking's identifier and location. Usually here you would create and show the views.
}
});
// Dispose the frame when you have finished processing it. If the frame is not properly disposed,
// different issues could arise, e.g. a frozen, non-responsive, or "severely stuttering" video feed.
frameData.Dispose();
}