Adding AR Overlays
In the previous section we covered how to vizualize the scan process using the BarcodeBatchBasicOverlay
. In this section we will cover how to add custom AR overlays to your MatrixScan application.
There are two ways to add custom AR overlays to your application:
- Using the
BarcodeBatchAdvancedOverlay
class, our ready-to-use implementation for view-based AR overlays. - Provide your own fully custom implementation by using the
SDCBarcodeBatchListener.barcodeBatch:didUpdate:frameData:
function to retrieve the tracking information and implement your own AR overlay.
The first option is the easiest and recommended approach for most applications. It covers adding, removing, and animating the overlay’s views whenever needed and is also flexible enough to cover the majority of use cases.
Using BarcodeBatchAdvancedOverlay
The advanced overlay combined with its listener offers an easy way of adding augmentations to your SDCDataCaptureView
. In this example we'll add a view above each barcode showing its content.
First, create a new instance of SDCBarcodeBatchAdvancedOverlay
and add it to your SDCDataCaptureView
:
let overlay = BarcodeBatchAdvancedOverlay(barcodeBatch: barcodeBatch, for: captureView)
There are two ways to proceed from here:
- Add a
SDCBarcodeBatchAdvancedOverlayDelegate
to the overlay. - Use the
setters
in theoverlay
to specify the view, anchor, and offset for each barcode.
The second way will take priority over the first one, meaning that if a view for a barcode has been set using SDCBarcodeBatchAdvancedOverlay.setView:forTrackedBarcode:
, the function SDCBarcodeBatchAdvancedOverlayDelegate.barcodeBatchAdvancedOverlay:viewForTrackedBarcode:
won’t be invoked for that specific barcode.
Using the Delegate
For this option, keep in mind that:
- You need to conform to
SDCBarcodeBatchAdvancedOverlayDelegate
. This protocol’s methods are invoked every time a barcode is newly tracked. SDCBarcodeBatchAdvancedOverlayDelegate.barcodeBatchAdvancedOverlay:viewForTrackedBarcode:
asks for a view to animate on top of the barcode. Returningnil
will show no view.SDCBarcodeBatchAdvancedOverlayDelegate.barcodeBatchAdvancedOverlay:anchorForTrackedBarcode:
asks how to anchor the view to the barcode throughSDCAnchor
. Be aware that it anchors the view’s center to the anchor point. To achieve anchoring the top of the view or the bottom you will have to set an offset.SDCBarcodeBatchAdvancedOverlayDelegate.barcodeBatchAdvancedOverlay:offsetForTrackedBarcode:
asks for an offset that is applied on the already anchored view. This offset is expressed through aSDCPointWithUnit
.
func barcodeBatchAdvancedOverlay(_ overlay: BarcodeBatchAdvancedOverlay,
viewFor trackedBarcode: TrackedBarcode) -> UIView? {
// Create and return the view you want to show for this tracked barcode. You can also return nil, to have no view for this barcode.
let label = UILabel()
label.text = trackedBarcode.barcode.data
label.backgroundColor = .white
label.sizeToFit()
return label
}
func barcodeBatchAdvancedOverlay(_ overlay: BarcodeBatchAdvancedOverlay,
anchorFor trackedBarcode: TrackedBarcode) -> Anchor {
// As we want the view to be above the barcode, we anchor the view's center to the top-center of the barcode quadrilateral.
// Use the function below to adjust the position of the view by providing an offset.
return .topCenter
}
func barcodeBatchAdvancedOverlay(_ overlay: BarcodeBatchAdvancedOverlay,
offsetFor trackedBarcode: TrackedBarcode) -> PointWithUnit {
// This is the offset that will be applied to the view.
// You can use .fraction to give a measure relative to the view itself, the sdk will take care of transforming this into pixel size.
// We now center horizontally and move up the view to make sure it's centered and above the barcode quadrilateral by half of the view's height.
return PointWithUnit(
x: FloatWithUnit(value: 0, unit: .fraction),
y: FloatWithUnit(value: -1, unit: .fraction)
)
}
Using the Setters
The function SDCBarcodeBatchListener.barcodeBatch:didUpdate:frameData:
gives you access to a session. This session object contains all added, updated, and removed tracked barcodes.
From there you can create the view you want to display, and then call:
SDCBarcodeBatchAdvancedOverlay.setView:forTrackedBarcode:
SDCBarcodeBatchAdvancedOverlay.setAnchor:forTrackedBarcode:
SDCBarcodeBatchAdvancedOverlay.setOffset:forTrackedBarcode:
func barcodeBatch(_ barcodeBatch: BarcodeBatch,
didUpdate session: BarcodeBatchSession,
frameData: FrameData) {
DispatchQueue.main.async {
for trackedBarcode in session.addedTrackedBarcodes {
let label = UILabel()
label.text = trackedBarcode.barcode.data
label.backgroundColor = .white
label.sizeToFit()
self.overlay.setView(label, for: trackedBarcode)
self.overlay.setAnchor(.topCenter, for: trackedBarcode)
let point = PointWithUnit(x: FloatWithUnit(value: 0, unit: .fraction),
y: FloatWithUnit(value: -1, unit: .fraction))
self.overlay.setOffset(point, for: trackedBarcode)
}
}
}
Using Custom Implementations
If you do not want to use the overlay, it is also possible to add augmented reality features based on the tracking identifier and the quadrilateral coordinates of every tracked barcode. When doing so, keep the following in mind:
- Set a
SDCBarcodeBatchListener
on the barcode tracking - In the
SDCBarcodeBatchListener.barcodeBatch:didUpdate:frameData:
function fetch the added and removed tracked barcodes - Create and show the views for the added barcodes
- Remove the views for the lost barcode
- Via
CADisplayLink
, add a method called 60fps whenSDCBarcodeBatch
is enabled. In this method, for eachSDCTrackedBarcode
on-screen, update the position based onSDCTrackedBarcode.location
. There is no need to animate the change of location, the change of position will happen frequently enough that the view will look animated. - The frame coordinates from
SDCTrackedBarcode.location
need to be mapped to view coordinates usingSDCDataCaptureView.viewQuadrilateralForFrameQuadrilateral:
.
func barcodeBatch(_ barcodeBatch: BarcodeBatch,
didUpdate session: BarcodeBatchSession,
frameData: FrameData) {
DispatchQueue.main.async {
for lostTrackIdentifier in session.removedTrackedBarcodes {
// You now know the identifier of the tracked barcode that has been lost. Usually here you would remove the views associated.
}
for trackedBarcode in session.addedTrackedBarcodes {
// Fixed identifier for the tracked barcode.
let trackingIdentifier = trackedBarcode.identifier
// Current location of the tracked barcode.
let location = trackedBarcode.location
let quadrilateral = self.captureView.viewQuadrilateral(forFrameQuadrilateral: location)
// You now know this new tracking's identifier and location. Usually here you would create and show the views.
}
}
}