Batch Scanning And AR

Get Started With MatrixScan

With MatrixScan, you can highlight and interact multiple barcodes within the same frame and build AR experiences. MatrixScan use cases are implemented through functionality provided by SDCBarcodeTracking.

In this guide you will learn step by step how to add SDCBarcodeTracking to your application. Roughly, the steps are:

• Include the ScanditBarcodeCapture framework and its dependencies to your project, if any.

• Create a new data capture context, initialized with your license key.

• Create a barcode tracking settings instance where you enable the barcode symbologies you want to read in your application.

• Create a new barcode tracking object and initialize it with the settings created above.

• Obtain a camera instance and set it as the frame source on the data capture context previously created.

• Create a new data capture view and add a basic overlay instance to it for visual feedback.

• Register an overlay listener and implement BrushForTrackedBarcode(), which is called whenever a new tracked barcode appears.

Prerequisites

Before starting with adding a capture mode, make sure that you have a valid Scandit Data Capture SDK license key and that you added the necessary dependencies. If you have not done that yet, check out this guide.

Note

You can retrieve your Scandit Data Capture SDK license key, by signing in to your account at ssl.scandit.com/dashboard/sign-in.

Internal dependencies

Some of the Scandit Data Capture SDK modules depend on others to work:

Module	Dependencies
ScanditCaptureCore	No dependencies
ScanditBarcodeCapture	ScanditCaptureCore
ScanditParser	No dependencies
ScanditOCR	No dependencies
ScanditTextCapture	ScanditCaptureCore ScanditTXT
ScanditIdCapture	ScanditCaptureCore ScanditIDC (VIZ documents)
ScanditIDC	No dependencies
ScanditTXT	No dependencies

Create the Data Capture Context

The first step to add capture capabilities to your application is to create a new data capture context. The context expects a valid Scandit Data Capture SDK license key during construction.

self.context = DataCaptureContext(licenseKey: "-- ENTER YOUR SCANDIT LICENSE KEY HERE --")

Configure the Barcode Tracking Mode

The main entry point for the Barcode Tracking Mode is the SDCBarcodeTracking object. It is configured through SDCBarcodeTrackingSettings and allows to register one or more listeners that will get informed whenever a new frame has been processed.

Most of the times, you will not need to conform to a SDCBarcodeTrackingListener, instead you will add a SDCBarcodeTrackingBasicOverlay and conform to a SDCBarcodeTrackingBasicOverlayDelegate.

For this tutorial, we will setup Barcode Tracking for tracking QR codes.

let settings = BarcodeTrackingSettings()
settings.set(symbology: .qr, enabled: true)

Note

If your scenario is similar to one described in Barcode Tracking Scenarios, then you should consider using SDCBarcodeTrackingSettings.settingsWithScenario: for better results.

Next, create a SDCBarcodeTracking instance with the data capture context and the settings initialized in the previous steps:

barcodeTracking = BarcodeTracking(context: context, settings: settings)

Use the Built-in Camera

The data capture context supports using different frame sources to perform recognition on. Most applications will use the built-in camera of the device, e.g. the world-facing camera of a device. The remainder of this tutorial will assume that you use the built-in camera.

Important

In iOS, the user must explicitly grant permission for each app to access cameras. Your app needs to provide static messages to display to the user when the system asks for camera permission. To do that include the NSCameraUsageDescription key in your app’s Info.plist file.

When using the built-in camera there are recommended settings for each capture mode. These should be used to achieve the best performance and user experience for the respective mode. The following couple of lines show how to get the recommended settings and create the camera from it:

let cameraSettings = BarcodeTracking.recommendedCameraSettings

// Depending on the use case further camera settings adjustments can be made here.

let camera = Camera.default
camera?.apply(cameraSettings)

Because the frame source is configurable, the data capture context must be told which frame source to use. This is done with a call to SDCDataCaptureContext.setFrameSource:completionHandler::

context.setFrameSource(camera)

The camera is off by default and must be turned on. This is done by calling SDCFrameSource.switchToDesiredState:completionHandler: with a value of SDCFrameSourceStateOn:

camera?.switch(toDesiredState: .on)

There is a separate guide for more advanced camera functionality.

Use a Capture View to Visualize the Scan Process

When using the built-in camera as frame source, you will typically want to display the camera preview on the screen together with UI elements that guide the user through the capturing process. To do that, add a SDCDataCaptureView to your view hierarchy:

let captureView = DataCaptureView(for: context, frame: view.bounds)
captureView.dataCaptureContext = context
captureView.autoresizingMask = [.flexibleWidth, .flexibleHeight]
view.addSubview(captureView)

To visualize the results of Barcode Tracking, first you need to add the following overlay:

let overlay = BarcodeTrackingBasicOverlay(barcodeTracking: barcodeTracking, view: captureView)

Once the overlay has been added, you should conform to the SDCBarcodeTrackingBasicOverlayDelegate protocol. The method SDCBarcodeTrackingBasicOverlayDelegate.barcodeTrackingBasicOverlay:brushForTrackedBarcode: is invoked every time a new tracked barcode appears and it can be used to set a brush that will be used to highlight that specific barcode in the overlay.

extension ViewController: BarcodeTrackingBasicOverlayDelegate {
    func barcodeTrackingBasicOverlay(_ overlay: BarcodeTrackingBasicOverlay,
                                brushFor trackedBarcode: TrackedBarcode) -> Brush? {
        // Return a custom Brush based on the tracked barcode.
    }
}

If you would like to make the highlights tappable, you need to implement the SDCBarcodeTrackingBasicOverlayDelegate.barcodeTrackingBasicOverlay:didTapTrackedBarcode: method.

extension ViewController: BarcodeTrackingBasicOverlayDelegate {
    func barcodeTrackingBasicOverlay(_ overlay: BarcodeTrackingBasicOverlay,
                                didTap trackedBarcode: TrackedBarcode) {
        // A tracked barcode was tapped.
    }
}

Get Barcode Tracking Feedback

Barcode Tracking, unlike Barcode Capture, doesn’t emit feedback (sound or vibration) when a new barcode is recognized. However, you may implement a SDCBarcodeTrackingListener to provide a similar experience. Below, we use the default SDCFeedback, but you may configure it with your own sound or vibration if you want.

First, let’s create a feedback.

override func viewDidLoad() {
    super.viewDidLoad()
    feedback = Feedback.default
}

Next, use this feedback in a SDCBarcodeTrackingListener:

extension ScanningViewController: BarcodeTrackingListener {
    func barcodeTracking(_ barcodeTracking: BarcodeTracking,
                            didUpdate session: BarcodeTrackingSession,
                            frameData: FrameData) {
        if !session.addedTrackedBarcodes.isEmpty {
            feedback?.emit()
        }
    }
}

SDCBarcodeTrackingListener.barcodeTracking:didUpdate:frameData: is invoked for every processed frame. The session parameter contains information about the currently tracked barcodes, in particular, the newly recognized ones. We check if there are any and if so, we emit the feedback.

As the last step, register the delegate responsible for emitting the feedback with the SDCBarcodeTracking instance.

barcodeTracking.addListener(self)

Disabling Barcode Tracking

To disable barcode tracking set SDCBarcodeTracking.enabled to NO. The effect is immediate: no more frames will be processed after the change. However, if a frame is currently being processed, this frame will be completely processed and deliver any results/callbacks to the registered listeners.

Note that disabling the capture mode does not stop the camera, the camera continues to stream frames until it is turned off or put it in standby calling SwitchToDesiredState with a value of StandBy.

Limitations

MatrixScan does not support the following symbologies:

• DotCode

• MaxiCode

• All postal codes (KIX, RM4SCC)

What’s next?

To dive further into the Scandit Data Capture SDK we recommend the following articles:

• Configure Which Barcodes Are Read

• Add AR Overlays in MatrixScan

Add AR Overlays in MatrixScan

Prerequisites

To proceed, you need to setup a project that uses MatrixScan first, check out this guide (you can ignore the bottom section about the visualization of tracked barcodes using SDCBarcodeTrackingBasicOverlay).

Getting started

There are two ways to add advanced AR overlays to a Data Capture View:

• Take advantage of the SDCBarcodeTrackingAdvancedOverlay class, which provides a ready-to-use implementation for view-based AR overlays.

• Provide your own custom implementation, using the function SDCBarcodeTrackingListener.barcodeTracking:didUpdate:frameData: to retrieve the barcode’s current screen position for each frame.

Note

• The first way is the easiest, as it takes care of adding, removing and animating the overlay’s views whenever needed. It’s also flexible enough to cover the majority of use cases.

• You can always handle touch events on the views you create like you normally would.

Using BarcodeTrackingAdvancedOverlay

As mentioned above, the advanced overlay combined with its listener offers an easy way of adding augmentations to your SDCDataCaptureView. In this guide we will add a view above each barcode showing its content.

First of all, create a new instance of SDCBarcodeTrackingAdvancedOverlay and add it to the SDCDataCaptureView.

let overlay = BarcodeTrackingAdvancedOverlay(barcodeTracking: barcodeTracking, for: captureView)

At this point, you have two options.

• Add a SDCBarcodeTrackingAdvancedOverlayDelegate to the overlay.

• Use the setters in the overlay to specify the view, anchor and offset for each barcode.

Note

The second way will take priority over the first one, which means that if a view for a barcode has been set using SDCBarcodeTrackingAdvancedOverlay.setView:forTrackedBarcode:, the function SDCBarcodeTrackingAdvancedOverlayDelegate.barcodeTrackingAdvancedOverlay:viewForTrackedBarcode: won’t be invoked for that specific barcode.

Using SDCBarcodeTrackingAdvancedOverlayDelegate

• You need to conform to SDCBarcodeTrackingAdvancedOverlayDelegate. This protocol’s methods are invoked every time a barcode is newly tracked.

• SDCBarcodeTrackingAdvancedOverlayDelegate.barcodeTrackingAdvancedOverlay:viewForTrackedBarcode: asks for a view to animate on top of the barcode. Returning nil will show no view.

• SDCBarcodeTrackingAdvancedOverlayDelegate.barcodeTrackingAdvancedOverlay:anchorForTrackedBarcode: asks how to anchor the view to the barcode through SDCAnchor. Be aware that it anchors the view’s center to the anchor point. To achieve anchoring the top of the view or the bottom etc. you will have to set an offset as explained in the next point.

• SDCBarcodeTrackingAdvancedOverlayDelegate.barcodeTrackingAdvancedOverlay:offsetForTrackedBarcode: asks for an offset that is applied on the already anchored view. This offset is expressed through a SDCPointWithUnit.

func barcodeTrackingAdvancedOverlay(_ overlay: BarcodeTrackingAdvancedOverlay,
                                    viewFor trackedBarcode: TrackedBarcode) -> UIView? {
    // Create and return the view you want to show for this tracked barcode. You can also return nil, to have no view for this barcode.
    let label = UILabel()
    label.text = trackedBarcode.barcode.data
    label.backgroundColor = .white
    label.sizeToFit()
    return label
}

func barcodeTrackingAdvancedOverlay(_ overlay: BarcodeTrackingAdvancedOverlay,
                                    anchorFor trackedBarcode: TrackedBarcode) -> Anchor {
    // As we want the view to be above the barcode, we anchor the view's center to the top-center of the barcode quadrilateral.
    // Use the function below to adjust the position of the view by providing an offset.
    return .topCenter
}

func barcodeTrackingAdvancedOverlay(_ overlay: BarcodeTrackingAdvancedOverlay,
                                    offsetFor trackedBarcode: TrackedBarcode) -> PointWithUnit {

    // This is the offset that will be applied to the view.
    // You can use .fraction to give a measure relative to the view itself, the sdk will take care of transforming this into pixel size.
    // We now center horizontally and move up the view to make sure it's centered and above the barcode quadrilateral by half of the view's height.
    return PointWithUnit(
        x: FloatWithUnit(value: 0, unit: .fraction),
        y: FloatWithUnit(value: -1, unit: .fraction)
    )
}

Using the setters in the overlay

The function SDCBarcodeTrackingListener.barcodeTracking:didUpdate:frameData: gives you access to a session, which contains all added, updated and removed tracked barcodes. From here you can create the view you want to display, and then call SDCBarcodeTrackingAdvancedOverlay.setView:forTrackedBarcode:, SDCBarcodeTrackingAdvancedOverlay.setAnchor:forTrackedBarcode: and SDCBarcodeTrackingAdvancedOverlay.setOffset:forTrackedBarcode:

func barcodeTracking(_ barcodeTracking: BarcodeTracking,
                    didUpdate session: BarcodeTrackingSession,
                    frameData: FrameData) {
    DispatchQueue.main.async {
        for trackedBarcode in session.addedTrackedBarcodes {
            let label = UILabel()
            label.text = trackedBarcode.barcode.data
            label.backgroundColor = .white
            label.sizeToFit()
            self.overlay.setView(label, for: trackedBarcode)
            self.overlay.setAnchor(.topCenter, for: trackedBarcode)
            let point = PointWithUnit(x: FloatWithUnit(value: 0, unit: .fraction),
                                      y: FloatWithUnit(value: -1, unit: .fraction))
            self.overlay.setOffset(point, for: trackedBarcode)
        }
    }
}

Provide your own custom implementation

If you do not want to use the overlay, it is also possible to add augmented reality features based on the tracking identifier and the quadrilateral coordinates that every tracked barcode has. Below are some pointers.

• Set a SDCBarcodeTrackingListener on the barcode tracking

• In the SDCBarcodeTrackingListener.barcodeTracking:didUpdate:frameData: function fetch the added and removed tracked barcodes.

• Create and show the views for the added barcodes.

• Remove the views for the lost barcodes.

• Add a method that is called 60fps when SDCBarcodeTracking is enabled. In this method, for each SDCTrackedBarcode on-screen, update the position based on SDCTrackedBarcode.location. Please note that there is no need to animate the change of location, the change of position will happen frequently enough that the view will look as it is animated.

The best way to have a method called 60fps, is by using CADisplayLink.

Note

The frame coordinates from SDCTrackedBarcode.location need to be mapped to view coordinates, using SDCDataCaptureView.viewQuadrilateralForFrameQuadrilateral:.

func barcodeTracking(_ barcodeTracking: BarcodeTracking,
                    didUpdate session: BarcodeTrackingSession,
                    frameData: FrameData) {
    DispatchQueue.main.async {
        for lostTrackIdentifier in session.removedTrackedBarcodes {
            // You now know the identifier of the tracked barcode that has been lost. Usually here you would remove the views associated.
        }

        for trackedBarcode in session.addedTrackedBarcodes {
            // Fixed identifier for the tracked barcode.
            let trackingIdentifier = trackedBarcode.identifier

            // Current location of the tracked barcode.
            let location = trackedBarcode.location
            let quadrilateral = self.captureView.viewQuadrilateral(forFrameQuadrilateral: location)

            // You now know this new tracking's identifier and location. Usually here you would create and show the views.
        }
    }
}