Batch Scanning And AR

Get Started With MatrixScan

With MatrixScan, you can highlight and interact multiple barcodes within the same frame and build AR experiences. MatrixScan use cases are implemented through functionality provided by BarcodeTracking.

In this guide you will learn step by step how to add BarcodeTracking to your application. Roughly, the steps are:

• Include the ScanditBarcodeCapture library and its dependencies to your project, if any.

• Create a new data capture context, initialized with your license key.

• Create a barcode tracking settings instance where you enable the barcode symbologies you want to read in your application.

• Create a new barcode tracking object and initialize it with the settings created above.

• Obtain a camera instance and set it as the frame source on the data capture context previously created.

• Create a new data capture view and add a basic overlay instance to it for visual feedback.

• Register an overlay listener and implement BrushForTrackedBarcode(), which is called whenever a new tracked barcode appears.

Prerequisites

Before starting with adding a capture mode, make sure that you have a valid Scandit Data Capture SDK license key and that you added the necessary dependencies. If you have not done that yet, check out this guide.

Note

You can retrieve your Scandit Data Capture SDK license key, by signing in to your account at ssl.scandit.com/dashboard/sign-in.

Internal dependencies

Some of the Scandit Data Capture SDK modules depend on others to work:

Module	Dependencies
scandit-capacitor-datacapture-core	No dependencies
scandit-capacitor-datacapture-barcode	scandit-capacitor-datacapture-core
scandit-capacitor-datacapture-id	scandit-capacitor-datacapture-core scandit-capacitor-datacapture-text (VIZ documents)
scandit-capacitor-datacapture-text	scandit-capacitor-datacapture-core
scandit-capacitor-datacapture-parser	scandit-capacitor-datacapture-core

Create the Data Capture Context

The first step to add capture capabilities to your application is to create a new data capture context. The context expects a valid Scandit Data Capture SDK license key during construction.

const context = Scandit.DataCaptureContext.forLicenseKey("-- ENTER YOUR SCANDIT LICENSE KEY HERE --");

Configure the Barcode Tracking Mode

The main entry point for the Barcode Tracking Mode is the BarcodeTracking object. It is configured through BarcodeTrackingSettings and allows to register one or more listeners that will get informed whenever a new frame has been processed.

Most of the times, you will not need to implement a BarcodeTrackingListener, instead you will add a BarcodeTrackingBasicOverlay and implement a BarcodeTrackingBasicOverlayListener.

For this tutorial, we will setup Barcode Tracking for tracking QR codes.

const settings = new Scandit.BarcodeTrackingSettings();
settings.enableSymbology(Scandit.Symbology.QR, true);

Note

If your scenario is similar to one described in Barcode Tracking Scenarios, then you should consider using BarcodeTrackingSettings.forScenario() for better results.

Next, create a BarcodeTracking instance with the data capture context and the settings initialized in the previous steps:

const barcodeTracking = Scandit.BarcodeTracking.forContext(context, settings);

Use the Built-in Camera

The data capture context supports using different frame sources to perform recognition on. Most applications will use the built-in camera of the device, e.g. the world-facing camera of a device. The remainder of this tutorial will assume that you use the built-in camera.

Important

In iOS, the user must explicitly grant permission for each app to access cameras. Your app needs to provide static messages to display to the user when the system asks for camera permission. To do that include the NSCameraUsageDescription key in your app’s Info.plist file.

Important

In Android, the user must explicitly grant permission for each app to access cameras. Your app needs to declare the use of the Camera permission in the AndroidManifest.xml file and request it at runtime so the user can grant or deny the permission. To do that follow the guidelines from Request app permissions to request the android.permission.CAMERA permission.

When using the built-in camera there are recommended settings for each capture mode. These should be used to achieve the best performance and user experience for the respective mode. The following couple of lines show how to get the recommended settings and create the camera from it:

const cameraSettings = Scandit.BarcodeTracking.recommendedCameraSettings;

// Depending on the use case further camera settings adjustments can be made here.

const camera = Scandit.Camera.default;
if (camera != null) {
  camera.applySettings(cameraSettings);
}

Because the frame source is configurable, the data capture context must be told which frame source to use. This is done with a call to DataCaptureContext.setFrameSource():

context.setFrameSource(camera);

The camera is off by default and must be turned on. This is done by calling FrameSource.switchToDesiredState() with a value of FrameSourceState.On:

camera.switchToDesiredState(Scandit.FrameSourceState.On);

There is a separate guide for more advanced camera functionality.

Use a Capture View to Visualize the Scan Process

When using the built-in camera as frame source, you will typically want to display the camera preview on the screen together with UI elements that guide the user through the capturing process. To do that, add a DataCaptureView to your view hierarchy:

const view = Scandit.DataCaptureView.forContext(context);
view.connectToElement(htmlElement);

To visualize the results of Barcode Tracking, first you need to add the following overlay:

const overlay = Scandit.BarcodeTrackingBasicOverlay.withBarcodeTrackingForView(barcodeTracking, view);

Once the overlay has been added, you should implement the BarcodeTrackingBasicOverlayListener interface. The method BarcodeTrackingBasicOverlayListener.brushForTrackedBarcode() is invoked every time a new tracked barcode appears and it can be used to set a brush that will be used to highlight that specific barcode in the overlay.

overlay.listener = {
  brushForTrackedBarcode: (overlay, trackedBarcode) => {
    // Return a custom Brush based on the tracked barcode.
  },
};

If you would like to make the highlights tappable, you need to implement the BarcodeTrackingBasicOverlayListener.didTapTrackedBarcode() method.

overlay.listener = {
  didTapTrackedBarcode: (overlay, trackedBarcode) => {
    // A tracked barcode was tapped.
  },
};

Get Barcode Tracking Feedback

Barcode Tracking, unlike Barcode Capture, doesn’t emit feedback (sound or vibration) when a new barcode is recognized. However, you may implement a BarcodeTrackingListener to provide a similar experience. Below, we use the default Feedback, but you may configure it with your own sound or vibration if you want.

const feedback = Scandit.Feedback.defaultFeedback;

Next, use this feedback in a BarcodeTrackingListener:

const feedbackListener = {
  didUpdateSession: (barcodeTracking, session) => {
    if (session.addedTrackedBarcodes.length > 0) {
      feedback.emit();
    }
  }
};

BarcodeTrackingListener.didUpdateSession() is invoked for every processed frame. The session parameter contains information about the currently tracked barcodes, in particular, the newly recognized ones. We check if there are any and if so, we emit the feedback.

As the last step, register the listener responsible for emitting the feedback with the BarcodeTracking instance.

barcodeTracking.addListener(feedbackListener);

Disabling Barcode Tracking

To disable barcode tracking set BarcodeTracking.isEnabled to false. The effect is immediate: no more frames will be processed after the change. However, if a frame is currently being processed, this frame will be completely processed and deliver any results/callbacks to the registered listeners.

Note that disabling the capture mode does not stop the camera, the camera continues to stream frames until it is turned off or put it in standby calling SwitchToDesiredState with a value of StandBy.

Limitations

MatrixScan does not support the following symbologies:

• DotCode

• MaxiCode

• All postal codes (KIX, RM4SCC)

What’s next?

To dive further into the Scandit Data Capture SDK we recommend the following articles:

• Configure Which Barcodes Are Read

• Add AR Overlays in MatrixScan

Add AR Overlays in MatrixScan

Prerequisites

To proceed, you need to setup a project that uses MatrixScan first, check out this guide (you can ignore the bottom section about the visualization of tracked barcodes using BarcodeTrackingBasicOverlay).

Getting started

To add advanced AR overlays to a Data Capture View you can take advantage of the BarcodeTrackingAdvancedOverlay class, which provides a ready-to-use implementation for view-based AR overlays.

Using BarcodeTrackingAdvancedOverlay

As mentioned above, the advanced overlay combined with its listener offers an easy way of adding augmentations to your DataCaptureView. In this guide we will add a view above each barcode showing its content.

First of all, create a new instance of BarcodeTrackingAdvancedOverlay and add it to the DataCaptureView.

const overlay = Scandit.BarcodeTrackingAdvancedOverlay.withBarcodeTrackingForView(barcodeTracking, view);

At this point, you have two options.

• Add a BarcodeTrackingAdvancedOverlayListener to the overlay.

• Use the setters in the overlay to specify the view, anchor and offset for each barcode.

Note

The second way will take priority over the first one, which means that if a view for a barcode has been set using BarcodeTrackingAdvancedOverlay.setViewForTrackedBarcode(), the function BarcodeTrackingAdvancedOverlayListener.viewForTrackedBarcode() won’t be invoked for that specific barcode.

Using BarcodeTrackingAdvancedOverlayListener

• You need to implement BarcodeTrackingAdvancedOverlayListener. This interface’s methods are invoked every time a barcode is newly tracked.

• BarcodeTrackingAdvancedOverlayListener.viewForTrackedBarcode() asks for a view to animate on top of the barcode. Returning null will show no view.

• BarcodeTrackingAdvancedOverlayListener.anchorForTrackedBarcode() asks how to anchor the view to the barcode through Anchor. Be aware that it anchors the view’s center to the anchor point. To achieve anchoring the top of the view or the bottom etc. you will have to set an offset as explained in the next point.

• BarcodeTrackingAdvancedOverlayListener.offsetForTrackedBarcode() asks for an offset that is applied on the already anchored view. This offset is expressed through a PointWithUnit.

overlay.listener = {
  viewForTrackedBarcode: (overlay, trackedBarcode) => {
    // Create and return the view you want to show for this tracked barcode. You can also return null, to have no view for this barcode.
    let element = document.createElement('span');
    element.innerText = trackedBarcode.barcode.data;
    element.style.backgroundColor = '#FFFFFFFF';
    return Scandit.TrackedBarcodeView.withHTMLElement(element, null);
  },

  anchorForTrackedBarcode: (overlay, trackedBarcode) => {
    // As we want the view to be above the barcode, we anchor the view's center to the top-center of the barcode quadrilateral.
    // Use the function 'offsetForTrackedBarcode' below to adjust the position of the view by providing an offset.
    return Scandit.Anchor.TopCenter;
  },

  offsetForTrackedBarcode: (overlay, trackedBarcode) => {
    // This is the offset that will be applied to the view.
    // You can use .fraction to give a measure relative to the view itself, the sdk will take care of transforming this into pixel size.
    // We now center horizontally and move up the view to make sure it's centered and above the barcode quadrilateral by half of the view's height.
    return new Scandit.PointWithUnit(
      new Scandit.NumberWithUnit(0, Scandit.MeasureUnit.Fraction),
      new Scandit.NumberWithUnit(-1, Scandit.MeasureUnit.Fraction),
    );
  },
};

Using the setters in the overlay

The function BarcodeTrackingListener.didUpdateSession() gives you access to a session, which contains all added, updated and removed tracked barcodes. From here you can create the view you want to display, and then call BarcodeTrackingAdvancedOverlay.setViewForTrackedBarcode(), BarcodeTrackingAdvancedOverlay.setAnchorForTrackedBarcode() and BarcodeTrackingAdvancedOverlay.setOffsetForTrackedBarcode()

didUpdateSession: (barcodeTracking, session) => {
  session.addedTrackedBarcodes.forEach(trackedBarcode => {
    let element = document.createElement('span');
    element.innerText = trackedBarcode.barcode.data;
    element.style.backgroundColor = '#FFFFFFFF';
    let trackedBarcodeView = Scandit.TrackedBarcodeView.withHTMLElement(element, null);

    window.overlay.setViewForTrackedBarcode(trackedBarcodeView, trackedBarcode);
    window.overlay.setAnchorForTrackedBarcode(Scandit.Anchor.TopCenter, trackedBarcode);
    window.overlay.setOffsetForTrackedBarcode(
      new Scandit.PointWithUnit(
        new Scandit.NumberWithUnit(0, Scandit.MeasureUnit.Fraction), new Scandit.NumberWithUnit(-1, Scandit.MeasureUnit.Fraction)
      ), trackedBarcode);
  });
}