Batch Scanning And AR

Get Started With MatrixScan

With MatrixScan, you can highlight and interact multiple barcodes within the same frame and build AR experiences. MatrixScan use cases are implemented through functionality provided by BarcodeTracking.

In this guide you will learn step by step how to add BarcodeTracking to your application. Roughly, the steps are:

Prerequisites

Before starting with adding a capture mode, make sure that you have a valid Scandit Data Capture SDK license key and that you added the necessary dependencies. If you have not done that yet, check out this guide.

Note

You can retrieve your Scandit Data Capture SDK license key, by signing in to your account at ssl.scandit.com/dashboard/sign-in.

Enable browser multithreading

You can achieve better performance by enabling multithreading in the browser that supports it. Check the Requirements Page to know what are the minimum versions that can take advantage of multithreading. To enable multithreading you must set your site to be crossOriginIsolated. This will enable the possibility for the sdk to use multithreading and significatively boost the performance. If the environment supports it the sdk will automatically use multithreading. You can programmatically check for multithreading supports using BrowserHelper.checkMultithreadingSupport()

Important

Multithreading is particularly critical for MatrixScan so be sure to configure it correctly following this tutorial, Making your website “cross-origin isolated” using COOP, COEP and CORP. You can also check this link A guide to enable cross-origin isolation and this one Safely reviving shared memory

# An example of how headers could be set.
Cross-Origin-Embedder-Policy: require-corp;
Cross-Origin-Opener-Policy: same-origin;
Cross-Origin-Resource-Policy: cross-origin allow-credentials; require-corp origin https://example.com https://example.net;

Internal dependencies

Some of the Scandit Data Capture SDK modules depend on others to work:

Module

Dependencies

ScanditCaptureCore

No dependencies

ScanditBarcodeCapture

  • ScanditCaptureCore

Create the Data Capture Context

The first step to add capture capabilities to your application is to create a new data capture context.

// the license key used in configure() will be used
const context = await SDCCore.DataCaptureContext.create();

Configure the Barcode Tracking Mode

The main entry point for the Barcode Tracking Mode is the BarcodeTracking object. It is configured through BarcodeTrackingSettings and allows to register one or more listeners that will get informed whenever a new frame has been processed.

Most of the times, you will not need to implement a BarcodeTrackingListener, instead you will add a BarcodeTrackingBasicOverlay and implement a BarcodeTrackingBasicOverlayListener.

For this tutorial, we will setup Barcode Tracking for tracking QR codes.

Note

If your scenario is similar to one described in Barcode Tracking Scenarios, then you should consider using BarcodeTrackingSettings.forScenario() for better results.

Next, create a BarcodeTracking instance with the data capture context and the settings initialized in the previous steps:

const barcodeTracking = await SDCBarcode.BarcodeTracking.forContext(context, settings);

Use the Built-in Camera

The data capture context supports using different frame sources to perform recognition on. Most applications will use the built-in camera of the device, e.g. the world-facing camera of a device. The remainder of this tutorial will assume that you use the built-in camera.

When using the built-in camera there are recommended settings for each capture mode. These should be used to achieve the best performance and user experience for the respective mode. The following couple of lines show how to get the recommended settings and create the camera from it:

const camera = SDCCore.Camera.default;

const cameraSettings = SDCBarcode.BarcodeTracking.recommendedCameraSettings;
await camera.applySettings(cameraSettings);

Because the frame source is configurable, the data capture context must be told which frame source to use. This is done with a call to DataCaptureContext.setFrameSource():

await context.setFrameSource(camera);

The camera is off by default and must be turned on. This is done by calling FrameSource.switchToDesiredState() with a value of FrameSourceState.On:

await camera.switchToDesiredState(Scandit.FrameSourceState.On);

There is a separate guide for more advanced camera functionality.

Use a Capture View to Visualize the Scan Process

When using the built-in camera as frame source, you will typically want to display the camera preview on the screen together with UI elements that guide the user through the capturing process. To do that, add a DataCaptureView to your view hierarchy:

const view = await SDCCore.DataCaptureView.forContext(context);
view.connectToElement(htmlElement);

To visualize the results of Barcode Tracking, first you need to add the following overlay:

const overlay = await SDCBarcode.BarcodeTrackingBasicOverlay.withBarcodeTrackingForView(barcodeTracking, view);

Once the overlay has been added, you should implement the BarcodeTrackingBasicOverlayListener interface. The method BarcodeTrackingBasicOverlayListener.brushForTrackedBarcode() is invoked every time a new tracked barcode appears and it can be used to set a brush that will be used to highlight that specific barcode in the overlay.

overlay.listener = {
  brushForTrackedBarcode: (overlay, trackedBarcode) => {
    // Return a custom Brush based on the tracked barcode.
  },
};

If you would like to make the highlights tappable, you need to implement the BarcodeTrackingBasicOverlayListener.didTapTrackedBarcode() method.

overlay.listener = {
  didTapTrackedBarcode: (overlay, trackedBarcode) => {
    // A tracked barcode was tapped.
  },
};

Get Barcode Tracking Feedback

Barcode Tracking, unlike Barcode Capture, doesn’t emit feedback (sound or vibration) when a new barcode is recognized. However, you may implement a BarcodeTrackingListener to provide a similar experience. Below, we use the default Feedback, but you may configure it with your own sound or vibration if you want.

Next, use this feedback in a BarcodeTrackingListener:

const feedbackListener = {
  didUpdateSession: (barcodeTracking, session) => {
    if (session.addedTrackedBarcodes.length > 0) {
      feedback.emit();
    }
  }
};

BarcodeTrackingListener.didUpdateSession() is invoked for every processed frame. The session parameter contains information about the currently tracked barcodes, in particular, the newly recognized ones. We check if there are any and if so, we emit the feedback.

As the last step, register the listener responsible for emitting the feedback with the BarcodeTracking instance.

barcodeTracking.addListener(feedbackListener);

Disabling Barcode Tracking

To disable barcode tracking call BarcodeTracking.setEnabled() passing false. No more frames will be processed after the change. However, if a frame is currently being processed, this frame will be completely processed and deliver any results/callbacks to the registered listeners.

Note that disabling the capture mode does not stop the camera, the camera continues to stream frames until it is turned off or put it in standby calling SwitchToDesiredState with a value of StandBy.

Limitations

MatrixScan does not support the following symbologies:

  • DotCode

  • MaxiCode

  • All postal codes (KIX, RM4SCC)

Important

Barcode Tracking needs browser multithreading to run. Check the minimum browser support in the Requirements Page and how to enable it Enable Multithreading

What’s next?

To dive further into the Scandit Data Capture SDK we recommend the following articles:

Add AR Overlays in MatrixScan

Prerequisites

To proceed, you need to setup a project that uses MatrixScan first, check out this guide (you can ignore the bottom section about the visualization of tracked barcodes using BarcodeTrackingBasicOverlay).

Getting started

There are two ways to add advanced AR overlays to a Data Capture View:

Note

  • The first way is the easiest, as it takes care of adding, removing and animating the overlay’s views whenever needed. It’s also flexible enough to cover the majority of use cases.

  • You can always handle touch events on the views you create like you normally would.

To add advanced AR overlays to a Data Capture View you can take advantage of the BarcodeTrackingAdvancedOverlay class, which provides a ready-to-use implementation for view-based AR overlays.

Using BarcodeTrackingAdvancedOverlay

As mentioned above, the advanced overlay combined with its listener offers an easy way of adding augmentations to your DataCaptureView. In this guide we will add a view above each barcode showing its content.

First of all, create a new instance of BarcodeTrackingAdvancedOverlay and add it to the DataCaptureView.

const overlay = await BarcodeTrackingAdvancedOverlay.withBarcodeTrackingForView(barcodeTracking, view);

At this point, you have two options.

Note

The second way will take priority over the first one, which means that if a view for a barcode has been set using BarcodeTrackingAdvancedOverlay.setViewForTrackedBarcode(), the function BarcodeTrackingAdvancedOverlayListener.viewForTrackedBarcode() won’t be invoked for that specific barcode.

Using BarcodeTrackingAdvancedOverlayListener

import { PointWithUnit, MeasureUnit, NumberWithUnit, Anchor } from "scandit-web-datacapture-core"
import { TrackedBarcodeView } from "scandit-web-datacapture-barcode"

// ...
overlay.listener = {
  viewForTrackedBarcode: (overlay, trackedBarcode) => {
    // Create and return the view you want to show for this tracked barcode. You can also return null, to have no view for this barcode.
    let element = document.createElement('span');
    element.innerText = trackedBarcode.barcode.data;
    element.style.backgroundColor = '#FFFFFFFF';
    return TrackedBarcodeView.withHTMLElement(element, null);
  },

  anchorForTrackedBarcode: (overlay, trackedBarcode) => {
    // As we want the view to be above the barcode, we anchor the view's center to the top-center of the barcode quadrilateral.
    // Use the function 'offsetForTrackedBarcode' below to adjust the position of the view by providing an offset.
    return Anchor.TopCenter;
  },

  offsetForTrackedBarcode: (overlay, trackedBarcode) => {
    // This is the offset that will be applied to the view.
    // You can use .fraction to give a measure relative to the view itself, the sdk will take care of transforming this into pixel size.
    // We now center horizontally and move up the view to make sure it's centered and above the barcode quadrilateral by half of the view's height.
    return new PointWithUnit(
      new NumberWithUnit(0, MeasureUnit.Fraction),
      new NumberWithUnit(-1, MeasureUnit.Fraction),
    );
  },
};

Using the setters in the overlay

The function BarcodeTrackingListener.didUpdateSession() gives you access to a session, which contains all added, updated and removed tracked barcodes. From here you can create the view you want to display, and then call BarcodeTrackingAdvancedOverlay.setViewForTrackedBarcode(), BarcodeTrackingAdvancedOverlay.setAnchorForTrackedBarcode() and BarcodeTrackingAdvancedOverlay.setOffsetForTrackedBarcode()

import { PointWithUnit, MeasureUnit, NumberWithUnit, Anchor } from "scandit-web-datacapture-core"
import { TrackedBarcodeView } from "scandit-web-datacapture-barcode"

// ...
barcodeTracking.addListener({
  didUpdateSession: (barcodeTracking, session) => {
    session.addedTrackedBarcodes.forEach(trackedBarcode => {
      let element = document.createElement('span');
      element.innerText = trackedBarcode.barcode.data;
      element.style.backgroundColor = '#FFFFFFFF';
      let trackedBarcodeView = TrackedBarcodeView.withHTMLElement(element, null);

      overlay.setViewForTrackedBarcode(trackedBarcodeView, trackedBarcode);
      overlay.setAnchorForTrackedBarcode(Anchor.TopCenter, trackedBarcode);
      overlay.setOffsetForTrackedBarcode(
        new PointWithUnit(
          new NumberWithUnit(0, MeasureUnit.Fraction), new NumberWithUnit(-1, MeasureUnit.Fraction)
        ), trackedBarcode);
    })
});