Scanning Multiple Barcodes (including Matrix Scan)

Motivation

Sometimes you have packages/pallets with several codes, with the Scandit Barcode Scanner you can scan all the codes or just a subset at the same time, saving time and money. To do this you have to options, simple multi scan mode or matrix scan.

Simple Multi Scan

Simple multi scan is when the barcode scanner tries to recognize multiple barcodes every frame. The codes will not be tracked over time but simply all end up in the scan session. To implement simple multi scanning

• Set the code duplicated filter in the scan settings by setting setCodeDuplicateFilter(int) to -1, so each unique code is only added once to the session.
• Set the maximum number of codes to be decoded every frame through setMaxNumberOfCodesPerFrame(int) to something higher than 1, depending on the environment of the codes it might be a good idea to set it higher than the number of codes that you actually want to scan.
• In the didScan function, wait until the number of expected codes have been decoded, then store the codes and pause/stop the session. If you pause and plan to resume but want to start a new session, make sure that you clear the session first.

public void initializeScanner() {
    ScanSettings settings = ScanSettings.create();

    // Enable the relevant symbologies.
    settings.setSymbologyEnabled(Barcode.SYMBOLOGY_EAN13, true);
    settings.setSymbologyEnabled(Barcode.SYMBOLOGY_CODE_39, true);

    // Barcodes are filtered as duplicates if they match an already decoded barcode in the session.
    settings.setCodeDuplicateFilter(-1);
    // The maximum number of codes to be decoded every frame.
    settings.setMaxNumberOfCodesPerFrame(3);

    // Create the picker, set the listener, add the picker to the view hierarchy and start it.
    BarcodePicker picker = new BarcodePicker(scanSettings);
    picker.setOnScanListener(this);
    setContentView(picker)
    picker.startScanning();
}

@Override
public void didScan(ScanSession session) {
    // Number of expected barcodes.
    int numExpectedCodes = 3;
    // Get all the scanned barcodes from the session.
    List<Barcode> allCodes = session.getAllRecognizedCodes();

    // If the number of scanned codes is greater than or equal to the number of expected barcodes,
    // pause the scanning and clear the session (to remove recognized barcodes).
    if (allCodes.size() >= numExpectedCodes) {
        // Stop scanning or pause and clear the session.
        session.stopScanning();
        ...
    }
}

Matrix Scan

Matrix Scan improves on simple multi scan by tracking recognized barcodes over time. The tracking improves the visual feedback over simple multi scanning which makes it easier to use.

Note: Matrix Scan currently only works with 1D codes. We are working on adding 2D!

Enabling Matrix Scan

To implement Matrix Scanning

• Enable matrix scanning in the scan settings by setting setMatrixScanEnabled(boolean) to true.
• Set the maximum number of codes to be decoded every frame through setMaxNumberOfCodesPerFrame(int) to something higher than 1, depending on the environment of the codes it is a good idea to set it higher than the number of codes that you actually want to scan.
• Change the GUI style through setGuiStyle(int) to GUI_STYLE_MATRIX_SCAN
• In the didScan function, wait until the number of expected codes have been decoded, then store the codes and pause/stop the session. If you pause and plan to resume but want to start a new session, make sure that you clear the session first.

At this point you can start the barcode picker and any recognized barcodes of the enabled symbologies will be highlighted by a filled green rectangle. Barcodes that have been localized but not recognized will be highlighted by a green border.

public void initializeScanner() {
    ScanSettings settings = ScanSettings.create();

    // Enable the relevant symbologies.
    settings.setSymbologyEnabled(Barcode.SYMBOLOGY_EAN13, true);
    settings.setSymbologyEnabled(Barcode.SYMBOLOGY_CODE_39, true);

    // Enable Matrix Scanning.
    settings.setMatrixScanEnabled(true);
    // The maximum number of codes to be decoded every frame.
    settings.setMaxNumberOfCodesPerFrame(3);

    // Create the picker and set the listener.
    BarcodePicker picker = new BarcodePicker(scanSettings);
    picker.setOnScanListener(this);

    // Set the GUI style accordingly.
    picker.getOverlayView().setGuiStyle(ScanOverlay.GUI_STYLE_MATRIX_SCAN);

    // Add the picker to the view hierarchy and start it.
    setContentView(picker)
    picker.startScanning();
}

@Override
public void didScan(ScanSession session) {
    // Number of expected barcodes.
    int numExpectedCodes = 3;
    // Get all the scanned barcodes from the session.
    List<Barcode> allCodes = session.getAllRecognizedCodes();

    // If the number of scanned codes is greater than or equal to the number of expected barcodes,
    // pause the scanning and clear the session (to remove recognized barcodes).
    if (allCodes.size() >= numExpectedCodes) {
        // Stop scanning or pause and clear the session.
        session.stopScanning();
        ...
    }
}

Rejecting unwanted codes

Like normal scanning Matrix Scanning provides you the option to reject codes. Just like for normal scanning rejected codes are not added to the scan session and do not provide any feedback like vibrating or beeping. In Multi Scan the rejected codes are still drawn on the screen but in a different color. This gives the user visual feedback that a certain code is not the one you are looking for which is something not possible through simple multi scanning. To implement rejection

• Set a ProcessFrameListener on the picker setProcessFrameListener(ProcessFrameListener)
• In the didProcess function you fetch the currently tracked barcodes from the session through getTrackedCodes() .
• Once you've checked the tracked codes for their symbologies and/or data you can reject them through rejectTrackedCode(TrackedBarcode)

Note: Rejecting in the didScan function is not allowed and will throw an exception.

public void initializeScanner() {
    ... // Same initialization as above.

    // Set the process frame listener.
    picker.setProcessFrameListener();
}

@Override
public void didProcess(byte[] imageBuffer, int width, int height, ScanSession session) {
    Map<Long, TrackedBarcode> trackedCodes = session.getTrackedCodes();

    for (TrackedBarcode trackedCode : trackedCodes.values()) {
        // Reject all Code39 barcodes.
        if (trackedCode.getSymbology() == Barcode.SYMBOLOGY_CODE39) {
            session.rejectTrackedCode(trackedCode);
        }
    }
}

Providing your own visualization

Matrix Scanning returns the locations of tracked barcodes with every processed frame. This allows you to replace the default visualization with your own. For example, you might want to overlay the barcodes with images of the products they refer to. To add your own visualization

• Set a ProcessFrameListener on the picker setProcessFrameListener(ProcessFrameListener)
• In the didProcess function you fetch the currently tracked barcodes from the session through getTrackedCodes() . The object is a map with Long keys which identify the TrackedBarcodes over different frames.
• Compare the tracked barcodes of the current frame with the one from the previous one and change your visualization accordingly. Instead of using getLocation() which gives you the exact location of the barcode in the frame you should generally use getPredictedLocation() which gives you a predicted location for the barcode. Using the predicted location avoids lagging behind the camera feed that is displayed.
• There are some state transitions where animating from the previous location of a tracked barcode to its current position can end up with weird artifacts as the edge ordering of the location is not stable. If your visualization depends on the edge ordering make sure that you query shouldAnimateFromPreviousToNextState() before animating. The default visualization makes use of this as it draws the location with all four edges and therefore depends on the edge ordering, if the order of the edges were to change it is possible that an animation would flip the visualization which is not what happens to the actual barcode. However, a visualization that just draws something in the center of the location does not depend on the edge ordering and can ignore shouldAnimateFromPreviousToNextState() .
• When animating your visualization take into consideration getDeltaTimeForPrediction which tells you how long it will take the code to move to the predicted location. This time generally is about as long as it will take the barcode picker to process the next frame, giving you a new predicted location right as your previous animation is coming to an end.

Important: The coordinates you get back from getLocation() and getPredictedLocation() are in the coordinate system of the processed frame. To visualize them on top of the picker's camera feed you have to convert them to the picker's coordinate system which you do by calling convertPointToPickerCoordinates(Point) .

Important: Don't forget that the didProcess function is not called on the mainThread, you have to switch to it to change the UI.

public Map<Long, Visualization> visualizations = new HashMap<>();

public class Visualization {
    // This is your view or an object holding data for your view.
}

public void initializeScanner() {
    ... // Same initialization as at the top.

    // Set the process frame listener.
    picker.setProcessFrameListener();
}

@Override
public void didScan(ScanSession session) {
    Map<Long, TrackedBarcode> trackedCodes = session.getTrackedCodes();

    // Iterate over the current visualizations and remove all that are no longer tracked.
    List<Long> currentIds = new ArrayList(visualizations.keySet());
    for (Long id : currentIds) {
        if (!trackedCodes.containsKey(id)) {
            visualizations.remove(id);
            // Maybe add an animation for barcodes that vanish.
        }
    }

    // Iterate over all the currently tracked barcodes, update old ones and add new ones.
    for (Map.Entry<Long, TrackedBarcode> entry : trackedCodes.entrySet()) {
        if (visualizations.containsKey(entry.getKey())) {
            // Update the visualization
            if (entry.getValue().shouldAnimateFromPreviousToNextState()) {
                // Animate to the new predicted location.
            }
        } else {
            // Add a new visualization, maybe animate the appearance.
        }
    }
}