Accessing the camera frames directly

The Scandit Barcode Scanner gives you access to the camera frames after a frame has been processed. This can be useful if you want to further process the camera frame after a barcode was or was not recognized.

Receiving the frames

To receive the camera frames you have to first set the process frame listener through setProcessFrameListener

mPicker.setProcessFrameListener(this);

And then implement the ProcessFrameListener interface, which consists of a single method. The method is invoked whenever a frame has been processed by the barcode scanner. The frame that you receive is the raw frame data that the scanner itself receives from the Android camera API. Be aware that the frame is not rotated with the phone but is always in the natural sensor orientation, like originally captured by the camera.

class YourFrameListener implements ProcessFrameListener {
...
public void didProcess(byte[] imageBuffer, int width, int height, ScanSession session) {
}
}

Careful: The ProcessFrameListener method is invoked on a picker-internal queue. To perform any UI work, you must dispatch to the main UI thread.


Converting the raw frame data to a Bitmap

To convert the frame data to a bitmap, you can use the YuvImage class, which is part of the Android SDK. This is illustrated in the following code-snippet:

@Override
public void didProcess(byte[] imageBuffer, int width, int height, ScanSession session) {
YuvImage image = new YuvImage(imageBuffer, ImageFormat.NV21, width, height, null);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
image.compressToJpeg(new Rect(0, 0, width, height), 100, stream);
byte[] jpegData = stream.toByteArray();
final Bitmap theBitmap = BitmapFactory.decodeByteArray(jpegData, 0, jpegData.length);
...
}

Note that this conversion is relatively slow, so make sure to only perform the conversion when you really have to. For example, it is generally a bad idea to perform the conversion every frame, if you only need an image of the frame in which the barcode was recognized.

Manually convert YUV420 to RGBA

The pixel data is stored in YUV 4:2:0 format. The Y information and UV information are spread across 2 image planes. First comes Y, then the interleaved UV data. The UV data is subsampled: A block of 2x2 pixels share the same UV information. To convert YUV420 to RGBA, the following code can be used:

static public void decodeYUV420(int[] rgba, byte[] yuv420, int width, int height) {
final int frameSize = width * height;
int r, g, b, y, uvp, u, v;
for (int j = 0, yp = 0; j < height; j++) {
uvp = frameSize + (j >> 1) * width;
u = 0;
v = 0;
for (int i = 0; i < width; i++, yp++) {
y = 0xff & ((int) yuv420[yp]);
if ((i & 1) == 0) {
v = (0xff & yuv420[uvp++]) - 128;
u = (0xff & yuv420[uvp++]) - 128;
}
r = y + (int)(1.370705f * v);
g = y - (int)((0.698001f * v) - (0.337633f * u));
b = y + (int)(1.732446f * u);
r = Math.max(0, Math.min(r, 255));
g = Math.max(0, Math.min(g, 255));
b = Math.max(0, Math.min(b, 255));
rgba[yp] = Color.argb(255, r, g, b);
}
}
}
@Override
public void didProcess(byte[] imageBuffer, int width, int height, ScanSession session) {
if (session.getNewlyRecognizedCodes().isEmpty()) return;
int[] rgba = new int[width*height];
decodeYUV420(rgba, imageBuffer, width, height);
final Bitmap theBitmap = Bitmap.createBitmap(rgba, width, height, Bitmap.Config.ARGB_8888);
...
}