Sequence Frame Source

Defined in library scandit_datacapture_core

SequenceFrameSource

class SequenceFrameSource : FrameSource

Added in version 8.6.0

Emits frames added via addFrame().

Use this class when your application, and not the Scandit Data Capture SDK, owns the camera.

That is the case when the frames come from a framework that runs its own capture session, such as ARKit on iOS or ARCore on Android, or from a camera plugin your application configures itself.

Register an instance of this class as the frame source via DataCaptureContext.frameSource, switch it to on, and then add the frames coming from the camera via addFrame(). The frame source never opens, selects or configures a camera itself — it only forwards the frames you push into it.

Frames reach the Scandit Data Capture SDK only while FrameSource.currentState is on and the frame source is attached to a DataCaptureContext, which is why the state has to be switched explicitly, just as it is for Camera. A frame added before both conditions are true stays in a short internal queue and is emitted as soon as they are. If the queue fills up, the oldest frames are dropped first. Switching the frame source off again stops processing without discarding that queue.

Only one frame is in flight at a time — the next queued frame is emitted once the DataCaptureContext has finished with the previous one. Pushing frames faster than they can be processed therefore drops frames rather than building up latency.

Note

Register the frame source and switch it on once, outside of your frame loop, and then call addFrame() for each frame. Do not create a frame source per frame, and do not set one on the DataCaptureContext from inside the frame loop. In particular, do not feed a continuous camera stream through ImageFrameSource — it is meant for single images and turns off after each one.

On this platform both mistakes are expensive rather than merely wasteful: setting a frame source re-serializes the whole DataCaptureContext, and ImageFrameSource encodes every image it is given to pass it over the bridge. Measured on a Flutter sample at 720p, the per-frame variant cost 520 ms per frame against 21 ms for the loop below.

Set up the frame source once, for example in your state’s initState:

final frameSource = SequenceFrameSource.create(CameraPosition.worldFacing);
await context.setFrameSource(frameSource);
await frameSource.switchToDesiredState(FrameSourceState.on);

Then, for every frame delivered by your camera, call addFrame() with the raw NV21 bytes:

await frameSource.addFrame(image.width, image.height, nv21Bytes);

The camera plugin delivers yuv420 frames whose planes are padded: each row is bytesPerRow bytes long rather than width bytes, and the chroma planes may be interleaved with a bytesPerPixel stride. addFrame() expects tightly packed NV21 — the full-resolution Y plane followed by interleaved V/U samples at half resolution — so the padding has to be removed and the chroma planes interleaved first:

Uint8List toNv21(CameraImage image) {
  final width = image.width;
  final height = image.height;
  // Chroma dimensions round up, so odd frame sizes are handled correctly.
  final chromaWidth = (width + 1) ~/ 2;
  final chromaHeight = (height + 1) ~/ 2;
  final nv21 = Uint8List(width * height + chromaWidth * chromaHeight * 2);

  // Y plane: copy row by row, dropping the row padding.
  final y = image.planes[0];
  var offset = 0;
  for (var row = 0; row < height; row++) {
    nv21.setRange(offset, offset + width, y.bytes, row * y.bytesPerRow);
    offset += width;
  }

  // Chroma planes: NV21 expects V and U interleaved, V first.
  final u = image.planes[1];
  final v = image.planes[2];
  final uPixelStride = u.bytesPerPixel ?? 1;
  final vPixelStride = v.bytesPerPixel ?? 1;
  for (var row = 0; row < chromaHeight; row++) {
    var uIndex = row * u.bytesPerRow;
    var vIndex = row * v.bytesPerRow;
    for (var column = 0; column < chromaWidth; column++) {
      nv21[offset++] = v.bytes[vIndex];
      nv21[offset++] = u.bytes[uIndex];
      uIndex += uPixelStride;
      vIndex += vPixelStride;
    }
  }

  return nv21;
}

Configure your camera controller with ImageFormatGroup.yuv420 so that the frames arrive in this layout, and skip a frame rather than queueing it if the previous addFrame() call has not completed yet.

create(position, lensPosition)

static SequenceFrameSource create(CameraPosition position,
        {double? lensPosition})

Added in version 8.6.0

Constructs a new SequenceFrameSource for the given CameraPosition. The lens position (0.0-1.0) sets the capture device lens position on iOS and is ignored on Android.

The capture device position neither selects nor opens a camera. Instead, it tells the Scandit Data Capture SDK how to interpret the frames you push: frames declared as coming from a user-facing camera are treated as mirrored. Pass the position of the camera your application is actually reading from.

desiredState

FrameSourceState get desiredState

Added in version 8.6.0

Implemented from FrameSource. See FrameSource.desiredState.

currentState

Future<FrameSourceState> get currentState

Added in version 8.6.0

Implemented from FrameSource. See FrameSource.currentState.

context

DataCaptureContext? context

Added in version 8.6.0

Implemented from FrameSource. See FrameSource.context.

addListener(listener)

void addListener(FrameSourceListener? listener)

Added in version 8.6.0

Implemented from FrameSource. See FrameSource.addListener().

removeListener(listener)

void removeListener(FrameSourceListener? listener)

Added in version 8.6.0

Implemented from FrameSource. See FrameSource.removeListener().

switchToDesiredState(state)

Future<void> switchToDesiredState(FrameSourceState state)

Added in version 8.6.0

Convenience method for FrameSource.switchToDesiredState(): it is the same as calling FrameSource.switchToDesiredState() without a completion handler.

addFrame(width, height, frameData)

Future<void> addFrame(int width,
        int height,
        Uint8List frameData)

Added in version 8.6.0

Adds a frame with the given width and height. The frame data must be the raw NV21 bytes of the frame. If this frame source is on and connected to a DataCaptureContext this is the next frame that will be processed.