Vision

pipecat.services.moondream.vision.detect_device()

Detects the appropriate device to run on, and return the device and dtype.

class pipecat.services.moondream.vision.MoondreamService(*, model='vikhyatk/moondream2', revision='2024-08-26', use_cpu=False, **kwargs)

Bases: VisionService

async run_vision(frame)

Process a vision image frame and generate results.

This method must be implemented by subclasses to provide actual computer vision functionality such as image description, object detection, or visual question answering.

Parameters:

frame (VisionImageRawFrame) – The vision image frame to process, containing image data.

Yields:

Frame – Frames containing the vision analysis results, typically TextFrame objects with descriptions or answers.

Return type:

AsyncGenerator[Frame, None]