Skip to main content

Filters Overview

What Are Filters?​

Filters are Docker-based computer vision applications that process images or video data frame by frame and facilitate communication between multiple stages in a vision pipeline. They are designed to be modular, composable, and scalable, enabling developers to create sophisticated vision workloads by chaining Filters together.

Depending on your use case, a Filter may or may not use GPU acceleration:

  • GPU-Accelerated Filters: Ideal for tasks that use machine learning models .
  • CPU-Based Filters: Useful for simpler operations, or when GPUs are not available.

Filters can be deployed in various environments, including:

  • Vision Stream – For real-time streaming inference.
  • Vision Flow – For batch or asynchronous processing.
  • Vision Edge – For on-device or edge computing deployments.

How Filters Work​

Filters operate within a directed graph, where each Filter performs a specific function, transforms data, and passes it downstream. Depending on their role, Filters may generate, modify, or transmit different types of data through the pipeline.

Types of Data in a Filter Pipeline​

Filters handle three primary types of data:

  1. Subject Data: This is structured information about what is happening in a video frame. Subject data is usually the output of a specific computer vision application. Examples include:

    • Counting objects (e.g., how many cows are in frame).
    • Detecting objects and their positions (e.g., where cars are located in a parking lot).
    • Recognizing text, faces, or other features.

  2. Visual Data: Visual Data is Image/Video Data, including the raw or processed images and videos used within the vision pipeline. Filters can work on various formats, applying transformations or analyses to improve downstream processing.

  3. Telemetry: This is operational data about the filter itself, providing insights into:

    • Model performance (if applicable)
    • Resource usage (CPU, GPU, memory).
    • Processing times and frame rates.

    The filter framework provides some standard metrics as well as tools for logging, emitting data lineage events and outputting traces.

Filters can pass subject data and image/video data down the pipeline, allowing subsequent Filters to refine, analyze, or store the results.

Machine Learning Models in Filters​

Some Filters contain embedded machine learning models, while others do not. For Filters that include models, the Plainsight filter framework provides:

  • Easy Model Loading & Versioning: Built-in utilities let you package, version, and load models without complicated boilerplate.
  • Flexible Model Support: Filters can embed a wide range of model architectures—image classification, instance segmentation, object detection, keypoint detection, or other specialized CV techniques.
  • Interpretation & Post-Processing: Filters often include logic to interpret model outputs—like object tracking, counting, or geometric calculations (e.g., trigonometry for measuring distances).

Types of Filters​

Filters are categorized based on their role within the pipeline. Plainsight provides a library of Filters across these categories:

1. Application Filters​

Application Filters are responsible for generating subject data. They analyze video frames and extract structured information that is passed downstream or stored for further processing. These Filters can also pass along image data if required by other Filters in the chain.

Examples:

  • Object Detection Filters (e.g., detecting people, animals, or vehicles).
  • Text Recognition Filters (OCR).
  • Face Detection or Attribute Extraction Filters.
  • Tracking or Counting Filters (e.g., object tracking, crowd counting).

2. Utility Filters​

Utility Filters do not usually generate subject data but perform useful transformations or operations on video and image data. These Filters enhance, modify, or otherwise preprocess the media before it reaches application Filters or final outputs.

Examples:

  • De-warping Filters – Correct fisheye lens distortion from cameras.
  • Frame Deduplication Filters – Remove duplicate frames to optimize processing.
  • Test Data Display Filters – Visualize bounding boxes, metadata, or debug overlays.

3. Connector Filters​

Connector Filters are typically the last stage in a vision pipeline, responsible for sending processed data to a final destination. They may output subject data, image/video data, or telemetry to an external system.

Examples:

  • Cloud Storage Connectors – Save processed images/videos to cloud services like GCS or S3.
  • Message Queue Connectors – Publish subject data to Kafka, MQTT, or other messaging systems.
  • Database Connectors – Store structured results in a database for further analysis.

Utilities for Testing​

The Plainsight filter framework also includes testing utilities to help you:

  • Validate that your Filters produce correct subject data outputs.
  • Verify that model-loading logic and GPU usage work as expected.
  • Check performance metrics like latency and throughput under different loads.

By streamlining test setups and providing out-of-the-box performance metrics, you can quickly iterate on your pipeline, fix bugs, and ensure reliable operation in production.