Skip to main content

Models in Filters

This guide explains how machine learning models fit into Plainsight Filters. We’ll briefly introduce what a model is, how it’s typically created, and how you can integrate it into a filter. We’ll also cover how you can bake a model directly into the filter Docker image for versioning and distribution.

1. What Is a Machine Learning Model?​

A machine learning model is essentially a set of learned parameters (weights, biases) that can make predictions based on data input. In the context of computer vision, models learn to interpret images or video frames—for example:

  • Object Detection: Locating specific objects and labeling them (e.g., bounding boxes).
  • Image Classification: Classifying an entire image into categories (dog, cat, car, etc.).
  • Instance Segmentation: Identifying object boundaries as polygons.
  • Keypoint Detection: Pinpointing landmarks on people or objects (e.g., body joints).

You can find countless resources online discussing the fundamentals of machine learning and computer vision:

These resources will help you understand the basic pipeline of collecting data → training a model → deploying a model.

2. Working with PyTorch Models in Plainsight Filters​

Plainsight Filters rely heavily on PyTorch for deep learning tasks, though the underlying framework is flexible enough to handle other backends. Typically:

  1. Load your PyTorch model inside a filter (e.g., a Docker container).
  2. Receive video frames (or images) from an upstream source (another filter).
  3. Run inference on each frame.
  4. Pass the resulting subject data (e.g., bounding boxes, classes, or segmentation masks) downstream.

Key Points​

  • You can integrate any model—classification, detection, segmentation, etc.
  • The filter can load the model from a local path in the container (baked in) or from a remote location (JFrog, GCP bucket).
  • Protege is Plainsight’s recommended tool to train PyTorch models for use in filters, but if you already have a pretrained model from a third-party library or custom code, that’s fine too.

3. Training Models with Protege (Optional)​

Protege is Plainsight’s command-line tool for training models in the cloud. It simplifies the pipeline from:

  1. Collecting data (often labeled in Encord or similar)
  2. Spinning up a GCP VM with GPUs
  3. Running your training job
  4. Exporting a final artifact (e.g., .zip) with weights

Once you have a trained artifact, you can embed it into your filter’s Docker image, or load it at runtime.

4. YOLO Support & Licensing​

Many developers are familiar with YOLO (You Only Look Once) for object detection. However, note:

  1. YOLOv8 is GPL-licensed (version of GPL 3). This means incorporating YOLOv8 directly into a commercial product can create licensing obligations.
  2. Plainsight filters do not provide official YOLO-based application filters out-of-the-box. If you want to integrate YOLO, you must carefully review licensing and possibly host it in a separate container or set up a bridging mechanism.

Our basic recommendation is that while YOLO is easy to get started, it probably should not be used in filters.

5. Baking Models into the Docker Image​

Typically, when you’re ready to deploy a filter with a specific model version, you can bundle (bake) the model file inside the filter’s Docker image:

  1. Download or copy the model artifact (e.g., model.zip) into your filter repository.
  2. Update the Dockerfile to copy that artifact into the image:
FROM python:3.11
# ...
COPY model.zip /app/model.zip
# Install dependencies...
  1. In your filter code, load /app/model.zip or the extracted weights.

This ensures:

  • Version Control: Each Docker tag corresponds to a specific model version.
  • Portability: Deploying the filter container automatically includes the correct model.

Alternatively, you can pull the model at runtime from an artifact registry (JFrog, GCP, etc.) if you prefer more dynamic updates.

Best Practices​

  • Use environment variables or config files to specify which model path to load.
  • Keep large model files under revision control with caution—often, you store them in an artifact repository, not Git, for size reasons.
  • Tag your Docker image with semantic versioning (e.g., filter-faceblur:1.0.0) to indicate the filter + model pairing.

6. Model Versioning​

When your model improves over time, you can:

  1. Bump the filter image version (e.g., from 1.0.0 to 1.1.0)
  2. Swap in the new model artifact.
  3. Rebuild & publish the Docker image.

This lets you maintain a clean history of model improvements. If something breaks or performance regresses, you can roll back to a known-good version.

7. Summary​

  • Machine Learning Models are at the heart of computer vision tasks. They learn patterns from labeled data and produce inferences on new frames.
  • Plainsight Filters can load any PyTorch-based model—your own or something trained with Protege.
  • YOLO integration requires care due to licensing (particularly YOLOv8’s GPL license). Plainsight does not provide official YOLO filters.
  • Baking a model into the Docker image ensures consistent versioning and easy deployment.

With this approach, you can develop, train, and deploy your ML models seamlessly into a filter-based pipeline—whether you use Protege or any other training methodology. Just be sure to keep track of model versions, handle licensing responsibly, and enjoy the benefits of modular filter architecture!