Using Docker Compose with Plainsight Filters

Below is a high-level guide on using Docker Compose to run Filters, manage configuration via environment files (.env), and handle different deployment scenarios. You can adapt this template to your own project’s needs.

1. Basic Structure of a `docker-compose.yaml`

A typical Docker Compose file includes:

Services: Each service corresponds to one or more Filter containers or auxiliary containers (e.g., for data storage, message queues, or other support services).
Networks: Usually one or more shared networks that let the filters communicate.
Volumes: Shared volumes for passing data or storing logs and telemetry.

Here’s a simplified snippet you might see in a docker-compose.yaml for running Filters:

version: "3.9"

services:
  video_in:
    image: plainsight.jfrog.io/tenant-lab-docker/filters/base/video_in:1.4.4
    environment:
      FILTER_ID: video_in
      # Pull values from .env
      FILTER_SOURCES: "${VIDEO_INPUT:-file:///input.mp4!sync}"
      FILTER_OUTPUTS: "tcp://*"
    ports:
      - "5550:5550"
      - "5551:5551"
    volumes:
      - ./data:/app/data  # example volume for local video files
    networks:
      - filter-network

  filter_vizcal:
    image: plainsight.jfrog.io/tenant-lab-docker/filters/vizcal:1.0.2
    environment:
      FILTER_ID: filter_vizcal
      FILTER_SOURCES: "tcp://video_in" 
      FILTER_OUTPUTS: "tcp://*"
      # other environment variables...
    networks:
      - filter-network

  webvis:
    image: plainsight.jfrog.io/tenant-lab-docker/filters/base/webvis:1.4.4
    environment:
      FILTER_ID: webvis
      FILTER_SOURCES: "tcp://filter_vizcal"
    ports:
      - "8000:8000"
    networks:
      - filter-network

networks:
  filter-network:

Key Points

Each service points to a Docker image.
Environment variables like FILTER_SOURCES and FILTER_OUTPUTS control how filters connect to each other.
ports: can be mapped to expose a filter’s internal ports for debugging or visualization (e.g. webvis).
All containers share a single filter-network for easy internal communication.

2. Creating and Using Environment (`.env`) Files

Why .env files?

Centralize configuration details (e.g., image versions, environment variables).
Keep secrets out of version control.
Let you easily switch configurations between dev/staging/production.

Example `.env`

# Image Versions
FILTERS_VERSION_BASE=1.4.4
VIZCAL_VERSION=1.0.2

# Video Files or RTSP Streams
VIDEO_INPUT=file:///home/user/videos/sample.mp4!sync

# Webvis Port
PORT=8000

# Optional JFROG API key for private images
JFROG_API_KEY=example123

# GPU usage toggle
# If you have an Nvidia GPU, set this to 'all' or 'none'
GPU_COUNT=none

Within your docker-compose.yaml, you can reference these variables:

services:
  video_in:
    image: "plainsight.jfrog.io/tenant-lab-docker/filters/base/video_in:${FILTERS_VERSION_BASE}"
    environment:
      FILTER_SOURCES: "${VIDEO_INPUT}"
      FILTER_OUTPUTS: "tcp://*"
    # ...

Tip: Docker Compose automatically loads variables from a file named .env in the same directory as your docker-compose.yaml. If you rename it, you’ll need to pass --env-file <filename> on the command line.

3. Connecting Filters Together

Plainsight Filters communicate via ZeroMQ. You configure them by setting FILTER_SOURCES and FILTER_OUTPUTS:

FILTER_OUTPUTS="tcp://*": The filter binds to a local TCP port (by default 5550, plus 5551 for the request/response channel). Downstream filters can connect to it.
FILTER_SOURCES="tcp://service_name": Connect to a service whose FILTER_OUTPUTS is tcp://*.

In Docker Compose:

Name your upstream service’s container (e.g. video_in).
Your downstream service sets FILTER_SOURCES: "tcp://video_in".

When Docker Compose starts, it resolves video_in to the network alias for that container. Under the hood, ZeroMQ sees tcp://video_in:5550.

Topics & Subscriptions

Each Filter can publish different “topics.”
Default topics for images (e.g., main), plus _metrics topics for telemetry.
You can subscribe to them with syntax like tcp://video_in;main.
For more advanced usage, you can do ephemeral connections (? or ??) or topic remapping (topic_in>topic_out).

4. Common Configurations

Real-Time RTSP Input

environment:
  FILTER_SOURCES: "rtsp://username:[email protected]:554/Streaming/Channels/1!sync!maxfps=15"

!sync processes all frames without skipping.
!maxfps=15 caps the reading at 15 frames per second.

File-Based Input

# in .env
VIDEO_INPUT=file:///app/data/video.mp4!sync!maxfps=10

# in docker-compose.yaml
environment:
  FILTER_SOURCES: "${VIDEO_INPUT}"
  FILTER_OUTPUTS: "tcp://*"

GPU Acceleration
If your filters require a GPU:

Install the NVIDIA Container Toolkit on the host.

Add a deploy: block to your filter service:

filter_gpuload:
  image: ...
  deploy:
    resources:
      reservations:
        devices:
          - driver: nvidia
            count: ${GPU_COUNT:-all}
            capabilities: [gpu]

Or simply pass --gpus all via Compose overrides if your Docker version supports it.

Output to Cloud Storage

filter_out:
  image: ...
  environment:
    FILTER_SOURCES: "tcp://some_filter"
    FILTER_OUTPUTS: "file://somefile.mp4"
    # or "gs://bucket/filename.mp4!segtime=00:05" for 5-minute segments

Make sure your credentials (e.g. google_application_credentials.json) are mounted and your environment is set:

volumes:
  - "./google_application_credentials.json:/google_application_credentials.json:ro"

5. Managing Different Deployment Scenarios

Many teams maintain multiple Compose files or environment files for dev, staging, and production. Typical patterns include:

Single Compose, Multiple `.env`

Keep a single docker-compose.yaml.
Provide multiple .env files: .env.dev, .env.staging, .env.prod.

Deploy with:

docker compose --env-file .env.dev up

docker compose --env-file .env.prod up -d

Multiple Compose Files

A base docker-compose.yaml plus environment-specific overrides:
- docker-compose.dev.yaml
- docker-compose.prod.yaml

Then:

docker compose -f docker-compose.yaml -f docker-compose.prod.yaml up

Example: Local vs. RTSP

docker-compose-video.yaml might read from a local file.
docker-compose-rtsp.yaml might read from an RTSP stream.
Both share the same webvis and utility filter definitions.

This pattern is especially helpful if you have large differences in your pipeline structure or external dependencies.

6. Troubleshooting

Images Not Found
- Make sure you are logged in to the correct registry (e.g., docker login plainsight.jfrog.io) and your .env points to valid image tags.
Filters Exiting Immediately
- Check logs: docker compose logs <service-name>.
- Typical reasons:
  - Wrong FILTER_SOURCES (camera not reachable, file not found).
  - Invalid environment variable referencing an empty string.
Ports Already In Use
- If you see address in use, another container or process might be binding to the same port. Adjust ports: or FILTER_OUTPUTS: to avoid conflict.
RTSP Connection Fails
- Could be firewall or camera credentials. Test with a direct RTSP client (like VLC) to confirm.
GPU Not Detected
- Ensure nvidia-smi works on the host.
- Check Docker logs for “Could not select device driver” messages.
- Confirm you have the right Docker Compose deploy: config or the --gpus all parameter.

Example: Putting It All Together

Here’s a simplified example with a video input and two filters—VizCal (a utility filter for camera movement detection) and Webvis for live preview:

version: "3.9"

services:
  video_in:
    image: "plainsight.jfrog.io/${ARTIFACTORY_USERNAME:-tenant-lab}-docker/filters/base/video_in:${FILTERS_VERSION_BASE}"
    environment:
      FILTER_ID: "video_in"
      FILTER_SOURCES: "${VIDEO_INPUT:-file:///app/data/video.mp4!sync}"
      FILTER_OUTPUTS: "tcp://*"
    volumes:
      - "./data:/app/data"
    networks:
      - filter-network

  vizcal_filter:
    image: "plainsight.jfrog.io/${ARTIFACTORY_USERNAME:-tenant-lab}-docker/filters/vizcal:${VIZCAL_VERSION}"
    environment:
      FILTER_ID: "vizcal_filter"
      FILTER_SOURCES: "tcp://video_in"
      FILTER_OUTPUTS: "tcp://*"
      # e.g., pass thresholds, etc.:
      # VIZCAL_THRESHOLD=5
    networks:
      - filter-network

  webvis:
    image: "plainsight.jfrog.io/${ARTIFACTORY_USERNAME:-tenant-lab}-docker/filters/base/webvis:${FILTERS_VERSION_BASE}"
    environment:
      FILTER_ID: "webvis"
      FILTER_SOURCES: "tcp://vizcal_filter"
      # By default listens on :8000
    ports:
      - "${PORT:-8000}:8000"
    networks:
      - filter-network

networks:
  filter-network:

Steps to Run

Create a .env file:

FILTERS_VERSION_BASE=1.4.4
VIZCAL_VERSION=1.0.2
VIDEO_INPUT=file:///home/ubuntu/test.mp4!sync
PORT=8000

Log in to JFrog (if your images are private):

docker login plainsight.jfrog.io
# enter username/password

Start Up:
```
docker compose up -d
```
Check Logs:
```
docker compose logs -f
```
Open Browser:
- Go to http://<server-ip>:8000 to view the webvis preview (if you mapped port 8000).

If all is correct, you’ll see logs indicating the filters connected to each other. Your local video file (test.mp4) will stream from video_in → vizcal_filter → webvis.

Summary

Using Docker Compose with Plainsight Filters involves:

Defining services in docker-compose.yaml (video inputs, transform filters, connectors, visualizers).
Configuring each filter via environment variables for input, output, and any model or special parameters.
Optionally customizing .env files to handle different environments or data sources.
Checking logs and container states if anything fails.

Once you get comfortable with this setup, you can add more advanced features like ephemeral subscribers, GPU-based filters, or dynamic load balancing. Docker Compose remains a convenient tool for local dev, basic edge deployments, or even cloud-based solutions—so long as you keep your configurations consistent and well-documented.

Need more help?

Refer to Plainsight’s internal docs for advanced pipeline topics like ephemeral sources, balancing, or multi-filter synchronization.

For private images, ensure your docker login credentials are valid.

For GPU usage, confirm your host machine has the NVIDIA drivers + Container Toolkit.

That’s it! You now have a framework to build, run, and manage your filter pipelines using Docker Compose and environment files.

1. Basic Structure of a docker-compose.yaml​

Key Points​

2. Creating and Using Environment (.env) Files​

Example .env​

3. Connecting Filters Together​

Topics & Subscriptions​

4. Common Configurations​

5. Managing Different Deployment Scenarios​

Single Compose, Multiple .env​

Multiple Compose Files​

Example: Local vs. RTSP​

6. Troubleshooting​