Video Engine

A high-performance video analysis engine featuring GPU-accelerated machine learning for intelligent scene detection, object recognition, and automated library organization.

Features

• Frame extraction - Uniform sampling or keyframe/scene detection
• Video metadata - Resolution, duration, codec, FPS
• Scene change detection - Automatic keyframe extraction
• ML-based analysis (GPU-accelerated)
  • CLIP scene classification (20+ categories)
  • YOLO object detection (80 COCO classes)
  • Content quality metrics (blur, exposure, saturation)
• Batch processing - Optimized GPU utilization
• Embedding cache - Avoid re-analyzing frames
• SQLite database - Searchable indexing with tags
• File organization - Auto-organize videos by tags/quality/scene
• Configuration - YAML-based settings

Installation

pip install -r requirements.txt

For ML-based analysis, also install:

pip install torch transformers ultralytics

Quick Start

Command Line

# Show video metadata
python cli.py info video.mp4

# Analyze without ML (fast, basic scene detection)
python cli.py analyze video.mp4 --db videos.db

# Analyze with full ML pipeline (GPU-accelerated)
python cli.py analyze video.mp4 --ml --db videos.db

# Analyze with specific ML models
python cli.py analyze video.mp4 --ml --clip --yolo --quality --db videos.db

# Analyze directory (batch processing)
python cli.py analyze ./videos --ml --db videos.db --output results.json

# Search database
python cli.py search --db videos.db --min-duration 60 --resolution 1080p

# Organize videos by analysis tags (preview with --dry-run)
python cli.py organize --db videos.db --dest ./organized --pattern by-quality --dry-run

# Execute organization
python cli.py organize --db videos.db --dest ./organized --pattern by-quality

# Show database stats
python cli.py stats --db videos.db

Python API

from src.analyzers import VideoAnalyzer
from src.utils import get_video_info, extract_keyframes

# Get video metadata
info = get_video_info("video.mp4")
print(f"Duration: {info['duration_formatted']}")
print(f"Resolution: {info['resolution']}")

# Extract keyframes
keyframes = extract_keyframes("video.mp4", threshold=30.0)
print(f"Found {len(keyframes)} scenes")

# Full analysis (without ML)
analyzer = VideoAnalyzer()
result = analyzer.analyze("video.mp4")

Database Usage

from src.database import VideoDatabase

# Import analysis results
with VideoDatabase("videos.db") as db:
    db.import_analysis(results)

    # Search by tags
    fitness_videos = db.search(tags=["fitness", "exercise"])

    # Search by metadata
    long_4k = db.search(min_duration=300, resolution="4k")

CLI Commands

info

Display video metadata without analysis.

python cli.py info video.mp4 [--json]

analyze

Analyze video(s) with scene detection and optional ML.

python cli.py analyze <input> [options]

Options:
  -o, --output FILE          Save results to JSON
  --db FILE                  Import to database
  --max-frames N             Max frames per video (default: 50)
  --threshold N              Scene detection threshold (default: 30.0)
  --min-scene-duration N     Min scene duration in seconds (default: 1.0)
  --uniform                  Use uniform sampling vs keyframes
  --no-recursive             Don't search subdirectories
  -q, --quiet                Suppress progress output

  # ML Options
  --ml                       Enable ML analysis (auto-enables all models)
  --clip                     Use CLIP for scene classification
  --clip-model MODEL         CLIP model name (default: openai/clip-vit-base-patch32)
  --yolo                     Use YOLO for object detection
  --yolo-model MODEL         YOLO model variant (default: yolov8n.pt)
  --yolo-conf N              YOLO confidence threshold (default: 0.25)
  --quality                  Analyze content quality
  --blur-threshold N         Blur detection threshold (default: 100.0)
  --device DEVICE            Device for ML (auto/cuda/cpu, default: auto)

search

Query video database.

python cli.py search [options]

Options:
  --db FILE                  Database path (default: video_library.db)
  --tags TAG [TAG ...]       Search by tags
  --min-duration N           Minimum duration in seconds
  --max-duration N           Maximum duration in seconds
  --resolution RES           Filter by resolution (e.g., 1080p, 4k)
  --min-quality N            Minimum quality score
  --limit N                  Max results (default: 100)
  --json                     Output as JSON

organize

Organize videos based on analysis tags into directory structures.

python cli.py organize [options]

Options:
  --db FILE                  Database path (required)
  --dest DIR                 Destination root directory (required)
  --pattern PATTERN          Organization pattern (default: by-quality)
                             Patterns: by-quality, by-scene, by-activity,
                             by-resolution, quality-scene, resolution-quality,
                             comprehensive, or custom template
  --mode MODE                Operation mode: copy, move, symlink (default: copy)
  --duplicates MODE          Duplicate handling: skip, overwrite, rename (default: skip)
  --dry-run                  Preview operations without executing
  --rollback FILE            Rollback using history file
  --tags TAG [TAG ...]       Filter by tags
  --min-duration N           Minimum duration in seconds
  --max-duration N           Maximum duration in seconds
  --resolution RES           Filter by resolution
  --min-quality N            Minimum quality score
  --limit N                  Max videos to organize (default: 10000)

See WORKFLOW.md for complete workflow examples and patterns.

stats

Show database statistics and available tags.

python cli.py stats [--db FILE] [--json]

Architecture

nivo-flow/
├── cli.py                              # CLI with ML support
├── config.yaml                         # Configuration file
├── src/
│   ├── analyzers/
│   │   ├── video_analyzer.py          # Main pipeline (batch processing)
│   │   ├── clip_analyzer.py           # CLIP scene classification
│   │   ├── yolo_detector.py           # YOLO object detection
│   │   └── content_analyzer.py        # Quality metrics
│   ├── utils/
│   │   └── video_io.py                # Frame extraction & I/O
│   ├── database/
│   │   └── video_db.py                # SQLite storage & search
│   ├── workflow/
│   │   └── organize.py                # File organization system
│   ├── config.py                      # Configuration manager
│   └── cache.py                       # Embedding cache
├── examples/                           # Example scripts
└── tests/                              # Test suite

Requirements

• Python 3.10+
• OpenCV
• NumPy
• PyYAML

Optional (for ML analysis):

• PyTorch with CUDA support
• Transformers (for CLIP)
• Ultralytics (for YOLO)
• NVIDIA GPU with CUDA 12.1+

Additional Documentation

• FEATURES.md - Complete feature list and capabilities
• WORKFLOW.md - File organization workflows and patterns
• PERFORMANCE.md - Optimization guide and benchmarks
• CLAUDE.md - Project setup and development workflow
• examples/README.md - Example scripts usage

Benchmarking

Run performance tests:

python benchmark.py [video_file]

Creates synthetic test video if none provided. Outputs:

• Basic analysis timing
• ML analysis with/without cache
• Cache effectiveness (speedup)
• Results saved to benchmark_results.json