Research paper reads - Video Pipeline architecture

Paper #1 - VideoPipe: Building Video Stream Processing Pipelines at the Edge

Key Notes

• Challenges are resource limitations, computationally expensive machine learning algorithms
• Edge Devices have limited battery, processing power, network latency
• Optimize Models - Model quantization and model compression

Edge Analytics Challenges

• Support heterogeneous devices
• Support high frame rates
• Avoid delays perceivable by the user
• Video frames will flow through the modules
• Pipeline processing distributed across devices
• Different services/models - object detection, face detection, activity recognition, and object tracking

Message Transfer Protocol

• ZeroMQ - high-performance asynchronous messaging library
• Images that are passed between devices are encoded/decoded and transferred using ZeroMQ
• Evaluation Metrics - Time for Load Frame, Pose, Activity Detect, Total Duration

Time Analysis

• Frames per second
• Encode / decode
• Model Loading
• Model Inference
• Result
• Network Latency
• Round Trip Time

Alternative Solutions

• Vigil
• VideoEdge
• Chameleon
• GStreamer
• Gstreamer drawback - did not support video processing pipelines across multiple devices

Paper #2 - Enabling Scalable Edge Video Analytics with Computing-InNetwork
Requirements

• Adaptive to real-time video content
• Leveraging real-time feedback from the consumer

Configuration parameters

• Resolution
• Frames rate
• Object detector

Periodic reprofiling to decide on optimal parameters. Driver video streaming by server-side logic

Paper #3 - EdgeEye - An Edge Service Framework for Real-time Intelligent Video Analytics
Key Notes

• Deployable components with minimal effort
• Optimized inference engines
• GStreamer for pipeline
• Kurento is an open source WebRTC [4] media server

More Reads
Networked Cameras Are the New Big Data Clusters
Edge Enhanced Deep Learning System for Large-scale Video Stream Analytics
Chameleon: Scalable Adaptation of Video Analytics
SIAT: A Distributed Video Analytics Framework for Intelligent Video Surveillance
Webrtc on rasberry pi, gstreamer
Webrtc streams for surveillance
aiortc is a library for Web Real-Time Communication (WebRTC) and Object Real-Time Communication (ORTC) in Python

Keep Thinking!!!