daniulive/SmarterStreaming
SmarterStreaming
SmarterStreaming is a cross-platform SDK for hardware-accelerated audio and video capture, encoding, and streaming. It provides a complete media pipeline for low-latency RTMP and RTSP streaming, and includes an embedded lightweight RTSP server that can serve live feeds directly from the source device to local network clients without requiring a separate server. The SDK also integrates with GB28181 surveillance platforms, enabling compliant device registration and streaming for standardized video monitoring systems.
The project distinguishes itself through a set of integrated capabilities for live production and distribution. It supports multi-protocol stream relaying, allowing streams to be pulled from one protocol and republished to another for cross-platform distribution. A real-time video compositing engine synthesizes multiple camera feeds, screen captures, and watermarks into a single output frame during live broadcasting. For interactive scenarios, it provides bidirectional audio intercom with echo cancellation and noise suppression, enabling one-on-one real-time communication sessions.
Beyond its core streaming and compositing functions, the SDK covers live stream playback with hardware-accelerated decoding, multi-protocol support, and network resilience features such as automatic reconnection and adaptation to fluctuating conditions. It includes local stream recording to files with support for multiple codecs and session control, as well as monitoring capabilities for stream connections, download speeds, and playback events through configurable callbacks. The SDK also offers Unity3D integration for rendering live video streams as textures within interactive applications.
Features
- Cross-Platform RTMP/RTSP Streaming SDKs - A cross-platform SDK for hardware-accelerated audio/video capture, encoding, and streaming.
- One-on-One Media Sessions - Establishes low-latency two-way audio/video calls using standard streaming protocols with echo cancellation.
- Video Frame Decoders - Decodes standard video formats using software or hardware acceleration.
- GB28181 Surveillance Registrations - Connects video devices to standardized surveillance platforms using SIP signaling and RTP transmission.
- RTMP Stream Pushing - Captures and transmits hardware-encoded audio and video to a remote RTMP server.
- Multi-Protocol Live Stream Players - Plays live streams over RTMP, RTSP, and HTTP-FLV protocols with low latency.
- Network Disruption Reconnections - Automatically reconnects after disconnection and adapts to varying network conditions during playback.
- GB28181 Surveillance Streaming - Receives live streams from GB28181 surveillance devices and processes them for further use.
- H.265 Video Decoders - Decodes H.264 and H.265 video streams using software or hardware acceleration.
- Hardware-Accelerated Encoders - Uses platform-specific hardware encoders for H.264 and H.265 to reduce CPU load.
- GB28181 Platform Integration - Registers camera or screen capture devices with GB28181-compliant surveillance platforms for streaming.
- Low-Latency Playback - Plays live streams with ultra-low latency, supporting multiple video codec extensions and concurrent playback.
- RTSP Streamers - Captures local audio and video and serves them as an RTSP stream accessible by local clients.
- Playback Clients - Plays RTSP live streams with ultra-low latency across multiple platforms including Unity3D.
- Playback Clients - Plays live streams with millisecond-level latency across public networks and local devices.
- Stream Compositing - Combines several live streams, local captures, or files into a single output stream for broadcasting.
- Stream Pushers - Captures and pushes live audio/video to streaming servers or CDNs for broadcasting.
- Low-Latency RTMP/RTSP Streams - Captures and pushes live audio/video with sub-second delay using RTMP and RTSP protocols.
- Low-Latency Video Streaming - Delivers live audio and video with end-to-end delay as low as 100 to 200 milliseconds.
- Multi-Source Video Composition - Combines camera feed, screen capture, and watermarks into a single output stream for recording or transmission.
- Multi-Format Inputs - Accepts input from cameras, screens, microphones, speakers, and external raw or encoded data.
- Protocol Routing & Forwarding - Transcodes and relays streams between different protocols like RTSP, RTMP, and GB28181 for multi-platform distribution.
- Embedded RTSP Servers - Exposes a built-in lightweight RTSP endpoint so clients can pull the live stream directly.
- Multi-Protocol Relays - Pulls streams from one protocol and republishes them to another for cross-platform distribution.
- H.265 Stream Decoders - Plays back live streams encoded with H.265 on supported platforms.
- Hardware-Accelerated Decoders - Uses software decoding for broad compatibility or hardware decoding on supported platforms to reduce CPU usage.
- Video Stream Compositing - Overlays or arranges multiple video sources into a single composited frame in real time.
- Hardware-Accelerated Playback - Uses platform-specific hardware decoders to reduce CPU load during video playback.
- GB28181 Device Integrations - Captures camera or screen data and registers it with a GB28181 surveillance platform.
- Camera Feed Capture - Captures live video from device cameras with front/rear camera switching for streaming.
- GB28181 Device Registrations - Registers Android devices as front-end units to GB/T 28181 video platforms for compliant live streaming and control.
- Audio and Video Forwarding - Captures live audio and video from local devices and encodes them using hardware acceleration for low-latency streaming.
- Noise Suppression Frameworks - Applies acoustic echo cancellation, noise suppression, and automatic gain control for clear two-way audio.
- Real-Time Video Overlayers - Adds text or image overlays onto decoded video frames during local preview for secondary processing.
- Two-Way Audio Talkbacks - Establishes real-time audio talkback channels between the platform and Android devices.
- Concurrent Stream Management - Manages and plays multiple live streams simultaneously within a single application instance.
- Simultaneous Video Stream Rendering - Decodes and renders multiple live streams at once for multi-view monitoring and video wall setups.
- Stream Recorders - Records live streams locally with support for multiple video codecs and audio format conversion.
- MP4 Recordings - Captures relayed streams and saves them directly as MP4 files on disk.
- Relay Recordings to MP4 - Saves relayed streams to MP4 files with status callbacks during forwarding.
- Media Session Recording Controls - Provides start, stop, pause, resume, and file-splitting controls for live stream recording sessions.
- Screen Capture Utilities - Captures the device screen for streaming, enabling remote desktop sharing or mobile screen mirroring.
- Media Streaming Renderers - Renders live streams within a Unity3D application across multiple platforms.
- Unity Integration SDKs - Provides an interface for Unity applications to use the streaming module for real-time video output.
- Echo-Cancelling Mixers - Establishes real-time two-way audio channels with acoustic echo suppression and noise reduction.
- Audio Blending - Blends audio from multiple inputs into a single output stream for multi-microphone scenarios.
- AAC and PCM Audio Decoders - Decodes AAC and PCM audio formats for real-time playback from cameras and encoders.
- Playback Controls - Mutes, unmutes, and adjusts volume in real time while a stream is playing.
- Playback Event Handlers - Reports network status, buffer state, and other playback events through configurable callbacks.
- Audio Processing - Applies noise reduction, automatic gain control, and voice activity detection to improve audio quality.
- Real-Time Microphone Captures - Captures live audio from a device microphone for inclusion in the live stream.
- Combined Microphone and System Audio Captures - Combines audio from the microphone and system speakers into a single stream for richer audio content.
- AAC Transcoders - Transcodes incoming audio formats like PCMA or PCMU to AAC before forwarding for broader compatibility.
- Unity3D Integrations - Supports Unity3D on multiple platforms for building cross-platform real-time video applications.
- Audio Transcoding - Converts incoming audio formats like PCMU and PCMA to AAC during live streaming.
- RTSP Audio Transcoders - Converts incoming audio formats such as PCMU, PCMA, or Speex to AAC before injecting them into the RTSP stream.
- Adaptive Bitrate Playback - Automatically reconnects and adapts playback to changing network conditions for stream continuity.
- Network Fluctuation Reconnections - Reconnects automatically and adapts to network jitter to maintain playback continuity.
- Network Instability Reconnections - Reconnects on disconnection and adapts to network fluctuations for playback continuity.
- Video Frame Rotators - Sets the rendered video to one of four rotation angles or applies horizontal/vertical mirroring.
- Dynamic Stream Adaptation - Automatically adjusts playback parameters in response to mid-stream changes in resolution or codecs.
- SEI Message Extensions - Embeds custom supplemental enhancement information into H.264 video streams for metadata.
- Variable Bitrate Encoding - Uses hardware-accelerated H.265 encoding or adjusts H.264 bitrate dynamically.
- Live Stream URL Switches - Changes the playback URL on the fly for multi-channel patrol or device switching without stopping playback.
- Screen Streams - Captures the device screen and pushes it as a live video stream for remote sharing.
- Remote Control Low-Latency Modes - Reduces end-to-end playback delay for real-time remote control and inspection scenarios.
- Local RTSP Gateways - Makes relayed streams available as RTSP URLs on the local network for client access.
- Protocol Bridging - Accepts RTSP input and re-publishes it as RTMP or other protocols for wider distribution.
- RTMP-to-RTSP Relays - Pulls RTSP or RTMP streams from the internet and re-serves them on a local RTSP server.
- RTSP-to-RTMP Relays - Pulls streams from RTSP or RTMP sources and republishes them to an RTMP server.
- Local Media Previews - Decodes and renders encoded media locally for real-time monitoring before or during transmission.
- HTTP-FLV Delivery - Plays incoming HTTP-FLV streams with low latency across multiple platforms.
- Live Stream Recorders - Saves incoming live streams to video files with support for multiple codecs and audio conversion.
- Automatic File Segmentation - Captures live media to local storage while automatically splitting files by size or URL changes.
- Intercom Audio Bridges - Establishes real-time audio talkback channels between the platform and Android devices.
- Broadcast Output Previews - Displays a real-time visual preview of the merged video stream before or during transmission.
- Automatic Reconnections - Automatically reconnects to the streaming server after a network drop to maintain stream reliability.
- Video Overlay Displays - Draws custom graphics on top of the video output for custom overlays during live broadcasting.
- Custom Data Stream Muxers - Embeds supplemental data into video streams for metadata or application-specific signaling.
- Dynamic Source Switching - Changes the source URL of a relay session on the fly without interrupting the broadcast.
- Stream Recording - Records audio/video streams to local files as a standalone module independent of push or playback.
- Real-Time Playback Controls - Mutes audio, changes volume, switches stream URLs, and rotates or mirrors video during playback.
- RTSP Media Servers - Restricts access to RTSP streams by requiring username and password from connecting clients.
- Authentication Mechanisms - Restricts playback of served RTSP streams by requiring username and password authentication.
- RTP-over-UDP Streamers - Streams real-time captured audio and video data to a platform using RTP over UDP or TCP.
- Runtime Encoding Configurations - Adjusts frame rate, bitrate, GOP, resolution, and codec to balance quality and performance.
- Video Frame Capture - Takes a snapshot of the current video frame while the stream is playing.
- Stable File Outputs - Handles network jitter, timestamp drift, and out-of-order packets to maintain synchronized audio-video and continuous timelines.
- First-Frame Optimizations - Displays the first video frame immediately upon connection for near-instant playback start.
- Video Mirroring Tools - Flips the video horizontally or vertically for front-facing cameras or special mounting angles.
- Video Playback Rendering Controls - Rotates the video display by standard angles and applies horizontal or vertical mirroring.
- Video Stream Metadata Embedding - Embeds custom text or binary data into video streams for real-time metadata delivery.
- Video Stream Metadata Parsing - Extracts and processes text or binary data from video stream extensions on the player side.
- Unity Texture Bridges - Renders live video streams as textures within the Unity engine for cross-platform interactive applications.
- Hardware-Accelerated Captures - Captures real-time video from local cameras with hardware acceleration for low-latency streaming.
- Media Channel Switching - Changes the playback URL on the fly for multi-channel patrolling without stopping the player.
- Network Reliability - Automatically reconnects after a network drop and reports connection status to maintain stream reliability.
- Jitter Adaptation Tools - Reconnects on disconnection and adapts to network jitter to maintain playback continuity.
- Automatic Reconnections - Reconnects automatically after disconnection and adapts to changing network conditions.
- Playback Monitoring - Reports network state, buffer status, and download speed through configurable event callbacks.
- Dynamic Watermark Overlays - Adds real-time text or image watermarks onto the video stream before encoding and pushing.
