Fannovel16/comfyui_controlnet_aux
Comfyui Controlnet Aux
Acest proiect este o suită de preprocesare ComfyUI ControlNet și un toolkit de analiză computer vision. Acesta funcționează ca un preprocesor de imagini stable diffusion care extrage indicii structurale din imagini pentru a ghida fluxurile de lucru de latent diffusion.
Sistemul oferă modele specializate pentru estimarea posturii umane, inclusiv keypoints scheletice și mesh-uri faciale, precum și maparea scenelor 3D prin estimarea adâncimii și a normalelor suprafeței. Include, de asemenea, instrumente pentru controlul mișcării video AI folosind analiza fluxului optic.
Suprafața mai largă de capabilități acoperă analiza structurală a imaginilor — cum ar fi line art, extragerea marginilor și segmentarea semantică — precum și izolarea luminanței și generarea de indicii de culoare și stil. Aceste procese sunt susținute de runtime-uri accelerate hardware și caching-ul model-checkpoint pentru a reduce latența de inferență.
Features
- ControlNet Preprocessing - Generates structural hint images and maps from source photos to guide the generation of AI images.
- Latent Diffusion Preprocessors - Extracts structural hints from images to guide and modulate latent diffusion workflows.
- Computer Vision Toolkits - Provides a comprehensive suite of models for semantic segmentation, optical flow, and human pose detection.
- ControlNet Guidance - Feeds processed hint images into secondary networks to modulate and constrain the primary generative process.
- 3D Spatial Mapping - Creates depth and surface normal maps to capture the spatial layout and geometry of an image for AI.
- Structural Analysis - Extracts line art, semantic segments, and luminance maps to define boundaries and lighting for editing.
- Surface Normal Estimators - Produces depth and surface normal maps to capture the three dimensional spatial layout of a scene.
- Human Pose Estimation - Detects skeletal keypoints and facial meshes to map human posture and expression for character control.
- ComfyUI Custom Node Suites - Provides a collection of auxiliary nodes for generating depth, pose, and edge maps within ComfyUI.
- Edge Extraction Utilities - Generates edge-based hint images to preserve structural boundaries and outlines within a visual composition.
- Image Guidance Map Generation - Processes raw images through specialized models to create guidance maps that direct neural network spatial layouts.
- Video Motion Controllers - Analyzes optical flow and pixel motion to ensure smooth transitions in AI generated videos.
- Luminance Extraction - Isolates lighting and intensity information to separate brightness from color for flexible recoloring.
- Feature Extraction Pipelines - Combines outputs from multiple specialized networks to capture depth, normals, and skeletal data simultaneously.
- Hardware-Accelerated Inference - Implements optimized compute backends to execute neural network operations for faster pose and depth estimation.
- Semantic Image Segmentations - Classifies image regions into specific categories to create labeled masks for targeted area editing.
- Structural Image Generation - Processes input images to create structural hints that enforce geometric or semantic layouts.
- Hardware-Accelerated Implementations - Implements optimized runtimes to reduce latency during skeletal keypoint detection.
- Inference Optimizations - Uses hardware-accelerated runtimes to increase pose detection speed and reduce inference latency.
- Pixel Motion Analysis - Estimates pixel motion between frames using optical flow to create smooth transitions in AI videos.
