# Fine-tuning and training > Search results for `Fine-tuning and training` on awesome-repositories.com. 114 total matches; showing the first 50. Explore on the web: https://awesome-repositories.com/q/fine-tuning-and-training **Attribution required: if you use, quote, or summarise this content, you must credit and link back to [this search on awesome-repositories.com](https://awesome-repositories.com/q/fine-tuning-and-training).** ## Results - [hiyouga/llama-efficient-tuning](https://awesome-repositories.com/repository/hiyouga-llama-efficient-tuning.md) (72,239 ⭐) — This project is a fine-tuning framework and training pipeline designed to optimize and adapt large language and vision models. It provides a specialized toolkit for parameter-efficient tuning and supervised learning, serving as both a trainer for multimodal models and a deployment tool for serving fine-tuned models via high-performance inference engines. The framework focuses on reducing memory and compute requirements by updating a small subset of model parameters. It supports a wide range of adaptation strategies, including vision-language model training to align text, image, video, and audio data, as well as preference alignment to match model behavior with human expectations. The system covers a broad set of capabilities including supervised fine-tuning, instruction tuning, and core pre-training. It incorporates memory optimization through quantization and weight-merging pipelines, alongside data management for importing and preparing custom datasets. For operational management, it includes a web-based interface for task execution and integration with external dashboards for experiment metric tracking. The project provides utilities for exporting model checkpoints and deploying tuned models as web services using standardized, OpenAI-compatible API interfaces. - [mlabonne/llm-course](https://awesome-repositories.com/repository/mlabonne-llm-course.md) (80,178 ⭐) — This project is a comprehensive educational curriculum and engineering handbook focused on the lifecycle of large language models. It serves as a structured knowledge base for machine learning practitioners, covering the fundamental mathematical and architectural principles of transformer-based sequence modeling, as well as the practical implementation of supervised instruction fine-tuning and preference-based model alignment. The repository distinguishes itself by providing a deep dive into advanced model composition and optimization techniques. It details methodologies for weight-space model merging and mixture-of-experts strategies, alongside practical guidance on low-precision parameter quantization and inference optimization to manage hardware requirements. Furthermore, it explores the development of autonomous agentic systems capable of tool-use orchestration and the construction of retrieval-augmented generation pipelines to ground model outputs in external data. The content spans the entire technical stack, from foundational deep learning concepts and neural network design to the complexities of deploying, evaluating, and securing models in production environments. It includes a curated collection of technical articles, blog posts, and interactive notebooks that track state-of-the-art research trends and experimental methodologies in generative artificial intelligence. - [d2l-ai/d2l-en](https://awesome-repositories.com/repository/d2l-ai-d2l-en.md) (29,001 ⭐) — This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flexible model development through modular layer composition, deferred parameter initialization, and symbolic graph hybridization, which balances the ease of imperative coding with the performance benefits of compiled execution. The project covers a broad capability surface, including computer vision, natural language processing, recommender systems, and reinforcement learning. It provides infrastructure for data pipeline management, gradient-based optimization, and distributed training across multiple hardware accelerators. Users can leverage built-in utilities for hyperparameter tuning, model regularization, and performance monitoring to diagnose and refine their architectures. The documentation is delivered as a series of interactive notebooks that can be executed locally or on remote cloud infrastructure, providing a standardized interface for deep learning research and experimentation. - [kvcache-ai/ktransformers](https://awesome-repositories.com/repository/kvcache-ai-ktransformers.md) (17,288 ⭐) — Ktransformers is a comprehensive framework designed for the operation, fine-tuning, and serving of large language models. It functions as a heterogeneous inference engine and quantized execution runtime, enabling the deployment of massive models by distributing computational workloads across both CPU and GPU resources. This architecture allows users to bypass local memory constraints, making it possible to run and train models that exceed the capacity of a single device. The project distinguishes itself through specialized support for sparse architectures, particularly mixture-of-experts models. It employs pipelined expert offloading and layer-wise sharding to balance memory usage and processing speed across heterogeneous hardware. By utilizing hardware-specific kernel optimizations, such as specialized instruction sets for server processors, the framework maximizes throughput for both inference and fine-tuning tasks. Beyond its core execution capabilities, the project provides a production-ready serving environment that exposes models via an OpenAI-compatible HTTP interface. It includes a suite of command-line tools for managing model deployments, configuring system environments, and performing performance benchmarking. The framework also supports the integration of custom inference kernels and operator injection, allowing for architectural modifications and fine-tuned control over model placement strategies. - [jamescj60/universal-x86-tuning-utility](https://awesome-repositories.com/repository/jamescj60-universal-x86-tuning-utility.md) (2,452 ⭐) — Universal-x86-Tuning-Utility is a system tuning tool for x86 hardware that adjusts CPU, GPU, and memory settings to optimize performance and power consumption. It provides an adaptive power optimization algorithm that dynamically adjusts processor power limits based on real-time temperature monitoring, balancing performance with thermal safety margins. The utility also includes a hardware specification viewer that displays detailed system information for reference. The tool distinguishes itself through event-driven profile automation, which applies pre-configured tuning profiles automatically when specified system events occur, enabling hands-off performance management. It offers both premade tuning presets tailored for specific use cases and the ability to create custom tuning profiles by configuring advanced parameters like power limits, voltages, and clock speeds. A built-in game launcher scans for installed game executables and presents them in a browsable catalog for one-click launch with applied tuning profiles. The utility supports tuning both CPU and GPU performance, including undervolting to reduce temperatures while maintaining stable operation. It also provides an AMD Zen tuning preset manager for quickly achieving optimized performance on AMD Zen-based processors. The documentation covers installation and usage of the application's tuning capabilities. - [jpmorganchase/python-training](https://awesome-repositories.com/repository/jpmorganchase-python-training.md) (12,714 ⭐) — This project is a comprehensive educational curriculum designed to teach Python programming through the lens of data science and financial analysis. It provides a structured guide for learning how to process complex numerical information, build data models, and perform scientific computing tasks using standard industry libraries. The materials focus on practical applications, enabling users to develop skills in financial data analysis and interactive exploration. By working through these resources, learners gain experience in executing high-performance mathematical operations, transforming raw datasets, and creating graphical representations to identify trends and patterns. The repository consists of a collection of interactive notebooks that facilitate iterative development and real-time visualization. These educational materials are organized to support the transition from fundamental programming concepts to advanced workflows involving large-scale data processing and quantitative decision-making. - [holms-ur/fine-tuning](https://awesome-repositories.com/repository/holms-ur-fine-tuning.md) (72 ⭐) — Close-Domain fine-tuning for table detection - [mudler/localai](https://awesome-repositories.com/repository/mudler-localai.md) (46,889 ⭐) — LocalAI is a self-hosted inference server that enables the execution of machine learning models directly on local hardware. By providing a unified interface for text, image, and audio processing, it allows users to maintain full control over data privacy and infrastructure costs while eliminating dependencies on external network services. The platform functions as an API gateway that mimics standard cloud-based artificial intelligence interfaces, allowing existing applications to integrate local models as drop-in replacements. It utilizes a container-based architecture to package runtimes and dependencies, ensuring consistent deployment across diverse hardware configurations. To optimize system performance, the server employs an on-demand orchestration layer that dynamically loads and unloads models based on active requests, minimizing memory usage during periods of inactivity. The system supports a wide range of model architectures through a flexible backend abstraction that allows for driver switching at runtime. Users can manage their models and interact with the service through a web interface or via standard web requests, which the proxy translates into model-specific execution commands. The software is distributed as a containerized application to facilitate deployment across various server and cloud environments. - [nndl/llm-beginner](https://awesome-repositories.com/repository/nndl-llm-beginner.md) (6,421 ⭐) — This project is a collection of educational resources and technical guides focused on the development and implementation of large language models. It provides a comprehensive curriculum covering transformer architectures, training methods, and deployment strategies. The materials provide detailed instructions for building autonomous agents using reasoning loops and tool integration, as well as guides for fine-tuning models through supervised learning and preference optimization. It also includes tutorials for constructing retrieval augmented generation pipelines and implementing transformer models from scratch. The project covers a broad range of capabilities, including model architecture design, parameter-efficient tuning, and the creation of vector-based retrieval systems. It further addresses natural language processing tasks such as text classification, semantic analysis, and tokenization, alongside methods for monitoring model performance through execution tracing and attention visualization. - [datawhalechina/self-llm](https://awesome-repositories.com/repository/datawhalechina-self-llm.md) (30,941 ⭐) — This project is an open-source educational resource providing structured, step-by-step guides for fine-tuning large language models. It focuses on adapting pre-trained transformer-based causal models to custom datasets, enabling users to transfer specific writing styles or domain knowledge into generative AI models. The repository distinguishes itself by emphasizing parameter-efficient training techniques, specifically low-rank adaptation. By providing practical implementations for updating only a small subset of model weights, it allows for the customization of massive neural networks on consumer-grade hardware. The guides cover the entire machine learning workflow, including instruction-based dataset formatting, configuration of training parameters, and the use of gradient accumulation to manage memory constraints. The documentation provides a comprehensive technical walkthrough for the fine-tuning process, from environment setup and data preparation to model training and weight saving. It includes specific code examples for loading models in half-precision formats and configuring training arguments to optimize performance for various tasks. - [tsinghuac3i/intuitive-fine-tuning](https://awesome-repositories.com/repository/tsinghuac3i-intuitive-fine-tuning.md) (0 ⭐) — This repository contains the code for the paper "Intuitive Fine-Tuning: Towards Simplifying Alignment into a Single Process". - [kohya-ss/sd-scripts](https://awesome-repositories.com/repository/kohya-ss-sd-scripts.md) (7,133 ⭐) — sd-scripts is a suite of utilities designed for fine-tuning generative models, preprocessing datasets, and converting model weights. It provides a collection of scripts for executing Stable Diffusion training through methods such as DreamBooth, textual inversion, and full fine-tuning, alongside a framework for creating and managing Low-Rank Adaptation weights. The project features specialized capabilities for model weight conversion between different architectures and precision formats. It includes tools for merging adaptation weights into base models, extracting weights from trained models, and integrating multiple adaptation modules to blend styles or concepts. The toolkit covers a broad range of generative AI operations, including image dataset preparation with automated tagging and aspect ratio bucketing, and various inference methods such as text-to-image, image-to-image, and inpainting. It also incorporates memory and performance optimizations, including VRAM management, latent caching, and just-in-time training acceleration. The software provides integration for synchronizing training states and model checkpoints with the Hugging Face Hub. - [zasder3/train-clip](https://awesome-repositories.com/repository/zasder3-train-clip.md) (0 ⭐) — A PyTorch Lightning solution to training CLIP from both scratch and fine-tuning. - [ml-explore/mlx-examples](https://awesome-repositories.com/repository/ml-explore-mlx-examples.md) (8,254 ⭐) — This repository provides a collection of reference implementations and code examples for training and deploying machine learning models using the MLX framework. It serves as a practical guide for executing distributed training, fine-tuning large language models, converting model weights, and implementing multimodal generative workflows. The project distinguishes itself through specialized examples for local hardware execution, featuring weight quantization to reduce memory usage and low-rank adaptation for parameter-efficient fine-tuning. It also includes scripts for transforming external model formats into MLX-compatible versions and merging adapter weights for standalone deployment. The examples cover a broad range of capabilities, including natural language processing with decoder-only and mixture-of-experts architectures, computer vision for image classification and segmentation, and audio processing for speech-to-text and music generation. Additionally, it demonstrates generative AI workflows for text-to-image and text-to-video synthesis, alongside graph-based neural networks and multimodal systems that utilize shared embedding spaces. - [huggingface/transformers](https://awesome-repositories.com/repository/huggingface-transformers.md) (161,630 ⭐) — Transformers is a comprehensive library for machine learning that provides a unified interface for training, fine-tuning, and deploying transformer-based models. It supports a wide range of tasks, including text classification, language modeling, question answering, and sequence-to-sequence translation, while offering specialized architectures for both text and vision processing. The framework includes tools for managing the entire model lifecycle, from data preprocessing and tokenization to distributed training and inference. The library features extensive support for model optimization and performance, including techniques like quantization, speculative decoding, and paged memory management for key-value caches. It provides native integration for distributed training across multi-node clusters, as well as flexible APIs for serving models via compatible inference servers. Developers can also utilize built-in utilities for model patching, custom kernel execution, and automated documentation generation to streamline development workflows. - [microsoft/unilm](https://awesome-repositories.com/repository/microsoft-unilm.md) (22,030 ⭐) — This project is a comprehensive framework and toolkit for developing, optimizing, and deploying transformer-based models across multimodal, document intelligence, and natural language processing tasks. It provides a unified neural architecture that processes text, vision, audio, and document layout data through a shared set of weights, enabling researchers and developers to build foundational models that align cross-modal representations. The platform distinguishes itself through advanced training and inference strategies designed for large-scale deep learning. It incorporates specialized mechanisms such as retentive state processing for efficient sequence generation, differential attention for improved focus, and distributed weight partitioning to handle memory-intensive computations. These capabilities are complemented by techniques for sparse decoding and model compression, which maintain performance while reducing the computational footprint of large-scale architectures. The project covers a broad capability surface, including end-to-end pipelines for data curation, synthetic data generation, and tokenization across diverse modalities. It supports extensive workflows for pre-training, instruction tuning, and fine-tuning, with specific focus areas in document understanding, speech synthesis, and cross-lingual transfer. Diagnostic tools for attention analysis and benchmarking further assist in evaluating model performance on complex reasoning and retrieval tasks. - [thudm/p-tuning-v2](https://awesome-repositories.com/repository/thudm-p-tuning-v2.md) (2,078 ⭐) — An optimized deep prompt tuning strategy comparable to fine-tuning across scales and tasks - [microsoft/generative-ai-for-beginners](https://awesome-repositories.com/repository/microsoft-generative-ai-for-beginners.md) (112,045 ⭐) — This project is a comprehensive, open-source educational curriculum designed to guide developers through the mastery of generative artificial intelligence. It provides a structured learning path that covers foundational concepts, prompt engineering, and the practical application of large language models. The repository serves as a central hub for skill acquisition, offering sequential modules that progress from basic model mechanics to advanced architectural patterns. The curriculum distinguishes itself by focusing on the end-to-end lifecycle of intelligent software, including the implementation of retrieval-augmented generation and agentic workflow orchestration. It provides technical guidance on integrating diverse models—ranging from open-source options to cloud-based services—while emphasizing responsible development through systematic safety guardrails and ethical design practices. Learners are equipped to build functional applications, such as conversational interfaces, semantic search tools, and automated content generators, using standardized interfaces and modern development techniques. Beyond core model implementation, the resource covers operational practices for monitoring and maintaining AI systems in production. It includes practical modules on fine-tuning, vector-based indexing, and designing intuitive user experiences for intelligent systems. The repository is structured to support developers through every stage of the process, from initial environment configuration and dependency management to deployment readiness and troubleshooting. - [conardli/easy-dataset](https://awesome-repositories.com/repository/conardli-easy-dataset.md) (13,394 ⭐) — Easy-dataset is a comprehensive platform designed for the end-to-end management of machine learning datasets, specifically tailored for language and vision model fine-tuning. It functions as a centralized environment for the entire data lifecycle, encompassing the automated generation of synthetic training data, the structural organization of document collections, and the systematic annotation of individual data points. The platform distinguishes itself through its integrated evaluation and orchestration capabilities. It provides a dedicated suite for benchmarking models, featuring blind side-by-side human testing and automated grading to ensure objective performance metrics. Users can orchestrate complex data pipelines that transform raw documents into structured formats through recursive segmentation, automated taxonomy classification, and customizable text refinement. Beyond core generation and management, the system supports a wide range of data processing tasks, including visual document extraction, content augmentation, and the creation of multi-turn conversational datasets. It offers flexible configuration for model connections and generation parameters, allowing for fine-grained control over output quality and consistency. The platform is designed for local deployment to maintain data privacy and security. It includes built-in tools for programmatic quality assessment and supports the export of processed datasets into standard formats compatible with various fine-tuning pipelines. - [lightly-ai/lightly-train](https://awesome-repositories.com/repository/lightly-ai-lightly-train.md) (1,582 ⭐) — All-in-one training for vision models (YOLO, ViTs, RT-DETR, DINOv3): pretraining, fine-tuning, distillation. - [espnet/espnet](https://awesome-repositories.com/repository/espnet-espnet.md) (9,861 ⭐) — ESPnet is a comprehensive speech processing toolkit and PyTorch-based trainer designed for building end-to-end speech recognition, synthesis, and translation models. It provides a structured framework for developing automatic speech recognition systems using transducer and encoder-decoder architectures, alongside engines for text-to-speech synthesis and speech translation pipelines. The project distinguishes itself through a recipe-based workflow execution system that ensures experimental reproducibility by running standardized sequences of scripts for data preparation and model training. It leverages containerized environments to provide consistent execution across platforms and supports large-scale distributed training across multiple GPUs and nodes. The toolkit covers a broad range of capabilities, including spoken language understanding for intent and sentiment classification, audio enhancement and separation, and singing voice synthesis. It also incorporates advanced training techniques such as self-supervised learning, parameter-efficient fine-tuning, and transfer learning. Model development is supported by utilities for audio data formatting, spectral augmentation, and the integration of pretrained encoders, while inference is optimized through blockwise beam search for real-time streaming execution. - [openai/openai-cookbook](https://awesome-repositories.com/repository/openai-openai-cookbook.md) (74,196 ⭐) — This project is a technical learning resource and developer knowledge base focused on the integration of large language models into software applications. It provides a structured collection of guides and code examples designed to teach developers how to implement intelligent features using proven patterns and best practices. The repository distinguishes itself through a library of functional demonstrations that cover complex topics such as retrieval-augmented generation, function calling, and prompt engineering workflows. These materials are organized into a modular structure, allowing for the rapid development and testing of prototypes and proof-of-concept applications before moving toward production-ready software. The content is delivered as a version-controlled knowledge base, utilizing markdown-based documentation and executable code blocks. These resources are designed to be copied directly into external development environments or cloud-based notebooks for hands-on experimentation. The entire collection is compiled into a static site to ensure consistent accessibility and navigation. - [hiyouga/chatglm-efficient-tuning](https://awesome-repositories.com/repository/hiyouga-chatglm-efficient-tuning.md) (3,720 ⭐) — Fine-tuning ChatGLM-6B with PEFT | 基于 PEFT 的高效 ChatGLM 微调 - [allenai/olmo](https://awesome-repositories.com/repository/allenai-olmo.md) (6,313 ⭐) - [oxford-cs-deepnlp-2017/lectures](https://awesome-repositories.com/repository/oxford-cs-deepnlp-2017-lectures.md) (15,854 ⭐) — This repository is a deep learning for natural language processing course and curriculum. It provides educational material and guides focused on neural network architectures used for processing natural language, speech signals, and text classification. The content includes instructional tutorials on sequence modeling and neural language modeling, covering the implementation of n-gram and recurrent neural networks. It also provides a framework for studying word embeddings to map linguistic meanings into numerical representations. The curriculum covers a broad range of capabilities, including speech signal processing, text classification workflows, and the implementation of sequence models. Additionally, it includes technical guidance on deep learning hardware optimization to improve memory bandwidth and throughput during model execution. - [rednaga/training](https://awesome-repositories.com/repository/rednaga-training.md) (431 ⭐) — Training materials crafted and publicly provided by Red Naga members - [facebookresearch/fairseq](https://awesome-repositories.com/repository/facebookresearch-fairseq.md) (32,228 ⭐) — Fairseq is a PyTorch toolkit for sequence-to-sequence modeling, specializing in neural machine translation, automatic speech recognition, and large-scale language model training. It provides a framework for processing and aligning diverse data sources, including text, audio, and video, to support tasks such as speech-to-text conversion and multimodal sequence learning. The project is distinguished by its distributed training capabilities, which utilize parameter sharding, mixed-precision training, and CPU offloading to handle models that exceed single-device memory. It also includes specialized tools for data engineering, such as parallel data mining for unsupervised learning and back-translation for expanding training corpora. Its capability surface extends to comprehensive inference and generation tools, including beam search and lexical constraint enforcement, as well as model compression techniques like layer pruning and product quantization. The toolkit also provides utilities for feature extraction, model evaluation via metrics like perplexity and BLEU scores, and a registry-based system for extending models and tasks. Training and evaluation workflows are managed through a command-line interface that orchestrates hyperparameter configuration and model execution. - [deepspeedai/deepspeed](https://awesome-repositories.com/repository/deepspeedai-deepspeed.md) (42,528 ⭐) — DeepSpeed is a high-performance library designed to scale deep learning model training and inference across massive clusters of GPUs and compute nodes. It provides a comprehensive suite of tools for distributed training, enabling the execution of models that exceed the memory capacity of single devices through advanced parameter partitioning, pipeline-based model parallelism, and memory-efficient state offloading. The framework distinguishes itself through specialized communication-efficient optimizers and hardware-aware acceleration techniques. By utilizing gradient compression, quantization, and custom-compiled kernels, it minimizes network bandwidth bottlenecks and maximizes computational throughput. It further supports complex architectures like mixture-of-experts and long-context models by integrating sequence parallelism and sparse attention mechanisms, ensuring efficient resource utilization across heterogeneous hardware topologies. Beyond its core training capabilities, the project includes a robust set of utilities for automated performance tuning, model profiling, and universal checkpointing. It provides infrastructure support for diverse processor architectures and cloud-based cluster deployment, allowing users to optimize execution environments through targeted kernel compilation and diagnostic monitoring. - [morizeyao/gpt2-chinese](https://awesome-repositories.com/repository/morizeyao-gpt2-chinese.md) (7,596 ⭐) — GPT2-Chinese is a Chinese language model implementation based on the GPT-2 architecture. It provides a causal language model trainer and a natural language generation tool designed for training and generating human-like Chinese text sequences. The system integrates a BERT tokenizer to process Chinese corpora into manageable units for machine learning. It enables the development of predictive text models that can generate specific patterns, such as news or poetry, through prompt-based text completion. The project covers a full workflow including text tokenization, model training using a transformer-based decoder, and performance evaluation via perplexity measurement. It also includes utilities for weight-based model serialization to support inference and checkpointing. - [thudm/p-tuning](https://awesome-repositories.com/repository/thudm-p-tuning.md) (0 ⭐) — 🌟 [2022-10-06] Thrilled to present GLM-130B: An Open Bilingual Pre-trained Model. It is an open-sourced LLM outperforming GPT-3 175B over various benchmarks. Get model weights and do inference and P-Tuning with only 4 RTX 3090 or 8 RTX 2080 Ti FOR FREE! - [paulescu/hands-on-train-and-deploy-ml](https://awesome-repositories.com/repository/paulescu-hands-on-train-and-deploy-ml.md) (885 ⭐) — Train and Deploy an ML REST API to predict crypto prices, in 10 steps - [aishwaryanr/awesome-generative-ai-guide](https://awesome-repositories.com/repository/aishwaryanr-awesome-generative-ai-guide.md) (24,755 ⭐) — This project is a community-driven knowledge repository and technical learning resource focused on the field of generative artificial intelligence. It serves as a centralized hub for developers and practitioners to access curated research, tutorials, and foundational concepts necessary for building and deploying modern artificial intelligence applications. The platform distinguishes itself through a collaborative, distributed contribution model that aggregates diverse learning materials into a structured, searchable knowledge base. It covers a wide range of specialized topics, including retrieval-augmented generation, large language model training, fine-tuning techniques, and agentic workflows. Beyond technical skill development, the repository functions as a professional development hub, offering interview preparation resources and guidance for those pursuing careers in the artificial intelligence industry. The content is organized through a hierarchical taxonomy, allowing users to navigate complex subjects such as system evaluation, multimodal models, and security tools. The repository provides access to comprehensive code notebooks and structured tutorials, all maintained as static documentation within a version control system to ensure accessibility and ease of discovery. - [yangjianxin1/firefly](https://awesome-repositories.com/repository/yangjianxin1-firefly.md) (6,642 ⭐) — Firefly is a training framework and inference engine for large language models. It functions as a toolkit for pre-training and fine-tuning various open-weight architectures, providing a system for model alignment and parameter-efficient fine-tuning. The project includes utilities for merging adapter weights back into base models to create standalone files. It also provides a model alignment toolkit to format training data according to specific prompt templates, ensuring conversational consistency across different models. The framework supports distributed model training and preference-based optimization. Inference capabilities include multi-turn dialogue execution with low-precision memory optimizations to reduce hardware requirements. - [appsecco/breaking-and-pwning-apps-and-servers-aws-azure-training](https://awesome-repositories.com/repository/appsecco-breaking-and-pwning-apps-and-servers-aws-azure-training.md) (952 ⭐) — Course content, lab setup instructions and documentation of our very popular Breaking and Pwning Apps and Servers on AWS and Azure hands on training! - [nirdiamant/agents-towards-production](https://awesome-repositories.com/repository/nirdiamant-agents-towards-production.md) (17,375 ⭐) — This project is a comprehensive framework for developing, orchestrating, and deploying autonomous agents. It provides a structured environment for building agents that utilize reasoning loops to perform multi-step tasks, manage state through graph-based workflows, and interact with external tools. By mapping unstructured model outputs into typed schemas, the framework ensures reliable integration with downstream application logic. The platform distinguishes itself through a focus on production-grade reliability and security. It incorporates hybrid memory systems that combine vector embeddings with structured knowledge graphs to maintain long-term context. To ensure operational safety, the framework includes built-in guardrails that intercept and validate inputs and outputs, mitigating risks such as injection attacks and enforcing strict security policies during agent execution. The system covers the entire agent lifecycle, including intelligent web scraping, retrieval-augmented generation, and containerized serverless deployment. It provides tools for monitoring agent performance, evaluating behavioral reliability, and managing complex multi-agent interactions. Developers can package these applications into portable container images for scalable execution, with built-in support for dynamic resource management and performance optimization in high-traffic environments. The repository is structured as a collection of Jupyter Notebooks that demonstrate the implementation of these agentic patterns and infrastructure components. - [paddlepaddle/lark](https://awesome-repositories.com/repository/paddlepaddle-lark.md) (7,717 ⭐) — LARK is a development toolkit for training, fine-tuning, and deploying large language models and multimodal models based on PaddlePaddle. It functions as a comprehensive framework that includes an LLM training orchestrator, an inference server, and a multimodal model framework for processing text, image, and video inputs. The project features a retrieval-augmented generation system for building conversational applications that integrate web search and private knowledge bases. It provides specific capabilities for multimodal reasoning and complex logic, enabling the extraction of structured data and visual knowledge from documents, charts, and images. The toolkit covers large-scale model training through supervised fine-tuning and preference optimization, as well as model compression via quantization to reduce memory usage. It includes production infrastructure for deploying inference servers with hardware acceleration and load balancing. A web-based graphical user interface is provided to control conversations and manage the training processes of vision-language models. - [alibaba/roll](https://awesome-repositories.com/repository/alibaba-roll.md) (2,844 ⭐) — ROLL is a distributed reinforcement learning framework and model alignment toolkit designed for large language models. It serves as a scalable training pipeline and GPU cluster manager, providing the infrastructure to align model behavior using reinforcement learning algorithms and preference optimization techniques. The project distinguishes itself through an agentic rollout orchestrator that generates and collects multi-turn interaction trajectories between AI agents and simulated environments. It supports specialized alignment methods including Direct Preference Optimization, reinforcement learning from verifiable rewards, and group-relative reward optimization. The framework covers a broad range of capabilities for large-scale distributed training, including tensor, pipeline, and expert parallelism to support ultra-large-scale models. It manages hardware resources through GPU multiplexing and disaggregated deployment, while providing tools for automated reward evaluation using code sandboxes and mathematical verification. Pre-configured environment deployments are provided for different GPU architectures and library versions to accelerate setup. - [raaminz/training](https://awesome-repositories.com/repository/raaminz-training.md) (0 ⭐) — This Repository is all about my training classes - [karpathy/mingpt](https://awesome-repositories.com/repository/karpathy-mingpt.md) (23,639 ⭐) — minGPT is a minimal implementation of the Transformer architecture designed for training and experimenting with language models. It functions as a neural network training framework and a text generation engine, providing the necessary tools to manage data loading, backpropagation, and parameter updates for custom deep learning models. The project is structured as an educational resource for understanding how transformer architectures function by building and training models from scratch. It utilizes a modular block architecture and transformer-based self-attention to process sequences, allowing users to define custom model configurations and execute the full training loop on their own datasets. Beyond its core training capabilities, the library supports byte-pair-encoding for text processing and provides mechanisms for serializing model parameters. It includes functionality for extending training logic through custom callbacks and packaging models for distribution, facilitating both neural network prototyping and text generation inference. - [timescale/timescaledb-tune](https://awesome-repositories.com/repository/timescale-timescaledb-tune.md) (501 ⭐) — A tool for tuning TimescaleDB for better performance by adjusting settings to match your system's CPU and memory resources. - [karpathy/build-nanogpt](https://awesome-repositories.com/repository/karpathy-build-nanogpt.md) (4,746 ⭐) — This is an educational implementation that builds a generative pre-trained transformer (GPT) language model from scratch using PyTorch. The project is structured as a step-by-step tutorial, walking through the construction of a decoder-only transformer architecture and its training loop with clean git commits and an accompanying video lecture for a hands-on learning experience. What sets this implementation apart is its focus on practical reproduction: it provides a workflow to train a 124-million-parameter model from scratch in about one hour on cloud GPU hardware, costing under ten dollars. The tutorial covers both the architecture construction and the full training pipeline, making it suitable for those who want to understand the inner workings of a GPT-scale model without relying on pre-built frameworks. The technical implementation covers the core components of a decoder-only transformer, including causal masked self-attention where each token attends only to preceding tokens, cross-entropy loss minimization for next-token prediction, weight-decay regularization to prevent overfitting, and GPU-accelerated training through PyTorch for large-scale computation. While the project is small in scale, it mirrors the architectural patterns used in larger language models. - [intel-analytics/ipex-llm](https://awesome-repositories.com/repository/intel-analytics-ipex-llm.md) (8,836 ⭐) — ipex-llm is an acceleration library and inference engine designed to optimize the execution and finetuning of large language models on Intel GPUs and NPUs. It provides a HuggingFace compatible model backend and a dedicated quantization toolkit for converting model weights into low-bit precision formats. The project facilitates distributed inference by splitting large model workloads across multiple accelerators using pipeline and tensor parallelism. It enables the deployment of models on Intel Arc, Flex, and Max GPUs to increase throughput and reduce latency. The library covers a broad range of optimization capabilities, including low-precision finetuning for local model updates and the loading of diverse community model formats. It also includes tools for measuring model predictive performance using standard perplexity metrics. - [fineuploader/fine-uploader](https://awesome-repositories.com/repository/fineuploader-fine-uploader.md) (8,149 ⭐) — Fine Uploader is a browser file upload widget and manager that provides a frontend interface for transferring multiple files. It functions as a chunked file upload manager and a client-side image processor. The project enables the direct transfer of files to cloud storage providers, specifically Amazon S3 and Microsoft Azure, to reduce the load on application servers. It includes tools for scaling and resizing image dimensions during the upload process to save bandwidth. The system manages large file transfers by splitting them into small pieces, allowing for pause and resume functionality. It also provides a drag-and-drop interface and progress tracking for multiple simultaneous uploads. - [ohf-voice/piper1-gpl](https://awesome-repositories.com/repository/ohf-voice-piper1-gpl.md) (2,897 ⭐) — This project is a neural text-to-speech system and voice trainer that converts written text into spoken audio across a variety of global languages and regional dialects. It functions as an ONNX-based engine capable of performing fast offline inference and uses a phoneme-based controller to manage precise pronunciation. The system distinguishes itself through a comprehensive toolkit for neural voice training, allowing for the creation of custom single-speaker or multi-speaker models. It supports the export of these models to a standardized open format and provides hardware acceleration via graphics processors to increase the speed of audio generation. The engine covers a wide range of synthesis capabilities, including real-time chunked audio streaming and file-based export. It provides granular control over vocal delivery through raw phoneme injection, punctuation-based prosody adjustments, and the modification of speaking speed and volume. - [ai4finance-foundation/fingpt](https://awesome-repositories.com/repository/ai4finance-foundation-fingpt.md) (20,507 ⭐) — FinGPT is a suite of specialized financial tools and a framework for adapting large language models to the financial domain. It provides a set of pipelines for financial entity extraction, sentiment analysis, and retrieval-augmented generation to improve the accuracy of financial information systems. The project distinguishes itself through efficient training workflows, utilizing low-rank adaptation and quantized low-rank adaptation to fine-tune models on consumer-grade hardware. It employs market-labeled datasets and reinforcement learning that uses actual stock price movements as reward signals to refine model performance. The framework covers broad capability areas including algorithmic trading signal generation, automated investment research, and stock price movement prediction. It also provides tools for collecting global financial data and generating source code for quantitative trading factors. The project is primarily implemented and demonstrated through Jupyter Notebooks. - [v-rusu/tuning-fork](https://awesome-repositories.com/repository/v-rusu-tuning-fork.md) (0 ⭐) — A configurable client-side JavaScript library for guitar tuning with real-time pitch detection. Supports standard and alternate tunings for guitar, bass, ukulele, banjo, and custom instruments. - [unslothai/unsloth](https://awesome-repositories.com/repository/unslothai-unsloth.md) (66,628 ⭐) — Unsloth is a high-performance training and inference platform designed to optimize the lifecycle of large language and multimodal models. It provides a comprehensive engine for fine-tuning, executing, and managing models locally, with a focus on reducing memory consumption and increasing compute speed on consumer-grade hardware. The platform distinguishes itself through hand-optimized kernels and automated computational graph techniques that maximize hardware throughput. It supports advanced training methodologies, including reinforcement learning for reasoning and efficient adapter-based fine-tuning, while offering a unified web-based interface for no-code model training, data preparation, and real-time performance monitoring. Beyond its core training capabilities, the project includes a local inference runtime that supports API-based deployment, tool-calling, and automated output verification. It manages the entire model development process, from dataset generation and hyperparameter configuration to model exporting and performance benchmarking across diverse hardware configurations. The software provides setup utilities for local development environments and includes diagnostic tools to assist with installation and hardware compatibility. - [llm-tuning-safety/llms-finetuning-safety](https://awesome-repositories.com/repository/llm-tuning-safety-llms-finetuning-safety.md) (0 ⭐) — Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! - [facebookresearch/map-anything](https://awesome-repositories.com/repository/facebookresearch-map-anything.md) (2,915 ⭐) — Map-anything is a 3D scene reconstruction framework and neural geometry estimator designed to transform two-dimensional images into metric three-dimensional spatial representations using feed-forward neural networks. It provides a specialized toolkit for predicting camera intrinsics and ray directions from single images without requiring external geometric metadata. The project includes a 3D model benchmarking suite that utilizes a unified model wrapper to standardize outputs from diverse reconstruction models. This allows for consistent evaluation and accuracy measurement across various spatial datasets. To facilitate downstream use, it includes a COLMAP data exporter that converts neural reconstruction predictions into formats compatible with photogrammetry and splatting pipelines. The framework covers a broad capability surface including distributed geometry model training, multi-node cluster orchestration, and inference memory optimization. It also provides tools for metric depth visualization, spatial data standardization, and geometry artifact filtering using normal-based masking. - [axolotl-ai-cloud/axolotl](https://awesome-repositories.com/repository/axolotl-ai-cloud-axolotl.md) (12,059 ⭐) — Axolotl is a configuration-driven framework designed for the fine-tuning, evaluation, and quantization of large language models. It functions as a comprehensive orchestrator for distributed training, enabling users to manage complex workflows across multi-node and multi-GPU environments. By utilizing structured configuration files, the platform streamlines the setup of training parameters, dataset paths, and hardware distribution strategies. The project distinguishes itself through its support for diverse training methodologies, including full-parameter tuning, parameter-efficient adaptation, and reinforcement learning alignment. It provides specialized capabilities for multimodal model training, allowing for the integration of text, image, and media inputs. Furthermore, the framework includes advanced optimization tools such as quantization-aware training, which simulates precision loss to maintain model accuracy, and dynamic reward signal integration for aligning model behavior with human preferences. The framework covers a broad capability surface, including data management, performance optimization, and model lifecycle management. It handles data ingestion, preprocessing, and streaming, while offering advanced techniques like sequence packing and replay buffers to improve training efficiency. Performance is managed through distributed parallelism strategies, memory-efficient training pipelines, and custom kernel implementations. The project provides pre-configured container images to ensure consistent deployment across local and cloud-based compute environments. Users can manage the entire model lifecycle, from initial configuration and training to adapter merging and final inference execution.