28 repository-uri
Mechanisms for exchanging large data arrays between managed and native memory using direct buffers.
Distinct from Shared Memory Data Exchange: Distinct from general shared memory exchange: focuses on Java-to-native array transfer.
Explore 28 awesome GitHub repositories matching data & databases · Direct Memory Data Transfer. Refine with filters or upvote what's useful.
Sharp is a high-performance image processing library for Node.js. It serves as a native extension and wrapper for the libvips framework, providing tools for image resizing, format conversion, and programmatic data manipulation. The project enables the transformation of images into web-friendly formats such as WebP and AVIF while preserving color profiles and alpha channels. It also provides capabilities for generating blank image buffers with specified dimensions and background colors. The library covers a broad range of image manipulation utilities, including rotation, extraction, compositi
Implements direct memory data transfer between the Node.js heap and native memory for high-performance image data handling.
This project is a comprehensive JavaScript programming tutorial and language reference. It serves as a web development education resource providing instruction on modern language fundamentals, object-oriented design, and advanced asynchronous programming patterns. The resource functions as both a frontend development guide and a technical reference. It covers core language features such as closures, prototypes, promises, and typed arrays, while providing practical lessons on managing browser data and handling network requests. The content spans several key capability areas, including browser
Teaches the use of fixed-type views over raw binary buffers to handle numeric data.
LibGDX is a Java-based framework designed for cross-platform game development, enabling the creation and deployment of 2D and 3D games across desktop, mobile, and web environments from a single codebase. It functions as a comprehensive library that abstracts hardware-accelerated graphics, audio, input, and file system access, providing a unified interface for developers to manage game logic and application lifecycles. The framework distinguishes itself through a high-performance architecture that prioritizes efficiency and native interoperability. It utilizes a batch-oriented graphics pipelin
Exchanges large data arrays between Java and native code using direct memory buffers to minimize overhead.
This library is a JavaScript framework for general-purpose computing on graphics processing units. It enables the execution of parallel mathematical operations directly within the browser by offloading data-heavy calculations to graphics hardware. The project functions as a web-based math accelerator that converts standard JavaScript functions into shader code for execution on the graphics processor. It provides a unified interface that detects available graphics APIs and manages data transfer between system and graphics memory. To ensure compatibility across diverse environments, the library
Manages data transfer between system and graphics memory using structured buffers to minimize latency.
sql.js is a serverless, in-memory SQL database engine that ports SQLite to WebAssembly for use within a web browser. It provides a JavaScript interface to initialize relational databases, execute SQL queries, and manage structured data without requiring a backend server. The project enables the import and export of database states using typed arrays, allowing in-memory data to be persisted as files. It supports high-precision integer retrieval via BigInt and utilizes prepared statements to increase execution speed and security. The engine includes capabilities for client-side data management
Uses JavaScript typed arrays to map the database's linear memory and internal state for efficient access.
LearningNotes este o bază de cunoștințe tehnice și un ghid de studiu de inginerie axat pe internele framework-ului Android, arhitectura sistemului și optimizarea performanței mobile. Servește drept referință pentru analizarea secvenței de boot Android, bootstrapping-ul proceselor și inițializarea serviciilor de sistem. Proiectul oferă ghiduri detaliate despre performanța mobilă, inclusiv strategii pentru reducerea amprentei de memorie, identificarea scurgerilor de memorie și optimizarea decodării imaginilor. Acoperă, de asemenea, comunicarea inter-proces Android folosind AIDL și driverul de kernel Binder, precum și manuale de arhitectură software pentru decuplarea logicii de business de interfețele utilizator prin tipare precum MVVM și MVP. Dincolo de dezvoltarea mobilă, repository-ul include o bază de cunoștințe de informatică pentru pregătirea interviurilor tehnice, acoperind structuri de date, algoritmi și concepte de sistem de operare. De asemenea, dispune de o referință practică pentru controlul versiunilor Git, detaliind gestionarea repository-ului, sincronizarea și fluxurile de lucru de branching.
Reduces data copying overhead during cross-process communication by mapping memory between kernel and user spaces.
xLua is a scripting bridge and C++ wrapper used to embed the Lua language into host applications. It facilitates bidirectional data exchange and function calls between scripts and the host environment. The project includes a runtime patching tool for replacing application logic and fixing bugs without requiring a system restart. It features a coroutine orchestrator that wraps asynchronous operations into linear code and a script validator that verifies digital signatures to ensure code authenticity and integrity before execution. To minimize memory overhead and garbage collection, the system
Passes complex data structures via raw memory pointers to avoid expensive object allocation and garbage collection overhead.
This project is a collection of PyTorch learning resources and educational guides designed to teach the construction and training of neural networks. It serves as a comprehensive deep learning tutorial covering various model architectures and practical implementation strategies. The resources provide specific guidance on implementing computer vision tasks, such as image classification and synthetic imagery generation, as well as reinforcement learning agents using value networks and experience replay. It also covers sequential data modeling through recurrent networks and generative modeling u
Provides utilities for transferring tensors between GPU and CPU memory to enable operations not supported on graphics hardware.
JerryScript is a lightweight, ECMAScript-compliant JavaScript engine and bytecode compiler designed for resource-constrained devices. It serves as an embedded interpreter and IoT scripting runtime, enabling the execution of JavaScript code within native C applications on hardware with limited memory. The project differentiates itself through a focus on low-memory runtime management, utilizing bytecode precompilation and pre-compiled state snapshots to reduce startup time and memory overhead. It features a C-binding native bridge for bidirectional communication between native code and scripts,
Constructs JavaScript typed arrays directly from raw memory buffers using specific offsets and lengths.
sjcl is a JavaScript cryptography library providing a collection of primitives for encryption, hashing, and encoding within a web browser. It functions as an AES symmetric encryption tool, a cryptographic hashing library, and a Base32 data encoder. The project provides implementations for the Advanced Encryption Standard to secure data through symmetric key encryption and decryption. It also enables the generation of fixed-length data fingerprints to verify information integrity and authenticity. The library covers a broader range of security capabilities, including client-side data hashing,
Uses JavaScript typed arrays to manage raw bytes and binary buffers for efficient memory access.
Goja is a JavaScript engine and ECMAScript compliant interpreter implemented entirely in Go. It serves as an embedded scripting engine that allows Go applications to execute JavaScript code and integrate a programmable scripting layer without relying on Cgo or external native dependencies. The project functions as a bridge between Go and JavaScript, enabling bidirectional data exchange and function invocation. It allows Go hosts to expose native structs, slices, and maps as JavaScript objects and arrays, while providing mechanisms to export script values and functions back into native Go type
Handles fixed-length binary data buffers and views using standardized integer array types.
This project is a learning guide and collection of study notes designed to teach Node.js backend development. It provides a comprehensive core API reference and practical demonstrations for implementing server-side logic, network programming, and system APIs. The guide specifically covers advanced technical domains including process management for scaling applications via clusters and child processes, as well as network programming for building TCP, UDP, and HTTP services. It also includes detailed instructional material on security implementation, focusing on cryptographic hashing and encryp
Implements raw byte handling using binary buffers for memory-efficient data manipulation.
Transfers decoded image data directly to CV-CUDA, PyTorch, or CuPy without copying through host memory.
Pako is a pure JavaScript compression library that ports the C zlib library to JavaScript, providing deflate and gzip compression and decompression capabilities. It runs in both browser and Node.js environments by using typed arrays and universal JavaScript, avoiding platform-specific APIs for cross-platform compatibility. The library handles data through bitwise operations for Huffman coding and LZ77 matching, and automatically encodes string inputs to UTF-8 before compression while decoding them back on decompression. It supports streaming chunk-based processing, allowing incremental data h
Uses Uint8Array for raw binary data input and output, avoiding string conversions for performance.
ssh2 is a JavaScript implementation of the SSH2 protocol for Node.js, providing the core components necessary to create secure clients and servers. It enables the establishment of authenticated sessions for remote server automation and secure communication. The project distinguishes itself by providing a complete suite of tools for identity management, including utilities for generating and parsing cryptographic key pairs and integrating with external authentication agents. It also functions as a tunneling proxy capable of routing TCP, HTTP, X11, and SOCKSv5 network traffic through encrypted
Implements low-level binary parsing using buffers to handle the SSH protocol's packet structures.
gl-matrix is a high-performance JavaScript library for vector and matrix math, purpose-built for real-time 3D graphics and physics simulations. It stores all vectors and matrices as typed arrays (Float32Array or Float64Array) in column-major order, matching the memory layout expected by OpenGL and WebGL shaders without requiring transposition. The library is implemented from scratch with zero external dependencies, keeping its bundle size minimal for web and Node.js environments. The library distinguishes itself through its immutable operation pattern, where each math function returns a new t
Computes vector and matrix operations with typed arrays for high-performance 3D graphics and physics calculations.
wxBot este un framework și toolkit pentru automatizarea mesajelor WeChat. Oferă o interfață programatică pentru a trimite și primi mesaje, permițând crearea și implementarea de chatbot-uri automatizați și boți de social media în cadrul platformei WeChat. Sistemul integrează servicii și scripturi externe pentru a automatiza fluxurile de lucru bazate pe chat și comunicațiile utilizatorilor. Permite operarea programatică a boților pentru a gestiona conversațiile și notificările în mod automat.
Implements binary packet parsing to decode structured data from the WeChat network buffer.
SynapseML este o bibliotecă de machine learning Apache Spark concepută pentru construirea și scalarea fluxurilor de lucru de machine learning și a pipeline-urilor de date în clustere distribuite. Servește drept framework de pipeline distribuit de machine learning și motor de inferență distribuit pentru executarea predicțiilor accelerate hardware și a sarcinilor de deep learning pe seturi de date la scară largă. Proiectul funcționează ca un strat de integrare AI în cloud, permițând utilizatorilor să aplice servicii de inteligență artificială pre-antrenate pentru text, viziune și vorbire în cadrul pipeline-urilor distribuite. Include, de asemenea, o suită dedicată de instrumente pentru detectarea distribuită a anomaliilor pentru a identifica outlierii multivariati și de serie temporală în date de înaltă dimensionalitate. Biblioteca acoperă o gamă largă de capabilități, inclusiv viziune computerizată distribuită pentru analiza feței și a imaginilor, procesarea limbajului natural scalabilă pentru analiza și traducerea textului și antrenarea arborilor de decizie cu gradient boosting. Oferă instrumente pentru căutarea similarității prin modelare k-nearest neighbor, explicabilitatea modelului prin atribuirea trăsăturilor și orchestrarea fluxurilor de lucru de reinforcement learning. Sistemul utilizează o arhitectură de pipeline compozibilă și suportă inferența modelelor bazată pe ONNX pentru compatibilitate cross-platform.
Optimizes data movement and memory usage between distributed partitions and native datasets using direct memory transfer.
exif-js este o bibliotecă JavaScript pentru extragerea metadatelor imaginilor direct în browser. Aceasta funcționează ca un parser de buffer de imagine binar care citește octeții imaginii brute pentru a regăsi detalii tehnice fără a necesita încărcarea fișierelor pe un server. Biblioteca parsează datele urmând standardele EXIF și IPTC pentru a extrage setările camerei, marcajele temporale, coordonatele GPS, subtitrările, cuvintele cheie și informațiile despre drepturile de autor. Utilizează array-uri tipizate și parcurgerea bazată pe decalaje pentru a naviga în structurile imaginilor și a mapa identificatorii numerici la etichete lizibile pentru oameni.
Uses typed arrays to interpret sequences of binary data for metadata extraction.
Acest proiect este o resursă educațională cuprinzătoare și un curs pentru construirea de rețele neuronale folosind PyTorch. Acoperă elementele fundamentale ale deep learning-ului, inclusiv manipularea tensorilor, diferențierea automată și construcția componentelor modulare de rețele neuronale. Repository-ul servește drept ghid tehnic pentru mai multe domenii specializate. Oferă detalii de implementare pentru sarcini de computer vision, cum ar fi clasificarea imaginilor, detecția obiectelor și segmentarea semantică, precum și fluxuri de lucru de procesare a limbajului natural (NLP) care implică transformatoare, rețele recurente și modele generative. În plus, include o referință pentru AI generativ, concentrându-se în mod specific pe sinteza de imagini prin modele de difuzie și rețele adversariale. Materialul se extinde către optimizarea modelelor și pipeline-uri de deployment. Acoperă tehnici pentru reducerea dimensiunii modelelor și creșterea vitezei de inferență prin cuantizare și exportul modelelor în formate precum ONNX și TensorRT. Alte domenii de capabilitate includ ingineria datelor pentru încărcarea paralelă, evaluarea modelelor folosind metrici personalizate și deployment-ul modelelor de limbaj mari (LLM) open-source. Proiectul este livrat în principal sub formă de serie de Jupyter Notebooks.
Pins tensors in memory to prevent disk swapping and accelerate data transfer from CPU to GPU.