30 open-source projects similar to awslabs/dynamodb-streams-kinesis-adapter, ranked by shared indexed features. Tags may describe platforms or build tools rather than the same primary purpose. Check each project’s use case, license, and deployment requirements before treating it as a replacement.
DeepLake is AI data infrastructure consisting of a multimodal data lake, a hybrid search engine, and a serverless vector database. It provides a PostgreSQL-based AI data runtime that combines multimodal storage with streaming pipelines to load and shuffle datasets from cloud storage directly into deep learning training pipelines. The system utilizes lazy indexing to store and slice images, audio, and video without loading entire files into memory. It enables retrieval-augmented generation by persisting high-dimensional embeddings in a serverless vector store and implementing hybrid search tha
Hub is a multimodal AI data lake and vector database designed for storing and querying embeddings, text, audio, and images. It functions as a dataset version control system and a machine learning data streaming engine to support large-scale model training. The system utilizes a serverless PostgreSQL vector store to index high-dimensional embeddings for semantic search. It provides a visual interface for inspecting multimodal datasets and viewing annotations such as bounding boxes and masks. The platform handles cloud-agnostic storage synchronization and implements lazy, compressed data strea
AWS Lambda Container Image with Python Rasterio for querying Cloud Optimised GeoTiffs.
WaveStitch is a deep generative framework for conditional time series synthesis. It enables the generation of realistic time series data conditioned on auxiliary features (e.g., labels, metadata) and signal anchors (e.g., partial observations). This codebase provides tools for experimentation…
JSONExport is a multi-language code generator and JSON schema converter that transforms JSON data structures into strongly typed source code classes. It serves as an API response mapper, converting JSON objects into data transfer objects to automate the creation of model classes. The tool specializes in multi-language model synthesis, allowing users to define data models across different programming languages using a single JSON input. It generates class boilerplate, including constructors and accessors, and provides a preview pipeline to review the resulting source code before it is saved.
AlgoliaSearch integration to your favorite ORM
Olmocr is a distributed document processing framework designed to convert PDF and image files into structured markdown. It functions as a vision-based document parser that utilizes multimodal neural networks to interpret complex visual layouts and translate them into standardized text representations. The system operates as a remote inference orchestrator, offloading heavy document analysis tasks to external servers or cloud APIs to minimize local computational requirements. By employing a stateless worker architecture, it decouples document ingestion from inference, allowing for the distribu
Groupdate is a PostgreSQL time series aggregator and date grouping tool. It provides a set of SQL functions to group and aggregate temporal records into discrete buckets, such as days, weeks, or months, to calculate sums and averages for reports. The project focuses on ensuring continuous timelines through time series gap filling, which inserts default values for periods where no data exists. It also includes a temporal data formatter that converts grouped date-time keys into localized strings or custom formatting patterns. The tool covers broad temporal data operations, including time range
Searchkick is an integration library and wrapper that connects application models to search engines such as Elasticsearch and OpenSearch. It functions as a search index synchronizer, automatically mirroring database records to a search server to enable full-text and vector retrieval. The project provides a high-level interface for implementing keyword search, semantic vector search, and hybrid search. It distinguishes itself through the ability to combine traditional keyword matching with vector embeddings using reranking and fusion techniques to improve precision. The library covers the end
A comprehensive Spark guide collated from multiple sources that can be referred to learn more about Spark or as an interview refresher.
Fx is a command-line processing suite designed for the transformation, conversion, exploration, and visualization of structured data. It functions as a terminal-based utility that handles both automated shell pipelines and interactive navigation of complex, nested data hierarchies. The tool distinguishes itself by integrating a JavaScript-based engine that executes user-provided logic to filter, map, or modify data fields within a sandboxed runtime. It maintains a responsive interface by decoupling data processing from the display loop, allowing users to explore large datasets through an inte
Hadoop is a big data infrastructure suite and distributed data processing framework designed to store and process massive datasets across clusters of computers. It consists of a distributed storage system for managing large files across multiple nodes and a parallel computing engine for processing data across a distributed cluster. The framework implements a distributed file system to ensure fault tolerance and high throughput, paired with a programming model that processes large datasets in parallel. It manages the underlying hardware and software environment required for distributed big dat
Apache IoTDB is a time-series database designed for the Internet of Things, purpose-built to ingest high-volume data from millions of low-power devices and store timestamp-value pairs with configurable data types and encoding schemes. It organizes time series data and device metadata in a tree-like hierarchy, enabling efficient management of complex industrial sensor networks. The database supports rich querying capabilities, including time-aligned data retrieval across multiple devices, time-based aggregation like downsampling, and frequency-domain signal analysis. It provides high-throughpu
Apache Spark is a unified distributed data processing engine designed for large-scale data analysis and computation graphs. It functions as a distributed machine learning framework, a graph processing system, a real-time stream processor, and a SQL analytics engine. The system enables the execution of distributed SQL querying, large-scale graph analysis, and real-time stream analytics across clusters of machines. It also provides a scalable environment for implementing machine learning algorithms and predictive model development on massive datasets. The engine incorporates relational query e
Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verified research papers.
CMSIS-DSP is an optimized compute library for embedded systems (DSP is in the name for legacy reasons).
data compression library for embedded/real-time systems
Fast sorted collections for Swift using in-memory B-trees
ARCHIVED: Use https://github.com/aws/aws-sdk-android/
Official repository for Amazon Cognito Sync Manager SDK for Dotnet.