phiresky/sql.js-httpvfs
Sql.js Httpvfs
sql.js-httpvfs is a virtual file system and loader designed to execute SQL queries against read-only SQLite databases hosted on remote static file servers. It enables the retrieval of database files over HTTP without requiring the entire file to be downloaded to the client.
The project distinguishes itself by using byte-range HTTP fetching to simulate local random-access reads. It includes a chunking tool that splits large databases into smaller pieces via a JSON configuration, which helps bypass file size limits and optimizes caching.
The system covers a range of capabilities including remote resource mapping, lazy-loading of data segments, and synchronous network simulation. It also provides mechanisms for cache corruption prevention through query-string identifiers and bandwidth control via data transfer limiting.
Features
- Remote SQL Querying - Provides a mechanism to execute SQL queries against remote SQLite databases using HTTP range requests to avoid downloading the entire file.
- Virtual File Systems - Abstracts remote HTTP resources as a local filesystem to allow a database engine to read data on demand.
- On-Demand Remote - Connects a remote database to a worker using a virtual file system that fetches data chunks on demand.
- Remote HTTP Database Access - Accesses database files from a remote server using HTTP requests to query data without downloading the full file.
- Remote Query Execution - Executes SQL queries against a remote database by fetching only the required data chunks via HTTP range requests.
- SQL Query Execution - Executes SQL queries against a remote database by fetching only the necessary data chunks over HTTP.
- Database Chunking - Implements a tool to split large SQLite databases into smaller chunks via JSON configuration for optimized remote loading.
- Remote Data File Accessors - Fetches specific byte ranges of a remote file using synchronous requests to simulate local random-access reads.
- Byte-Range Requesting - Implements byte-range HTTP requests to simulate random-access reads of remote database files.
- Read-Only Virtual Mounts - Provides a restricted virtual file system layer that only permits read operations for static hosting compatibility.
- Remote Storage Mapping - Links remote HTTP resources to a virtual file system so an engine can query remote databases without full downloads.
- Synchronous Network Simulation - Executes blocking network requests to satisfy the requirement for synchronous file system calls within the database engine.
- Client-Side Data Analysis - Enables loading large datasets into a browser for querying and analysis without downloading the entire database.
- Remote Database Loaders - Fetches specific byte ranges of a remote SQLite database to simulate local random-access reads.
- Database File Chunking - Allows splitting large databases into smaller pieces using JSON configuration to bypass file size limits.
- Database - Uses JSON manifest files to map database offsets to physical file segments across remote assets.
- Static Data Distributions - Distributes read-only database files through a content delivery network or static host to minimize infrastructure costs.
- Static Database Hosting - Hosts read-only SQLite databases on static file servers to provide data-driven content without a dedicated backend.
