kahing/goofys
Goofys
Goofys is a POSIX-compliant cloud object storage gateway that presents remote storage buckets as local system directories. It implements a user-space filesystem that maps S3-compliant and Azure Blob storage services to local mount points, allowing remote objects to be accessed through standard file system operations.
The project provides specific mounting capabilities for Amazon S3, Azure Blob Storage, and Azure Data Lake accounts. It utilizes a FUSE-based implementation to interface cloud object storage with the operating system kernel.
The system includes performance optimizations such as local read caching to reduce latency and concurrent range-request fetching to optimize the download of large objects. It simulates hierarchical folder structures by parsing object key prefixes to provide directory emulation.
Features
- Cloud Data Access - Provides a gateway to access remote cloud object storage as local folders for simplified file management.
- Azure Blob Storage Integration - Connects Azure cloud storage to a local file system to manage remote objects through native OS tools.
- Azure Blob Storage Mounting - Maps Azure Blob Storage buckets to local directories for interaction via standard file system operations.
- Azure Blob Storage Mounts - Presents Azure Blob and Data Lake Storage accounts as local directories via a filesystem interface.
- Azure Data Lake Integration - Connects data lake accounts to a local directory for file system operations on large cloud datasets.
- Azure Data Lake Mounting - Maps Azure Data Lake accounts to local directories to access cloud files through the native file system.
- Cloud Storage Gateways - Acts as a bridge that maps S3-compliant and Azure Blob storage services to local system mount points.
- Multi-Provider Object Storage - Provides a unified interface for connecting to multiple S3-compliant and Azure storage providers.
- Object Key Prefix Emulation - Simulates a hierarchical folder structure by parsing object key prefixes to provide standard directory navigation.
- S3 Object Mounting - Maps S3-compliant cloud storage buckets to local mount points as a compliant local file system.
- User-Space Implementations - Implements filesystem logic in user-space via Go to avoid the need for kernel-level drivers.
- FUSE-Based Filesystem Mounts - Uses a FUSE-based implementation to map remote cloud storage buckets as local system directories.
- S3 FUSE Clients - Implements a POSIX-compliant S3 FUSE filesystem to mount Amazon S3 buckets as local directories.
- Read-Through Memory Caches - Stores recently read remote data locally in a cache to reduce latency and increase throughput.
- Caching and Performance - Implements high-performance local caching to reduce latency when accessing remote cloud files.
- Local Disk Caching - Provides local disk caching for frequently accessed data and pending writes to reduce network latency.
- Byte-Range Requesting - Implements parallel downloads of large objects using HTTP byte-range requests to optimize read performance.
- Stateless Architectures - Translates POSIX system calls directly into cloud API requests without maintaining persistent local state.
- File Systems - High-performance POSIX-like file system for Amazon S3.
- Filesystem Mounts - Modern filesystem driver focused on performance and POSIX compatibility.
Star history
How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
