- Rust 100%
|
|
||
|---|---|---|
| examples | ||
| src | ||
| tests | ||
| .gitignore | ||
| Cargo.toml | ||
| LICENSE | ||
| notes.txt | ||
| performance.md | ||
| PLAN.md | ||
| README.md | ||
| summary.md | ||
| USER_GUIDE.md | ||
gdrive4fs
A Rust crate that exposes Google Drive as a filesystem-flavoured API, designed to be
wrapped by a FUSE filesystem (and friends). It wraps the Google Drive HTTP API
(google-drive3) and presents a POSIX-like, async interface —
getattr/lookup/readdir/open/read/write/flush/unlink/mkdir/rename/statfs —
so FUSE glue stays trivial. The full design and milestone plan live in
PLAN.md.
Status
Implemented (M1–M17). v1 is complete; M11 adds the optional
"Shared with me" virtual folder; M12 adds an executable-bit policy
(exec_policy); M13 adds the freedesktop /.Trash namespace over Drive's own
trash; M14 replaces google-drive3's single-shot resumable upload with a
chunked, byte-contiguous resumable uploader (2 MiB bands, reseeded from the
server's Range header on 308 and from transfer-status queries after failures —
the bandwidth milestone §7.1, re-scoped since Drive sessions can only start at
byte 0); M15 implements duplicate_policy=RenameRemote (name collisions
actually rename the Drive holder out of the way, race-guarded by a version
compare, with a graceful EACCES fallback when the holder isn't owned) and
extended attributes over Drive appProperties under a gdrive4fs. namespace
(getxattr/setxattr/listxattr/removexattr, coexisting with the M12 exec
bit in the same property map); M16 is the v2 polish + release pass (FUSE example
now configures and walks every v2 feature — see Examples and Feature
build-out below); M17 adds session logging & metrics — every op logs a
per-op line (getattr, cache.open, cache.flush, drive.upload.resumable,
…) and each GDrive session collects a shared metrics ledger
(op timings, byte totals, cache hit/reuse/eviction counters, retries, resumable
band/stall/conflict counters) that is written out as one info! summary when
the mount drops — enough to diagnose performance and corrupt-file issues from a
plain log (RUST_LOG=debug for per-op tracing, default info). 196 unit
tests (174 without the goa feature) —
zero warnings on cargo build/cargo test/cargo doc in both feature
configurations. cargo clippy is not installable in this environment (no
rustup), so the zero-warning build/test/doc gate stands in its place.
Two runnable examples ship with the crate; live end-to-end smoke tests (basic +
GOA round-trips, a multi-band ~9 MiB resumable round-trip, a real
files.update(name) rename-remote, an appProperties xattr round-trip, trash
walk + empty_trash, and the shared feed) are opt-in behind the integration
feature — 7/7 green against a live GOA account.
Feature build-out
| Feature | Milestone | Status |
|---|---|---|
Namespace + content ops (getattr/lookup/readdir/open/read/write/flush/create/mkdir/unlink/rmdir/rename/truncate/statfs) |
M9 | ✓ |
| Whole-file on-disk content cache (export/import of Google-native files) | M8 | ✓ |
| Chunked resumable uploads (≥5 MiB; banded, byte-contiguous) | M14 | ✓ |
Optimistic conflict handling (FailOnRemoteChange) |
M9 | ✓ |
"Shared with me" virtual folder (mount_shared) |
M11 | ✓ |
Executable bit — exec_policy (Property/Sidecar/Heuristic) |
M12 | ✓ |
freedesktop /.Trash (walk, trash/restore, permanent delete in-trash, empty_trash) |
M13 | ✓ |
duplicate_policy=RenameRemote (colliding create renames the holder away) |
M15 | ✓ |
xattr over appProperties (gdrive4fs. namespace) |
M15 | ✓ |
GOA token provider (optional goa feature) |
M5 | ✓ |
FUSE bridge example (examples/fuse-mount.rs) + v2 ops |
M10 / M16 | ✓ |
Session logging & metrics (log/Metrics, RUST_LOG-tunable) |
M17 | ✓ |
Live-Drive smoke tests (optional integration feature) |
M10+ | ✓ |
Usage
[dependencies]
gdrive4fs = { version = "0.1", default-features = false }
Connect with a token you manage yourself,
use gdrive4fs::config::GDriveConfig;
use gdrive4fs::fs::GDrive;
let drive = GDrive::connect_with_token(token.into(), GDriveConfig::default()).await?;
or reuse an existing token source (GDrive::connect_with_source). With the
default goa feature (Linux), a ready-made Gnome Online Accounts provider
picks the Drive-enabled Google account for you.
Examples
examples/basic.rs— a credential + feature walk over a live account:stat /,mkdir,create,open→write→flush,read,readdir, then the v2 ops — an xattr round-trip, an exec-bit chmod, a freedesktop trash walk (trash a file, list/.Trash/<uid>/files/, read its synthesized.trashinfo, rename it back out to restore), a slice of the/Shared with mefeed (mount_sharedon), and cleanup. Run withGDRIVE_TOKEN=<token> cargo run --example basic.examples/fuse-mount.rs— the FUSE bridge described below. Run withcargo run --example fuse-mount -- <mountpoint> [--shared] [--delete=trash|permanent] [--exec=property|sidecar|heuristic] [--duplicates=suffix|first-wins|rename-remote](unmount:fusermount -u <mountpoint>). The flags configure the v2 features the mount exposes:--sharedshows the/Shared with meview,--deleteselects the/.Trashpresentation or permanent deletes,--execpicks the executable-bit policy (thesetattr(mode)callback is the exec-bit chmod path), and--duplicatesselects the collision policy. The four xattr callbacks (get/set/list/removexattr) are wired unconditionally. fuser exposes noempty_trashop, so emptying the trash uses the API (thebasicexample calls it after its walk).
Both connect from GDRIVE_TOKEN if set, otherwise from the GOA account.
A FUSE mount in one file
gdrive4fs is deliberately FUSE-agnostic — it does not depend on fuser.
The bridge lives in the consumer (examples/fuse-mount.rs shows the whole
thing):
GDriveis async and path/handle-based;fuseris a synchronous, inode-based kernel protocol. The glue holds atokio::runtime::Handle, an inode → path map (inodes are deterministic — the crate's inode table is stable per Drive id), and anfh → OpenFilemap; every fuser callback simplyblock_ons the matchingGDrivecall.- Because inodes are deterministic and paths stable, the inode → path map
fills up naturally as the kernel walks
lookup/readdir. - Every v2 feature rides the same glue:
.Trashand/Shared with meare ordinary virtual directories from the kernel's point of view (walk, restore and in-trash delete are plainreaddir/rename/unlink),chmodis exec-bit-only, and xattrs map to DriveappProperties.
Feature flags
| Feature | Default | Purpose |
|---|---|---|
goa |
on | GOA/D-Bus token provider (dbus dependency, Linux only) |
integration |
off | live-Drive + live-D-Bus smoke tests (needs credentials) |
Use default-features = false for a portable, dependency-light build.
Token sources
The crate accepts a TokenSource (a cloneable, Send + Sync provider of fresh
tokens), or a plain token for short-lived sessions:
connect_with_token(token, cfg)— fixed token (tests / quick experiments).connect_with_source(source, cfg)— application-owned acquisition and refresh.CallableTokenSource— wrap a closure so the crate re-fetches on 401.
Design notes
- Whole-file caching: reads/writes go through a local on-disk cache; modified files are re-uploaded whole on flush/close. No partial-write emulation.
- Resumable uploads (M14): payloads above 5 MiB bypass the simple multipart
upload and go through a banded resumable session (2 MiB PUT bands with
Content-Range, the cursor reseeded from the server'sRangeheader on 308 and from transfer-status queries after any failed band). Drive sessions are byte-contiguous from offset 0 — there is no way to start a session at a mid-file offset — so "upload only the dirty range" is impossible and the whole file is always re-sent; the bands merely bound memory and resume state. - Deterministic inodes: stable FNV-1a-64 of the Drive file id.
- Google-native files (Docs/Sheets/Slides): shown as editable regular files — exported on open and re-imported on flush.
- Errno-flavored errors (
Error::to_errno()) for FUSE glue. - Conflicts: optimistic — the
files.versiontoken is replayed on upload;fail_on_remote_changesurfaces a stale copy instead of clobbering silently. - "Shared with me" (
config.mount_shared, default off): a virtual/Shared with mefolder at the mount root lists Drive's shared feed (sharedWithMe = true). Its items are keyed by file id, not path, so content, export and mutation ops work unchanged once an id is resolved; a file you also own under "My Drive" keeps a single inode in both views. - freedesktop Trash (
delete_mode=Trash, the default): a virtual/.Trashfolder at the mount root presents Drive's own trash. FM trash actions map onto it — renaming a file into/.Trash/<uid>/files/trashes it, the/.Trash/<uid>/info/<name>.trashinfowrite is accepted-and-dropped,mkdirof theuid/files/infoscaffolding is ignored, renaming a file out of the trash restores it, and unlinking inside the trash permanently deletes it..trashinforecords are synthesized (freedesktop[Trash Info];Pathfrom the last-known location,DeletionDatefromtrashedTime, falling back tomodifiedTime).empty_trashcallsfiles.emptyTrash.delete_mode=Permanentlyomits/.Trashentirely. - Executable bit (
config.exec_policy, defaultProperty): Drive has no perms, so mounts expose an exec bit per file and honour exec-bit-onlychmods.Propertypersists it in the file'sappProperties(gdrive4fs.exec— 15 bytes, well under Drive's 124-byte cap) for files you own; for shared/read-only files the chmod falls back to a machine-local manifest.Sidecarstores every chmod in that manifest;Heuristic(the default look-on-a-mount) just marks.sh/.py/.exe/… by extension and rejects chmods (ENOTSUP). - RenameRemote (
config.duplicate_policy, defaultSuffix): instead of presenting duplicates with(N)suffixes, a collidingcreate/mkdir/renameactually renames the Drive holder to the next freename (N)(files.update(name)) so the caller can claim the bare name — consistent for every client and persistent beyond this mount. Opt-in for a reason: it mutates Drive. Two safety rails: the freed slot is picked against a fresh listing, and the rename replays the holder'sversionso two racing renames can't both win (a lost race surfaces asEEXIST; a holder this account can't rename — e.g. a file only shared with you — degrades toEACCES, nothing moves). - Extended attributes (M15):
getxattr/setxattr/listxattr/removexattrread and write DriveappPropertiesunder agdrive4fs.namespace — the xattr name is the property key, sosetfattr -n gdrive4fs.user.color -v blue f.txtstoresappProperties["gdrive4fs.user.color"] = "blue". Names must match Drive's property-key charset (A-Za-z0-9.!@$%^&*()-_/— Drive rejects:and friends with a 403; discovered live, hence the.namespace). Outside the namespace:ENOTSUP; missing attribute:ENODATA; non-UTF-8 value:EINVAL; key+value over Drive's 124-byte budget:EFBIG. xattrs share the map with the exec bit, so chmod and xattrs coexist. FUSE glue for all four ops ships inexamples/fuse-mount.rs. Removal caveat (M15, live-verified): Drive'sfiles.updatemergesappProperties, and google-drive3 cannot express the null value Drive needs to truly delete a key — soremovexattrpersists an empty-string tombstone thatgetxattr/listxattrreport as absent; the property lingers in the Drive metadata (a documented trade-off, consistent across remounts). - Logging & metrics (M17): the crate logs through the standard
logfaçade —RUST_LOGcontrols verbosity (both examples initializeenv_logger:basicdefaults todebugfor full per-op tracing,fuse-mounttoinfo). Every public op reports one foreground line (e.g.getattr /a/b,cache.open <id>with the stale-reason and fetched bytes,cache.flush <id>with the re-upload size and conflict guard,drive.upload.resumable <size>with per-band progress and running MiB/s). EachGDrivesession also keeps a per-instanceMetricsledger — shared with the cache and drive connection — of op timings, byte totals, cache hits/reuses/evictions, 401 replays, pagination and resumable band/stall/conflict counters; when the mount drops, the whole ledger is written as oneinfo!summary (with derived MiB/s and thecache.reuse.ratio). Size-mismatchwarn!s inside the cache (slot filled ≠ metadata size; upload reported ≠ media sent) flag the truncated/corrupt-file class with the file id attached.
Limitations
Per feature (each maps to a "Design notes" bullet above):
- Permissions — no
chown/mknod/symlink(ENOTSUP);chmodis exec-bit-only (underHeuristicthe chmod itself fails withENOTSUP). - Shared with me (
mount_shared) — a filter view, so you cannot create files directly inside/Shared with me(create inside a shared folder instead), shared-drive full listing is out of scope, and mutating a shared file requires the owner's edit permission (otherwise the write fails with a permission error). - Slash/backslash/dot names (§7.9) — Drive allows
/in file names and exact./.., neither of which is a valid POSIX path component. Such names are presented C-style-escaped (/→\/,\→\\, exact./..get a leading\), consistently acrosslookup/readdir(one pipe inchildren_of), and decoded back only at the Drive write (create/mkdir/rename). Names without/or\pass through unchanged. Caveats: an escaped\/still contains a/byte, so a kernel FUSE mount still drops those entries fromreaddirand noPathcan name them — slash-names are addressable through the crate'sOsStr-name ops (lookup/create/mkdir/unlink/rmdir) in escaped form, while\- and dot-names work everywhere including FUSE; presented names over 255 bytes are skipped (with a warning) as unlistable viagetdents. - Duplicates — Drive's duplicate-name allowance is reconciled by presented
(N)suffixes (deterministic bycreatedTime); under the default policy the duplicates stay on Drive.RenameRemotefixes Drive itself (opt-in, mutates your data), and a holder this account can't rename degrades toEACCES. - Trash —
rmdirremoves a Drive folder regardless of contents (Drive has folder-level granularity, noENOTEMPTY); the trash feed is keyed by file id, and fuser exposes noempty_trashop on Linux (empty viaGDrive::empty_trash()— seeexamples/basic.rs). - Writes — whole-file: large edits upload the entire file on flush (the
5 MiB resumable path sends it in 2 MiB bands — configurable — from the on-disk slot, so RAM stays flat; no partial-Drive-write exists, so whole-file re-upload stays — see the M14 design note above). Single files are capped by
max_file_bytes(unlimited by default,EFBIGwhen set); uploads are MD5-verified against the server checksum. Drive caps single files at 5 TB and idle resumable sessions expire after ~1 week. - xattr removal —
removexattrpersists an""tombstone (see the design note) that reads as absent; the property lingers in the Drive metadata. - Conflicts — optimistic (
files.versionreplayed on upload);fail_on_remote_changesurfaces a stale copy instead of clobbering silently. - GOA — needs a running GNOME session D-Bus;
connect_goapicks the single Drive-enabled Google account and errors helpfully when ambiguous.
Live smoke tests
GDRIVE_TOKEN=<token> cargo test --features integration live_
These hit the real Drive API (and a live D-Bus session for the GOA test); they
skip at runtime when credentials are absent, so they never fail a CI run. The
suite (tests/live_drive.rs, all green against a live GOA account):
live_drive_basic_round_trip— mkdir/create/write/flush/read/unlink/rmdir on a real Drive tree.live_goa_smoke— GOA token acquisition + connect (+filter/Scopes).live_shared_with_me_smoke— the/Shared with mefeed counts and lists.live_trash_smoke— trash a file, walk/.Trash, read the synthesized.trashinfo(Path+DeletionDate), permanently delete,empty_trash.live_resumable_upload_smoke— ~9 MiB payload, 5 × 2 MiB bands, byte-exact read-back round-trip.live_rename_remote_smoke— a collidingcreateactually renames the Drive holder toname (1).live_xattr_smoke—setxattr/getxattr/listxattr/removexattragainst real DriveappProperties.