cephmount
Provides a local filesystem implementation for Reva that emulates CephFS operations using the local filesystem.
- Package
github.com/cs3org/reva/v3/pkg/storage/fs/cephmount- Source
- pkg/storage/fs/cephmount · Go reference
Enable it
Select the driver in the service that uses it, and put its settings in the driver's own table:
[grpc.services.storageprovider]
driver = "cephmount"
[grpc.services.storageprovider.drivers.cephmount]
# settings belowOverview
Package cephmount provides a local filesystem implementation for Reva that emulates CephFS operations using the local filesystem.
This driver implements the storage.FS interface using standard Go os package operations instead of libcephfs, making it suitable for development, testing, or single-node deployments.
CephMount provides a CephFS-compatible filesystem interface for Reva using fstab-based configuration.
Requirements
1. Ceph Development Libraries
Install Ceph development libraries on your system:
# Fedora 42
sudo dnf install libcephfs-devel librados-devel
# CentOS/RHEL
sudo yum install libcephfs-devel librados-devel
2. CephFS Kernel Mount
You need a valid persistent CephFS kernel mount accessible by the user running Reva (typically root). Refer to Ceph fstab configuration.
ceph-mon.example.com:6789:/volumes/shared /mnt/cephfs ceph defaults,name=admin,secretfile=/etc/ceph/ceph.client.admin.key,conf=/etc/ceph/ceph.conf 0 2
This entry must be configured in your Reva config as the fstabentry parameter.
Quick Start
Configuration
Configure in your config.toml:
[grpc.services.storageprovider]
driver = "cephmount"
[grpc.services.storageprovider.drivers.cephmount]
fstabentry = "ceph-mon.example.com:6789:/volumes/shared /mnt/cephfs ceph defaults,name=admin,secretfile=/etc/ceph/ceph.client.admin.key,conf=/etc/ceph/ceph.conf 0 2"
root = "/"
[http.services.dataprovider]
driver = "cephmount"
[http.services.dataprovider.drivers.cephmount]
fstabentry = "ceph-mon.example.com:6789:/volumes/shared /mnt/cephfs ceph defaults,name=admin,secretfile=/etc/ceph/ceph.client.admin.key,conf=/etc/ceph/ceph.conf 0 2"
root = "/"
This example will convert Reva paths like /projects/alabasta/planning.md, to /mnt/cephfs/projects/alabasta/planning.md (when using localfs functions) and to /volumes/shared/projects/alabasta/planning.md when connecting directly to MDS.
Testing
There are 2 major testing scenarios:
# Unit tests (no CephFS required)
go test ./pkg/storage/fs/cephmount -v
# Integration tests (requires valid CephFS mount)
export CEPHMOUNT_FSTAB_ENTRY="your-fstab-entry-here"
go test -tags ceph ./pkg/storage/fs/cephmount -v
Depending if you run the tests as root or as another user (typically in your dev machines), some tests like the privilege verification ones may be skipped due to lack of permsissions to switch uid/guid for the forked processes.
Benchmarks
go test ./pkg/storage/fs/cephmount -bench "Benchmark.*" -v
go test -tags ceph ./pkg/storage/fs/cephmount -bench "BenchmarkCeph.*" -v
Use benchstat to compare if needed.
Development Mode
For testing without CephFS:
[grpc.services.storageprovider.drivers.cephmount]
testing_allow_local_mode = true # Testing only!
root = "/tmp/testing"
Never use testing_allow_local_mode in production!
Functions that require conversion between file ID (inode) and and path will always require access to the MDS and therefore having a valid Ceph configuration. The tests handle this transparently, skipping any tests necessary.
Configuration
| Key | Type | Default | Description |
|---|---|---|---|
uploads | string | ".uploads" | source |
dir_perms | uint32 | dirPermDefault | source |
file_perms | uint32 | filePermDefault | source |
user_quota_bytes | uint64 | 50000000000 | source |
nobody_uid | int | 65534 | Nobody user/group for fallback operations (instead of root) source |
nobody_gid | int | 65534 | source |
external_accounts_user_name | string | "cboxexternal" | External (lightweight, federated, guest) accounts have no uid of their own, so operations on their behalf run as this local service account instead. What they are allowed to do is decided by reva from the stored grants, not by the permissions of this account. source |
external_accounts_user_uid | int | 193339 | source |
external_accounts_user_gid | int | 2766 | source |
max_user_threads | int | - | Go stops the process above 10000 OS threads. One revad has two cephmount drivers (storageprovider and dataprovider), and each has its own user threads. source |
max_threads_per_user | int | - | The requests of one uid run in parallel on up to this number of threads. All external accounts share one uid, thus they have a separate limit. source |
max_threads_external | int | - | source |
space_depth | int | - | SpaceDepth defines how many levels below the chroot the space (project)
roots sit: with roots at /winspaces/c/ It is injected by the storage provider, which rebases its own space_depth on the paths it hands to the driver: its mount path is trimmed off first, so the number here is smaller than the one in its configuration. It should not be set in the driver configuration. source |
fstabentry | string | - | Simplified Ceph configuration - just paste the fstab entry source |
root_dir | string | - | Root directory for the Ceph mount source |
fs_name | string | - | Ceph filesystem name (e.g., “cephfs”) source |
testing_allow_local_mode | bool | - | Testing-only option - allows running without Ceph configuration for local filesystem tests source |