cephmount

Provides a local filesystem implementation for Reva that emulates CephFS operations using the local filesystem.

DriversStorageStorage driversbuild tag: ceph
Package
github.com/cs3org/reva/v3/pkg/storage/fs/cephmount
Source
pkg/storage/fs/cephmount · Go reference

Enable it

Select the driver in the service that uses it, and put its settings in the driver's own table:

[grpc.services.storageprovider]
driver = "cephmount"

[grpc.services.storageprovider.drivers.cephmount]
# settings below

Overview

Package cephmount provides a local filesystem implementation for Reva that emulates CephFS operations using the local filesystem.

This driver implements the storage.FS interface using standard Go os package operations instead of libcephfs, making it suitable for development, testing, or single-node deployments.

CephMount provides a CephFS-compatible filesystem interface for Reva using fstab-based configuration.

Requirements

1. Ceph Development Libraries

Install Ceph development libraries on your system:

# Fedora 42
sudo dnf install libcephfs-devel librados-devel

# CentOS/RHEL
sudo yum install libcephfs-devel librados-devel

2. CephFS Kernel Mount

You need a valid persistent CephFS kernel mount accessible by the user running Reva (typically root). Refer to Ceph fstab configuration.

ceph-mon.example.com:6789:/volumes/shared /mnt/cephfs ceph defaults,name=admin,secretfile=/etc/ceph/ceph.client.admin.key,conf=/etc/ceph/ceph.conf 0 2

This entry must be configured in your Reva config as the fstabentry parameter.

Quick Start

Configuration

Configure in your config.toml:

[grpc.services.storageprovider]
driver = "cephmount"

[grpc.services.storageprovider.drivers.cephmount]
fstabentry = "ceph-mon.example.com:6789:/volumes/shared /mnt/cephfs ceph defaults,name=admin,secretfile=/etc/ceph/ceph.client.admin.key,conf=/etc/ceph/ceph.conf 0 2"
root = "/"

[http.services.dataprovider]
driver = "cephmount"

[http.services.dataprovider.drivers.cephmount]
fstabentry = "ceph-mon.example.com:6789:/volumes/shared /mnt/cephfs ceph defaults,name=admin,secretfile=/etc/ceph/ceph.client.admin.key,conf=/etc/ceph/ceph.conf 0 2"
root = "/"

This example will convert Reva paths like /projects/alabasta/planning.md, to /mnt/cephfs/projects/alabasta/planning.md (when using localfs functions) and to /volumes/shared/projects/alabasta/planning.md when connecting directly to MDS.

Testing

There are 2 major testing scenarios:

# Unit tests (no CephFS required)
go test ./pkg/storage/fs/cephmount -v

# Integration tests (requires valid CephFS mount)
export CEPHMOUNT_FSTAB_ENTRY="your-fstab-entry-here"
go test -tags ceph ./pkg/storage/fs/cephmount -v

Depending if you run the tests as root or as another user (typically in your dev machines), some tests like the privilege verification ones may be skipped due to lack of permsissions to switch uid/guid for the forked processes.

Benchmarks

go test ./pkg/storage/fs/cephmount -bench "Benchmark.*" -v
go test -tags ceph ./pkg/storage/fs/cephmount -bench "BenchmarkCeph.*" -v

Use benchstat to compare if needed.

Development Mode

For testing without CephFS:

[grpc.services.storageprovider.drivers.cephmount]
testing_allow_local_mode = true  # Testing only!
root = "/tmp/testing"

Never use testing_allow_local_mode in production!

Functions that require conversion between file ID (inode) and and path will always require access to the MDS and therefore having a valid Ceph configuration. The tests handle this transparently, skipping any tests necessary.

Configuration

KeyTypeDefaultDescription
uploadsstring".uploads"source
dir_permsuint32dirPermDefaultsource
file_permsuint32filePermDefaultsource
user_quota_bytesuint6450000000000source
nobody_uidint65534Nobody user/group for fallback operations (instead of root) source
nobody_gidint65534source
external_accounts_user_namestring"cboxexternal"External (lightweight, federated, guest) accounts have no uid of their own, so operations on their behalf run as this local service account instead. What they are allowed to do is decided by reva from the stored grants, not by the permissions of this account. source
external_accounts_user_uidint193339source
external_accounts_user_gidint2766source
max_user_threadsint-Go stops the process above 10000 OS threads. One revad has two cephmount drivers (storageprovider and dataprovider), and each has its own user threads. source
max_threads_per_userint-The requests of one uid run in parallel on up to this number of threads. All external accounts share one uid, thus they have a separate limit. source
max_threads_externalint-source
space_depthint-

SpaceDepth defines how many levels below the chroot the space (project) roots sit: with roots at /winspaces/c/, served by a provider mounted at /winspaces, it is 2. The service account is granted rwx on the space root when something inside it is shared with an external account.

It is injected by the storage provider, which rebases its own space_depth on the paths it hands to the driver: its mount path is trimmed off first, so the number here is smaller than the one in its configuration. It should not be set in the driver configuration.

source
fstabentrystring-Simplified Ceph configuration - just paste the fstab entry source
root_dirstring-Root directory for the Ceph mount source
fs_namestring-Ceph filesystem name (e.g., “cephfs”) source
testing_allow_local_modebool-Testing-only option - allows running without Ceph configuration for local filesystem tests source