On Mon, 2019-03-04 at 09:11 -0500, Jeff Layton wrote:
This list has been deprecated. Please subscribe to the new devel list at lists.nfs-ganesha.org. On Fri, 2019-03-01 at 15:49 +0000, David C wrote:
This list has been deprecated. Please subscribe to the new devel list at lists.nfs-ganesha.org. Hi All
Exporting cephfs with the CEPH_FSAL
I set the following on a dir:
setfattr -n ceph.quota.max_bytes -v 100000000 /dir setfattr -n ceph.quota.max_files -v 10 /dir
From an NFSv4 client, the quota.max_bytes appears to be completely ignored, I can go GBs over the quota in the dir. The quota.max_files DOES work however, if I try and create more than 10 files, I'll get "Error opening file 'dir/new file': Disk quota exceeded" as expected.
From a fuse-mount on the same server that is running nfs-ganesha, I've confirmed ceph.quota.max_bytes is enforcing the quota, I'm unable to copy more than 100MB into the dir.
According to [1] and [2] this should work.
Cluster is Luminous 12.2.10
Package versions on nfs-ganesha server:
nfs-ganesha-rados-grace-2.7.1-0.1.el7.x86_64 nfs-ganesha-2.7.1-0.1.el7.x86_64 nfs-ganesha-vfs-2.7.1-0.1.el7.x86_64 nfs-ganesha-ceph-2.7.1-0.1.el7.x86_64 libcephfs2-13.2.2-0.el7.x86_64 ceph-fuse-12.2.10-0.el7.x86_64
My Ganesha export:
EXPORT { Export_ID=100; Protocols = 4; Transports = TCP; Path = /; Pseudo = /ceph/; Access_Type = RW; Attr_Expiration_Time = 0; #Manage_Gids = TRUE; Filesystem_Id = 100.1; FSAL { Name = CEPH; } }
My ceph.conf client section:
[client] mon host = 10.10.10.210:6789, 10.10.10.211:6789, 10.10.10.212:6789 client_oc_size = 8388608000 #fuse_default_permission=0 client_acl_type=posix_acl client_quota = true client_quota_df = true
Related links:
[1] http://tracker.ceph.com/issues/16526 [2] https://github.com/nfs-ganesha/nfs-ganesha/issues/100
Thanks David
It looks like you're having ganesha do the mount as "client.admin", and I suspect that that may allow you to bypass quotas? You may want to try creating a cephx user with less privileges, have ganesha connect as that user and see if it changes things?
Actually, this may be wrong info. How are you testing being able to write to the file past quota? Are you using O_DIRECT I/O? If not, then it may just be that you're seeing the effect of the NFS client caching writes. -- Jeff Layton <jlayton@redhat.com>