This list has been deprecated. Please subscribe to the new devel list
at
lists.nfs-ganesha.org.
On Fri, 2019-03-01 at 15:49 +0000, David C wrote:
> This list has been deprecated. Please subscribe to the new devel list at
lists.nfs-ganesha.org.
> Hi All
>
> Exporting cephfs with the CEPH_FSAL
>
> I set the following on a dir:
>
> setfattr -n ceph.quota.max_bytes -v 100000000 /dir
> setfattr -n ceph.quota.max_files -v 10 /dir
>
> From an NFSv4 client, the quota.max_bytes appears to be completely ignored, I can go
GBs over the quota in the dir. The quota.max_files DOES work however, if I try and create
more than 10 files, I'll get "Error opening file 'dir/new file': Disk
quota exceeded" as expected.
>
> From a fuse-mount on the same server that is running nfs-ganesha, I've confirmed
ceph.quota.max_bytes is enforcing the quota, I'm unable to copy more than 100MB into
the dir.
>
> According to [1] and [2] this should work.
>
> Cluster is Luminous 12.2.10
>
> Package versions on nfs-ganesha server:
>
> nfs-ganesha-rados-grace-2.7.1-0.1.el7.x86_64
> nfs-ganesha-2.7.1-0.1.el7.x86_64
> nfs-ganesha-vfs-2.7.1-0.1.el7.x86_64
> nfs-ganesha-ceph-2.7.1-0.1.el7.x86_64
> libcephfs2-13.2.2-0.el7.x86_64
> ceph-fuse-12.2.10-0.el7.x86_64
>
> My Ganesha export:
>
> EXPORT
> {
> Export_ID=100;
> Protocols = 4;
> Transports = TCP;
> Path = /;
> Pseudo = /ceph/;
> Access_Type = RW;
> Attr_Expiration_Time = 0;
> #Manage_Gids = TRUE;
> Filesystem_Id = 100.1;
> FSAL {
> Name = CEPH;
> }
> }
>
> My ceph.conf client section:
>
> [client]
> mon host = 10.10.10.210:6789, 10.10.10.211:6789, 10.10.10.212:6789
> client_oc_size = 8388608000
> #fuse_default_permission=0
> client_acl_type=posix_acl
> client_quota = true
> client_quota_df = true
>
> Related links:
>
> [1]
http://tracker.ceph.com/issues/16526
> [2]
https://github.com/nfs-ganesha/nfs-ganesha/issues/100
>
> Thanks
> David
>
It looks like you're having ganesha do the mount as "client.admin", and
I suspect that that may allow you to bypass quotas? You may want to try
creating a cephx user with less privileges, have ganesha connect as that
user and see if it changes things?
Actually, this may be wrong info.
How are you testing being able to write to the file past quota? Are you
using O_DIRECT I/O? If not, then it may just be that you're seeing the
effect of the NFS client caching writes.
--
Jeff Layton <jlayton(a)redhat.com>