1:34 a.m.
On 10/12/18 7:22 AM, Kinglong Mee wrote:
On 2018/10/11 19:09, Soumya Koduri wrote:
> NFS-Ganesha's md-cache layer already does extensive caching of attributes and
ACLs of each file looked upon. Do you see any additional benefit with turning on gluster
md-cache as well? More replies inline..
Yes, I think.
The logical is different between md-cache and ganesha's cache,
Ganesha caches xattr data depends on timeout, if timeout, ganesha get it from back-end
glusterfs;
Md-cache caches depneds on timeout too, but md-cache can delay the timeout for some
cases.
Could you please list out which xattrs fall into that category. AFAIK,
like iatt, even all xattrs are invalidated post timeout in md-cache xlator.
By turning on both these caches, the process shall consume more memory
and CPUs to store and invalidate same set of attributes at two different
layers right. Do you see much better performance when compared to that
cost?
Thanks,
Soumya
>
>>
>> On 10/11/18 7:47 AM, Kinglong Mee wrote:
>>> Cc nfs-ganesha,
>>>
>>> Md-cache has option "cache-posix-acl" that controls caching of
posix ACLs
>>> ("system.posix_acl_access"/"system.posix_acl_default")
and virtual glusterfs ACLs
>>> ("glusterfs.posix.acl"/"glusterfs.posix.default_acl")
now.
>>>
>>> But, _posix_xattr_get_set does not fill virtual glusterfs ACLs when lookup
requests.
>>> So, md-cache caches bad virtual glusterfs ACLs.
>>>
>>> After I turn on "cache-posix-acl" option to cache ACLs at md-cache,
nfs client gets many EIO errors.
>>>
>>> https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/427305
>>>
>>> There are two chooses for cache virtual glusterfs ACLs in md-cache,
>>> 1. Cache it separately as posix ACLs (a new option maybe
"cache-glusterfs-acl" is added);
>>> And make sure _posix_xattr_get_set fills them when lookup requests.
>>>
>>
>> I am not sure if posix layer can handle it. Virtual xattrs are in-memory and not
stored on disk. They are converted to/from posix-acl in posix-acl xlator. So FWIU,
posix-acl xlator should handle setting these attributes as part of LOOKUP response if
needed. Same shall apply for any virtual xattr cached in md-cache. Request Poornima to
comment.
>
> Posix-acl can hand it correctly now.
>
>>
>> At a time, any gfapi consumer would use either posix-acl or virtual glusterfs
ACLs. So having two options to selectively choose which one of them to cache sounds better
to me instead of unnecessarily storing two different representations of the same ACL.
>
> Make sense.
> I will add another option for virtual glusterfs ACLs in md-cache.
>
> thanks,
> Kinglong Mee
>
>>
>> Thanks,
>> Soumya
>>
>>> 2. Does not cache it, only cache posix ACLs;
>>> If gfapi request it, md-cache lookup according posix ACL at cache,
>>> if exist, make the virtual glusterfs ACL locally and return to gfapi;
>>> otherwise, send the request to glusterfsd.
>>>
>>> Virtual glusterfs ACLs are another format of posix ACLs, there are larger
than posix ACLs,
>>> and always exist no matter the really posix ACL exist or not.
>>
>>>
>>> So, I'd prefer #2.
>>> Any comments are welcome.
>>>
>>> thanks,
>>> Kinglong Mee
>>>
>>> _______________________________________________
>>> Gluster-devel mailing list
>>> Gluster-devel(a)gluster.org
>>> https://lists.gluster.org/mailman/listinfo/gluster-devel
>>>
>>
> _______________________________________________
> Devel mailing list -- devel(a)lists.nfs-ganesha.org
> To unsubscribe send an email to devel-leave(a)lists.nfs-ganesha.org
>