Hi Bjorn, Bjorn Leffler wrote on Sun, Jan 27, 2019 at 04:40:06PM -0800:
I'm a software engineer at Google, working on cloud and NFS related projects. I've been working with the Ganesha codebase to implement data caching as stackable FSALs. These can be stacked onto any other FSAL, but my immediate goal is to use this on top of FSAL_PROXY. That turns Ganesha into an efficient NFS caching proxy across WAN links, for example between different on-premise locations and/or cloud providers.
Welcome! Great to see interest in FSAL_PROXY, there haven't been many users recently and it's been a struggle to keep it up and running. A developer on that area is definitely more than welcome.
I've got a few patches that I'd like to contribute to the main code base: 1. A small change to make FSAL_NULL reusable. 2. Add a generic FSAL_DATACACHE for the caching logic. 3. Add specific cache implementation. I've implemented two: Memcache and local disk/ssd.
It would probably makes sense to implement some a ram only cache as well.
Would be curious to see how the local disk/ssd works before answering that, is it a local fs? using the disk as a block device directly? Either way there also are ways of working with ram there, and memcached is basically a ram backend as well, I'm not sure it makes sense to have many options that aren't used often - most other fsal have some other form of data cache (e.g. VFS has the VFS cache, gluster/ceph have some cache in the libs afaik? etc) so I'd rather keep a minimal set of such FSALs and focus on keeping them working.
Memcache may sound like a strange choice, but is interesting because of the scale out nature, and that it's already battle tested in large scale production. Initial performance with memcache/ssd caches looks really good: I'm getting up to 1.5 GB/s for streaming reads, which is slightly better than I get from the kernel NFS server serving from ram.
Nice results! How do you handle stuff like cache invalidation, send a stat to the real server and look at ctime? Well, either way feel free to send the patches to gerrit, we won't bite. Our wiki is a bit of a mess but you should have the minimum required informatinos here: https://github.com/nfs-ganesha/nfs-ganesha/wiki/DevPolicy#Pushing_to_gerrit
I'm based in Sydney, Australia, so it's tricky for me to join the weekly call, which is ~2.30am my time. I'm currently in the US and plan to join the weekly call on Tuesday.
I'm boycotting the call as well when it's just half past midnight here (JST), we can find other ways to work :) In particular there often are people in #ganesha on freenode if you have any question. -- Dominique