May 2018 - Devel - NFS Ganesha List Archives

Change in ffilz/nfs-ganesha[next]: Update Flex File Layout XDR defintion refrence: draft-ietf-nfsv4-flex...
by GerritHub 18 May '18

18 May '18

From <supriti.singh(a)suse.com>: supriti.singh(a)suse.com has uploaded this change for review. ( https://review.gerrithub.io/411695 Change subject: Update Flex File Layout XDR defintion refrence: draft-ietf-nfsv4-flex-files-19 ...................................................................... Update Flex File Layout XDR defintion refrence: draft-ietf-nfsv4-flex-files-19 Change-Id: I61fdc6be74dcf0d16b33d769fec5ac06eca19c6d Signed-off-by: Supriti Singh <supriti.singh(a)suse.com> --- M src/Protocols/XDR/flex_files_prot.x M src/include/nfsv41.h 2 files changed, 59 insertions(+), 12 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/95/411695/1 -- To view, visit https://review.gerrithub.io/411695 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-MessageType: newchange Gerrit-Change-Id: I61fdc6be74dcf0d16b33d769fec5ac06eca19c6d Gerrit-Change-Number: 411695 Gerrit-PatchSet: 1 Gerrit-Owner: supriti.singh(a)suse.com

1 0

Change in ffilz/nfs-ganesha[next]: FSAL_PROXY : fix open_fd_count decrement
by GerritHub 18 May '18

18 May '18

From Patrice LUCAS <patrice.lucas(a)cea.fr>: Patrice LUCAS has uploaded this change for review. ( https://review.gerrithub.io/411675 Change subject: FSAL_PROXY : fix open_fd_count decrement ...................................................................... FSAL_PROXY : fix open_fd_count decrement We change the error status returned by pxy_close from ERR_FSAL_NO_ERROR to ERR_FSAL_NOT_OPENED to avoid decrement of global open_fd_count in fsal_close helper function. Change-Id: I965dbe8624566cfa76b7e9ae69fab0faa1dcab65 Signed-off-by: Patrice LUCAS <patrice.lucas(a)cea.fr> --- M src/FSAL/FSAL_PROXY/handle.c 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/75/411675/1 -- To view, visit https://review.gerrithub.io/411675 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-MessageType: newchange Gerrit-Change-Id: I965dbe8624566cfa76b7e9ae69fab0faa1dcab65 Gerrit-Change-Number: 411675 Gerrit-PatchSet: 1 Gerrit-Owner: Patrice LUCAS <patrice.lucas(a)cea.fr>

1 0

FSAL_PROXY : "open_fd_count is negative"
by patrice.lucas＠cea.fr 18 May '18

18 May '18

Since 2.7-dev.8 and the Daniel's patch fc5e13ecb "adding a message when open_fd_count comes negative", FSAL_PROXY tests logs are full of these critical messages : "open_fd_count is negative". Should I worry about and try to fix it ? Or is this open_fd_count deprecated and unused ? Best regards, Patrice

4 10

FSAL Ceph + Pnfs
by Supriti Singh 17 May '18

17 May '18

Hello all, I have been investigating on how to implement pnfs for FSAL CEPH [1]. As suggested by Jeff , I spent time reading about flex file layout and wrote a POC just for my own understanding of pnfs protocol [2]. But as nfs-ganesha lacks support for flex filelayout, just to ensure if libcephfs api work, I wrote existing code using file layout. I have few doubts and would also like to understand whats the best way to breakdown the work here. 1. Advantage of using flex over file layout: As I understood, the advantage is that in flex file layout storage nodes can be nfs version 3 or 4. Hence, we can make the data path more simpler. Is there any other advantage wrt how ceph stores data (crush maps) that makes flex better than file layout. 2. Implementing Flex file layout support In existing code we have a file "flex_files_prot.x " but that does not match the latest rfc [1]. There are some other data structures defined in nfsv41.h, but we would require to implement the functions. I will submit a patch. 3. While working on POC, there were some implementation issue as well: - In LAYOUTGET, we populate opaque object device_id (16 bytes) and in GETDEVICEINFO, we should be able to send the network address for the storage device. To get started, I assumed we will co-locate the osds with storage device. And in GETDEVICEINFO, we use api ceph_ll_osdaddr to get the osd address. We will need some way to encode the osd_ids in the device_id and pass it to GETDEVICEINFO. Its a implementation detail, but just want to be sure I am not overlooking something obvious - In LAYOUTGET, using ceph_ll_get_stripe_osd to get the osd corresponding to stripe. But I see a assert failure at [3], and I have to investigate if its because of my ceph setup or bug in the code. And as there is no documentation for this function, I am not sure exactly what is it doing. Any suggestions are welcome. [1] https://tools.ietf.org/html/draft-ietf-nfsv4-flex-files-19 [2] https://github.com/supriti/nfs-ganesha/tree/pnfs [3] https://github.com/ceph/ceph/blob/master/src/osd/OSDMap.h#L1049 ------ Supriti Singh SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

2 1

[nfs-ganesha RFC PATCH v2 00/13] experimental rados_cluster recovery backend
by Jeff Layton 17 May '18

17 May '18

From: Jeff Layton <jlayton(a)redhat.com> This is an update of the patchset I had originally posted back in late January. The basic idea is to add a new rados_cluster recovery backend that allows running ganesha servers to self-aggregate and work out the grace period amongst themselves by following a set of simple rules and indicating their current and desired state in a rados object. The patchset starts by extending the recovery_backend operations interface to cover handling the grace period. All of these collapse down to being no-ops on the singleton recovery_backends. It then adds a new support library that abstracts out management of the shared rados object. This object tracks whether there is a cluster-wide grace period in effect, and from what reboot epoch recovery is allowed. It also allows the cluster nodes to indicate whether they need a grace period (in order to allow recovery) and whether they are currently enforcing the grace period. Then, a new command-line tool for directly manipulating the shared object. This gives an admin a way to do things like request a grace period manually, remove a dead host from the cluster, and "fake up" other nodes in the cluster for purposes of testing. Finally, it adds a new recovery backend that plugs into the same library to allow ganesha to participate as a clustered node. The immediate aim here is to allow us to do an active/active export of FSAL_CEPH from multiple heads, probably under some sort of container orchestration (e.g., Kubernetes). The underlying design however should be extendible to other clustered backends. While this does work, it's still very much proof-of-concept code at this point. There is quite a bit of room for improvement here so, I don't think this is quite ready for merge, but I'd appreciate any early feedback on the approach. Does anyone see any major red flags in this design that I haven't yet spotted? There is one prerequisite for this set -- it currently relies on a patch to ceph that is not yet in tree (to allow ganesha to immediately kill off the Ceph MDS session of its previous incarnation). That's still under development, but it's fairly straightforward. Jeff Layton (13): HASHTABLE: add a hashtable_for_each function reaper: add a way to wake up the reaper immediately main: initialize recovery backend earlier SAL: make some rados_kv symbols public SAL: add new try_lift_grace recovery operation SAL: add recovery operation to maybe start a grace period SAL: add new set_enforcing operation SAL: add a way to check for grace period being enforced cluster-wide main: add way to stall server until grace is being enforced support: add a rados_grace support library tools: add new rados_grace manipulation tool SAL: add new clustered RADOS recovery backend FSAL_CEPH: kill off old session before the mount src/CMakeLists.txt | 1 + src/FSAL/FSAL_CEPH/main.c | 39 ++ src/MainNFSD/nfs_init.c | 8 - src/MainNFSD/nfs_lib.c | 13 + src/MainNFSD/nfs_main.c | 12 + src/MainNFSD/nfs_reaper_thread.c | 11 + src/SAL/CMakeLists.txt | 3 +- src/SAL/nfs4_recovery.c | 90 ++- src/SAL/recovery/recovery_rados.h | 6 + src/SAL/recovery/recovery_rados_cluster.c | 406 +++++++++++++ src/SAL/recovery/recovery_rados_kv.c | 7 +- src/cmake/modules/FindCEPHFS.cmake | 8 + src/doc/man/ganesha-core-config.rst | 1 + src/hashtable/hashtable.c | 17 + src/include/config-h.in.cmake | 1 + src/include/hashtable.h | 3 + src/include/nfs_core.h | 1 + src/include/rados_grace.h | 82 +++ src/include/sal_functions.h | 11 +- src/nfs-ganesha.spec-in.cmake | 2 + src/support/CMakeLists.txt | 4 + src/support/rados_grace.c | 678 ++++++++++++++++++++++ src/tools/CMakeLists.txt | 4 + src/tools/rados_grace_tool.c | 178 ++++++ 24 files changed, 1567 insertions(+), 19 deletions(-) create mode 100644 src/SAL/recovery/recovery_rados_cluster.c create mode 100644 src/include/rados_grace.h create mode 100644 src/support/rados_grace.c create mode 100644 src/tools/rados_grace_tool.c -- 2.17.0

2 16

Change in ffilz/nfs-ganesha[next]: build: add BuildRequires: wbclient-devel for MSPAC support
by GerritHub 17 May '18

17 May '18

From <kaleb(a)redhat.com>: kaleb(a)redhat.com has uploaded this change for review. ( https://review.gerrithub.io/411562 Change subject: build: add BuildRequires: wbclient-devel for MSPAC support ...................................................................... build: add BuildRequires: wbclient-devel for MSPAC support While trying to build FSAL_CEPH and/or FSAL_RGW for nfs-ganesha-2.5.5 in Fedora 27, discovered that WBCLIENT_LIBRARIES is not found by cmake, i.e. missing MSPAC support and libwbclient-devel. (Unclear why this is only found when building the Ceph FSALs; with those not enabled it builds fine.) jlayton's recent fix to FindWBclient.cmake in change Ie1ba82261e1728f300037127d3efd3f22a70ebfe allows cmake to silently disable MSPAC when wbclient(-devel) is not installed. But that patch hasn't been backported to 2.5-stable (yet). Independent of that, adding the BuildRequires so that we do get the MSPAC support. Change-Id: I8fb5abb278bbddd0ee8f93924737891c485a527f Signed-off-by: Kaleb S. KEITHLEY <kkeithle(a)redhat.com> --- M src/nfs-ganesha.spec-in.cmake 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/62/411562/1 -- To view, visit https://review.gerrithub.io/411562 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-MessageType: newchange Gerrit-Change-Id: I8fb5abb278bbddd0ee8f93924737891c485a527f Gerrit-Change-Number: 411562 Gerrit-PatchSet: 1 Gerrit-Owner: kaleb(a)redhat.com

1 0

Change in ffilz/nfs-ganesha[next]: Fixing getattr for fs locations
by GerritHub 17 May '18

17 May '18

From Sriram Patil <sriramp(a)vmware.com>: Sriram Patil has uploaded this change for review. ( https://review.gerrithub.io/411514 Change subject: Fixing getattr for fs locations ...................................................................... Fixing getattr for fs locations Fixed encode_fs_locations to allow using referral points other than directories. Also, removed an assert in nfs4_op_getattr which cause ganesha to crash if FSAL is not able to return fs locations. Change-Id: I79c1c56f4c9669d0a09e80d224fa3e3e3d5ba2e7 Signed-off-by: Sriram Patil <sriramp(a)vmware.com> --- M src/Protocols/NFS/nfs4_op_getattr.c M src/Protocols/NFS/nfs_proto_tools.c 2 files changed, 0 insertions(+), 6 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/14/411514/1 -- To view, visit https://review.gerrithub.io/411514 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-MessageType: newchange Gerrit-Change-Id: I79c1c56f4c9669d0a09e80d224fa3e3e3d5ba2e7 Gerrit-Change-Number: 411514 Gerrit-PatchSet: 1 Gerrit-Owner: Sriram Patil <sriramp(a)vmware.com>

1 0

Change in ffilz/nfs-ganesha[next]: gtest/test_mknode_latency: mknode latency microbenchmark
by GerritHub 16 May '18

16 May '18

From Girjesh Rajoria <grajoria(a)redhat.com>: Girjesh Rajoria has uploaded this change for review. ( https://review.gerrithub.io/411382 Change subject: gtest/test_mknode_latency: mknode latency microbenchmark ...................................................................... gtest/test_mknode_latency: mknode latency microbenchmark gtest unit test that run mknode on single case, large loops and bypass cases to find average latency of the calls. Change-Id: I754c13832c898e161c3054b65f276af493d2e597 Signed-off-by: Girjesh Rajoria <grajoria(a)redhat.com> --- M src/gtest/CMakeLists.txt A src/gtest/test_mknode_latency.cc 2 files changed, 450 insertions(+), 0 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/82/411382/1 -- To view, visit https://review.gerrithub.io/411382 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-MessageType: newchange Gerrit-Change-Id: I754c13832c898e161c3054b65f276af493d2e597 Gerrit-Change-Number: 411382 Gerrit-PatchSet: 1 Gerrit-Owner: Girjesh Rajoria <grajoria(a)redhat.com>

1 0

Don't open global fd for getattr/setattr
by Frank Filz 15 May '18

15 May '18

I just posted this patch: https://review.gerrithub.io/411347 to address the use of the global fd for getattr/setattr. This has caused concern by various users. I also thought of an API change we could make that could help things. In NFS v4, there will be GETATTR calls after I/O which may have used a stateid, however, the GETATTR itself does not use the stateid. We could still fetch the state_t and have an option of passing that on the GETATTR if available. That would then allow use of the state_t file descriptor instead of opening the temporary file descriptor when we already have a state_t file descriptor that would work just fine for the getattr. That API could also be utilized if I ever get around to coding up a search for lock or share state for NFS v3 I/O. Frank

1 0

Change in ffilz/nfs-ganesha[next]: Use temporary fd instead of global fd for getattr/setattr/LOCKT
by GerritHub 15 May '18

15 May '18

From Frank Filz <ffilzlnx(a)mindspring.com>: Frank Filz has uploaded this change for review. ( https://review.gerrithub.io/411347 Change subject: Use temporary fd instead of global fd for getattr/setattr/LOCKT ...................................................................... Use temporary fd instead of global fd for getattr/setattr/LOCKT Some use cases have shown issues with opening a global file descriptor for getattr/setattr which then remains open until the LRU thread gets around to closing it. This can result in a lot of open fd, or preventing Samba clients from opening the file. This patch simply prevents opening the global file descriptor when the FSAL_O_ANY openflags mode is selected, instead if the global file descriptor is NOT open, it opens a temporary file descriptor. If the global file descriptor IS open, it will still be utilized, so we avoid extraneous open for getattr that occurs amidst anonymous I/O. Change-Id: Ic2283e7150b1e0797cc2017016ab5c298ba7232a Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com> --- M src/FSAL/commonlib.c 1 file changed, 35 insertions(+), 5 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/47/411347/1 -- To view, visit https://review.gerrithub.io/411347 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-MessageType: newchange Gerrit-Change-Id: Ic2283e7150b1e0797cc2017016ab5c298ba7232a Gerrit-Change-Number: 411347 Gerrit-PatchSet: 1 Gerrit-Owner: Frank Filz <ffilzlnx(a)mindspring.com>

1 0