Change in ffilz/nfs-ganesha[next]: Update Flex File Layout XDR defintion refrence: draft-ietf-nfsv4-flex...
by GerritHub
From <supriti.singh(a)suse.com>:
supriti.singh(a)suse.com has uploaded this change for review. ( https://review.gerrithub.io/411695
Change subject: Update Flex File Layout XDR defintion refrence: draft-ietf-nfsv4-flex-files-19
......................................................................
Update Flex File Layout XDR defintion
refrence: draft-ietf-nfsv4-flex-files-19
Change-Id: I61fdc6be74dcf0d16b33d769fec5ac06eca19c6d
Signed-off-by: Supriti Singh <supriti.singh(a)suse.com>
---
M src/Protocols/XDR/flex_files_prot.x
M src/include/nfsv41.h
2 files changed, 59 insertions(+), 12 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/95/411695/1
--
To view, visit https://review.gerrithub.io/411695
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-MessageType: newchange
Gerrit-Change-Id: I61fdc6be74dcf0d16b33d769fec5ac06eca19c6d
Gerrit-Change-Number: 411695
Gerrit-PatchSet: 1
Gerrit-Owner: supriti.singh(a)suse.com
6 years, 7 months
Change in ffilz/nfs-ganesha[next]: FSAL_PROXY : fix open_fd_count decrement
by GerritHub
From Patrice LUCAS <patrice.lucas(a)cea.fr>:
Patrice LUCAS has uploaded this change for review. ( https://review.gerrithub.io/411675
Change subject: FSAL_PROXY : fix open_fd_count decrement
......................................................................
FSAL_PROXY : fix open_fd_count decrement
We change the error status returned by pxy_close from
ERR_FSAL_NO_ERROR to ERR_FSAL_NOT_OPENED to avoid
decrement of global open_fd_count in fsal_close helper function.
Change-Id: I965dbe8624566cfa76b7e9ae69fab0faa1dcab65
Signed-off-by: Patrice LUCAS <patrice.lucas(a)cea.fr>
---
M src/FSAL/FSAL_PROXY/handle.c
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/75/411675/1
--
To view, visit https://review.gerrithub.io/411675
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-MessageType: newchange
Gerrit-Change-Id: I965dbe8624566cfa76b7e9ae69fab0faa1dcab65
Gerrit-Change-Number: 411675
Gerrit-PatchSet: 1
Gerrit-Owner: Patrice LUCAS <patrice.lucas(a)cea.fr>
6 years, 7 months
FSAL_PROXY : "open_fd_count is negative"
by patrice.lucas＠cea.fr
Since 2.7-dev.8 and the Daniel's patch fc5e13ecb "adding a message when
open_fd_count comes negative", FSAL_PROXY tests logs are full of these
critical messages : "open_fd_count is negative".
Should I worry about and try to fix it ? Or is this open_fd_count
deprecated and unused ?
Best regards,
Patrice
6 years, 7 months
FSAL Ceph + Pnfs
by Supriti Singh
Hello all,
I have been investigating on how to implement pnfs for FSAL CEPH [1]. As suggested by Jeff , I spent time reading about
flex file layout and wrote a POC just for my own understanding of pnfs protocol [2]. But as nfs-ganesha lacks support
for flex filelayout, just to ensure if libcephfs api work, I wrote existing code using file layout. I have few doubts
and would also like to understand whats the best way to breakdown the work here.
1. Advantage of using flex over file layout:
As I understood, the advantage is that in flex file layout storage nodes can be nfs version 3 or 4. Hence, we can
make the data path more simpler. Is there any other advantage wrt how ceph stores data (crush maps) that makes flex
better than file layout.
2. Implementing Flex file layout support
In existing code we have a file "flex_files_prot.x " but that does not match the latest rfc [1]. There are
some other data structures defined in nfsv41.h, but we would require to implement the functions. I will submit a patch.
3. While working on POC, there were some implementation issue as well:
- In LAYOUTGET, we populate opaque object device_id (16 bytes) and in GETDEVICEINFO, we should be able to send the
network address for the storage device. To get started, I assumed we will co-locate the osds with storage device. And in
GETDEVICEINFO, we use api ceph_ll_osdaddr to get the osd address. We will need some way to encode the osd_ids in the
device_id and pass it to GETDEVICEINFO. Its a implementation detail, but just want to be sure I am not overlooking
something obvious
- In LAYOUTGET, using ceph_ll_get_stripe_osd to get the osd corresponding to stripe. But I see a assert failure at [3],
and I have to investigate if its because of my ceph setup or bug in the code. And as there is no documentation for this
function, I am not sure exactly what is it doing.
Any suggestions are welcome.
[1] https://tools.ietf.org/html/draft-ietf-nfsv4-flex-files-19
[2] https://github.com/supriti/nfs-ganesha/tree/pnfs
[3] https://github.com/ceph/ceph/blob/master/src/osd/OSDMap.h#L1049
------
Supriti Singh SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton,
HRB 21284 (AG Nürnberg)
6 years, 7 months
[nfs-ganesha RFC PATCH v2 00/13] experimental rados_cluster recovery backend
by Jeff Layton
From: Jeff Layton <jlayton(a)redhat.com>
This is an update of the patchset I had originally posted back in late
January. The basic idea is to add a new rados_cluster recovery backend
that allows running ganesha servers to self-aggregate and work out the
grace period amongst themselves by following a set of simple rules and
indicating their current and desired state in a rados object.
The patchset starts by extending the recovery_backend operations
interface to cover handling the grace period. All of these collapse
down to being no-ops on the singleton recovery_backends.
It then adds a new support library that abstracts out management of the
shared rados object. This object tracks whether there is a cluster-wide
grace period in effect, and from what reboot epoch recovery is allowed.
It also allows the cluster nodes to indicate whether they need a grace
period (in order to allow recovery) and whether they are currently
enforcing the grace period.
Then, a new command-line tool for directly manipulating the shared
object. This gives an admin a way to do things like request a grace
period manually, remove a dead host from the cluster, and "fake up"
other nodes in the cluster for purposes of testing.
Finally, it adds a new recovery backend that plugs into the same
library to allow ganesha to participate as a clustered node.
The immediate aim here is to allow us to do an active/active export of
FSAL_CEPH from multiple heads, probably under some sort of container
orchestration (e.g., Kubernetes). The underlying design however should
be extendible to other clustered backends.
While this does work, it's still very much proof-of-concept code at this
point. There is quite a bit of room for improvement here so, I don't
think this is quite ready for merge, but I'd appreciate any early
feedback on the approach. Does anyone see any major red flags in this
design that I haven't yet spotted?
There is one prerequisite for this set -- it currently relies on a patch
to ceph that is not yet in tree (to allow ganesha to immediately kill
off the Ceph MDS session of its previous incarnation). That's still
under development, but it's fairly straightforward.
Jeff Layton (13):
HASHTABLE: add a hashtable_for_each function
reaper: add a way to wake up the reaper immediately
main: initialize recovery backend earlier
SAL: make some rados_kv symbols public
SAL: add new try_lift_grace recovery operation
SAL: add recovery operation to maybe start a grace period
SAL: add new set_enforcing operation
SAL: add a way to check for grace period being enforced cluster-wide
main: add way to stall server until grace is being enforced
support: add a rados_grace support library
tools: add new rados_grace manipulation tool
SAL: add new clustered RADOS recovery backend
FSAL_CEPH: kill off old session before the mount
src/CMakeLists.txt | 1 +
src/FSAL/FSAL_CEPH/main.c | 39 ++
src/MainNFSD/nfs_init.c | 8 -
src/MainNFSD/nfs_lib.c | 13 +
src/MainNFSD/nfs_main.c | 12 +
src/MainNFSD/nfs_reaper_thread.c | 11 +
src/SAL/CMakeLists.txt | 3 +-
src/SAL/nfs4_recovery.c | 90 ++-
src/SAL/recovery/recovery_rados.h | 6 +
src/SAL/recovery/recovery_rados_cluster.c | 406 +++++++++++++
src/SAL/recovery/recovery_rados_kv.c | 7 +-
src/cmake/modules/FindCEPHFS.cmake | 8 +
src/doc/man/ganesha-core-config.rst | 1 +
src/hashtable/hashtable.c | 17 +
src/include/config-h.in.cmake | 1 +
src/include/hashtable.h | 3 +
src/include/nfs_core.h | 1 +
src/include/rados_grace.h | 82 +++
src/include/sal_functions.h | 11 +-
src/nfs-ganesha.spec-in.cmake | 2 +
src/support/CMakeLists.txt | 4 +
src/support/rados_grace.c | 678 ++++++++++++++++++++++
src/tools/CMakeLists.txt | 4 +
src/tools/rados_grace_tool.c | 178 ++++++
24 files changed, 1567 insertions(+), 19 deletions(-)
create mode 100644 src/SAL/recovery/recovery_rados_cluster.c
create mode 100644 src/include/rados_grace.h
create mode 100644 src/support/rados_grace.c
create mode 100644 src/tools/rados_grace_tool.c
--
2.17.0
6 years, 7 months
Change in ffilz/nfs-ganesha[next]: build: add BuildRequires: wbclient-devel for MSPAC support
by GerritHub
From <kaleb(a)redhat.com>:
kaleb(a)redhat.com has uploaded this change for review. ( https://review.gerrithub.io/411562
Change subject: build: add BuildRequires: wbclient-devel for MSPAC support
......................................................................
build: add BuildRequires: wbclient-devel for MSPAC support
While trying to build FSAL_CEPH and/or FSAL_RGW for nfs-ganesha-2.5.5 in
Fedora 27, discovered that WBCLIENT_LIBRARIES is not found by cmake, i.e.
missing MSPAC support and libwbclient-devel. (Unclear why this is only
found when building the Ceph FSALs; with those not enabled it builds fine.)
jlayton's recent fix to FindWBclient.cmake in change
Ie1ba82261e1728f300037127d3efd3f22a70ebfe
allows cmake to silently disable MSPAC when wbclient(-devel) is not
installed. But that patch hasn't been backported to 2.5-stable (yet).
Independent of that, adding the BuildRequires so that we do get the
MSPAC support.
Change-Id: I8fb5abb278bbddd0ee8f93924737891c485a527f
Signed-off-by: Kaleb S. KEITHLEY <kkeithle(a)redhat.com>
---
M src/nfs-ganesha.spec-in.cmake
1 file changed, 1 insertion(+), 0 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/62/411562/1
--
To view, visit https://review.gerrithub.io/411562
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8fb5abb278bbddd0ee8f93924737891c485a527f
Gerrit-Change-Number: 411562
Gerrit-PatchSet: 1
Gerrit-Owner: kaleb(a)redhat.com
6 years, 7 months
Change in ffilz/nfs-ganesha[next]: Fixing getattr for fs locations
by GerritHub
From Sriram Patil <sriramp(a)vmware.com>:
Sriram Patil has uploaded this change for review. ( https://review.gerrithub.io/411514
Change subject: Fixing getattr for fs locations
......................................................................
Fixing getattr for fs locations
Fixed encode_fs_locations to allow using referral points other than
directories. Also, removed an assert in nfs4_op_getattr which cause
ganesha to crash if FSAL is not able to return fs locations.
Change-Id: I79c1c56f4c9669d0a09e80d224fa3e3e3d5ba2e7
Signed-off-by: Sriram Patil <sriramp(a)vmware.com>
---
M src/Protocols/NFS/nfs4_op_getattr.c
M src/Protocols/NFS/nfs_proto_tools.c
2 files changed, 0 insertions(+), 6 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/14/411514/1
--
To view, visit https://review.gerrithub.io/411514
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-MessageType: newchange
Gerrit-Change-Id: I79c1c56f4c9669d0a09e80d224fa3e3e3d5ba2e7
Gerrit-Change-Number: 411514
Gerrit-PatchSet: 1
Gerrit-Owner: Sriram Patil <sriramp(a)vmware.com>
6 years, 7 months
Change in ffilz/nfs-ganesha[next]: gtest/test_mknode_latency: mknode latency microbenchmark
by GerritHub
From Girjesh Rajoria <grajoria(a)redhat.com>:
Girjesh Rajoria has uploaded this change for review. ( https://review.gerrithub.io/411382
Change subject: gtest/test_mknode_latency: mknode latency microbenchmark
......................................................................
gtest/test_mknode_latency: mknode latency microbenchmark
gtest unit test that run mknode on single case, large loops
and bypass cases to find average latency of the calls.
Change-Id: I754c13832c898e161c3054b65f276af493d2e597
Signed-off-by: Girjesh Rajoria <grajoria(a)redhat.com>
---
M src/gtest/CMakeLists.txt
A src/gtest/test_mknode_latency.cc
2 files changed, 450 insertions(+), 0 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/82/411382/1
--
To view, visit https://review.gerrithub.io/411382
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-MessageType: newchange
Gerrit-Change-Id: I754c13832c898e161c3054b65f276af493d2e597
Gerrit-Change-Number: 411382
Gerrit-PatchSet: 1
Gerrit-Owner: Girjesh Rajoria <grajoria(a)redhat.com>
6 years, 7 months
Don't open global fd for getattr/setattr
by Frank Filz
I just posted this patch: https://review.gerrithub.io/411347 to address the
use of the global fd for getattr/setattr. This has caused concern by various
users.
I also thought of an API change we could make that could help things. In NFS
v4, there will be GETATTR calls after I/O which may have used a stateid,
however, the GETATTR itself does not use the stateid. We could still fetch
the state_t and have an option of passing that on the GETATTR if available.
That would then allow use of the state_t file descriptor instead of opening
the temporary file descriptor when we already have a state_t file descriptor
that would work just fine for the getattr. That API could also be utilized
if I ever get around to coding up a search for lock or share state for NFS
v3 I/O.
Frank
6 years, 7 months
Change in ffilz/nfs-ganesha[next]: Use temporary fd instead of global fd for getattr/setattr/LOCKT
by GerritHub
From Frank Filz <ffilzlnx(a)mindspring.com>:
Frank Filz has uploaded this change for review. ( https://review.gerrithub.io/411347
Change subject: Use temporary fd instead of global fd for getattr/setattr/LOCKT
......................................................................
Use temporary fd instead of global fd for getattr/setattr/LOCKT
Some use cases have shown issues with opening a global file descriptor
for getattr/setattr which then remains open until the LRU thread gets
around to closing it. This can result in a lot of open fd, or preventing
Samba clients from opening the file.
This patch simply prevents opening the global file descriptor when the
FSAL_O_ANY openflags mode is selected, instead if the global file
descriptor is NOT open, it opens a temporary file descriptor. If the
global file descriptor IS open, it will still be utilized, so we avoid
extraneous open for getattr that occurs amidst anonymous I/O.
Change-Id: Ic2283e7150b1e0797cc2017016ab5c298ba7232a
Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com>
---
M src/FSAL/commonlib.c
1 file changed, 35 insertions(+), 5 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/47/411347/1
--
To view, visit https://review.gerrithub.io/411347
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic2283e7150b1e0797cc2017016ab5c298ba7232a
Gerrit-Change-Number: 411347
Gerrit-PatchSet: 1
Gerrit-Owner: Frank Filz <ffilzlnx(a)mindspring.com>
6 years, 7 months
