Issues seen with krb5i and krb5p mounts
by Trishali Nayar
Hi all,
There are a few clients eg- Ubuntu 18.04.3 (4.15.0-55-generic) and RH7.8
(3.10.0-1127.el7.x86_64) for which we have observed... simple command like
'dd' either hangs or returns EIO. This is happening only on krb5i and krb5p
mounts. It seems to happen for file sizes eg- 100MB and larger mostly. But
sometimes even a 30 MB file sees failures.
A client eg- RH7.6 (3.10.0-957.el7.x86_64) does not seem to hit this
issue...so might be with more recent kernels?
We fixed the issue with check-in
https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/490802 The idea was to
let clients know that Ganesha denied the request VS just dropping the
request.
This fix did seem to help and hangs/errors stopped completely... but for
larger file sizes eg- 1000MB we started seeing "Permission Denied" errors.
This was different than the EIO errors seen earlier. Reason could be we are
now sending an "AUTH DENIED" error so clients translate it to this new
error.
We tried to add more logging into Ganesha and observe that these particular
clients seem to send a lot of requests together. When we process same, the
sequence no. is pretty much out or order and we drop the requests outside
the sequence window, as per the RFC 2203 Section 7.2.1. The sequence window
that we have is 32.
Testing these clients with kNFS does not hit the issue...The kNFS sequence
window seems to be larger and is 128.
So, tried to increase the sequence window as well to 128 for ganesha. That
does not seem to help fix the issue.
We also have below additional 'seqmask' check and many of the requests went
into that category as well and were dropped.
"libntirpc/src/svc_auth_gss.c":
} else if (offset >= gd->win || (gd->seqmask & (1 << offset))) {
*no_dispatch = true;
goto gd_free;
}
Also observed that now these clients sent many requests above the 128
window...which we would again drop.
Wondering what is the proper way to fix this and any idea on what these
clients are doing different.
Thanks and regards,
Trishali.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Trishali Nayar
IBM Systems
ETZ, Pune.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3 years, 6 months
Change in ...nfs-ganesha[next]: Gather unix uid and gid using wbcGetpwsid
by Name of user not set (GerritHub)
blankburian(a)wwu.de has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/504443 )
Change subject: Gather unix uid and gid using wbcGetpwsid
......................................................................
Gather unix uid and gid using wbcGetpwsid
Currently, the primary group is determined from the second sid in the sids
list of the user info. When the flag unix_primary_group is used in the
winbind configuration, the primary gid is instead being looked up from attribute
primaryGroupID while the sids list keeps the same order. The function
wbcGetpwsid asks winbind to determine the primary gid. It also returns the
uid, so that the call to wbcSidToUid can be omitted.
Change-Id: I919dfb1b32b4f51353efee5a3482cf9aac498b3a
Signed-off-by: Markus Blank-Burian <blankburian(a)wwu.de>
---
M src/idmapper/idmapper.c
1 file changed, 7 insertions(+), 18 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/43/504443/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/504443
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: I919dfb1b32b4f51353efee5a3482cf9aac498b3a
Gerrit-Change-Number: 504443
Gerrit-PatchSet: 1
Gerrit-Owner: blankburian(a)wwu.de
Gerrit-MessageType: newchange
3 years, 6 months
Change in ...nfs-ganesha[next]: Honor NFS_COMMIT export option.
by Malahal (GerritHub)
Malahal has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/504428 )
Change subject: Honor NFS_COMMIT export option.
......................................................................
Honor NFS_COMMIT export option.
This was lost when we moved to support_ex API. Made changes in
nfs3_write and nfs4_write to handle this.
Change-Id: I93ebdb3f9c195f8b7c3e1b8e492f41177f44d206
Signed-off-by: Malahal Naineni <malahal(a)us.ibm.com>
---
M src/Protocols/NFS/nfs3_write.c
M src/Protocols/NFS/nfs4_op_write.c
2 files changed, 6 insertions(+), 6 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/28/504428/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/504428
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: I93ebdb3f9c195f8b7c3e1b8e492f41177f44d206
Gerrit-Change-Number: 504428
Gerrit-PatchSet: 1
Gerrit-Owner: Malahal <malahal(a)gmail.com>
Gerrit-MessageType: newchange
3 years, 6 months
Announce Push of V4-dev.38
by Frank Filz
Branch next
Tag:V4-dev.38
Merge Highlights
* Fix double export ref with set_op_context_export calls
* Fix execute only permission check (EPERM vs EACCESS confustion)
* Update pynfs URLs
* Set root of FSAL_MEM exports to writeable by all
* FSAL_MEM: remove redundant wrlock acquire
* Documenting the Security_Label option
* ProxyV3: Issue last-chance LOOKUP in READDIRPLUS.
Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com>
Contents:
711f255 Frank S. Filz V4-dev.38
d7939db Solomon Boulos ProxyV3: Issue last-chance LOOKUP in READDIRPLUS.
6038a78 Arjun Sharma Documenting the Security_Label option
131b3de Matthew DeVore FSAL_MEM: remove redundant wrlock acquire
766c07a Matthew DeVore Set root of FSAL_MEM exports to writeable by all
fd261a6 Matthew DeVore Update pynfs URLs
38c236c Frank S. Filz Fix execute only permission check (EPERM vs EACCESS
confustion)
10874a5 Frank S. Filz Fix double export ref with set_op_context_export calls
3 years, 6 months
Change in ...nfs-ganesha[next]: Fix execute only permission check (EPERM vs EACCESS confustion)
by Frank Filz (GerritHub)
Frank Filz has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/504062 )
Change subject: Fix execute only permission check (EPERM vs EACCESS confustion)
......................................................................
Fix execute only permission check (EPERM vs EACCESS confustion)
Permission check failures return EACCESS/ERR_FSAL_ACCESS not
EPERM/ERR_FSAL_PERM.
Also put in some logging.
Change-Id: Ie71bee342f61d0e3060f65bc46a3a755a55b08a5
Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com>
---
M src/FSAL/fsal_helper.c
1 file changed, 9 insertions(+), 1 deletion(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/62/504062/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/504062
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: Ie71bee342f61d0e3060f65bc46a3a755a55b08a5
Gerrit-Change-Number: 504062
Gerrit-PatchSet: 1
Gerrit-Owner: Frank Filz <ffilzlnx(a)mindspring.com>
Gerrit-MessageType: newchange
3 years, 6 months
Change in ...nfs-ganesha[next]: FSAL_MEM: remove redundant wrlock acquire
by Matthew DeVore (GerritHub)
Matthew DeVore has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/504056 )
Change subject: FSAL_MEM: remove redundant wrlock acquire
......................................................................
FSAL_MEM: remove redundant wrlock acquire
This function is only ever called after a rwlock is acquired by
fsal_reopen_obj at src/FSAL/commonlib.c:2756. In mem_close_func, use the
same implementation pattern as used in e.g. glusterfs_close_func and
other _close_func's.
This avoids a fatal EDEADLK from occuring in the st_write.testStateidOne
test in pynfs. This is necessary but not sufficient to get FSAL_MEM
passing in pynfs.
Change-Id: I65c45aafdfd86b4ba9d1e87901b7a3f74df9f034
Signed-off-by: Matthew DeVore <matvore(a)google.com>
---
M src/FSAL/FSAL_MEM/mem_handle.c
M src/FSAL/commonlib.c
2 files changed, 8 insertions(+), 11 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/56/504056/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/504056
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: I65c45aafdfd86b4ba9d1e87901b7a3f74df9f034
Gerrit-Change-Number: 504056
Gerrit-PatchSet: 1
Gerrit-Owner: Matthew DeVore <matvore(a)google.com>
Gerrit-MessageType: newchange
3 years, 6 months
Change in ...nfs-ganesha[next]: Fix double export ref with set_op_context_export calls
by Frank Filz (GerritHub)
Frank Filz has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/504037 )
Change subject: Fix double export ref with set_op_context_export calls
......................................................................
Fix double export ref with set_op_context_export calls
Several invocations of set_op_context_export assume it's a function
not a macro and have a "parameter" with side effects that must only
occur once. Use a local variable inside the macro to assure these
side effects only occur once.
Change-Id: I583cbbbcd633ecc6c17982bf55be0c6f3661641b
Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com>
---
M src/include/fsal.h
1 file changed, 5 insertions(+), 2 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/37/504037/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/504037
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: I583cbbbcd633ecc6c17982bf55be0c6f3661641b
Gerrit-Change-Number: 504037
Gerrit-PatchSet: 1
Gerrit-Owner: Frank Filz <ffilzlnx(a)mindspring.com>
Gerrit-MessageType: newchange
3 years, 6 months
Change in ...nfs-ganesha[next]: Fix redundant refcnt with set op context marco
by freeze (GerritHub)
freeze has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/503998 )
Change subject: Fix redundant refcnt with set op context marco
......................................................................
Fix redundant refcnt with set op context marco
The set_op_context_export macro need a export to checking something
like fsal_export exist or not.
When we use the get_gsh_export_by_pseudo to get export, that will
increase the reference count.
If using the get_gsh_export_by_pseudo directly with
set_op_context_export, that will call three times (to get export)
then the redundant reference count is counted.
That would case that the obj handles still exists when destroy_fsal.
Depends on different implementation of release handle.
Sometimes meet the SEGFULT because op_ctx is null in ADMIN thread.
In destroy_fsal stage, all obj handles should be released.
Change-Id: Icf6670bbf7bfcdf5c46be5cb53b3095a2d93edcb
Signed-off-by: Vicente Cheng <freeze.bilsted(a)gmail.com>
---
M src/Protocols/NFS/nfs4_op_putrootfh.c
M src/Protocols/NFS/nfs4_pseudo.c
2 files changed, 7 insertions(+), 2 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/98/503998/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/503998
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: Icf6670bbf7bfcdf5c46be5cb53b3095a2d93edcb
Gerrit-Change-Number: 503998
Gerrit-PatchSet: 1
Gerrit-Owner: freeze <freeze.bilsted(a)gmail.com>
Gerrit-MessageType: newchange
3 years, 6 months
Change in ...nfs-ganesha[next]: Update pynfs URLs
by Matthew DeVore (GerritHub)
Matthew DeVore has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/503994 )
Change subject: Update pynfs URLs
......................................................................
Update pynfs URLs
Change-Id: I53ee26644156c2b3e7a08e2d6c320e60ea54203d
Signed-off-by: Matthew DeVore <matvore(a)google.com>
---
M jenkins/sigmund_as_root.rc
M src/doc/Resources.txt
2 files changed, 3 insertions(+), 3 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/94/503994/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/503994
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: I53ee26644156c2b3e7a08e2d6c320e60ea54203d
Gerrit-Change-Number: 503994
Gerrit-PatchSet: 1
Gerrit-Owner: Matthew DeVore <matvore(a)google.com>
Gerrit-MessageType: newchange
3 years, 6 months
Change in ...nfs-ganesha[next]: ProxyV3: Issue last-chance LOOKUP in READDIRPLUS.
by Solomon Boulos (GerritHub)
Solomon Boulos has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/503953 )
Change subject: ProxyV3: Issue last-chance LOOKUP in READDIRPLUS.
......................................................................
ProxyV3: Issue last-chance LOOKUP in READDIRPLUS.
READDIR responses can also be "here's the name" with no handle
or attributes. For example, knfsd (and NetApp) return just the
name for nested mounts/exports. The NFS client is then responsible
for issuing a LOOKUP in response.
Until we add whatever support is needed in Ganesha to return these
sort of partial responses ("just a name", or "name and handle, but
not attributes"), the proxy needs to act more like a full client.
This also finally pushed me to move the readdir entry loop into a
little helper function. The indentation was out of control :).
Change-Id: I96ba4b8791fd25e8273b46df873c812741c2acbe
Signed-off-by: Solomon Boulos <boulos(a)google.com>
---
M src/FSAL/FSAL_PROXY_V3/main.c
1 file changed, 171 insertions(+), 113 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/53/503953/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/503953
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: I96ba4b8791fd25e8273b46df873c812741c2acbe
Gerrit-Change-Number: 503953
Gerrit-PatchSet: 1
Gerrit-Owner: Solomon Boulos <boulos(a)google.com>
Gerrit-MessageType: newchange
3 years, 6 months
