November 2022 - Devel - NFS Ganesha List Archives

[M] Change in ...nfs-ganesha[next]: ganesha_mgr:Fix show exports/clients "IndexError:"
by Name of user not set (GerritHub) 21 Nov '22

21 Nov '22

skmprabhu2(a)gmail.com has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546426 ) Change subject: ganesha_mgr:Fix show exports/clients "IndexError:" ...................................................................... ganesha_mgr:Fix show exports/clients "IndexError:" $ ganesha_mgr show exports Show exports Traceback (most recent call last): File "/usr/bin/ganesha_mgr", line 470, in <module> exportmgr.showexports() File "/usr/bin/ganesha_mgr", line 92, in showexports status, msg, reply = self.exportmgr.ShowExports() File "/usr/lib/python3.6/site-packages/Ganesha/ganesha_mgr_utils.py", line 223, in ShowExports lasttime = ex[10] IndexError: tuple index out of range $ ganesha_mgr show clients Show clients Traceback (most recent call last): File "/usr/bin/ganesha_mgr", line 466, in <module> clientmgr.showclients() File "/usr/bin/ganesha_mgr", line 56, in showclients status, errormsg, reply = self.clientmgr.ShowClients() File "/usr/lib/python3.6/site-packages/Ganesha/ganesha_mgr_utils.py", line 94, in ShowClients lasttime = cl_[9] IndexError: tuple index out of range This is regression of commit id :0c1047500d2. commit id : 0c1047500d2 has changed format of show exports[clints] return value in server_stats_summary() Signed-off-by: Prabhu Murugesan <skmprabhu2(a)gmail.com> Change-Id: I173271b8395a4314f5e27fb4f9e7849aef9c5f99 --- M src/scripts/ganeshactl/Ganesha/ganesha_mgr_utils.py 1 file changed, 86 insertions(+), 23 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/26/546426/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546426 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I173271b8395a4314f5e27fb4f9e7849aef9c5f99 Gerrit-Change-Number: 546426 Gerrit-PatchSet: 1 Gerrit-Owner: skmprabhu2(a)gmail.com Gerrit-MessageType: newchange

1 0

Announce Push of V4.1
by Frank Filz 18 Nov '22

18 Nov '22

Branch next Tag:V4.1 NOTE: This merge includes an ntirpc pullup. Please update your submodule. Release Notes can be found on the github project wiki: https://github.com/nfs-ganesha/nfs-ganesha/wiki/ReleaseNotes_4 Merge Highlights * ntirpc 4.1 pullup * FSAL_PROXY_V3: Ensure that the used xid is unique * build: use CMAKE_DL_LIBS * Retry communication with NSM service * init_fds_limit(): Log EVENT to print current set FD limit * Add -C option to Ganesha daemon startup * Fix ganesha_stats with dbus import Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com> Contents: 6a84f9532 Frank S. Filz V4.1 7ba9ef6d2 Daniel Gryniewicz Pullup ntirpc to 4.1 cb9f3afd5 Assaf Yaari FSAL_PROXY_V3: Ensure that the used xid is unique dd13bbbe8 Adrian Freihofer build: use CMAKE_DL_LIBS 275539c02 Malahal Naineni Retry communication with NSM service f2e6765bb Madhu Thorat init_fds_limit(): Log EVENT to print current set FD limit cf7768e66 Malahal Naineni Add -C option to Ganesha daemon startup 6a7880c3d Malahal Naineni Fix ganesha_stats with dbus import

1 0

[S] Change in ...nfs-ganesha[next]: Pullup ntirpc to 4.1
by Daniel Gryniewicz (GerritHub) 18 Nov '22

18 Nov '22

Daniel Gryniewicz has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546350 ) Change subject: Pullup ntirpc to 4.1 ...................................................................... Pullup ntirpc to 4.1 Change-Id: I1c42b1d8b153587664cbc620dc93d9ad8b49b1c2 Signed-off-by: Daniel Gryniewicz <dang(a)redhat.com> --- M src/libntirpc 1 file changed, 11 insertions(+), 1 deletion(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/50/546350/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546350 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I1c42b1d8b153587664cbc620dc93d9ad8b49b1c2 Gerrit-Change-Number: 546350 Gerrit-PatchSet: 1 Gerrit-Owner: Daniel Gryniewicz <dang(a)redhat.com> Gerrit-MessageType: newchange

1 0

[S] Change in ...nfs-ganesha[next]: FSAL_PROXY_V3: Ensure that the used xid is unique
by Assaf Yaari (GerritHub) 17 Nov '22

17 Nov '22

Assaf Yaari has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546260 ) Change subject: FSAL_PROXY_V3: Ensure that the used xid is unique ...................................................................... FSAL_PROXY_V3: Ensure that the used xid is unique If the used XID is the same as other call that is identical, it may cause the DRC to drop the request which cause the FSAL thread and client to halt. Using rand_r is not good enough as mutiple threads calling it at the same time (using same 'seedp') may produce the same number. Signed-off-by: Assaf Yaari <assafy(a)google.com> Change-Id: I725e62f91638ee39aad4c990cffaa4b470f0aa45 --- M src/FSAL/FSAL_PROXY_V3/rpc.c 1 file changed, 27 insertions(+), 10 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/60/546260/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546260 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I725e62f91638ee39aad4c990cffaa4b470f0aa45 Gerrit-Change-Number: 546260 Gerrit-PatchSet: 1 Gerrit-Owner: Assaf Yaari <assafy(a)google.com> Gerrit-MessageType: newchange

1 0

[S] Change in ...nfs-ganesha[next]: DS-3041: check FSAL library path before load it
by Name of user not set (GerritHub) 17 Nov '22

17 Nov '22

skmprabhu2(a)gmail.com has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546250 ) Change subject: DS-3041: check FSAL library path before load it ...................................................................... DS-3041: check FSAL library path before load it Ganesha is exiting if wrong FSAL is mentioned in export config. We should not terminate ganesha for just one wrong export config. Added code to check file existence before loading FSAL lib. Change-Id: I5106b8b153c1df2dd4bf326ec62a241061d2b476 Signed-off-by: Prabhu Murugesan <pmurugesan(a)ddn.com> (cherry picked from commit 5c9e9ae36426e4de66c3557ec6f40bfbffcd1f97) --- M src/FSAL/fsal_manager.c 1 file changed, 28 insertions(+), 1 deletion(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/50/546250/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546250 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I5106b8b153c1df2dd4bf326ec62a241061d2b476 Gerrit-Change-Number: 546250 Gerrit-PatchSet: 1 Gerrit-Owner: skmprabhu2(a)gmail.com Gerrit-MessageType: newchange

1 0

Performance scalability issues - linear drops in IOPs when adding clients
by jakub.jurcaga＠datera.cz 16 Nov '22

16 Nov '22

Hello, We are currently testing an all-flash storage with FIO. Our infrastructure is as follows: A two node all-flash storage which is connected to two GPFS servers (version 5.1.4-1) via point-to-point 16Gb FC connection (4 paths per server). Three ESXi hosts (v 7.0 U3), each running four NFS clients (NFS v4.0), which are connected to the GPFS servers using 100GbE connection via a high-end Ethernet switch with support of jumbo frames. The GPFS servers are using 2x 100 GbE LACP to the Ethernet switch, ESX servers are using 1x 100GbE connection. Connection throughput was tested with iperf and works well. The GPFS servers are exporting the following share: /gpfs/gpfs22/nfs1 *(rw,no_root_squash,async,fsid=402) The NFS clients are mounting the share with the following options: 192.168.5.202:/gpfs/gpfs22/nfs1 on /mnt/nfs1 type nfs4 (rw,noatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,nconnect=8,timeo=600,retrans=2,sec=sys,clientaddr=192.168.5.54,local_lock=none,addr=192.168.5.202) We are experiencing issues with scaling - performance does not scale with added clients We always hit a ceiling of around 165k IOPs regardless of the number of NFS clients. It seems that the performance is directly impacted by the number of NFS servers. We've run tests utilizing 12, 6, 4 and 2 clients respectively. When running the tests with only two clients, we are reaching 82,5k (8k, 70/30 r/w) IOPs per client. However, when running the tests with 4 clients, we are only reaching roughly a half. The performance drops linearly with added clients. We've looked around the archives and found a topic which deals with scalability issues. The following statement seems close to what we are experiencing. > If I'm understanding this right, it appears to be using a single thread per interface. That would explain the observation that a single thread is blocked in writev() whenever I look. It also explains why when clients are bandwidth-limited, the aggregate throughput does not scale much above that consumed by a single client. Could it be that we are running into a similar issue? Is there a tunable that could help us with scaling? The thread is quite old. We are running a newer version of NFS-Ganesha: # ganesha_mgr show version NFS-Ganesha Release = V3.5-ibm071.16 ganesha compiled on Jul 11 2022 at 15:28:41 Release comment = GANESHA file server is 64 bits compliant and supports NFS v3,4.0,4.1 (pNFS) and 9P Git HEAD = NOT-GIT Git Describe = /home/ppsbld/workspace/SS_5.1.4/Build_Prod/Ganesha_5.1.4_ProdBuild/label/BLDLNX125_X86_64_REDHAT_8.0.0/ganesha-rpmdir/rpmbuild/BUILD/nfs-ganesha-3.5-ibm071.16 When tested locally, the performance of the storage was around 400k IOPs. We are at our wits end and would really appreciate any insight. Thank you. Regards, Jakub

1 0

[M] Change in ...nfs-ganesha[next]: MDCACHE: Don't need DQ_SAFE any more
by Frank Filz (GerritHub) 12 Nov '22

12 Nov '22

Frank Filz has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546035 ) Change subject: MDCACHE: Don't need DQ_SAFE any more ...................................................................... MDCACHE: Don't need DQ_SAFE any more The qlane lock is no longer dropped during iterating through the queue so no need for the gyrations of DQ_SAFE. CHUNK_DQ_SAFE was never needed since it never dropped the qlane lock during the iterator. Change-Id: I5e92f637e64dbd4ae113fe24a0efe03dbbd5b36d Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com> --- M src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_lru.c 1 file changed, 41 insertions(+), 66 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/35/546035/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546035 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I5e92f637e64dbd4ae113fe24a0efe03dbbd5b36d Gerrit-Change-Number: 546035 Gerrit-PatchSet: 1 Gerrit-Owner: Frank Filz <ffilzlnx(a)mindspring.com> Gerrit-MessageType: newchange

1 0

[M] Change in ...nfs-ganesha[next]: Retry communication with NSM service
by Name of user not set (GerritHub) 11 Nov '22

11 Nov '22

Attention is currently required from: Malahal. Hello Malahal, I'd like you to do a code review. Please visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546024 to review the following change. Change subject: Retry communication with NSM service ...................................................................... Retry communication with NSM service We have seen incidents where Ganesha couldn't communicate to rpc.statd. It uses the same socket over and over resulting into the same TIMEOUT error every time. Destroy the nsm client on a failure and recreate it to avoid this persistent error. This also helps when rpc.statd service gets restarted for any reason. Change-Id: Ia9ef2b5a8c82b79efe2f57b755d0e25adb46b26a Signed-off-by: Malahal Naineni <malahal(a)us.ibm.com> (cherry picked from commit 7b1d58899bc13c1f02b6ea1e4849a37e884d212c) (cherry picked from commit a77f95b7ac21862cffad6e8cbb979562c73baa57) --- M src/Protocols/NLM/nsm.c 1 file changed, 55 insertions(+), 9 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/24/546024/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546024 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: Ia9ef2b5a8c82b79efe2f57b755d0e25adb46b26a Gerrit-Change-Number: 546024 Gerrit-PatchSet: 1 Gerrit-Owner: skmprabhu2(a)gmail.com Gerrit-Reviewer: Malahal <malahal(a)gmail.com> Gerrit-Attention: Malahal <malahal(a)gmail.com> Gerrit-MessageType: newchange

1 0

[S] Change in ...nfs-ganesha[next]: init_fds_limit(): Log EVENT to print current set FD limit
by Name of user not set (GerritHub) 11 Nov '22

11 Nov '22

Attention is currently required from: Madhu Thorat. Hello Madhu Thorat, I'd like you to do a code review. Please visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546015 to review the following change. Change subject: init_fds_limit(): Log EVENT to print current set FD limit ...................................................................... init_fds_limit(): Log EVENT to print current set FD limit In init_fds_limit() Log EVENT to print the set system-imposed FD limit. Change-Id: I73839073c5c7f73c2f18a93ebc901647719261cc Signed-off-by: Madhu Thorat <madhu.punjabi(a)in.ibm.com> (cherry picked from commit e8f4fd2f64c4105021f7cc79d4c760826cc251f0) (cherry picked from commit 262e35028648f57cdd0c0e613d024f96e1018bb5) --- M src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_lru.c 1 file changed, 19 insertions(+), 3 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/15/546015/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546015 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I73839073c5c7f73c2f18a93ebc901647719261cc Gerrit-Change-Number: 546015 Gerrit-PatchSet: 1 Gerrit-Owner: skmprabhu2(a)gmail.com Gerrit-Reviewer: Madhu Thorat <madhu.punjabi(a)in.ibm.com> Gerrit-Attention: Madhu Thorat <madhu.punjabi(a)in.ibm.com> Gerrit-MessageType: newchange

1 0

[S] Change in ...nfs-ganesha[next]: Add -C option to Ganesha daemon startup
by Name of user not set (GerritHub) 11 Nov '22

11 Nov '22

Attention is currently required from: Malahal. Hello Malahal, I'd like you to do a code review. Please visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546005 to review the following change. Change subject: Add -C option to Ganesha daemon startup ...................................................................... Add -C option to Ganesha daemon startup Option -C would make Ganesha daemon dump stack backtrace on crash. Change-Id: I1790ffcd67219bd4a46da073d43651109822c886 Signed-off-by: Malahal Naineni <malahal(a)us.ibm.com> (cherry picked from commit 8c137d46284840f952027b837258e27ff29b342d) --- M src/scripts/systemd/nfs-ganesha.service.debian8 M src/scripts/systemd/nfs-ganesha.service.el7 2 files changed, 15 insertions(+), 2 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/05/546005/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/546005 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I1790ffcd67219bd4a46da073d43651109822c886 Gerrit-Change-Number: 546005 Gerrit-PatchSet: 1 Gerrit-Owner: skmprabhu2(a)gmail.com Gerrit-Reviewer: Malahal <malahal(a)gmail.com> Gerrit-Attention: Malahal <malahal(a)gmail.com> Gerrit-MessageType: newchange

1 0