August 2019 - Devel - NFS Ganesha List Archives

Change in ...nfs-ganesha[next]: python: run setup.py for gpfs-epoch(.py) to fix its shebang
by Kaleb KEITHLEY (GerritHub) 20 Aug '19

20 Aug '19

Kaleb KEITHLEY has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465799 ) Change subject: python: run setup.py for gpfs-epoch(.py) to fix its shebang ...................................................................... python: run setup.py for gpfs-epoch(.py) to fix its shebang `setup.py build ...` modifies the shebang appropriately for the platform, i.e. using the python (e.g. /usr/bin/python3) that setup.py was invoked with. This might seem like a heavyweight solution for one file, but it's ostensibly better than some other solution like sed editing the shebang in an rpmbuild scriptlet. This way you get (almost) the right thing in a regular cmake/make/make-install build. Almost, because the `setup.py install ...` step installs gpfs-epoch in /usr/bin instead of where we infer it's wanted, in /usr/libexec, based on the install scriptlets in the rpm .spec file. I contemplated changing the --prefix for the `setup.py install ...` step but then that scribbles other things in /usr/libexec or /usr/local/libexec I also contemplated putting gpfs-epoch(.py) in its own subdir, e.g., .../src/scripts/gpfs-epoch/... or .../src/FSAL/FSAL_GPFS/gpfs-epoch/... and could be convinced to do that. Signed-off-by: Kaleb S. KEITHLEY <kkeithle(a)redhat.com> Change-Id: I5ff746393b7bac53c9f12c9d33f0b0aeb24cd5ce --- M src/nfs-ganesha.spec-in.cmake M src/scripts/CMakeLists.txt M src/scripts/ganeshactl/CMakeLists.txt R src/scripts/gpfs-epoch.py A src/scripts/setup.py.in 5 files changed, 80 insertions(+), 3 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/99/465799/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465799 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I5ff746393b7bac53c9f12c9d33f0b0aeb24cd5ce Gerrit-Change-Number: 465799 Gerrit-PatchSet: 1 Gerrit-Owner: Kaleb KEITHLEY <kaleb(a)redhat.com> Gerrit-MessageType: newchange

1 0

Change in ...nfs-ganesha[next]: Fix max open files limit for ganesha daemon
by Malahal (GerritHub) 19 Aug '19

19 Aug '19

Malahal has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465631 ) Change subject: Fix max open files limit for ganesha daemon ...................................................................... Fix max open files limit for ganesha daemon The current method involves setting proper resource limit using prlimit command after starting the daemon, and then calling a dbus method to recalculate the open fd limit. systemd environment file and environment variables are used only in Exec* lines but not setting limits like this. The best way to teach systemd to start the daemon with a correct resource limit is having a drop-in configuration file, a bit tedious but works! Change-Id: I1c8332ca5f1e49073ac9fb9d74ce310bf652ff68 Signed-off-by: Malahal Naineni <malahal(a)us.ibm.com> --- M src/scripts/nfs-ganesha-config.sh M src/scripts/systemd/nfs-ganesha.service.debian8 M src/scripts/systemd/nfs-ganesha.service.el7 3 files changed, 7 insertions(+), 6 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/31/465631/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465631 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I1c8332ca5f1e49073ac9fb9d74ce310bf652ff68 Gerrit-Change-Number: 465631 Gerrit-PatchSet: 1 Gerrit-Owner: Malahal <malahal(a)gmail.com> Gerrit-MessageType: newchange

1 0

improperly booting aarch64 node (ganesha nfs client) x86 OK
by Erik Jacobson 17 Aug '19

17 Aug '19

First, just a quick thanks for all you do. I'm new to Ganesha. I am using Gluster and Gluster NFS as an NFS-boot solution. I've done a lot of testing with 2,500 aarch64 nodes spread across 9 x86_64 NFS servers for booting (we get abut 10 minute boot times with NFS roots on Gluster with Gluster NFS). We're using CTDB for a level of "HA" and it works great. We've lost nodes and haven't even noticed until we checked alerting/logs. We didn't start with Ganesha because I was on a deadline and it was crashing on us under heavy boot load (daemons would disappear over time). I didn't have time to do proper debugging and reporting and the customer had to move on too at that time. It wouldn't have been fair to you guys to post at that time. Since then, I've learned a lot more about Gluster and system tuning, etc. Our customer likes the boot times but their job launch times are taking too long. Compared to kernel NFS (and full TMPFS no-network-FS) roots, Gluster NFS is taking 2 more minutes to launch. They have a large LD_LIBRARY_PATH and tons of libraries being loaded by all nodes at the same time. Some gluster profiling is showing caching seems to be working, we are guessing the issue is metadata load and related latency. Because Gluster NFS is being depreciated anyway and because we want to improve the job launch time, I have come back to trying Ganesha. I noticed it has 'MDCACHE' and we wanted to try that out. Maybe we can get our 2 minutes back. The NFS solution is unique in that it combines a read-only NFS root with TMPFS using overlay FS on each node. This gives the client nodes a true "writable" feel even though they are based on read-only NFS. Kernel NFS exports from standard ext4, and Gluster NFS exports are working ok in this configuration for both architectures. Ganesha is working OK for x86_64 nodes - normal bootup, no problem. However, Ganesha for aarch64 client nodes is causing us trouble. A typical RHEL7.6 bootup has many errors. Among services that never start is "sshd" and "login". Sort of a bummer! :) [FAILED] Failed to start Login Service. over and over. At this time, I'm using our systems in the lab - a tiny version of the big one - to debug the above. If I can solve the above, I will be able to test on the ARM supercomputer a week from today. It will be my last chance to run on the system personally. My goal of dedicated time next week, if I can resolve the above, is to improve job launch times. If there are stability problems, I want to know the right method to report them to you guys before we revert to Gluster NFS if needed. To get to that point, I need to solve the above issue. Gluster info: gluster-4.1.6 * My own build of it Volume type: 3x3 (3 replicas for 3 total subvolumes, 9 total x86_64 servers) Gluster server OS: RHEL 7.6 Server architecture: x86_64 Ganesha info: Ganesha 2.8.2 * I did a personal build of this for testing, with gluster 4.1.6 Ganesha clients running on the Gluster Servers themselves. NFS Client info: RHEL 7.6 aarch64 (fails) RHEL 7.6 x86_64 (works) NFS Client environment: - UEFI pxe to retrieve grub2 grub2 loads kernel and initrd with tftp initrd loads a special "miniroot" with a vendor script miniroot is an "expanded initrd" that has our toolchain * I can enable a rescue mode where I shell out just before SWITCHROOT if that helps NFS / Overlay mount commands done by client: mount -n -o ro,noatime,nocto,actimeo=3600,lookupcache=all,nolock,tcp,vers=3 172.23.255.249:/cm_shared/image/images_ro_nfs/rhel7.6 mount -t tmpfs -o mpol=interleave tmpfs /rootfs.rw mount -t overlay overlay -o lowerdir=/root_ro_nfs,upperdir=/rootfs.rw/upperdir,workdir=/rootfs.rw/work /a (then later the magic to make the above work with SWITCHROOT). - I don't think it's a squash issue - from the miniroot in rescue mode, I can poke around the RO NFS + TMPFS overlay and see the contents of root-only readable files. - If I chroot from rescue mode in to the root, sshd stats by hand, so I think the issue is related to systemd. - Not being able to log in after switchroot makes it harder but I'll start researching a way to get in, perhaps starting systemd by hand. Any advice appreciated. Sorry this got so long. PS: Ganesha configuration: Disable_ACL = TRUE; # To enable/disable ACL NFS_CORE_PARAM { Clustered = TRUE; } EXPORT { # Export Id (mandatory, each EXPORT must have a unique Export_Id) Export_Id = 10; # Exported path (mandatory) Path = "/cm_shared"; # assuming 'testvol' is the Gluster volume name # Pseudo Path (required for NFS v4) Pseudo = "/cm_shared"; # Required for access (default is None) # Could use CLIENT blocks instead Access_Type = RW; # Allow root access Squash = none; #Squash = No_Root_Squash; # Security flavor supported SecType = "sys"; # Exporting FSAL FSAL { Name = "GLUSTER"; Hostname = "127.0.0.1"; # IP of one of the nodes in the trusted pool Volume = "cm_shared"; } }

4 43

Announce Push of V2.9-dev.5
by Frank Filz 16 Aug '19

16 Aug '19

Branch next Tag:V2.9-dev.5 Release Highlights * A number of Coverity fixes * Various python script fixes * FSAL_RGW - Track openflags per state * MDCACHE: save all dirents from last READDIR in dir-map * mdcache_get_parent: Don't do lookup of parent for a root object * gtest: portability to gcc-4.8.5 in RHEL/CentOS 7 * Drop loglevel for principal to uid/gid failures Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com> Contents: 9fcdf27 Frank S. Filz V2.9-dev.5 6882b29 Shangzhong Zhu Coverity: use dirent after free in mdcache_avl_remove f46c0a3 Shangzhong Zhu Coverity: check if data->session is NULL after deref a26ab86 Shangzhong Zhu Coverity: dead code in fsal_create 7380359 Shangzhong Zhu Coverity: invalid type to printf format specifier e1d3f39 Shangzhong Zhu Test function "nfs4_acls_test" should be removed f26bbed Shangzhong Zhu Copy/paste error in state_nlm_share 4bf71dd Shangzhong Zhu Coverity: dereference NULL pointer in dbus_proc_property 24c33ea Shangzhong Zhu Coverity: missing unlock in ceph_fsal_lock_op2 94117be Shangzhong Zhu Coverity: overrun to index array 73df505 Shangzhong Zhu Coverity: resource leak in rados_url_setup_watch bd1ab5b sepia-liu op_ctx->ctx_export->fsal_export maybe dereference error 1c8fae5 sepia-liu export is not NULL 7153993 sepia-liu exp dereference error if get_gsh_export() return NULL 3ca23ea sepia-liu res maybe double_free, when the first time res has released of the loop,the second getaddrinfo() failed and res not NULL. b4b95e7 Trishali Nayar Kerberos Keytab file path should be configurable 4fc6c2c Malahal Naineni Drop loglevel for principal to uid/gid failures 1974118 Madhu Thorat mdcache_get_parent: Don't do lookup of parent for a root object a28a854 Kaleb S. KEITHLEY gpfs: python2 -> python3, .spec file fix 4b08c21 Kaleb S. KEITHLEY ganeshactl: more fixes for selected pylint warnings and errors 9b5a362 Kaleb S. KEITHLEY ganeshactl: fix selected pylint warnings and errors 58b1639 Kaleb S. KEITHLEY gtest: portability to gcc-4.8.5 in RHEL/CentOS 7 c1c70e8 Daniel Gryniewicz MDCACHE: save all dirents from last READDIR in dir-map 6a2a2a7 Daniel Gryniewicz FSAL_RGW - Track openflags per state

1 0

Change in ...nfs-ganesha[next]: MDCACHE: save all dirents from last READDIR in dir-map
by Matt Benjamin (GerritHub) 14 Aug '19

14 Aug '19

Hello Daniel Gryniewicz, I'd like you to do a code review. Please visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465197 to review the following change. Change subject: MDCACHE: save all dirents from last READDIR in dir-map ...................................................................... MDCACHE: save all dirents from last READDIR in dir-map Change-Id: Icc5cf96c2f3380bc640b5ccc0e9fc455af2ba794 Signed-off-by: Daniel Gryniewicz <dang(a)redhat.com> --- M src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_helpers.c M src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_lru.c 2 files changed, 9 insertions(+), 11 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/97/465197/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465197 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: Icc5cf96c2f3380bc640b5ccc0e9fc455af2ba794 Gerrit-Change-Number: 465197 Gerrit-PatchSet: 1 Gerrit-Owner: Matt Benjamin <mbenjamin(a)redhat.com> Gerrit-Reviewer: Daniel Gryniewicz <dang(a)redhat.com> Gerrit-MessageType: newchange

1 0

Change in ...nfs-ganesha[next]: gpfs: python2 -> python3, .spec file fix
by Kaleb KEITHLEY (GerritHub) 14 Aug '19

14 Aug '19

Kaleb KEITHLEY has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465157 ) Change subject: gpfs: python2 -> python3, .spec file fix ...................................................................... gpfs: python2 -> python3, .spec file fix missed a py2->py3 switch in .../scripts/gpfs-epoch nfs-ganesha.spec file uses IBM package name for Requires: (Although I wonder if the -utils RPM should really depend on -gpfs. Aren't the bulk of the utils usable independent of whether nfs-ganesha-gpfs is installed?) Signed-off-by: Kaleb S. KEITHLEY <kkeithle(a)redhat.com> Change-Id: Idbe7f36a200b408c776f2c20ff179e9169a0f123 --- M src/nfs-ganesha.spec-in.cmake M src/scripts/gpfs-epoch 2 files changed, 3 insertions(+), 3 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/57/465157/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465157 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: Idbe7f36a200b408c776f2c20ff179e9169a0f123 Gerrit-Change-Number: 465157 Gerrit-PatchSet: 1 Gerrit-Owner: Kaleb KEITHLEY <kaleb(a)redhat.com> Gerrit-MessageType: newchange

1 0

Change in ...nfs-ganesha[next]: Protocol: add config option to skip utf8 string scan
by Name of user not set (GerritHub) 14 Aug '19

14 Aug '19

xujiatong(a)cmss.chinamobile.com has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465149 ) Change subject: Protocol: add config option to skip utf8 string scan ...................................................................... Protocol: add config option to skip utf8 string scan Client got "invalid arg"return when filename appeared garbled with NFSv4 mount.Besides,NFSv3 & any protocols exporting NFSD could do ops exactly. Provide config option to keep the consistency of NFSv3/v4. Signed-off-by: Jerome.Xu <xujiatong(a)cmss.chinamobile.com> Change-Id: Iadf4f83c377ada0fb2d50e855827d8968dde4093 --- M src/Protocols/NFS/nfs_proto_tools.c M src/include/gsh_config.h M src/support/nfs_read_conf.c 3 files changed, 9 insertions(+), 0 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/49/465149/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465149 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: Iadf4f83c377ada0fb2d50e855827d8968dde4093 Gerrit-Change-Number: 465149 Gerrit-PatchSet: 1 Gerrit-Owner: xujiatong(a)cmss.chinamobile.com Gerrit-MessageType: newchange

1 0

Change in ...nfs-ganesha[next]: mdcache_create_handle: Don't lookup for parent for a root object
by Madhu Thorat (GerritHub) 14 Aug '19

14 Aug '19

Madhu Thorat has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465126 ) Change subject: mdcache_create_handle: Don't lookup for parent for a root object ...................................................................... mdcache_create_handle: Don't lookup for parent for a root object Currently in mdcache_create_handle() after getting an 'entry' we call mdc_get_parent() for the 'entry'. But if the entry->obj_handle points to root object then we would be doing lookup of parent for the root, which we should not be doing. To avoid this prior to call mdc_get_parent() check if the entry->obj_handle points to a root object, if yes then don't call mdc_get_parent() for the 'entry'. Change-Id: Ibf25fd283573c9fa28ae0452d96f00f294f66828 Signed-off-by: Madhu Thorat <madhu.punjabi(a)in.ibm.com> --- M src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_handle.c 1 file changed, 21 insertions(+), 2 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/26/465126/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465126 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: Ibf25fd283573c9fa28ae0452d96f00f294f66828 Gerrit-Change-Number: 465126 Gerrit-PatchSet: 1 Gerrit-Owner: Madhu Thorat <madhu.punjabi(a)in.ibm.com> Gerrit-MessageType: newchange

1 0

Change in ...nfs-ganesha[next]: fix wrong package name and dependency
by Ji-Hyeon Gim (GerritHub) 13 Aug '19

13 Aug '19

Ji-Hyeon Gim has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465123 ) Change subject: fix wrong package name and dependency ...................................................................... fix wrong package name and dependency Problem In RPM spec file, the package name for GPFS is gpfs.nfs-ganesha but that it is a name for IBM only. And also, nfs-ganesha-utils has a dependency with gpfs flag but this requirement should not be indispensable. Signed-off-by: Ji-Hyeon Gim <potatogim(a)gluesys.com> Change-Id: I67a2b76a1b41a62c1d280d1461d7baf5d013a3a5 --- M src/nfs-ganesha.spec-in.cmake 1 file changed, 12 insertions(+), 6 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/23/465123/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465123 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I67a2b76a1b41a62c1d280d1461d7baf5d013a3a5 Gerrit-Change-Number: 465123 Gerrit-PatchSet: 1 Gerrit-Owner: Ji-Hyeon Gim <potatogim(a)potatogim.net> Gerrit-MessageType: newchange

1 0

Change in ...nfs-ganesha[next]: gtest: portability to gcc-4.8.5 in RHEL/CentOS 7
by Kaleb KEITHLEY (GerritHub) 13 Aug '19

13 Aug '19

Kaleb KEITHLEY has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465099 ) Change subject: gtest: portability to gcc-4.8.5 in RHEL/CentOS 7 ...................................................................... gtest: portability to gcc-4.8.5 in RHEL/CentOS 7 gcc-4.8.5 has no namespace std::literals lacking literals, use std::chrono::seconds() std=gnu-g++11 seems to adequate to get typeof, esp. as gcc-4.8.5 doesn't know gnu-g++14 All to be able to build gtest in the CentOS CI Signed-off-by: Kaleb S. KEITHLEY <kkeithle(a)redhat.com> Change-Id: I8611a8a1219524fafc0c2fa9f0521b22bb81a59f --- M src/gtest/CMakeLists.txt M src/gtest/fsal_api/test_close2_latency.cc M src/gtest/fsal_api/test_close_latency.cc M src/gtest/fsal_api/test_commit2_latency.cc M src/gtest/fsal_api/test_getattrs_latency.cc M src/gtest/fsal_api/test_handle_to_key_latency.cc M src/gtest/fsal_api/test_handle_to_wire_latency.cc M src/gtest/fsal_api/test_link_latency.cc M src/gtest/fsal_api/test_lock_op2_latency.cc M src/gtest/fsal_api/test_lookup_latency.cc M src/gtest/fsal_api/test_mkdir_latency.cc M src/gtest/fsal_api/test_mknode_latency.cc M src/gtest/fsal_api/test_open2_latency.cc M src/gtest/fsal_api/test_read2_latency.cc M src/gtest/fsal_api/test_readdir_correctness.cc M src/gtest/fsal_api/test_readdir_latency.cc M src/gtest/fsal_api/test_readlink_latency.cc M src/gtest/fsal_api/test_release_latency.cc M src/gtest/fsal_api/test_rename_latency.cc M src/gtest/fsal_api/test_reopen2_latency.cc M src/gtest/fsal_api/test_setattr2_latency.cc M src/gtest/fsal_api/test_symlink_latency.cc M src/gtest/fsal_api/test_unlink_latency.cc M src/gtest/fsal_api/test_write2_latency.cc M src/gtest/gtest.hh M src/gtest/nfs4/test_nfs4_link_latency.cc M src/gtest/nfs4/test_nfs4_lookup_latency.cc M src/gtest/nfs4/test_nfs4_putfh_latency.cc M src/gtest/nfs4/test_nfs4_rename_latency.cc M src/gtest/test_ci_hash_dist1.cc 30 files changed, 3 insertions(+), 32 deletions(-) git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/99/465099/1 -- To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465099 To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings Gerrit-Project: ffilz/nfs-ganesha Gerrit-Branch: next Gerrit-Change-Id: I8611a8a1219524fafc0c2fa9f0521b22bb81a59f Gerrit-Change-Number: 465099 Gerrit-PatchSet: 1 Gerrit-Owner: Kaleb KEITHLEY <kaleb(a)redhat.com> Gerrit-MessageType: newchange

1 0