Change in ...nfs-ganesha[next]: python: run setup.py for gpfs-epoch(.py) to fix its shebang
by Kaleb KEITHLEY (GerritHub)
Kaleb KEITHLEY has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465799 )
Change subject: python: run setup.py for gpfs-epoch(.py) to fix its shebang
......................................................................
python: run setup.py for gpfs-epoch(.py) to fix its shebang
`setup.py build ...` modifies the shebang appropriately for the
platform, i.e. using the python (e.g. /usr/bin/python3) that setup.py
was invoked with.
This might seem like a heavyweight solution for one file, but it's
ostensibly better than some other solution like sed editing the shebang
in an rpmbuild scriptlet. This way you get (almost) the right thing in
a regular cmake/make/make-install build. Almost, because the `setup.py
install ...` step installs gpfs-epoch in /usr/bin instead of where we
infer it's wanted, in /usr/libexec, based on the install scriptlets in
the rpm .spec file.
I contemplated changing the --prefix for the `setup.py install ...` step
but then that scribbles other things in /usr/libexec or /usr/local/libexec
I also contemplated putting gpfs-epoch(.py) in its own subdir, e.g.,
.../src/scripts/gpfs-epoch/... or .../src/FSAL/FSAL_GPFS/gpfs-epoch/...
and could be convinced to do that.
Signed-off-by: Kaleb S. KEITHLEY <kkeithle(a)redhat.com>
Change-Id: I5ff746393b7bac53c9f12c9d33f0b0aeb24cd5ce
---
M src/nfs-ganesha.spec-in.cmake
M src/scripts/CMakeLists.txt
M src/scripts/ganeshactl/CMakeLists.txt
R src/scripts/gpfs-epoch.py
A src/scripts/setup.py.in
5 files changed, 80 insertions(+), 3 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/99/465799/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465799
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: I5ff746393b7bac53c9f12c9d33f0b0aeb24cd5ce
Gerrit-Change-Number: 465799
Gerrit-PatchSet: 1
Gerrit-Owner: Kaleb KEITHLEY <kaleb(a)redhat.com>
Gerrit-MessageType: newchange
4 years, 8 months
Change in ...nfs-ganesha[next]: Fix max open files limit for ganesha daemon
by Malahal (GerritHub)
Malahal has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465631 )
Change subject: Fix max open files limit for ganesha daemon
......................................................................
Fix max open files limit for ganesha daemon
The current method involves setting proper resource limit using prlimit
command after starting the daemon, and then calling a dbus method to
recalculate the open fd limit.
systemd environment file and environment variables are used only in
Exec* lines but not setting limits like this. The best way to teach
systemd to start the daemon with a correct resource limit is having a
drop-in configuration file, a bit tedious but works!
Change-Id: I1c8332ca5f1e49073ac9fb9d74ce310bf652ff68
Signed-off-by: Malahal Naineni <malahal(a)us.ibm.com>
---
M src/scripts/nfs-ganesha-config.sh
M src/scripts/systemd/nfs-ganesha.service.debian8
M src/scripts/systemd/nfs-ganesha.service.el7
3 files changed, 7 insertions(+), 6 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/31/465631/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465631
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: I1c8332ca5f1e49073ac9fb9d74ce310bf652ff68
Gerrit-Change-Number: 465631
Gerrit-PatchSet: 1
Gerrit-Owner: Malahal <malahal(a)gmail.com>
Gerrit-MessageType: newchange
4 years, 8 months
improperly booting aarch64 node (ganesha nfs client) x86 OK
by Erik Jacobson
First, just a quick thanks for all you do. I'm new to Ganesha.
I am using Gluster and Gluster NFS as an NFS-boot solution. I've done a
lot of testing with 2,500 aarch64 nodes spread across 9 x86_64 NFS servers for
booting (we get abut 10 minute boot times with NFS roots on Gluster with
Gluster NFS). We're using CTDB for a level of "HA" and it works great.
We've lost nodes and haven't even noticed until we checked
alerting/logs.
We didn't start with Ganesha because I was on a deadline and it was
crashing on us under heavy boot load (daemons would disappear over
time). I didn't have time to do proper debugging and reporting and the
customer had to move on too at that time. It wouldn't have been fair to
you guys to post at that time.
Since then, I've learned a lot more about Gluster and system tuning,
etc.
Our customer likes the boot times but their job launch times are taking
too long. Compared to kernel NFS (and full TMPFS no-network-FS) roots,
Gluster NFS is taking 2 more minutes to launch. They have a large
LD_LIBRARY_PATH and tons of libraries being loaded by all nodes at the
same time. Some gluster profiling is showing caching seems to be
working, we are guessing the issue is metadata load and related latency.
Because Gluster NFS is being depreciated anyway and because we want to
improve the job launch time, I have come back to trying Ganesha. I
noticed it has 'MDCACHE' and we wanted to try that out. Maybe we can
get our 2 minutes back.
The NFS solution is unique in that it combines a read-only NFS root with
TMPFS using overlay FS on each node. This gives the client nodes a true
"writable" feel even though they are based on read-only NFS.
Kernel NFS exports from standard ext4, and Gluster NFS exports are
working ok in this configuration for both architectures.
Ganesha is working OK for x86_64 nodes - normal bootup, no problem.
However, Ganesha for aarch64 client nodes is causing us trouble. A typical
RHEL7.6 bootup has many errors. Among services that never start is
"sshd" and "login". Sort of a bummer! :)
[FAILED] Failed to start Login Service.
over and over.
At this time, I'm using our systems in the lab - a tiny version of the
big one - to debug the above. If I can solve the above, I will be able
to test on the ARM supercomputer a week from today. It will be my last
chance to run on the system personally.
My goal of dedicated time next week, if I can resolve the above, is to
improve job launch times. If there are stability problems, I want to
know the right method to report them to you guys before we revert to
Gluster NFS if needed. To get to that point, I need to solve the above
issue.
Gluster info:
gluster-4.1.6
* My own build of it
Volume type: 3x3 (3 replicas for 3 total subvolumes, 9 total x86_64 servers)
Gluster server OS: RHEL 7.6
Server architecture: x86_64
Ganesha info:
Ganesha 2.8.2
* I did a personal build of this for testing, with gluster 4.1.6
Ganesha clients running on the Gluster Servers themselves.
NFS Client info:
RHEL 7.6 aarch64 (fails)
RHEL 7.6 x86_64 (works)
NFS Client environment:
- UEFI pxe to retrieve grub2
grub2 loads kernel and initrd with tftp
initrd loads a special "miniroot" with a vendor script
miniroot is an "expanded initrd" that has our toolchain
* I can enable a rescue mode where I shell out just before
SWITCHROOT if that helps
NFS / Overlay mount commands done by client:
mount -n -o ro,noatime,nocto,actimeo=3600,lookupcache=all,nolock,tcp,vers=3 172.23.255.249:/cm_shared/image/images_ro_nfs/rhel7.6
mount -t tmpfs -o mpol=interleave tmpfs /rootfs.rw
mount -t overlay overlay -o lowerdir=/root_ro_nfs,upperdir=/rootfs.rw/upperdir,workdir=/rootfs.rw/work /a
(then later the magic to make the above work with SWITCHROOT).
- I don't think it's a squash issue - from the miniroot in rescue mode,
I can poke around the RO NFS + TMPFS overlay and see the contents of
root-only readable files.
- If I chroot from rescue mode in to the root, sshd stats by hand, so I think
the issue is related to systemd.
- Not being able to log in after switchroot makes it harder but I'll
start researching a way to get in, perhaps starting systemd by hand.
Any advice appreciated. Sorry this got so long.
PS: Ganesha configuration:
Disable_ACL = TRUE; # To enable/disable ACL
NFS_CORE_PARAM {
Clustered = TRUE;
}
EXPORT
{
# Export Id (mandatory, each EXPORT must have a unique Export_Id)
Export_Id = 10;
# Exported path (mandatory)
Path = "/cm_shared"; # assuming 'testvol' is the Gluster volume name
# Pseudo Path (required for NFS v4)
Pseudo = "/cm_shared";
# Required for access (default is None)
# Could use CLIENT blocks instead
Access_Type = RW;
# Allow root access
Squash = none;
#Squash = No_Root_Squash;
# Security flavor supported
SecType = "sys";
# Exporting FSAL
FSAL {
Name = "GLUSTER";
Hostname = "127.0.0.1"; # IP of one of the nodes in the trusted pool
Volume = "cm_shared";
}
}
4 years, 8 months
Announce Push of V2.9-dev.5
by Frank Filz
Branch next
Tag:V2.9-dev.5
Release Highlights
* A number of Coverity fixes
* Various python script fixes
* FSAL_RGW - Track openflags per state
* MDCACHE: save all dirents from last READDIR in dir-map
* mdcache_get_parent: Don't do lookup of parent for a root object
* gtest: portability to gcc-4.8.5 in RHEL/CentOS 7
* Drop loglevel for principal to uid/gid failures
Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com>
Contents:
9fcdf27 Frank S. Filz V2.9-dev.5
6882b29 Shangzhong Zhu Coverity: use dirent after free in mdcache_avl_remove
f46c0a3 Shangzhong Zhu Coverity: check if data->session is NULL after deref
a26ab86 Shangzhong Zhu Coverity: dead code in fsal_create
7380359 Shangzhong Zhu Coverity: invalid type to printf format specifier
e1d3f39 Shangzhong Zhu Test function "nfs4_acls_test" should be removed
f26bbed Shangzhong Zhu Copy/paste error in state_nlm_share
4bf71dd Shangzhong Zhu Coverity: dereference NULL pointer in
dbus_proc_property
24c33ea Shangzhong Zhu Coverity: missing unlock in ceph_fsal_lock_op2
94117be Shangzhong Zhu Coverity: overrun to index array
73df505 Shangzhong Zhu Coverity: resource leak in rados_url_setup_watch
bd1ab5b sepia-liu op_ctx->ctx_export->fsal_export maybe dereference error
1c8fae5 sepia-liu export is not NULL
7153993 sepia-liu exp dereference error if get_gsh_export() return NULL
3ca23ea sepia-liu res maybe double_free, when the first time res has
released of the loop,the second getaddrinfo() failed and res not NULL.
b4b95e7 Trishali Nayar Kerberos Keytab file path should be configurable
4fc6c2c Malahal Naineni Drop loglevel for principal to uid/gid failures
1974118 Madhu Thorat mdcache_get_parent: Don't do lookup of parent for a
root object
a28a854 Kaleb S. KEITHLEY gpfs: python2 -> python3, .spec file fix
4b08c21 Kaleb S. KEITHLEY ganeshactl: more fixes for selected pylint
warnings and errors
9b5a362 Kaleb S. KEITHLEY ganeshactl: fix selected pylint warnings and
errors
58b1639 Kaleb S. KEITHLEY gtest: portability to gcc-4.8.5 in RHEL/CentOS 7
c1c70e8 Daniel Gryniewicz MDCACHE: save all dirents from last READDIR in
dir-map
6a2a2a7 Daniel Gryniewicz FSAL_RGW - Track openflags per state
4 years, 8 months
Change in ...nfs-ganesha[next]: MDCACHE: save all dirents from last READDIR in dir-map
by Matt Benjamin (GerritHub)
Hello Daniel Gryniewicz,
I'd like you to do a code review. Please visit
https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465197
to review the following change.
Change subject: MDCACHE: save all dirents from last READDIR in dir-map
......................................................................
MDCACHE: save all dirents from last READDIR in dir-map
Change-Id: Icc5cf96c2f3380bc640b5ccc0e9fc455af2ba794
Signed-off-by: Daniel Gryniewicz <dang(a)redhat.com>
---
M src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_helpers.c
M src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_lru.c
2 files changed, 9 insertions(+), 11 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/97/465197/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465197
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: Icc5cf96c2f3380bc640b5ccc0e9fc455af2ba794
Gerrit-Change-Number: 465197
Gerrit-PatchSet: 1
Gerrit-Owner: Matt Benjamin <mbenjamin(a)redhat.com>
Gerrit-Reviewer: Daniel Gryniewicz <dang(a)redhat.com>
Gerrit-MessageType: newchange
4 years, 8 months
Change in ...nfs-ganesha[next]: gpfs: python2 -> python3, .spec file fix
by Kaleb KEITHLEY (GerritHub)
Kaleb KEITHLEY has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465157 )
Change subject: gpfs: python2 -> python3, .spec file fix
......................................................................
gpfs: python2 -> python3, .spec file fix
missed a py2->py3 switch in .../scripts/gpfs-epoch
nfs-ganesha.spec file uses IBM package name for Requires:
(Although I wonder if the -utils RPM should really depend on -gpfs.
Aren't the bulk of the utils usable independent of whether nfs-ganesha-gpfs
is installed?)
Signed-off-by: Kaleb S. KEITHLEY <kkeithle(a)redhat.com>
Change-Id: Idbe7f36a200b408c776f2c20ff179e9169a0f123
---
M src/nfs-ganesha.spec-in.cmake
M src/scripts/gpfs-epoch
2 files changed, 3 insertions(+), 3 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/57/465157/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465157
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: Idbe7f36a200b408c776f2c20ff179e9169a0f123
Gerrit-Change-Number: 465157
Gerrit-PatchSet: 1
Gerrit-Owner: Kaleb KEITHLEY <kaleb(a)redhat.com>
Gerrit-MessageType: newchange
4 years, 8 months
Change in ...nfs-ganesha[next]: Protocol: add config option to skip utf8 string scan
by Name of user not set (GerritHub)
xujiatong(a)cmss.chinamobile.com has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465149 )
Change subject: Protocol: add config option to skip utf8 string scan
......................................................................
Protocol: add config option to skip utf8 string scan
Client got "invalid arg"return when filename appeared garbled
with NFSv4 mount.Besides,NFSv3 & any protocols exporting NFSD
could do ops exactly.
Provide config option to keep the consistency of NFSv3/v4.
Signed-off-by: Jerome.Xu <xujiatong(a)cmss.chinamobile.com>
Change-Id: Iadf4f83c377ada0fb2d50e855827d8968dde4093
---
M src/Protocols/NFS/nfs_proto_tools.c
M src/include/gsh_config.h
M src/support/nfs_read_conf.c
3 files changed, 9 insertions(+), 0 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/49/465149/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465149
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: Iadf4f83c377ada0fb2d50e855827d8968dde4093
Gerrit-Change-Number: 465149
Gerrit-PatchSet: 1
Gerrit-Owner: xujiatong(a)cmss.chinamobile.com
Gerrit-MessageType: newchange
4 years, 8 months
Change in ...nfs-ganesha[next]: mdcache_create_handle: Don't lookup for parent for a root object
by Madhu Thorat (GerritHub)
Madhu Thorat has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465126 )
Change subject: mdcache_create_handle: Don't lookup for parent for a root object
......................................................................
mdcache_create_handle: Don't lookup for parent for a root object
Currently in mdcache_create_handle() after getting an 'entry' we
call mdc_get_parent() for the 'entry'. But if the entry->obj_handle
points to root object then we would be doing lookup of parent for the
root, which we should not be doing.
To avoid this prior to call mdc_get_parent() check if the
entry->obj_handle points to a root object, if yes then don't call
mdc_get_parent() for the 'entry'.
Change-Id: Ibf25fd283573c9fa28ae0452d96f00f294f66828
Signed-off-by: Madhu Thorat <madhu.punjabi(a)in.ibm.com>
---
M src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_handle.c
1 file changed, 21 insertions(+), 2 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/26/465126/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465126
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: Ibf25fd283573c9fa28ae0452d96f00f294f66828
Gerrit-Change-Number: 465126
Gerrit-PatchSet: 1
Gerrit-Owner: Madhu Thorat <madhu.punjabi(a)in.ibm.com>
Gerrit-MessageType: newchange
4 years, 8 months
Change in ...nfs-ganesha[next]: fix wrong package name and dependency
by Ji-Hyeon Gim (GerritHub)
Ji-Hyeon Gim has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465123 )
Change subject: fix wrong package name and dependency
......................................................................
fix wrong package name and dependency
Problem
In RPM spec file, the package name for GPFS is gpfs.nfs-ganesha but that
it is a name for IBM only.
And also, nfs-ganesha-utils has a dependency with gpfs flag but this
requirement should not be indispensable.
Signed-off-by: Ji-Hyeon Gim <potatogim(a)gluesys.com>
Change-Id: I67a2b76a1b41a62c1d280d1461d7baf5d013a3a5
---
M src/nfs-ganesha.spec-in.cmake
1 file changed, 12 insertions(+), 6 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/23/465123/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465123
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: I67a2b76a1b41a62c1d280d1461d7baf5d013a3a5
Gerrit-Change-Number: 465123
Gerrit-PatchSet: 1
Gerrit-Owner: Ji-Hyeon Gim <potatogim(a)potatogim.net>
Gerrit-MessageType: newchange
4 years, 8 months
Change in ...nfs-ganesha[next]: gtest: portability to gcc-4.8.5 in RHEL/CentOS 7
by Kaleb KEITHLEY (GerritHub)
Kaleb KEITHLEY has uploaded this change for review. ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465099 )
Change subject: gtest: portability to gcc-4.8.5 in RHEL/CentOS 7
......................................................................
gtest: portability to gcc-4.8.5 in RHEL/CentOS 7
gcc-4.8.5 has no namespace std::literals
lacking literals, use std::chrono::seconds()
std=gnu-g++11 seems to adequate to get typeof, esp. as gcc-4.8.5 doesn't
know gnu-g++14
All to be able to build gtest in the CentOS CI
Signed-off-by: Kaleb S. KEITHLEY <kkeithle(a)redhat.com>
Change-Id: I8611a8a1219524fafc0c2fa9f0521b22bb81a59f
---
M src/gtest/CMakeLists.txt
M src/gtest/fsal_api/test_close2_latency.cc
M src/gtest/fsal_api/test_close_latency.cc
M src/gtest/fsal_api/test_commit2_latency.cc
M src/gtest/fsal_api/test_getattrs_latency.cc
M src/gtest/fsal_api/test_handle_to_key_latency.cc
M src/gtest/fsal_api/test_handle_to_wire_latency.cc
M src/gtest/fsal_api/test_link_latency.cc
M src/gtest/fsal_api/test_lock_op2_latency.cc
M src/gtest/fsal_api/test_lookup_latency.cc
M src/gtest/fsal_api/test_mkdir_latency.cc
M src/gtest/fsal_api/test_mknode_latency.cc
M src/gtest/fsal_api/test_open2_latency.cc
M src/gtest/fsal_api/test_read2_latency.cc
M src/gtest/fsal_api/test_readdir_correctness.cc
M src/gtest/fsal_api/test_readdir_latency.cc
M src/gtest/fsal_api/test_readlink_latency.cc
M src/gtest/fsal_api/test_release_latency.cc
M src/gtest/fsal_api/test_rename_latency.cc
M src/gtest/fsal_api/test_reopen2_latency.cc
M src/gtest/fsal_api/test_setattr2_latency.cc
M src/gtest/fsal_api/test_symlink_latency.cc
M src/gtest/fsal_api/test_unlink_latency.cc
M src/gtest/fsal_api/test_write2_latency.cc
M src/gtest/gtest.hh
M src/gtest/nfs4/test_nfs4_link_latency.cc
M src/gtest/nfs4/test_nfs4_lookup_latency.cc
M src/gtest/nfs4/test_nfs4_putfh_latency.cc
M src/gtest/nfs4/test_nfs4_rename_latency.cc
M src/gtest/test_ci_hash_dist1.cc
30 files changed, 3 insertions(+), 32 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/99/465099/1
--
To view, visit https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/465099
To unsubscribe, or for help writing mail filters, visit https://review.gerrithub.io/settings
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Change-Id: I8611a8a1219524fafc0c2fa9f0521b22bb81a59f
Gerrit-Change-Number: 465099
Gerrit-PatchSet: 1
Gerrit-Owner: Kaleb KEITHLEY <kaleb(a)redhat.com>
Gerrit-MessageType: newchange
4 years, 8 months