FSAL_GLUSTER passes locks down through the gluster stack where eventually gluster uses POSIX locking in the backing file system. Hmmm. Apropos of nothing, I thought I'd been told that Samba keeps all its lock state in CTDB, but the Gluster VFS in Samba 4.10 clearly uses gluster locking. Interesting. On Fri, Nov 22, 2019 at 10:59 AM Malahal Naineni <malahal@gmail.com> wrote:
Oh ok, Ganesha being a user level program, it loses all POSIX locks! GPFS fsal has some kernel part (not needed for graceful shutdown but abrupt shutdown needs kernel part) where it notifies some monitoring tool which puts the entire cluster of Ganesha nodes in grace period to process only reclaims! I assume neither Gluster nor Cephs use POSIX locks for NFS locks, correct? Didn't realize that graceful shutdown is worse than abrupt shutdown with some FSALs.
On Fri, Nov 22, 2019 at 8:30 PM Daniel Gryniewicz <dang@redhat.com> wrote:
We've had problems with Gluster and Ceph, where the clean shutdown closes all the files and releases the locks. Then, another client can get those locks before the failover is complete, and they can't be recovered. For Ceph, we added a special mode where it doesn't release any state.
Daniel
On 11/22/19 9:57 AM, Daniel Gryniewicz wrote:
We've definitely had problems with graceful shutdown on Gluster. It closed all the files, dropping all the locks, and another client was able to take those locks before failover occurred. For Ceph, we added a special mode where it doesn't release locks or state.
Daniel
On 11/22/19 9:33 AM, Malahal Naineni wrote:
Even a graceful shutdown should work fine unless there are some bugs in the shutdown path. The client should NOT use "soft" mount option though. We have used graceful shutdown (with GPFS fsal), and didn't encounter any issue yet.
On Fri, Nov 22, 2019 at 1:06 AM Daniel Gryniewicz <dgryniew@redhat.com <mailto:dgryniew@redhat.com>> wrote:
Expanding on that, the information we save for recovery hasn't changed in a while, so upgrades should be fine. However, make sure you do an unclean shutdown of the Ganesha to be upgraded (e.g., kill -9 rather than kill) since a clean shutdown will clean up all the state/locks/open files and so on, making recovery by active clients impossible.
Daniel
On 11/21/19 2:17 PM, Kaleb Keithley wrote: > Hi, > > To the best of my knowledge there's no reason why a rolling upgrade > wouldn't work seamlessly. > > There's nothing version specific about the recovery and no reason I'm > aware of that differing versions of ganesha wouldn't work side by side > in an HA cluster over the course of an upgrade. > > I suggest you play it safe and try it in a non-production environment > before doing it on a production system. > > HTH. > > > > > On Thu, Nov 21, 2019 at 2:07 PM Charles Hedrick <hedrick@rutgers.edu <mailto:hedrick@rutgers.edu> > <mailto:hedrick@rutgers.edu <mailto:hedrick@rutgers.edu>>> wrote: > > I understand that Genesha supports HA configurations. Software > upgrades are a lot more frequent than hardware failures. The > ability to do upgrades is actually more of a motivation for HA > than handling failures. Do you support rolling upgrades, i.e. > upgrading one node while the other is active? That would require > that the two nodes would end up with different versions for at > least some period of time. > _______________________________________________ > Support mailing list -- support@lists.nfs-ganesha.org <mailto:support@lists.nfs-ganesha.org> > <mailto:support@lists.nfs-ganesha.org <mailto:support@lists.nfs-ganesha.org>> > To unsubscribe send an email to > support-leave@lists.nfs-ganesha.org <mailto:support-leave@lists.nfs-ganesha.org> > <mailto:support-leave@lists.nfs-ganesha.org <mailto:support-leave@lists.nfs-ganesha.org>> > > > _______________________________________________ > Support mailing list -- support@lists.nfs-ganesha.org <mailto:support@lists.nfs-ganesha.org> > To unsubscribe send an email to support-leave@lists.nfs-ganesha.org <mailto:support-leave@lists.nfs-ganesha.org> _______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org <mailto:support@lists.nfs-ganesha.org> To unsubscribe send an email to support-leave@lists.nfs-ganesha.org <mailto:support-leave@lists.nfs-ganesha.org>