We've had problems with Gluster and Ceph, where the clean shutdown
closes all the files and releases the locks.  Then, another client can
get those locks before the failover is complete, and they can't be
recovered.  For Ceph, we added a special mode where it doesn't release
any state.

Daniel

On 11/22/19 9:57 AM, Daniel Gryniewicz wrote:
> We've definitely had problems with graceful shutdown on Gluster. It
> closed all the files, dropping all the locks, and another client was
> able to take those locks before failover occurred.  For Ceph, we added a
> special mode where it doesn't release locks or state.
>
> Daniel
>
> On 11/22/19 9:33 AM, Malahal Naineni wrote:
>> Even a graceful shutdown should work fine unless there are some bugs
>> in the shutdown path. The client should NOT use "soft" mount option
>> though. We have used graceful shutdown (with GPFS fsal), and didn't
>> encounter any issue yet.
>>
>> On Fri, Nov 22, 2019 at 1:06 AM Daniel Gryniewicz <dgryniew@redhat.com
>> <mailto:dgryniew@redhat.com>> wrote:
>>
>>     Expanding on that, the information we save for recovery hasn't
>>     changed
>>     in a while, so upgrades should be fine.  However, make sure you do an
>>     unclean shutdown of the Ganesha to be upgraded (e.g., kill -9 rather
>>     than kill) since a clean shutdown will clean up all the
>>     state/locks/open
>>     files and so on, making recovery by active clients impossible.
>>
>>     Daniel
>>
>>     On 11/21/19 2:17 PM, Kaleb Keithley wrote:
>>     > Hi,
>>     >
>>     > To the best of my knowledge there's no reason why a rolling upgrade
>>     > wouldn't work seamlessly.
>>     >
>>     > There's nothing version specific about the recovery and no
>>     reason I'm
>>     > aware of that differing versions of ganesha wouldn't work side
>>     by side
>>     > in an HA cluster over the course of an upgrade.
>>     >
>>     > I suggest you play it safe and try it in a non-production
>>     environment
>>     > before doing it on a production system.
>>     >
>>     > HTH.
>>     >
>>     >
>>     >
>>     >
>>     > On Thu, Nov 21, 2019 at 2:07 PM Charles Hedrick
>>     <hedrick@rutgers.edu <mailto:hedrick@rutgers.edu>
>>     > <mailto:hedrick@rutgers.edu <mailto:hedrick@rutgers.edu>>> wrote:
>>     >
>>     >     I understand that Genesha supports HA configurations. Software
>>     >     upgrades are a lot more frequent than hardware failures. The
>>     >     ability to do upgrades is actually more of a motivation for HA
>>     >     than handling failures. Do you support rolling upgrades, i.e.
>>     >     upgrading one node while the other is active? That would
>> require
>>     >     that the two nodes would end up with different versions for at
>>     >     least some period of time.
>>     >     _______________________________________________
>>     >     Support mailing list -- support@lists.nfs-ganesha.org
>>     <mailto:support@lists.nfs-ganesha.org>
>>     >     <mailto:support@lists.nfs-ganesha.org
>>     <mailto:support@lists.nfs-ganesha.org>>
>>     >     To unsubscribe send an email to
>>     > support-leave@lists.nfs-ganesha.org
>>     <mailto:support-leave@lists.nfs-ganesha.org>
>>     >     <mailto:support-leave@lists.nfs-ganesha.org
>>     <mailto:support-leave@lists.nfs-ganesha.org>>
>>     >
>>     >
>>     > _______________________________________________
>>     > Support mailing list -- support@lists.nfs-ganesha.org
>>     <mailto:support@lists.nfs-ganesha.org>
>>     > To unsubscribe send an email to
>>     support-leave@lists.nfs-ganesha.org
>>     <mailto:support-leave@lists.nfs-ganesha.org>
>>     _______________________________________________
>>     Support mailing list -- support@lists.nfs-ganesha.org
>>     <mailto:support@lists.nfs-ganesha.org>
>>     To unsubscribe send an email to
>>     support-leave@lists.nfs-ganesha.org
>>     <mailto:support-leave@lists.nfs-ganesha.org>
>>