[NFS-Ganesha-Support] Re: consistently reproduceable problem with kvm guest booting root filesystem via nfs-ganesha 2.8 proxy fsal

On Tue, 10 Mar 2020, Daniel Gryniewicz wrote: I've tried that by changing "-N NIV_CRIT" to "-N NIV_DEBUG" in /etc/sysconfig/ganesha - is there a better way? - but this did not reveal anything interesting to my eyes. I'm quite willing to accept that my eyes may have missed something. Something that I neglected to mention previously is that sometimes (maybe always, not sure yet) when the error condition occurs in the kvm guest, the kvm host's nfs-ganesha proxy mount has gone into a faulty state and requires an unmount and remount to recover. I'll consider this. I may need to first need to set up a proper test environment, unless I can figure out (or someone can tell me :) how best to focus on the useful packets and not be overwhelmed by extraneous data. All of our many kvm images are on NFS and have been for years so I do have plenty of personal experience with this, but having them on nfs via ganesha is a new experiment for us. We do have many terabytes of data filesystems served via nfs-ganesha proxy fsal though, with generally good results. Also CentOS 7. Everything in our mix is CO6 or CO7. Thanks, Todd > > Daniel > > On 3/9/20 5:46 PM, Todd Pfaff wrote: >> I have a reproduceable problem when trying to boot a kvm guest whose >> root filesystem is on an nfs path that is accessed from the kvm virt >> host via nfs-ganesha-2.8 and the proxy fsal. >> >> All hosts involved: kvm host, kvm guest, and nfs-ganesha server, run >> CentOS 7. >> >> The guest begins to boot but consistently fails just after the switch >> root with ext4 errors like this: >> >> [  OK  ] Started Plymouth switch root service. >>          Starting Switch Root... >> [   15.154967] systemd-journald[145]: Received SIGTERM from PID 1 >> (systemd). >> [   15.168447] EXT4-fs error (device vda1): __ext4_get_inode_loc:4247: >> inode #131073: block 524320: comm systemd: unable to read itable block >> [   15.269495] EXT4-fs warning (device vda1): >> __ext4_read_dirblock:676: error reading directory block (ino 524296, >> block 0) >> [   15.278651] systemd[1]: Failed to execute /sbin/init, giving up: >> Input/output error >> >> >> I'm using nfs-ganesha from the CentOS 7 Storage SIG. >> >> I've tested with both the latest stable 2.8 version from repo >> [centos-nfs-ganesha28]: >> >>   nfs-ganesha-2.8.3-3.el7.x86_64 >> >> and the latest test version from [centos-nfs-ganesha28-test]: >> >>   nfs-ganesha-2.8.3-4.el7.x86_64 >> >> with similar results.  I've also tried with nfs v3 and nfs v4.1 mounts >> from the kvm host. >> >> >> The kvm guest disk is defined as: >> >>   <disk type='file' device='disk'> >>     <driver name='qemu' type='qcow2' cache='none'/> >>     <source file='/mnt/ganesha/kvm/guest.qcow2'/> >>     <target dev='vdc' bus='virtio'/> >>     <boot order='1'/> >>     <address type='pci' domain='0x0000' bus='0x00' slot='0x08' >>       function='0x0'/> >>   </disk> >> >> >> The kvm guest works fine if the kvm host accesses this qcow2 image via >> a direct nfs mount instead of via nfs-ganesha proxy. >> >> Thoughts? >> >> Thanks, >> Todd >> _______________________________________________ >> Support mailing list -- support(a)lists.nfs-ganesha.org >> To unsubscribe send an email to support-leave(a)lists.nfs-ganesha.org > _______________________________________________ > Support mailing list -- support(a)lists.nfs-ganesha.org > To unsubscribe send an email to support-leave(a)lists.nfs-ganesha.org >