September 2021 - Support - NFS Ganesha List Archives

ESXI 6.7 client creating Thick Eager zeroed vmdk files using ceph fsal
by Robert Toole 07 Jan '25

07 Jan '25

Hi, I have a 3 node Ceph octopus 15.2.7 cluster running on fully up to date Centos 7 with nfs-ganesha 3.5. After following the Ceph install guide https://docs.ceph.com/en/octopus/cephadm/install/#deploying-nfs-ganesha I am able to create a NFS 4.1 Datastore in vmware using the ip address of all three nodes. Everything appears to work OK.. The issue however is that for some reason esxi is creating thick provisioned eager zeroed disks instead of thin provisioned disks on this datastore, whether I am migrating, cloning, or creating new vms. Even running vmkfstools -i disk.vmdk -d thin thin_disk.vmdk still results in a thick eager zeroed vmdk file. This should not be possible on an NFS datastore, because vmware requires a VAAI NAS plugin to accomplish thick provisioning over NFS before it can thick provision disks. Linux clients to the same datastore can create thin qcow2 images, and when looking at the images created by esxi from the linux hosts you can see that the vmdks are indeed thick: ls -lsh total 81G 512 -rw-r--r--. 1 root root 230 Mar 25 15:17 test_vm-2221e939.hlog 40G -rw-------. 1 root root 40G Mar 25 15:17 test_vm-flat.vmdk 40G -rw-------. 1 root root 40G Mar 25 15:56 test_vm_thin-flat.vmdk 512 -rw-------. 1 root root 501 Mar 25 15:57 test_vm_thin.vmdk 512 -rw-------. 1 root root 473 Mar 25 15:17 test_vm.vmdk 0 -rw-r--r--. 1 root root 0 Jan 6 1970 test_vm.vmsd 2.0K -rwxr-xr-x. 1 root root 2.0K Mar 25 15:17 test_vm.vmx but the qcow2 files from the linux hosts are thin as one would expect: qemu-img create -f qcow2 big_disk_2.img 500G ls -lsh total 401K 200K -rw-r--r--. 1 root root 200K Mar 25 15:47 big_disk_2.img 200K -rw-r--r--. 1 root root 200K Mar 25 15:44 big_disk.img 512 drwxr-xr-x. 2 root root 81G Mar 25 15:57 test_vm These ls -lsh results are the same from esx, linux nfs clients and from cephfs kernel client. What is happening here? Are there undocumented VAAI features in nfs-ganesha with the cephfs fsal ? If so, how do I turn them off ? I want thin provisioned disks. ceph nfs export ls dev-nfs-cluster --detailed [ { "export_id": 1, "path": "/Development-Datastore", "cluster_id": "dev-nfs-cluster", "pseudo": "/Development-Datastore", "access_type": "RW", "squash": "no_root_squash", "security_label": true, "protocols": [ 4 ], "transports": [ "TCP" ], "fsal": { "name": "CEPH", "user_id": "dev-nfs-cluster1", "fs_name": "dev_cephfs_vol", "sec_label_xattr": "" }, "clients": [] } ] rpm -qa | grep ganesha nfs-ganesha-ceph-3.5-1.el7.x86_64 nfs-ganesha-rados-grace-3.5-1.el7.x86_64 nfs-ganesha-rados-urls-3.5-1.el7.x86_64 nfs-ganesha-3.5-1.el7.x86_64 centos-release-nfs-ganesha30-1.0-2.el7.centos.noarch rpm -qa | grep ceph python3-cephfs-15.2.7-0.el7.x86_64 nfs-ganesha-ceph-3.5-1.el7.x86_64 python3-ceph-argparse-15.2.7-0.el7.x86_64 python3-ceph-common-15.2.7-0.el7.x86_64 cephadm-15.2.7-0.el7.x86_64 libcephfs2-15.2.7-0.el7.x86_64 ceph-common-15.2.7-0.el7.x86_64 ceph -v ceph version 15.2.7 (<ceph_uuid>) octopus (stable) The ceph cluster is healthy using bluestore on raw 3.84TB sata 7200 rpm disks. -- Robert Toole rtoole(a)tooleweb.ca 403 368 5680

4 5

Mount Failed on a GlusterFS running on OEL 7.9
by mkanaga＠gmail.com 23 Sep '21

23 Sep '21

Greetings! I have followed this example & created 3 Node Gluster FS (https://oracle.github.io/linux-labs/HA-NFS/) Everything is working except When I tried to mount this export to a client, It fails: --------------------------------------------- [root@master1 sysconfig]# pcs status Cluster name: HA-NFS Stack: corosync Current DC: master1 (version 1.1.23-1.0.1.el7-9acf116022) - partition with quorum Last updated: Wed Sep 22 04:56:33 2021 Last change: Wed Sep 22 02:28:21 2021 by root via cibadmin on master1 3 nodes configured 2 resource instances configured Online: [ master1 master2 master3 ] Full list of resources: Resource Group: nfs_group nfs_server (systemd:nfs-ganesha): Started master1 nfs_ip (ocf::heartbeat:IPaddr2): Started master1 Daemon Status: corosync: active/enabled pacemaker: active/enabled pcsd: active/enabled ---------------------------------------------- Here is the Mount Error: [root@client1 sysconfig]# mount -t nfs nfs.vagrant.vm:/data/glusterfs/sharedvol /sharedvol mount.nfs: mounting nfs.vagrant.vm:/data/glusterfs/sharedvol failed, reason given by server: No such file or directory [root@client1 sysconfig]# Even showmount is not providing any info: [root@client1 sysconfig]# showmount nfs.vagrant.vm Hosts on nfs.vagrant.vm: [root@client1 sysconfig]# Any help or pointers would be very much appreciated. Thank You Murali

2 1

Weekly Tuesday community call changes
by Frank Filz 08 Sep '21

08 Sep '21

We are moving to Google Meet to host the community meetings. I have sent a Google invite to the mailing list, or use this URL: https://meet.google.com/mkh-ctnj-tqz Phone numbers for dialing are available here: https://tel.meet/mkh-ctnj-tqz?pin=2844708468265 <https://tel.meet/mkh-ctnj-tqz?pin=2844708468265&hs=0> &hs=0 Due to my schedule changing the meetings will be limited to 30 minutes. At the next meeting we can discuss changing the time, otherwise it will be at the same 7:00 AM Pacific Time starting time. Maybe we'll try and hold off changing it until the time is forced to change by the ending of Daylight Saving Time in October. Frank

1 0