ESXI 6.7 client creating Thick Eager zeroed vmdk files using ceph fsal
by Robert Toole
Hi,
I have a 3 node Ceph octopus 15.2.7 cluster running on fully up to date
Centos 7 with nfs-ganesha 3.5.
After following the Ceph install guide
https://docs.ceph.com/en/octopus/cephadm/install/#deploying-nfs-ganesha
I am able to create a NFS 4.1 Datastore in vmware using the ip address
of all three nodes. Everything appears to work OK..
The issue however is that for some reason esxi is creating thick
provisioned eager zeroed disks instead of thin provisioned disks on this
datastore, whether I am migrating, cloning, or creating new vms. Even
running vmkfstools -i disk.vmdk -d thin thin_disk.vmdk still results in
a thick eager zeroed vmdk file.
This should not be possible on an NFS datastore, because vmware requires
a VAAI NAS plugin to accomplish thick provisioning over NFS before it
can thick provision disks.
Linux clients to the same datastore can create thin qcow2 images, and
when looking at the images created by esxi from the linux hosts you can
see that the vmdks are indeed thick:
ls -lsh
total 81G
512 -rw-r--r--. 1 root root 230 Mar 25 15:17 test_vm-2221e939.hlog
40G -rw-------. 1 root root 40G Mar 25 15:17 test_vm-flat.vmdk
40G -rw-------. 1 root root 40G Mar 25 15:56 test_vm_thin-flat.vmdk
512 -rw-------. 1 root root 501 Mar 25 15:57 test_vm_thin.vmdk
512 -rw-------. 1 root root 473 Mar 25 15:17 test_vm.vmdk
0 -rw-r--r--. 1 root root 0 Jan 6 1970 test_vm.vmsd
2.0K -rwxr-xr-x. 1 root root 2.0K Mar 25 15:17 test_vm.vmx
but the qcow2 files from the linux hosts are thin as one would expect:
qemu-img create -f qcow2 big_disk_2.img 500G
ls -lsh
total 401K
200K -rw-r--r--. 1 root root 200K Mar 25 15:47 big_disk_2.img
200K -rw-r--r--. 1 root root 200K Mar 25 15:44 big_disk.img
512 drwxr-xr-x. 2 root root 81G Mar 25 15:57 test_vm
These ls -lsh results are the same from esx, linux nfs clients and from
cephfs kernel client.
What is happening here? Are there undocumented VAAI features in
nfs-ganesha with the cephfs fsal ? If so, how do I turn them off ? I
want thin provisioned disks.
ceph nfs export ls dev-nfs-cluster --detailed
[
{
"export_id": 1,
"path": "/Development-Datastore",
"cluster_id": "dev-nfs-cluster",
"pseudo": "/Development-Datastore",
"access_type": "RW",
"squash": "no_root_squash",
"security_label": true,
"protocols": [
4
],
"transports": [
"TCP"
],
"fsal": {
"name": "CEPH",
"user_id": "dev-nfs-cluster1",
"fs_name": "dev_cephfs_vol",
"sec_label_xattr": ""
},
"clients": []
}
]
rpm -qa | grep ganesha
nfs-ganesha-ceph-3.5-1.el7.x86_64
nfs-ganesha-rados-grace-3.5-1.el7.x86_64
nfs-ganesha-rados-urls-3.5-1.el7.x86_64
nfs-ganesha-3.5-1.el7.x86_64
centos-release-nfs-ganesha30-1.0-2.el7.centos.noarch
rpm -qa | grep ceph
python3-cephfs-15.2.7-0.el7.x86_64
nfs-ganesha-ceph-3.5-1.el7.x86_64
python3-ceph-argparse-15.2.7-0.el7.x86_64
python3-ceph-common-15.2.7-0.el7.x86_64
cephadm-15.2.7-0.el7.x86_64
libcephfs2-15.2.7-0.el7.x86_64
ceph-common-15.2.7-0.el7.x86_64
ceph -v
ceph version 15.2.7 (<ceph_uuid>) octopus (stable)
The ceph cluster is healthy using bluestore on raw 3.84TB sata 7200 rpm
disks.
--
Robert Toole
rtoole(a)tooleweb.ca
403 368 5680
4 weeks
Announce Push of V6.4
by Frank Filz
Branch next
Tag:V6.4
Merge Highlights
This merge is mostly small patches from Google to closer match their
internal repos. Patches that are bug fixes are listed first followed by
some minor "features".
* Fix build break for USE_NFS3=OFF
* Replace _LINUX macro with linux
* Handle suid, sgid bits reset during chown operation
* src/SAL/state_lock: Fix a bug of adressing the wrong field
* Free allocated strings
* nfs4_op_open: Check permissions on upgrade re-open
* Added a metric for dropped gss requests
* fsal_manager.c: Add support to configure / init static fsals
* Add dbus method reread_config
* client_mgr: Expose DBUS method to disconnect a nfsv41 client
* nfs4_pseudo.c: Update some comments
* Added logs and lttng traces
* Added missing includes and initializations
* Clang format cc files
* src: Clang formatting
* CMakeLists: Resolve __FILE__ to a relative path to the project in logs by
default
* Replace malloc and free with gsh_malloc and gsh_free respectively
* idmapping cache cleanup: log number of entries removed
* Gss-callbacks: Remove rpc callback initialisation on startup
* Extracted credentials squashing logic to a separate function
* Extracted setting the extended groups to a separate function
* Use cmake3 to build Ganesha on centos
* Added an option to print the client XID as part of the log entry
* Added nfs request to op context if a request is originated from nfs client
Signed-off-by: Frank S. Filz <ffilzlnx(a)mindspring.com>
Contents:
178e02de9 Frank S. Filz V6.4
a7bd32d8b izxl007 nfs4_pseudo.c: Update some comments
72b4544ca Sachin Punadikar Fix build break for USE_NFS3=OFF
e090a299d Frank S. Filz Replace _LINUX macro with linux
c6343a211 Lior Suliman Added a metric for dropped gss requests
0b9d7a984 Lior Suliman Added logs and lttng traces
7eeafc8b9 Dipit Grover Handle suid, sgid bits reset during chown operation
0d70330da Lior Suliman Added missing includes and initializations
ea6d183e6 Lior Suliman Clang format cc files
f2db73ff6 Lior Suliman fsal_manager.c: Add support to configure / init
static fsals
6b3f9e437 Lior Suliman src: Clang formatting
dfd26b7b9 Lior Suliman CMakeLists: Resolve __FILE__ to a relative path to
the project in logs by default
5c0b0ff8a Lior Suliman Add dbus method reread_config
dd2c3edc5 Lior Suliman src/SAL/state_lock: Fix a bug of adressing the wrong
field
a0ab1c690 Gaurav Sharma Replace malloc and free with gsh_malloc and gsh_free
respectively
74266e89e Ofir Vainshtein nfs_lib.c: Free allocated strings
cb098d40e Dipit Grover idmapping cache cleanup: log number of entries
removed
bacd45e72 Lior Suliman nfs4_op_open: Check permissions on upgrade re-open
b89ee1347 Dipit Grover client_mgr: Expose DBUS method to disconnect a nfsv41
client
789b30dc2 Lior Suliman Gss-callbacks: Remove rpc callback initialisation on
startup
ed0ef716d Lior Suliman Extracted credentials squashing logic to a separate
function
544b951e1 Lior Suliman Extracted setting the extended groups to a separate
function
053f1351c Lior Suliman Use cmake3 to build Ganesha on centos
2b4682f04 Lior Suliman Added an option to print the client XID as part of
the log entry
078e7281f Lior Suliman Added nfs request to op context if a request is
originated from nfs client
1 month, 3 weeks
About Top Adult SEO
by wotebo3207＠pokeline.com
So, when choosing private escort services, remember the things mentioned above and get you a model like no other. You will also find that it is difficult to get a good one, but you have to try very hard in order to find someone who can satisfy you. It is important that you make sure they are honest and reliable and they know what they are doing.
https://www.primeporndirectory.com/
1 month, 4 weeks
