CephFS via Kernel client + NFS Ganesha
Hi all, I am trying to implement a highly available (active active) nfs server cluster for our infrastructure. My initial implementation was with CephFS -> FSAL_CEPH with rados_cluster recovery backend but it didn't perform very well for speeds maybe due to libcephfs ( ceph-fuse) being really slow compared to cephfs kernel client. I was getting 1.8GB/s ( kernel client) compared to 60MB/s (ceph-fuse and NFS Ganesha) I am now trying to implement CephFS -> Kernel client mount -> FSAL_VFS with rados cluster recovery backed but seems like FSAL_VFS doesn't like the cephfs kernel mount. Whenever I am trying to mount nfs on the client i am getting errors listed below in ganesha. Please let me know what is the best way out to get good performance. 12/04/2019 16:01:14 : epoch 5cb0b61c : ganesha2-9b76ccf7b-wdpds : ganesha.nfsd-1[svc_3] nfs4_Compound :NFS4 :DEBUG :Status of OP_GETFH in position 3 = NFS4_OK, op response size is 40 total response size is 136 12/04/2019 16:01:14 : epoch 5cb0b61c : ganesha2-9b76ccf7b-wdpds : ganesha.nfsd-1[svc_3] nfs4_Compound :NFS4 :DEBUG :Request 4: opcode 9 is OP_GETATTR 12/04/2019 16:01:14 : epoch 5cb0b61c : ganesha2-9b76ccf7b-wdpds : ganesha.nfsd-1[svc_3] file_To_Fattr :NFS4 ACL :DEBUG :No permission check for ACL for obj 0x56358b04bb08 12/04/2019 16:01:14 : epoch 5cb0b61c : ganesha2-9b76ccf7b-wdpds : ganesha.nfsd-1[svc_3] posix2fsal_error :FSAL :CRIT :Mapping 38(default) to ERR_FSAL_SERVERFAULT 12/04/2019 16:01:14 : epoch 5cb0b61c : ganesha2-9b76ccf7b-wdpds : ganesha.nfsd-1[svc_3] vfs_open_by_handle :FSAL :DEBUG :Failed with Function not implemented openflags 0x00000000 12/04/2019 16:01:14 : epoch 5cb0b61c : ganesha2-9b76ccf7b-wdpds : ganesha.nfsd-1[svc_3] find_fd :FSAL :DEBUG :Failed with Function not implemented openflags 0x00000020 12/04/2019 16:01:14 : epoch 5cb0b61c : ganesha2-9b76ccf7b-wdpds : ganesha.nfsd-1[svc_3] posix2fsal_error :FSAL :CRIT :Mapping 38(default) to ERR_FSAL_SERVERFAULT 12/04/2019 16:01:14 : epoch 5cb0b61c : ganesha2-9b76ccf7b-wdpds : ganesha.nfsd-1[svc_3] posix2fsal_error :FSAL :CRIT :Mapping 38(default) to ERR_FSAL_SERVERFAULT 12/04/2019 16:01:14 : epoch 5cb0b61c : ganesha2-9b76ccf7b-wdpds : ganesha.nfsd-1[svc_3] vfs_open_by_handle :FSAL :DEBUG :Failed with Function not implemented openflags 0x00000000 12/04/2019 16:01:14 : epoch 5cb0b61c : ganesha2-9b76ccf7b-wdpds : ganesha.nfsd-1[svc_3] find_fd :FSAL :DEBUG :Failed with Function not implemented openflags 0x00000020 12/04/2019 16:01:14 : epoch 5cb0b61c : ganesha2-9b76ccf7b-wdpds : ganesha.nfsd-1[svc_3] posix2fsal_error :FSAL :CRIT :Mapping 38(default) to ERR_FSAL_SERVERFAULT 12/04/2019 16:01:14 : epoch 5cb0b61c : ganesha2-9b76ccf7b-wdpds : ganesha.nfsd-1[svc_3] fsal_common_is_referral :FSAL :EVENT :Failed to get attributes for referral, request_mask: 1433550 12/04/2019 16:01:14 : epoch 5cb0b61c : ganesha2-9b76ccf7b-wdpds : ganesha.nfsd-1[svc_3] nfs4_Compound :NFS4 :DEBUG :Status of OP_GETATTR in position 4 = NFS4ERR_SERVERFAULT, op response size is 4 total response size is 144 Thanks, Shantur
participants (1)
-
Shantur Rathore