Many thanks for the quick response Daniel! On Fri, May 17, 2019 at 4:55 PM Daniel Gryniewicz <dang@redhat.com> wrote:
Hi, David.
Answers inline.
On Fri, May 17, 2019 at 11:42 AM David C <dcsysengineer@gmail.com> wrote:
Hi All
I recently put an nfs-ganesha CEPH_FSAL deployment into production, so
far so good but I'm seeing some errors in the logs I didn't see when testing and was hoping someone could shed some light on what they mean. I haven't had any adverse behaviour reported from the clients (apart from a potential issue with slow 'ls' operations which I'm investigating).
Versions:
libcephfs2-13.2.2-0.el7.x86_64 nfs-ganesha-2.7.1-0.1.el7.x86_64 nfs-ganesha-ceph-2.7.1-0.1.el7.x86_64 Ceph cluster is 12.2.10
Log errors:
"posix2fsal_error :FSAL :INFO :Mapping 11 to ERR_FSAL_DELAY"
This is an INFO, so it's not an error. You might not want to run a production deployment at INFO, because it will be a bit chatty. The default is to run at EVENT.
I'm seeing this one frequently although seems to spam the log with 20 or
so occurrences in a second.
"15/05/2019 18:27:01 : epoch 5cd99ef1 : nfsserver :
ganesha.nfsd-1990[svc_1653] posix2fsal_error :FSAL :INFO :Mapping 5 to ERR_FSAL_IO, rlim_cur=1048576 rlim_max=1048576
15/05/2019 18:27:01 : epoch 5cd99ef1 : nfsserver : ganesha.nfsd-1990[svc_1653] nfs4_Errno_verbose :NFS4 :CRIT :Error I/O error in nfs4_mds_putfh converted to NFS4ERR_IO but was set non-retryable"
The IO error log is there because NFS4ERR_IO is a catchall error, that's used for lots and lots of situations. When a client sees NFS4ERR_IO, it can be really hard to know what caused the error. This one has traditionally been a CRIT message. I think that's high, I would put it at WARN, but that's the way it is. Something *did* go wrong, but not in Ganesha, there was some error in the underlying FS, and it was returned to the client.
I've only seen a few occurrences of this one
17/05/2019 15:34:24 : epoch 5cdd9df8 : nfsserver :
ganesha.nfsd-4696[svc_258] xdr_encode_nfs4_princ :ID MAPPER :INFO :nfs4_gid_to_name failed with code -2.
17/05/2019 15:34:24 : epoch 5cdd9df8 : nfsserver : ganesha.nfsd-4696[svc_258] xdr_encode_nfs4_princ :ID MAPPER :INFO :Lookup for 1664 failed, using numeric group
These, again, are INFO, and so don't indicate a true failure anywhere.
This one doesn't seem too serious, my guess is there are accounts on the
clients with gids/uids that the server can't look up. The server is using SSSD to bind to AD if that helps.
Correct.
Daniel