I understood what was going on.
This is due to "timeo=2,retrans=3". In dmesg, I saw "nfs: server 127.0.0.1 not responding, timed out" and in tcpdump I saw that some WRITE were not acked.
Because of that, the kernel decided that some write failed (and indeed, they have…) and thus fdatasync() returned EIO.
Increasing the timeout parameter and/or the number of retransmissions helped to no longer have IO errors.
--
Boris Faure
Software Engineer


On Tue, Sep 25, 2018 at 7:01 PM Malahal Naineni <malahal@gmail.com> wrote:
tcpdump would be good to chase this.

On Tue, Sep 25, 2018 at 6:10 PM, Daniel Gryniewicz <dang@redhat.com> wrote:
Hi, Boris.

Which FSAL is this?

Daniel

On 09/25/2018 07:50 AM, Boris Faure wrote:
Hi everyone,
I'm running a simple test with ganesha-next (but the issue exists at least in latest V2.5.5-stable):
$ sudo mount -t nfs -o v4.0,users,nolock,soft,timeo=2,retrans=3 127.0.0.1:/  /mnt/nfs
$ dd if=/dev/zero of=/mnt/nfs/dump/pw.1 bs=1024k count=2048 conv=fdatasync
dd: error writing '/mnt/nfs/dump/pw.1': Input/output error
1616+0 records in
1615+0 records out
1693450240 bytes (1,7 GB, 1,6 GiB) copied, 8,28076 s, 205 MB/s
$ sudo mount -t nfs -o users,nolock,soft,timeo=2,retrans=3 127.0.0.1:/  /mnt/nfs
$ dd if=/dev/zero of=/mnt/nfs/dump/pw.1 bs=1024k count=2048 conv=fdatasync
2048+0 records in
2048+0 records out
2147483648 bytes (2,1 GB, 2,0 GiB) copied, 6,27708 s, 342 MB/s

The second mount is using nfs v4.1.
I don't have anything special in my configuration.

If I do the dd with a blocksize of 512k, it works fine.
If I set rsize/wsize to 524288 as mount options, the dd with bs=1024k still fails.
The only log line I get is the following:
25/09/2018 13:47:25 : epoch 5baa2036 : caipirinha : ganesha.nfsd-2381[svc_182] rpc :TIRPC :EVENT :svc_ioq_flushv() writev failed (104)

Thanks for your help.
Best Regards
--
Boris Faure
Software Engineer

<https://www.scality.com/download-idc-marketscape/?utm_source=ScalityEmail&utm_medium=Email&utm_campaign=2018Predictions>


_______________________________________________
Devel mailing list -- devel@lists.nfs-ganesha.org
To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org

_______________________________________________
Devel mailing list -- devel@lists.nfs-ganesha.org
To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org

_______________________________________________
Devel mailing list -- devel@lists.nfs-ganesha.org
To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org