In case this is of any interest, ESTALE was not generated on the server side. Enabling
debug on the client side showed that while resolving the pathname, the fsid received was
seen as crossing the fs boundary, and resolve of the next part of the path was tried in
this new context and not found so the client said ESTALE.
Why was this happening and why only after some idle time (initially the same thing worked
ok) no idea, needs more testing. The point where it happened on the client is
nfs_xdev_get_sb function.