Kirill Gibelgaus has uploaded this change for review.
[RFC] FSAL_VFS: add opt-in strict O_DIRECT I/O
Streaming NFS reads can repeatedly fill and reclaim a container's file
cache when the dataset exceeds its memory limit. Client-side direct I/O
does not bypass this server-side cache. Issue #1439 describes the
experiments that motivated this proposal.
Add a Linux-only VFS module option, ODirect, disabled by default:
VFS { ODirect = true; }
Require O_DIRECT on regular-file data descriptors, including create and
reopen. Keep descriptor reuse and the existing NFS state/share ordering.
Use STATX_DIOALIGN to check alignment. Requests whose transport buffers
are misaligned use a request-local aligned RAM buffer, preserving the
original TCP/RDMA vectors and their ownership. Scatter only bytes
actually read, and recognize short EOF reads even when protocol-level
Getattrs_In_Complete_Read is disabled.
This is deliberately a strict mode. Unaligned file offsets or lengths
return EINVAL; unknown alignment or a non-direct data descriptor returns
EOPNOTSUPP. Direct-open errors propagate without retrying buffered.
Defer named-open truncation until direct-mode setup succeeds so a failed
request cannot erase existing data. No rounded read-modify-write is
introduced.
Disable native VFS COPY in this mode because copy_file_range and its
fd-level fallback can use page cache. Generic FSAL COPY paths retain
the strict READ/WRITE restrictions. Metadata caches and backend caches,
including ZFS ARC, are outside this option. The filesystem still decides
its internal direct-I/O behavior; the option does not imply O_SYNC.
Local Linux/ext4 probes compile the actual changed source functions with
FSAL bookkeeping stubbed and real file I/O. ASan/UBSan passed 47 I/O
checks, 23 open/create checks, and four native COPY cases. These include
misaligned buffers, rejected byte ranges, short EOF without overwriting
the unread tail, descriptor flags, failure before truncation, and no
resident data pages after the tested direct write. A build with
STATX_DIOALIGN absent rejects strict I/O and preserves the default path.
clang-format, checkpatch and git diff --check also passed.
A complete Ganesha build, NFS protocol E2E, RDMA hardware tests and new
throughput measurements of this upstream variant have not been run.
This RFC asks whether an explicitly strict, opt-in alignment contract is
useful for VFS. General byte-granular NFS applications can fail when it is
enabled. Feedback on this policy and the preferred upstream approach to
avoiding page-cache pressure would be welcome.
Link: https://github.com/nfs-ganesha/nfs-ganesha/issues/1439
Change-Id: I7c627fc7243b771cd7a798176971e1ffb6f4f73c
Signed-off-by: Kirill Gibelgaus <kgibelgaus@ozon.ru>
---
M src/FSAL/FSAL_VFS/file.c
M src/FSAL/FSAL_VFS/handle.c
M src/FSAL/FSAL_VFS/vfs/main-c.in.cmake
M src/FSAL/FSAL_VFS/vfs_methods.h
M src/doc/man/ganesha-vfs-config.rst
5 files changed, 193 insertions(+), 4 deletions(-)
git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha refs/changes/14/1260114/1
To view, visit change 1260114. To unsubscribe, or for help writing mail filters, visit settings.