NFS over UNIX domain socket
Hello, It turns out that the macOS NFS client supports connecting to an NFS server over AF_LOCAL (UNIX domain) sockets. It is undocumented, but a hint of this is here: https://github.com/apple/darwin-xnu/blob/xnu-7195.101.1/bsd/nfs/nfs_subs.c#L... This has performance improvements over using IP loopback. I'm wondering if this area has ever been explored or worked on? After a quick look I feel it should be an approachable task. There is already AF_VSOCK support, which sounds very similar. FWIW it looks like this was upgraded from a debug build feature around 2020-02: https://github.com/apple/darwin-xnu/blame/18c0ee9fc565c37827391eaa2f42f265a3... Thank you, Matt
Hi Matt, I don't have a particular exception to someone contributing an AF_LOCAL personality, provided it is stream-based (clts-ord, like vsock). It's hard to see it having a great deal of utility, though. The idea of vsock is to provide an actual remote nfs filesystem, that happens to use a pipe transport. It's also worth pointing out that most nfs implementations (including nfs-ganesha) probably can run into problems (e.g., livelock) doing loopback, and this would be similar. Matt On Wed, May 26, 2021 at 10:09 AM <matvore@comcast.net> wrote:
Hello,
It turns out that the macOS NFS client supports connecting to an NFS server over AF_LOCAL (UNIX domain) sockets. It is undocumented, but a hint of this is here: https://github.com/apple/darwin-xnu/blob/xnu-7195.101.1/bsd/nfs/nfs_subs.c#L...
This has performance improvements over using IP loopback.
I'm wondering if this area has ever been explored or worked on? After a quick look I feel it should be an approachable task. There is already AF_VSOCK support, which sounds very similar.
FWIW it looks like this was upgraded from a debug build feature around 2020-02: https://github.com/apple/darwin-xnu/blame/18c0ee9fc565c37827391eaa2f42f265a3...
Thank you, Matt _______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
-- Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://www.redhat.com/en/technologies/storage tel. 734-821-5101 fax. 734-769-8938 cel. 734-216-5309
On Wed, May 26, 2021 at 10:57:46AM -0400, Matt Benjamin wrote:
Hi Matt,
I don't have a particular exception to someone contributing an AF_LOCAL personality, provided it is stream-based (clts-ord, like
I think you mean cots-ord, for connection-oriented, not connection-less?
vsock). It's hard to see it having a great deal of utility, though.
Preliminary benchmarks saw a ~12% speed boost over localhost IP. This is useful for us, but understood that it is highly specialized. I should clarify that my colleagues and I (added to CC) will support our contributions for specialized functionality going forward in case needed. Most of said functionality will be related to macOS support.
The idea of vsock is to provide an actual remote nfs filesystem, that happens to use a pipe transport. It's also worth pointing out that most nfs implementations (including nfs-ganesha) probably can run into problems (e.g., livelock) doing loopback, and this would be similar.
Thank you for the heads-up. We have noticed some livelock issues when using OSXFUSE for trivial use-cases (macOS changes a certain kernel struct that needs accommodating to get thread QoS correctly set), but tests of NFS on localhost has so far shown to be stable. I'll definitely keep an eye out for this.
On Wed, May 26, 2021 at 10:09 AM <matvore@comcast.net> wrote:
Hello,
It turns out that the macOS NFS client supports connecting to an NFS server over AF_LOCAL (UNIX domain) sockets. It is undocumented, but a hint of this is here: https://github.com/apple/darwin-xnu/blob/xnu-7195.101.1/bsd/nfs/nfs_subs.c#L...
This has performance improvements over using IP loopback.
I'm wondering if this area has ever been explored or worked on? After a quick look I feel it should be an approachable task. There is already AF_VSOCK support, which sounds very similar.
FWIW it looks like this was upgraded from a debug build feature around 2020-02: https://github.com/apple/darwin-xnu/blame/18c0ee9fc565c37827391eaa2f42f265a3...
Sorry, yes, cots-ord. Matt On Wed, May 26, 2021 at 6:06 PM Matthew DeVore <matvore@comcast.net> wrote:
On Wed, May 26, 2021 at 10:57:46AM -0400, Matt Benjamin wrote:
Hi Matt,
I don't have a particular exception to someone contributing an AF_LOCAL personality, provided it is stream-based (clts-ord, like
I think you mean cots-ord, for connection-oriented, not connection-less?
vsock). It's hard to see it having a great deal of utility, though.
Preliminary benchmarks saw a ~12% speed boost over localhost IP. This is useful for us, but understood that it is highly specialized. I should clarify that my colleagues and I (added to CC) will support our contributions for specialized functionality going forward in case needed.
Most of said functionality will be related to macOS support.
The idea of vsock is to provide an actual remote nfs filesystem, that happens to use a pipe transport. It's also worth pointing out that most nfs implementations (including nfs-ganesha) probably can run into problems (e.g., livelock) doing loopback, and this would be similar.
Thank you for the heads-up. We have noticed some livelock issues when using OSXFUSE for trivial use-cases (macOS changes a certain kernel struct that needs accommodating to get thread QoS correctly set), but tests of NFS on localhost has so far shown to be stable. I'll definitely keep an eye out for this.
On Wed, May 26, 2021 at 10:09 AM <matvore@comcast.net> wrote:
Hello,
It turns out that the macOS NFS client supports connecting to an NFS server over AF_LOCAL (UNIX domain) sockets. It is undocumented, but a hint of this is here: https://github.com/apple/darwin-xnu/blob/xnu-7195.101.1/bsd/nfs/nfs_subs.c#L...
This has performance improvements over using IP loopback.
I'm wondering if this area has ever been explored or worked on? After a quick look I feel it should be an approachable task. There is already AF_VSOCK support, which sounds very similar.
FWIW it looks like this was upgraded from a debug build feature around 2020-02: https://github.com/apple/darwin-xnu/blame/18c0ee9fc565c37827391eaa2f42f265a3...
-- Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://www.redhat.com/en/technologies/storage tel. 734-821-5101 fax. 734-769-8938 cel. 734-216-5309
On 5/26/21 6:06 PM, Matthew DeVore wrote:
On Wed, May 26, 2021 at 10:57:46AM -0400, Matt Benjamin wrote:
Hi Matt,
I don't have a particular exception to someone contributing an AF_LOCAL personality, provided it is stream-based (clts-ord, like
I think you mean cots-ord, for connection-oriented, not connection-less?
vsock). It's hard to see it having a great deal of utility, though.
Preliminary benchmarks saw a ~12% speed boost over localhost IP. This is useful for us, but understood that it is highly specialized. I should clarify that my colleagues and I (added to CC) will support our contributions for specialized functionality going forward in case needed.
This is interesting, and may very well be worth a prototype to check, but keep in mind that Linux will likely have very different performance characteristics for both IP loopback and for AF_LOCAL, so it may be that it's not faster on Linux. Daniel
participants (4)
-
Daniel Gryniewicz -
Matt Benjamin -
Matthew DeVore -
matvore＠comcast.net