Glad to hear it. The v3 proxy will automatically respect the maxread / maxwrite settings reported by the backend, so you don't *have* to set them: https://github.com/nfs-ganesha/nfs-ganesha/blob/ef07e96e25e2b8521311d594b6bd... (But I made that a WARN, so maybe it's a bit noisy). On Wed, Nov 3, 2021 at 11:15 AM Matthew Hess <mxhess@gmail.com> wrote:
I pasted my current config below. I just turned on MDCACHE and set up PROXY_V3 to align r and w sizes with the clients and other settings we use just in case I was causing an issue. I also set num_sockets to something crazy large. I've gotten much further in my testing now since these changes.
fstab nfs mount opts atm: nofail,vers=4,proto=tcp,nolock,sync,noatime,nodiratime,noac,lookupcache=none
I also went crazy and set: LimitAS=infinity LimitRSS=infinity LimitCORE=infinity LimitNOFILE=infinity LimitNPROC=infinity LimitMEMLOCK=infinity LimitLOCKS=infinity LimitSIGPENDING=infinity
in the unit config. Will try to make this more rational as I fine-tune.
## These are core parameters that affect Ganesha as a whole. NFS_CORE_PARAM { ## Allow NFSv3 to mount paths with the Pseudo path, the same as NFSv4, ## instead of using the physical paths. mount_path_pseudo = true;
## Configure the protocols that Ganesha will listen for. This is a hard ## limit, as this list determines which sockets are opened. This list ## can be restricted per export, but cannot be expanded. Protocols = 3,4; }
## These are defaults for exports. They can be overridden per-export. EXPORT_DEFAULTS { ## Access type for clients. Default is None, so some access must be ## given either here or in the export itself. Access_Type = RW; }
## Configure settings for the object handle cache MDCACHE { ## The point at which object cache entries will start being reused. Entries_HWMark = 100000; }
## Configure logging. Default is to log to Syslog. Basic logging can also be ## configured from the command line LOG { ## Default log level for all components Default_Log_Level = WARN;
## Configure per-component log levels. Components { FSAL = INFO; NFS4 = EVENT; }
## Where to log Facility { name = FILE; destination = "/var/log/ganesha/ganesha.log"; enable = active; } }
EXPORT { Export_Id = 2345; Path = "/prod"; Pseudo = "/prod";
Access_Type = RW; Squash = No_Root_Squash; SecType = "sys"; Protocols = 3,4; Transports = "TCP"; MaxRead = 65536; MaxWrite = 65536;
FSAL { Name = proxy_v3; Srv_Addr = <netapp ip>; }
}
PROXY_V3 { # How many sockets to keep open in the connection pool. num_sockets = 512; maxread = 65536; maxwrite = 65536; }
On Wed, Nov 3, 2021 at 12:01 PM Solomon Boulos <boulos@waymo.com> wrote:
The fds for the proxy_v3 code *must* be from the privileged port range (0 to 1024) because ~all NFSv3 servers require "privileged" clients. But it should also only be opening a few. Num sockets defaults to 32:
https://github.com/nfs-ganesha/nfs-ganesha/blob/ef07e96e25e2b8521311d594b6bd...
what does your proxy config look like?
The proxy has okay logging as well, if you want to add a:
``` LOG { COMPONENTS { FSAL = FULL_DEBUG; NFSPROTO = DEBUG; } } ```
stanza to your config.
On Wed, Nov 3, 2021 at 10:38 AM Matthew Hess <mxhess@gmail.com> wrote:
I've set:
LimitAS=infinity LimitRSS=infinity LimitCORE=infinity LimitNOFILE=infinity
in /usr/lib/systemd/system/nfs-ganesha.service and reloaded it then restarted the service but I keep hitting that error when writing large files.
I'm using last night's next branch and built rpms from that.
_______________________________________________ Support mailing list -- support@lists.nfs-ganesha.org To unsubscribe send an email to support-leave@lists.nfs-ganesha.org