[NFS-Ganesha-Support] Re: proxyv3_getfd_blocking :FSAL :CRIT :Failed to ever acquire a new fd, dying

I pasted my current config below. I just turned on MDCACHE and set up PROXY_V3 to align r and w sizes with the clients and other settings we use just in case I was causing an issue. I also set num_sockets to something crazy large. I've gotten much further in my testing now since these changes. fstab nfs mount opts atm: nofail,vers=4,proto=tcp,nolock,sync,noatime,nodiratime,noac,lookupcache=none I also went crazy and set: LimitAS=infinity LimitRSS=infinity LimitCORE=infinity LimitNOFILE=infinity LimitNPROC=infinity LimitMEMLOCK=infinity LimitLOCKS=infinity LimitSIGPENDING=infinity in the unit config. Will try to make this more rational as I fine-tune. ## These are core parameters that affect Ganesha as a whole. NFS_CORE_PARAM { ## Allow NFSv3 to mount paths with the Pseudo path, the same as NFSv4, ## instead of using the physical paths. mount_path_pseudo = true; ## Configure the protocols that Ganesha will listen for. This is a hard ## limit, as this list determines which sockets are opened. This list ## can be restricted per export, but cannot be expanded. Protocols = 3,4; } ## These are defaults for exports. They can be overridden per-export. EXPORT_DEFAULTS { ## Access type for clients. Default is None, so some access must be ## given either here or in the export itself. Access_Type = RW; } ## Configure settings for the object handle cache MDCACHE { ## The point at which object cache entries will start being reused. Entries_HWMark = 100000; } ## Configure logging. Default is to log to Syslog. Basic logging can also be ## configured from the command line LOG { ## Default log level for all components Default_Log_Level = WARN; ## Configure per-component log levels. Components { FSAL = INFO; NFS4 = EVENT; } ## Where to log Facility { name = FILE; destination = "/var/log/ganesha/ganesha.log"; enable = active; } } EXPORT { Export_Id = 2345; Path = "/prod"; Pseudo = "/prod"; Access_Type = RW; Squash = No_Root_Squash; SecType = "sys"; Protocols = 3,4; Transports = "TCP"; MaxRead = 65536; MaxWrite = 65536; FSAL { Name = proxy_v3; Srv_Addr = <netapp ip>; } } PROXY_V3 { # How many sockets to keep open in the connection pool. num_sockets = 512; maxread = 65536; maxwrite = 65536; } On Wed, Nov 3, 2021 at 12:01 PM Solomon Boulos <boulos(a)waymo.com&gt; wrote: > The fds for the proxy_v3 code *must* be from the privileged port range (0 > to 1024) because ~all NFSv3 servers require "privileged" clients. But it > should also only be opening a few. Num sockets defaults to 32: > > > https://github.com/nfs-ganesha/nfs-ganesha/blob/ef07e96e25e2b8521311d594b... > > what does your proxy config look like? > > The proxy has okay logging as well, if you want to add a: > > ``` > LOG { > COMPONENTS { > FSAL = FULL_DEBUG; > NFSPROTO = DEBUG; > } > } > ``` > > stanza to your config. > > On Wed, Nov 3, 2021 at 10:38 AM Matthew Hess <mxhess(a)gmail.com&gt; wrote: > >> I've set: >> >> LimitAS=infinity >> LimitRSS=infinity >> LimitCORE=infinity >> LimitNOFILE=infinity >> >> in /usr/lib/systemd/system/nfs-ganesha.service and reloaded it then >> restarted the service but I keep hitting that error when writing large >> files. >> >> I'm using last night's next branch and built rpms from that. >> >> _______________________________________________ >> Support mailing list -- support(a)lists.nfs-ganesha.org >> To unsubscribe send an email to support-leave(a)lists.nfs-ganesha.org >> >