Glad to hear it. The v3 proxy will automatically respect the maxread / maxwrite settings reported by the backend, so you don't *have* to set them:

 https://github.com/nfs-ganesha/nfs-ganesha/blob/ef07e96e25e2b8521311d594b6bdfb600c05c2f0/src/FSAL/FSAL_PROXY_V3/main.c#L2547

(But I made that a WARN, so maybe it's a bit noisy).

On Wed, Nov 3, 2021 at 11:15 AM Matthew Hess <mxhess@gmail.com> wrote:
I pasted my current config below. I just turned on MDCACHE and set up PROXY_V3 to align r and w sizes with the clients and other settings we use just in case I was causing an issue. I also set num_sockets to something crazy large. I've gotten much further in my testing now since these changes.

fstab nfs mount opts atm: nofail,vers=4,proto=tcp,nolock,sync,noatime,nodiratime,noac,lookupcache=none

I also went crazy and set:
LimitAS=infinity
LimitRSS=infinity
LimitCORE=infinity
LimitNOFILE=infinity
LimitNPROC=infinity
LimitMEMLOCK=infinity
LimitLOCKS=infinity
LimitSIGPENDING=infinity

in the unit config. Will try to make this more rational as I fine-tune.

## These are core parameters that affect Ganesha as a whole.
NFS_CORE_PARAM {
        ## Allow NFSv3 to mount paths with the Pseudo path, the same as NFSv4,
        ## instead of using the physical paths.
        mount_path_pseudo = true;

        ## Configure the protocols that Ganesha will listen for.  This is a hard
        ## limit, as this list determines which sockets are opened.  This list
        ## can be restricted per export, but cannot be expanded.
        Protocols = 3,4;
}

## These are defaults for exports.  They can be overridden per-export.
EXPORT_DEFAULTS {
        ## Access type for clients.  Default is None, so some access must be
        ## given either here or in the export itself.
        Access_Type = RW;
}

## Configure settings for the object handle cache
MDCACHE {
        ## The point at which object cache entries will start being reused.
        Entries_HWMark = 100000;
}

## Configure logging.  Default is to log to Syslog.  Basic logging can also be
## configured from the command line
LOG {
        ## Default log level for all components
        Default_Log_Level = WARN;

        ## Configure per-component log levels.
        Components {
                FSAL = INFO;
                NFS4 = EVENT;
        }

        ## Where to log
        Facility {
                name = FILE;
                destination = "/var/log/ganesha/ganesha.log";
                enable = active;
        }
}

EXPORT {
  Export_Id = 2345;
  Path = "/prod";
  Pseudo = "/prod";

  Access_Type = RW;
  Squash = No_Root_Squash;
  SecType = "sys";
  Protocols = 3,4;
  Transports = "TCP";
  MaxRead = 65536;
  MaxWrite = 65536;

  FSAL {
    Name = proxy_v3;
    Srv_Addr = <netapp ip>;
  }

}

PROXY_V3 {
        # How many sockets to keep open in the connection pool.
        num_sockets = 512;
        maxread = 65536;
        maxwrite = 65536;
}


On Wed, Nov 3, 2021 at 12:01 PM Solomon Boulos <boulos@waymo.com> wrote:
The fds for the proxy_v3 code *must* be from the privileged port range (0 to 1024) because ~all NFSv3 servers require "privileged" clients. But it should also only be opening a few. Num sockets defaults to 32:


what does your proxy config look like?

The proxy has okay logging as well, if you want to add a:

```
LOG {
  COMPONENTS {
    FSAL = FULL_DEBUG;
    NFSPROTO = DEBUG;
  }
}
```

stanza to your config.

On Wed, Nov 3, 2021 at 10:38 AM Matthew Hess <mxhess@gmail.com> wrote:
I've set:

LimitAS=infinity
LimitRSS=infinity
LimitCORE=infinity
LimitNOFILE=infinity

in /usr/lib/systemd/system/nfs-ganesha.service and reloaded it then restarted the service but I keep hitting that error when writing large files.

I'm using last night's next branch and built rpms from that.

_______________________________________________
Support mailing list -- support@lists.nfs-ganesha.org
To unsubscribe send an email to support-leave@lists.nfs-ganesha.org