I pasted my current config below. I just turned on MDCACHE and set up PROXY_V3 to align r and w sizes with the clients and other settings we use just in case I was causing an issue. I also set num_sockets to something crazy large. I've gotten much further in my testing now since these changes.
fstab nfs mount opts atm: nofail,vers=4,proto=tcp,nolock,sync,noatime,nodiratime,noac,lookupcache=none
I also went crazy and set:
LimitAS=infinity
LimitRSS=infinity
LimitCORE=infinity
LimitNOFILE=infinity
LimitNPROC=infinity
LimitMEMLOCK=infinity
LimitLOCKS=infinity
LimitSIGPENDING=infinity
in the unit config. Will try to make this more rational as I fine-tune.
## These are core parameters that affect Ganesha as a whole.
NFS_CORE_PARAM {
## Allow NFSv3 to mount paths with the Pseudo path, the same as NFSv4,
## instead of using the physical paths.
mount_path_pseudo = true;
## Configure the protocols that Ganesha will listen for. This is a hard
## limit, as this list determines which sockets are opened. This list
## can be restricted per export, but cannot be expanded.
Protocols = 3,4;
}
## These are defaults for exports. They can be overridden per-export.
EXPORT_DEFAULTS {
## Access type for clients. Default is None, so some access must be
## given either here or in the export itself.
Access_Type = RW;
}
## Configure settings for the object handle cache
MDCACHE {
## The point at which object cache entries will start being reused.
Entries_HWMark = 100000;
}
## Configure logging. Default is to log to Syslog. Basic logging can also be
## configured from the command line
LOG {
## Default log level for all components
Default_Log_Level = WARN;
## Configure per-component log levels.
Components {
FSAL = INFO;
NFS4 = EVENT;
}
## Where to log
Facility {
name = FILE;
destination = "/var/log/ganesha/ganesha.log";
enable = active;
}
}
EXPORT {
Export_Id = 2345;
Path = "/prod";
Pseudo = "/prod";
Access_Type = RW;
Squash = No_Root_Squash;
SecType = "sys";
Protocols = 3,4;
Transports = "TCP";
MaxRead = 65536;
MaxWrite = 65536;
FSAL {
Name = proxy_v3;
Srv_Addr = <netapp ip>;
}
}
PROXY_V3 {
# How many sockets to keep open in the connection pool.
num_sockets = 512;
maxread = 65536;
maxwrite = 65536;
}