[NFS-Ganesha-Devel] Re: Scalability issue with VFS FSAL and large amounts of read i/o in flight

On 9/20/18 7:27 AM, Malahal Naineni wrote: Olga suggested trying V4.1. (She had also asked for the mount parameters.) So I passed it along. DanG told me the rest. Now we know why GPFS needs 4K threads and 200GB RAM. But the NFS protocol is telling you those sizes are too big. They'll only work well after you increase the system and socket buffer sizes, and feed the correctly sized data accordingly. Otherwise, you just have a huge amount of bufferbloat. Anyway, talked with Bruce about what knfsd is doing. He says that they used to enforce sizing and reject piggy clients, but there were too many badly behaved operators. So they took the checks out. You are free to self-denial-of-service. The ideal is to have just enough data arrive just in time to the queue, on one CPU that is closest feed to the interface. That will give the best performance. We've been doing Fair Queuing for 30+ years. I'm old; I remember when John proposed it. :) Whereas I learned about Weighted Fair Queuing from Fred Baker in the early '90s, when he was implementing it for ACC. I was part of the bufferbloat project, and one of the earliest testers of fq_codel. That's the best practice, the current state of the art. Look, I've mentioned that I was overridden on this code design. The evidence is in the list archives. You have shipping product. It needs to work in the field. So let's massage your patch a little, and get stuff working better. We can argue another day about the best design. Bake-a-thon is about ensuring things work!