6:47 p.m.
Our test programs are very tightly coupled to our test environment. We'll try to see
if we can eliminate the dependencies and get something useful
Meanwhile here's complete details on our workload.
Pre condition:
Create a subdirectory with 2000 subdirectories and 2500 1K files in each subdir for a
total of 5 million files.
One thread doing a python (os.walk) readdir of the entire share (pre-populated subdir as
well as the 2nd subdir where files are being created, deleted, recreated). This thread
continues for the duration of test.
One test run is 6 hours.
N files/dir, M dirs, S file size
Files to create are submitted to the thread pool in the following order:
Dir1/file1, Dir2/file1, DirM/file1, Dir1/file2, Dir2/file2..
5 threads will pull work from the thread pool and create files.
Once the writes are complete, one thread will read the files, verify MD5 and then delete
them. The process repeats. N,M,S stay constant for the test run.
The names of the files in the 2nd iteration are the same as the first iteration.
After 6 hours the test run is repeated with a different set of N,M,S and the readdir
thread is restarted.
N is one of 10, 50, 100, 500, 1000
M is from 1 to 10
S is one if 1K, 5K, 10K, 50K, 100K, 500K, 1M, 5M, 10M, 50M
One other thought:
The new code in cih_get_by_key_latch will now return not found even though it is still in
the hash with a refcount of 0. mdcache_new_entry will go through oentry=NULL and will add
the new entry to the hash (avltree?). Can this result in collisions between the oentry
that still exists and the new entry being added?