read-only open causing file to be truncated with NFSv4/ganesha.
Hello, I'm seeing an issue with a specific application flow and with ganesha server (2.6.5). Here is what the application does: - fd1 = open (file1, O_WRONLY|O_CREAT|O_TRUNC) Ganesha creates a state and stores the flags in fd->openflags (see vfs_open2()) - write (fd1, ....) - fd2 = open(file1, O_RDONLY) Ganesha finds the state for the first open, finds the flags and calls fsal_reopen2() with old and new flags OR'd together. This causes the file to be truncated because the original open was done with O_TRUNC. I don't see this behavior with kernel NFS or a local filesystem. Any suggestions on fixing this? Here is a quick and dirty program to reproduce it - you can see that the second stat prints zero as size with a mount from Ganesha server. #include <stdio.h> #include <stdlib.h> #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <sys/stat.h> #include <unistd.h> int main(int argc, char **argv) { char *fn = argv[1]; int fd; char buf[1024]; struct stat stbuf; int rc; fd = open(fn, O_WRONLY|O_CREAT|O_TRUNC, 0666); printf("open(%s) = %d\n", fn, fd); ssize_t ret; ret = write(fd, buf, sizeof(buf)); printf("write returned %ld\n", ret); rc = stat(fn, &stbuf); printf("size: %ld\n", stbuf.st_size); int fd1; fd1 = open(fn, O_RDONLY); printf("open(%s) = %d\n", fn, fd1); rc = stat(fn, &stbuf); printf("size: %ld\n", stbuf.st_size); } Thanks, Pradeep
This seems like a bug. My guess is that either status2() shouldn't return O_TRUNC, or it shouldn't be saved in the fd to begin with. O_TRUNC is a transitory flag, only useful at the time of the open itself. Fixing the common status2() is simplest; otherwise we need to fix all the FSALs. Daniel On 10/12/2018 08:12 PM, Pradeep wrote:
Hello,
I'm seeing an issue with a specific application flow and with ganesha server (2.6.5). Here is what the application does:
- fd1 = open (file1, O_WRONLY|O_CREAT|O_TRUNC) Ganesha creates a state and stores the flags in fd->openflags (see vfs_open2()) - write (fd1, ....) - fd2 = open(file1, O_RDONLY) Ganesha finds the state for the first open, finds the flags and calls fsal_reopen2() with old and new flags OR'd together. This causes the file to be truncated because the original open was done with O_TRUNC.
I don't see this behavior with kernel NFS or a local filesystem. Any suggestions on fixing this? Here is a quick and dirty program to reproduce it - you can see that the second stat prints zero as size with a mount from Ganesha server.
#include <stdio.h> #include <stdlib.h> #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <sys/stat.h> #include <unistd.h>
int main(int argc, char **argv) { char *fn = argv[1]; int fd; char buf[1024]; struct stat stbuf; int rc;
fd = open(fn, O_WRONLY|O_CREAT|O_TRUNC, 0666); printf("open(%s) = %d\n", fn, fd);
ssize_t ret; ret = write(fd, buf, sizeof(buf)); printf("write returned %ld\n", ret);
rc = stat(fn, &stbuf); printf("size: %ld\n", stbuf.st_size);
int fd1; fd1 = open(fn, O_RDONLY); printf("open(%s) = %d\n", fn, fd1);
rc = stat(fn, &stbuf); printf("size: %ld\n", stbuf.st_size); }
Thanks, Pradeep
_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
I have uploaded a patch to fix this ( https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/430435) But I have a question on this. As per the code, the O_TRUNC gets applied (via open_flags) only when it is file create. If the file pre-exists, O_TRUNC is not applied to open_flags. Was wondering, why we need O_TRUNC while creating new file, anyway it will be of size 0 ? As per my little understanding the O_TRUNC should be applicable if we are opening a pre-existing file. Correct me if I am wrong. On Mon, Oct 15, 2018 at 6:32 PM Daniel Gryniewicz <dang@redhat.com> wrote:
This seems like a bug. My guess is that either status2() shouldn't return O_TRUNC, or it shouldn't be saved in the fd to begin with. O_TRUNC is a transitory flag, only useful at the time of the open itself.
Fixing the common status2() is simplest; otherwise we need to fix all the FSALs.
Daniel
On 10/12/2018 08:12 PM, Pradeep wrote:
Hello,
I'm seeing an issue with a specific application flow and with ganesha server (2.6.5). Here is what the application does:
- fd1 = open (file1, O_WRONLY|O_CREAT|O_TRUNC) Ganesha creates a state and stores the flags in fd->openflags (see vfs_open2()) - write (fd1, ....) - fd2 = open(file1, O_RDONLY) Ganesha finds the state for the first open, finds the flags and calls fsal_reopen2() with old and new flags OR'd together. This causes the file to be truncated because the original open was done with O_TRUNC.
I don't see this behavior with kernel NFS or a local filesystem. Any suggestions on fixing this? Here is a quick and dirty program to reproduce it - you can see that the second stat prints zero as size with a mount from Ganesha server.
#include <stdio.h> #include <stdlib.h> #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <sys/stat.h> #include <unistd.h>
int main(int argc, char **argv) { char *fn = argv[1]; int fd; char buf[1024]; struct stat stbuf; int rc;
fd = open(fn, O_WRONLY|O_CREAT|O_TRUNC, 0666); printf("open(%s) = %d\n", fn, fd);
ssize_t ret; ret = write(fd, buf, sizeof(buf)); printf("write returned %ld\n", ret);
rc = stat(fn, &stbuf); printf("size: %ld\n", stbuf.st_size);
int fd1; fd1 = open(fn, O_RDONLY); printf("open(%s) = %d\n", fn, fd1);
rc = stat(fn, &stbuf); printf("size: %ld\n", stbuf.st_size); }
Thanks, Pradeep
_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org
-- with regards, Sachin Punadikar
Where are you seeing this? It looks to me like FSAL_O_TRUNC is applied on all opens. Daniel On 10/23/2018 06:42 AM, Sachin Punadikar wrote:
I have uploaded a patch to fix this (https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/430435) But I have a question on this. As per the code, the O_TRUNC gets applied (via open_flags) only when it is file create. If the file pre-exists, O_TRUNC is not applied to open_flags. Was wondering, why we need O_TRUNC while creating new file, anyway it will be of size 0 ? As per my little understanding the O_TRUNC should be applicable if we are opening a pre-existing file. Correct me if I am wrong.
On Mon, Oct 15, 2018 at 6:32 PM Daniel Gryniewicz <dang@redhat.com <mailto:dang@redhat.com>> wrote:
This seems like a bug. My guess is that either status2() shouldn't return O_TRUNC, or it shouldn't be saved in the fd to begin with. O_TRUNC is a transitory flag, only useful at the time of the open itself.
Fixing the common status2() is simplest; otherwise we need to fix all the FSALs.
Daniel
On 10/12/2018 08:12 PM, Pradeep wrote: > Hello, > > I'm seeing an issue with a specific application flow and with ganesha > server (2.6.5). Here is what the application does: > > - fd1 = open (file1, O_WRONLY|O_CREAT|O_TRUNC) > Ganesha creates a state and stores the flags in fd->openflags (see > vfs_open2()) > - write (fd1, ....) > - fd2 = open(file1, O_RDONLY) > Ganesha finds the state for the first open, finds the flags and calls > fsal_reopen2() with old and new flags OR'd together. This causes the > file to be truncated because the original open was done with O_TRUNC. > > I don't see this behavior with kernel NFS or a local filesystem. Any > suggestions on fixing this? Here is a quick and dirty program to > reproduce it - you can see that the second stat prints zero as size with > a mount from Ganesha server. > > #include <stdio.h> > #include <stdlib.h> > #include <sys/types.h> > #include <sys/stat.h> > #include <fcntl.h> > #include <sys/stat.h> > #include <unistd.h> > > int main(int argc, char **argv) > { > char *fn = argv[1]; > int fd; > char buf[1024]; > struct stat stbuf; > int rc; > > > fd = open(fn, O_WRONLY|O_CREAT|O_TRUNC, 0666); > printf("open(%s) = %d\n", fn, fd); > > ssize_t ret; > ret = write(fd, buf, sizeof(buf)); > printf("write returned %ld\n", ret); > > rc = stat(fn, &stbuf); > printf("size: %ld\n", stbuf.st_size); > > int fd1; > fd1 = open(fn, O_RDONLY); > printf("open(%s) = %d\n", fn, fd1); > > rc = stat(fn, &stbuf); > printf("size: %ld\n", stbuf.st_size); > } > > Thanks, > Pradeep > > > > _______________________________________________ > Devel mailing list -- devel@lists.nfs-ganesha.org <mailto:devel@lists.nfs-ganesha.org> > To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org <mailto:devel-leave@lists.nfs-ganesha.org> > _______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org <mailto:devel@lists.nfs-ganesha.org> To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org <mailto:devel-leave@lists.nfs-ganesha.org>
-- with regards, Sachin Punadikar
Hi Daniel, Here is the code snippet from fsal_open2 function: * /* Handle attribute size = 0 here, normalize to FSAL_O_TRUNC* * * instead of setting ATTR_SIZE.* * */* * if (attr != NULL &&* * FSAL_TEST_MASK(attr->valid_mask, ATTR_SIZE) &&* * attr->filesize == 0) {* * LogFullDebug(COMPONENT_FSAL, "Truncate");* * /* Handle truncate to zero on open */* * openflags |= FSAL_O_TRUNC;* * /* Don't set the size if we later set the attributes */* * FSAL_UNSET_MASK(attr->valid_mask, ATTR_SIZE);* * }* If I enable full debug, I could see "Truncate" being logged only when the file is new. For existing files, "Truncate" log entry was never seen. Hence I mentioned that it gets applied to only new files. On Tue, Oct 23, 2018 at 7:27 PM Daniel Gryniewicz <dang@redhat.com> wrote:
Where are you seeing this? It looks to me like FSAL_O_TRUNC is applied on all opens.
Daniel
I have uploaded a patch to fix this (https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/430435) But I have a question on this. As per the code, the O_TRUNC gets applied (via open_flags) only when it is file create. If the file pre-exists, O_TRUNC is not applied to open_flags. Was wondering, why we need O_TRUNC while creating new file, anyway it will be of size 0 ? As per my little understanding the O_TRUNC should be applicable if we are opening a pre-existing file. Correct me if I am wrong.
On Mon, Oct 15, 2018 at 6:32 PM Daniel Gryniewicz <dang@redhat.com <mailto:dang@redhat.com>> wrote:
This seems like a bug. My guess is that either status2() shouldn't return O_TRUNC, or it shouldn't be saved in the fd to begin with. O_TRUNC is a transitory flag, only useful at the time of the open itself.
Fixing the common status2() is simplest; otherwise we need to fix all the FSALs.
Daniel
On 10/12/2018 08:12 PM, Pradeep wrote: > Hello, > > I'm seeing an issue with a specific application flow and with ganesha > server (2.6.5). Here is what the application does: > > - fd1 = open (file1, O_WRONLY|O_CREAT|O_TRUNC) > Ganesha creates a state and stores the flags in fd->openflags (see > vfs_open2()) > - write (fd1, ....) > - fd2 = open(file1, O_RDONLY) > Ganesha finds the state for the first open, finds the flags and calls > fsal_reopen2() with old and new flags OR'd together. This causes
On 10/23/2018 06:42 AM, Sachin Punadikar wrote: the
> file to be truncated because the original open was done with
O_TRUNC.
> > I don't see this behavior with kernel NFS or a local filesystem.
Any
> suggestions on fixing this? Here is a quick and dirty program to > reproduce it - you can see that the second stat prints zero as size with > a mount from Ganesha server. > > #include <stdio.h> > #include <stdlib.h> > #include <sys/types.h> > #include <sys/stat.h> > #include <fcntl.h> > #include <sys/stat.h> > #include <unistd.h> > > int main(int argc, char **argv) > { > char *fn = argv[1]; > int fd; > char buf[1024]; > struct stat stbuf; > int rc; > > > fd = open(fn, O_WRONLY|O_CREAT|O_TRUNC, 0666); > printf("open(%s) = %d\n", fn, fd); > > ssize_t ret; > ret = write(fd, buf, sizeof(buf)); > printf("write returned %ld\n", ret); > > rc = stat(fn, &stbuf); > printf("size: %ld\n", stbuf.st_size); > > int fd1; > fd1 = open(fn, O_RDONLY); > printf("open(%s) = %d\n", fn, fd1); > > rc = stat(fn, &stbuf); > printf("size: %ld\n", stbuf.st_size); > } > > Thanks, > Pradeep > > > > _______________________________________________ > Devel mailing list -- devel@lists.nfs-ganesha.org <mailto:devel@lists.nfs-ganesha.org> > To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org <mailto:devel-leave@lists.nfs-ganesha.org> > _______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org <mailto:devel@lists.nfs-ganesha.org> To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org <mailto:devel-leave@lists.nfs-ganesha.org>
-- with regards, Sachin Punadikar
-- with regards, Sachin Punadikar
You may not have done anything from client that does: fd = open(“foo”, O_RDWR + O_CREAT + O_TRUNC, 0666) Try echo foo > /mnt/foo from the client, and then run it again (so you get both the create case and the already exists case. Frank From: Sachin Punadikar [mailto:punadikar.sachin@gmail.com] Sent: Wednesday, October 24, 2018 4:07 AM To: Daniel Gryniewicz <dang@redhat.com> Cc: devel@lists.nfs-ganesha.org Subject: [NFS-Ganesha-Devel] Re: read-only open causing file to be truncated with NFSv4/ganesha. Hi Daniel, Here is the code snippet from fsal_open2 function: /* Handle attribute size = 0 here, normalize to FSAL_O_TRUNC * instead of setting ATTR_SIZE. */ if (attr != NULL && FSAL_TEST_MASK(attr->valid_mask, ATTR_SIZE) && attr->filesize == 0) { LogFullDebug(COMPONENT_FSAL, "Truncate"); /* Handle truncate to zero on open */ openflags |= FSAL_O_TRUNC; /* Don't set the size if we later set the attributes */ FSAL_UNSET_MASK(attr->valid_mask, ATTR_SIZE); } If I enable full debug, I could see "Truncate" being logged only when the file is new. For existing files, "Truncate" log entry was never seen. Hence I mentioned that it gets applied to only new files. On Tue, Oct 23, 2018 at 7:27 PM Daniel Gryniewicz <dang@redhat.com <mailto:dang@redhat.com> > wrote: Where are you seeing this? It looks to me like FSAL_O_TRUNC is applied on all opens. Daniel On 10/23/2018 06:42 AM, Sachin Punadikar wrote:
I have uploaded a patch to fix this (https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/430435) But I have a question on this. As per the code, the O_TRUNC gets applied (via open_flags) only when it is file create. If the file pre-exists, O_TRUNC is not applied to open_flags. Was wondering, why we need O_TRUNC while creating new file, anyway it will be of size 0 ? As per my little understanding the O_TRUNC should be applicable if we are opening a pre-existing file. Correct me if I am wrong.
On Mon, Oct 15, 2018 at 6:32 PM Daniel Gryniewicz <dang@redhat.com <mailto:dang@redhat.com> <mailto:dang@redhat.com <mailto:dang@redhat.com> >> wrote:
This seems like a bug. My guess is that either status2() shouldn't return O_TRUNC, or it shouldn't be saved in the fd to begin with. O_TRUNC is a transitory flag, only useful at the time of the open itself.
Fixing the common status2() is simplest; otherwise we need to fix all the FSALs.
Daniel
On 10/12/2018 08:12 PM, Pradeep wrote: > Hello, > > I'm seeing an issue with a specific application flow and with ganesha > server (2.6.5). Here is what the application does: > > - fd1 = open (file1, O_WRONLY|O_CREAT|O_TRUNC) > Ganesha creates a state and stores the flags in fd->openflags (see > vfs_open2()) > - write (fd1, ....) > - fd2 = open(file1, O_RDONLY) > Ganesha finds the state for the first open, finds the flags and calls > fsal_reopen2() with old and new flags OR'd together. This causes the > file to be truncated because the original open was done with O_TRUNC. > > I don't see this behavior with kernel NFS or a local filesystem. Any > suggestions on fixing this? Here is a quick and dirty program to > reproduce it - you can see that the second stat prints zero as size with > a mount from Ganesha server. > > #include <stdio.h> > #include <stdlib.h> > #include <sys/types.h> > #include <sys/stat.h> > #include <fcntl.h> > #include <sys/stat.h> > #include <unistd.h> > > int main(int argc, char **argv) > { > char *fn = argv[1]; > int fd; > char buf[1024]; > struct stat stbuf; > int rc; > > > fd = open(fn, O_WRONLY|O_CREAT|O_TRUNC, 0666); > printf("open(%s) = %d\n", fn, fd); > > ssize_t ret; > ret = write(fd, buf, sizeof(buf)); > printf("write returned %ld\n", ret); > > rc = stat(fn, &stbuf); > printf("size: %ld\n", stbuf.st_size); > > int fd1; > fd1 = open(fn, O_RDONLY); > printf("open(%s) = %d\n", fn, fd1); > > rc = stat(fn, &stbuf); > printf("size: %ld\n", stbuf.st_size); > } > > Thanks, > Pradeep > > > > _______________________________________________ > Devel mailing list -- devel@lists.nfs-ganesha.org <mailto:devel@lists.nfs-ganesha.org> <mailto:devel@lists.nfs-ganesha.org <mailto:devel@lists.nfs-ganesha.org> > > To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org <mailto:devel-leave@lists.nfs-ganesha.org> <mailto:devel-leave@lists.nfs-ganesha.org <mailto:devel-leave@lists.nfs-ganesha.org> > > _______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org <mailto:devel@lists.nfs-ganesha.org> <mailto:devel@lists.nfs-ganesha.org <mailto:devel@lists.nfs-ganesha.org> > To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org <mailto:devel-leave@lists.nfs-ganesha.org> <mailto:devel-leave@lists.nfs-ganesha.org <mailto:devel-leave@lists.nfs-ganesha.org> >
-- with regards, Sachin Punadikar
-- with regards, Sachin Punadikar
We do need O_TRUNC on a non-exclusive create even if we think the file doesn’t exist because someone else might open it and write data to it. Of course that’s a bit dangerous… We should not be saving O_TRUNC in the open flags for the fd though… That would be the best fix, to remove it from the open flags and only store the mode opened, then the flag isn’t there for status2 to return. Frank From: Sachin Punadikar [mailto:punadikar.sachin@gmail.com] Sent: Tuesday, October 23, 2018 3:43 AM To: Daniel Gryniewicz <dang@redhat.com> Cc: devel@lists.nfs-ganesha.org Subject: [NFS-Ganesha-Devel] Re: read-only open causing file to be truncated with NFSv4/ganesha. I have uploaded a patch to fix this (https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/430435) But I have a question on this. As per the code, the O_TRUNC gets applied (via open_flags) only when it is file create. If the file pre-exists, O_TRUNC is not applied to open_flags. Was wondering, why we need O_TRUNC while creating new file, anyway it will be of size 0 ? As per my little understanding the O_TRUNC should be applicable if we are opening a pre-existing file. Correct me if I am wrong. On Mon, Oct 15, 2018 at 6:32 PM Daniel Gryniewicz <dang@redhat.com <mailto:dang@redhat.com> > wrote: This seems like a bug. My guess is that either status2() shouldn't return O_TRUNC, or it shouldn't be saved in the fd to begin with. O_TRUNC is a transitory flag, only useful at the time of the open itself. Fixing the common status2() is simplest; otherwise we need to fix all the FSALs. Daniel On 10/12/2018 08:12 PM, Pradeep wrote:
Hello,
I'm seeing an issue with a specific application flow and with ganesha server (2.6.5). Here is what the application does:
- fd1 = open (file1, O_WRONLY|O_CREAT|O_TRUNC) Ganesha creates a state and stores the flags in fd->openflags (see vfs_open2()) - write (fd1, ....) - fd2 = open(file1, O_RDONLY) Ganesha finds the state for the first open, finds the flags and calls fsal_reopen2() with old and new flags OR'd together. This causes the file to be truncated because the original open was done with O_TRUNC.
I don't see this behavior with kernel NFS or a local filesystem. Any suggestions on fixing this? Here is a quick and dirty program to reproduce it - you can see that the second stat prints zero as size with a mount from Ganesha server.
#include <stdio.h> #include <stdlib.h> #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <sys/stat.h> #include <unistd.h>
int main(int argc, char **argv) { char *fn = argv[1]; int fd; char buf[1024]; struct stat stbuf; int rc;
fd = open(fn, O_WRONLY|O_CREAT|O_TRUNC, 0666); printf("open(%s) = %d\n", fn, fd);
ssize_t ret; ret = write(fd, buf, sizeof(buf)); printf("write returned %ld\n", ret);
rc = stat(fn, &stbuf); printf("size: %ld\n", stbuf.st_size);
int fd1; fd1 = open(fn, O_RDONLY); printf("open(%s) = %d\n", fn, fd1);
rc = stat(fn, &stbuf); printf("size: %ld\n", stbuf.st_size); }
Thanks, Pradeep
_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org <mailto:devel@lists.nfs-ganesha.org> To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org <mailto:devel-leave@lists.nfs-ganesha.org>
_______________________________________________ Devel mailing list -- devel@lists.nfs-ganesha.org <mailto:devel@lists.nfs-ganesha.org> To unsubscribe send an email to devel-leave@lists.nfs-ganesha.org <mailto:devel-leave@lists.nfs-ganesha.org> -- with regards, Sachin Punadikar
participants (4)
-
Daniel Gryniewicz -
Frank Filz -
Pradeep -
Sachin Punadikar