| Message ID | 7561c096c7de603ac39fcfcff7bd2ec80589cae1.1418618044.git.osandov@osandov.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Mon, Dec 15, 2014 at 12:26 AM, Omar Sandoval <osandov@osandov.com> wrote: > The generic callers of direct_IO lock i_mutex before doing a write. NFS > doesn't use the generic write code, so it doesn't follow this > convention. This is now a problem because the interface introduced for > swap-over-NFS calls direct_IO for a write without holding i_mutex, but > other implementations of direct_IO will expect to have it locked. I really don't care much about swap-over-NFS performance; that's a niche usage at best. I _do_ care about O_DIRECT performance, and the ability to run multiple WRITE calls in parallel. IOW: Patch NACKed... Please find another solution. Trond -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Dec 15, 2014 at 07:49:20AM -0500, Trond Myklebust wrote: > On Mon, Dec 15, 2014 at 12:26 AM, Omar Sandoval <osandov@osandov.com> wrote: > > The generic callers of direct_IO lock i_mutex before doing a write. NFS > > doesn't use the generic write code, so it doesn't follow this > > convention. This is now a problem because the interface introduced for > > swap-over-NFS calls direct_IO for a write without holding i_mutex, but > > other implementations of direct_IO will expect to have it locked. > > I really don't care much about swap-over-NFS performance; that's a > niche usage at best. I _do_ care about O_DIRECT performance, and the > ability to run multiple WRITE calls in parallel. > > IOW: Patch NACKed... Please find another solution. > > Trond So the patch formatting doesn't make it completely clear what's going on here, but here's what the original nfs_file_direct_write code did: - called with i_mutex unlocked - collects stats and does some generic checks - locks i_mutex - syncs the mapping, schedules the write - unlocks i_mutex - waits for the write to complete if synchronous After this patch, nfs_file_direct_write works like: - called with i_mutex locked - collects stats and does some generic checks - syncs the mapping, schedules the write - drops i_mutex - waits for the write to complete if synchronous - picks i_mutex back up There's an extra lock and unlock as a result and a slightly longer critical section, but we drop i_mutex to wait for the write, so multiple writes still work in parallel.
diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index 10bf072..9402b96 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -906,17 +906,15 @@ ssize_t nfs_file_direct_write(struct kiocb *iocb, struct iov_iter *iter, if (!count) goto out; - mutex_lock(&inode->i_mutex); - result = nfs_sync_mapping(mapping); if (result) - goto out_unlock; + goto out; if (mapping->nrpages) { result = invalidate_inode_pages2_range(mapping, pos >> PAGE_CACHE_SHIFT, end); if (result) - goto out_unlock; + goto out; } task_io_account_write(count); @@ -924,7 +922,7 @@ ssize_t nfs_file_direct_write(struct kiocb *iocb, struct iov_iter *iter, result = -ENOMEM; dreq = nfs_direct_req_alloc(); if (!dreq) - goto out_unlock; + goto out; dreq->inode = inode; dreq->bytes_left = count; @@ -960,12 +958,12 @@ ssize_t nfs_file_direct_write(struct kiocb *iocb, struct iov_iter *iter, } } nfs_direct_req_release(dreq); + + mutex_lock(&inode->i_mutex); return result; out_release: nfs_direct_req_release(dreq); -out_unlock: - mutex_unlock(&inode->i_mutex); out: return result; } diff --git a/fs/nfs/file.c b/fs/nfs/file.c index 2ab6f00..8b80276 100644 --- a/fs/nfs/file.c +++ b/fs/nfs/file.c @@ -675,8 +675,12 @@ ssize_t nfs_file_write(struct kiocb *iocb, struct iov_iter *from) if (result) return result; - if (file->f_flags & O_DIRECT) - return nfs_file_direct_write(iocb, from, pos); + if (file->f_flags & O_DIRECT) { + mutex_lock(&inode->i_mutex); + result = nfs_file_direct_write(iocb, from, pos); + mutex_unlock(&inode->i_mutex); + return result; + } dprintk("NFS: write(%pD2, %zu@%Ld)\n", file, count, (long long) pos);
The generic callers of direct_IO lock i_mutex before doing a write. NFS doesn't use the generic write code, so it doesn't follow this convention. This is now a problem because the interface introduced for swap-over-NFS calls direct_IO for a write without holding i_mutex, but other implementations of direct_IO will expect to have it locked. Signed-off-by: Omar Sandoval <osandov@osandov.com> --- fs/nfs/direct.c | 12 +++++------- fs/nfs/file.c | 8 ++++++-- 2 files changed, 11 insertions(+), 9 deletions(-)