From patchwork Sat Apr 11 12:10:49 2009 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 17742 Received: from vger.kernel.org (vger.kernel.org [209.132.176.167]) by demeter.kernel.org (8.14.2/8.14.2) with ESMTP id n3BCBHIx023710 for ; Sat, 11 Apr 2009 12:11:17 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753679AbZDKMLB (ORCPT ); Sat, 11 Apr 2009 08:11:01 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1753031AbZDKMLB (ORCPT ); Sat, 11 Apr 2009 08:11:01 -0400 Received: from out01.mta.xmission.com ([166.70.13.231]:53778 "EHLO out01.mta.xmission.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751003AbZDKMK4 (ORCPT ); Sat, 11 Apr 2009 08:10:56 -0400 Received: from in01.mta.xmission.com ([166.70.13.51]) by out01.mta.xmission.com with esmtp (Exim 4.62) (envelope-from ) id 1Lsc3B-0004Y8-B4; Sat, 11 Apr 2009 06:11:09 -0600 Received: from c-67-169-126-145.hsd1.ca.comcast.net ([67.169.126.145] helo=fess.ebiederm.org) by in01.mta.xmission.com with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69) (envelope-from ) id 1Lsc2v-0004Dp-4e; Sat, 11 Apr 2009 06:10:55 -0600 Received: from fess.ebiederm.org (localhost [127.0.0.1]) by fess.ebiederm.org (8.14.3/8.14.3/Debian-4) with ESMTP id n3BCAnjH002069; Sat, 11 Apr 2009 05:10:49 -0700 Received: (from eric@localhost) by fess.ebiederm.org (8.14.3/8.14.3/Submit) id n3BCAnxB002068; Sat, 11 Apr 2009 05:10:49 -0700 X-Authentication-Warning: fess.ebiederm.org: eric set sender to ebiederm@xmission.com using -f To: Andrew Morton Cc: , , , , Al Viro , Hugh Dickins , Tejun Heo , Alexey Dobriyan , Linus Torvalds , Alan Cox , Greg Kroah-Hartman References: From: ebiederm@xmission.com (Eric W. Biederman) Date: Sat, 11 Apr 2009 05:10:49 -0700 In-Reply-To: (Eric W. Biederman's message of "Sat\, 11 Apr 2009 05\:01\:29 -0700") Message-ID: User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.2 (gnu/linux) MIME-Version: 1.0 X-XM-SPF: eid=; ; ; mid=; ; ; hst=in01.mta.xmission.com; ; ; ip=67.169.126.145; ; ; frm=ebiederm@xmission.com; ; ; spf=neutral X-SA-Exim-Connect-IP: 67.169.126.145 X-SA-Exim-Rcpt-To: akpm@linux-foundation.org, gregkh@suse.de, alan@lxorguk.ukuu.org.uk, torvalds@linux-foundation.org, adobriyan@gmail.com, tj@kernel.org, hugh@veritas.com, viro@ZenIV.linux.org.uk, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: ebiederm@xmission.com X-Spam-DCC: XMission; sa01 1397; Body=1 Fuz1=1 Fuz2=1 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on sa01.xmission.com X-Spam-Level: X-Spam-Status: No, score=-2.5 required=8.0 tests=ALL_TRUSTED,BAYES_00, DCC_CHECK_NEGATIVE,UNTRUSTED_Relay,XMNoVowels,XM_SPF_Neutral autolearn=disabled version=3.2.5 X-Spam-Combo: ;Andrew Morton X-Spam-Relay-Country: X-Spam-Report: * -1.8 ALL_TRUSTED Passed through trusted hosts only via SMTP * 1.5 XMNoVowels Alpha-numberic number with no vowels * -2.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1% * [score: 0.0000] * -0.0 DCC_CHECK_NEGATIVE Not listed in DCC * [sa01 1397; Body=1 Fuz1=1 Fuz2=1] * 0.0 XM_SPF_Neutral SPF-Neutral * 0.4 UNTRUSTED_Relay Comes from a non-trusted relay Subject: [RFC][PATCH 6/9] vfs: Utilize fops_read_lock where appropriate X-SA-Exim-Version: 4.2.1 (built Thu, 25 Oct 2007 00:26:12 +0000) X-SA-Exim-Scanned: Yes (on in01.mta.xmission.com) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Signed-off-by: Eric W. Biederman --- fs/compat.c | 31 ++++++++---- fs/fcntl.c | 32 ++++++++---- fs/ioctl.c | 39 +++++++++------ fs/locks.c | 81 +++++++++++++++++++++++++------ fs/open.c | 12 ++++- fs/read_write.c | 143 ++++++++++++++++++++++++++++++++++++++++++------------- fs/readdir.c | 14 ++++- fs/select.c | 17 +++++- 8 files changed, 276 insertions(+), 93 deletions(-) diff --git a/fs/compat.c b/fs/compat.c index 3f84d5f..a73ca0d 100644 --- a/fs/compat.c +++ b/fs/compat.c @@ -1090,6 +1090,7 @@ out: #endif /* ! __ARCH_OMIT_COMPAT_SYS_GETDENTS64 */ static ssize_t compat_do_readv_writev(int type, struct file *file, + const struct file_operations *f_op, const struct compat_iovec __user *uvector, unsigned long nr_segs, loff_t *pos) { @@ -1117,7 +1118,7 @@ static ssize_t compat_do_readv_writev(int type, struct file *file, ret = -EINVAL; if ((nr_segs > UIO_MAXIOV) || (nr_segs <= 0)) goto out; - if (!file->f_op) + if (!f_op) goto out; if (nr_segs > UIO_FASTIOV) { ret = -ENOMEM; @@ -1170,11 +1171,11 @@ static ssize_t compat_do_readv_writev(int type, struct file *file, fnv = NULL; if (type == READ) { - fn = file->f_op->read; - fnv = file->f_op->aio_read; + fn = f_op->read; + fnv = f_op->aio_read; } else { - fn = (io_fn_t)file->f_op->write; - fnv = file->f_op->aio_write; + fn = (io_fn_t)f_op->write; + fnv = f_op->aio_write; } if (fnv) @@ -1200,21 +1201,27 @@ static size_t compat_readv(struct file *file, const struct compat_iovec __user *vec, unsigned long vlen, loff_t *pos) { + const struct file_operations *f_op; + int fops_idx; ssize_t ret = -EBADF; + fops_idx = fops_read_lock(file); + f_op = rcu_dereference(file->f_op); + if (!(file->f_mode & FMODE_READ)) goto out; ret = -EINVAL; - if (!file->f_op || (!file->f_op->aio_read && !file->f_op->read)) + if (!f_op || (!f_op->aio_read && !f_op->read)) goto out; - ret = compat_do_readv_writev(READ, file, vec, vlen, pos); + ret = compat_do_readv_writev(READ, file, f_op, vec, vlen, pos); out: if (ret > 0) add_rchar(current, ret); inc_syscr(current); + fops_read_unlock(file, fops_idx); return ret; } @@ -1257,21 +1264,27 @@ static size_t compat_writev(struct file *file, const struct compat_iovec __user *vec, unsigned long vlen, loff_t *pos) { + const struct file_operations *f_op; + int fops_idx; ssize_t ret = -EBADF; + fops_idx = fops_read_lock(file); + f_op = rcu_dereference(file->f_op); + if (!(file->f_mode & FMODE_WRITE)) goto out; ret = -EINVAL; - if (!file->f_op || (!file->f_op->aio_write && !file->f_op->write)) + if (!f_op || (!f_op->aio_write && !f_op->write)) goto out; - ret = compat_do_readv_writev(WRITE, file, vec, vlen, pos); + ret = compat_do_readv_writev(WRITE, file, f_op, vec, vlen, pos); out: if (ret > 0) add_wchar(current, ret); inc_syscw(current); + fops_read_unlock(file, fops_idx); return ret; } diff --git a/fs/fcntl.c b/fs/fcntl.c index cc8e4de..2718aea 100644 --- a/fs/fcntl.c +++ b/fs/fcntl.c @@ -146,42 +146,51 @@ SYSCALL_DEFINE1(dup, unsigned int, fildes) static int setfl(int fd, struct file * filp, unsigned long arg) { struct inode * inode = filp->f_path.dentry->d_inode; - int error = 0; + const struct file_operations *f_op; + int fops_idx; + int error; + + fops_idx = fops_read_lock(filp); + f_op = rcu_dereference(filp->f_op); /* * O_APPEND cannot be cleared if the file is marked as append-only * and the file is open for write. */ + error = -EPERM; if (((arg ^ filp->f_flags) & O_APPEND) && IS_APPEND(inode)) - return -EPERM; + goto out; /* O_NOATIME can only be set by the owner or superuser */ + error = -EPERM; if ((arg & O_NOATIME) && !(filp->f_flags & O_NOATIME)) if (!is_owner_or_cap(inode)) - return -EPERM; + goto out; /* required for strict SunOS emulation */ if (O_NONBLOCK != O_NDELAY) if (arg & O_NDELAY) arg |= O_NONBLOCK; + error = -EINVAL; if (arg & O_DIRECT) { if (!filp->f_mapping || !filp->f_mapping->a_ops || - !filp->f_mapping->a_ops->direct_IO) - return -EINVAL; + !filp->f_mapping->a_ops->direct_IO) + goto out; } - if (filp->f_op && filp->f_op->check_flags) - error = filp->f_op->check_flags(arg); + error = 0; + if (f_op && f_op->check_flags) + error = f_op->check_flags(arg); if (error) - return error; + goto out; /* * ->fasync() is responsible for setting the FASYNC bit. */ - if (((arg ^ filp->f_flags) & FASYNC) && filp->f_op && - filp->f_op->fasync) { - error = filp->f_op->fasync(fd, filp, (arg & FASYNC) != 0); + if (((arg ^ filp->f_flags) & FASYNC) && f_op && + f_op->fasync) { + error = f_op->fasync(fd, filp, (arg & FASYNC) != 0); if (error < 0) goto out; if (error > 0) @@ -192,6 +201,7 @@ static int setfl(int fd, struct file * filp, unsigned long arg) spin_unlock(&filp->f_lock); out: + fops_read_unlock(filp, fops_idx); return error; } diff --git a/fs/ioctl.c b/fs/ioctl.c index ac2d47e..158030b 100644 --- a/fs/ioctl.c +++ b/fs/ioctl.c @@ -33,23 +33,23 @@ * * Returns 0 on success, -errno on error. */ -static long vfs_ioctl(struct file *filp, unsigned int cmd, - unsigned long arg) +static long vfs_ioctl(struct file *filp, const struct file_operations *f_op, + unsigned int cmd, unsigned long arg) { int error = -ENOTTY; - if (!filp->f_op) + if (!f_op) goto out; - if (filp->f_op->unlocked_ioctl) { - error = filp->f_op->unlocked_ioctl(filp, cmd, arg); + if (f_op->unlocked_ioctl) { + error = f_op->unlocked_ioctl(filp, cmd, arg); if (error == -ENOIOCTLCMD) error = -EINVAL; goto out; - } else if (filp->f_op->ioctl) { + } else if (f_op->ioctl) { lock_kernel(); - error = filp->f_op->ioctl(filp->f_path.dentry->d_inode, - filp, cmd, arg); + error = f_op->ioctl(filp->f_path.dentry->d_inode, filp, + cmd, arg); unlock_kernel(); } @@ -370,8 +370,8 @@ EXPORT_SYMBOL(generic_block_fiemap); #endif /* CONFIG_BLOCK */ -static int file_ioctl(struct file *filp, unsigned int cmd, - unsigned long arg) +static int file_ioctl(struct file *filp, const struct file_operations *f_op, + unsigned int cmd, unsigned long arg) { struct inode *inode = filp->f_path.dentry->d_inode; int __user *p = (int __user *)arg; @@ -387,7 +387,7 @@ static int file_ioctl(struct file *filp, unsigned int cmd, return put_user(i_size_read(inode) - filp->f_pos, p); } - return vfs_ioctl(filp, cmd, arg); + return vfs_ioctl(filp, f_op, cmd, arg); } static int ioctl_fionbio(struct file *filp, int __user *argp) @@ -414,6 +414,7 @@ static int ioctl_fionbio(struct file *filp, int __user *argp) } static int ioctl_fioasync(unsigned int fd, struct file *filp, + const struct file_operations *f_op, int __user *argp) { unsigned int flag; @@ -426,9 +427,9 @@ static int ioctl_fioasync(unsigned int fd, struct file *filp, /* Did FASYNC state change ? */ if ((flag ^ filp->f_flags) & FASYNC) { - if (filp->f_op && filp->f_op->fasync) + if (f_op && f_op->fasync) /* fasync() adjusts filp->f_flags */ - error = filp->f_op->fasync(fd, filp, on); + error = f_op->fasync(fd, filp, on); else error = -ENOTTY; } @@ -482,9 +483,14 @@ static int ioctl_fsthaw(struct file *filp) int do_vfs_ioctl(struct file *filp, unsigned int fd, unsigned int cmd, unsigned long arg) { + const struct file_operations *f_op; + int fops_idx; int error = 0; int __user *argp = (int __user *)arg; + fops_idx = fops_read_lock(filp); + f_op = rcu_dereference(filp->f_op); + switch (cmd) { case FIOCLEX: set_close_on_exec(fd, 1); @@ -499,7 +505,7 @@ int do_vfs_ioctl(struct file *filp, unsigned int fd, unsigned int cmd, break; case FIOASYNC: - error = ioctl_fioasync(fd, filp, argp); + error = ioctl_fioasync(fd, filp, f_op, argp); break; case FIOQSIZE: @@ -524,11 +530,12 @@ int do_vfs_ioctl(struct file *filp, unsigned int fd, unsigned int cmd, default: if (S_ISREG(filp->f_path.dentry->d_inode->i_mode)) - error = file_ioctl(filp, cmd, arg); + error = file_ioctl(filp, f_op, cmd, arg); else - error = vfs_ioctl(filp, cmd, arg); + error = vfs_ioctl(filp, f_op, cmd, arg); break; } + fops_read_unlock(filp, fops_idx); return error; } diff --git a/fs/locks.c b/fs/locks.c index ec3deea..5ff959e 100644 --- a/fs/locks.c +++ b/fs/locks.c @@ -1463,15 +1463,21 @@ EXPORT_SYMBOL(generic_setlease); int vfs_setlease(struct file *filp, long arg, struct file_lock **lease) { + const struct file_operations *f_op; + int fops_idx; int error; + fops_idx = fops_read_lock(filp); + f_op = rcu_dereference(filp->f_op); + lock_kernel(); - if (filp->f_op && filp->f_op->setlease) - error = filp->f_op->setlease(filp, arg, lease); + if (f_op && f_op->setlease) + error = f_op->setlease(filp, arg, lease); else error = generic_setlease(filp, arg, lease); unlock_kernel(); + fops_read_unlock(filp, fops_idx); return error; } EXPORT_SYMBOL_GPL(vfs_setlease); @@ -1566,9 +1572,11 @@ EXPORT_SYMBOL(flock_lock_file_wait); */ SYSCALL_DEFINE2(flock, unsigned int, fd, unsigned int, cmd) { + const struct file_operations *f_op; struct file *filp; struct file_lock *lock; int can_sleep, unlock; + int fops_idx; int error; error = -EBADF; @@ -1594,13 +1602,18 @@ SYSCALL_DEFINE2(flock, unsigned int, fd, unsigned int, cmd) if (error) goto out_free; - if (filp->f_op && filp->f_op->flock) - error = filp->f_op->flock(filp, + fops_idx = fops_read_lock(filp); + f_op = rcu_dereference(filp->f_op); + + if (f_op && f_op->flock) + error = f_op->flock(filp, (can_sleep) ? F_SETLKW : F_SETLK, lock); else error = flock_lock_file_wait(filp, lock); + fops_read_unlock(filp, fops_idx); + out_free: locks_free_lock(lock); @@ -1620,10 +1633,20 @@ SYSCALL_DEFINE2(flock, unsigned int, fd, unsigned int, cmd) */ int vfs_test_lock(struct file *filp, struct file_lock *fl) { - if (filp->f_op && filp->f_op->lock) - return filp->f_op->lock(filp, F_GETLK, fl); - posix_test_lock(filp, fl); - return 0; + const struct file_operations *f_op; + int fops_idx; + int ret = 0; + + fops_idx = fops_read_lock(filp); + f_op = rcu_dereference(filp->f_op); + + if (f_op && f_op->lock) + ret = filp->f_op->lock(filp, F_GETLK, fl); + else + posix_test_lock(filp, fl); + + fops_read_unlock(filp, fops_idx); + return ret; } EXPORT_SYMBOL_GPL(vfs_test_lock); @@ -1732,10 +1755,20 @@ out: */ int vfs_lock_file(struct file *filp, unsigned int cmd, struct file_lock *fl, struct file_lock *conf) { - if (filp->f_op && filp->f_op->lock) - return filp->f_op->lock(filp, cmd, fl); + const struct file_operations *f_op; + int fops_idx; + int ret; + + fops_idx = fops_read_lock(filp); + f_op = rcu_dereference(filp->f_op); + + if (f_op && f_op->lock) + ret = f_op->lock(filp, cmd, fl); else - return posix_lock_file(filp, fl, conf); + ret = posix_lock_file(filp, fl, conf); + + fops_read_unlock(filp, fops_idx); + return ret; } EXPORT_SYMBOL_GPL(vfs_lock_file); @@ -1999,13 +2032,18 @@ EXPORT_SYMBOL(locks_remove_posix); void locks_remove_flock(struct file *filp) { struct inode * inode = filp->f_path.dentry->d_inode; + const struct file_operations *f_op; struct file_lock *fl; struct file_lock **before; + int fops_idx; if (!inode->i_flock) return; - if (filp->f_op && filp->f_op->flock) { + fops_idx = fops_read_lock(filp); + f_op = rcu_dereference(filp->f_op); + + if (f_op && f_op->flock) { struct file_lock fl = { .fl_pid = current->tgid, .fl_file = filp, @@ -2013,11 +2051,13 @@ void locks_remove_flock(struct file *filp) .fl_type = F_UNLCK, .fl_end = OFFSET_MAX, }; - filp->f_op->flock(filp, F_SETLKW, &fl); + f_op->flock(filp, F_SETLKW, &fl); if (fl.fl_ops && fl.fl_ops->fl_release_private) fl.fl_ops->fl_release_private(&fl); } + fops_read_unlock(filp, fops_idx); + lock_kernel(); before = &inode->i_flock; @@ -2071,9 +2111,18 @@ EXPORT_SYMBOL(posix_unblock_lock); */ int vfs_cancel_lock(struct file *filp, struct file_lock *fl) { - if (filp->f_op && filp->f_op->lock) - return filp->f_op->lock(filp, F_CANCELLK, fl); - return 0; + const struct file_operations *f_op; + int fops_idx; + int ret = 0; + + fops_idx = fops_read_lock(filp); + f_op = rcu_dereference(filp->f_op); + + if (f_op && f_op->lock) + ret = f_op->lock(filp, F_CANCELLK, fl); + + fops_read_unlock(filp, fops_idx); + return ret; } EXPORT_SYMBOL_GPL(vfs_cancel_lock); diff --git a/fs/open.c b/fs/open.c index 0b75dde..67031e7 100644 --- a/fs/open.c +++ b/fs/open.c @@ -398,6 +398,7 @@ SYSCALL_DEFINE(fallocate)(int fd, int mode, loff_t offset, loff_t len) goto out; if (!(file->f_mode & FMODE_WRITE)) goto out_fput; + /* * Revalidate the write permissions, in case security policy has * changed since the files were opened. @@ -1107,6 +1108,8 @@ SYSCALL_DEFINE2(creat, const char __user *, pathname, int, mode) */ int filp_close(struct file *filp, fl_owner_t id) { + const struct file_operations *f_op; + int fops_idx; int retval = 0; if (!file_count(filp)) { @@ -1114,8 +1117,13 @@ int filp_close(struct file *filp, fl_owner_t id) return 0; } - if (filp->f_op && filp->f_op->flush) - retval = filp->f_op->flush(filp, id); + fops_idx = fops_read_lock(filp); + f_op = rcu_dereference(filp->f_op); + + if (f_op && f_op->flush) + retval = f_op->flush(filp, id); + + fops_read_unlock(filp, fops_idx); dnotify_flush(filp, id); locks_remove_posix(filp, id); diff --git a/fs/read_write.c b/fs/read_write.c index 9d1e76b..4def2ee 100644 --- a/fs/read_write.c +++ b/fs/read_write.c @@ -136,14 +136,23 @@ EXPORT_SYMBOL(default_llseek); loff_t vfs_llseek(struct file *file, loff_t offset, int origin) { loff_t (*fn)(struct file *, loff_t, int); + const struct file_operations *f_op; + int fops_idx; + loff_t ret; + + fops_idx = fops_read_lock(file); + f_op = rcu_dereference(file->f_op); fn = no_llseek; if (file->f_mode & FMODE_LSEEK) { fn = default_llseek; - if (file->f_op && file->f_op->llseek) - fn = file->f_op->llseek; + if (f_op && f_op->llseek) + fn = f_op->llseek; } - return fn(file, offset, origin); + ret = fn(file, offset, origin); + + fops_read_unlock(file, fops_idx); + return ret; } EXPORT_SYMBOL(vfs_llseek); @@ -252,15 +261,20 @@ static void wait_on_retry_sync_kiocb(struct kiocb *iocb) ssize_t do_sync_read(struct file *filp, char __user *buf, size_t len, loff_t *ppos) { struct iovec iov = { .iov_base = buf, .iov_len = len }; + const struct file_operations *f_op; struct kiocb kiocb; + int fops_idx; ssize_t ret; + fops_idx = fops_read_lock(filp); + f_op = rcu_dereference(filp->f_op); + init_sync_kiocb(&kiocb, filp); kiocb.ki_pos = *ppos; kiocb.ki_left = len; for (;;) { - ret = filp->f_op->aio_read(&kiocb, &iov, 1, kiocb.ki_pos); + ret = f_op->aio_read(&kiocb, &iov, 1, kiocb.ki_pos); if (ret != -EIOCBRETRY) break; wait_on_retry_sync_kiocb(&kiocb); @@ -269,6 +283,8 @@ ssize_t do_sync_read(struct file *filp, char __user *buf, size_t len, loff_t *pp if (-EIOCBQUEUED == ret) ret = wait_on_sync_kiocb(&kiocb); *ppos = kiocb.ki_pos; + fops_read_unlock(filp, fops_idx); + return ret; } @@ -276,20 +292,28 @@ EXPORT_SYMBOL(do_sync_read); ssize_t vfs_read(struct file *file, char __user *buf, size_t count, loff_t *pos) { + const struct file_operations *f_op; + int fops_idx; ssize_t ret; + fops_idx = fops_read_lock(file); + f_op = rcu_dereference(file->f_op); + + ret = -EBADF; if (!(file->f_mode & FMODE_READ)) - return -EBADF; - if (!file->f_op || (!file->f_op->read && !file->f_op->aio_read)) - return -EINVAL; + goto out; + ret = -EINVAL; + if (!f_op || (!f_op->read && !f_op->aio_read)) + goto out; + ret = -EFAULT; if (unlikely(!access_ok(VERIFY_WRITE, buf, count))) - return -EFAULT; + goto out; ret = rw_verify_area(READ, file, pos, count); if (ret >= 0) { count = ret; - if (file->f_op->read) - ret = file->f_op->read(file, buf, count, pos); + if (f_op->read) + ret = f_op->read(file, buf, count, pos); else ret = do_sync_read(file, buf, count, pos); if (ret > 0) { @@ -298,7 +322,8 @@ ssize_t vfs_read(struct file *file, char __user *buf, size_t count, loff_t *pos) } inc_syscr(current); } - +out: + fops_read_unlock(file, fops_idx); return ret; } @@ -307,15 +332,20 @@ EXPORT_SYMBOL(vfs_read); ssize_t do_sync_write(struct file *filp, const char __user *buf, size_t len, loff_t *ppos) { struct iovec iov = { .iov_base = (void __user *)buf, .iov_len = len }; + const struct file_operations *f_op; struct kiocb kiocb; + int fops_idx; ssize_t ret; + fops_idx = fops_read_lock(filp); + f_op = rcu_dereference(filp->f_op); + init_sync_kiocb(&kiocb, filp); kiocb.ki_pos = *ppos; kiocb.ki_left = len; for (;;) { - ret = filp->f_op->aio_write(&kiocb, &iov, 1, kiocb.ki_pos); + ret = f_op->aio_write(&kiocb, &iov, 1, kiocb.ki_pos); if (ret != -EIOCBRETRY) break; wait_on_retry_sync_kiocb(&kiocb); @@ -324,6 +354,8 @@ ssize_t do_sync_write(struct file *filp, const char __user *buf, size_t len, lof if (-EIOCBQUEUED == ret) ret = wait_on_sync_kiocb(&kiocb); *ppos = kiocb.ki_pos; + + fops_read_unlock(filp, fops_idx); return ret; } @@ -331,20 +363,28 @@ EXPORT_SYMBOL(do_sync_write); ssize_t vfs_write(struct file *file, const char __user *buf, size_t count, loff_t *pos) { + const struct file_operations *f_op; + int fops_idx; ssize_t ret; + fops_idx = fops_read_lock(file); + f_op = rcu_dereference(file->f_op); + + ret = -EBADF; if (!(file->f_mode & FMODE_WRITE)) - return -EBADF; - if (!file->f_op || (!file->f_op->write && !file->f_op->aio_write)) - return -EINVAL; + goto out; + ret = -EINVAL; + if (!f_op || (!f_op->write && !f_op->aio_write)) + goto out; + ret = -EFAULT; if (unlikely(!access_ok(VERIFY_READ, buf, count))) - return -EFAULT; + goto out; ret = rw_verify_area(WRITE, file, pos, count); if (ret >= 0) { count = ret; - if (file->f_op->write) - ret = file->f_op->write(file, buf, count, pos); + if (f_op->write) + ret = f_op->write(file, buf, count, pos); else ret = do_sync_write(file, buf, count, pos); if (ret > 0) { @@ -354,6 +394,8 @@ ssize_t vfs_write(struct file *file, const char __user *buf, size_t count, loff_ inc_syscw(current); } +out: + fops_read_unlock(file, fops_idx); return ret; } @@ -611,6 +653,7 @@ out: } static ssize_t do_readv_writev(int type, struct file *file, + const struct file_operations *f_op, const struct iovec __user * uvector, unsigned long nr_segs, loff_t *pos) { @@ -621,7 +664,7 @@ static ssize_t do_readv_writev(int type, struct file *file, io_fn_t fn; iov_fn_t fnv; - if (!file->f_op) { + if (!f_op) { ret = -EINVAL; goto out; } @@ -638,11 +681,11 @@ static ssize_t do_readv_writev(int type, struct file *file, fnv = NULL; if (type == READ) { - fn = file->f_op->read; - fnv = file->f_op->aio_read; + fn = f_op->read; + fnv = f_op->aio_read; } else { - fn = (io_fn_t)file->f_op->write; - fnv = file->f_op->aio_write; + fn = (io_fn_t)f_op->write; + fnv = f_op->aio_write; } if (fnv) @@ -666,12 +709,25 @@ out: ssize_t vfs_readv(struct file *file, const struct iovec __user *vec, unsigned long vlen, loff_t *pos) { + const struct file_operations *f_op; + int fops_idx; + ssize_t ret; + + fops_idx = fops_read_lock(file); + f_op = rcu_dereference(file->f_op); + + ret = -EBADF; if (!(file->f_mode & FMODE_READ)) - return -EBADF; - if (!file->f_op || (!file->f_op->aio_read && !file->f_op->read)) - return -EINVAL; + goto out; + ret = -EINVAL; + if (!f_op || (!f_op->aio_read && !f_op->read)) + goto out; + + ret = do_readv_writev(READ, file, f_op, vec, vlen, pos); - return do_readv_writev(READ, file, vec, vlen, pos); +out: + fops_read_unlock(file, fops_idx); + return ret; } EXPORT_SYMBOL(vfs_readv); @@ -679,12 +735,25 @@ EXPORT_SYMBOL(vfs_readv); ssize_t vfs_writev(struct file *file, const struct iovec __user *vec, unsigned long vlen, loff_t *pos) { + const struct file_operations *f_op; + int fops_idx; + ssize_t ret; + + fops_idx = fops_read_lock(file); + f_op = rcu_dereference(file->f_op); + + ret = -EBADF; if (!(file->f_mode & FMODE_WRITE)) - return -EBADF; - if (!file->f_op || (!file->f_op->aio_write && !file->f_op->write)) - return -EINVAL; + goto out; + ret = -EINVAL; + if (!f_op || (!f_op->aio_write && !f_op->write)) + goto out; + + ret = do_readv_writev(WRITE, file, f_op, vec, vlen, pos); - return do_readv_writev(WRITE, file, vec, vlen, pos); +out: + fops_read_unlock(file, fops_idx); + return ret; } EXPORT_SYMBOL(vfs_writev); @@ -790,8 +859,10 @@ SYSCALL_DEFINE5(pwritev, unsigned long, fd, const struct iovec __user *, vec, static ssize_t do_sendfile(int out_fd, int in_fd, loff_t *ppos, size_t count, loff_t max) { + const struct file_operations *in_f_op, *out_f_op; struct file * in_file, * out_file; struct inode * in_inode, * out_inode; + int in_fops_idx, out_fops_idx; loff_t pos; ssize_t retval; int fput_needed_in, fput_needed_out, fl; @@ -803,13 +874,15 @@ static ssize_t do_sendfile(int out_fd, int in_fd, loff_t *ppos, in_file = fget_light(in_fd, &fput_needed_in); if (!in_file) goto out; + in_fops_idx = fops_read_lock(in_file); if (!(in_file->f_mode & FMODE_READ)) goto fput_in; retval = -EINVAL; in_inode = in_file->f_path.dentry->d_inode; if (!in_inode) goto fput_in; - if (!in_file->f_op || !in_file->f_op->splice_read) + in_f_op = rcu_dereference(in_file->f_op); + if (!in_f_op || !in_f_op->splice_read) goto fput_in; retval = -ESPIPE; if (!ppos) @@ -829,10 +902,12 @@ static ssize_t do_sendfile(int out_fd, int in_fd, loff_t *ppos, out_file = fget_light(out_fd, &fput_needed_out); if (!out_file) goto fput_in; + out_fops_idx = fops_read_lock(out_file); if (!(out_file->f_mode & FMODE_WRITE)) goto fput_out; retval = -EINVAL; - if (!out_file->f_op || !out_file->f_op->sendpage) + out_f_op = rcu_dereference(out_file->f_op); + if (!out_f_op || !out_f_op->sendpage) goto fput_out; out_inode = out_file->f_path.dentry->d_inode; retval = rw_verify_area(WRITE, out_file, &out_file->f_pos, count); @@ -878,8 +953,10 @@ static ssize_t do_sendfile(int out_fd, int in_fd, loff_t *ppos, retval = -EOVERFLOW; fput_out: + fops_read_unlock(out_file, out_fops_idx); fput_light(out_file, fput_needed_out); fput_in: + fops_read_unlock(in_file, in_fops_idx); fput_light(in_file, fput_needed_in); out: return retval; diff --git a/fs/readdir.c b/fs/readdir.c index 7723401..6017fa6 100644 --- a/fs/readdir.c +++ b/fs/readdir.c @@ -21,9 +21,16 @@ int vfs_readdir(struct file *file, filldir_t filler, void *buf) { - struct inode *inode = file->f_path.dentry->d_inode; + const struct file_operations *f_op; + struct inode *inode; + int fops_idx; int res = -ENOTDIR; - if (!file->f_op || !file->f_op->readdir) + + fops_idx = fops_read_lock(file); + f_op = rcu_dereference(file->f_op); + + inode = file->f_path.dentry->d_inode; + if (!f_op || !f_op->readdir) goto out; res = security_file_permission(file, MAY_READ); @@ -36,11 +43,12 @@ int vfs_readdir(struct file *file, filldir_t filler, void *buf) res = -ENOENT; if (!IS_DEADDIR(inode)) { - res = file->f_op->readdir(file, buf, filler); + res = f_op->readdir(file, buf, filler); file_accessed(file); } mutex_unlock(&inode->i_mutex); out: + fops_read_unlock(file, fops_idx); return res; } diff --git a/fs/select.c b/fs/select.c index 0fe0e14..8f736a9 100644 --- a/fs/select.c +++ b/fs/select.c @@ -416,10 +416,12 @@ int do_select(int n, fd_set_bits *fds, struct timespec *end_time) continue; file = fget_light(i, &fput_needed); if (file) { - f_op = file->f_op; + int fops_idx = fops_read_lock(file); + f_op = rcu_dereference(file->f_op); mask = DEFAULT_POLLMASK; if (f_op && f_op->poll) mask = (*f_op->poll)(file, retval ? NULL : wait); + fops_read_unlock(file, fops_idx); fput_light(file, fput_needed); if ((mask & POLLIN_SET) && (in & bit)) { res_in |= bit; @@ -684,11 +686,20 @@ static inline unsigned int do_pollfd(struct pollfd *pollfd, poll_table *pwait) file = fget_light(fd, &fput_needed); mask = POLLNVAL; if (file != NULL) { + const struct file_operations *f_op; + int fops_idx; + + fops_idx = fops_read_lock(file); + f_op = rcu_dereference(file->f_op); + mask = DEFAULT_POLLMASK; - if (file->f_op && file->f_op->poll) - mask = file->f_op->poll(file, pwait); + if (f_op && f_op->poll) + mask = f_op->poll(file, pwait); + /* Mask out unneeded events. */ mask &= pollfd->events | POLLERR | POLLHUP; + + fops_read_unlock(file, fops_idx); fput_light(file, fput_needed); } }