From patchwork Wed May 18 08:40:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12853349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D46CC433EF for ; Wed, 18 May 2022 08:40:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233410AbiERIkQ (ORCPT ); Wed, 18 May 2022 04:40:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47026 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233321AbiERIkQ (ORCPT ); Wed, 18 May 2022 04:40:16 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7896BC8BE0 for ; Wed, 18 May 2022 01:40:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=SDLfs/lKOmEecD7rLW8XM7B/pNb5s2qArw4hhBChVwE=; b=FCGlgd+bYmwZTJb6TVKcKBf9R7 y0EjU0m7aOIyJhXksSiS5/NAqVu74UjZGlTAhgNGPjrUPnfXUsJznqywf5sQhd8dHLsM8+XwLKIbq WF5kQMcQSA9idV9tiM9mprjmHb/5IfUvz5+VFeKYu+VruR8eDp+sDySxN4E9zXGJbkYIGjJQeDQYw L7Fy+7V0y8AUmo6mANsMVvSpu8JuYljJcr46/lU9aGWp1FMPrF9zGGL1pQddv9rC4iWxOx8l+jSco k3wZJB+X2lfj4u5ba11nB1mHc9V/kjIcxe2Yna7qgpWvo70Tfl97hf7NDYt14km+oX/zp0hESw9SD MJ6I7bZw==; Received: from [2001:4bb8:19a:7bdf:8143:492c:c3b:39b6] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nrFE6-000dTK-2P; Wed, 18 May 2022 08:40:14 +0000 From: Christoph Hellwig To: axboe@kernel.dk Cc: asml.silence@gmail.com, io-uring@vger.kernel.org Subject: [PATCH 1/6] io_uring: use a rwf_t for io_rw.flags Date: Wed, 18 May 2022 10:40:00 +0200 Message-Id: <20220518084005.3255380-2-hch@lst.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220518084005.3255380-1-hch@lst.de> References: <20220518084005.3255380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org Remove the bogus __force casts and just use the proper type instead. Signed-off-by: Christoph Hellwig --- fs/io_uring.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 0b05d5aa5891a..99862cbc1041c 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -644,7 +644,7 @@ struct io_rw { struct kiocb kiocb; u64 addr; u32 len; - u32 flags; + rwf_t flags; }; struct io_connect { @@ -3636,7 +3636,7 @@ static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe) req->imu = NULL; req->rw.addr = READ_ONCE(sqe->addr); req->rw.len = READ_ONCE(sqe->len); - req->rw.flags = (__force u32)READ_ONCE(sqe->rw_flags); + req->rw.flags = READ_ONCE(sqe->rw_flags); /* used for fixed read/write too - just read unconditionally */ req->buf_index = READ_ONCE(sqe->buf_index); return 0; @@ -4274,7 +4274,7 @@ static int io_rw_init_file(struct io_kiocb *req, fmode_t mode) req->flags |= io_file_get_flags(file) << REQ_F_SUPPORT_NOWAIT_BIT; kiocb->ki_flags = iocb_flags(file); - ret = kiocb_set_rw_flags(kiocb, (__force rwf_t)req->rw.flags); + ret = kiocb_set_rw_flags(kiocb, req->rw.flags); if (unlikely(ret)) return ret; From patchwork Wed May 18 08:40:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12853350 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4F89C433F5 for ; Wed, 18 May 2022 08:40:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233412AbiERIkU (ORCPT ); Wed, 18 May 2022 04:40:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233424AbiERIkT (ORCPT ); Wed, 18 May 2022 04:40:19 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A52E1238B1 for ; Wed, 18 May 2022 01:40:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=P8s2iw1N3sL3IRMVRz72GXdwH+YWNdJ1pUpX8T3gav4=; b=khvJYo5vmjIDvNXDtCYKldVBk9 iWuzJ8C8A9rV6hQnKkuZy1sYxTszGc1xI2aZzyqtzhALlpWcbz3oOSwxQrTA+ZOBzYbcHp7PODlOh fTSOQTH1Wx6Xxcyd6VjsLLIhU73cIdh2AWHbIYpyLszoZ9ysdxi9R6ciMza0W4hnMmKmivlxzARlU hT1sLAeF2LsJMKCwphzfCBF5VLFZifwtOmhLr2Gb3oFO+FxB1lbgoFD+HuEaCQNRq5dZCY4bObcEo wacDGCeIA4xb4VZ67JeYQPHBpWuSwc7zwaMtRtBHuadVljp8h6rakC/haOJTGwkkBMXDw0Gk7O5pk /Jmx4/Og==; Received: from [2001:4bb8:19a:7bdf:8143:492c:c3b:39b6] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nrFE8-000dTW-P3; Wed, 18 May 2022 08:40:17 +0000 From: Christoph Hellwig To: axboe@kernel.dk Cc: asml.silence@gmail.com, io-uring@vger.kernel.org Subject: [PATCH 2/6] io_uring: don't use ERR_PTR for user pointers Date: Wed, 18 May 2022 10:40:01 +0200 Message-Id: <20220518084005.3255380-3-hch@lst.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220518084005.3255380-1-hch@lst.de> References: <20220518084005.3255380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org ERR_PTR abuses the high bits of a pointer to transport error information. This is only safe for kernel pointers and not user pointers. Fix io_buffer_select and its helpers to just return NULL for failure and get rid of this abuse. Signed-off-by: Christoph Hellwig --- fs/io_uring.c | 83 +++++++++++++++++++++++---------------------------- 1 file changed, 37 insertions(+), 46 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 99862cbc1041c..abb7108258f96 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -3797,11 +3797,8 @@ static int io_buffer_add_list(struct io_ring_ctx *ctx, } static void __user *io_provided_buffer_select(struct io_kiocb *req, size_t *len, - struct io_buffer_list *bl, - unsigned int issue_flags) + struct io_buffer_list *bl) { - void __user *ret = ERR_PTR(-ENOBUFS); - if (!list_empty(&bl->buf_list)) { struct io_buffer *kbuf; @@ -3812,11 +3809,9 @@ static void __user *io_provided_buffer_select(struct io_kiocb *req, size_t *len, req->flags |= REQ_F_BUFFER_SELECTED; req->kbuf = kbuf; req->buf_index = kbuf->bid; - ret = u64_to_user_ptr(kbuf->addr); + return u64_to_user_ptr(kbuf->addr); } - - io_ring_submit_unlock(req->ctx, issue_flags); - return ret; + return NULL; } static void __user *io_ring_buffer_select(struct io_kiocb *req, size_t *len, @@ -3829,7 +3824,7 @@ static void __user *io_ring_buffer_select(struct io_kiocb *req, size_t *len, if (unlikely(smp_load_acquire(&br->tail) == head)) { io_ring_submit_unlock(req->ctx, issue_flags); - return ERR_PTR(-ENOBUFS); + return NULL; } head &= bl->mask; @@ -3847,22 +3842,19 @@ static void __user *io_ring_buffer_select(struct io_kiocb *req, size_t *len, req->buf_list = bl; req->buf_index = buf->bid; - if (!(issue_flags & IO_URING_F_UNLOCKED)) - return u64_to_user_ptr(buf->addr); - - /* - * If we came in unlocked, we have no choice but to - * consume the buffer here. This does mean it'll be - * pinned until the IO completes. But coming in - * unlocked means we're in io-wq context, hence there - * should be no further retry. For the locked case, the - * caller must ensure to call the commit when the - * transfer completes (or if we get -EAGAIN and must - * poll or retry). - */ - req->buf_list = NULL; - bl->head++; - io_ring_submit_unlock(req->ctx, issue_flags); + if (issue_flags & IO_URING_F_UNLOCKED) { + /* + * If we came in unlocked, we have no choice but to consume the + * buffer here. This does mean it'll be pinned until the IO + * completes. But coming in unlocked means we're in io-wq + * context, hence there should be no further retry. For the + * locked case, the caller must ensure to call the commit when + * the transfer completes (or if we get -EAGAIN and must poll + * or retry). + */ + req->buf_list = NULL; + bl->head++; + } return u64_to_user_ptr(buf->addr); } @@ -3871,20 +3863,19 @@ static void __user *io_buffer_select(struct io_kiocb *req, size_t *len, { struct io_ring_ctx *ctx = req->ctx; struct io_buffer_list *bl; + void __user *ret = NULL; io_ring_submit_lock(req->ctx, issue_flags); bl = io_buffer_get_list(ctx, req->buf_index); - if (unlikely(!bl)) { - io_ring_submit_unlock(req->ctx, issue_flags); - return ERR_PTR(-ENOBUFS); + if (likely(bl)) { + if (bl->buf_nr_pages) + ret = io_ring_buffer_select(req, len, bl, issue_flags); + else + ret = io_provided_buffer_select(req, len, bl); } - - /* selection helpers drop the submit lock again, if needed */ - if (bl->buf_nr_pages) - return io_ring_buffer_select(req, len, bl, issue_flags); - - return io_provided_buffer_select(req, len, bl, issue_flags); + io_ring_submit_unlock(req->ctx, issue_flags); + return ret; } #ifdef CONFIG_COMPAT @@ -3906,8 +3897,8 @@ static ssize_t io_compat_import(struct io_kiocb *req, struct iovec *iov, len = clen; buf = io_buffer_select(req, &len, issue_flags); - if (IS_ERR(buf)) - return PTR_ERR(buf); + if (!buf) + return -ENOBUFS; req->rw.addr = (unsigned long) buf; iov[0].iov_base = buf; req->rw.len = iov[0].iov_len = (compat_size_t) len; @@ -3929,8 +3920,8 @@ static ssize_t __io_iov_buffer_select(struct io_kiocb *req, struct iovec *iov, if (len < 0) return -EINVAL; buf = io_buffer_select(req, &len, issue_flags); - if (IS_ERR(buf)) - return PTR_ERR(buf); + if (!buf) + return -ENOBUFS; req->rw.addr = (unsigned long) buf; iov[0].iov_base = buf; req->rw.len = iov[0].iov_len = len; @@ -3987,8 +3978,8 @@ static struct iovec *__io_import_iovec(int rw, struct io_kiocb *req, if (opcode == IORING_OP_READ || opcode == IORING_OP_WRITE) { if (io_do_buffer_select(req)) { buf = io_buffer_select(req, &sqe_len, issue_flags); - if (IS_ERR(buf)) - return ERR_CAST(buf); + if (!buf) + return ERR_PTR(-ENOBUFS); req->rw.addr = (unsigned long) buf; req->rw.len = sqe_len; } @@ -5259,8 +5250,8 @@ static int io_nop(struct io_kiocb *req, unsigned int issue_flags) size_t len = 1; buf = io_buffer_select(req, &len, issue_flags); - if (IS_ERR(buf)) - return PTR_ERR(buf); + if (!buf) + return -ENOBUFS; } if (!(req->ctx->flags & IORING_SETUP_CQE32)) @@ -6394,8 +6385,8 @@ static int io_recvmsg(struct io_kiocb *req, unsigned int issue_flags) void __user *buf; buf = io_buffer_select(req, &sr->len, issue_flags); - if (IS_ERR(buf)) - return PTR_ERR(buf); + if (!buf) + return -ENOBUFS; kmsg->fast_iov[0].iov_base = buf; kmsg->fast_iov[0].iov_len = sr->len; iov_iter_init(&kmsg->msg.msg_iter, READ, kmsg->fast_iov, 1, @@ -6464,8 +6455,8 @@ static int io_recv(struct io_kiocb *req, unsigned int issue_flags) void __user *buf; buf = io_buffer_select(req, &sr->len, issue_flags); - if (IS_ERR(buf)) - return PTR_ERR(buf); + if (!buf) + return -ENOBUFS; sr->buf = buf; } From patchwork Wed May 18 08:40:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12853354 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10E64C433EF for ; Wed, 18 May 2022 08:40:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233424AbiERIkW (ORCPT ); Wed, 18 May 2022 04:40:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233427AbiERIkV (ORCPT ); Wed, 18 May 2022 04:40:21 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 06786126983 for ; Wed, 18 May 2022 01:40:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=O3Hwh9FMK/Y+mVRf2nPDA4yKBB8tjOZN8Y7yWPvmkuY=; b=ryFUQfaaLxMx4y9FsJ7dzJtyF3 qZkotTPtaiiErrFefMMsrS8arylISg3ZPjW1EaKLMv1RZCyP70CRvVJAmodEy5VskFpYB8f5Od15a NYrkuSdy3+DAe3468ADP1pDo45BXrmXI7AsHY731IYyA/s+ajATLDQyifKjm9jQlhpksoeqT/XdfN 8cyrC8tgIkwKnsbzxaO0mxcUrKVI57YtIu1ixnNs0TZGKa8h3UcU9xpm6ZThy8OBpr50rZGbJ8rly 27PO2cl3S1N+W1Cwq7NVj8XIJqchB6hsl6sUTsnSr9/z1gy0A0v3hd+mrO0OJWfVj1yeQpz7RN+gH HdybGynQ==; Received: from [2001:4bb8:19a:7bdf:8143:492c:c3b:39b6] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nrFEB-000dUY-DN; Wed, 18 May 2022 08:40:19 +0000 From: Christoph Hellwig To: axboe@kernel.dk Cc: asml.silence@gmail.com, io-uring@vger.kernel.org Subject: [PATCH 3/6] io_uring: drop a spurious inline on a forward declaration Date: Wed, 18 May 2022 10:40:02 +0200 Message-Id: <20220518084005.3255380-4-hch@lst.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220518084005.3255380-1-hch@lst.de> References: <20220518084005.3255380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org io_file_get_normal isn't marked inline, so don't claim it as such in the forward declaration. Signed-off-by: Christoph Hellwig --- fs/io_uring.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index abb7108258f96..fc435f95ef340 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -1354,7 +1354,7 @@ static int __io_register_rsrc_update(struct io_ring_ctx *ctx, unsigned type, static void io_clean_op(struct io_kiocb *req); static inline struct file *io_file_get_fixed(struct io_kiocb *req, int fd, unsigned issue_flags); -static inline struct file *io_file_get_normal(struct io_kiocb *req, int fd); +static struct file *io_file_get_normal(struct io_kiocb *req, int fd); static void io_drop_inflight_file(struct io_kiocb *req); static bool io_assign_file(struct io_kiocb *req, unsigned int issue_flags); static void io_queue_sqe(struct io_kiocb *req); From patchwork Wed May 18 08:40:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12853351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85BBBC433FE for ; Wed, 18 May 2022 08:40:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233427AbiERIk2 (ORCPT ); Wed, 18 May 2022 04:40:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47192 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233321AbiERIkY (ORCPT ); Wed, 18 May 2022 04:40:24 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 72C84C8BE0 for ; Wed, 18 May 2022 01:40:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=g4fhh/jaoaiXgD0A58bF6TxSzg6yFwnWkwOfK0pR8Q8=; b=L+nASTp0xS4U9STthd9Ou6HGnq CFjHk3HjuJybhNHVnyhNt5dzerWeLtxeXI65UseEdfzUkEmUQhbSuZ1MHYUnZOlBkoE/ybS66vx2A dxnP0JMvT/X84o/JhlZB0UC7URoHC5nT2xK9S7iGXekXgslXVR+wLBU08rCG9RrIDAEi4cprQCazw LVevJM1wY22Bkd2u58wGWp97codEOqDigX6LBYrlu6CF04sBa+3R3H+nEfn1chOWZkWXseXkTYqz2 ymW+A7c+nn5cfX6JXhiqAbjnzGxVr0wG0CNUap9r5BRK2O1dFmBHE5mCmRatvaV/Ln9Ue3xbNjvGJ G6T++Kig==; Received: from [2001:4bb8:19a:7bdf:8143:492c:c3b:39b6] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nrFEE-000dV1-1l; Wed, 18 May 2022 08:40:22 +0000 From: Christoph Hellwig To: axboe@kernel.dk Cc: asml.silence@gmail.com, io-uring@vger.kernel.org Subject: [PATCH 4/6] io_uring: make apoll_events a __poll_t Date: Wed, 18 May 2022 10:40:03 +0200 Message-Id: <20220518084005.3255380-5-hch@lst.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220518084005.3255380-1-hch@lst.de> References: <20220518084005.3255380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org apoll_events is fed to vfs_poll and the poll tables, so it should be a __poll_t. Signed-off-by: Christoph Hellwig --- fs/io_uring.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index fc435f95ef340..1b46c3e9df33a 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -1031,7 +1031,7 @@ struct io_kiocb { /* used by request caches, completion batching and iopoll */ struct io_wq_work_node comp_list; /* cache ->apoll->events */ - int apoll_events; + __poll_t apoll_events; }; atomic_t refs; atomic_t poll_refs; @@ -6977,7 +6977,7 @@ static void io_apoll_task_func(struct io_kiocb *req, bool *locked) io_req_complete_failed(req, ret); } -static void __io_poll_execute(struct io_kiocb *req, int mask, int events) +static void __io_poll_execute(struct io_kiocb *req, int mask, __poll_t events) { req->cqe.res = mask; /* @@ -6996,7 +6996,8 @@ static void __io_poll_execute(struct io_kiocb *req, int mask, int events) io_req_task_work_add(req, false); } -static inline void io_poll_execute(struct io_kiocb *req, int res, int events) +static inline void io_poll_execute(struct io_kiocb *req, int res, + __poll_t events) { if (io_poll_get_ownership(req)) __io_poll_execute(req, res, events); From patchwork Wed May 18 08:40:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12853352 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0F63C433F5 for ; Wed, 18 May 2022 08:40:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233321AbiERIk2 (ORCPT ); Wed, 18 May 2022 04:40:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47214 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233426AbiERIk0 (ORCPT ); Wed, 18 May 2022 04:40:26 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 39EF4C8BE0 for ; Wed, 18 May 2022 01:40:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=vSVwa3ZNe0YJsyTijBcRUJbZ86ohZvBBmKLNw22MzNs=; b=4XO4gTtG8O6xAdYGlMgKVU/ckW MBjYMjraLmG9GOfqvkS38+xwNBJNzHJOKGRux963cHzps0Vf3EUFFQX7yYsi09rJDaxt9tGsPGKWb fWz5AUQq1cmAE7a5I1QrjZBmRFO1hSPs4JHOoDB56rtk9nSoxnjT1rGXPF+rKlb0DqMmDNI6hQkrf DkwPp2z6c6EdwWyCsbM/THMQyyiRkz/PgIx2q8KLnvHNCXtsSRkEkr+oAx2XB6EeGr/PLBr0hpVLh q/muxsLIGkhjGmcIQw1stj8/7/XDiYMS0rQ/w+XW0LG2/Nwv0AT5FPnMXwpEgEc6DQrYsbIgMjKal xVURU0Kg==; Received: from [2001:4bb8:19a:7bdf:8143:492c:c3b:39b6] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nrFEG-000dVd-Ni; Wed, 18 May 2022 08:40:25 +0000 From: Christoph Hellwig To: axboe@kernel.dk Cc: asml.silence@gmail.com, io-uring@vger.kernel.org Subject: [PATCH 5/6] io_uring: consistently use the EPOLL* defines Date: Wed, 18 May 2022 10:40:04 +0200 Message-Id: <20220518084005.3255380-6-hch@lst.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220518084005.3255380-1-hch@lst.de> References: <20220518084005.3255380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org POLL* are unannotated values for the userspace ABI, while everything in-kernel should use EPOLL* and the __poll_t type. Signed-off-by: Christoph Hellwig --- fs/io_uring.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 1b46c3e9df33a..c9596d551bd67 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -7012,7 +7012,7 @@ static void io_poll_cancel_req(struct io_kiocb *req) #define wqe_to_req(wait) ((void *)((unsigned long) (wait)->private & ~1)) #define wqe_is_double(wait) ((unsigned long) (wait)->private & 1) -#define IO_ASYNC_POLL_COMMON (EPOLLONESHOT | POLLPRI) +#define IO_ASYNC_POLL_COMMON (EPOLLONESHOT | EPOLLPRI) static int io_poll_wake(struct wait_queue_entry *wait, unsigned mode, int sync, void *key) @@ -7217,14 +7217,14 @@ static int io_arm_poll_handler(struct io_kiocb *req, unsigned issue_flags) mask |= EPOLLONESHOT; if (def->pollin) { - mask |= POLLIN | POLLRDNORM; + mask |= EPOLLIN | EPOLLRDNORM; /* If reading from MSG_ERRQUEUE using recvmsg, ignore POLLIN */ if ((req->opcode == IORING_OP_RECVMSG) && (req->sr_msg.msg_flags & MSG_ERRQUEUE)) - mask &= ~POLLIN; + mask &= ~EPOLLIN; } else { - mask |= POLLOUT | POLLWRNORM; + mask |= EPOLLOUT | EPOLLWRNORM; } if (def->poll_exclusive) mask |= EPOLLEXCLUSIVE; From patchwork Wed May 18 08:40:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12853353 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5A0EC433FE for ; Wed, 18 May 2022 08:40:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233433AbiERIkb (ORCPT ); Wed, 18 May 2022 04:40:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233426AbiERIk3 (ORCPT ); Wed, 18 May 2022 04:40:29 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD7921078A7 for ; Wed, 18 May 2022 01:40:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=aYeW7fe3tqt7eOZhO6qodf1YLqFtMMH8JlWFnC3q7gs=; b=PAExOTorQo2Vt9Fy4qzCi5Rt/G pDM4nFbNXFUxNksoitZzN17X39tSGIV1fn2vc/DuSU7TboOsc5iA9axdDDloUAZPDu9kmBqZoqC6s iFb9bcn9PLcQ3o06p615GkCMVP6dZK3hB7TbfrVmaLweINxXBWWCMXTsBehznjpFKUG87ZSgrTvKL dpROGVngMlhdHtVg4gxZmEkFKQH1vdMJm/YNg7/ZUrEQIPCsH63x/6HVEp0YeNJM1NqOjjQUsSLz7 dInCAf8fcn0LYUZF3IaLgF9oy8UtymN2tKHn8ERQyFfkaxZ5IsfWabPOrILHp7GMsTe3JOkfdfBm6 5KEdQS7g==; Received: from [2001:4bb8:19a:7bdf:8143:492c:c3b:39b6] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nrFEJ-000dWE-AB; Wed, 18 May 2022 08:40:27 +0000 From: Christoph Hellwig To: axboe@kernel.dk Cc: asml.silence@gmail.com, io-uring@vger.kernel.org Subject: [PATCH 6/6] io_uring: use rcu_dereference in io_close Date: Wed, 18 May 2022 10:40:05 +0200 Message-Id: <20220518084005.3255380-7-hch@lst.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220518084005.3255380-1-hch@lst.de> References: <20220518084005.3255380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org Accessing the file table needs a rcu_dereference_protected(). Signed-off-by: Christoph Hellwig --- fs/io_uring.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index c9596d551bd67..2b848a8dfc46c 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -5984,7 +5984,8 @@ static int io_close(struct io_kiocb *req, unsigned int issue_flags) spin_unlock(&files->file_lock); goto err; } - file = fdt->fd[close->fd]; + file = rcu_dereference_protected(fdt->fd[close->fd], + lockdep_is_held(&files->file_lock)); if (!file || file->f_op == &io_uring_fops) { spin_unlock(&files->file_lock); file = NULL;