From patchwork Fri Aug 13 16:43:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12435843 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 278D7C4338F for ; Fri, 13 Aug 2021 16:43:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0ECD56103A for ; Fri, 13 Aug 2021 16:43:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236922AbhHMQoV (ORCPT ); Fri, 13 Aug 2021 12:44:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40254 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236473AbhHMQoT (ORCPT ); Fri, 13 Aug 2021 12:44:19 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 67E09C061756; Fri, 13 Aug 2021 09:43:52 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id x10so7802894wrt.8; Fri, 13 Aug 2021 09:43:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EaXGPCv4rG5J1NZC9xHyYL1z2IpeWIiNJsgYxDUUWOE=; b=gUnpy4Fd0r0qeQY0j9DV0aCJZjgEiE+I07XDM3P1LbQZ0evIDxiLp7lTovIDXUMVtF Zooe9e6zZ332888X6hRTKZBMFygxC0osuQd1iQcAsdoSNNS7tvPgl3GdR9FH50Nqo2MB g1bwdRxZsJBCklFtwYOYxN0dsSgZzi9i3LtuNwkgrjtL9jx8ol0iSdnGIhSaM1o5yl0i +5AKUsDelEySMuzJVcPrvHrpF9WYxoW480D7r/z6uPTeE6YYeC49cnNBOiFzfqusDvx/ vNDy//lKZDS+gv9omrS7fZK6W7CUCVs1g5NQoOzUTpMTdsjXeiOrQ/4+8Xoofu/Is5Wg U8Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EaXGPCv4rG5J1NZC9xHyYL1z2IpeWIiNJsgYxDUUWOE=; b=qmVwgmJsyemF+Wn1nu/IzIeh6ZZ5VBAwcE5b2d5YN8yNvU4qUkEOdrZiLH6P8aw9r3 DagnXOP3NVZTLdKgR4zKHc4B25kUSwSPjxQ3XJav+/YD7IioIEKcGMfgjQCkVbw0hL3a hZUZM58UeRqYmkT0D4kCE99oPbvoXZaz23gckvGHqqqMnEnK6q2lZA8DZupCAel+Iy8s lXB8IScDmGGgzoGz9B7/uTOL1y51C1HnjdMY8YbfqmkWGL4x1tCRLmBck5H6pA25G1FO kiDOvg3XB/9b02QSQ7RE52jf62OzJjoQa0TF0/xmfUIF3J+DSE6+gj9SEaZ8QE407+Ch 0JSQ== X-Gm-Message-State: AOAM530jZLOmo5INpDDspx1HGZN+LVvL7f8yvP5EKqPWarludkGTrcq5 Wr+p2cuD3fzYtD5fl8Dz98g= X-Google-Smtp-Source: ABdhPJz5tfnJNxOqwfFnaBDDBPs//t7Rss7F1h35igdy1t0QCvEF1HseGfyDmrdA3pBdzeVoJCgtcA== X-Received: by 2002:a05:6000:1a86:: with SMTP id f6mr4276552wry.345.1628873031039; Fri, 13 Aug 2021 09:43:51 -0700 (PDT) Received: from localhost.localdomain ([148.252.132.210]) by smtp.gmail.com with ESMTPSA id s10sm2495829wrv.54.2021.08.13.09.43.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 09:43:50 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , io-uring@vger.kernel.org, Josh Triplett Cc: "David S . Miller" , Jakub Kicinski , linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Stefan Metzmacher Subject: [PATCH v2 1/4] net: add accept helper not installing fd Date: Fri, 13 Aug 2021 17:43:10 +0100 Message-Id: <2fa1157f9fa4ed89ebe673a9dfed81385ba63811.1628871893.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Introduce and reuse a helper that acts similarly to __sys_accept4_file() but returns struct file instead of installing file descriptor. Will be used by io_uring. Signed-off-by: Pavel Begunkov --- include/linux/socket.h | 3 ++ net/socket.c | 71 ++++++++++++++++++++++-------------------- 2 files changed, 40 insertions(+), 34 deletions(-) diff --git a/include/linux/socket.h b/include/linux/socket.h index 0d8e3dcb7f88..d3c1a42a2edd 100644 --- a/include/linux/socket.h +++ b/include/linux/socket.h @@ -421,6 +421,9 @@ extern int __sys_accept4_file(struct file *file, unsigned file_flags, struct sockaddr __user *upeer_sockaddr, int __user *upeer_addrlen, int flags, unsigned long nofile); +extern struct file *do_accept(struct file *file, unsigned file_flags, + struct sockaddr __user *upeer_sockaddr, + int __user *upeer_addrlen, int flags); extern int __sys_accept4(int fd, struct sockaddr __user *upeer_sockaddr, int __user *upeer_addrlen, int flags); extern int __sys_socket(int family, int type, int protocol); diff --git a/net/socket.c b/net/socket.c index 0b2dad3bdf7f..532fff5a3684 100644 --- a/net/socket.c +++ b/net/socket.c @@ -1722,32 +1722,22 @@ SYSCALL_DEFINE2(listen, int, fd, int, backlog) return __sys_listen(fd, backlog); } -int __sys_accept4_file(struct file *file, unsigned file_flags, +struct file *do_accept(struct file *file, unsigned file_flags, struct sockaddr __user *upeer_sockaddr, - int __user *upeer_addrlen, int flags, - unsigned long nofile) + int __user *upeer_addrlen, int flags) { struct socket *sock, *newsock; struct file *newfile; - int err, len, newfd; + int err, len; struct sockaddr_storage address; - if (flags & ~(SOCK_CLOEXEC | SOCK_NONBLOCK)) - return -EINVAL; - - if (SOCK_NONBLOCK != O_NONBLOCK && (flags & SOCK_NONBLOCK)) - flags = (flags & ~SOCK_NONBLOCK) | O_NONBLOCK; - sock = sock_from_file(file); - if (!sock) { - err = -ENOTSOCK; - goto out; - } + if (!sock) + return ERR_PTR(-ENOTSOCK); - err = -ENFILE; newsock = sock_alloc(); if (!newsock) - goto out; + return ERR_PTR(-ENFILE); newsock->type = sock->type; newsock->ops = sock->ops; @@ -1758,18 +1748,9 @@ int __sys_accept4_file(struct file *file, unsigned file_flags, */ __module_get(newsock->ops->owner); - newfd = __get_unused_fd_flags(flags, nofile); - if (unlikely(newfd < 0)) { - err = newfd; - sock_release(newsock); - goto out; - } newfile = sock_alloc_file(newsock, flags, sock->sk->sk_prot_creator->name); - if (IS_ERR(newfile)) { - err = PTR_ERR(newfile); - put_unused_fd(newfd); - goto out; - } + if (IS_ERR(newfile)) + return newfile; err = security_socket_accept(sock, newsock); if (err) @@ -1794,16 +1775,38 @@ int __sys_accept4_file(struct file *file, unsigned file_flags, } /* File flags are not inherited via accept() unlike another OSes. */ - - fd_install(newfd, newfile); - err = newfd; -out: - return err; + return newfile; out_fd: fput(newfile); - put_unused_fd(newfd); - goto out; + return ERR_PTR(err); +} + +int __sys_accept4_file(struct file *file, unsigned file_flags, + struct sockaddr __user *upeer_sockaddr, + int __user *upeer_addrlen, int flags, + unsigned long nofile) +{ + struct file *newfile; + int newfd; + if (flags & ~(SOCK_CLOEXEC | SOCK_NONBLOCK)) + return -EINVAL; + + if (SOCK_NONBLOCK != O_NONBLOCK && (flags & SOCK_NONBLOCK)) + flags = (flags & ~SOCK_NONBLOCK) | O_NONBLOCK; + + newfd = __get_unused_fd_flags(flags, nofile); + if (unlikely(newfd < 0)) + return newfd; + + newfile = do_accept(file, file_flags, upeer_sockaddr, upeer_addrlen, + flags); + if (IS_ERR(newfile)) { + put_unused_fd(newfd); + return PTR_ERR(newfile); + } + fd_install(newfd, newfile); + return newfd; } /* From patchwork Fri Aug 13 16:43:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12435849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9A48C432BE for ; Fri, 13 Aug 2021 16:44:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A3841610FD for ; Fri, 13 Aug 2021 16:44:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237666AbhHMQoZ (ORCPT ); Fri, 13 Aug 2021 12:44:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40260 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236895AbhHMQoV (ORCPT ); Fri, 13 Aug 2021 12:44:21 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA734C061756; Fri, 13 Aug 2021 09:43:53 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id f9-20020a05600c1549b029025b0f5d8c6cso10071312wmg.4; Fri, 13 Aug 2021 09:43:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CNqoqrJYYQRcV/CRZqwgpOungHLgzziX626nbxoBaQM=; b=pSXtyls3J5jBvGn4dQz98QK2mqVDeqdIJLXPtDa1mqwSit+6R3pChg8YE5IWZCh5QA /O5pB1sN6ZJQdEKsPCJPcdCdumGpJ1pi0b8onrOTxzdoBc0GXt9kARKd6KUfRBCJPe1G +SFC001kdrWWiDwQqXt1KcTC3yOyij+XNBFK3jJqitaSRhHo+xE66OXSCNrwRzeFzB/9 dI+qUnvBUVJ4UeG9Lg50eH4dDhmQE3V/EZZheTCxGrnytBh79lWAW9jkkqodcwgfOEHH FHF9K/KwnyHB284M3XDSMT/CfaAMLR+9DjnDvN4MHr/+p/y21DphII5LMj364kY0JNud sGxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CNqoqrJYYQRcV/CRZqwgpOungHLgzziX626nbxoBaQM=; b=Hwu7beqJuYUGsgUuINseGuWKWabLdXSIkB8xfnFaRfBdr2oSSCTaAX26gltxllbMGi pDqtNImY3rDNBWYC0LioKKe+qi8b/oDie7ANZjak/pXXgAMEJM7elrL5HLGqhyt1pfvs iwMjL/T+SLbjrde8ZHvNtoDv7Mzfdwg/yH7pWd/FCSXYtrTPlSeBcTgH3tvWRJJKbku1 caEFJkI8W4EWrMtirkXisugpkojjTtnFNwDen6rikaidcd6YkdSpzXcg71K/v1VEKs8W O+EPNOscY7RvRVk1VqPMwcLGcCCpcPzvdwDOOLJS/Nj35gAnI0WzsG8ZEcKdg5b05YYv STVg== X-Gm-Message-State: AOAM532sB5m2OaZfsP342e+TFbw8B0DorvlPEZtl+9w7Km1Ms/77YWyy 0NKnJJooQYFjRBUAEvdA1YM= X-Google-Smtp-Source: ABdhPJxEKCk04KMoa0igrVrqwQmVAdRXy+TYG0qKKgWtGE57PIq8JmJmIbTRV2ze0jeJxsGceAkBzg== X-Received: by 2002:a05:600c:19c6:: with SMTP id u6mr3569811wmq.154.1628873032417; Fri, 13 Aug 2021 09:43:52 -0700 (PDT) Received: from localhost.localdomain ([148.252.132.210]) by smtp.gmail.com with ESMTPSA id s10sm2495829wrv.54.2021.08.13.09.43.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 09:43:52 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , io-uring@vger.kernel.org, Josh Triplett Cc: "David S . Miller" , Jakub Kicinski , linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Stefan Metzmacher Subject: [PATCH v2 2/4] io_uring: openat directly into fixed fd table Date: Fri, 13 Aug 2021 17:43:11 +0100 Message-Id: <686a860d973d0b751cf1fa3b1408179b2833720f.1628871893.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Instead of opening a file into a process's file table as usual and then registering the fd within io_uring, some users may want to skip the first step and place it directly into io_uring's fixed file table. This patch adds such a capability for IORING_OP_OPENAT and IORING_OP_OPENAT2. The behaviour is controlled by setting sqe->file_index, where 0 implies the old behaviour. If non-zero value is specified, then it will behave as described and place the file into a fixed file slot sqe->file_index - 1. A file table should be already created, the slot should be valid and empty, otherwise the operation will fail. Note 1: we can't use IOSQE_FIXED_FILE to switch between modes, because accept takes a file, and it already uses the flag with a different meaning. Note 2: it's u16, where in theory the limit for fixed file tables might get increased in the future. If would ever happen so, we'll better workaround later, e.g. by making ioprio to represent upper bits 16 bits. The layout for open is tight already enough. Suggested-by: Josh Triplett Signed-off-by: Pavel Begunkov --- fs/io_uring.c | 76 ++++++++++++++++++++++++++++++----- include/uapi/linux/io_uring.h | 2 + 2 files changed, 69 insertions(+), 9 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 51c4166f68b5..b4f7de5147dc 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -1060,6 +1060,9 @@ static void io_req_task_queue(struct io_kiocb *req); static void io_submit_flush_completions(struct io_ring_ctx *ctx); static int io_req_prep_async(struct io_kiocb *req); +static int io_install_fixed_file(struct io_kiocb *req, struct file *file, + unsigned int issue_flags); + static struct kmem_cache *req_cachep; static const struct file_operations io_uring_fops; @@ -3803,11 +3806,10 @@ static int __io_openat_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe if (unlikely(req->ctx->flags & IORING_SETUP_IOPOLL)) return -EINVAL; - if (unlikely(sqe->ioprio || sqe->buf_index)) + if (unlikely(sqe->ioprio)) return -EINVAL; if (unlikely(req->flags & REQ_F_FIXED_FILE)) return -EBADF; - /* open.how should be already initialised */ if (!(req->open.how.flags & O_PATH) && force_o_largefile()) req->open.how.flags |= O_LARGEFILE; @@ -3820,6 +3822,10 @@ static int __io_openat_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe req->open.filename = NULL; return ret; } + req->buf_index = READ_ONCE(sqe->file_index); + if (req->buf_index && (req->open.how.flags & O_CLOEXEC)) + return -EINVAL; + req->open.nofile = rlimit(RLIMIT_NOFILE); req->flags |= REQ_F_NEED_CLEANUP; return 0; @@ -3857,8 +3863,8 @@ static int io_openat2(struct io_kiocb *req, unsigned int issue_flags) { struct open_flags op; struct file *file; - bool nonblock_set; - bool resolve_nonblock; + bool resolve_nonblock, nonblock_set; + bool fixed = !!req->buf_index; int ret; ret = build_open_flags(&req->open.how, &op); @@ -3877,9 +3883,11 @@ static int io_openat2(struct io_kiocb *req, unsigned int issue_flags) op.open_flag |= O_NONBLOCK; } - ret = __get_unused_fd_flags(req->open.how.flags, req->open.nofile); - if (ret < 0) - goto err; + if (!fixed) { + ret = __get_unused_fd_flags(req->open.how.flags, req->open.nofile); + if (ret < 0) + goto err; + } file = do_filp_open(req->open.dfd, req->open.filename, &op); if (IS_ERR(file)) { @@ -3888,7 +3896,8 @@ static int io_openat2(struct io_kiocb *req, unsigned int issue_flags) * marginal gain for something that is now known to be a slower * path. So just put it, and we'll get a new one when we retry. */ - put_unused_fd(ret); + if (!fixed) + put_unused_fd(ret); ret = PTR_ERR(file); /* only retry if RESOLVE_CACHED wasn't already set by application */ @@ -3901,7 +3910,11 @@ static int io_openat2(struct io_kiocb *req, unsigned int issue_flags) if ((issue_flags & IO_URING_F_NONBLOCK) && !nonblock_set) file->f_flags &= ~O_NONBLOCK; fsnotify_open(file); - fd_install(ret, file); + + if (!fixed) + fd_install(ret, file); + else + ret = io_install_fixed_file(req, file, issue_flags); err: putname(req->open.filename); req->flags &= ~REQ_F_NEED_CLEANUP; @@ -7835,6 +7848,50 @@ static int io_sqe_file_register(struct io_ring_ctx *ctx, struct file *file, #endif } +static int io_install_fixed_file(struct io_kiocb *req, struct file *file, + unsigned int issue_flags) +{ + struct io_ring_ctx *ctx = req->ctx; + int i = req->buf_index - 1; + bool force_nonblock = issue_flags & IO_URING_F_NONBLOCK; + struct io_fixed_file *file_slot; + int ret = -EBADF; + + if (WARN_ON_ONCE(req->buf_index == 0)) + goto err; + + io_ring_submit_lock(ctx, !force_nonblock); + if (file->f_op == &io_uring_fops) + goto err; + ret = -ENXIO; + if (!ctx->file_data) + goto err; + ret = -EINVAL; + if (i > ctx->nr_user_files) + goto err; + + i = array_index_nospec(i, ctx->nr_user_files); + file_slot = io_fixed_file_slot(&ctx->file_table, i); + ret = -EEXIST; + if (file_slot->file_ptr) + goto err; + + *io_get_tag_slot(ctx->file_data, i) = 0; + io_fixed_file_set(file_slot, file); + ret = io_sqe_file_register(ctx, file, i); + if (ret) { + file_slot->file_ptr = 0; + goto err; + } + + ret = 0; +err: + io_ring_submit_unlock(ctx, !force_nonblock); + if (ret) + fput(file); + return ret; +} + static int io_queue_rsrc_removal(struct io_rsrc_data *data, unsigned idx, struct io_rsrc_node *node, void *rsrc) { @@ -10298,6 +10355,7 @@ static int __init io_uring_init(void) BUILD_BUG_SQE_ELEM(32, __u64, user_data); BUILD_BUG_SQE_ELEM(40, __u16, buf_index); BUILD_BUG_SQE_ELEM(40, __u16, buf_group); + BUILD_BUG_SQE_ELEM(40, __u16, file_index); BUILD_BUG_SQE_ELEM(42, __u16, personality); BUILD_BUG_SQE_ELEM(44, __s32, splice_fd_in); diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h index 79126d5cd289..f105deb4da1d 100644 --- a/include/uapi/linux/io_uring.h +++ b/include/uapi/linux/io_uring.h @@ -52,6 +52,8 @@ struct io_uring_sqe { __u16 buf_index; /* for grouped buffer selection */ __u16 buf_group; + /* index into fixed files */ + __u16 file_index; } __attribute__((packed)); /* personality to use, if used */ __u16 personality; From patchwork Fri Aug 13 16:43:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12435847 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C446C432BE for ; Fri, 13 Aug 2021 16:44:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 37F51610EA for ; Fri, 13 Aug 2021 16:44:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237860AbhHMQo0 (ORCPT ); Fri, 13 Aug 2021 12:44:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40266 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237118AbhHMQoW (ORCPT ); Fri, 13 Aug 2021 12:44:22 -0400 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D00BBC061756; Fri, 13 Aug 2021 09:43:54 -0700 (PDT) Received: by mail-wm1-x336.google.com with SMTP id l34-20020a05600c1d22b02902573c214807so10089435wms.2; Fri, 13 Aug 2021 09:43:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NelZKyRIOnA4dGpeG33lwddD90VVcPtbBw8as+67pdQ=; b=pMGbJX2KgNHCeBUnxAsO9J5dTQ34bwvP7JE9+fORl7tgeh9h2uacIQ9rgWTDnwOIKR wQ3RmvEAq2qTu2FCWcgHFRAIV8KpEoP4S1sLXCmOyiricKEL+uXADqPYfupMfY96NAJw /0Q5sza2eBEd/6FiNN0hyRRBxXkpTzU2Vp8Ed02KA9APkIrd0Xr8+f2gtrhY7LBk8DIF lO75QwldNKzMY/TWiwf+2bMxcw6T8Hq8oZ3zNJRwpORyBcPqf+tufIa/t9poVUDszdsE DOAptjnx3luJkr7YHQSBf4UZvO+riyDmJaXVf+Daj+ZoPIzHjQ+kgiAXDgoCambOo4t7 mU1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NelZKyRIOnA4dGpeG33lwddD90VVcPtbBw8as+67pdQ=; b=eV0ctbL6ftFe5JgMtPtL720tX1e/F410ULPW41yDLNyGSOY20o/ofXJb7eqLG7MhwS y/F0ZRoF35ysW1D8El1EoZFfBzN/qk2QOva8DDm9PivqrBQOea6+Nw/TbQggB+3+Ehp2 08HTIKCs/n3CyybUA3NY0RgJXfpXKzBZZF2aMjPTXuS0kY6C1KkCqhUhHNrLvMJRKshi AGwdcr/IzMn05LUZGcdcn7zEw7879wvvFH7lT2CKC9MMkMCYlkTYSCDqZjlO0OVU+Bj+ PAqbjde3NL3RWSd6vGwnkjOihM/ZNX6nk13WfELqGh3M40s07ZwcoXeMmgZNkDAwTkQH ZKEQ== X-Gm-Message-State: AOAM533H1W3XpIE2pwZTSxjgQhx4Qi+eTOIz9MnXBV0O4RaDc9qUItOZ xchUpjrHgHUvWmyXPNk/RiE= X-Google-Smtp-Source: ABdhPJwLklC5mLfwRq9IGHnBA4JB7RU/Cb9lir3Gt0KFaponZBJJr68qMiG+8cv6lMCWJZ1J+GjGuQ== X-Received: by 2002:a1c:1b87:: with SMTP id b129mr3450129wmb.80.1628873033531; Fri, 13 Aug 2021 09:43:53 -0700 (PDT) Received: from localhost.localdomain ([148.252.132.210]) by smtp.gmail.com with ESMTPSA id s10sm2495829wrv.54.2021.08.13.09.43.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 09:43:53 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , io-uring@vger.kernel.org, Josh Triplett Cc: "David S . Miller" , Jakub Kicinski , linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Stefan Metzmacher Subject: [PATCH v2 3/4] io_uring: hand code io_accept() fd installing Date: Fri, 13 Aug 2021 17:43:12 +0100 Message-Id: <72145792a5db2cd3511e45234a3b2f3302e69459.1628871893.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Make io_accept() to handle file descriptor allocations and installation. A preparation patch for bypassing file tables. Signed-off-by: Pavel Begunkov --- fs/io_uring.c | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index b4f7de5147dc..f92adfbc9a6b 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -4743,6 +4743,11 @@ static int io_accept_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) accept->addr_len = u64_to_user_ptr(READ_ONCE(sqe->addr2)); accept->flags = READ_ONCE(sqe->accept_flags); accept->nofile = rlimit(RLIMIT_NOFILE); + + if (accept->flags & ~(SOCK_CLOEXEC | SOCK_NONBLOCK)) + return -EINVAL; + if (SOCK_NONBLOCK != O_NONBLOCK && (accept->flags & SOCK_NONBLOCK)) + accept->flags = (accept->flags & ~SOCK_NONBLOCK) | O_NONBLOCK; return 0; } @@ -4751,20 +4756,28 @@ static int io_accept(struct io_kiocb *req, unsigned int issue_flags) struct io_accept *accept = &req->accept; bool force_nonblock = issue_flags & IO_URING_F_NONBLOCK; unsigned int file_flags = force_nonblock ? O_NONBLOCK : 0; - int ret; + struct file *file; + int ret, fd; if (req->file->f_flags & O_NONBLOCK) req->flags |= REQ_F_NOWAIT; - ret = __sys_accept4_file(req->file, file_flags, accept->addr, - accept->addr_len, accept->flags, - accept->nofile); - if (ret == -EAGAIN && force_nonblock) - return -EAGAIN; - if (ret < 0) { + fd = __get_unused_fd_flags(accept->flags, accept->nofile); + if (unlikely(fd < 0)) + return fd; + + file = do_accept(req->file, file_flags, accept->addr, accept->addr_len, + accept->flags); + if (IS_ERR(file)) { + ret = PTR_ERR(file); + if (ret == -EAGAIN && force_nonblock) + return -EAGAIN; if (ret == -ERESTARTSYS) ret = -EINTR; req_set_fail(req); + } else { + fd_install(fd, file); + ret = fd; } __io_req_complete(req, issue_flags, ret, 0); return 0; From patchwork Fri Aug 13 16:43:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 12435845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B414CC4320A for ; Fri, 13 Aug 2021 16:44:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 955686103A for ; Fri, 13 Aug 2021 16:44:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238025AbhHMQo2 (ORCPT ); Fri, 13 Aug 2021 12:44:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40274 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237262AbhHMQoX (ORCPT ); Fri, 13 Aug 2021 12:44:23 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F36EAC0617AD; Fri, 13 Aug 2021 09:43:55 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id r7so14188157wrs.0; Fri, 13 Aug 2021 09:43:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KGUf89VstkMW69bAYZr9iyqMebO2WnyTgQSpQt32rdk=; b=Nke1JSVLr4L4GXr63jVCc9h5JkTOZ+BeTR0rq2ONhwX5DkN10gSKNtSJU9mAJHldGK Cjsmk+Wdn/kYKpeuhi4U5Gl9aJAYbqC13cfN1U/gDDv7CJrI2u9hiU6JwpFAxxC3Q82f TTVWBDUKwoTvruCeOmrNxNrC+LE/4Sc+7s2oyvSVI5pkeA9Gdaa4qbcKnOT3XnaoDoKF ma438YO9zIC0Rj3/RS3pFY2V4At+RqRS0NZwhPb34ju93WmQ99n8OY5OuvbZCkaIV6Ku hwS2dOunq9g9Er+fNWaDa0e1ydZhyakYx8a2TD4x5NngUwU+DQz39uphJnNRtjIwH0yb Ah1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KGUf89VstkMW69bAYZr9iyqMebO2WnyTgQSpQt32rdk=; b=m7Ht6A3YDkqlsraBxKBs0keCj2ay595lQgLwtC94p8ebMK6RNUxkE/ghbicHr7WaaV GK37xTt9//OWbZgxsClKxyJb7mYmFj9kRnU7M43oDkb1dJPW53J35//fM2l0xYCgCJsZ ZUS14ydxfX2jtrASEDEzEAMQSrk03tTLNY1pkPDClOGzhNS7R655tbYIIdsTWaZz6rc2 oruHYVyXKxBK5YOoTb2EviA4fcva2Dh1k7IgPCSVHFNdW+X/QQ4mS9hv1yJkwGExhfKW YPVvRbUlB4Orrx9BJ2J8anDVWCnDfWe5ghElgs6wU72nJTRquvMohIT49cbwDhygo0jM hgMg== X-Gm-Message-State: AOAM532t60Jyhu2dNkR0WHSPpP8aOJruD9J895WkESLryhmiLwy5FBGp nr/hiokkzMWsEhE8zMFj16FNji08pDw= X-Google-Smtp-Source: ABdhPJyXDhiUH3fMG+2Ci4GnXD3FELWqG6MpVUuan6xzGoz5/bsaQI65YrKshWpcGocHOQ+JzwC+Gg== X-Received: by 2002:a5d:4b49:: with SMTP id w9mr4276354wrs.242.1628873034660; Fri, 13 Aug 2021 09:43:54 -0700 (PDT) Received: from localhost.localdomain ([148.252.132.210]) by smtp.gmail.com with ESMTPSA id s10sm2495829wrv.54.2021.08.13.09.43.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 09:43:54 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , io-uring@vger.kernel.org, Josh Triplett Cc: "David S . Miller" , Jakub Kicinski , linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Stefan Metzmacher Subject: [PATCH v2 4/4] io_uring: accept directly into fixed file table Date: Fri, 13 Aug 2021 17:43:13 +0100 Message-Id: X-Mailer: git-send-email 2.32.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org As done with open opcodes, allow accept to skip installing fd into processes' file tables and put it directly into io_uring's fixed file table. Same restrictions and design as for open. Suggested-by: Josh Triplett Signed-off-by: Pavel Begunkov --- fs/io_uring.c | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index f92adfbc9a6b..0e6189864d12 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -4736,14 +4736,17 @@ static int io_accept_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) if (unlikely(req->ctx->flags & IORING_SETUP_IOPOLL)) return -EINVAL; - if (sqe->ioprio || sqe->len || sqe->buf_index) + if (sqe->ioprio || sqe->len) return -EINVAL; accept->addr = u64_to_user_ptr(READ_ONCE(sqe->addr)); accept->addr_len = u64_to_user_ptr(READ_ONCE(sqe->addr2)); accept->flags = READ_ONCE(sqe->accept_flags); accept->nofile = rlimit(RLIMIT_NOFILE); + req->buf_index = READ_ONCE(sqe->file_index); + if (req->buf_index && (accept->flags & SOCK_CLOEXEC)) + return -EINVAL; if (accept->flags & ~(SOCK_CLOEXEC | SOCK_NONBLOCK)) return -EINVAL; if (SOCK_NONBLOCK != O_NONBLOCK && (accept->flags & SOCK_NONBLOCK)) @@ -4756,28 +4759,34 @@ static int io_accept(struct io_kiocb *req, unsigned int issue_flags) struct io_accept *accept = &req->accept; bool force_nonblock = issue_flags & IO_URING_F_NONBLOCK; unsigned int file_flags = force_nonblock ? O_NONBLOCK : 0; + bool fixed = !!req->buf_index; struct file *file; int ret, fd; if (req->file->f_flags & O_NONBLOCK) req->flags |= REQ_F_NOWAIT; - fd = __get_unused_fd_flags(accept->flags, accept->nofile); - if (unlikely(fd < 0)) - return fd; - + if (!fixed) { + fd = __get_unused_fd_flags(accept->flags, accept->nofile); + if (unlikely(fd < 0)) + return fd; + } file = do_accept(req->file, file_flags, accept->addr, accept->addr_len, accept->flags); if (IS_ERR(file)) { + if (!fixed) + put_unused_fd(fd); ret = PTR_ERR(file); if (ret == -EAGAIN && force_nonblock) return -EAGAIN; if (ret == -ERESTARTSYS) ret = -EINTR; req_set_fail(req); - } else { + } else if (!fixed) { fd_install(fd, file); ret = fd; + } else { + ret = io_install_fixed_file(req, file, issue_flags); } __io_req_complete(req, issue_flags, ret, 0); return 0;