From patchwork Fri Aug 11 12:53:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13350836 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1949BC001DE for ; Fri, 11 Aug 2023 12:55:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232009AbjHKMzO (ORCPT ); Fri, 11 Aug 2023 08:55:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51700 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229503AbjHKMzO (ORCPT ); Fri, 11 Aug 2023 08:55:14 -0400 Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F305D114 for ; Fri, 11 Aug 2023 05:55:12 -0700 (PDT) Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-99c4923195dso270795566b.2 for ; Fri, 11 Aug 2023 05:55:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691758511; x=1692363311; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xViQ+UkUWQ0OXWR1P9K62Qd578cNB7ZDE1O1vUIN/qo=; b=shUpGLPyFf8zQn6qw22UJYkIqOF+D2gro1W0m5PPXAkb5YdgVM2I/pbL2uxDZ3brTm JTCBI+I2oc3qdcc5i+NLucRPwuIF4OBcumcAgNtj/L54qm5zx9YS3IKJa6glPcD5qKU9 KCOOyzp3PmEWEtVkfd8XGZrptK6IvyKpze88YOxUTA1CYo9BZ5pENh8gx/Pjvd33uMer B5vILywQig8DHdyCzaM/YuSVBbKAJgkhl2pNiY1o9E5qqeMfXu5fEbLuuCFr0p1fCG0B GGnb5VZbh9bor0qY3DqXnSFj0y3pllePC/XQMrWNrDh33Ceap1SqL/YIZzhctHMMHLL6 nDMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691758511; x=1692363311; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xViQ+UkUWQ0OXWR1P9K62Qd578cNB7ZDE1O1vUIN/qo=; b=Jbdw9mNINHoJlWoT+UBvYIjTd9yQTzZLF/rt9pbo751qaqmwMiOvoE60+crqEWO8fe b66MRC8A3LTCmNp/2S3rEgwSv0L7fhzauZR41bLE1sfVWcOnPTf7QpPtWJO6oOD1lhuy h7phBbpLjA4j7hFaicnNrk9D1Ng+3CjsSwyoSnQCBJwS4hhaMzxfiI4Hv/SVPuMVBihq w7rfixCM5p/eaQ0kMD+3YWf/3ZtuWcZkAs34kjqor76hSwA8ZPwCaCmN9uM5Ql4cKdik pYQh7qWhTImV5ny9j63a2uBjwg3Mmp1BZJS6lOVD+GyzknfC2xEaE3QGLSbKVR9+QHk2 qXyA== X-Gm-Message-State: AOJu0Yz1d4FiX/6+byAfW0/lBStVjk4kEI3h0jJ2RBS+xNKM98d9cHtV Zs8HEnC7IiVsC+ugI3Qx2syQZkGkk90= X-Google-Smtp-Source: AGHT+IGLntDlh/d7T3z+iMePP2P+zb/maxJeE2zAEHDJOHl0ypAU6XduapsDCwMrPL5ue7SGegp+jA== X-Received: by 2002:a17:907:270e:b0:99b:bc51:8ca3 with SMTP id w14-20020a170907270e00b0099bbc518ca3mr1686677ejk.1.1691758511186; Fri, 11 Aug 2023 05:55:11 -0700 (PDT) Received: from 127.com ([2620:10d:c092:600::2:a57e]) by smtp.gmail.com with ESMTPSA id kk9-20020a170907766900b0099cc36c4681sm2206943ejc.157.2023.08.11.05.55.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Aug 2023 05:55:11 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com Subject: [PATCH 1/7] io_uring/net: don't overflow multishot accept Date: Fri, 11 Aug 2023 13:53:41 +0100 Message-ID: <7d0d749649244873772623dd7747966f516fe6e2.1691757663.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org Don't allow overflowing multishot accept CQEs, we want to limit the grows of the overflow list. Cc: stable@vger.kernel.org Fixes: 4e86a2c980137 ("io_uring: implement multishot mode for accept") Signed-off-by: Pavel Begunkov --- io_uring/net.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/io_uring/net.c b/io_uring/net.c index eb1f51ddcb23..1599493544a5 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -1367,7 +1367,7 @@ int io_accept(struct io_kiocb *req, unsigned int issue_flags) if (ret < 0) return ret; if (io_aux_cqe(req, issue_flags & IO_URING_F_COMPLETE_DEFER, ret, - IORING_CQE_F_MORE, true)) + IORING_CQE_F_MORE, false)) goto retry; return -ECANCELED; From patchwork Fri Aug 11 12:53:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13350837 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7899C41513 for ; Fri, 11 Aug 2023 12:55:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229503AbjHKMzP (ORCPT ); Fri, 11 Aug 2023 08:55:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51720 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233952AbjHKMzO (ORCPT ); Fri, 11 Aug 2023 08:55:14 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C04A126A0 for ; Fri, 11 Aug 2023 05:55:13 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id a640c23a62f3a-99bdcade7fbso268288766b.1 for ; Fri, 11 Aug 2023 05:55:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691758512; x=1692363312; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=93K4NPI31nusFtmR4h3522J9OQwMlEfdCozpShx5wv0=; b=VVKyMkrRHSYe1wYEvaxFOyn6lpv9HT3enjhuKVnvXkQZzpoGa+kf/pO555m0d6JIRu k0lV9w/VpuVZ3FcA0kWbk3WVv/gcaLtnfSKVG5kaaJVdwu4ygpphDRSth51Ct1jtwMLy VHHBE7F7jAxZuf6sHmEeRxZ16yv6vKGJELeCoC2jmpizxP3AAVwQjLGzZ5QuOvcgSeSA CCMhUWsXfw1DMB10mTB7uYJjYjjM0Elflfqq32JrPzdzuBhv/xUnyoa86fzFEbc9L7GV /etLbYXoM2p3ZEhJAAHDAouSszSXlk+RqvGkEoCQOsXtC3Ndg6n9su0ZojRS3cl/YVvI +ftQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691758512; x=1692363312; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=93K4NPI31nusFtmR4h3522J9OQwMlEfdCozpShx5wv0=; b=eb8pHPSI9jhocx1aYmQuadR32wz8zmjuBVOwZUcb6vXmkvm/BWLw5nO+c1OOGUAvNn 4XUqMnEgkHLy7NB0bFi3e77AFx0nO6HpsOIkTYhdtVzFkxolRvd9L1yG8IxdJtU7scIt gNUxBO+HBkE/J+fJkV4k1/diKa8Zjpx9MbayyKOeK5/+DMm8AnK4Ghu2cds8ocXvvo6e AoVaUPzvFTEHMHxBs0Max/3ZezS+pvcH1y3/sOK7tpj/cBWY7B+kTlY8qKYKXMWybdG0 tMvR8mR1VxvhaV/yKPKsWUHsPF+s4H90YvLr7B758dXr24Oc1+XjhkF00JC6y2A4KZVK U/+w== X-Gm-Message-State: AOJu0Yz/rL8u2wc5whqDRgd4qCkDX0KnGlhQkQZg/JZ9QAY0ln0J2q3J 4USD8qbLXjHjCko3SAZA/ykB/gVa2Po= X-Google-Smtp-Source: AGHT+IFcA0t4lPMJM6ctIuBsNBsyX4bxgp+Q027iqZEPVqGyVMhQkS99+nyMdU2ypH/MKtJRp1rK/A== X-Received: by 2002:a17:906:3112:b0:99c:ca27:c447 with SMTP id 18-20020a170906311200b0099cca27c447mr1493421ejx.43.1691758512027; Fri, 11 Aug 2023 05:55:12 -0700 (PDT) Received: from 127.com ([2620:10d:c092:600::2:a57e]) by smtp.gmail.com with ESMTPSA id kk9-20020a170907766900b0099cc36c4681sm2206943ejc.157.2023.08.11.05.55.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Aug 2023 05:55:11 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com Subject: [PATCH 2/7] io_uring/net: don't overflow multishot recv Date: Fri, 11 Aug 2023 13:53:42 +0100 Message-ID: <0b295634e8f1b71aa764c984608c22d85f88f75c.1691757663.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org Don't allow overflowing multishot recv CQEs, it might get out of hand, hurt performanece, and in the worst case scenario OOM the task. Cc: stable@vger.kernel.org Fixes: b3fdea6ecb55c ("io_uring: multishot recv") Signed-off-by: Pavel Begunkov --- io_uring/net.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/io_uring/net.c b/io_uring/net.c index 1599493544a5..8c419c01a5db 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -642,7 +642,7 @@ static inline bool io_recv_finish(struct io_kiocb *req, int *ret, if (!mshot_finished) { if (io_aux_cqe(req, issue_flags & IO_URING_F_COMPLETE_DEFER, - *ret, cflags | IORING_CQE_F_MORE, true)) { + *ret, cflags | IORING_CQE_F_MORE, false)) { io_recv_prep_retry(req); /* Known not-empty or unknown state, retry */ if (cflags & IORING_CQE_F_SOCK_NONEMPTY || From patchwork Fri Aug 11 12:53:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13350839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7164C001B0 for ; Fri, 11 Aug 2023 12:55:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235068AbjHKMzR (ORCPT ); Fri, 11 Aug 2023 08:55:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51776 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229449AbjHKMzP (ORCPT ); Fri, 11 Aug 2023 08:55:15 -0400 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C37ED109 for ; Fri, 11 Aug 2023 05:55:14 -0700 (PDT) Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-99bcfe28909so266636066b.3 for ; Fri, 11 Aug 2023 05:55:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691758513; x=1692363313; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3Gq8cfrl9gYUlpuskr6LbK2kqwA488iyjn0SSaKnIaM=; b=QCNwAp/YikysLZ9JyaMgFdUksbktzfAjAWxIwxfrff/SaTI80wwZLR/j+qR9029HXj awXTfa+SxelngWnx/bYYettzYd76EmSqjhF1aDFOaWOh9ybRiD33i0WbJStN8ea6lqeh B8hY5gAdtaZGSIrU5ym97gK9X7URBPVH2fe+dyemv+UpiVjJokI5tcU3FJEJIrFFODie OGaeMYjDSaRZJ1huAm+v03CZvkpQ+rfGN7Fov/dr9HOzFQYzkvmRkRg8wNAxvlEiykRr sxkAHHvKVYnmvNwqXNTjcUUP8BvSqQyJ29WhurBpM3WaLqdUPJTJWkZKRGsM3BmSzewc ci5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691758513; x=1692363313; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3Gq8cfrl9gYUlpuskr6LbK2kqwA488iyjn0SSaKnIaM=; b=X58wxW3HvaFGhPWjzszgUhx5zqRhkzZFQxT42LFLVLXKH31ahmO9FMqrlAU8JjuiqL plF/q1g88ZvgvIuSwtl/FwWRMczmCpjT+KRfQuccxZdMO7YytpIKMFCpX1QZJJgSEtQD /DkanVkpDIvc6Cy2vWeJMySyz9t7Y5IVf9G+J3aaAGp/YQm3bmMbqBzLOhVi5VSkjQYH 6UC60XgE5xt0l8z0tnfcha1dA3ryyNQCRQmFcZXY/qwl+0lmp5pBiPLFOx39C63luYAg Vdv8Ebgicpkfc5u3xGKyYQYmEQEbzZ6ofjNH6OjBga3GrbKUTmP17kc0MzPhJyJ+33M3 NxRQ== X-Gm-Message-State: AOJu0Yx+jXhcZF1/88AdaUNGcEw4HN8YTOl3zEvsr2BnHnlOJFkKn0He 3xGBsdbaNXXC8fZBQRL4PCzpDE5Ln2M= X-Google-Smtp-Source: AGHT+IHkmtQswa5dgjfLsrINWwsjRF9qTnT/kdotZ698eALX6obvLuyLr8AeNPJMC1C/hjWUdMnA8Q== X-Received: by 2002:a17:906:76ca:b0:991:b834:af83 with SMTP id q10-20020a17090676ca00b00991b834af83mr1595744ejn.59.1691758512427; Fri, 11 Aug 2023 05:55:12 -0700 (PDT) Received: from 127.com ([2620:10d:c092:600::2:a57e]) by smtp.gmail.com with ESMTPSA id kk9-20020a170907766900b0099cc36c4681sm2206943ejc.157.2023.08.11.05.55.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Aug 2023 05:55:12 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com Subject: [PATCH 3/7] io_uring: open code io_fill_cqe_req() Date: Fri, 11 Aug 2023 13:53:43 +0100 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org io_fill_cqe_req() is only called from one place, open code it, and rename __io_fill_cqe_req(). Signed-off-by: Pavel Begunkov --- io_uring/io_uring.c | 8 +++++--- io_uring/io_uring.h | 11 +---------- io_uring/rw.c | 2 +- 3 files changed, 7 insertions(+), 14 deletions(-) diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 047576bc98d0..e969b4ca1c47 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -978,8 +978,10 @@ static void __io_req_complete_post(struct io_kiocb *req, unsigned issue_flags) struct io_rsrc_node *rsrc_node = NULL; io_cq_lock(ctx); - if (!(req->flags & REQ_F_CQE_SKIP)) - io_fill_cqe_req(ctx, req); + if (!(req->flags & REQ_F_CQE_SKIP)) { + if (!io_fill_cqe_req(ctx, req)) + io_req_cqe_overflow(req); + } /* * If we're the last reference to this request, add to our locked @@ -1556,7 +1558,7 @@ static void __io_submit_flush_completions(struct io_ring_ctx *ctx) comp_list); if (!(req->flags & REQ_F_CQE_SKIP) && - unlikely(!__io_fill_cqe_req(ctx, req))) { + unlikely(!io_fill_cqe_req(ctx, req))) { if (ctx->task_complete) { spin_lock(&ctx->completion_lock); io_req_cqe_overflow(req); diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index ff153af28236..3aa208fbe905 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -133,8 +133,7 @@ static inline struct io_uring_cqe *io_get_cqe(struct io_ring_ctx *ctx) return io_get_cqe_overflow(ctx, false); } -static inline bool __io_fill_cqe_req(struct io_ring_ctx *ctx, - struct io_kiocb *req) +static inline bool io_fill_cqe_req(struct io_ring_ctx *ctx, struct io_kiocb *req) { struct io_uring_cqe *cqe; @@ -168,14 +167,6 @@ static inline bool __io_fill_cqe_req(struct io_ring_ctx *ctx, return true; } -static inline bool io_fill_cqe_req(struct io_ring_ctx *ctx, - struct io_kiocb *req) -{ - if (likely(__io_fill_cqe_req(ctx, req))) - return true; - return io_req_cqe_overflow(req); -} - static inline void req_set_fail(struct io_kiocb *req) { req->flags |= REQ_F_FAIL; diff --git a/io_uring/rw.c b/io_uring/rw.c index 1bce2208b65c..9b51afdae505 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -1064,7 +1064,7 @@ int io_do_iopoll(struct io_ring_ctx *ctx, bool force_nonspin) continue; req->cqe.flags = io_put_kbuf(req, 0); - if (unlikely(!__io_fill_cqe_req(ctx, req))) { + if (unlikely(!io_fill_cqe_req(ctx, req))) { spin_lock(&ctx->completion_lock); io_req_cqe_overflow(req); spin_unlock(&ctx->completion_lock); From patchwork Fri Aug 11 12:53:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13350838 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 919A1C001DB for ; Fri, 11 Aug 2023 12:55:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233952AbjHKMzQ (ORCPT ); Fri, 11 Aug 2023 08:55:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235248AbjHKMzP (ORCPT ); Fri, 11 Aug 2023 08:55:15 -0400 Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com [IPv6:2a00:1450:4864:20::232]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 29A1C115 for ; Fri, 11 Aug 2023 05:55:15 -0700 (PDT) Received: by mail-lj1-x232.google.com with SMTP id 38308e7fff4ca-2b962c226ceso31223191fa.3 for ; Fri, 11 Aug 2023 05:55:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691758513; x=1692363313; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jEqAyf8Y7JpC7PfPYXoIYZ/kuhGCFcsqdd0va/xWKGo=; b=HZMEQ+EKsFOUJNxzC+oivr0YjH219E1R9AIr8FKq8ofypyJCpdW28vGa9iO7vZA3gh yq5OUSoR62Bt2UWuvtp5MtyPIYgY9rkVI4nuSy6KSKKuSBPk1gA5DN0xHsLBxlR6I190 xC0w4JRqhvknD1lcAODIN6ZE7Zy+dvITCRMq0wgeXsulSK/XGbKU7/kPOML5Cs4tLMG8 SBycxIOENu9MQT9s12ufjTw9cD7dlh6cfjgiJms5ZK3icJoBkeQYHimDxpgi5OIvXNCT 8cPNvaFKzuhiHuDaNF7MntCsaSYmNE+2LCU6zc0avkCiuRka7VNToJaeXU+3/8SXwMKE ksOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691758513; x=1692363313; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jEqAyf8Y7JpC7PfPYXoIYZ/kuhGCFcsqdd0va/xWKGo=; b=j0kGNxmlY7EC+hnIkikGuZz5bzYst6Ig3VqtNCkmvXGpG9IcafZKxW6Oszgub68qll ePZ+Cm5WSjzHsY2Df21XuQgKqMcuLLucUXz0DjwqDF7M+xAgE+lgx6Z9QIwqU+S+huNy vbSoMNMTW8iSTOk4s9xLNBjYvZGvuWfJ+HUpPD5W64mUzgrwfQhrZrzFXlXvkWDwq1qf niqgZWUxo4hHpgMLyBX6mLSv9dVDV+gj/12cMLwFozw4a1yxQX/LlNLWTmByg3PjKJY1 8L5EdLIDEF3AWAYB2TaPL7YMeKvgdRDeCczmN4dY1BZ8GK7RtNk4d+JZUEurn//A6xa+ pJwA== X-Gm-Message-State: AOJu0Yy5ttLs8vcvPkp6rtn7l3HQ2oWeeZKG1Te4v/qHjiORQfXLSJ64 TincZGLMOU60P+MM1hmLh1M5T4EzUik= X-Google-Smtp-Source: AGHT+IE5iZj29cY2rtCzaDO1udS3/Ur9Af7ssVsNyJmQSD/heauKKoCS3AP+IKTa6O680bXoTqS+TQ== X-Received: by 2002:a05:651c:1028:b0:2b9:44c5:ac15 with SMTP id w8-20020a05651c102800b002b944c5ac15mr1599889ljm.41.1691758512959; Fri, 11 Aug 2023 05:55:12 -0700 (PDT) Received: from 127.com ([2620:10d:c092:600::2:a57e]) by smtp.gmail.com with ESMTPSA id kk9-20020a170907766900b0099cc36c4681sm2206943ejc.157.2023.08.11.05.55.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Aug 2023 05:55:12 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com Subject: [PATCH 4/7] io_uring: remove return from io_req_cqe_overflow() Date: Fri, 11 Aug 2023 13:53:44 +0100 Message-ID: <8f2029ad0c22f73451664172d834372608ee0a77.1691757663.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org Nobody checks io_req_cqe_overflow()'s return, make it return void. Signed-off-by: Pavel Begunkov --- io_uring/io_uring.c | 8 ++++---- io_uring/io_uring.h | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index e969b4ca1c47..7595658a5073 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -813,15 +813,15 @@ static bool io_cqring_event_overflow(struct io_ring_ctx *ctx, u64 user_data, return true; } -bool io_req_cqe_overflow(struct io_kiocb *req) +void io_req_cqe_overflow(struct io_kiocb *req) { if (!(req->flags & REQ_F_CQE32_INIT)) { req->extra1 = 0; req->extra2 = 0; } - return io_cqring_event_overflow(req->ctx, req->cqe.user_data, - req->cqe.res, req->cqe.flags, - req->extra1, req->extra2); + io_cqring_event_overflow(req->ctx, req->cqe.user_data, + req->cqe.res, req->cqe.flags, + req->extra1, req->extra2); } /* diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index 3aa208fbe905..3dc0b6fb0ef7 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -39,7 +39,7 @@ enum { }; struct io_uring_cqe *__io_get_cqe(struct io_ring_ctx *ctx, bool overflow); -bool io_req_cqe_overflow(struct io_kiocb *req); +void io_req_cqe_overflow(struct io_kiocb *req); int io_run_task_work_sig(struct io_ring_ctx *ctx); void io_req_defer_failed(struct io_kiocb *req, s32 res); void io_req_complete_post(struct io_kiocb *req, unsigned issue_flags); From patchwork Fri Aug 11 12:53:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13350841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B63E6C001DE for ; Fri, 11 Aug 2023 12:55:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232875AbjHKMzR (ORCPT ); Fri, 11 Aug 2023 08:55:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51806 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235048AbjHKMzQ (ORCPT ); Fri, 11 Aug 2023 08:55:16 -0400 Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [IPv6:2a00:1450:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A386114 for ; Fri, 11 Aug 2023 05:55:16 -0700 (PDT) Received: by mail-ej1-x631.google.com with SMTP id a640c23a62f3a-99c47ef365cso281980266b.0 for ; Fri, 11 Aug 2023 05:55:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691758514; x=1692363314; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YJbLz9QdYPZV5kn6edsU70ey95VfZY7HvhquINjrY44=; b=iAOOICYcaIDRQzRgQkZvD65NfcZ9h93G4bHmYfLj8txChX+vfjB3IDGy+M4hCtj0qr 3KAvvBRa6dJY8uqS8Kfh0GWlT+QrRQoYvjSyX+g5ibu8iNMux2QrzRiCnK4fn5Eb0GWT 8J8iHguoFRcukAalm0j9FY6tz04m7bGsiokKEepzRZ0sr1/NAEOlP/+Db3+/ZoWoKLot zOX84L1UiaFMdqGfYIr73ZtHoNLLExb9O3lWqu7WDhqXufkiEhiEeUIuERMh5IvM3C6R nb8e8HPDISay1isiqItZI7UdK4vSoNm59wqlJ+18hCGMQZvN24irSeLjwSrZ9atq9GbE tTCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691758514; x=1692363314; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YJbLz9QdYPZV5kn6edsU70ey95VfZY7HvhquINjrY44=; b=FAnche3XZyrbNFd4pQzd6pUWfQpwRBgDrL8fQJ41OdTEs1sa4QPnCPIvFT69LaN3da FLU6dqIUC2+kRwwMScphYqHntdkw35R53CdwRYhN83zTLzNGdpi2iDr31+PRokmHH72F yhTTc0JFlf2DS8lS2jB47nWixAXFMD2OKrBB5Fl44MvP7ZIQRL1eCIiJT4AGOsGWshqI pLwaKk0ZTPV+IZinwCC/m+0yRNmYQQDw6XmdTpaLQXvi92n3or8M2dzBuzvUb39Mu472 2KlRxabpJ5qDDrXc/eBai8OxBmBN/51p2OF/5I7P8ksuAJR2QfvVzB+fhv7gNxAmZyBz c5Kg== X-Gm-Message-State: AOJu0Yyae9sniOsKcNf9XJAcFDa2klHwK+1UJ6VDJN9N9z9pxBho5GwR CR8W/XWBPhOjnbJfkwyUWJ6xDOY6Esc= X-Google-Smtp-Source: AGHT+IGp26d3omOxY9UWuDjRWyX3296Xtvudj7s9cxmfD0/9NQKK43UPD39Bd6EMurMj3wAclJkMvw== X-Received: by 2002:a17:906:31cb:b0:99c:e38d:e47f with SMTP id f11-20020a17090631cb00b0099ce38de47fmr1561738ejf.33.1691758513983; Fri, 11 Aug 2023 05:55:13 -0700 (PDT) Received: from 127.com ([2620:10d:c092:600::2:a57e]) by smtp.gmail.com with ESMTPSA id kk9-20020a170907766900b0099cc36c4681sm2206943ejc.157.2023.08.11.05.55.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Aug 2023 05:55:13 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com Subject: [PATCH 5/7] io_uring: never overflow io_aux_cqe Date: Fri, 11 Aug 2023 13:53:45 +0100 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org Now all callers of io_aux_cqe() set allow_overflow to false, remove the parameter and not allow overflowing auxilary multishot cqes. When CQ is full the function callers and all multishot requests in general are expected to complete the request. That prevents indefinite in-background grows of the overflow list and let's the userspace to handle the backlog at its own pace. Resubmitting a request should also be faster than accounting a bunch of overflows, so it should be better for perf when it happens, but a well behaving userspace should be trying to avoid overflows in any case. Signed-off-by: Pavel Begunkov --- io_uring/io_uring.c | 11 +++++++---- io_uring/io_uring.h | 3 +-- io_uring/net.c | 8 ++++---- io_uring/poll.c | 4 ++-- io_uring/timeout.c | 4 ++-- 5 files changed, 16 insertions(+), 14 deletions(-) diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 7595658a5073..e57d00939ab9 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -939,15 +939,18 @@ bool io_post_aux_cqe(struct io_ring_ctx *ctx, u64 user_data, s32 res, u32 cflags return __io_post_aux_cqe(ctx, user_data, res, cflags, true); } -bool io_aux_cqe(const struct io_kiocb *req, bool defer, s32 res, u32 cflags, - bool allow_overflow) +/* + * A helper for multishot requests posting additional CQEs. + * Should only be used from a task_work including IO_URING_F_MULTISHOT. + */ +bool io_fill_cqe_req_aux(struct io_kiocb *req, bool defer, s32 res, u32 cflags) { struct io_ring_ctx *ctx = req->ctx; u64 user_data = req->cqe.user_data; struct io_uring_cqe *cqe; if (!defer) - return __io_post_aux_cqe(ctx, user_data, res, cflags, allow_overflow); + return __io_post_aux_cqe(ctx, user_data, res, cflags, false); lockdep_assert_held(&ctx->uring_lock); @@ -962,7 +965,7 @@ bool io_aux_cqe(const struct io_kiocb *req, bool defer, s32 res, u32 cflags, * however it's main job is to prevent unbounded posted completions, * and in that it works just as well. */ - if (!allow_overflow && test_bit(IO_CHECK_CQ_OVERFLOW_BIT, &ctx->check_cq)) + if (test_bit(IO_CHECK_CQ_OVERFLOW_BIT, &ctx->check_cq)) return false; cqe = &ctx->submit_state.cqes[ctx->submit_state.cqes_count++]; diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index 3dc0b6fb0ef7..3e6ff3cd9a24 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -44,8 +44,7 @@ int io_run_task_work_sig(struct io_ring_ctx *ctx); void io_req_defer_failed(struct io_kiocb *req, s32 res); void io_req_complete_post(struct io_kiocb *req, unsigned issue_flags); bool io_post_aux_cqe(struct io_ring_ctx *ctx, u64 user_data, s32 res, u32 cflags); -bool io_aux_cqe(const struct io_kiocb *req, bool defer, s32 res, u32 cflags, - bool allow_overflow); +bool io_fill_cqe_req_aux(struct io_kiocb *req, bool defer, s32 res, u32 cflags); void __io_commit_cqring_flush(struct io_ring_ctx *ctx); struct page **io_pin_pages(unsigned long ubuf, unsigned long len, int *npages); diff --git a/io_uring/net.c b/io_uring/net.c index 8c419c01a5db..3d07bf79c1e0 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -641,8 +641,8 @@ static inline bool io_recv_finish(struct io_kiocb *req, int *ret, } if (!mshot_finished) { - if (io_aux_cqe(req, issue_flags & IO_URING_F_COMPLETE_DEFER, - *ret, cflags | IORING_CQE_F_MORE, false)) { + if (io_fill_cqe_req_aux(req, issue_flags & IO_URING_F_COMPLETE_DEFER, + *ret, cflags | IORING_CQE_F_MORE)) { io_recv_prep_retry(req); /* Known not-empty or unknown state, retry */ if (cflags & IORING_CQE_F_SOCK_NONEMPTY || @@ -1366,8 +1366,8 @@ int io_accept(struct io_kiocb *req, unsigned int issue_flags) if (ret < 0) return ret; - if (io_aux_cqe(req, issue_flags & IO_URING_F_COMPLETE_DEFER, ret, - IORING_CQE_F_MORE, false)) + if (io_fill_cqe_req_aux(req, issue_flags & IO_URING_F_COMPLETE_DEFER, + ret, IORING_CQE_F_MORE)) goto retry; return -ECANCELED; diff --git a/io_uring/poll.c b/io_uring/poll.c index 65ec363f6377..4c360ba8793a 100644 --- a/io_uring/poll.c +++ b/io_uring/poll.c @@ -300,8 +300,8 @@ static int io_poll_check_events(struct io_kiocb *req, struct io_tw_state *ts) __poll_t mask = mangle_poll(req->cqe.res & req->apoll_events); - if (!io_aux_cqe(req, ts->locked, mask, - IORING_CQE_F_MORE, false)) { + if (!io_fill_cqe_req_aux(req, ts->locked, mask, + IORING_CQE_F_MORE)) { io_req_set_res(req, mask, 0); return IOU_POLL_REMOVE_POLL_USE_RES; } diff --git a/io_uring/timeout.c b/io_uring/timeout.c index 6242130e73c6..7fd7dbb211d6 100644 --- a/io_uring/timeout.c +++ b/io_uring/timeout.c @@ -73,8 +73,8 @@ static void io_timeout_complete(struct io_kiocb *req, struct io_tw_state *ts) if (!io_timeout_finish(timeout, data)) { bool filled; - filled = io_aux_cqe(req, ts->locked, -ETIME, IORING_CQE_F_MORE, - false); + filled = io_fill_cqe_req_aux(req, ts->locked, -ETIME, + IORING_CQE_F_MORE); if (filled) { /* re-arm timer */ spin_lock_irq(&ctx->timeout_lock); From patchwork Fri Aug 11 12:53:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13350842 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D562C04FDF for ; Fri, 11 Aug 2023 12:55:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235048AbjHKMzS (ORCPT ); Fri, 11 Aug 2023 08:55:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229449AbjHKMzR (ORCPT ); Fri, 11 Aug 2023 08:55:17 -0400 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6985B115 for ; Fri, 11 Aug 2023 05:55:16 -0700 (PDT) Received: by mail-ej1-x633.google.com with SMTP id a640c23a62f3a-99c47ef365cso281981266b.0 for ; Fri, 11 Aug 2023 05:55:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691758514; x=1692363314; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WyPJ8ImW+Enn/JA4TnwWypJPSlU1aIeQGsKxhR8THfU=; b=BkJJvuraa5dwDPTR1vx6NaFNRAzNH6hFEKrQIP9cd8o0QPH+UCqY/TXnyYe0ULdmW4 JJlCPM7eFLZtVVIxEPC2wMWsv07PA9HnM2xCtCv+urBW70ewPWxgmT/YLeJDyOLkg2fj LMEUpmVS8Wxi7Nm92kd4dR2SV0Khb17RMirDxXCNyLNFSAxrSmCVn4dGUUyhsHKsoqHQ xOKfvQp1uVy4HcBzkWF06ZwRjHkExSzfOYyjECY3InjhPsA/jAZMsIKyYe7tX6ic9vzi otM4DJtQIUALBlkXz8M/jIxD0RU5nl/rZrPq3safipwLYYWVGZSHwWs+/zMVHkaQC6Xw KBdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691758514; x=1692363314; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WyPJ8ImW+Enn/JA4TnwWypJPSlU1aIeQGsKxhR8THfU=; b=Khb3RjfEokliYFdMWEFj3sNMG5WqzmtlPfOiT4GUd/dLBsOAtwUzUvCbhXe4r+i1yy jHJ0fGDo1xkfRL1ykG6CRvXcDq+SI3SBmS6+pssuO4h4GAubCxDKiAF/1GfTDM5gB5zU 3GdG/NzDZPPF+ZzN6579qgIwikOaOm+4KPMM330tYHyUkXmbKabNlctQ/F6EN93EqQFc z7Aw/vBeMw2386kMbUGcZAINlEDrin4kjrTLnyqkkB+Kh+zaISoz8IS7xkXRfRj1bw9h +heFmimfJE8Aeu2FF0sya56nqc7WxNKVPL/9ixxL5PZXNEHRDpPa/JzN1NviY1O7sEVj 9GfQ== X-Gm-Message-State: AOJu0Yw14/N8MYaAjmLEcSxlqpAOxwA+gHtQQVs2BdTVvMFIPOMsSa0S oFQrbUqc2jPaQKmmmQcuTQlmciGZnzE= X-Google-Smtp-Source: AGHT+IEn37CeDZpDg8BiQl0RikAPK4Yz6FJeure6kLQ6+0+wnhSmZQV3MRQRLEx6z0smTp6//BkDoA== X-Received: by 2002:a17:906:9bc1:b0:99c:ad52:b0a with SMTP id de1-20020a1709069bc100b0099cad520b0amr1773354ejc.38.1691758514563; Fri, 11 Aug 2023 05:55:14 -0700 (PDT) Received: from 127.com ([2620:10d:c092:600::2:a57e]) by smtp.gmail.com with ESMTPSA id kk9-20020a170907766900b0099cc36c4681sm2206943ejc.157.2023.08.11.05.55.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Aug 2023 05:55:14 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com Subject: [PATCH 6/7] io_uring/rsrc: keep one global dummy_ubuf Date: Fri, 11 Aug 2023 13:53:46 +0100 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org We set empty registered buffers to dummy_ubuf as an optimisation. Currently, we allocate the dummy entry for each ring, whenever we can simply have one global instance. We're casting out const on assignment, it's fine as we're not going to change the content of the dummy, the constness gives us an extra layer of protection if sth ever goes wrong. Signed-off-by: Pavel Begunkov --- io_uring/io_uring.c | 9 --------- io_uring/rsrc.c | 14 ++++++++++---- 2 files changed, 10 insertions(+), 13 deletions(-) diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index e57d00939ab9..a7a4d637aee0 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -290,13 +290,6 @@ static __cold struct io_ring_ctx *io_ring_ctx_alloc(struct io_uring_params *p) goto err; if (io_alloc_hash_table(&ctx->cancel_table_locked, hash_bits)) goto err; - - ctx->dummy_ubuf = kzalloc(sizeof(*ctx->dummy_ubuf), GFP_KERNEL); - if (!ctx->dummy_ubuf) - goto err; - /* set invalid range, so io_import_fixed() fails meeting it */ - ctx->dummy_ubuf->ubuf = -1UL; - if (percpu_ref_init(&ctx->refs, io_ring_ctx_ref_free, 0, GFP_KERNEL)) goto err; @@ -335,7 +328,6 @@ static __cold struct io_ring_ctx *io_ring_ctx_alloc(struct io_uring_params *p) INIT_WQ_LIST(&ctx->submit_state.compl_reqs); return ctx; err: - kfree(ctx->dummy_ubuf); kfree(ctx->cancel_table.hbs); kfree(ctx->cancel_table_locked.hbs); kfree(ctx->io_bl); @@ -2897,7 +2889,6 @@ static __cold void io_ring_ctx_free(struct io_ring_ctx *ctx) io_wq_put_hash(ctx->hash_map); kfree(ctx->cancel_table.hbs); kfree(ctx->cancel_table_locked.hbs); - kfree(ctx->dummy_ubuf); kfree(ctx->io_bl); xa_destroy(&ctx->io_bl_xa); kfree(ctx); diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index 5e8fdd9b8ca6..d9c853d10587 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -33,6 +33,12 @@ static int io_sqe_buffer_register(struct io_ring_ctx *ctx, struct iovec *iov, #define IORING_MAX_FIXED_FILES (1U << 20) #define IORING_MAX_REG_BUFFERS (1U << 14) +static const struct io_mapped_ubuf dummy_ubuf = { + /* set invalid range, so io_import_fixed() fails meeting it */ + .ubuf = -1UL, + .ubuf_end = 0, +}; + int __io_account_mem(struct user_struct *user, unsigned long nr_pages) { unsigned long page_limit, cur_pages, new_pages; @@ -132,7 +138,7 @@ static void io_buffer_unmap(struct io_ring_ctx *ctx, struct io_mapped_ubuf **slo struct io_mapped_ubuf *imu = *slot; unsigned int i; - if (imu != ctx->dummy_ubuf) { + if (imu != &dummy_ubuf) { for (i = 0; i < imu->nr_bvecs; i++) unpin_user_page(imu->bvec[i].bv_page); if (imu->acct_pages) @@ -459,14 +465,14 @@ static int __io_sqe_buffers_update(struct io_ring_ctx *ctx, break; i = array_index_nospec(up->offset + done, ctx->nr_user_bufs); - if (ctx->user_bufs[i] != ctx->dummy_ubuf) { + if (ctx->user_bufs[i] != &dummy_ubuf) { err = io_queue_rsrc_removal(ctx->buf_data, i, ctx->user_bufs[i]); if (unlikely(err)) { io_buffer_unmap(ctx, &imu); break; } - ctx->user_bufs[i] = ctx->dummy_ubuf; + ctx->user_bufs[i] = (struct io_mapped_ubuf *)&dummy_ubuf; } ctx->user_bufs[i] = imu; @@ -1077,7 +1083,7 @@ static int io_sqe_buffer_register(struct io_ring_ctx *ctx, struct iovec *iov, int ret, nr_pages, i; struct folio *folio = NULL; - *pimu = ctx->dummy_ubuf; + *pimu = (struct io_mapped_ubuf *)&dummy_ubuf; if (!iov->iov_base) return 0; From patchwork Fri Aug 11 12:53:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13350840 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37D5FC04E69 for ; Fri, 11 Aug 2023 12:55:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234954AbjHKMzS (ORCPT ); Fri, 11 Aug 2023 08:55:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51840 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235116AbjHKMzR (ORCPT ); Fri, 11 Aug 2023 08:55:17 -0400 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA8A8109 for ; Fri, 11 Aug 2023 05:55:16 -0700 (PDT) Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-98377c5d53eso269360566b.0 for ; Fri, 11 Aug 2023 05:55:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691758515; x=1692363315; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=b44YcyjPbFap3yV03J2dsu1qQ4z7LC8pimOtpsft2Qc=; b=HbQ+nEF5gOkVhKlVEVlJq9A9JRZYcs2GtXA1jM1EXMiweB6aSVOO7WGqXqU/zkGNHY y7dU2wpbbrs2Hs2LEKE39ZsTgrk3T7lnKU5wIZdhcx1Xfj0Cb5UN+RAgLcYuPF9062tZ Y4LBmFPhdWGjzVZb8wRF1F3MxNjWCiDA5hx+P4PP6ZzD+c3CiV0dBuvgOzNPlQd1dA93 +auaX9AjlRO3UTCBymzaA7T5x49UR8x0gm/4U5HWfgwrg1v/5fM9LtAUg447SvrgVdHK gRb9bEurIaRy/zxe+eTXuK5hN9c/C2e7k+IJu8qOsr/Qv250QGO+6y2BU8hGz6EIWT/J s3sQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691758515; x=1692363315; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=b44YcyjPbFap3yV03J2dsu1qQ4z7LC8pimOtpsft2Qc=; b=NG5EqKWHEe45A3vLWndfkwsmLwoanqD1aA8+xd9a6QAsBnahQWpqifgGSKdWORvmrr UBf7Ic1YQATe7F5pbD2joEjXwGJCEfYs/eykyaOqPPlW6FvrqyXqS+R1V8WEBoVlyYYv IP0TYqjp682/O6uiv/mRP96dSf5zxUzMxyMPB6OX5FP5ovfZN7jJgGc85sm7injqWGek 3RHMTstlH+PzFjxBphzRm3Lk4H4KM7T1DW3iNEscK8HVXmpFUruae+yH0DGEiJbM90zN K/9dDtK4EsyleFaDhODNBM5jecbc5bgYxK3Sfs+tXJX090VMnZSwbes/BV6cffJ5s/9e 1Paw== X-Gm-Message-State: AOJu0YwvgST70j8bkJO0I4aZSXFQXfcSxHIy6N+FwjYP5/VVCzzZVl/b EILsglrvDqwYJWGQVgCrLzZhptXlIiU= X-Google-Smtp-Source: AGHT+IFRoQL/vUAtm6YvdlGoo1WVdWemmsvHCeY2Cpdgp0VEB7e6gE9CcdZ8yXaXC1xSW4aOi+zQhw== X-Received: by 2002:a17:906:18aa:b0:99c:bb4d:f590 with SMTP id c10-20020a17090618aa00b0099cbb4df590mr1439525ejf.47.1691758515080; Fri, 11 Aug 2023 05:55:15 -0700 (PDT) Received: from 127.com ([2620:10d:c092:600::2:a57e]) by smtp.gmail.com with ESMTPSA id kk9-20020a170907766900b0099cc36c4681sm2206943ejc.157.2023.08.11.05.55.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Aug 2023 05:55:14 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com Subject: [PATCH 7/7] io_uring: simplify io_run_task_work_sig return Date: Fri, 11 Aug 2023 13:53:47 +0100 Message-ID: <3aec8a532c003d6e50739b969a82989402696170.1691757663.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org Nobody cares about io_run_task_work_sig returning 1, we only check for negative errors. Simplify by keeping to 0/-error returns. Signed-off-by: Pavel Begunkov --- io_uring/io_uring.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index a7a4d637aee0..e189158ebbdd 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -2477,10 +2477,10 @@ int io_run_task_work_sig(struct io_ring_ctx *ctx) if (!llist_empty(&ctx->work_llist)) { __set_current_state(TASK_RUNNING); if (io_run_local_work(ctx) > 0) - return 1; + return 0; } if (io_run_task_work() > 0) - return 1; + return 0; if (task_sigpending(current)) return -EINTR; return 0;