From patchwork Mon Feb 24 19:45:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13988861 Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16A81200138 for ; Mon, 24 Feb 2025 19:44:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740426263; cv=none; b=Pg98e4R4qi9RwoZwv+pDsx9XdnmNF02bXaYjFbED2rcs2A1X6aEFKzMkc47ajCctUGmVF7dvK3grfCKxatymAdGA9slUkzfEuATVVUJKmIJFSoBQITqgsW2zj8Tv3PEiqnt6DZRpP1qiyptWNV2t8S56on7A7YqM7XkL9Pddo0U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740426263; c=relaxed/simple; bh=Rtd1vr5SAaF02h6dyLb9Ksi5yf1TT9r9MZf01jdx/pM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jOg2wRjRsKGa0OocAONxQ7Gmo3wMM3kdJJ6gjP5JK5mt5iDnsBVwkAX3UFGCapFZ7Ti4ig59o+RK6vjZBKvNVWZSnNJsNd19HKNKUZXwd3HebM5YYzzNlWtevaFVkHVDWVSfCVTH4/bShBR3nAY/fYSZT+TtS6ttHsLJcjqv6eU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=h38yGW7+; arc=none smtp.client-ip=209.85.221.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="h38yGW7+" Received: by mail-wr1-f41.google.com with SMTP id ffacd0b85a97d-388cae9eb9fso2520163f8f.3 for ; Mon, 24 Feb 2025 11:44:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740426260; x=1741031060; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3QwZm2v050ucBGPYTt5nSqW/wI1mc4sDWfiJMAxM5Nk=; b=h38yGW7+/zQiSNBG+7qj9Tl111hoBVm4gIM5W+fO5Q9hV5V8W/fEtCn/OtiYyEim6x Mbe3BahUi6SfSAyhhh9D4WTXXFE0I7Mzc9iDqd+w4F7gskRbN41M01Rcy2lWTHZqW4pG SDmWsLEEaP+86sVMgNNUIGUvxLT6geexa1F2Voeq6TbxEFN7OHHVl/TtiMLvd+U3bt5y /ACCcqQlira/pXPR2cIXTWGUlMJvjS576zp/wYQZCLtRQsIjqosalZ5ewOkEjdJHYTuX PkMlPWP/BZHWAKmveAnU5tk4c+AET5/SWV4LozMccvKbCHFbJxxa/LNVeFcym+eIq8v2 PclQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740426260; x=1741031060; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3QwZm2v050ucBGPYTt5nSqW/wI1mc4sDWfiJMAxM5Nk=; b=rjEjcsR9+KCdyGLHQEdRY9DDK4BSJ8gN5q1KO93zCXjXmu5EdB/sOEgfvHe2n1VGZG mPvYxqgmOxPUHesZAf43C5Z2IgUPamMUQalgQtDC5zJ202PBhZzpSOz4LMFxWcwQq6Tf bgwA9PenihBD0OnP4K2BMqG18KfRFH9QaVfrXVGN2ZOZRyksoIAXeZNbvW34JcVN+glX HL2Qkt1u0jqvErWjXXu1UNfBvZUeYtqhCu8d5IKbpX9zKeaY9jCDDHOOj9JuzoIqkF9C +MRGey1vKLKG7c2hRfPAP7U0/MoO5L1BJpe2UTVA/DxtDkMZksbF3IRAp7Obwah1QXP2 Oj/w== X-Gm-Message-State: AOJu0Yzd+ITUCokuVtcuesiGh9g3XA8IjET4gRePHIKEnXgXDJgcBoH3 UEC/HMQl3xvOmKlsCH0E3kmcS8vXYjKyH4WHRZc0Y7vZrL8Mxixj34sASQ== X-Gm-Gg: ASbGncuD5Efrq+wMFE31B+v3VjS3foKDgxZU5Wd5RHXAkeocryM854Sz9sZt2m0SWwL 6yI4euDAmj4ecsKwLhWMhbQXQ9PSvPI0PLQv/j6F2geJXCxBNILvSmkkvuD9g1cDfj+i4fEKE1e ZbDG9MRraoJjqQv81L2eo6vUyUc0Otnq5pdxP0HNJRoXfPW2mZkJYNsNxmD9QKqAR5iESbnZ9cA ldO2okUlq/LCjSVYx8FJCh+EC8yTwvypowbU+E675hbhtDj4Ll8PIXMeU4EJDDFzKDmNz0xXTTp 7jO3qZr2uX8AW+NUgWh1OWo0FbLMhVak8r4LIs8= X-Google-Smtp-Source: AGHT+IEZ+/3z+ctNhMqN5FVLH9mxpg2OZxKJtFIffD0DuMlU55JxHI7OwFOBrzynYsepth3XFPxmkA== X-Received: by 2002:a5d:5887:0:b0:38f:2065:3f20 with SMTP id ffacd0b85a97d-390cc604b28mr218205f8f.17.1740426259783; Mon, 24 Feb 2025 11:44:19 -0800 (PST) Received: from 127.0.0.1localhost ([148.252.146.93]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43ab14caa5esm1548305e9.0.2025.02.24.11.44.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2025 11:44:19 -0800 (PST) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com, Caleb Sander Mateos Subject: [PATCH v2 1/4] io_uring/rw: allocate async data in io_prep_rw() Date: Mon, 24 Feb 2025 19:45:03 +0000 Message-ID: <5ead621051bc3374d1e8d96f816454906a6afd71.1740425922.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 rw always allocates async_data, so instead of doing that deeper in prep calls inside of io_prep_rw_setup(), be a bit more explicit and do that early on in io_prep_rw(). Signed-off-by: Pavel Begunkov --- io_uring/rw.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/io_uring/rw.c b/io_uring/rw.c index 22612a956e75..7efc2337c5a0 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -203,9 +203,6 @@ static int io_prep_rw_setup(struct io_kiocb *req, int ddir, bool do_import) { struct io_async_rw *rw; - if (io_rw_alloc_async(req)) - return -ENOMEM; - if (!do_import || io_do_buffer_select(req)) return 0; @@ -262,6 +259,9 @@ static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe, u64 attr_type_mask; int ret; + if (io_rw_alloc_async(req)) + return -ENOMEM; + rw->kiocb.ki_pos = READ_ONCE(sqe->off); /* used for fixed read/write too - just read unconditionally */ req->buf_index = READ_ONCE(sqe->buf_index); From patchwork Mon Feb 24 19:45:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13988862 Received: from mail-wm1-f49.google.com (mail-wm1-f49.google.com [209.85.128.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69C4C200138 for ; Mon, 24 Feb 2025 19:44:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740426266; cv=none; b=uqYn0WVRpvlx2fiWiIrzLWsI5ckMZQuqvMSyyCN2nhPkFndVsk2fiFhLDIBjrCfszEqLtIWWq8xg3uiNPxUUG2DjpVSidy940uIAvnYmsovFPdhf+TI74Y3ozWNDJcpLscRnYJP1zL5s+MuTikocnw4AadPlkybkEQ2OnMABBUQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740426266; c=relaxed/simple; bh=ZiWEpyrbRZKpS6hkr7hzo1dG6dzpZJ5uoEu210Dzmgs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kEAk+mggLlK7Vl4QcTXASH0WFLi6t0ZM4fa6QGKZ0GVLkZHeX4VSo2f1lMRvs3Nh/MOls64OrRV0Ek4Z4e/XTLDFFkQzmUmHAcwD4ZNnHqsOMDshzWZAg2I0gNNiUF0mIKqKBwbuTv9c+QwXSdZcgVTssEHM072kInbMF9Gv6Xo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GbIUk0Qq; arc=none smtp.client-ip=209.85.128.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GbIUk0Qq" Received: by mail-wm1-f49.google.com with SMTP id 5b1f17b1804b1-43948f77f1aso29488125e9.0 for ; Mon, 24 Feb 2025 11:44:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740426262; x=1741031062; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QK9YGtEODcnKoP57rreMyeESG4woNuiJuZC9WxFY5T0=; b=GbIUk0QqtpWVJLe3CD7I+wt8TgRx1/1gtN0gzj1tLFG+tfwDF8Y0r0DyDIFgRUVHQo zOzI9Ai+knACBmTZCYFhxzlSlE/727otBeEQ+QdIyKVd0HABKnDRsJIVvUUK6hDEFwDr Hj6l8GyzzIim/C4VR8VGtdARotkP0au7LqZ95VBrih9R75pOCm9cETnUthtaE0JMlnS8 vZtwfu2xxkyspik/Bra3rz3UnOBJBDfATUCUiFGixDXoDcO6kzNz06upPD1Vi7e16a3F bsS5GkOGPeRH33qQLvV5ONUI3I4WSoTnnzq6oaNuaTkMcHHDgwr23A7IG+fyGmkGr2Qt N2dA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740426262; x=1741031062; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QK9YGtEODcnKoP57rreMyeESG4woNuiJuZC9WxFY5T0=; b=fxRsxknXY9ZFwWDes4FTG06sh7gwqs3BqsQJX5trObOhsdlnHcZO20LY5JF6XA7OQi JHVe7BgEwN7v2pct8GHLN5tkzu5CXnNFT1ONpFlP1hekBxkV5LQAGQObUxKtFhb+dJxt nzdeJMWQuRne0IfhGkMQym+Z4Y4pH5bnHSzbhj9sJlRDWFU/u8rtaqlErknaaL9MJ93S N8D73F0tm7lZAwQ/cxoYsKskeeTm4tqQLBZOCvSB7jGEQFaL30zjwXqQaka3h1UJbeQn TQgDo0WKRKGMLBLpmydjrRyZmZVfgxh8ljImx3o///sDs2cOwrER5ipCa3V136Ky/ZvT W9xw== X-Gm-Message-State: AOJu0Yyhe4EXGVBzJgc5Qho1YijLYYOsYTPoVHMth2cBp4LYOqrYH7PK UbCsF4CAwYvMdszIfAbGhughVOzMO4CvOKZHV/8tC9XM/eUatQuN47YQiA== X-Gm-Gg: ASbGncsVODfIHoboSUC9OWUD7bQkyHGW4vlmGirQCRCa/BjLF9P4HZWLYY1zkt9xeIy hRnJXkHqVOe1R3pBPWUPvmJ7yQYF3QljZcnWeLv/hl6f2bb7AvXBH8wD0rAhiQenxohfCpWA+g5 MpOwZ3re8uGUKvYZNB4X5oqW0GiD4wJXYKdGcg8BtFHZYToM87j7PSt1ErEMHdnMrZlgqgLeTPk Cnb5y7JC7nkE1I5cal6M5+vf43RE7cPB62HafzwnSi5bzgFouMB94ySfu4Pl4I9soFZhOu3Mtgu WaAe8mGB8uOs7/Qi5qc2hrYHFncO/bAlBuCiryY= X-Google-Smtp-Source: AGHT+IFvOlj1GXW5meFdjtgZYBguANTagkdRz8UgjH/w8fSwCAJlLMGQxQUtO2vFdeQWZJ6kk+YPJQ== X-Received: by 2002:a05:600c:19cd:b0:439:8340:637 with SMTP id 5b1f17b1804b1-43ab0fa248bmr6070095e9.30.1740426261961; Mon, 24 Feb 2025 11:44:21 -0800 (PST) Received: from 127.0.0.1localhost ([148.252.146.93]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43ab14caa5esm1548305e9.0.2025.02.24.11.44.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2025 11:44:20 -0800 (PST) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com, Caleb Sander Mateos Subject: [PATCH v2 2/4] io_uring/rw: rename io_import_iovec() Date: Mon, 24 Feb 2025 19:45:04 +0000 Message-ID: <91cea59340b61a8f52dc7b8e720274577a25188c.1740425922.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 io_import_iovec() is not limited to iovecs but also imports buffers for normal reads and selected buffers, rename it for clarity. Signed-off-by: Pavel Begunkov --- io_uring/rw.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/io_uring/rw.c b/io_uring/rw.c index 7efc2337c5a0..e636be4850a7 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -76,7 +76,7 @@ static int io_iov_buffer_select_prep(struct io_kiocb *req) return 0; } -static int __io_import_iovec(int ddir, struct io_kiocb *req, +static int __io_import_rw_buffer(int ddir, struct io_kiocb *req, struct io_async_rw *io, unsigned int issue_flags) { @@ -122,13 +122,13 @@ static int __io_import_iovec(int ddir, struct io_kiocb *req, return 0; } -static inline int io_import_iovec(int rw, struct io_kiocb *req, - struct io_async_rw *io, - unsigned int issue_flags) +static inline int io_import_rw_buffer(int rw, struct io_kiocb *req, + struct io_async_rw *io, + unsigned int issue_flags) { int ret; - ret = __io_import_iovec(rw, req, io, issue_flags); + ret = __io_import_rw_buffer(rw, req, io, issue_flags); if (unlikely(ret < 0)) return ret; @@ -207,7 +207,7 @@ static int io_prep_rw_setup(struct io_kiocb *req, int ddir, bool do_import) return 0; rw = req->async_data; - return io_import_iovec(ddir, req, rw, 0); + return io_import_rw_buffer(ddir, req, rw, 0); } static inline void io_meta_save_state(struct io_async_rw *io) @@ -845,7 +845,7 @@ static int __io_read(struct io_kiocb *req, unsigned int issue_flags) loff_t *ppos; if (io_do_buffer_select(req)) { - ret = io_import_iovec(ITER_DEST, req, io, issue_flags); + ret = io_import_rw_buffer(ITER_DEST, req, io, issue_flags); if (unlikely(ret < 0)) return ret; } From patchwork Mon Feb 24 19:45:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13988863 Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8ACAA200138 for ; Mon, 24 Feb 2025 19:44:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740426268; cv=none; b=qxDOGiWdfhriOnbsv9j0zYvm3opqdjkcf0ArWfFotLriirAO4S4LyMHqEIOsFD9bgFrqEs8vvAy3cDUPoYVkkzBUPJPswa5Uz+AFA4033Oyaq6yDX6c1zLxJGWmWLIuTZD5QVP6iuj8xxpuMXBIu8w+GrpjteKLum2Ad/kfG5BY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740426268; c=relaxed/simple; bh=c2dr56Otg6QzVB7KxPlpb7otqsKJtZ8w8OfoNqMegvI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Xf//I40eZJYkle7BjvlpbzEbDeA6Hm7mPsT6mvT36CYseyXY8Q/6ujCRfIiwjIZU6M1Y/x2SAECZ0lCLi2qLen5i6Bmtc5+aRR99dzC8FaY8ioGOGdR3czkkW3jwzVwf1YlZ+xkI+cg1p3DXfJB8rCsJDHoPZonknGaxDxEXqNw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=U5CugzmN; arc=none smtp.client-ip=209.85.221.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="U5CugzmN" Received: by mail-wr1-f44.google.com with SMTP id ffacd0b85a97d-38f286b5281so2381017f8f.1 for ; Mon, 24 Feb 2025 11:44:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740426264; x=1741031064; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pry5JnUH8YKwWMbGiu+3GyU5H3hss/Z+v4KLGyqRHE8=; b=U5CugzmNgvk8R+6/uyLgWdjfFCXnzBh/NvreyfDRHdplKagy03vdv42Ax86LdYZ7/8 x5ZLTNQpJGGdcq9RRANeiQsmsEjvMq3hZjKTxAvXR+1q7dAhxHTTJaPmy5WMs6x0s2TC NgaL+kP0AnBau4vWxc6FU4wjlTa/RFNaQUJ9rZmenXdHcbtreWFO1B8CN2jRsiOvrP3Y JOHJfKOpjsNlaRhA3r0FB3UscFSIpeJGzKGVbLcsSule8zhax79si1m+5A45jsGstS97 IvhB/mHpAGAl7umWs1cd2dlxgwaU1ezg3BkwBGF8H6tBcUfWS6eD3k8yITVEFITCaZ1d zIFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740426264; x=1741031064; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pry5JnUH8YKwWMbGiu+3GyU5H3hss/Z+v4KLGyqRHE8=; b=g3whiRT+RxwYTbovRLflihiR5YiT8spmGfqxk9xhf3nLEZTP+Ui2Elf3JLQ2AYfQ1D GEeeQMCWncvbMkj/Qfi6KTXPw2BMGJRiwzilYZDJicYl+1qVyKliBukORWpRpYoOCnId s9CPnFQ45vxeFNiVAcXIAGqVPGiTxI8nJhUb+2qcpoheGn353tEF/GDdX6gcb23i+jew V0ZomelMCOcMJJMBpigNwNAXb8RQE8DvTGfZ9kvJbr5+J7jEOEf0h04+5mwtSd83CoTl 3wOvvn1lYnBDqvQDD2Wpeww7gEkHXf1JDDXPzcfTZ/Gir2RZhmgFMmJm9KeHdw7NTyPf 9t3w== X-Gm-Message-State: AOJu0YxsvmnGt7MbGs9iTnOUAa4QUvEC1R3/gTwWHm7bHz/YpdNilXxr 52xqoqrY866j+tbsiAN7QB2452YMApJghzqX/XcUTKi/fsXB+iWj5VlXWQ== X-Gm-Gg: ASbGncun1S9hH/E8WekLlh/C3/tHBeHe0zHTadVAKMiVSQ8OkPsmzXA3vtZIayQ72b2 wj82YQSdoiL0kNHA9CYtRCwvtYRVJ0/ZbgRXJEiED/rJq/4F39+phtQZAiRB/xJOba9DT86ZdJj wAnTL4FGJrc5gSGz8S8TBpDA57erIVGP1G9ASwCb/MUBVYpdtHrrlPMvkHryYeB4gBJV/snDLuQ RntJ7Ew2BDEKRJgH3PeFuwjn5GRntWW2VUipjh3Efm9A4f73MQEWWxE0UfuAWKUtk1ZVHPTB+zl shsoQSMVKqUDXEjJ14dfPSVhVZaXnSIrFyZ+soE= X-Google-Smtp-Source: AGHT+IEa2V7jE1QCv+WkeJPq4WeMpoUkXRVW5Em9Ao9ZoVuhrsxz08eD43yRaxsIHksW3Rz6ozq1bQ== X-Received: by 2002:a5d:584b:0:b0:38d:b28f:564a with SMTP id ffacd0b85a97d-38f7078bf21mr12447784f8f.21.1740426264198; Mon, 24 Feb 2025 11:44:24 -0800 (PST) Received: from 127.0.0.1localhost ([148.252.146.93]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43ab14caa5esm1548305e9.0.2025.02.24.11.44.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2025 11:44:22 -0800 (PST) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com, Caleb Sander Mateos Subject: [PATCH v2 3/4] io_uring/rw: extract helper for iovec import Date: Mon, 24 Feb 2025 19:45:05 +0000 Message-ID: <075470cfb24be38709d946815f35ec846d966f41.1740425922.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Split out a helper out of __io_import_rw_buffer() that handles vectored buffers. I'll need it for registered vectored buffers, but it also looks cleaner, especially with parameters being properly named. Signed-off-by: Pavel Begunkov --- io_uring/rw.c | 57 ++++++++++++++++++++++++++++----------------------- 1 file changed, 31 insertions(+), 26 deletions(-) diff --git a/io_uring/rw.c b/io_uring/rw.c index e636be4850a7..4f7fa2520820 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -76,41 +76,24 @@ static int io_iov_buffer_select_prep(struct io_kiocb *req) return 0; } -static int __io_import_rw_buffer(int ddir, struct io_kiocb *req, - struct io_async_rw *io, - unsigned int issue_flags) +static int io_import_vec(int ddir, struct io_kiocb *req, + struct io_async_rw *io, + const struct iovec __user *uvec, + size_t uvec_segs) { - const struct io_issue_def *def = &io_issue_defs[req->opcode]; - struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw); + int ret, nr_segs; struct iovec *iov; - void __user *buf; - int nr_segs, ret; - size_t sqe_len; - - buf = u64_to_user_ptr(rw->addr); - sqe_len = rw->len; - - if (!def->vectored || req->flags & REQ_F_BUFFER_SELECT) { - if (io_do_buffer_select(req)) { - buf = io_buffer_select(req, &sqe_len, issue_flags); - if (!buf) - return -ENOBUFS; - rw->addr = (unsigned long) buf; - rw->len = sqe_len; - } - - return import_ubuf(ddir, buf, sqe_len, &io->iter); - } if (io->free_iovec) { nr_segs = io->free_iov_nr; iov = io->free_iovec; } else { - iov = &io->fast_iov; nr_segs = 1; + iov = &io->fast_iov; } - ret = __import_iovec(ddir, buf, sqe_len, nr_segs, &iov, &io->iter, - io_is_compat(req->ctx)); + + ret = __import_iovec(ddir, uvec, uvec_segs, nr_segs, &iov, &io->iter, + io_is_compat(req->ctx)); if (unlikely(ret < 0)) return ret; if (iov) { @@ -122,6 +105,28 @@ static int __io_import_rw_buffer(int ddir, struct io_kiocb *req, return 0; } +static int __io_import_rw_buffer(int ddir, struct io_kiocb *req, + struct io_async_rw *io, + unsigned int issue_flags) +{ + const struct io_issue_def *def = &io_issue_defs[req->opcode]; + struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw); + void __user *buf = u64_to_user_ptr(rw->addr); + size_t sqe_len = rw->len; + + if (def->vectored && !(req->flags & REQ_F_BUFFER_SELECT)) + return io_import_vec(ddir, req, io, buf, sqe_len); + + if (io_do_buffer_select(req)) { + buf = io_buffer_select(req, &sqe_len, issue_flags); + if (!buf) + return -ENOBUFS; + rw->addr = (unsigned long) buf; + rw->len = sqe_len; + } + return import_ubuf(ddir, buf, sqe_len, &io->iter); +} + static inline int io_import_rw_buffer(int rw, struct io_kiocb *req, struct io_async_rw *io, unsigned int issue_flags) From patchwork Mon Feb 24 19:45:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13988864 Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3478204F8B for ; Mon, 24 Feb 2025 19:44:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740426269; cv=none; b=bnILVEGyIRRl2GK0ThIIXQTkUQ0QFM3yEn+7j/S/8/t+TayYt+izs5SSTGRgOrg3Y3HmY24Db7Lc1dZ+zZztzCa3R2eR6aZa63SJjXXULBfpcVnUvq+VX+Wj3S0byuZXbJqlGH+vVNW1kaUoYn9Vq7rgJxAnQx6mcxrUe0oVDto= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740426269; c=relaxed/simple; bh=qFOqKTxZOFX9KEJWaWg6+yPiS4lhn8M7Yd4RpOprTWI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kx8i+kYe+/HJO5G0LwxI/8Xdy6oVNeqpT7dwld9MruoHsxevHRoSwKmazfDeExpbSoEnMQMd1WBQEA8w1EXdiOyXAO0VXzZRbX517JDWxU3VLojPlYCvXSUef62R5gNhs1dOWBjoLpP7lYhbGdA6JEXfwTKABwzpKTyiDayIgeI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KQJk4jyP; arc=none smtp.client-ip=209.85.128.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KQJk4jyP" Received: by mail-wm1-f50.google.com with SMTP id 5b1f17b1804b1-43aac0390e8so7427435e9.2 for ; Mon, 24 Feb 2025 11:44:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740426266; x=1741031066; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3k/XwABibW+INb7cZJK+UXq7tXwfthMztjYYatrtEns=; b=KQJk4jyPe8SbWXtDaCwz3a2K3IQ5ZaoUWnfXrTFpAa0JHGPsioXk2c1BXpMo6FK6HO 5jjZnC368117jqI5DctnKCOWz85O+0rd0bviD/YHqTFNIFfBBJbeCva06iWc94I2j1q8 /Tw0Wl6hxto++xrgdR2d4ekSG1k3CLsybdNV/ygFKmfzpBnfC7j8Gt7K80ESjLBfDoAW cZuD6t8MixIaNjBr1IZQgTa5sDA3c49m2WtEqHo9nNQvuhjPU3PeBYV4V3gxyjA/z4V8 E2QfY8yWU6UNj/g7maV9a3DQeV0FsOApzmujMY0V40w6r2gF5utoRkWUU4HK6gtLBH6h Acyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740426266; x=1741031066; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3k/XwABibW+INb7cZJK+UXq7tXwfthMztjYYatrtEns=; b=JpudTYDUbMfhO48A/7sUS6ZkX+F2TIhggTJIkR+Tlo7Q9n8HNcuk1yO1ybL9uHAunE 8bi4CeyXEul3ugUY+tdgcXJoPyuenKC3RD4tRbhgGPG7bPH+J7gwFHg+RBkHWMAZePNS VZZdV1IQUi8clFUyst/DDZfS0oFGKFoeA5M7tdrkJJdxLBOsirahz2RJXD7AH+suPj/k 0HJsd4YPhMBfuR4gu5csSzIzRnnfYB2VG4Mc/xXZiciAZPlL9FTjibiqU6BlgrcyVKIz I/wDbCiS0FZA/ykh9SPjv9WVhIp+jOK5OO6+QmGWsG2/SLwNw+3rS5JaZUrD8sl9+UNE FwmQ== X-Gm-Message-State: AOJu0YwpCoYdekZjPP7xKJFUbtxqWXbLxQ2y0CmYVjRSnLBPV/eWev1+ /LqgkxfMY+bFFiQvKE2VdupY3GL3GozS6ZnMPsfMoPLWPUdOaiJ2qu2KQQ== X-Gm-Gg: ASbGncspSTK3MPAOLWWpBPS4JQ8odTDR0dXQZY2aQ48OIMP5zxm2Z4XyW6M2i8uYb3G cNtaAj8x63R0R/WhDcWDvL2JIXPpywBaHxopadYI70cRqt3H1qbWWX15UHol2+xWU8GTFzSXQOH s1wn7SnDFkbnyVDeejlTpGpctgdKeyrsXlNPWsnEbwkENH6KVl4zad3hjIvzdsATdTIA6amhe3L uQVIEAAFISOA4xDWwNNu0vpxoQx2FyM0ZdD3LkuwKTcvnfS5nG1ux3tejdCkjDKR2L9ejE7CDaa F6rUjVi/hAHOmS4IhPW0NQELny1GcvyNqVV376Y= X-Google-Smtp-Source: AGHT+IHkma8nETQpcFC3zRQNeua888TCLcQHV4ooN6UMH0dpez9iNOgat/L8phmHYWTyhokUdmnGGQ== X-Received: by 2002:a05:600c:4686:b0:439:a25b:e7d3 with SMTP id 5b1f17b1804b1-439ae1f4153mr150455055e9.14.1740426265524; Mon, 24 Feb 2025 11:44:25 -0800 (PST) Received: from 127.0.0.1localhost ([148.252.146.93]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43ab14caa5esm1548305e9.0.2025.02.24.11.44.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2025 11:44:25 -0800 (PST) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com, Caleb Sander Mateos Subject: [PATCH v2 4/4] io_uring/rw: open code io_prep_rw_setup() Date: Mon, 24 Feb 2025 19:45:06 +0000 Message-ID: <61ba72e2d46119db71f27ab908018e6a6cd6c064.1740425922.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Open code io_prep_rw_setup() into its only caller, it doesn't provide any meaningful abstraction anymore. Signed-off-by: Pavel Begunkov --- io_uring/rw.c | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/io_uring/rw.c b/io_uring/rw.c index 4f7fa2520820..cb660a224e90 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -204,17 +204,6 @@ static int io_rw_alloc_async(struct io_kiocb *req) return 0; } -static int io_prep_rw_setup(struct io_kiocb *req, int ddir, bool do_import) -{ - struct io_async_rw *rw; - - if (!do_import || io_do_buffer_select(req)) - return 0; - - rw = req->async_data; - return io_import_rw_buffer(ddir, req, rw, 0); -} - static inline void io_meta_save_state(struct io_async_rw *io) { io->meta_state.seed = io->meta.seed; @@ -287,10 +276,14 @@ static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe, rw->addr = READ_ONCE(sqe->addr); rw->len = READ_ONCE(sqe->len); rw->flags = READ_ONCE(sqe->rw_flags); - ret = io_prep_rw_setup(req, ddir, do_import); - if (unlikely(ret)) - return ret; + if (do_import && !io_do_buffer_select(req)) { + struct io_async_rw *io = req->async_data; + + ret = io_import_rw_buffer(ddir, req, io, 0); + if (unlikely(ret)) + return ret; + } attr_type_mask = READ_ONCE(sqe->attr_type_mask); if (attr_type_mask) {