From patchwork Mon Feb 24 16:07:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13988442 Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 664C725A625 for ; Mon, 24 Feb 2025 16:06:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740413200; cv=none; b=OUkE+V7YyDnqI+PwzKYwMvE+syR5HLtxZu2Qj9MYngLCBrOHui1qh+3DEpV+z0TAHSJYLKHiYjtcjMeXlVJJLwd+xhqotGCmc+3giCQuho7LMAInC4SkqOysc3vDrzM+ESwlYEdGL+UiNcRDWQalf11I6x3Y0vYqsHznjv+HsLs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740413200; c=relaxed/simple; bh=JmJIAh32myNNvoDyHd1rD5NGh99j0D+7ZJU8Yx75BOQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZOqb98C8HVN7BCLtvfy5P+9gIhASBws1mLvMjFIGqMspfA4HOgMMG19WVICwaJsnUjh9MC4ZoWl7v7l4BR4JK+OqulYYPTPn/LoCREuhUhQsA0XjOlzrYA+eqyPmx8ZpjaMCHQcRoTDUcZGyk+dUdVc2W+4jyIohRYFDkOW1+9c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=M9I97/EN; arc=none smtp.client-ip=209.85.218.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="M9I97/EN" Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-aaecf50578eso901634766b.2 for ; Mon, 24 Feb 2025 08:06:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740413196; x=1741017996; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IqBysNL4N/FiHuDdD1Yn9dA7GNQUgwsmQ1Zzh8zAUbU=; b=M9I97/ENqcSf3jfTCmw3lJyGwgONofNJij3zmxFOfocbYzLlxXHgjK6famCbND9ykW ZNK1kImdYFl282spOs2jGJ1CqrIq0v+Vq1H8VTg0wQfA232kx2I9QoZbwpFZqd4cAtKw o6CyTEmouNZZ99pMvjJPAAZWwZcjOH+JHJes2oiliWD6W5ZDM/zlQOHoupgMX2CR4X+U 5INw/lKa7/znYIZkl6D+xL303clTXJ0GRnVwpFCLqS8c5cn0xzpGSEdhYaQtDfuDFGTd z+D2ZdZjXEk/LkajU/ScY1SEOGiFVZ1nTSJhwgs8nxABcV5gaEGwgwk28iWErlyHf5j5 y0qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740413196; x=1741017996; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IqBysNL4N/FiHuDdD1Yn9dA7GNQUgwsmQ1Zzh8zAUbU=; b=e50Ht7yieasG7sRDssilwEQkCIAF/uKGqqpHvkD6GGE1ReN0a2Xep6eXp8vY0cMDsm 9fpD4p57XU1DRBpu/+2ytJxAJg3LcRCTdXqgXLiaNVIornFfcWlgjVjqrqkLlXfQ/qCR 38odTyOshRy3CXoGuYloaYCpp7NZQdfnNZwe/mVfbQ0kS/jS5jNM+xq+Nxc5jyWIatnh KOlxoPYgrpv5DhD7DmivzZllv86r2uCSrvV3uHvR/noRbTQVzMjPlkz4ajOT39A6mBTJ tOo1DgFUXPPTRDjrCgQq83MqVOAiG41GhM39kEsXzZxCsA5gBfhIA73BIUK59CnT883g Kueg== X-Gm-Message-State: AOJu0YyIeeabGZ2Bh9S8pV0oOZYItMCJys3lNL07AvDORy6OuXmNWYl4 XFm5zbGDGUHJAt1LijZ6bYYJuHovGspIP2LnmOrSm8E98ADBrqSz/ouhRA== X-Gm-Gg: ASbGncu4CGa9Dsf4ggXwMmV2WYZfsIt5IIEk2u60HKhJ2Ef0JhZdn1WKIY2UG+hd7ZB EI0ZAxHxHYKCauxKTeQ7gv5wyDS9Q5kxkrHr0o6/SBL1NYRLZBKSeFdqw1P+u+VlqQEcXbj3tTs sqxPTK1c3gNxhya4ra8bvCgt4tkR1iKNJjk99cc+FtdPJT4I9WXdB3l1ysmZDMuOaBrEo3rpg8Q D+w5ukxm9VdEbFfrmg4HRyid2BRfsXzRbsCLkJH7P9cLqfiAUeHHD5f7n49hkuXMV5VykCDCH8w G0b5HsGWpg== X-Google-Smtp-Source: AGHT+IHk7swaQCLWHaTkBo8uFir+pbTMZ+A2SIPZjEnaxJwog1tewwBI1KnRObGqllTHGw7iUX1K7g== X-Received: by 2002:a17:906:3119:b0:abb:d04c:6947 with SMTP id a640c23a62f3a-abc0d994a14mr1034640366b.8.1740413195944; Mon, 24 Feb 2025 08:06:35 -0800 (PST) Received: from 127.com ([2620:10d:c092:600::1:bd30]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-abb95cc7451sm1664684566b.92.2025.02.24.08.06.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2025 08:06:35 -0800 (PST) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com Subject: [PATCH 1/4] io_uring/rw: allocate async data in io_prep_rw() Date: Mon, 24 Feb 2025 16:07:22 +0000 Message-ID: <2bedcfe941cd2b594c4ee1658276f5c1b008feb8.1740412523.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 io_prep_rw() relies on async_data being allocated in io_prep_rw_setup(). Be a bit more explicit and move the allocation earlier into io_prep_rw() and don't hide it in a call chain. Signed-off-by: Pavel Begunkov --- io_uring/rw.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/io_uring/rw.c b/io_uring/rw.c index 22612a956e75..7efc2337c5a0 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -203,9 +203,6 @@ static int io_prep_rw_setup(struct io_kiocb *req, int ddir, bool do_import) { struct io_async_rw *rw; - if (io_rw_alloc_async(req)) - return -ENOMEM; - if (!do_import || io_do_buffer_select(req)) return 0; @@ -262,6 +259,9 @@ static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe, u64 attr_type_mask; int ret; + if (io_rw_alloc_async(req)) + return -ENOMEM; + rw->kiocb.ki_pos = READ_ONCE(sqe->off); /* used for fixed read/write too - just read unconditionally */ req->buf_index = READ_ONCE(sqe->buf_index); From patchwork Mon Feb 24 16:07:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13988443 Received: from mail-ed1-f47.google.com (mail-ed1-f47.google.com [209.85.208.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 61823261583 for ; Mon, 24 Feb 2025 16:06:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740413201; cv=none; b=nMN8GNgyEW2r+gxCZ4JFhICVcyyMPh1je9zZdTC619mYYoln8k4OSB9gvFn2dkeS3aaNpCckoiLFW2+i0gicDFvx+ykt7I82g6o0O1Btm0hfJhZgYf1v5AyBcOmcE+ZdDhEj7e8YaMGTZfYf70+gJyIT0CF8dwG9pGOYOihCtFk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740413201; c=relaxed/simple; bh=ZiWEpyrbRZKpS6hkr7hzo1dG6dzpZJ5uoEu210Dzmgs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DhMg0qT1ozmlKYiBeY/DiVUiQEsLobueiSD0F/dcvenRKlQkmXzeOlBOaENT0QFUh0ykL00V7xqiq1AM/FDdyWLShSHn04XlGw3ZXuOZ14pSBVlqEzN5OyrWXGhdYeqrrpYNK7aHxHWDV46sk5A1sE3MlT2jun74fYy0puZ8KYE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=eMJCwuXR; arc=none smtp.client-ip=209.85.208.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eMJCwuXR" Received: by mail-ed1-f47.google.com with SMTP id 4fb4d7f45d1cf-5dec817f453so7604327a12.2 for ; Mon, 24 Feb 2025 08:06:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740413197; x=1741017997; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QK9YGtEODcnKoP57rreMyeESG4woNuiJuZC9WxFY5T0=; b=eMJCwuXRVGIIb27HT2WgaGf++Qh5ScYIGOyyt8E0KsFZ4uA+iiK0sMOsHCepxKTIHO 0TtDkiImTb+CSaKw8tqW6X4ErkOYJinyupfZNPpmWQSKfJfji27sy6v+3jaYhPI8qRf/ L1YmFXdq13C1cK777kapmuJUO42QDvQdSrZcT6SgRhLM632f2DY22uMOCjV0WkCW8/6G pQpTHVtM5um91EOHXH3BMOjAgL/sDQNEMwQJxA9HNE2pFXJIKGl5Ovb9vxO6PLVCSe2N IZ/5clkykn1JCzshDV37Vq3YFltDsoEdcO6JMZEX3f0p6Wf3Hxh0CaYZGHvr0tEQDRVs knaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740413197; x=1741017997; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QK9YGtEODcnKoP57rreMyeESG4woNuiJuZC9WxFY5T0=; b=E7B5gDOPjBhKAetiVkJNMMmvkFR1pftowQqRWeO7OH+ffMok4SDtaWotwKQkc96ElA mcWUays5b+2yuSwOu19hqb5LO20zhWbfcE+sliaNqqC4NvTlV2gNs0LW5W6LrgITKlar MMkRViZ92vtp/e1zKtcIewHCpnkXIFwbntiDKMas33sZewXZTRUxQBBY04+MRd1TLwaH l+eaJlg35wAiaukShwwBKP2/aK9FDOTZKhkMIe34fhR7/9+sRkpBf3iEvUBl/Q4W2neB QpySsSMHNFWKMSC2qUjhw/2eGAAZQoqcD85wdkklFdwZI63D8tyJYVaBeEOVty0LLaLM 4jjg== X-Gm-Message-State: AOJu0Yxzj2CmZ8XX7oiUSaRy939hVvOFi/XGP3XDJRdYRdeuLM0z3NSJ yMdOgWrfu0CMIpwTelWb27uSAh8hTakFhZduJV1kTVCZqcr/YqHN1hRM6A== X-Gm-Gg: ASbGncvY3sybpCIouU7UZfiwVUZOi+5VT/x6IuFitMUY7/2N6gXnkCMU9Jdp3lCUhVn M+t9eAeMOaaczDaAbOgBH/qaHp0YkJqheO/hGc4wRgFVWwt0Vy+aGbKk4Fc3fS/uOF9PVI89Xul RtLTMjq+TSDfm+Jc0g1aD4jUA3agh2KQ1diXyRL7LDBn2oWPRJVe2MOTMZDlREiTWN06z57Ya6+ tLiD4PdpSpv6SoLUALwfb4Zc6WXq4hj4acaVu1IF0pFXVzjhxngBUKyMkL/lVaicsV1W5l5ohP5 wq9Pj7M0zg== X-Google-Smtp-Source: AGHT+IFXIYK5WVyWjAr4xNyXz8iaGrsF6jytUfZykN8TEJQd33k3335cHz42s1KoUerTZyJ2zat/3w== X-Received: by 2002:a17:906:3290:b0:aa6:9eac:4b8e with SMTP id a640c23a62f3a-abc09bf5188mr1283654866b.41.1740413196804; Mon, 24 Feb 2025 08:06:36 -0800 (PST) Received: from 127.com ([2620:10d:c092:600::1:bd30]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-abb95cc7451sm1664684566b.92.2025.02.24.08.06.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2025 08:06:36 -0800 (PST) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com Subject: [PATCH 2/4] io_uring/rw: rename io_import_iovec() Date: Mon, 24 Feb 2025 16:07:23 +0000 Message-ID: <2c8fa231c1096b81f909d327fcdbf0ddf1855afd.1740412523.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 io_import_iovec() is not limited to iovecs but also imports buffers for normal reads and selected buffers, rename it for clarity. Signed-off-by: Pavel Begunkov --- io_uring/rw.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/io_uring/rw.c b/io_uring/rw.c index 7efc2337c5a0..e636be4850a7 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -76,7 +76,7 @@ static int io_iov_buffer_select_prep(struct io_kiocb *req) return 0; } -static int __io_import_iovec(int ddir, struct io_kiocb *req, +static int __io_import_rw_buffer(int ddir, struct io_kiocb *req, struct io_async_rw *io, unsigned int issue_flags) { @@ -122,13 +122,13 @@ static int __io_import_iovec(int ddir, struct io_kiocb *req, return 0; } -static inline int io_import_iovec(int rw, struct io_kiocb *req, - struct io_async_rw *io, - unsigned int issue_flags) +static inline int io_import_rw_buffer(int rw, struct io_kiocb *req, + struct io_async_rw *io, + unsigned int issue_flags) { int ret; - ret = __io_import_iovec(rw, req, io, issue_flags); + ret = __io_import_rw_buffer(rw, req, io, issue_flags); if (unlikely(ret < 0)) return ret; @@ -207,7 +207,7 @@ static int io_prep_rw_setup(struct io_kiocb *req, int ddir, bool do_import) return 0; rw = req->async_data; - return io_import_iovec(ddir, req, rw, 0); + return io_import_rw_buffer(ddir, req, rw, 0); } static inline void io_meta_save_state(struct io_async_rw *io) @@ -845,7 +845,7 @@ static int __io_read(struct io_kiocb *req, unsigned int issue_flags) loff_t *ppos; if (io_do_buffer_select(req)) { - ret = io_import_iovec(ITER_DEST, req, io, issue_flags); + ret = io_import_rw_buffer(ITER_DEST, req, io, issue_flags); if (unlikely(ret < 0)) return ret; } From patchwork Mon Feb 24 16:07:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13988444 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 311A826158D for ; Mon, 24 Feb 2025 16:06:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740413202; cv=none; b=H/n4UPU2l1s+N8HD6csmAcaMWJS2urqp/sMrG/p9OIji7UxcOfWMO5qg+E4kLjl4Tw2DtEgTdCtdp2LZme14M0KVwEMPObuCj71js6cF1idhfCOAQqJ/uH5/vAChArBFKV5o/TLIisHt14DnQaR/sJqfPFGUQLzmtuXNvOBSaaM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740413202; c=relaxed/simple; bh=PvuHJY1tomndor+upuIhjvvcJnPklhdETpzFR/6V5qI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lBlsRxp6NAfj40UimKv72KbIRQ5BKxSldVrt6Ov7vElDgUKspml+W/g/U34wSELq8s25Yi41t3B2AU0fV4A5lres06A86Yf1I6fA3l6A+6aCQeIoIYSVqzBbwP3eKx1sitsHAmRXG0X7lJGOUGUNKERygWCYg2YZXsSH3Vwm6UA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QLz9su/U; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QLz9su/U" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-aaee2c5ee6eso669807266b.1 for ; Mon, 24 Feb 2025 08:06:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740413198; x=1741017998; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YrxwZjz3oqJZKxsLAKf5JWC6HBJTle6L1MVSIMNCL3M=; b=QLz9su/UN2/mRUJiJh4Q78abQi8F1FmW+1uW+PcXTDWe5G5GiCZx9jCX+AWutVI80w Uk28Acp7cvLpS9vlMz0Alg++e3snZMg6uuepab5h7SsFQmI2XzMuPEq18J4yQnMRJ5Pz pjyCF4HJKNWDJme9F4BESEnJ+BCM2R9h2B/YAVP1WsiraQw3dBeg+BLsnIoHARP10UvY Cc1yI9mfB75jw7n00fT7UZHM20cvOlzNNDgOws1yDAIVTqJIxTOMyUKarLeDuRGrqvIG relVywWOZFFfkszfS38MEsOPYxIjfA7ZzJVj0Ci9E4D6jJUqC8sb8tddBG31ywsHmIUF UkVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740413198; x=1741017998; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YrxwZjz3oqJZKxsLAKf5JWC6HBJTle6L1MVSIMNCL3M=; b=gNTV5wAAkEK90Cf4I/jr3n6n+F/jjMJD1YC4UwpAAuAoZ5iqdt8WKRIdQUmFvLsZ66 2z7JIcTfxtkYaEUPw8xdCz65zR+KSMXcyi8AottR3H4UD/Ymms4oJaPPGrPGAnj0B+Bz 6pew9+PuAwm8BYL4oG1hrIkB5Y+OmnataQVEdq/IbckCOXP8BRw/Gr/3j9H/uC6sWDpZ gjoPf2kDdilmi/sifrphv/3g/yiwPHLfWq+XUODih9Qw4Bm8MLU7/eQIyJWZbzGYDK0P 9JkRwRVd3C9cBvYxtN+U73g65GCM0yEHTXzvXajTkNsSLTTqvtU3AQmDXGwnH6Y/iyXh j30g== X-Gm-Message-State: AOJu0Yx79/5trt5gVvGO04LA1+xi+UD2AchxO7nzvbUI4B6VE7+9Ho/F 5TIa4Ht8yiWiBUx9KMbJCeYgHzxT3FHjIu2niczooHkR8pNWi1SBKgcpvw== X-Gm-Gg: ASbGncu1AERppHQsQMrRqdl8mQ4fA9NMi50okeTt1e2ahtjzeFPy6bvKKQX93t9YwCi VcmdfNt8lBXgq8Oj7HSNV+6B0oMOsdXx/1OqTx0JhloORDT025wF8WeaAcK3vb24vNPoBLFuirO Jox5/qcpN/VfzuOC3uPBsEQaWe5vYA8tGE6RgDcPeYfGkYA0GZeMDt/V7cXSvoKXXY9O12SN4ss A0jsGDAWeOXNnRS9CjeIe4jnKeC/ynSj9HdYjFZMRQ1fhLNFBpOawtp6D2jYdWgxiU1+Mq/Yq4G 1eb7Tn9llQ== X-Google-Smtp-Source: AGHT+IGYu44Iaa9x7wRVHyam1a72n7LbdGx1oKbelMKLDPqHKgK0eADpPip4J+WydVYRKwlQyM/C2g== X-Received: by 2002:a17:907:948c:b0:ab7:e3cb:ca81 with SMTP id a640c23a62f3a-abc09aac8cbmr1399448766b.30.1740413197750; Mon, 24 Feb 2025 08:06:37 -0800 (PST) Received: from 127.com ([2620:10d:c092:600::1:bd30]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-abb95cc7451sm1664684566b.92.2025.02.24.08.06.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2025 08:06:37 -0800 (PST) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com Subject: [PATCH 3/4] io_uring/rw: extract helper for iovec import Date: Mon, 24 Feb 2025 16:07:24 +0000 Message-ID: <5cf589c0efb611bfe32fc3b69b47d2b067fc8a65.1740412523.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Split out a helper out of __io_import_rw_buffer() that handles vectored buffers. I'll need it for registered vectored buffers, but it also looks cleaner, especially with parameters being properly named. Signed-off-by: Pavel Begunkov --- io_uring/rw.c | 57 ++++++++++++++++++++++++++++----------------------- 1 file changed, 31 insertions(+), 26 deletions(-) diff --git a/io_uring/rw.c b/io_uring/rw.c index e636be4850a7..0e0d2a19f21d 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -76,41 +76,23 @@ static int io_iov_buffer_select_prep(struct io_kiocb *req) return 0; } -static int __io_import_rw_buffer(int ddir, struct io_kiocb *req, - struct io_async_rw *io, - unsigned int issue_flags) +static int io_import_vec(int ddir, struct io_kiocb *req, + struct io_async_rw *io, void __user *uvec, + size_t uvec_segs) { - const struct io_issue_def *def = &io_issue_defs[req->opcode]; - struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw); + int ret, nr_segs; struct iovec *iov; - void __user *buf; - int nr_segs, ret; - size_t sqe_len; - - buf = u64_to_user_ptr(rw->addr); - sqe_len = rw->len; - - if (!def->vectored || req->flags & REQ_F_BUFFER_SELECT) { - if (io_do_buffer_select(req)) { - buf = io_buffer_select(req, &sqe_len, issue_flags); - if (!buf) - return -ENOBUFS; - rw->addr = (unsigned long) buf; - rw->len = sqe_len; - } - - return import_ubuf(ddir, buf, sqe_len, &io->iter); - } if (io->free_iovec) { nr_segs = io->free_iov_nr; iov = io->free_iovec; } else { - iov = &io->fast_iov; nr_segs = 1; + iov = &io->fast_iov; } - ret = __import_iovec(ddir, buf, sqe_len, nr_segs, &iov, &io->iter, - io_is_compat(req->ctx)); + + ret = __import_iovec(ddir, uvec, uvec_segs, nr_segs, &iov, &io->iter, + io_is_compat(req->ctx)); if (unlikely(ret < 0)) return ret; if (iov) { @@ -122,6 +104,29 @@ static int __io_import_rw_buffer(int ddir, struct io_kiocb *req, return 0; } +static int __io_import_rw_buffer(int ddir, struct io_kiocb *req, + struct io_async_rw *io, + unsigned int issue_flags) +{ + const struct io_issue_def *def = &io_issue_defs[req->opcode]; + struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw); + void __user *buf = u64_to_user_ptr(rw->addr); + size_t sqe_len = rw->len; + + if (!def->vectored || req->flags & REQ_F_BUFFER_SELECT) { + if (io_do_buffer_select(req)) { + buf = io_buffer_select(req, &sqe_len, issue_flags); + if (!buf) + return -ENOBUFS; + rw->addr = (unsigned long) buf; + rw->len = sqe_len; + } + return import_ubuf(ddir, buf, sqe_len, &io->iter); + } + + return io_import_vec(ddir, req, io, buf, sqe_len); +} + static inline int io_import_rw_buffer(int rw, struct io_kiocb *req, struct io_async_rw *io, unsigned int issue_flags) From patchwork Mon Feb 24 16:07:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13988445 Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 12D6A261591 for ; Mon, 24 Feb 2025 16:06:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740413202; cv=none; b=SQgLHxfopy5kaXoErGJAw+J/fROkXAWoPzCAID3aBCwfJGXkzBBhnaTlC2YitgZEJ7jJtby9j6mNbeCfmv/FL4hUBUq4mm9foRm0BGi6/Ppuncp4aoYVyF5iIORe8K2Ynu36zGcUOWkR9zSTDVN8Ca3UZ3qw4MqdNxS7xfzBFVQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740413202; c=relaxed/simple; bh=iyEeXQ7xsxxrWsm1X4Rszb9X10yIS5n63SBfynG342A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qQOKSRgr7dZsyVwOE2t0WhF6A5NCBjUUbhT/v3pyg91T48mZ6ULqSQbGNJwszxV9/LQfcmohdbJYck0qukjqi+KCk8qfb5dlfhpzbOc8mimerwZnN8xOU6jyraarW0W3S8QVKzxqLaLq6D0EiVS5pvfRjHm1Dy19iRmlbwsBRO4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=k9FX35e0; arc=none smtp.client-ip=209.85.218.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="k9FX35e0" Received: by mail-ej1-f47.google.com with SMTP id a640c23a62f3a-ab7483b9bf7so750697166b.3 for ; Mon, 24 Feb 2025 08:06:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740413199; x=1741017999; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=33bFwdlXi6q1zIThWD9p1qAnKpNPjhHJq3izUsokAd4=; b=k9FX35e0nxdvUi3s4R8pdgCeRKCdYZdbrh5AH8Pm81VDs8gVBb6Yb5cu2qfNZDCLyS di+kcli3JkDLS5Uxo2FOsEKQv6xj3oU4uuvMCTJ4yY+Q3vafLCW5KAbBvRQaJsYxBDeG dTZK0MQhC55uj/Ukrxh6YmN7jOTOpwyYCl/lYk00SfFTOQ71BmPM1tyG24dIrKZlMyKA 6EeO5v37+EfIJjFdVvd5rMxwRvyQXFM2uO+gu54mbRMg9PDSeb6C6Z802Y0+t35wzV8d CdsaUetS/6DhqdDptwzEFFkYFoJA4OdAM84hc8arhA/bpEk4V2P4GjDFNmvqyzbXP16h w6aA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740413199; x=1741017999; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=33bFwdlXi6q1zIThWD9p1qAnKpNPjhHJq3izUsokAd4=; b=pUG15Gu0e9bYrP2VPeRR81atIev3nhaF2Bz0cz2BtzHCWpaR0mAVRgTfiTwgD9EmDC KLs8IIYcHQMcF+hwag0q4kRKKv7kYzEiikE5rkwObfvku0YzoFGRrvWIrvDHIcjFnBdj hEXgkVibRARxP9uBFkQuDftiJlsgExoVA6RVEGMth59y2i+dYW9BA74Uuba+zCGqih0R QeMKsoCGc2EOLWYxLpjVC+wvs+P3gtYQve/PdfMvppjvn80Rvt0FUPHwLqSG6/P1B9yt hvzQGJ85qn0E8XSRMfdcPzgMzpHUvePW1ws4yJriq0DI2Mr6mOW0zyDAP0plMzP9nDLY nygg== X-Gm-Message-State: AOJu0YzPbKemkf78YZ5N7cvoiPKDFu0jYWCW7mt1qEV5MGzgLsIbZ+y9 kBOBwjCiTZX52QzSsSJLFaUw+Jo37FAcg71fgfjd+4XDN3UtRY/gFeAoEg== X-Gm-Gg: ASbGncsb1A/mwGU16U4HXZAWBJg26Y6snVf96E2fEG3ABIr2gDazeAZamHZKA12u0Cr epISj3Zu5G4Bw/nuXrAgsp8n7m0J0aFrvdLzPD8D5spcEUjqSYKH/kjnBAglz7XJHoI5kR78Vcs VPXgra8bpJlBeynYEoeJvLOKlFWX2H0pMCkTFwvl8g6es/BIGVjW8d34JDJjDGBVPYU1m0BYkTK vyNOvr2UAiwG1lUh1aGdh5LsWTfSP1W0gSGx5sef0o2wGo+wd53k6+ysIR4REJ4k4iVzRw7aIG5 FhuhIpVVIw== X-Google-Smtp-Source: AGHT+IE7EF03dQx6L3t51xHutNk3Y+/C7khygHix/MgWo57O/dFPpAHV/wl3KtNk09RryyYF/UjYyA== X-Received: by 2002:a17:907:da7:b0:abb:33ff:c5f5 with SMTP id a640c23a62f3a-abc09c27017mr1513633166b.48.1740413198603; Mon, 24 Feb 2025 08:06:38 -0800 (PST) Received: from 127.com ([2620:10d:c092:600::1:bd30]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-abb95cc7451sm1664684566b.92.2025.02.24.08.06.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2025 08:06:38 -0800 (PST) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com Subject: [PATCH 4/4] io_uring/rw: open code io_prep_rw_setup() Date: Mon, 24 Feb 2025 16:07:25 +0000 Message-ID: <60470a586db34ed3a5d9c992564acf29aa846c9c.1740412523.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Open code io_prep_rw_setup() into its only caller, it doesn't provide any meaningful abstraction anymore. Signed-off-by: Pavel Begunkov --- io_uring/rw.c | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/io_uring/rw.c b/io_uring/rw.c index 0e0d2a19f21d..2e35e7e68ff8 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -204,17 +204,6 @@ static int io_rw_alloc_async(struct io_kiocb *req) return 0; } -static int io_prep_rw_setup(struct io_kiocb *req, int ddir, bool do_import) -{ - struct io_async_rw *rw; - - if (!do_import || io_do_buffer_select(req)) - return 0; - - rw = req->async_data; - return io_import_rw_buffer(ddir, req, rw, 0); -} - static inline void io_meta_save_state(struct io_async_rw *io) { io->meta_state.seed = io->meta.seed; @@ -287,10 +276,14 @@ static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe, rw->addr = READ_ONCE(sqe->addr); rw->len = READ_ONCE(sqe->len); rw->flags = READ_ONCE(sqe->rw_flags); - ret = io_prep_rw_setup(req, ddir, do_import); - if (unlikely(ret)) - return ret; + if (do_import && !io_do_buffer_select(req)) { + struct io_async_rw *io = req->async_data; + + ret = io_import_rw_buffer(ddir, req, io, 0); + if (unlikely(ret)) + return ret; + } attr_type_mask = READ_ONCE(sqe->attr_type_mask); if (attr_type_mask) {