From patchwork Mon Dec 20 14:17:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12688939 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A55EC4332F for ; Tue, 21 Dec 2021 02:56:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234048AbhLUC4S (ORCPT ); Mon, 20 Dec 2021 21:56:18 -0500 Received: from mailout3.samsung.com ([203.254.224.33]:45855 "EHLO mailout3.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232385AbhLUC4R (ORCPT ); Mon, 20 Dec 2021 21:56:17 -0500 Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20211221025615epoutp03767e2d013b0146d6fa6e34ced3052a10~Cpayw8HoP3071930719epoutp03a for ; Tue, 21 Dec 2021 02:56:15 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20211221025615epoutp03767e2d013b0146d6fa6e34ced3052a10~Cpayw8HoP3071930719epoutp03a DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1640055375; bh=lxZ3y46Zzqi42oELo2sbv5mkNnzILaZBKfOldBs6qM0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oVh0DdxsB36sAHAGqqsrnaFBm5GSqJ+/CYtwSFtv+NobcQrIk045yynrnQZMpYz/x qeLajGKNnDRV5rn/VqI9Cn2Nc7kvH9javOq6xgSwuLC/3bLn6Oy0Cf/HGOYZEiMd1x wTJeWJ/BELBUCRi8r8nVvDWm/MITakiqsKXe/IPw= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20211221025614epcas5p1efa0134e18e9b2f8bf039c2d82ab4931~CpayPW6G61825618256epcas5p1V; Tue, 21 Dec 2021 02:56:14 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.181]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4JJ1LL6Lm7z4x9Q9; Tue, 21 Dec 2021 02:56:10 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 4B.D1.46822.84241C16; Tue, 21 Dec 2021 11:56:08 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20211220142228epcas5p2978d92d38f2015148d5f72913d6dbc3e~CfIqTXAx22775527755epcas5p2B; Mon, 20 Dec 2021 14:22:28 +0000 (GMT) Received: from epsmgms1p2.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20211220142228epsmtrp1bc48eb49aa183de9078ecce0162bab08~CfIqSnczN2445924459epsmtrp1U; Mon, 20 Dec 2021 14:22:28 +0000 (GMT) X-AuditID: b6c32a4a-dfbff7000000b6e6-57-61c142489b41 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p2.samsung.com (Symantec Messaging Gateway) with SMTP id 24.65.08738.4A190C16; Mon, 20 Dec 2021 23:22:28 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20211220142227epsmtip1fb486648edb179508517924532dd8f9c~CfIolz2kX0637406374epsmtip1q; Mon, 20 Dec 2021 14:22:26 +0000 (GMT) From: Kanchan Joshi To: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, javier@javigon.com, anuj20.g@samsung.com, joshiiitr@gmail.com, pankydev8@gmail.com Subject: [RFC 01/13] io_uring: add infra for uring_cmd completion in submitter-task Date: Mon, 20 Dec 2021 19:47:22 +0530 Message-Id: <20211220141734.12206-2-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211220141734.12206-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrFJsWRmVeSWpSXmKPExsWy7bCmhq6H08FEg68bOCyaJvxltlh9t5/N YuXqo0wW71rPsVh0nr7AZHH+7WEmi0mHrjFa7L2lbTF/2VN2izU3n7I4cHnsnHWX3aN5wR0W j8tnSz02repk89i8pN5j980GNo++LasYPT5vkgvgiMq2yUhNTEktUkjNS85PycxLt1XyDo53 jjc1MzDUNbS0MFdSyEvMTbVVcvEJ0HXLzAG6UEmhLDGnFCgUkFhcrKRvZ1OUX1qSqpCRX1xi q5RakJJTYFKgV5yYW1yal66Xl1piZWhgYGQKVJiQnbH7pVrBBZGKP7/XszYwbhPsYuTkkBAw kTj3sZO9i5GLQ0hgN6PE6h8nGCGcT4wSOy/chcp8ZpSY0DCLHaZlxa6FTBCJXYwSV/Z8YIOr ujlpM1A/BwebgKbEhcmlIA0iAtESF55fYwOxmQU6gMZ224LYwgKhEr+nvQeLswioSrS9vsYK YvMKWEhcO30Hapm8xMxL38FsTgFLicOzl7FB1AhKnJz5hAViprxE89bZzCA3SAj0ckgsWPWL EaLZReLL8xWsELawxKvjW6CGSkm87G+Dsoslft05CtUMdNz1hpksEAl7iYt7/jKBPMMM9Mz6 XfoQYVmJqafWMUEs5pPo/f2ECSLOK7FjHoytKHFv0lOoveISD2csYQUZIyHgIXH2iAUkrHoY JRpuLWabwKgwC8k/s5D8Mwth8wJG5lWMkqkFxbnpqcWmBUZ5qeXwSE7Oz93ECE60Wl47GB8+ +KB3iJGJg/EQowQHs5II75bZ+xOFeFMSK6tSi/Lji0pzUosPMZoCA3wis5Rocj4w1eeVxBua WBqYmJmZmVgamxkqifOeTt+QKCSQnliSmp2aWpBaBNPHxMEp1cAkuOy8wORzkxcWbjfe2mNU I7s679CWL9suOfW5Jib0dYfVOC48p8MaqGxfZOfKyvdKbYtn8JmvUVv6Mj4dvKL01MbnhcZM BZsN21fo88j9n2gQvnTrhiXq1iYt63rqfkTxvOJkFU23Dv5Z3nR7Y3lhCzdbYue8/09iJKQe cDhP6/vhyekTYxjLtuSCidKcv2vuTmyZdMf+7TyuT1tPN3CnHV4lbTfXJ6NZ9sTcqgPrfTbx MfmZbPMqPz3RY+m+k09Ely/s/MFsbLjr9NwNX9yz0yeVZj/an5cj3MZufK5w7UWJnTpckWc8 evp/T0lPWXTp6IE0NebMiXnr19bld9Vut3nxav2SDkbvpbq8Vw8osRRnJBpqMRcVJwIAL7/F pz0EAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrPLMWRmVeSWpSXmKPExsWy7bCSnO6SiQcSDbZ91LNomvCX2WL13X42 i5WrjzJZvGs9x2LRefoCk8X5t4eZLCYdusZosfeWtsX8ZU/ZLdbcfMriwOWxc9Zddo/mBXdY PC6fLfXYtKqTzWPzknqP3Tcb2Dz6tqxi9Pi8SS6AI4rLJiU1J7MstUjfLoErY/dLtYILIhV/ fq9nbWDcJtjFyMkhIWAisWLXQiYQW0hgB6PE/1sVEHFxieZrP9ghbGGJlf+es0PUfGSUODYt t4uRg4NNQFPiwuRSkLCIQKzEh1/HgMZwcTALTGKU2ND/AKxeWCBYYvvub2wgNouAqkTb62us IDavgIXEtdN3oObLS8y89B3M5hSwlDg8exkbxC4LiRMfvrBA1AtKnJz5BMxmBqpv3jqbeQKj wCwkqVlIUgsYmVYxSqYWFOem5xYbFhjlpZbrFSfmFpfmpesl5+duYgTHgZbWDsY9qz7oHWJk 4mA8xCjBwawkwrtl9v5EId6UxMqq1KL8+KLSnNTiQ4zSHCxK4rwXuk7GCwmkJ5akZqemFqQW wWSZODilGpjqmHVfnwrz3N5w58rRJWFTqo1tmMTNnteU3f9+oTvMd+3aZVcX5TFe/KO/Pm2n w+eQknN3/V47zXn15PTtDSUn1h+Wia5k8y7US7/pu3KZuXBG+qPZ5xha2Ps0Z8fvttVYK3dh wpzSbK3feipn5+u39lydv/vht83C++XfxL7jaf/yvdUov6bt7d01eZ/SE/7Md3CYEOqzw9Jc 3irqgdtWOQe5vw/vWbeop9TVONffr3oU5yzKpMR0iO+P2U3z95OjecKkdm57JMb36v1MR7GP PmlnS6YHaQQFMbeJ3ZHN1JU/vPZ+vl/L7UdMx3lm16kJ/z3Iv0W5qdLp6A2+vy7b8798Cdxj Y+fcWqzftUWJpTgj0VCLuag4EQBVSk2G8gIAAA== X-CMS-MailID: 20211220142228epcas5p2978d92d38f2015148d5f72913d6dbc3e X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20211220142228epcas5p2978d92d38f2015148d5f72913d6dbc3e References: <20211220141734.12206-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Completion of a uring_cmd ioctl may involve referencing certain ioctl-specific fields, requiring original submitter context. Export an API that driver can use for this purpose. The API facilitates reusing task-work infra of io_uring, while driver gets to implement cmd-specific handling in a callback. Signed-off-by: Kanchan Joshi Signed-off-by: Anuj Gupta --- fs/io_uring.c | 16 ++++++++++++++++ include/linux/io_uring.h | 8 ++++++++ 2 files changed, 24 insertions(+) diff --git a/fs/io_uring.c b/fs/io_uring.c index e96ed3d0385e..246f1085404d 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -2450,6 +2450,22 @@ static void io_req_task_submit(struct io_kiocb *req, bool *locked) io_req_complete_failed(req, -EFAULT); } +static void io_uring_cmd_work(struct io_kiocb *req, bool *locked) +{ + req->uring_cmd.driver_cb(&req->uring_cmd); +} + +void io_uring_cmd_complete_in_task(struct io_uring_cmd *ioucmd, + void (*driver_cb)(struct io_uring_cmd *)) +{ + struct io_kiocb *req = container_of(ioucmd, struct io_kiocb, uring_cmd); + + req->uring_cmd.driver_cb = driver_cb; + req->io_task_work.func = io_uring_cmd_work; + io_req_task_work_add(req, !!(req->ctx->flags & IORING_SETUP_SQPOLL)); +} +EXPORT_SYMBOL_GPL(io_uring_cmd_complete_in_task); + static void io_req_task_queue_fail(struct io_kiocb *req, int ret) { req->result = ret; diff --git a/include/linux/io_uring.h b/include/linux/io_uring.h index 64e788b39a86..f4b4990a3b62 100644 --- a/include/linux/io_uring.h +++ b/include/linux/io_uring.h @@ -14,11 +14,15 @@ struct io_uring_cmd { __u16 op; __u16 unused; __u32 len; + /* used if driver requires update in task context*/ + void (*driver_cb)(struct io_uring_cmd *cmd); __u64 pdu[5]; /* 40 bytes available inline for free use */ }; #if defined(CONFIG_IO_URING) void io_uring_cmd_done(struct io_uring_cmd *cmd, ssize_t ret); +void io_uring_cmd_complete_in_task(struct io_uring_cmd *ioucmd, + void (*driver_cb)(struct io_uring_cmd *)); struct sock *io_uring_get_socket(struct file *file); void __io_uring_cancel(bool cancel_all); void __io_uring_free(struct task_struct *tsk); @@ -42,6 +46,10 @@ static inline void io_uring_free(struct task_struct *tsk) static inline void io_uring_cmd_done(struct io_uring_cmd *cmd, ssize_t ret) { } +static inline void io_uring_cmd_complete_in_task(struct io_uring_cmd *ioucmd, + void (*driver_cb)(struct io_uring_cmd *)) +{ +} static inline struct sock *io_uring_get_socket(struct file *file) { return NULL; From patchwork Mon Dec 20 14:17:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12688943 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2293BC43217 for ; Tue, 21 Dec 2021 02:56:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234082AbhLUC40 (ORCPT ); Mon, 20 Dec 2021 21:56:26 -0500 Received: from mailout4.samsung.com ([203.254.224.34]:14376 "EHLO mailout4.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234050AbhLUC40 (ORCPT ); Mon, 20 Dec 2021 21:56:26 -0500 Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20211221025624epoutp04dd7316d18c6f4be7d92103e4123385be~Cpa7KafMe1360713607epoutp04e for ; Tue, 21 Dec 2021 02:56:24 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20211221025624epoutp04dd7316d18c6f4be7d92103e4123385be~Cpa7KafMe1360713607epoutp04e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1640055384; bh=ZUldHq0a4MM0PQTzu09Hb3trWCm0rWfi963tcz/Zgn8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i9l9Kx9ryR/2nYTBgBZNlRWvxxLq+DOTNLyaNij/qipS/h0GsgqSJYyqMgRTR/KUC z0IPa/UuHQ6pNxWYgpdv/7KyhSdsx3nUaf64nCYgmAa2jlzSyUG/cN1SIhG6jyLtgP iJU+qILNuyHkAf2rCLHiAyXS92CeGv0FFr17Fi3I= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20211221025624epcas5p3ebff1d3eb39e17d688bc71255fef8d7c~Cpa6vWKzA2522725227epcas5p3N; Tue, 21 Dec 2021 02:56:24 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.175]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4JJ1LS6d6Gz4x9Pv; Tue, 21 Dec 2021 02:56:16 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 3E.59.05590.E4241C16; Tue, 21 Dec 2021 11:56:14 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20211220142231epcas5p1482c78f91feabdbc3e62341790ab22e1~CfIsP4ciX2429924299epcas5p1v; Mon, 20 Dec 2021 14:22:31 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20211220142231epsmtrp112a72f9309229ce911c426e5fd46f0f6~CfIsPEWMX2445924459epsmtrp1V; Mon, 20 Dec 2021 14:22:31 +0000 (GMT) X-AuditID: b6c32a4b-723ff700000015d6-b3-61c1424ef8bc Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 83.85.29871.6A190C16; Mon, 20 Dec 2021 23:22:30 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20211220142228epsmtip18a65d672b0ca5d59d7c46922940d2c31~CfIqVb2s_0040400404epsmtip1g; Mon, 20 Dec 2021 14:22:28 +0000 (GMT) From: Kanchan Joshi To: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, javier@javigon.com, anuj20.g@samsung.com, joshiiitr@gmail.com, pankydev8@gmail.com Subject: [RFC 02/13] nvme: wire-up support for async-passthru on char-device. Date: Mon, 20 Dec 2021 19:47:23 +0530 Message-Id: <20211220141734.12206-3-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211220141734.12206-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrGJsWRmVeSWpSXmKPExsWy7bCmuq6f08FEg10dohZNE/4yW6y+289m sXL1USaLd63nWCw6T19gsjj/9jCTxaRD1xgt9t7Stpi/7Cm7xZqbT1kcuDx2zrrL7tG84A6L x+WzpR6bVnWyeWxeUu+x+2YDm0ffllWMHp83yQVwRGXbZKQmpqQWKaTmJeenZOal2yp5B8c7 x5uaGRjqGlpamCsp5CXmptoqufgE6Lpl5gBdqKRQlphTChQKSCwuVtK3synKLy1JVcjILy6x VUotSMkpMCnQK07MLS7NS9fLSy2xMjQwMDIFKkzIztj1YApjwXz/ipbnC5gaGC85djFycEgI mEjM2xraxcjFISSwm1Gi+/pVJgjnE6PEhkMn2SCcb4wSt88/BspwgnXM33qdDcQWEtjLKLFk hiiE/ZlRYu7jWpCpbAKaEhcml4KERQSiJS48vwZWzizQwSixs9sWxBYW8Jf4fewEI0g5i4Cq xOlebpAwr4CFxOInH5khNslLzLz0nR3E5hSwlDg8exkbRI2gxMmZT1ggRspLNG+dDVU/kUOi 6yIrxF8uErfX8UKEhSVeHd/CDmFLSbzsb4OyiyV+3TnKDPKhBMhl1xtmskAk7CUu7vnLBDKH GeiT9bv0IcKyElNPrWOCWMsn0fv7CTRAeCV2zIOxFSXuTXrKCmGLSzycsQTK9pBY23MWGpo9 jBJzHu5gnsCoMAvJO7OQvDMLYfUCRuZVjJKpBcW56anFpgXGeanl8AhOzs/dxAhOsFreOxgf Pfigd4iRiYPxEKMEB7OSCO+W2fsThXhTEiurUovy44tKc1KLDzGaAoN7IrOUaHI+MMXnlcQb mlgamJiZmZlYGpsZKonznkrfkCgkkJ5YkpqdmlqQWgTTx8TBKdXAZGP9MCi26O8F+T0u7/XF l7OKnfiSGHii52dwypSle9oqqou37Hqv58MymWVjTW91B1eppFXxtxdFqw8pbd0hGcAfu070 vL5T15P7k1bevnFmt93LuSEHXbs++OwLsP10bsOmx4ZBrgemup05fTA6csGk/FmaNlUp01ar /DqSuH3mY1+vp5eFJv5/mX7Aodsrucl4CeO2fbGr5DTi9z78/OL0xRWsr41cvqwrlZNY01fM svb393XTnt1OPbZw7qwlEhrOm868m3NXTKbRaROrqFndBXnrV/dWq7vOiI35GpgZacFiEdv8 oXPSG9c+PVNNOf8lD3ICxYwzb669+bt58m/XCbnxly+8fJa3w3Aa8xQlluKMREMt5qLiRACl y/R+OQQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrALMWRmVeSWpSXmKPExsWy7bCSnO6yiQcSDfbdMrFomvCX2WL13X42 i5WrjzJZvGs9x2LRefoCk8X5t4eZLCYdusZosfeWtsX8ZU/ZLdbcfMriwOWxc9Zddo/mBXdY PC6fLfXYtKqTzWPzknqP3Tcb2Dz6tqxi9Pi8SS6AI4rLJiU1J7MstUjfLoErY9eDKYwF8/0r Wp4vYGpgvOTYxcjJISFgIjF/63U2EFtIYDejxPSdsRBxcYnmaz/YIWxhiZX/ngPZXEA1Hxkl NixZwNTFyMHBJqApcWFyKUiNiECsxIdfx5hAapgFJgHV9D8AaxYW8JXYPKWDHaSeRUBV4nQv N0iYV8BCYvGTj8wQ8+UlZl76DlbOKWApcXj2Mqh7LCROfPjCAlEvKHFy5hMwmxmovnnrbOYJ jAKzkKRmIUktYGRaxSiZWlCcm55bbFhgmJdarlecmFtcmpeul5yfu4kRHAlamjsYt6/6oHeI kYmD8RCjBAezkgjvltn7E4V4UxIrq1KL8uOLSnNSiw8xSnOwKInzXug6GS8kkJ5YkpqdmlqQ WgSTZeLglGpgarly9MDi/Xu2FwY/U2EXPaC9qD3hA4O+29NPsaELrZ9ffn7x1f1ZV1dEB336 6ik7Z2pl5Cz/97eWGy6MlCrd8DGhf+HprSbmKVO8H8RsNtTdE9UgcOi9QkfQp+bfCZKyHG9k F+s8+y//rmXtc96LK2r/v+lJPaYYV2Z13uGqyduSy5Fncr++q3d/qX4i9I2qJuOa5aWMc7a2 Zvz9ytcv4H/lw29d2VoTYXMZNvvun1e7zYwuz75YPPPMV9WuwMVObSu8BK94MpR8c3YUKJ9x +MUpd7MPh+b+2X+Gjc3r6OW71788X3JocXyyXObXbVzn6roKforv+Otd/liXKdfX4pFNy5/j jrtEBe2PRz19mKXEUpyRaKjFXFScCABMYHy38wIAAA== X-CMS-MailID: 20211220142231epcas5p1482c78f91feabdbc3e62341790ab22e1 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20211220142231epcas5p1482c78f91feabdbc3e62341790ab22e1 References: <20211220141734.12206-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Introduce handlers for fops->async_cmd(), implementing async passthru on char device (including the multipath one). The handlers supports NVME_IOCTL_IO64_CMD. Signed-off-by: Kanchan Joshi Signed-off-by: Anuj Gupta --- drivers/nvme/host/core.c | 1 + drivers/nvme/host/ioctl.c | 147 +++++++++++++++++++++++++++++++--- drivers/nvme/host/multipath.c | 1 + drivers/nvme/host/nvme.h | 5 ++ 4 files changed, 145 insertions(+), 9 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 290f26ed74c2..bce2e93d14a3 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -3670,6 +3670,7 @@ static const struct file_operations nvme_ns_chr_fops = { .release = nvme_ns_chr_release, .unlocked_ioctl = nvme_ns_chr_ioctl, .compat_ioctl = compat_ptr_ioctl, + .async_cmd = nvme_ns_chr_async_cmd, }; static int nvme_add_ns_cdev(struct nvme_ns *ns) diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index 22314962842d..7d9c51d9c0a8 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -18,6 +18,84 @@ static void __user *nvme_to_user_ptr(uintptr_t ptrval) ptrval = (compat_uptr_t)ptrval; return (void __user *)ptrval; } +/* This overlays struct io_uring_cmd pdu (40 bytes) */ +struct nvme_uring_cmd { + u32 ioctl_cmd; + u32 meta_len; + void __user *argp; + union { + struct bio *bio; + struct request *req; + }; + void *meta; /* kernel-resident buffer */ + void __user *meta_buffer; +}; + +static struct nvme_uring_cmd *nvme_uring_cmd(struct io_uring_cmd *ioucmd) +{ + return (struct nvme_uring_cmd *)&ioucmd->pdu; +} + +static void nvme_pt_task_cb(struct io_uring_cmd *ioucmd) +{ + struct nvme_uring_cmd *cmd = nvme_uring_cmd(ioucmd); + struct nvme_passthru_cmd64 __user *ptcmd64 = cmd->argp; + struct request *req = cmd->req; + int status; + u64 result; + + if (nvme_req(req)->flags & NVME_REQ_CANCELLED) + status = -EINTR; + else + status = nvme_req(req)->status; + result = le64_to_cpu(nvme_req(req)->result.u64); + + /* we can free request */ + blk_mq_free_request(req); + + if (cmd->meta) { + if (status) + if (copy_to_user(cmd->meta_buffer, cmd->meta, cmd->meta_len)) + status = -EFAULT; + kfree(cmd->meta); + } + + if (put_user(result, &ptcmd64->result)) + status = -EFAULT; + io_uring_cmd_done(ioucmd, status); +} + +static void nvme_end_async_pt(struct request *req, blk_status_t err) +{ + struct io_uring_cmd *ioucmd = req->end_io_data; + struct nvme_uring_cmd *cmd = nvme_uring_cmd(ioucmd); + /* extract bio before reusing the same field for request */ + struct bio *bio = cmd->bio; + + cmd->req = req; + /* this takes care of setting up task-work */ + io_uring_cmd_complete_in_task(ioucmd, nvme_pt_task_cb); + blk_rq_unmap_user(bio); +} + +static void nvme_setup_uring_cmd_data(struct request *rq, + struct io_uring_cmd *ioucmd, void *meta, + void __user *meta_buffer, u32 meta_len, bool write) +{ + struct nvme_uring_cmd *cmd = nvme_uring_cmd(ioucmd); + + /* to free bio on completion, as req->bio will be null at that time */ + cmd->bio = rq->bio; + /* meta update is required only for read requests */ + if (meta && !write) { + cmd->meta = meta; + cmd->meta_buffer = meta_buffer; + cmd->meta_len = meta_len; + } else { + cmd->meta = NULL; + } + rq->end_io_data = ioucmd; +} static void *nvme_add_user_metadata(struct bio *bio, void __user *ubuf, unsigned len, u32 seed, bool write) @@ -56,7 +134,8 @@ static void *nvme_add_user_metadata(struct bio *bio, void __user *ubuf, static int nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd, void __user *ubuffer, unsigned bufflen, void __user *meta_buffer, unsigned meta_len, - u32 meta_seed, u64 *result, unsigned timeout) + u32 meta_seed, u64 *result, unsigned timeout, + struct io_uring_cmd *ioucmd) { bool write = nvme_is_write(cmd); struct nvme_ns *ns = q->queuedata; @@ -92,6 +171,12 @@ static int nvme_submit_user_cmd(struct request_queue *q, req->cmd_flags |= REQ_INTEGRITY; } } + if (ioucmd) { /* async dispatch */ + nvme_setup_uring_cmd_data(req, ioucmd, meta, meta_buffer, + meta_len, write); + blk_execute_rq_nowait(req, 0, nvme_end_async_pt); + return 0; + } ret = nvme_execute_passthru_rq(req); if (result) @@ -170,7 +255,8 @@ static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio) return nvme_submit_user_cmd(ns->queue, &c, nvme_to_user_ptr(io.addr), length, - metadata, meta_len, lower_32_bits(io.slba), NULL, 0); + metadata, meta_len, lower_32_bits(io.slba), NULL, 0, + NULL); } static bool nvme_validate_passthru_nsid(struct nvme_ctrl *ctrl, @@ -224,7 +310,7 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns, status = nvme_submit_user_cmd(ns ? ns->queue : ctrl->admin_q, &c, nvme_to_user_ptr(cmd.addr), cmd.data_len, nvme_to_user_ptr(cmd.metadata), cmd.metadata_len, - 0, &result, timeout); + 0, &result, timeout, NULL); if (status >= 0) { if (put_user(result, &ucmd->result)) @@ -235,7 +321,8 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns, } static int nvme_user_cmd64(struct nvme_ctrl *ctrl, struct nvme_ns *ns, - struct nvme_passthru_cmd64 __user *ucmd) + struct nvme_passthru_cmd64 __user *ucmd, + struct io_uring_cmd *ioucmd) { struct nvme_passthru_cmd64 cmd; struct nvme_command c; @@ -270,9 +357,9 @@ static int nvme_user_cmd64(struct nvme_ctrl *ctrl, struct nvme_ns *ns, status = nvme_submit_user_cmd(ns ? ns->queue : ctrl->admin_q, &c, nvme_to_user_ptr(cmd.addr), cmd.data_len, nvme_to_user_ptr(cmd.metadata), cmd.metadata_len, - 0, &cmd.result, timeout); + 0, &cmd.result, timeout, ioucmd); - if (status >= 0) { + if (!ioucmd && status >= 0) { if (put_user(cmd.result, &ucmd->result)) return -EFAULT; } @@ -296,7 +383,7 @@ static int nvme_ctrl_ioctl(struct nvme_ctrl *ctrl, unsigned int cmd, case NVME_IOCTL_ADMIN_CMD: return nvme_user_cmd(ctrl, NULL, argp); case NVME_IOCTL_ADMIN64_CMD: - return nvme_user_cmd64(ctrl, NULL, argp); + return nvme_user_cmd64(ctrl, NULL, argp, NULL); default: return sed_ioctl(ctrl->opal_dev, cmd, argp); } @@ -340,7 +427,7 @@ static int nvme_ns_ioctl(struct nvme_ns *ns, unsigned int cmd, case NVME_IOCTL_SUBMIT_IO: return nvme_submit_io(ns, argp); case NVME_IOCTL_IO64_CMD: - return nvme_user_cmd64(ns->ctrl, ns, argp); + return nvme_user_cmd64(ns->ctrl, ns, argp, NULL); default: return -ENOTTY; } @@ -369,6 +456,33 @@ long nvme_ns_chr_ioctl(struct file *file, unsigned int cmd, unsigned long arg) return __nvme_ioctl(ns, cmd, (void __user *)arg); } +static int nvme_ns_async_ioctl(struct nvme_ns *ns, struct io_uring_cmd *ioucmd) +{ + struct nvme_uring_cmd *cmd = nvme_uring_cmd(ioucmd); + int ret; + + switch (cmd->ioctl_cmd) { + case NVME_IOCTL_IO64_CMD: + ret = nvme_user_cmd64(ns->ctrl, ns, cmd->argp, ioucmd); + break; + default: + ret = -ENOTTY; + } + + if (ret >= 0) + ret = -EIOCBQUEUED; + return ret; +} + +int nvme_ns_chr_async_cmd(struct io_uring_cmd *ioucmd, + enum io_uring_cmd_flags flags) +{ + struct nvme_ns *ns = container_of(file_inode(ioucmd->file)->i_cdev, + struct nvme_ns, cdev); + + return nvme_ns_async_ioctl(ns, ioucmd); +} + #ifdef CONFIG_NVME_MULTIPATH static int nvme_ns_head_ctrl_ioctl(struct nvme_ns *ns, unsigned int cmd, void __user *argp, struct nvme_ns_head *head, int srcu_idx) @@ -412,6 +526,21 @@ int nvme_ns_head_ioctl(struct block_device *bdev, fmode_t mode, return ret; } +int nvme_ns_head_chr_async_cmd(struct io_uring_cmd *ioucmd, + enum io_uring_cmd_flags flags) +{ + struct cdev *cdev = file_inode(ioucmd->file)->i_cdev; + struct nvme_ns_head *head = container_of(cdev, struct nvme_ns_head, cdev); + int srcu_idx = srcu_read_lock(&head->srcu); + struct nvme_ns *ns = nvme_find_path(head); + int ret = -EWOULDBLOCK; + + if (ns) + ret = nvme_ns_async_ioctl(ns, ioucmd); + srcu_read_unlock(&head->srcu, srcu_idx); + return ret; +} + long nvme_ns_head_chr_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -480,7 +609,7 @@ long nvme_dev_ioctl(struct file *file, unsigned int cmd, case NVME_IOCTL_ADMIN_CMD: return nvme_user_cmd(ctrl, NULL, argp); case NVME_IOCTL_ADMIN64_CMD: - return nvme_user_cmd64(ctrl, NULL, argp); + return nvme_user_cmd64(ctrl, NULL, argp, NULL); case NVME_IOCTL_IO_CMD: return nvme_dev_user_cmd(ctrl, argp); case NVME_IOCTL_RESET: diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index 13e5d503ed07..1e59c8e06622 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -423,6 +423,7 @@ static const struct file_operations nvme_ns_head_chr_fops = { .release = nvme_ns_head_chr_release, .unlocked_ioctl = nvme_ns_head_chr_ioctl, .compat_ioctl = compat_ptr_ioctl, + .async_cmd = nvme_ns_head_chr_async_cmd, }; static int nvme_add_ns_head_cdev(struct nvme_ns_head *head) diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 9b095ee01364..9a901b954a87 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -16,6 +16,7 @@ #include #include #include +#include #include @@ -747,6 +748,10 @@ long nvme_ns_head_chr_ioctl(struct file *file, unsigned int cmd, unsigned long arg); long nvme_dev_ioctl(struct file *file, unsigned int cmd, unsigned long arg); +int nvme_ns_chr_async_cmd(struct io_uring_cmd *ucmd, + enum io_uring_cmd_flags flags); +int nvme_ns_head_chr_async_cmd(struct io_uring_cmd *ucmd, + enum io_uring_cmd_flags flags); int nvme_getgeo(struct block_device *bdev, struct hd_geometry *geo); extern const struct attribute_group *nvme_ns_id_attr_groups[]; From patchwork Mon Dec 20 14:17:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12688941 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD4A8C433F5 for ; Tue, 21 Dec 2021 02:56:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234062AbhLUC4Z (ORCPT ); Mon, 20 Dec 2021 21:56:25 -0500 Received: from mailout1.samsung.com ([203.254.224.24]:12712 "EHLO mailout1.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234050AbhLUC4Y (ORCPT ); Mon, 20 Dec 2021 21:56:24 -0500 Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20211221025622epoutp01b119edfcca58bf85d26c9cb859398094~Cpa5n890t1830618306epoutp01a for ; Tue, 21 Dec 2021 02:56:22 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20211221025622epoutp01b119edfcca58bf85d26c9cb859398094~Cpa5n890t1830618306epoutp01a DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1640055382; bh=qQzwQeVQ/rdLDuaJEMVGex5uan9oUlYhMcsCMT4pHTs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iICAsI8DuKsC4HnxcA9MaNABeW26BnhwRk5VMoa1I/xSM543bwArrco7Iq3mJHRnK nkggI/9eknIZ3v4ncEHIdrTvMuA3Cn4OPyJpi3Nebt9Zhc6KXICqRCVSHdqkQjr8Oa xzSYoAL3oYgYN5hXnXMXRRLou8fK/5aYa+39yoiI= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20211221025622epcas5p26a3cf5e78abbd536c0cd175200d05009~Cpa5Mihas2587925879epcas5p2W; Tue, 21 Dec 2021 02:56:22 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.180]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4JJ1LV4CKSz4x9Q5; Tue, 21 Dec 2021 02:56:18 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 57.26.06423.25241C16; Tue, 21 Dec 2021 11:56:18 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20211220142233epcas5p3b54aa591fb7b81bfb58bc33b5f92a2d3~CfIuf3uti0906209062epcas5p31; Mon, 20 Dec 2021 14:22:33 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20211220142233epsmtrp17a48a03437f0a58e0662ed249523ee12~CfIueRL9p2445924459epsmtrp1Y; Mon, 20 Dec 2021 14:22:33 +0000 (GMT) X-AuditID: b6c32a49-b13ff70000001917-0c-61c142525b00 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 64.85.29871.9A190C16; Mon, 20 Dec 2021 23:22:33 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20211220142231epsmtip1c9366a0b7d0e3606cbcec7cdeb8c370e~CfIsWcVoS0637406374epsmtip1r; Mon, 20 Dec 2021 14:22:31 +0000 (GMT) From: Kanchan Joshi To: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, javier@javigon.com, anuj20.g@samsung.com, joshiiitr@gmail.com, pankydev8@gmail.com Subject: [RFC 03/13] io_uring: mark iopoll not supported for uring-cmd Date: Mon, 20 Dec 2021 19:47:24 +0530 Message-Id: <20211220141734.12206-4-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211220141734.12206-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrNJsWRmVeSWpSXmKPExsWy7bCmpm6Q08FEg66pChZNE/4yW6y+289m sXL1USaLd63nWCw6T19gsjj/9jCTxaRD1xgt9t7Stpi/7Cm7xZqbT1kcuDx2zrrL7tG84A6L x+WzpR6bVnWyeWxeUu+x+2YDm0ffllWMHp83yQVwRGXbZKQmpqQWKaTmJeenZOal2yp5B8c7 x5uaGRjqGlpamCsp5CXmptoqufgE6Lpl5gBdqKRQlphTChQKSCwuVtK3synKLy1JVcjILy6x VUotSMkpMCnQK07MLS7NS9fLSy2xMjQwMDIFKkzIzpjcdImx4CVrxdJN21kbGJ+zdDFyckgI mEhs/vmdqYuRi0NIYDejxK8LKxkhnE+MEntXn2CGcL4xSuzrfcsG07Km4ykrRGIvo8Ssy5NZ IJzPjBIdB3cCtXBwsAloSlyYXArSICIQLXHh+TWwZmaBDkaJnd22ILawgJvE4+fvWUDKWQRU JaYszwIJ8wpYSByZ0cgMsUteYual7+wgNqeApcTh2cvYIGoEJU7OfMICMVJeonnrbLBDJQQm ckhMev4A6lAXiba/P9khbGGJV8e3QNlSEi/726DsYolfd45CNQPddr1hJjRg7CUu7vnLBHIc M9Av63fpQ4RlJaaeWscEsZhPovf3EyaIOK/EjnkwtqLEvUmgAAKxxSUezlgCZXtIfHt5BBpW PYwSvz9NYJ3AqDALyUOzkDw0C2H1AkbmVYySqQXFuempxaYFhnmp5fBYTs7P3cQITrVanjsY 7z74oHeIkYmD8RCjBAezkgjvltn7E4V4UxIrq1KL8uOLSnNSiw8xmgIDfCKzlGhyPjDZ55XE G5pYGpiYmZmZWBqbGSqJ855O35AoJJCeWJKanZpakFoE08fEwSnVwKQ5/TTzmvzjf107Phbm Pk5RO5IldeduzI/QW9M6lqjMkSwNTOZXk1jAWdVf5fKu5pflbiWzgP1XD3wNna+3eetrLt/c zvTChzMKdsl273ZM+H7jRXzJyp2rtwfKC/kley3XfX9YPzfgX+aL535XVnsFbrm4/jdf4/Wj fhsn3ZjNbr3iWFTsM7fOvq6qU/0T5ywOzXtg8Si84H+T/uVZx97+0Z6m759y2XmX7MLJqcJ9 teKHL+hlHym913u16cmS+anrJ87n+88n8DVjVui9fPtVG+3eJjNV9Mb9vO54/ZxwfIrKKYm1 W9vPJxe6KJ7a8N30XJWo6ItplU/d9xol7/g2M9Voa5Lcnsrg5lW1+w4rsRRnJBpqMRcVJwIA SNOzMz4EAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrILMWRmVeSWpSXmKPExsWy7bCSnO7KiQcSDdbu4bVomvCX2WL13X42 i5WrjzJZvGs9x2LRefoCk8X5t4eZLCYdusZosfeWtsX8ZU/ZLdbcfMriwOWxc9Zddo/mBXdY PC6fLfXYtKqTzWPzknqP3Tcb2Dz6tqxi9Pi8SS6AI4rLJiU1J7MstUjfLoErY3LTJcaCl6wV SzdtZ21gfM7SxcjJISFgIrGm4ylrFyMXh5DAbkaJL9ua2SAS4hLN136wQ9jCEiv/PWeHKPrI KLHl0RugDg4ONgFNiQuTS0FqRARiJT78OsYEUsMsMIlRYkP/A7BmYQE3icfP37OA1LMIqEpM WZ4FEuYVsJA4MqORGWK+vMTMS9/ByjkFLCUOz14GdoMQUM2JD19YIOoFJU7OfAJmMwPVN2+d zTyBUWAWktQsJKkFjEyrGCVTC4pz03OLDQsM81LL9YoTc4tL89L1kvNzNzGCY0FLcwfj9lUf 9A4xMnEwHmKU4GBWEuHdMnt/ohBvSmJlVWpRfnxRaU5q8SFGaQ4WJXHeC10n44UE0hNLUrNT UwtSi2CyTBycUg1MjqLWNZtYlVV9kryiz/v+ymMJCpBkPlkacMGW5ejhPKXbYQuX+rEzP2Y9 ut8zs+fEFV1PmfsZMsc+ZfxJL5vlVhL5o92Nd1vn5/P5W3JrOFLfBqw68Pzj7btnWySZClRq PrIZvjN987tV/i136m+XoC0vGExX11+t3ZK4a/P8qZrcXe1m7fGG/CXx5w8ttsnrv1//cpe9 zcG1BovEXA8fOXdz4gw1nr8zolNsHB7L7WEI5H6q3HvlxhZr/mf7u7Of9S1tWfU3WKLKjuF3 9ZcPZ10ZftexG8tPE5A3jfxax9unx72wa17uAfYLW5esdyxqazmXr8U9id0smmtVy3fzd3Vv P+isWOho1VjU/FyJpTgj0VCLuag4EQBUGjSQ9AIAAA== X-CMS-MailID: 20211220142233epcas5p3b54aa591fb7b81bfb58bc33b5f92a2d3 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20211220142233epcas5p3b54aa591fb7b81bfb58bc33b5f92a2d3 References: <20211220141734.12206-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Anuj Gupta Currently uring-passthrough doesn't support iopoll. Bail out to avoid the panic. Signed-off-by: Anuj Gupta --- fs/io_uring.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/fs/io_uring.c b/fs/io_uring.c index 246f1085404d..1061b4cde4be 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -4131,6 +4131,11 @@ static int io_uring_cmd_prep(struct io_kiocb *req, if (!req->file->f_op->async_cmd) return -EOPNOTSUPP; + if (req->ctx->flags & IORING_SETUP_IOPOLL) { + printk_once(KERN_WARNING "io_uring: iopoll not supported!\n"); + return -EOPNOTSUPP; + } + cmd->op = READ_ONCE(csqe->op); cmd->len = READ_ONCE(csqe->len); From patchwork Mon Dec 20 14:17:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12688945 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2EC7C433FE for ; Tue, 21 Dec 2021 02:56:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234095AbhLUC4a (ORCPT ); Mon, 20 Dec 2021 21:56:30 -0500 Received: from mailout2.samsung.com ([203.254.224.25]:42761 "EHLO mailout2.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234076AbhLUC43 (ORCPT ); Mon, 20 Dec 2021 21:56:29 -0500 Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20211221025628epoutp02398988358506364cd499c11fa5a2d033~Cpa_pt5y42870328703epoutp02B for ; Tue, 21 Dec 2021 02:56:28 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20211221025628epoutp02398988358506364cd499c11fa5a2d033~Cpa_pt5y42870328703epoutp02B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1640055388; bh=/KXeRqRmtKhe4hlLhbh4cf3uc4RF2AMH9nkqKc284KQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UbQ/1KhZw9DjEtCyVV8mhwHKGEwRzSLOP0tJYlN7XIXDf6uC8r1mv6mmHXzZ0iJDR 4EHfxDItAKkm8pd3Lq4vVuCvh0hyhx2gKf6lJXiCpj7SjQJOM/XAl5Dq1u84DRagI1 zfDjs5LB3v8uNWGaDxNBJFM7+aYiMeoUkj9taQVE= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20211221025627epcas5p2e477d0414a432cbbd50852455c1f21cc~Cpa_A6Vkw2587925879epcas5p2o; Tue, 21 Dec 2021 02:56:27 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.183]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4JJ1Lb3VVnz4x9Q7; Tue, 21 Dec 2021 02:56:23 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 5D.E1.46822.75241C16; Tue, 21 Dec 2021 11:56:23 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20211220142235epcas5p3b8d56cd39d9710278ec3360be47f2cca~CfIwWbsyc2729127291epcas5p3K; Mon, 20 Dec 2021 14:22:35 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20211220142235epsmtrp1b36578b55c298b1b473d0ff329d0c514~CfIwVp8QL2445924459epsmtrp1b; Mon, 20 Dec 2021 14:22:35 +0000 (GMT) X-AuditID: b6c32a4a-de5ff7000000b6e6-91-61c142575288 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 35.85.29871.BA190C16; Mon, 20 Dec 2021 23:22:35 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20211220142233epsmtip18e0a7b5c3cb5a6b825868840f277c672~CfIuivTvM0637906379epsmtip1p; Mon, 20 Dec 2021 14:22:33 +0000 (GMT) From: Kanchan Joshi To: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, javier@javigon.com, anuj20.g@samsung.com, joshiiitr@gmail.com, pankydev8@gmail.com Subject: [RFC 04/13] io_uring: modify unused field in io_uring_cmd to store flags Date: Mon, 20 Dec 2021 19:47:25 +0530 Message-Id: <20211220141734.12206-5-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211220141734.12206-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrBJsWRmVeSWpSXmKPExsWy7bCmhm6408FEg7eXpCyaJvxltlh9t5/N YuXqo0wW71rPsVh0nr7AZHH+7WEmi0mHrjFa7L2lbTF/2VN2izU3n7I4cHnsnHWX3aN5wR0W j8tnSz02repk89i8pN5j980GNo++LasYPT5vkgvgiMq2yUhNTEktUkjNS85PycxLt1XyDo53 jjc1MzDUNbS0MFdSyEvMTbVVcvEJ0HXLzAG6UEmhLDGnFCgUkFhcrKRvZ1OUX1qSqpCRX1xi q5RakJJTYFKgV5yYW1yal66Xl1piZWhgYGQKVJiQnXHl/l2Wgr+sFevvHmRrYPzI0sXIySEh YCIx/fExRhBbSGA3o8T5tcpdjFxA9idGiXN75zNDJL4xSkxtEIBpeLbtFTtE0V5GiWe9n9gh ij4zSqy6wNPFyMHBJqApcWFyKUhYRCBa4sLza2wgNrNAB6PEzm5bEFtYIFji7MmfYK0sAqoS Jz/sBjuCV8BCYuGKGYwQu+QlZl76DlbDKWApcXj2MjaIGkGJkzOfsEDMlJdo3jqbGeQeCYGp HBJXVh5lhmh2kTh67gQbhC0s8er4FnYIW0riZX8blF0s8evOUahmoOOuN8yEBou9xMU9f5lA nmEGemb9Ln2IsKzE1FPrmCAW80n0/n7CBBHnldgxD8ZWlLg36SkrhC0u8XDGElaQMRICHhI/ V9RBwq2HUWLyiUfMExgVZiH5ZxaSf2YhbF7AyLyKUTK1oDg3PbXYtMAoL7UcHsXJ+bmbGMFJ VstrB+PDBx/0DjEycTAeYpTgYFYS4d0ye3+iEG9KYmVValF+fFFpTmrxIUZTYIBPZJYSTc4H pvm8knhDE0sDEzMzMxNLYzNDJXHe0+kbEoUE0hNLUrNTUwtSi2D6mDg4pRqYFI9sFjrS9jpX 73/s8cYbUvMOPLzvceHI0w3OjaucdsWWFWYkzkq3u5PwKnx776Mla/YYl8l9yLzRwHyrZJe8 5NHcKfUP3+Uve5Rru0j0eeia+FPpT6/P9v347skOj/VG2j/ZjLvWJJ7KWJEpz5Idv8rZ1uyR wt23i3WS13y/fXpx7Zm1nWbPtieIKrEbm5+4I+Z6/tkijTup92Z7XltT9cvgwRTr84+yFTy2 uz69U50xZd+09BUFe3/mat+M4V3pp2NhPc/vomZj8OPT+9fnemeHdC8oXhNiUzJPtN1hflNH 7eI3Sd7cETePsiguqFy66Oq+BztM9Ngcg6VeqdmeSi0SdFH94eu2VYiFh4tLTomlOCPRUIu5 qDgRAMGwUes7BAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrELMWRmVeSWpSXmKPExsWy7bCSnO7qiQcSDU6vEbRomvCX2WL13X42 i5WrjzJZvGs9x2LRefoCk8X5t4eZLCYdusZosfeWtsX8ZU/ZLdbcfMriwOWxc9Zddo/mBXdY PC6fLfXYtKqTzWPzknqP3Tcb2Dz6tqxi9Pi8SS6AI4rLJiU1J7MstUjfLoEr48r9uywFf1kr 1t89yNbA+JGli5GTQ0LAROLZtlfsXYxcHEICuxklfm18wAaREJdovvaDHcIWllj57zlU0UdG iY8rnjF3MXJwsAloSlyYXApSIyIQK/Hh1zEmkBpmgUmMEhv6H4A1CwsESmz+uYEJxGYRUJU4 +WE3I4jNK2AhsXDFDEaIBfISMy99B6vnFLCUODx7GdgRQkA1Jz58YYGoF5Q4OfMJmM0MVN+8 dTbzBEaBWUhSs5CkFjAyrWKUTC0ozk3PLTYsMMxLLdcrTswtLs1L10vOz93ECI4GLc0djNtX fdA7xMjEwXiIUYKDWUmEd8vs/YlCvCmJlVWpRfnxRaU5qcWHGKU5WJTEeS90nYwXEkhPLEnN Tk0tSC2CyTJxcEo1MImw9/x8d/hgZLrjZW9503+11v/371w036de3tMns+nUURlG4Z/S01lT 3kYFv+VfNKd4UXKK8dXfG1OuHr7eMN+1oGhxqVYfz/cV7tGfjPI0PD6eqzLyr/USYN36w/mP Y5z40yc3Jlsc3bV+0RSJurM3jC99n8CabMY4Y2O+as6L8NXb8h/3chmc7/J9Vr9nfWfp3mk/ 5ERrz9wujJfrcS7WnrbsYVPG1CtTT2nN6fO/rPnW9Gnj820r13q0/z8+4fIeZifuZ+YOyacn Xv6feGbmy08Vr2ad/OS5Kn3Ji6oPodvWSvWWx93dJ50S/nmupN2pO4c2CzW+v3Dm87I+yZX7 OT4/uizAt6T10vPQqawSSizFGYmGWsxFxYkAT4aFBfUCAAA= X-CMS-MailID: 20211220142235epcas5p3b8d56cd39d9710278ec3360be47f2cca X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20211220142235epcas5p3b8d56cd39d9710278ec3360be47f2cca References: <20211220141734.12206-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Anuj Gupta This patch modifies the unused field in io_uring_cmd structure to store the flags. This is a prep patch for providing fixedbufs support via passthrough ioctls. Signed-off-by: Anuj Gupta --- include/linux/io_uring.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/io_uring.h b/include/linux/io_uring.h index f4b4990a3b62..5ab824ced147 100644 --- a/include/linux/io_uring.h +++ b/include/linux/io_uring.h @@ -12,7 +12,7 @@ struct io_uring_cmd { struct file *file; __u16 op; - __u16 unused; + __u16 flags; __u32 len; /* used if driver requires update in task context*/ void (*driver_cb)(struct io_uring_cmd *cmd); From patchwork Mon Dec 20 14:17:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12688947 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A979C433F5 for ; Tue, 21 Dec 2021 02:56:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234076AbhLUC4i (ORCPT ); Mon, 20 Dec 2021 21:56:38 -0500 Received: from mailout3.samsung.com ([203.254.224.33]:46177 "EHLO mailout3.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234103AbhLUC4i (ORCPT ); Mon, 20 Dec 2021 21:56:38 -0500 Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20211221025633epoutp035358cb04a686b00eae0bf6df2547a164~CpbDz2Jsf3025530255epoutp03c for ; Tue, 21 Dec 2021 02:56:33 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20211221025633epoutp035358cb04a686b00eae0bf6df2547a164~CpbDz2Jsf3025530255epoutp03c DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1640055393; bh=lAWdIOcCOWl+B/sZ0GjEPv75N75abkg3WHQImh94oDg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YcmMdendvk+OPWcxJU4TY/Z8Up02JqTrN+ZVq3ZaJ+7LycgtvAgK3nHPZwafgcsFc vuVnYA2YPiB0f4mjbMRKCf+bBpnuU2ncvdcw7oOUoHm9MvhcyQQVHhAAFwT9c9Jcd+ N8WsQLMZm33GbNdAtZHRbR1QzxuYxg0WcFnSXj7Y= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20211221025632epcas5p3be3ee7d1f6023455a2bb2327e605d623~CpbC4Mj4x2522725227epcas5p3c; Tue, 21 Dec 2021 02:56:32 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.177]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4JJ1Lh2n7Rz4x9QJ; Tue, 21 Dec 2021 02:56:28 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 1B.36.06423.C5241C16; Tue, 21 Dec 2021 11:56:28 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20211220142237epcas5p48729a52293e4f7627e6ec53ca67b9c58~CfIylo9yo0647006470epcas5p4y; Mon, 20 Dec 2021 14:22:37 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20211220142237epsmtrp11641c51565e8b602ca6d4d5425fdeb0b~CfIyk2bis2445924459epsmtrp1d; Mon, 20 Dec 2021 14:22:37 +0000 (GMT) X-AuditID: b6c32a49-b13ff70000001917-49-61c1425cfa8e Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 06.85.29871.DA190C16; Mon, 20 Dec 2021 23:22:37 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20211220142235epsmtip101615df6543ec254f06c2f8a82760ba6~CfIwdsWZV0040100401epsmtip1h; Mon, 20 Dec 2021 14:22:35 +0000 (GMT) From: Kanchan Joshi To: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, javier@javigon.com, anuj20.g@samsung.com, joshiiitr@gmail.com, pankydev8@gmail.com Subject: [RFC 05/13] io_uring: add flag and helper for fixed-buffer uring-cmd Date: Mon, 20 Dec 2021 19:47:26 +0530 Message-Id: <20211220141734.12206-6-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211220141734.12206-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrFJsWRmVeSWpSXmKPExsWy7bCmhm6M08FEg7n7RCyaJvxltlh9t5/N YuXqo0wW71rPsVh0nr7AZHH+7WEmi0mHrjFa7L2lbTF/2VN2izU3n7I4cHnsnHWX3aN5wR0W j8tnSz02repk89i8pN5j980GNo++LasYPT5vkgvgiMq2yUhNTEktUkjNS85PycxLt1XyDo53 jjc1MzDUNbS0MFdSyEvMTbVVcvEJ0HXLzAG6UEmhLDGnFCgUkFhcrKRvZ1OUX1qSqpCRX1xi q5RakJJTYFKgV5yYW1yal66Xl1piZWhgYGQKVJiQnbGiaw9zwXbxiv23zjE3MK4V7mLk5JAQ MJG4v/o3axcjF4eQwG5GifudT9ggnE+MEs1vDkBlvjFKPHncB+RwgLXc+u0CEd/LKNH76As7 hPOZUeL6/y1MIEVsApoSFyaXgqwQEYiWuPD8GhuIzSzQwSixs9sWxBYW8JfourKUEcRmEVCV eHT0KyNIK6+AhUT/vHKI6+QlZl76zg5icwpYShyevQxsDK+AoMTJmU9YIEbKSzRvnc0McoKE wEQOiWnP1jBDNLtI9H0/zwZhC0u8Or6FHcKWknjZ3wZlF0v8unMUqhnotusNM1kgEvYSF/f8 BfuFGeiX9bv0IcKyElNPrWOCWMwn0fv7CRNEnFdixzwYW1Hi3qSnrBC2uMTDGUugbA+JBQ9+ M0PCqodRYsuON8wTGBVmIXloFpKHZiGsXsDIvIpRMrWgODc9tdi0wDAvtRweycn5uZsYwYlW y3MH490HH/QOMTJxMB5ilOBgVhLh3TJ7f6IQb0piZVVqUX58UWlOavEhRlNggE9klhJNzgem +rySeEMTSwMTMzMzE0tjM0Mlcd7T6RsShQTSE0tSs1NTC1KLYPqYODilGpgcm45fZWf13s/0 z7xgT8NB9lt8jlOiKnelnnvBW7J93ic1/eJpPwVfzbe0+5NoVHbDorbolTlvzI3Dy/vE/h/d 2i9m8ejtWY1Aj6Ai1qVbFm1YW/tQmqngurDdyRd5y7+/9j3X+DD043qr6liFi9JZknVbfvmZ T9ltIil2UWzZ1F1RBcK5V74J7eCx+mm4q4j7xvYZQd9eKQVMZWjnrPG9tT7swKNta7nD36Qa lh6VUXv2daMUp47wmQWnP3NGSTfcXv+jM/XzYxWOOwIP7YWMnXyWR6YZPPn0p6JIbX/OIaXO PS4Bh1IaGPwfCjf4rV8aeS+k0a8pxjPl/JelrRMtk25PLv6rvsfsZHOaoo4SS3FGoqEWc1Fx IgAiKoNJPQQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrPLMWRmVeSWpSXmKPExsWy7bCSnO7aiQcSDa4sU7domvCX2WL13X42 i5WrjzJZvGs9x2LRefoCk8X5t4eZLCYdusZosfeWtsX8ZU/ZLdbcfMriwOWxc9Zddo/mBXdY PC6fLfXYtKqTzWPzknqP3Tcb2Dz6tqxi9Pi8SS6AI4rLJiU1J7MstUjfLoErY0XXHuaC7eIV +2+dY25gXCvcxcjBISFgInHrt0sXIyeHkMBuRol7+7JAbAkBcYnmaz/YIWxhiZX/ngPZXEA1 Hxklrmy4wgTSyyagKXFhcilIjYhArMSHX8eYQGqYBSYxSmzofwDWLCzgK3Fm3kowm0VAVeLR 0a+MIL28AhYS/fPKIebLS8y89B2shFPAUuLw7GVsEPdYSJz48IUFxOYVEJQ4OfMJmM0MVN+8 dTbzBEaBWUhSs5CkFjAyrWKUTC0ozk3PLTYsMMxLLdcrTswtLs1L10vOz93ECI4DLc0djNtX fdA7xMjEwXiIUYKDWUmEd8vs/YlCvCmJlVWpRfnxRaU5qcWHGKU5WJTEeS90nYwXEkhPLEnN Tk0tSC2CyTJxcEo1MC1uLLJ8ZfEh5GxFkkScfTCn3cG/ifXRXnmHE94Vae/SU0hw4SxYecDN LOihzReFnWI6P48rnn7SbcjAITbp6Yt6LxfV6o02tup557Mzaj4taZF9Uz2nxKXyzkbVlNxj geweCWYXJZRjy4SOt/CXmJQYlRvdPVu7f1L5RP3ojLwtM2/84+TvufQz/PlBQYVN0Xt8Ba70 28zjsHZ+3RA0Rer5NM6cyYzS0bqtqgcTfE5k+fBL/RT/FS3FoVa91sJrSn6eSantz87Ni+bx Hiu5WnRohiZbAN/M5ZV35002jT3+d8J2v2uVGl5yM11r8z0D0kwLpNcUqFuY2n3ddP1zzNQt efJaGupS3ScO1iuxFGckGmoxFxUnAgAMk3+G8gIAAA== X-CMS-MailID: 20211220142237epcas5p48729a52293e4f7627e6ec53ca67b9c58 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20211220142237epcas5p48729a52293e4f7627e6ec53ca67b9c58 References: <20211220141734.12206-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Add URING_CMD_FIXEDBUFS flag to use fixedbufs enabled passthrough. Refactor the existing code and factor out helper that can be used for passthrough with fixed-buffer. This is a prep patch. Signed-off-by: Kanchan Joshi Signed-off-by: Anuj Gupta --- fs/io_uring.c | 20 ++++++++++++++------ include/linux/io_uring.h | 10 ++++++++++ 2 files changed, 24 insertions(+), 6 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 1061b4cde4be..cc6735913c4b 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -3125,12 +3125,10 @@ static void kiocb_done(struct io_kiocb *req, ssize_t ret, } } } - -static int __io_import_fixed(struct io_kiocb *req, int rw, struct iov_iter *iter, - struct io_mapped_ubuf *imu) +static int __io_import_fixed(u64 buf_addr, size_t len, int rw, + struct iov_iter *iter, struct io_mapped_ubuf *imu) { - size_t len = req->rw.len; - u64 buf_end, buf_addr = req->rw.addr; + u64 buf_end; size_t offset; if (unlikely(check_add_overflow(buf_addr, (u64)len, &buf_end))) @@ -3199,8 +3197,18 @@ static int io_import_fixed(struct io_kiocb *req, int rw, struct iov_iter *iter) imu = READ_ONCE(ctx->user_bufs[index]); req->imu = imu; } - return __io_import_fixed(req, rw, iter, imu); + return __io_import_fixed(req->rw.addr, req->rw.len, rw, iter, imu); +} + +int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, + int rw, struct iov_iter *iter, void *ioucmd) +{ + struct io_kiocb *req = container_of(ioucmd, struct io_kiocb, uring_cmd); + struct io_mapped_ubuf *imu = req->imu; + + return __io_import_fixed(ubuf, len, rw, iter, imu); } +EXPORT_SYMBOL_GPL(io_uring_cmd_import_fixed); static void io_ring_submit_unlock(struct io_ring_ctx *ctx, bool needs_lock) { diff --git a/include/linux/io_uring.h b/include/linux/io_uring.h index 5ab824ced147..07732bc850af 100644 --- a/include/linux/io_uring.h +++ b/include/linux/io_uring.h @@ -5,6 +5,9 @@ #include #include +enum { + URING_CMD_FIXEDBUFS = (1 << 1), +}; /* * Note that the first member here must be a struct file, as the * io_uring command layout depends on that. @@ -20,6 +23,8 @@ struct io_uring_cmd { }; #if defined(CONFIG_IO_URING) +int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, + int rw, struct iov_iter *iter, void *ioucmd); void io_uring_cmd_done(struct io_uring_cmd *cmd, ssize_t ret); void io_uring_cmd_complete_in_task(struct io_uring_cmd *ioucmd, void (*driver_cb)(struct io_uring_cmd *)); @@ -50,6 +55,11 @@ static inline void io_uring_cmd_complete_in_task(struct io_uring_cmd *ioucmd, void (*driver_cb)(struct io_uring_cmd *)) { } +int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, + int rw, struct iov_iter *iter, void *ioucmd) +{ + return -1; +} static inline struct sock *io_uring_get_socket(struct file *file) { return NULL; From patchwork Mon Dec 20 14:17:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12688949 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A86BC433F5 for ; Tue, 21 Dec 2021 02:56:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234102AbhLUC4l (ORCPT ); Mon, 20 Dec 2021 21:56:41 -0500 Received: from mailout3.samsung.com ([203.254.224.33]:46263 "EHLO mailout3.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234050AbhLUC4l (ORCPT ); Mon, 20 Dec 2021 21:56:41 -0500 Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20211221025639epoutp03bc84bf6df56281be396e2f4b88473832~CpbJUOIto3071930719epoutp03n for ; Tue, 21 Dec 2021 02:56:39 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20211221025639epoutp03bc84bf6df56281be396e2f4b88473832~CpbJUOIto3071930719epoutp03n DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1640055399; bh=2DlgsdVDD8TJPholpkMcAzQvduH7Afep+ScWGG73sRs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lkq56k4DAdRlRvkiJ0uV+2hmvWEtl1rHSzWoPvP/F+oWAoLywlABqL0XBPgw12ZnU VJjva84RyVdkGAFzW/hjaq0SQB8d/TFasqbLCALzAodYQZwqJPT32kfQgypkQit9qX ZX/7QmkTaezBNDCS0Xp7nPVJaNlPni4oo8+lZSOw= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20211221025639epcas5p2488e211d93d5212947c28175128a9947~CpbI0t8M82224322243epcas5p2b; Tue, 21 Dec 2021 02:56:39 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.178]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4JJ1Lp0ZG8z4x9QD; Tue, 21 Dec 2021 02:56:34 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 9D.F1.46822.16241C16; Tue, 21 Dec 2021 11:56:34 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20211220142239epcas5p3efc3c89bd536f3f5d728c81bc550e143~CfI0LXJGL2729127291epcas5p3O; Mon, 20 Dec 2021 14:22:39 +0000 (GMT) Received: from epsmgms1p2.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20211220142239epsmtrp185588a27f5a3594c45f233f4cadcac15~CfI0Kp2mA2445924459epsmtrp1f; Mon, 20 Dec 2021 14:22:39 +0000 (GMT) X-AuditID: b6c32a4a-dfbff7000000b6e6-bc-61c14261fd9b Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p2.samsung.com (Symantec Messaging Gateway) with SMTP id B5.65.08738.FA190C16; Mon, 20 Dec 2021 23:22:39 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20211220142237epsmtip186a84a7fff6983188e70229f09aa16c6~CfIyqUKf20040400404epsmtip1h; Mon, 20 Dec 2021 14:22:37 +0000 (GMT) From: Kanchan Joshi To: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, javier@javigon.com, anuj20.g@samsung.com, joshiiitr@gmail.com, pankydev8@gmail.com Subject: [RFC 06/13] io_uring: add support for uring_cmd with fixed-buffer Date: Mon, 20 Dec 2021 19:47:27 +0530 Message-Id: <20211220141734.12206-7-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211220141734.12206-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrBJsWRmVeSWpSXmKPExsWy7bCmpm6S08FEg5sHmCyaJvxltlh9t5/N YuXqo0wW71rPsVh0nr7AZHH+7WEmi0mHrjFa7L2lbTF/2VN2izU3n7I4cHnsnHWX3aN5wR0W j8tnSz02repk89i8pN5j980GNo++LasYPT5vkgvgiMq2yUhNTEktUkjNS85PycxLt1XyDo53 jjc1MzDUNbS0MFdSyEvMTbVVcvEJ0HXLzAG6UEmhLDGnFCgUkFhcrKRvZ1OUX1qSqpCRX1xi q5RakJJTYFKgV5yYW1yal66Xl1piZWhgYGQKVJiQnTFn6WbGgibpilsfZjE1MN4U7WLk5JAQ MJFYM+k3G4gtJLCbUeLKzOwuRi4g+xOjxL5fx1khEp8ZJdo/pcM07Dr2jQ2iaBejxPTXJ5ng ilq7NLsYOTjYBDQlLkwuBQmLCERLXHh+DWwBs0AHo8TOblsQW1jAS+L89c1gcRYBVYk5jafB bF4BC4l70z6xQuySl5h56Ts7iM0pYClxePYyqBpBiZMzn7BAzJSXaN46mxmifiqHxMobXBC2 i8SJtt1QcWGJV8e3sEPYUhIv+9ug7GKJX3eOMoP8IgFy2/WGmSwQCXuJi3v+MoH8wgz0y/pd +hBhWYmpp9YxQezlk+j9/YQJIs4rsWMejK0ocW/SU6j7xSUezlgCZXtI7Np2lwkSbj2MEtdW bGKfwKgwC8k/s5D8Mwth9QJG5lWMkqkFxbnpqcWmBUZ5qeXwKE7Oz93ECE6yWl47GB8++KB3 iJGJg/EQowQHs5II75bZ+xOFeFMSK6tSi/Lji0pzUosPMZoCA3wis5Rocj4wzeeVxBuaWBqY mJmZmVgamxkqifOeTt+QKCSQnliSmp2aWpBaBNPHxMEp1cC05Jerw4RHlyyZ9B5mair5/+A4 W37c1dbf7uXydXfyvPZO7RN9+08/Y0PGpOnn8s1aHhrMey8gnPEzzM5ki+cnQfu1Je9+7gjx vVL5/bHyPlNH4fz37Ykf8y+8fnmr90eR0hY271frv3I0TDy43VFwq0zJvYxri78YJjYIL/2a fvfmHJ0PX/brPF1YXHFO+bH/awY/GdmMQ9k20/qZu/+eTFPryAtsVC39MOfJ2oB5Tds1PXa+ uzzZQSxMWbk5qWv6hIslDyQZtd9qqol6GC6YeGaiSsS1rNzHxlsnyU+/Fzhn16qnbNuusU55 /ECLz7zXzeEng/qEK2wz8/s+BSsvVX+RsNdy1kkJk19+FXOPKbEUZyQaajEXFScCAAJY+xA7 BAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrELMWRmVeSWpSXmKPExsWy7bCSnO76iQcSDbb2Slg0TfjLbLH6bj+b xcrVR5ks3rWeY7HoPH2ByeL828NMFpMOXWO02HtL22L+sqfsFmtuPmVx4PLYOesuu0fzgjss HpfPlnpsWtXJ5rF5Sb3H7psNbB59W1YxenzeJBfAEcVlk5Kak1mWWqRvl8CVMWfpZsaCJumK Wx9mMTUw3hTtYuTkkBAwkdh17BtbFyMXh5DADkaJh9vnskEkxCWar/1gh7CFJVb+e84OUfSR UWLG7iaWLkYODjYBTYkLk0tBakQEYiU+/DrGBFLDLDCJUWJD/wOwZmEBL4nz1zeDDWURUJWY 03gazOYVsJC4N+0TK8QCeYmZl76D1XMKWEocnr0MrEYIqObEhy8sEPWCEidnPgGzmYHqm7fO Zp7AKDALSWoWktQCRqZVjJKpBcW56bnFhgVGeanlesWJucWleel6yfm5mxjB0aCltYNxz6oP eocYmTgYDzFKcDArifBumb0/UYg3JbGyKrUoP76oNCe1+BCjNAeLkjjvha6T8UIC6Yklqdmp qQWpRTBZJg5OqQYmv8TQm8989giXRs25f4NfiO+G13MjF92CN2eCV8s5nHi27kvDonW2ucWC 1Qq6QlVWMsq85899OPR5m8etPjf2FZs/bu65fUFpjWjw/QcfnA7Mm3Xpz8vaOon19566SNXK HP99IPzqikWnln2LiLd5ufJpPfepy13FJ26muzY9zC7/ZMW/09yg4nLCjKPNc3iNt09sbjSP frXD0V9VtSP6+N8Lf8scPxteqt8swfFIsPL9t+a9e080t6k+2FLEuOb/t1mhygp9sszb3H5G Xcl41DD797VZO580HPdc0nQ5/7rkmQf8BUJKM2fa6BUfe6W+4969ZQYR7TY7GkpfT9j4jc+Q u/nEh1KzuGeCM0sORCmxFGckGmoxFxUnAgBBUgm19QIAAA== X-CMS-MailID: 20211220142239epcas5p3efc3c89bd536f3f5d728c81bc550e143 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20211220142239epcas5p3efc3c89bd536f3f5d728c81bc550e143 References: <20211220141734.12206-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Anuj Gupta Add IORING_OP_URING_CMD_FIXED opcode that enables performing the operation with previously registered buffers. Signed-off-by: Anuj Gupta --- fs/io_uring.c | 29 ++++++++++++++++++++++++++++- include/uapi/linux/io_uring.h | 6 +++++- 2 files changed, 33 insertions(+), 2 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index cc6735913c4b..2870a891e441 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -1122,6 +1122,10 @@ static const struct io_op_def io_op_defs[] = { .needs_file = 1, .offsets = 1, }, + [IORING_OP_URING_CMD_FIXED] = { + .needs_file = 1, + .offsets = 1, + }, }; /* requests with any of those set should undergo io_disarm_next() */ @@ -4133,6 +4137,7 @@ EXPORT_SYMBOL_GPL(io_uring_cmd_done); static int io_uring_cmd_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) { + struct io_ring_ctx *ctx = req->ctx; const struct io_uring_cmd_sqe *csqe = (const void *) sqe; struct io_uring_cmd *cmd = &req->uring_cmd; @@ -4145,7 +4150,13 @@ static int io_uring_cmd_prep(struct io_kiocb *req, } cmd->op = READ_ONCE(csqe->op); - cmd->len = READ_ONCE(csqe->len); + if (req->opcode == IORING_OP_URING_CMD_FIXED) { + req->imu = NULL; + io_req_set_rsrc_node(req, ctx); + req->buf_index = READ_ONCE(csqe->buf_index); + req->uring_cmd.flags |= URING_CMD_FIXEDBUFS; + } else + cmd->len = READ_ONCE(csqe->len); /* * The payload is the last 40 bytes of an io_uring_cmd_sqe, with the @@ -4160,6 +4171,20 @@ static int io_uring_cmd(struct io_kiocb *req, unsigned int issue_flags) struct file *file = req->file; int ret; + if (req->opcode == IORING_OP_URING_CMD_FIXED) { + u32 index, buf_index = req->buf_index; + struct io_ring_ctx *ctx = req->ctx; + struct io_mapped_ubuf *imu = req->imu; + + if (likely(!imu)) { + if (unlikely(buf_index >= ctx->nr_user_bufs)) + return -EFAULT; + index = array_index_nospec(buf_index, ctx->nr_user_bufs); + imu = READ_ONCE(ctx->user_bufs[index]); + req->imu = imu; + } + } + ret = file->f_op->async_cmd(&req->uring_cmd, issue_flags); /* queued async, consumer will call io_uring_cmd_done() when complete */ if (ret == -EIOCBQUEUED) @@ -6675,6 +6700,7 @@ static int io_req_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) case IORING_OP_LINKAT: return io_linkat_prep(req, sqe); case IORING_OP_URING_CMD: + case IORING_OP_URING_CMD_FIXED: return io_uring_cmd_prep(req, sqe); } @@ -6960,6 +6986,7 @@ static int io_issue_sqe(struct io_kiocb *req, unsigned int issue_flags) ret = io_linkat(req, issue_flags); break; case IORING_OP_URING_CMD: + case IORING_OP_URING_CMD_FIXED: ret = io_uring_cmd(req, issue_flags); break; default: diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h index 7191419f2236..cb331f201255 100644 --- a/include/uapi/linux/io_uring.h +++ b/include/uapi/linux/io_uring.h @@ -79,7 +79,10 @@ struct io_uring_cmd_sqe { __u64 user_data; __u16 op; __u16 personality; - __u32 len; + union { + __u32 len; + __u16 buf_index; + }; __u64 pdu[5]; }; @@ -164,6 +167,7 @@ enum { IORING_OP_SYMLINKAT, IORING_OP_LINKAT, IORING_OP_URING_CMD, + IORING_OP_URING_CMD_FIXED, /* this goes last, obviously */ IORING_OP_LAST, From patchwork Mon Dec 20 14:17:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12688951 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59456C433EF for ; Tue, 21 Dec 2021 02:56:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234110AbhLUC4q (ORCPT ); Mon, 20 Dec 2021 21:56:46 -0500 Received: from mailout3.samsung.com ([203.254.224.33]:46337 "EHLO mailout3.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234089AbhLUC4q (ORCPT ); Mon, 20 Dec 2021 21:56:46 -0500 Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20211221025644epoutp03e599cde95554539b9affe894147a310b~CpbOD8Idr3025730257epoutp03t for ; Tue, 21 Dec 2021 02:56:44 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20211221025644epoutp03e599cde95554539b9affe894147a310b~CpbOD8Idr3025730257epoutp03t DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1640055404; bh=bss9OWagJehTewuWFbdwSOhXbbsGjfeINSnWwzxnnh0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bPZsbuQEMmLoq0MLmV/IY2HX5HsMXuubluPaCD5A+srVPoySkG7nmIjk20sLnKREO lmP8U1U7id3q8TECkgnnPRlTQE5/91pdvS9w7dHrqihi4mEMJ55FqMdIVeEr1nnOsL YlIxaVEhqRjgssUfH6PLRvqWCszwofLIewVx4VOM= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20211221025644epcas5p292b210d7cc5dfdfa8f79b3bed0d1ac8f~CpbNfxEBe2587925879epcas5p2Q; Tue, 21 Dec 2021 02:56:44 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.174]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4JJ1Lw3z2rz4x9Px; Tue, 21 Dec 2021 02:56:40 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 3A.02.46822.86241C16; Tue, 21 Dec 2021 11:56:40 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20211220142242epcas5p45dddab51a9f20a8ec3d8b8e4f1dda40a~CfI2l8kWO2261322613epcas5p4s; Mon, 20 Dec 2021 14:22:42 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20211220142242epsmtrp1aa7f27bbe814be26ba5fadc58ea0a8e6~CfI2lQtPn2445924459epsmtrp1g; Mon, 20 Dec 2021 14:22:42 +0000 (GMT) X-AuditID: b6c32a4a-de5ff7000000b6e6-d7-61c142687e44 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id B6.85.29871.1B190C16; Mon, 20 Dec 2021 23:22:41 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20211220142239epsmtip182586cf1250a6f94ad6f3bf56893816f~CfI0O3QeY0637906379epsmtip1q; Mon, 20 Dec 2021 14:22:39 +0000 (GMT) From: Kanchan Joshi To: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, javier@javigon.com, anuj20.g@samsung.com, joshiiitr@gmail.com, pankydev8@gmail.com Subject: [RFC 07/13] nvme: enable passthrough with fixed-buffer Date: Mon, 20 Dec 2021 19:47:28 +0530 Message-Id: <20211220141734.12206-8-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211220141734.12206-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrBJsWRmVeSWpSXmKPExsWy7bCmlm6G08FEg1PvpCyaJvxltlh9t5/N YuXqo0wW71rPsVh0nr7AZHH+7WEmi0mHrjFa7L2lbTF/2VN2izU3n7I4cHnsnHWX3aN5wR0W j8tnSz02repk89i8pN5j980GNo++LasYPT5vkgvgiMq2yUhNTEktUkjNS85PycxLt1XyDo53 jjc1MzDUNbS0MFdSyEvMTbVVcvEJ0HXLzAG6UEmhLDGnFCgUkFhcrKRvZ1OUX1qSqpCRX1xi q5RakJJTYFKgV5yYW1yal66Xl1piZWhgYGQKVJiQnfHo5QGmgmPaFat/r2duYHyg3MXIwSEh YCJxtNe7i5GLQ0hgN6NE085PLBDOJ0aJA2+uMEI43xglDn98BZThBOv4/u8WVGIvo8Tj32fY QRJCAp8ZJT5dYQEZyyagKXFhcilIWEQgWuLC82tsIDazQAejxM5uWxBbWMBeYuvTt2AzWQRU JR6s/g9WwytgIdF45C87xC55iZmXvoPZnAKWEodnL4OqEZQ4OfMJC8RMeYnmrbOZQe6REJjI IfF891JmiNdcJL4/cIaYIyzx6vgWqJlSEi/726DsYolfd45C9QLddr1hJtST9hIX9/xlApnD DPTL+l36EGFZiamn1jFB7OWT6P39hAkiziuxYx6MrShxb9JTVghbXOLhjCWsEOd4SPyZLgEJ th5Gif7NN1kmMCrMQvLOLCTvzELYvICReRWjZGpBcW56arFpgVFeajk8ipPzczcxgpOsltcO xocPPugdYmTiYDzEKMHBrCTCu2X2/kQh3pTEyqrUovz4otKc1OJDjKbA8J7ILCWanA9M83kl 8YYmlgYmZmZmJpbGZoZK4ryn0zckCgmkJ5akZqemFqQWwfQxcXBKNTBlzJebsGWOkWfmkz3N Uf+XvH+oUm/DEVBhdfiQg/wpxplNy0T+MlQsCbBLOrPb8/4LxTnzLd68SFL8Xa5bNLXNYvvJ oNpL1yVY/I8osmuf5jlUqvkgcdtP7dd/OwPLps374mPKJpCm5aATZ6hfJ6WWcf7e6R1Zc4+W ecXx7V7s7Xvsc+ynXTeb32mrCui8/SCVcKWhJ/OLzUWdSrc7ld9ZPDxtrPRDt06LP3Fx+b9n n9keHXjh2JWbI+65UKsi+Xnh5Nz2GSK7J+80nD6lOO3/2h9ey+VXHZcMXBDy6NNOVyNuJsmg OnXJU2weXcv5zj/UtHzfHZBiPcvCoHvRhilzNC37Vhezfv67hF3wQqYSS3FGoqEWc1FxIgBS p/JzOwQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrELMWRmVeSWpSXmKPExsWy7bCSnO7GiQcSDb6+t7ZomvCX2WL13X42 i5WrjzJZvGs9x2LRefoCk8X5t4eZLCYdusZosfeWtsX8ZU/ZLdbcfMriwOWxc9Zddo/mBXdY PC6fLfXYtKqTzWPzknqP3Tcb2Dz6tqxi9Pi8SS6AI4rLJiU1J7MstUjfLoEr49HLA0wFx7Qr Vv9ez9zA+EC5i5GTQ0LAROL7v1uMXYxcHEICuxkl7p9pYYRIiEs0X/vBDmELS6z895wdougj o8TPo++Yuxg5ONgENCUuTC4FqRERiJX48OsYE0gNs8AkRokN/Q/AmoUF7CW2Pn3LAmKzCKhK PFj9nw3E5hWwkGg88hdqgbzEzEvfwWxOAUuJw7OXgdUIAdWc+PCFBaJeUOLkzCdgNjNQffPW 2cwTGAVmIUnNQpJawMi0ilEytaA4Nz232LDAMC+1XK84Mbe4NC9dLzk/dxMjOBq0NHcwbl/1 Qe8QIxMH4yFGCQ5mJRHeLbP3JwrxpiRWVqUW5ccXleakFh9ilOZgURLnvdB1Ml5IID2xJDU7 NbUgtQgmy8TBKdXANEN9VTnn/HcdFmI9zzY8u7Nj9RLuVt/Uu9m2L1ff0FVv1MkziUspNorl lD5/qU+R+T6D64bHXle3/me1TijWf90UFPHb6IAiY1H0rr+ne6dETQs/WfNwybr3hhYR9d9z 1XdvEq7qaVVkMlv1eWfKZlfHOKE70d9nGc56//2C5OEjmZfVFPQ8ueOds89lbFD78nN1vXLY DKfg3J8pvxUUvvTU/nnFcP/+lKgdE0026i2bab7zo39DYO+eJWKL+KQYgl8yHuBveu9ns6H2 bPQWN4bXZ/ZEpLb+rni5KOSPF0vJtP+Vu1refUpwmvWk1rxXfzJ3jSvjTt9/p9mnLKyM2dvV w+F1LP6V67bOzaoxSizFGYmGWsxFxYkAgFq2cfUCAAA= X-CMS-MailID: 20211220142242epcas5p45dddab51a9f20a8ec3d8b8e4f1dda40a X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20211220142242epcas5p45dddab51a9f20a8ec3d8b8e4f1dda40a References: <20211220141734.12206-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Add support to carry out passthrough command with pre-mapped buffers. Signed-off-by: Kanchan Joshi --- block/blk-map.c | 46 +++++++++++++++++++++++++++++++++++++++ drivers/nvme/host/ioctl.c | 30 +++++++++++++++---------- include/linux/blk-mq.h | 3 +++ 3 files changed, 67 insertions(+), 12 deletions(-) diff --git a/block/blk-map.c b/block/blk-map.c index 4526adde0156..9aa9864eab55 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -8,6 +8,7 @@ #include #include #include +#include #include "blk.h" @@ -577,6 +578,51 @@ int blk_rq_map_user(struct request_queue *q, struct request *rq, } EXPORT_SYMBOL(blk_rq_map_user); +/* Unlike blk_rq_map_user () this is only for fixed-buffer async passthrough. */ +int blk_rq_map_user_fixedb(struct request_queue *q, struct request *rq, + u64 ubuf, unsigned long len, gfp_t gfp_mask, + struct io_uring_cmd *ioucmd) +{ + struct iov_iter iter; + size_t iter_count, nr_segs; + struct bio *bio; + int ret; + + /* + * Talk to io_uring to obtain BVEC iterator for the buffer. + * And use that iterator to form bio/request. + */ + ret = io_uring_cmd_import_fixed(ubuf, len, rq_data_dir(rq), &iter, + ioucmd); + if (unlikely(ret < 0)) + return ret; + iter_count = iov_iter_count(&iter); + nr_segs = iter.nr_segs; + + if (!iter_count || (iter_count >> 9) > queue_max_hw_sectors(q)) + return -EINVAL; + if (nr_segs > queue_max_segments(q)) + return -EINVAL; + /* no iovecs to alloc, as we already have a BVEC iterator */ + bio = bio_alloc(gfp_mask, 0); + if (!bio) + return -ENOMEM; + + bio->bi_opf |= req_op(rq); + ret = bio_iov_iter_get_pages(bio, &iter); + if (ret) + goto out_free; + + blk_rq_bio_prep(rq, bio, nr_segs); + return 0; + +out_free: + bio_release_pages(bio, false); + bio_put(bio); + return ret; +} +EXPORT_SYMBOL(blk_rq_map_user_fixedb); + /** * blk_rq_unmap_user - unmap a request with user data * @bio: start of bio list diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index 7d9c51d9c0a8..dc6a5f1b81ca 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -131,8 +131,13 @@ static void *nvme_add_user_metadata(struct bio *bio, void __user *ubuf, return ERR_PTR(ret); } +static inline bool nvme_is_fixedb_passthru(struct io_uring_cmd *ioucmd) +{ + return ((ioucmd) && (ioucmd->flags & URING_CMD_FIXEDBUFS)); +} + static int nvme_submit_user_cmd(struct request_queue *q, - struct nvme_command *cmd, void __user *ubuffer, + struct nvme_command *cmd, u64 ubuffer, unsigned bufflen, void __user *meta_buffer, unsigned meta_len, u32 meta_seed, u64 *result, unsigned timeout, struct io_uring_cmd *ioucmd) @@ -154,8 +159,12 @@ static int nvme_submit_user_cmd(struct request_queue *q, nvme_req(req)->flags |= NVME_REQ_USERCMD; if (ubuffer && bufflen) { - ret = blk_rq_map_user(q, req, NULL, ubuffer, bufflen, - GFP_KERNEL); + if (likely(!nvme_is_fixedb_passthru(ioucmd))) + ret = blk_rq_map_user(q, req, NULL, nvme_to_user_ptr(ubuffer), + bufflen, GFP_KERNEL); + else + ret = blk_rq_map_user_fixedb(q, req, ubuffer, bufflen, + GFP_KERNEL, ioucmd); if (ret) goto out; bio = req->bio; @@ -254,9 +263,8 @@ static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio) c.rw.appmask = cpu_to_le16(io.appmask); return nvme_submit_user_cmd(ns->queue, &c, - nvme_to_user_ptr(io.addr), length, - metadata, meta_len, lower_32_bits(io.slba), NULL, 0, - NULL); + io.addr, length, metadata, meta_len, + lower_32_bits(io.slba), NULL, 0, NULL); } static bool nvme_validate_passthru_nsid(struct nvme_ctrl *ctrl, @@ -308,9 +316,8 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns, timeout = msecs_to_jiffies(cmd.timeout_ms); status = nvme_submit_user_cmd(ns ? ns->queue : ctrl->admin_q, &c, - nvme_to_user_ptr(cmd.addr), cmd.data_len, - nvme_to_user_ptr(cmd.metadata), cmd.metadata_len, - 0, &result, timeout, NULL); + cmd.addr, cmd.data_len, nvme_to_user_ptr(cmd.metadata), + cmd.metadata_len, 0, &result, timeout, NULL); if (status >= 0) { if (put_user(result, &ucmd->result)) @@ -355,9 +362,8 @@ static int nvme_user_cmd64(struct nvme_ctrl *ctrl, struct nvme_ns *ns, timeout = msecs_to_jiffies(cmd.timeout_ms); status = nvme_submit_user_cmd(ns ? ns->queue : ctrl->admin_q, &c, - nvme_to_user_ptr(cmd.addr), cmd.data_len, - nvme_to_user_ptr(cmd.metadata), cmd.metadata_len, - 0, &cmd.result, timeout, ioucmd); + cmd.addr, cmd.data_len, nvme_to_user_ptr(cmd.metadata), + cmd.metadata_len, 0, &cmd.result, timeout, ioucmd); if (!ioucmd && status >= 0) { if (put_user(cmd.result, &ucmd->result)) diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 30e54ab05333..a82b054eebde 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -922,6 +922,9 @@ struct rq_map_data { int blk_rq_map_user(struct request_queue *, struct request *, struct rq_map_data *, void __user *, unsigned long, gfp_t); +int blk_rq_map_user_fixedb(struct request_queue *, struct request *, + u64 ubuf, unsigned long, gfp_t, + struct io_uring_cmd *); int blk_rq_map_user_iov(struct request_queue *, struct request *, struct rq_map_data *, const struct iov_iter *, gfp_t); int blk_rq_unmap_user(struct bio *); From patchwork Mon Dec 20 14:17:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12688953 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A35BC433F5 for ; Tue, 21 Dec 2021 02:57:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234154AbhLUC5D (ORCPT ); Mon, 20 Dec 2021 21:57:03 -0500 Received: from mailout3.samsung.com ([203.254.224.33]:46527 "EHLO mailout3.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234104AbhLUC4z (ORCPT ); Mon, 20 Dec 2021 21:56:55 -0500 Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20211221025653epoutp03f0a3b5cbc061173ffe3ffc28e28f5539~CpbWW3mOT3025730257epoutp030 for ; Tue, 21 Dec 2021 02:56:53 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20211221025653epoutp03f0a3b5cbc061173ffe3ffc28e28f5539~CpbWW3mOT3025730257epoutp030 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1640055413; bh=+fDobuNhmEqQD9o52/bcCFNDtnitrCw7j/s7XoX46j8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vYMuC7Ji92/wGbuXh0N6aoOoCzKlQCbvPlnRSfbYgvzveVkI8pdtChvQD8crKSJge Asn5AeimNgJPOFkUxMvbHVzKEUdkBpz86vVmd3lJ/Mlgy/pTGaf647suWgrJvGcqob HCSqxKYDXbuRjQUR03GGZgJZg/s26gJzqPM68wmk= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20211221025653epcas5p1719e87d09bfb48d21593ff5bce4a3e67~CpbV1OSFi0291802918epcas5p1c; Tue, 21 Dec 2021 02:56:53 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.183]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4JJ1M333RCz4x9Pp; Tue, 21 Dec 2021 02:56:47 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 57.66.06423.C6241C16; Tue, 21 Dec 2021 11:56:44 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20211220142244epcas5p2f311ed168b8f31b9301bcc2002076db4~CfI4p3oaf1916119161epcas5p22; Mon, 20 Dec 2021 14:22:44 +0000 (GMT) Received: from epsmgms1p2.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20211220142244epsmtrp11934831347aefccf2553b5dc3795b983~CfI4o8_Sa2445924459epsmtrp1i; Mon, 20 Dec 2021 14:22:44 +0000 (GMT) X-AuditID: b6c32a49-b01ff70000001917-87-61c1426c83c5 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p2.samsung.com (Symantec Messaging Gateway) with SMTP id 86.65.08738.4B190C16; Mon, 20 Dec 2021 23:22:44 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20211220142242epsmtip19aa44cf774b51f0552270d47f9c773b1~CfI2s34xu0040000400epsmtip1g; Mon, 20 Dec 2021 14:22:42 +0000 (GMT) From: Kanchan Joshi To: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, javier@javigon.com, anuj20.g@samsung.com, joshiiitr@gmail.com, pankydev8@gmail.com Subject: [RFC 08/13] io_uring: plug for async bypass Date: Mon, 20 Dec 2021 19:47:29 +0530 Message-Id: <20211220141734.12206-9-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211220141734.12206-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrFJsWRmVeSWpSXmKPExsWy7bCmhm6O08FEg8ktahZNE/4yW6y+289m sXL1USaLd63nWCw6T19gsjj/9jCTxaRD1xgt9t7Stpi/7Cm7xZqbT1kcuDx2zrrL7tG84A6L x+WzpR6bVnWyeWxeUu+x+2YDm0ffllWMHp83yQVwRGXbZKQmpqQWKaTmJeenZOal2yp5B8c7 x5uaGRjqGlpamCsp5CXmptoqufgE6Lpl5gBdqKRQlphTChQKSCwuVtK3synKLy1JVcjILy6x VUotSMkpMCnQK07MLS7NS9fLSy2xMjQwMDIFKkzIzri4eD1bwUaWirv7vrM0MK5g7mLk5JAQ MJHoXnGFrYuRi0NIYDejxJZn79lBEkICnxgl1vQVQSQ+M0p8XLKYBabj1JXHjBCJXYwSpx+v ZoToAKpaMcO7i5GDg01AU+LC5FKQsIhAtMSF59fYQGxmgQ5GiZ3dtiC2MNCcbSeug8VZBFQl 5s2ZDLaYV8BC4s+WK4wQu+QlZl76DhbnFLCUODx7GRtEjaDEyZlPWCBmyks0b53NDHKPhEAv h8SK20dYIZpdJA5/WwP1prDEq+Nb2CFsKYmX/W1QdrHErztHoZqBjrveMBPqS3uJi3v+MoE8 wwz0zPpd+hBhWYmpp9YxQSzmk+j9/YQJIs4rsWMejK0ocW/SU6gbxCUezlgCZXtIrNg4FRrU PYwS/2btYZnAqDALyUOzkDw0C2H1AkbmVYySqQXFuempxaYFhnmp5fBITs7P3cQITrRanjsY 7z74oHeIkYmD8RCjBAezkgjvltn7E4V4UxIrq1KL8uOLSnNSiw8xmgJDfCKzlGhyPjDV55XE G5pYGpiYmZmZWBqbGSqJ855O35AoJJCeWJKanZpakFoE08fEwSnVwMSvv65sbuTUG4ah/Gu+ TfG18HS8F7VWwGnrq7A4vV1Lrl3OqanTXqIfmb7Cqnqt36mDP7S4fs7d/Mexsme214XVoTlF 9/eq7TqzqTlOwY6ZRSOw67jBHGf7+9znXk/uVFf/9sLkEdsMRuH/T+OaX+9lMkn9qLX8+KPm xp1t4nyPt1qcX/L1jHGEi+aJ6/3/03es3TpnPw/QEXFOTav3+ASFri1fes7u0Z4UuaVrVCZ6 TeAxm/fjSdGa/lsHcwvXeJcy7Pj262DjtgVXZ509GKmmxG6cFsL+7ur/pt3McT/FZ9/mWXHV 9Ezagx8T1Eyaq8pYlmiescg/fbAqdYOb4mPNaO5HzC/Fyqw8XBiPliuxFGckGmoxFxUnAgBr jbdBPQQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrELMWRmVeSWpSXmKPExsWy7bCSnO6WiQcSDW5d57JomvCX2WL13X42 i5WrjzJZvGs9x2LRefoCk8X5t4eZLCYdusZosfeWtsX8ZU/ZLdbcfMriwOWxc9Zddo/mBXdY PC6fLfXYtKqTzWPzknqP3Tcb2Dz6tqxi9Pi8SS6AI4rLJiU1J7MstUjfLoEr4+Li9WwFG1kq 7u77ztLAuIK5i5GTQ0LAROLUlceMXYxcHEICOxglrq1oYoRIiEs0X/vBDmELS6z895wdougj o8SkI71MXYwcHGwCmhIXJpeC1IgIxEp8+HWMCaSGWWASo8SG/gdgzcJAG7aduM4GYrMIqErM mzMZLM4rYCHxZ8sVqGXyEjMvfQeLcwpYShyevQysXgio5sSHLywQ9YISJ2c+AbOZgeqbt85m nsAoMAtJahaS1AJGplWMkqkFxbnpucWGBUZ5qeV6xYm5xaV56XrJ+bmbGMHRoKW1g3HPqg96 hxiZOBgPMUpwMCuJ8G6ZvT9RiDclsbIqtSg/vqg0J7X4EKM0B4uSOO+FrpPxQgLpiSWp2amp BalFMFkmDk6pBqYGSUFBzXv/hY5udP1t0/Chwnx20xzzg1e8lsWvNDszdyPD7m+/n3i8DpFw +ROQrZ9VHunUJGH7+nrxkjmX9wW58d+pfBopO0N21Tn7vzstTDkVH39uCqxfzcL4b2nPrYBt MT5lEst+7zd8IziDiSEv7I/rTb92z9aTVpNvXbx7r+B9oe1MduVm418TlzHLLppx2+vwy0M7 ZhSclX61x7RQVnV22dk2sZui2zgt87P+Njz9zFy1gm9Rp2LAApvU66uM9T7PN3u2a0LUeSfR lJr6Zu/saAHpMpXdh16l3bO8EsTrxCfytenFqw4+a9UpjKKsAvom3Szcmb9jBFJbNCQ6xbYv VmyapxB7YPOPi0osxRmJhlrMRcWJAC+rRI/1AgAA X-CMS-MailID: 20211220142244epcas5p2f311ed168b8f31b9301bcc2002076db4 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20211220142244epcas5p2f311ed168b8f31b9301bcc2002076db4 References: <20211220141734.12206-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Jens Axboe Signed-off-by: Jens Axboe --- fs/io_uring.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/io_uring.c b/fs/io_uring.c index 2870a891e441..f77dde1bdc75 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -1121,10 +1121,12 @@ static const struct io_op_def io_op_defs[] = { [IORING_OP_URING_CMD] = { .needs_file = 1, .offsets = 1, + .plug = 1, }, [IORING_OP_URING_CMD_FIXED] = { .needs_file = 1, .offsets = 1, + .plug = 1, }, }; From patchwork Mon Dec 20 14:17:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12688955 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 391DBC4332F for ; Tue, 21 Dec 2021 02:57:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234104AbhLUC5D (ORCPT ); Mon, 20 Dec 2021 21:57:03 -0500 Received: from mailout4.samsung.com ([203.254.224.34]:14906 "EHLO mailout4.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234120AbhLUC46 (ORCPT ); Mon, 20 Dec 2021 21:56:58 -0500 Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20211221025657epoutp04c95b1a5e370d34dc314bce62c10b090b~CpbZn47bV1811318113epoutp04S for ; Tue, 21 Dec 2021 02:56:57 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20211221025657epoutp04c95b1a5e370d34dc314bce62c10b090b~CpbZn47bV1811318113epoutp04S DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1640055417; bh=rfpmkXvtBx85hRvQm9aTlYTjkImDVAUlbe+HZabwpAQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fEvyjsB5SLS1YJ9r6dDvNnu5Nva85ghwvQJXUPalQ5U13eKCzVfa25X6chQBFCBJ9 hHxJwlHb4LVIQnuiludHqwG14q/Aldxzi0OVKzY5NJ2Poo7ljDYnzqRBqa7y7hE7ZV /VKLQcC3xx2M0wC2WdXp4TlEqQXxaUrhFohCcYm0= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20211221025656epcas5p35ebb22a17637f6ebec2e55376f046c35~CpbZKwkJO1655516555epcas5p30; Tue, 21 Dec 2021 02:56:56 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.183]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4JJ1M55HCZz4x9Q7; Tue, 21 Dec 2021 02:56:49 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 52.12.46822.17241C16; Tue, 21 Dec 2021 11:56:49 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20211220142246epcas5p303c64b6b1b832c7fcd5ac31fc79c91d1~CfI6mkxip0906209062epcas5p3C; Mon, 20 Dec 2021 14:22:46 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20211220142246epsmtrp2306cb12bf125091665ae9fc0e3b32dac~CfI6lxQIF1266312663epsmtrp27; Mon, 20 Dec 2021 14:22:46 +0000 (GMT) X-AuditID: b6c32a4a-de5ff7000000b6e6-f7-61c142717702 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id B7.85.29871.6B190C16; Mon, 20 Dec 2021 23:22:46 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20211220142244epsmtip177f41ad25b8108696b2cf53fd1c239c9~CfI4txnRE0637606376epsmtip1b; Mon, 20 Dec 2021 14:22:44 +0000 (GMT) From: Kanchan Joshi To: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, javier@javigon.com, anuj20.g@samsung.com, joshiiitr@gmail.com, pankydev8@gmail.com Subject: [RFC 09/13] block: wire-up support for plugging Date: Mon, 20 Dec 2021 19:47:30 +0530 Message-Id: <20211220141734.12206-10-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211220141734.12206-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrNJsWRmVeSWpSXmKPExsWy7bCmum6h08FEg/67WhZNE/4yW6y+289m sXL1USaLd63nWCw6T19gsjj/9jCTxaRD1xgt9t7Stpi/7Cm7xZqbT1kcuDx2zrrL7tG84A6L x+WzpR6bVnWyeWxeUu+x+2YDm0ffllWMHp83yQVwRGXbZKQmpqQWKaTmJeenZOal2yp5B8c7 x5uaGRjqGlpamCsp5CXmptoqufgE6Lpl5gBdqKRQlphTChQKSCwuVtK3synKLy1JVcjILy6x VUotSMkpMCnQK07MLS7NS9fLSy2xMjQwMDIFKkzIzlj6+ylzwQG5irWnp7I1MM6R7GLk5JAQ MJFY13uCsYuRi0NIYDejxIHuZ1DOJ0aJ7Q/7WSGcb4wSm5ramGBaOjfPZYdI7GWUeHb8OBOE 85lRYvXTJpYuRg4ONgFNiQuTS0EaRASiJS48v8YGYjMLdDBK7Oy2BbGFBSwkvj6/wQJiswio Smz6cosdxOYVsJTou3SMHWKZvMTMS9/BbE6g+OHZy9ggagQlTs58wgIxU16ieetsZpAbJAQm ckhc61/ADNHsIvFmxnoWCFtY4tXxLVBDpSQ+v9vLBmEXS/y6cxSqGei46w0zoRrsJS7u+csE 8gwz0DPrd+lDhGUlpp5axwSxmE+i9/cTaKjwSuyYB2MrStyb9JQVwhaXeDhjCSvIGAkBD4mm qwmQsOphlLh4+y3zBEaFWUj+mYXkn1kImxcwMq9ilEwtKM5NTy02LTDKSy2Hx3Jyfu4mRnCq 1fLawfjwwQe9Q4xMHIyHGCU4mJVEeLfM3p8oxJuSWFmVWpQfX1Sak1p8iNEUGOATmaVEk/OB yT6vJN7QxNLAxMzMzMTS2MxQSZz3dPqGRCGB9MSS1OzU1ILUIpg+Jg5OqQYm6TUS9SZXM03v VQpOfH0y2uSyX7OepVbDfl8B9ba5Jq+WCXd+fjNbK14iWfiP8B0uhyzb+ZcvHNtULPg+Knlz dJ5xbcnWfw37/5c5OKovj5U4bOVV9Xtl4Llv1ceslTKTjpU3ewl2TXv+O3zLgffbJs8yyHSf 2R0nFdk/4eqVu7EJl6NF74lwnJ62yqqvfGfY2RciYhKrm2+Wryi/Mck6pdJSf2+CwZUP3PXf 04MfToqfunGGwaoTW1xtg9kXfLSrrL1gd6t4y/0Zi5rTehoqRRo8ThrMOn7lTleylFp9tvKP 2RdlvvMsUvbuX7erM/VXsC/fM7Yvm1Y//VG6UXdL9P2JKy7vijy76PDe3rVHlViKMxINtZiL ihMBKWrOXT4EAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrELMWRmVeSWpSXmKPExsWy7bCSnO62iQcSDd5vELBomvCX2WL13X42 i5WrjzJZvGs9x2LRefoCk8X5t4eZLCYdusZosfeWtsX8ZU/ZLdbcfMriwOWxc9Zddo/mBXdY PC6fLfXYtKqTzWPzknqP3Tcb2Dz6tqxi9Pi8SS6AI4rLJiU1J7MstUjfLoErY+nvp8wFB+Qq 1p6eytbAOEeyi5GTQ0LARKJz81z2LkYuDiGB3YwSN96/Z4ZIiEs0X/vBDmELS6z89xyq6COj xJ6dfUxdjBwcbAKaEhcml4LUiAjESnz4dYwJpIZZYBKjxIb+B2DNwgIWEl+f32ABsVkEVCU2 fbkFFucVsJTou3QMaoG8xMxL38FsTqD44dnL2EBsIaDeEx++sEDUC0qcnPkEzGYGqm/eOpt5 AqPALCSpWUhSCxiZVjFKphYU56bnFhsWGOallusVJ+YWl+al6yXn525iBEeDluYOxu2rPugd YmTiYDzEKMHBrCTCu2X2/kQh3pTEyqrUovz4otKc1OJDjNIcLErivBe6TsYLCaQnlqRmp6YW pBbBZJk4OKUamKor3OqVCmtKN50qW//zcpr3Vd+sya7y+3gmivS8ex5rwNPx4v3BSdwhxp+7 ZHq/C3SlbVjEx6brNm+xjdtR3e9zzXuKjye/ycv8yFp/M9hTr+xb2iPtWOM/XOvkS+v2NLO7 GV2b8zui6aO3zb+/Bq8XveiaUzVndsSc4K18qcIvr6xe4np1d1ZC0bZderM8JvadPnSN4WD5 xd9xq8RMZ02ZJ/F9qfkKP8n1c5InWIvNrTV96sXHPL8spo37wIUC78oDscvaYj4WPt954EDm 7tcrG35uCj4gvV5866/5L/Wvvvi/89hatwafS1YLza9FVB641fk/NPhCbWlGbb9IbHXbc9nP zmoq52X2TZC8uU2JpTgj0VCLuag4EQC1zYEN9QIAAA== X-CMS-MailID: 20211220142246epcas5p303c64b6b1b832c7fcd5ac31fc79c91d1 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20211220142246epcas5p303c64b6b1b832c7fcd5ac31fc79c91d1 References: <20211220141734.12206-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Jens Axboe Add support to use plugging if it is enabled, else use default path. Signed-off-by: Jens Axboe --- block/blk-mq.c | 90 ++++++++++++++++++++++++++------------------------ 1 file changed, 47 insertions(+), 43 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 0ebd09492aa5..c77991688bfd 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -2326,6 +2326,40 @@ void __blk_mq_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, blk_mq_hctx_mark_pending(hctx, ctx); } +/* + * Allow 2x BLK_MAX_REQUEST_COUNT requests on plug queue for multiple + * queues. This is important for md arrays to benefit from merging + * requests. + */ +static inline unsigned short blk_plug_max_rq_count(struct blk_plug *plug) +{ + if (plug->multiple_queues) + return BLK_MAX_REQUEST_COUNT * 2; + return BLK_MAX_REQUEST_COUNT; +} + +static void blk_add_rq_to_plug(struct blk_plug *plug, struct request *rq) +{ + struct request *last = rq_list_peek(&plug->mq_list); + + if (!plug->rq_count) { + trace_block_plug(rq->q); + } else if (plug->rq_count >= blk_plug_max_rq_count(plug) || + (!blk_queue_nomerges(rq->q) && + blk_rq_bytes(last) >= BLK_PLUG_FLUSH_SIZE)) { + blk_mq_flush_plug_list(plug, false); + trace_block_plug(rq->q); + } + + if (!plug->multiple_queues && last && last->q != rq->q) + plug->multiple_queues = true; + if (!plug->has_elevator && (rq->rq_flags & RQF_ELV)) + plug->has_elevator = true; + rq->rq_next = NULL; + rq_list_add(&plug->mq_list, rq); + plug->rq_count++; +} + /** * blk_mq_request_bypass_insert - Insert a request at dispatch list. * @rq: Pointer to request to be inserted. @@ -2339,16 +2373,20 @@ void blk_mq_request_bypass_insert(struct request *rq, bool at_head, bool run_queue) { struct blk_mq_hw_ctx *hctx = rq->mq_hctx; + struct blk_plug *plug = current->plug; - spin_lock(&hctx->lock); - if (at_head) - list_add(&rq->queuelist, &hctx->dispatch); - else - list_add_tail(&rq->queuelist, &hctx->dispatch); - spin_unlock(&hctx->lock); - - if (run_queue) - blk_mq_run_hw_queue(hctx, false); + if (plug) { + blk_add_rq_to_plug(plug, rq); + } else { + spin_lock(&hctx->lock); + if (at_head) + list_add(&rq->queuelist, &hctx->dispatch); + else + list_add_tail(&rq->queuelist, &hctx->dispatch); + spin_unlock(&hctx->lock); + if (run_queue) + blk_mq_run_hw_queue(hctx, false); + } } void blk_mq_insert_requests(struct blk_mq_hw_ctx *hctx, struct blk_mq_ctx *ctx, @@ -2658,40 +2696,6 @@ void blk_mq_try_issue_list_directly(struct blk_mq_hw_ctx *hctx, hctx->queue->mq_ops->commit_rqs(hctx); } -/* - * Allow 2x BLK_MAX_REQUEST_COUNT requests on plug queue for multiple - * queues. This is important for md arrays to benefit from merging - * requests. - */ -static inline unsigned short blk_plug_max_rq_count(struct blk_plug *plug) -{ - if (plug->multiple_queues) - return BLK_MAX_REQUEST_COUNT * 2; - return BLK_MAX_REQUEST_COUNT; -} - -static void blk_add_rq_to_plug(struct blk_plug *plug, struct request *rq) -{ - struct request *last = rq_list_peek(&plug->mq_list); - - if (!plug->rq_count) { - trace_block_plug(rq->q); - } else if (plug->rq_count >= blk_plug_max_rq_count(plug) || - (!blk_queue_nomerges(rq->q) && - blk_rq_bytes(last) >= BLK_PLUG_FLUSH_SIZE)) { - blk_mq_flush_plug_list(plug, false); - trace_block_plug(rq->q); - } - - if (!plug->multiple_queues && last && last->q != rq->q) - plug->multiple_queues = true; - if (!plug->has_elevator && (rq->rq_flags & RQF_ELV)) - plug->has_elevator = true; - rq->rq_next = NULL; - rq_list_add(&plug->mq_list, rq); - plug->rq_count++; -} - static bool blk_mq_attempt_bio_merge(struct request_queue *q, struct bio *bio, unsigned int nr_segs) { From patchwork Mon Dec 20 14:17:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12688959 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DEFBFC43217 for ; Tue, 21 Dec 2021 02:57:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234120AbhLUC5J (ORCPT ); Mon, 20 Dec 2021 21:57:09 -0500 Received: from mailout3.samsung.com ([203.254.224.33]:47139 "EHLO mailout3.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234124AbhLUC5I (ORCPT ); Mon, 20 Dec 2021 21:57:08 -0500 Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20211221025704epoutp031b7a25968917777c93c1ec7c2e7bc436~CpbgUwbZV2946629466epoutp03b for ; Tue, 21 Dec 2021 02:57:04 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20211221025704epoutp031b7a25968917777c93c1ec7c2e7bc436~CpbgUwbZV2946629466epoutp03b DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1640055424; bh=5/JeJfIydkPK+f5DVvzf7D85rmek1HpmQwDhHQuwAww=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=e3ulblZ4nW+TtDoj/USdpslunGTVcqD/FhJrJ2+qKdxYluGNw10cPMQmyrvrMMPk1 CXhPoB7GvXVjQ0CydNQSRoSF/Yb76AKSlI4kAnYskgUqDZVN42YJLOCf4f+75G4Fqa q4MGmgS6e9YiPytYXs9MVJUcxQK23YvhBUH2Denc= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20211221025703epcas5p3f754819ad6043991093990f2eaaeec67~Cpbfw6_tP2323723237epcas5p3K; Tue, 21 Dec 2021 02:57:03 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.182]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4JJ1MG10D3z4x9Pp; Tue, 21 Dec 2021 02:56:58 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 89.76.06423.67241C16; Tue, 21 Dec 2021 11:56:54 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20211220142248epcas5p1e5904e10396f8cdea54bbd8d7aeca9a6~CfI8YXvcb2842328423epcas5p1v; Mon, 20 Dec 2021 14:22:48 +0000 (GMT) Received: from epsmgms1p2.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20211220142248epsmtrp23a7411e80d9989d0aa94a47322086fd0~CfI8XkkyS1266312663epsmtrp28; Mon, 20 Dec 2021 14:22:48 +0000 (GMT) X-AuditID: b6c32a49-b01ff70000001917-b7-61c1427689c5 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p2.samsung.com (Symantec Messaging Gateway) with SMTP id 57.65.08738.8B190C16; Mon, 20 Dec 2021 23:22:48 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20211220142246epsmtip1002c69610ffbf8c65bb9b40f9d65a84f~CfI63Hcf80040400404epsmtip1j; Mon, 20 Dec 2021 14:22:46 +0000 (GMT) From: Kanchan Joshi To: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, javier@javigon.com, anuj20.g@samsung.com, joshiiitr@gmail.com, pankydev8@gmail.com Subject: [RFC 10/13] block: factor out helper for bio allocation from cache Date: Mon, 20 Dec 2021 19:47:31 +0530 Message-Id: <20211220141734.12206-11-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211220141734.12206-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrJJsWRmVeSWpSXmKPExsWy7bCmhm6Z08FEg4VT9SyaJvxltlh9t5/N YuXqo0wW71rPsVh0nr7AZHH+7WEmi0mHrjFa7L2lbTF/2VN2izU3n7I4cHnsnHWX3aN5wR0W j8tnSz02repk89i8pN5j980GNo++LasYPT5vkgvgiMq2yUhNTEktUkjNS85PycxLt1XyDo53 jjc1MzDUNbS0MFdSyEvMTbVVcvEJ0HXLzAG6UEmhLDGnFCgUkFhcrKRvZ1OUX1qSqpCRX1xi q5RakJJTYFKgV5yYW1yal66Xl1piZWhgYGQKVJiQnfFh/jmWgpOSFdt/PWdpYLwt0sXIySEh YCKx/u0sti5GLg4hgd2MEjsb7rJCOJ8YJfav/coO4XxmlLh3+AU7TMvK138ZQWwhgV1AiStl cEVbdu8GSnBwsAloSlyYXApSIyIQLXHh+TU2EJtZoANoRbctiC0s4C1xv+M0E0g5i4CqROcd YZAwr4ClxMV5v9ggVslLzLz0HWwtJ1D88OxlbBA1ghInZz5hgRgpL9G8dTYzyAkSAhM5JLZv f8gKMlNCwEXizI4kiDnCEq+Ob4E6X0ri87u9UPOLJX7dOQrVC3Ta9YaZLBAJe4mLe/6C3cYM 9Mr6XfoQYVmJqafWMUHs5ZPo/f2ECSLOK7FjHoytKHFv0lNWCFtc4uGMJVC2h8SB5U+g4dnD KDHnwXW2CYwKs5D8MwvJP7MQVi9gZF7FKJlaUJybnlpsWmCYl1oOj+Pk/NxNjOA0q+W5g/Hu gw96hxiZOBgPMUpwMCuJ8G6ZvT9RiDclsbIqtSg/vqg0J7X4EKMpMLwnMkuJJucDE31eSbyh iaWBiZmZmYmlsZmhkjjv6fQNiUIC6YklqdmpqQWpRTB9TBycUg1MSqwvrzyxa3p+L+Nab5Hj 2de7X7O5TtflrW+p5Nxj+fTkWfnIqSqiyguPbz1/51PHAi/Vnrv/++qS1WTuF7syLjvYkOGT fed1pYVR6Le02TvVfTss+HNMCjeHbK+/KzHzuJKneX9aQlqgjerfzXOs/748804qjOXOklnJ wXOjmwIKLhQk1xSGLPVhD+y8OG3Dh+nts9jr3oQFt62oPnTj/dt9S6dt3aaelGFYENA2+QbD NcVE9ZiSjtaVhUdv21W8ybXVf99RuqLp7qK4NXq/hKeZbc45myG/Nes7n2vBmd/nGZS0Dr98 4RK3+Ip+w2fOK2bHkq6+L4psM1qk4z+jimHy1DXzF/gkSbBxTq1TYinOSDTUYi4qTgQAuP9x FjwEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrILMWRmVeSWpSXmKPExsWy7bCSnO6OiQcSDQ4u57FomvCX2WL13X42 i5WrjzJZvGs9x2LRefoCk8X5t4eZLCYdusZosfeWtsX8ZU/ZLdbcfMriwOWxc9Zddo/mBXdY PC6fLfXYtKqTzWPzknqP3Tcb2Dz6tqxi9Pi8SS6AI4rLJiU1J7MstUjfLoEr48P8cywFJyUr tv96ztLAeFuki5GTQ0LARGLl67+MXYxcHEICOxglLm69zgyREJdovvaDHcIWllj57zk7RNFH RomDXxqBOjg42AQ0JS5MLgWpERGIlfjw6xgTSA2zwCRGiQ39D8CahQW8Je53nGYCqWcRUJXo vCMMEuYVsJS4OO8XG8R8eYmZl76DlXMCxQ/PXgYWFxKwkDjx4QsLRL2gxMmZT8BsZqD65q2z mScwCsxCkpqFJLWAkWkVo2RqQXFuem6xYYFRXmq5XnFibnFpXrpecn7uJkZwLGhp7WDcs+qD 3iFGJg7GQ4wSHMxKIrxbZu9PFOJNSaysSi3Kjy8qzUktPsQozcGiJM57oetkvJBAemJJanZq akFqEUyWiYNTqoFJY/fNU5ctNkf/W7r6qtK5yxYZqQuMRHiTP61VvTTr/NHuq6au0/a31JS7 tLQe8Y1mCHg7IyPXxJTJV9owam/LD742tV+rmE/Yfmb64yFcc3TeJDFugavnGB0dbs9Z9Xey RX7aydfSizYEbjd42ap+7OK3alZjkdQpRsocIQcXCBd+uHS02P6dCO+eL58XzWDPkLgRME28 c9Lt62ExztsVYsWSLcs+5/VFHrvRteGw4/VXb5cxfrto0PUu0WSj+H8WgZNznLPXT175oW7P th0bWNtYiuZ8j5OJ5T7458GHbzrr1mToSmbeqGU6YbOown/z9xtzl7/jClnz7dw5Me7grN2u zV+/PgnszFkZP+ntCSWW4oxEQy3mouJEAGwjypH0AgAA X-CMS-MailID: 20211220142248epcas5p1e5904e10396f8cdea54bbd8d7aeca9a6 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20211220142248epcas5p1e5904e10396f8cdea54bbd8d7aeca9a6 References: <20211220141734.12206-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Factor the code to pull out bio_from_cache helper which is not tied to kiocb. This is prep patch. Signed-off-by: Kanchan Joshi --- block/bio.c | 43 ++++++++++++++++++++++++++----------------- include/linux/bio.h | 1 + 2 files changed, 27 insertions(+), 17 deletions(-) diff --git a/block/bio.c b/block/bio.c index 6fadc977cd7f..46d0f278d3aa 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1682,27 +1682,12 @@ int bioset_init_from_src(struct bio_set *bs, struct bio_set *src) } EXPORT_SYMBOL(bioset_init_from_src); -/** - * bio_alloc_kiocb - Allocate a bio from bio_set based on kiocb - * @kiocb: kiocb describing the IO - * @nr_vecs: number of iovecs to pre-allocate - * @bs: bio_set to allocate from - * - * Description: - * Like @bio_alloc_bioset, but pass in the kiocb. The kiocb is only - * used to check if we should dip into the per-cpu bio_set allocation - * cache. The allocation uses GFP_KERNEL internally. On return, the - * bio is marked BIO_PERCPU_CACHEABLE, and the final put of the bio - * MUST be done from process context, not hard/soft IRQ. - * - */ -struct bio *bio_alloc_kiocb(struct kiocb *kiocb, unsigned short nr_vecs, - struct bio_set *bs) +struct bio *bio_from_cache(unsigned short nr_vecs, struct bio_set *bs) { struct bio_alloc_cache *cache; struct bio *bio; - if (!(kiocb->ki_flags & IOCB_ALLOC_CACHE) || nr_vecs > BIO_INLINE_VECS) + if (nr_vecs > BIO_INLINE_VECS) return bio_alloc_bioset(GFP_KERNEL, nr_vecs, bs); cache = per_cpu_ptr(bs->cache, get_cpu()); @@ -1721,6 +1706,30 @@ struct bio *bio_alloc_kiocb(struct kiocb *kiocb, unsigned short nr_vecs, bio_set_flag(bio, BIO_PERCPU_CACHE); return bio; } +EXPORT_SYMBOL_GPL(bio_from_cache); + +/** + * bio_alloc_kiocb - Allocate a bio from bio_set based on kiocb + * @kiocb: kiocb describing the IO + * @nr_vecs: number of iovecs to pre-allocate + * @bs: bio_set to allocate from + * + * Description: + * Like @bio_alloc_bioset, but pass in the kiocb. The kiocb is only + * used to check if we should dip into the per-cpu bio_set allocation + * cache. The allocation uses GFP_KERNEL internally. On return, the + * bio is marked BIO_PERCPU_CACHEABLE, and the final put of the bio + * MUST be done from process context, not hard/soft IRQ. + * + */ +struct bio *bio_alloc_kiocb(struct kiocb *kiocb, unsigned short nr_vecs, + struct bio_set *bs) +{ + if (!(kiocb->ki_flags & IOCB_ALLOC_CACHE)) + return bio_alloc_bioset(GFP_KERNEL, nr_vecs, bs); + + return bio_from_cache(nr_vecs, bs); +} EXPORT_SYMBOL_GPL(bio_alloc_kiocb); static int __init init_bio(void) diff --git a/include/linux/bio.h b/include/linux/bio.h index fe6bdfbbef66..77eceb2bda4b 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -358,6 +358,7 @@ struct bio *bio_alloc_bioset(gfp_t gfp, unsigned short nr_iovecs, struct bio_set *bs); struct bio *bio_alloc_kiocb(struct kiocb *kiocb, unsigned short nr_vecs, struct bio_set *bs); +struct bio *bio_from_cache(unsigned short nr_vecs, struct bio_set *bs); struct bio *bio_kmalloc(gfp_t gfp_mask, unsigned short nr_iovecs); extern void bio_put(struct bio *); From patchwork Mon Dec 20 14:17:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12688957 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CCD7C433F5 for ; Tue, 21 Dec 2021 02:57:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234121AbhLUC5I (ORCPT ); Mon, 20 Dec 2021 21:57:08 -0500 Received: from mailout4.samsung.com ([203.254.224.34]:15096 "EHLO mailout4.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234120AbhLUC5H (ORCPT ); Mon, 20 Dec 2021 21:57:07 -0500 Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20211221025706epoutp04d048f3ba333370c7f4de1d42c1f5fc6f~Cpbh4fINw1811318113epoutp04e for ; Tue, 21 Dec 2021 02:57:06 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20211221025706epoutp04d048f3ba333370c7f4de1d42c1f5fc6f~Cpbh4fINw1811318113epoutp04e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1640055426; bh=SWDMGTcULDyUUGJ03Gd9mGp0+xbdwZhQc470VcrJToQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WwsZYNtRqSN1e3qi7IZ7BE4Ua9mQb/QTBRdpCu962IkHeUOQi00hYdq6upyHxLUf9 6GMIlNMGe01Y08Ru0m084l7cP2jpdqXlu7LEhGYXr2HtdHfrz0w24JudZx0RJdbTfQ gneRXN4A/8UkSZvlx/gnOshWrGvG0lt/K7O9q9fo= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20211221025705epcas5p22b6929f51df31440c1b220fef405690a~CpbhSk1sY1195011950epcas5p2C; Tue, 21 Dec 2021 02:57:05 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.175]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4JJ1MK5GNCz4x9Pt; Tue, 21 Dec 2021 02:57:01 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 53.86.06423.C7241C16; Tue, 21 Dec 2021 11:57:00 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20211220142250epcas5p34b9d93b1dd3388af6209a4223befe40f~CfI_DoopR2729127291epcas5p3U; Mon, 20 Dec 2021 14:22:50 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20211220142250epsmtrp1113649aa5e250daf85cb0c63572a168d~CfI_CyWgt2445924459epsmtrp1k; Mon, 20 Dec 2021 14:22:50 +0000 (GMT) X-AuditID: b6c32a49-b13ff70000001917-d6-61c1427c8508 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 68.85.29871.9B190C16; Mon, 20 Dec 2021 23:22:49 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20211220142248epsmtip179e780c785e738f6bd68b52ce146b0ee~CfI8cl9-X0040100401epsmtip1k; Mon, 20 Dec 2021 14:22:48 +0000 (GMT) From: Kanchan Joshi To: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, javier@javigon.com, anuj20.g@samsung.com, joshiiitr@gmail.com, pankydev8@gmail.com Subject: [RFC 11/13] nvme: enable bio-cache for fixed-buffer passthru Date: Mon, 20 Dec 2021 19:47:32 +0530 Message-Id: <20211220141734.12206-12-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211220141734.12206-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrFJsWRmVeSWpSXmKPExsWy7bCmpm6N08FEg6Mf+CyaJvxltlh9t5/N YuXqo0wW71rPsVh0nr7AZHH+7WEmi0mHrjFa7L2lbTF/2VN2izU3n7I4cHnsnHWX3aN5wR0W j8tnSz02repk89i8pN5j980GNo++LasYPT5vkgvgiMq2yUhNTEktUkjNS85PycxLt1XyDo53 jjc1MzDUNbS0MFdSyEvMTbVVcvEJ0HXLzAG6UEmhLDGnFCgUkFhcrKRvZ1OUX1qSqpCRX1xi q5RakJJTYFKgV5yYW1yal66Xl1piZWhgYGQKVJiQnfFvjXHBNNWK2XdeMDYw7pLrYuTkkBAw kTgx6xkjiC0ksJtR4uXJvC5GLiD7E6NEU9suZgjnG6PE02nr2WE6mvqXskMk9jJKXHq5lhXC +cwo8X7/RKBZHBxsApoSFyaXgjSICERLXHh+jQ3EZhboYJTY2W0LYgsLuErMeredGcRmEVCV 2P3kHtgZvAKWEif/vGeDWCYvMfPSd7DFnEDxw7OXsUHUCEqcnPmEBWKmvETz1tlgl0oI9HJI 3FrfCXWpi8SkU9uYIWxhiVfHt0DFpSRe9rdB2cUSv+4chWoGOu56w0wWiIS9xMU9f5lAnmEG emb9Ln2IsKzE1FPrmCAW80n0/n7CBBHnldgxD8ZWlLg36SkrhC0u8XDGEijbQ2L13LnQkOth lNiw4QT7BEaFWUgemoXkoVkIqxcwMq9ilEwtKM5NTy02LTDMSy2HR3Jyfu4mRnCi1fLcwXj3 wQe9Q4xMHIyHGCU4mJVEeLfM3p8oxJuSWFmVWpQfX1Sak1p8iNEUGOITmaVEk/OBqT6vJN7Q xNLAxMzMzMTS2MxQSZz3dPqGRCGB9MSS1OzU1ILUIpg+Jg5OqQam4omSV+vvfr3qKJy0bFVj 6n5ulg+fF216b5QbcvzK3xmeB/jd1v7cZNgUHnMyvNym18/wodGu7nfl79ebfcwrqLgzwa+d 9d63Bs/OKV/esoWdlt2T1e7gGLci2H754nNLfzS9t0uOst+8LSfTr3jSH7/Lezg/H/FyMrDj Xu2fpBHAUfRy+Z/qk+mGF/66fNx3k3Xzui4jy3P8u2QF7CxZkk4qZXLf9upnSFhbL/vBbuYG h2SzXJtX6Qszw3YlVC7Ucrgkl7Ds9Ke5Gz8sOn/m2oWtOQ9VaxYr8NeqHz4dOjt6/fPcdgN9 of03ufZcP3cyMmSfJc961uuvdq/aVvjSe/HhvddVX+RbJzy6EPaHRYmlOCPRUIu5qDgRACPm GE49BAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrILMWRmVeSWpSXmKPExsWy7bCSnO7OiQcSDR7ttLZomvCX2WL13X42 i5WrjzJZvGs9x2LRefoCk8X5t4eZLCYdusZosfeWtsX8ZU/ZLdbcfMriwOWxc9Zddo/mBXdY PC6fLfXYtKqTzWPzknqP3Tcb2Dz6tqxi9Pi8SS6AI4rLJiU1J7MstUjfLoEr498a44JpqhWz 77xgbGDcJdfFyMkhIWAi0dS/lL2LkYtDSGA3o8TtpufMEAlxieZrP9ghbGGJlf+eQxV9ZJTY OHEmUBEHB5uApsSFyaUgNSICsRIffh1jAqlhFpjEKLGh/wFYs7CAq8Ssd9vBhrIIqErsfnKP EcTmFbCUOPnnPRvEAnmJmZe+g9VzAsUPz14GFhcSsJA48eELC0S9oMTJmU/AbGag+uats5kn MArMQpKahSS1gJFpFaNkakFxbnpusWGBYV5quV5xYm5xaV66XnJ+7iZGcCxoae5g3L7qg94h RiYOxkOMEhzMSiK8W2bvTxTiTUmsrEotyo8vKs1JLT7EKM3BoiTOe6HrZLyQQHpiSWp2ampB ahFMlomDU6qBifd1QvDbnX1Xbet8jtT/knmV+MzENN6S8XKmdgTvg/JPHpWXiyTSIo47mO8s WHiC1W1zlGuKhOfGOTf2hxQaB99efPTAu2UinQunrt1Rtye7ZXf1pNAz8xlSil983FyddL35 wsvdH/onySounxl0Yw+He/vNkBc1emJFhqtueYt7FR+4ve2zz+X9W12uhtpx233Z+lR2rsAC ruqGhlZt9csfM3p2FNYyvV6aPs9tTbqpV6v3+37fskDVranlXl/Pmudni0XMPnD37xHOnJzz zS8jH7htflE/ycJd8oeE4L1HGZatUuyn90+9lWVZv27SpNYXS7PnP7LxObjr6LWwp0XcnWJs uopRO469KfsnrMRSnJFoqMVcVJwIAMB0OKD0AgAA X-CMS-MailID: 20211220142250epcas5p34b9d93b1dd3388af6209a4223befe40f X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20211220142250epcas5p34b9d93b1dd3388af6209a4223befe40f References: <20211220141734.12206-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Since we do submission/completion in task, we can have this up. Add a bio-set for nvme as we need that for bio-cache. Signed-off-by: Kanchan Joshi Signed-off-by: Anuj Gupta --- block/blk-map.c | 4 ++-- drivers/nvme/host/core.c | 9 +++++++++ drivers/nvme/host/ioctl.c | 6 ++++-- drivers/nvme/host/nvme.h | 1 + include/linux/blk-mq.h | 2 +- 5 files changed, 17 insertions(+), 5 deletions(-) diff --git a/block/blk-map.c b/block/blk-map.c index 9aa9864eab55..e3e28b628fba 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -580,7 +580,7 @@ EXPORT_SYMBOL(blk_rq_map_user); /* Unlike blk_rq_map_user () this is only for fixed-buffer async passthrough. */ int blk_rq_map_user_fixedb(struct request_queue *q, struct request *rq, - u64 ubuf, unsigned long len, gfp_t gfp_mask, + u64 ubuf, unsigned long len, struct bio_set *bs, struct io_uring_cmd *ioucmd) { struct iov_iter iter; @@ -604,7 +604,7 @@ int blk_rq_map_user_fixedb(struct request_queue *q, struct request *rq, if (nr_segs > queue_max_segments(q)) return -EINVAL; /* no iovecs to alloc, as we already have a BVEC iterator */ - bio = bio_alloc(gfp_mask, 0); + bio = bio_from_cache(0, bs); if (!bio) return -ENOMEM; diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index bce2e93d14a3..0c231946a310 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -30,6 +30,9 @@ #define NVME_MINORS (1U << MINORBITS) +#define NVME_BIO_POOL_SZ (4) +struct bio_set nvme_bio_pool; + unsigned int admin_timeout = 60; module_param(admin_timeout, uint, 0644); MODULE_PARM_DESC(admin_timeout, "timeout in seconds for admin commands"); @@ -4793,6 +4796,11 @@ static int __init nvme_core_init(void) goto unregister_generic_ns; } + result = bioset_init(&nvme_bio_pool, NVME_BIO_POOL_SZ, 0, + BIOSET_NEED_BVECS | BIOSET_PERCPU_CACHE); + if (result < 0) + goto unregister_generic_ns; + return 0; unregister_generic_ns: @@ -4815,6 +4823,7 @@ static int __init nvme_core_init(void) static void __exit nvme_core_exit(void) { + bioset_exit(&nvme_bio_pool); class_destroy(nvme_ns_chr_class); class_destroy(nvme_subsys_class); class_destroy(nvme_class); diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index dc6a5f1b81ca..013ff9baa78e 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -43,6 +43,7 @@ static void nvme_pt_task_cb(struct io_uring_cmd *ioucmd) struct request *req = cmd->req; int status; u64 result; + struct bio *bio = req->bio; if (nvme_req(req)->flags & NVME_REQ_CANCELLED) status = -EINTR; @@ -52,6 +53,7 @@ static void nvme_pt_task_cb(struct io_uring_cmd *ioucmd) /* we can free request */ blk_mq_free_request(req); + blk_rq_unmap_user(bio); if (cmd->meta) { if (status) @@ -73,9 +75,9 @@ static void nvme_end_async_pt(struct request *req, blk_status_t err) struct bio *bio = cmd->bio; cmd->req = req; + req->bio = bio; /* this takes care of setting up task-work */ io_uring_cmd_complete_in_task(ioucmd, nvme_pt_task_cb); - blk_rq_unmap_user(bio); } static void nvme_setup_uring_cmd_data(struct request *rq, @@ -164,7 +166,7 @@ static int nvme_submit_user_cmd(struct request_queue *q, bufflen, GFP_KERNEL); else ret = blk_rq_map_user_fixedb(q, req, ubuffer, bufflen, - GFP_KERNEL, ioucmd); + &nvme_bio_pool, ioucmd); if (ret) goto out; bio = req->bio; diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 9a901b954a87..6bbb8ed868eb 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -47,6 +47,7 @@ extern unsigned int admin_timeout; extern struct workqueue_struct *nvme_wq; extern struct workqueue_struct *nvme_reset_wq; extern struct workqueue_struct *nvme_delete_wq; +extern struct bio_set nvme_bio_pool; /* * List of workarounds for devices that required behavior not specified in diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index a82b054eebde..e35a5d835b1f 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -923,7 +923,7 @@ struct rq_map_data { int blk_rq_map_user(struct request_queue *, struct request *, struct rq_map_data *, void __user *, unsigned long, gfp_t); int blk_rq_map_user_fixedb(struct request_queue *, struct request *, - u64 ubuf, unsigned long, gfp_t, + u64 ubuf, unsigned long, struct bio_set *, struct io_uring_cmd *); int blk_rq_map_user_iov(struct request_queue *, struct request *, struct rq_map_data *, const struct iov_iter *, gfp_t); From patchwork Mon Dec 20 14:17:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12688961 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14C98C433FE for ; Tue, 21 Dec 2021 02:57:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234181AbhLUC5M (ORCPT ); Mon, 20 Dec 2021 21:57:12 -0500 Received: from mailout2.samsung.com ([203.254.224.25]:43491 "EHLO mailout2.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234140AbhLUC5M (ORCPT ); Mon, 20 Dec 2021 21:57:12 -0500 Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20211221025710epoutp02ba67b7ed77eaa30f073bee22c9e0d90e~Cpbl_81_n2861028610epoutp02l for ; Tue, 21 Dec 2021 02:57:10 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20211221025710epoutp02ba67b7ed77eaa30f073bee22c9e0d90e~Cpbl_81_n2861028610epoutp02l DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1640055430; bh=mMtCI2RCEK+ZUNR58tFFC6bnhpSD5e53npox+XXYqX0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tTeu8nhYutWZCR4FdMqbK1JOxNtqftqD11u136+8At+IRf2VIDpeqvCgmpkEWr/En lCxrj87jfWzZSFRVA3Qx5yDdhMPXZP4DUDNi6PCGOBhcvdd0QPu1e0dwNrQNAjMQOP 8b9lCnlZVORvI6B2A9m5Fvwhh7Nm4B+ZGHVP9yNk= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p4.samsung.com (KnoxPortal) with ESMTP id 20211221025709epcas5p4974cf1ae578b63f8dd9cba6308f36ceb~CpblQ75NL0577005770epcas5p4N; Tue, 21 Dec 2021 02:57:09 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.180]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4JJ1MP73Jpz4x9Q0; Tue, 21 Dec 2021 02:57:05 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 33.A9.05590.08241C16; Tue, 21 Dec 2021 11:57:04 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20211220142252epcas5p4611297f9970acbc8ee3b0e325ca5ceec~CfJAA85bE2261322613epcas5p4z; Mon, 20 Dec 2021 14:22:52 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20211220142252epsmtrp1a0f228186fc3afef9aa91b10473b3e52~CfJAALb8B2445924459epsmtrp1m; Mon, 20 Dec 2021 14:22:52 +0000 (GMT) X-AuditID: b6c32a4b-723ff700000015d6-83-61c142804a57 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 49.85.29871.BB190C16; Mon, 20 Dec 2021 23:22:52 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20211220142250epsmtip143802eba4e3ea9f7728b4258c67d9eb5~CfI_JcFof0637906379epsmtip1r; Mon, 20 Dec 2021 14:22:50 +0000 (GMT) From: Kanchan Joshi To: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, javier@javigon.com, anuj20.g@samsung.com, joshiiitr@gmail.com, pankydev8@gmail.com Subject: [RFC 12/13] nvme: allow user passthrough commands to poll Date: Mon, 20 Dec 2021 19:47:33 +0530 Message-Id: <20211220141734.12206-13-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211220141734.12206-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrFJsWRmVeSWpSXmKPExsWy7bCmhm6D08FEg9PHpC2aJvxltlh9t5/N YuXqo0wW71rPsVh0nr7AZHH+7WEmi0mHrjFa7L2lbTF/2VN2izU3n7I4cHnsnHWX3aN5wR0W j8tnSz02repk89i8pN5j980GNo++LasYPT5vkgvgiMq2yUhNTEktUkjNS85PycxLt1XyDo53 jjc1MzDUNbS0MFdSyEvMTbVVcvEJ0HXLzAG6UEmhLDGnFCgUkFhcrKRvZ1OUX1qSqpCRX1xi q5RakJJTYFKgV5yYW1yal66Xl1piZWhgYGQKVJiQnXFl9R3GgufuFdNeT2FuYNxt3cXIySEh YCKxZ/YZpi5GLg4hgd2MEt/Wr2OBcD4xSpy7f5sZwvnGKLH96BYWmJazXX1QVXsZJb7P2scI 4XxmlHjdeIy9i5GDg01AU+LC5FKQBhGBaIkLz6+xgdjMAh2MEju7bUFsYQEniQ1908GGsgio ShxZcpURxOYVsJS4ua+dGWKZvMTMS9/ZQWxOoPjh2cvYIGoEJU7OfMICMVNeonnrbLBLJQR6 OSTaX7QyQjS7SPxpXs0KYQtLvDq+hR3ClpJ42d8GZRdL/LpzFKoZ6LjrDTOh3rSXuLjnLxPI M8xAz6zfpQ8RlpWYemodE8RiPone30+YIOK8EjvmwdiKEvcmPYXaKy7xcMYSVpAxEgIeEo+7 eEDCQgI9wHC4LjKBUWEWkndmIXlnFsLiBYzMqxglUwuKc9NTi00LjPNSy+GRnJyfu4kRnGi1 vHcwPnrwQe8QIxMH4yFGCQ5mJRHeLbP3JwrxpiRWVqUW5ccXleakFh9iNAWG90RmKdHkfGCq zyuJNzSxNDAxMzMzsTQ2M1QS5z2VviFRSCA9sSQ1OzW1ILUIpo+Jg1OqgWndiUNPH6qaeE/i ZRTlkLS36RDIsF30+Ha5e/YspajrWpnLfRikk+dU+7MKWlZaZPa8frxNKKmHZ6XjpG6Zzyec pDpaTPUvTp26+HufgJbA/S2tDw9pcWyOZjT7ts585ce3P1s2p9annmnPKbxrZGqXsGbTlvar uwQfez9NUw1856d0cMpc2dsz1i/qWXawMd2tPFptquP7WbMkmaoVwx59WtPdXNClGbpVsMhq l35EwhaDTN9PpSF/3FZ1exT6fQuf5iAQ+Ng07unb0FsnuSO+cy1w/xXtczHK7+GOk1bdy39P vnJz1sJw45pVMU38E2oy7VSyL/30YK09eDPui+pLgaPaxseP/ZjtxDT1vxJLcUaioRZzUXEi AN0E3l89BAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrILMWRmVeSWpSXmKPExsWy7bCSnO6eiQcSDeZNZLJomvCX2WL13X42 i5WrjzJZvGs9x2LRefoCk8X5t4eZLCYdusZosfeWtsX8ZU/ZLdbcfMriwOWxc9Zddo/mBXdY PC6fLfXYtKqTzWPzknqP3Tcb2Dz6tqxi9Pi8SS6AI4rLJiU1J7MstUjfLoEr48rqO4wFz90r pr2ewtzAuNu6i5GTQ0LAROJsVx8LiC0ksJtRYuV+Zoi4uETztR/sELawxMp/z4FsLqCaj4wS cx+0MXUxcnCwCWhKXJhcClIjIhAr8eHXMSaQGmaBSYwSG/ofgDULCzhJbOibDraARUBV4siS q4wgNq+ApcTNfe1Qy+QlZl76DlbPCRQ/PHsZG8RBFhInPnxhgagXlDg58wmYzQxU37x1NvME RoFZSFKzkKQWMDKtYpRMLSjOTc8tNiwwzEst1ytOzC0uzUvXS87P3cQIjgUtzR2M21d90DvE yMTBeIhRgoNZSYR3y+z9iUK8KYmVValF+fFFpTmpxYcYpTlYlMR5L3SdjBcSSE8sSc1OTS1I LYLJMnFwSjUwufOFhua27n25dSpX6YuFQZu/HT91vqZZfpJ5kKn7hmmCuds6G774XV8VJJCq usJg/v0/L5cv5mOKj/0/9cZejtyijM0dcjYHOSLXPxb82Rqj7D4lZsoG9inPVcu6rrUU3Vew 6TyjVj/NpvCE+Db50hnt/5Pu3GT+beUuafLd+brG30l/WeUYis+c/P/19fOCI3IBV7wNvD4y LF/fHKaiMzlZRm7pdOGMx4lxc5SaGtXe8LYpH8mpS9P8vXL2j88r177Qab3R9mJncWlTqfnZ xM7fnEdOT/Y0jNwq+lDm05rY92dKNHKZan6HLynbs/ZLvtY/1quilk+PNSVl6zP06D44MHWj Dk9B/9w714U3K7EUZyQaajEXFScCADhDwzL0AgAA X-CMS-MailID: 20211220142252epcas5p4611297f9970acbc8ee3b0e325ca5ceec X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20211220142252epcas5p4611297f9970acbc8ee3b0e325ca5ceec References: <20211220141734.12206-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Keith Busch The block layer knows how to deal with polled requests. Let the NVMe driver use the previously reserved user "flags" fields to define an option to allocate the request from the polled hardware contexts. If polling is not enabled, then the block layer will automatically fallback to a non-polled request.[1] [1] https://lore.kernel.org/linux-block/20210517171443.GB2709391@dhcp-10-100-145-180.wdc.com/ Signed-off-by: Keith Busch --- drivers/nvme/host/core.c | 10 ++++++---- drivers/nvme/host/ioctl.c | 33 +++++++++++++++++++-------------- drivers/nvme/host/nvme.h | 3 ++- drivers/nvme/host/pci.c | 4 ++-- drivers/nvme/target/passthru.c | 2 +- include/uapi/linux/nvme_ioctl.h | 4 ++++ 6 files changed, 34 insertions(+), 22 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 0c231946a310..5199adf7ae92 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -633,11 +633,13 @@ static inline void nvme_init_request(struct request *req, } struct request *nvme_alloc_request(struct request_queue *q, - struct nvme_command *cmd, blk_mq_req_flags_t flags) + struct nvme_command *cmd, blk_mq_req_flags_t flags, + unsigned int rq_flags) { + unsigned int cmd_flags = nvme_req_op(cmd) | rq_flags; struct request *req; - req = blk_mq_alloc_request(q, nvme_req_op(cmd), flags); + req = blk_mq_alloc_request(q, cmd_flags, flags); if (!IS_ERR(req)) nvme_init_request(req, cmd); return req; @@ -1081,7 +1083,7 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd, int ret; if (qid == NVME_QID_ANY) - req = nvme_alloc_request(q, cmd, flags); + req = nvme_alloc_request(q, cmd, flags, 0); else req = nvme_alloc_request_qid(q, cmd, flags, qid); if (IS_ERR(req)) @@ -1277,7 +1279,7 @@ static void nvme_keep_alive_work(struct work_struct *work) } rq = nvme_alloc_request(ctrl->admin_q, &ctrl->ka_cmd, - BLK_MQ_REQ_RESERVED | BLK_MQ_REQ_NOWAIT); + BLK_MQ_REQ_RESERVED | BLK_MQ_REQ_NOWAIT, 0); if (IS_ERR(rq)) { /* allocation failure, reset the controller */ dev_err(ctrl->device, "keep-alive failed: %ld\n", PTR_ERR(rq)); diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index 013ff9baa78e..bdaf8f317aa8 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -142,7 +142,7 @@ static int nvme_submit_user_cmd(struct request_queue *q, struct nvme_command *cmd, u64 ubuffer, unsigned bufflen, void __user *meta_buffer, unsigned meta_len, u32 meta_seed, u64 *result, unsigned timeout, - struct io_uring_cmd *ioucmd) + struct io_uring_cmd *ioucmd, unsigned int rq_flags) { bool write = nvme_is_write(cmd); struct nvme_ns *ns = q->queuedata; @@ -152,7 +152,7 @@ static int nvme_submit_user_cmd(struct request_queue *q, void *meta = NULL; int ret; - req = nvme_alloc_request(q, cmd, 0); + req = nvme_alloc_request(q, cmd, 0, rq_flags); if (IS_ERR(req)) return PTR_ERR(req); @@ -212,11 +212,13 @@ static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio) struct nvme_command c; unsigned length, meta_len; void __user *metadata; + unsigned int rq_flags = 0; if (copy_from_user(&io, uio, sizeof(io))) return -EFAULT; - if (io.flags) - return -EINVAL; + + if (io.flags & NVME_HIPRI) + rq_flags |= REQ_POLLED; switch (io.opcode) { case nvme_cmd_write: @@ -254,7 +256,7 @@ static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio) memset(&c, 0, sizeof(c)); c.rw.opcode = io.opcode; - c.rw.flags = io.flags; + c.rw.flags = 0; c.rw.nsid = cpu_to_le32(ns->head->ns_id); c.rw.slba = cpu_to_le64(io.slba); c.rw.length = cpu_to_le16(io.nblocks); @@ -266,7 +268,7 @@ static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio) return nvme_submit_user_cmd(ns->queue, &c, io.addr, length, metadata, meta_len, - lower_32_bits(io.slba), NULL, 0, NULL); + lower_32_bits(io.slba), NULL, 0, NULL, rq_flags); } static bool nvme_validate_passthru_nsid(struct nvme_ctrl *ctrl, @@ -288,6 +290,7 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns, { struct nvme_passthru_cmd cmd; struct nvme_command c; + unsigned int rq_flags = 0; unsigned timeout = 0; u64 result; int status; @@ -296,14 +299,14 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns, return -EACCES; if (copy_from_user(&cmd, ucmd, sizeof(cmd))) return -EFAULT; - if (cmd.flags) - return -EINVAL; + if (cmd.flags & NVME_HIPRI) + rq_flags |= REQ_POLLED; if (!nvme_validate_passthru_nsid(ctrl, ns, cmd.nsid)) return -EINVAL; memset(&c, 0, sizeof(c)); c.common.opcode = cmd.opcode; - c.common.flags = cmd.flags; + c.common.flags = 0; c.common.nsid = cpu_to_le32(cmd.nsid); c.common.cdw2[0] = cpu_to_le32(cmd.cdw2); c.common.cdw2[1] = cpu_to_le32(cmd.cdw3); @@ -319,7 +322,7 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns, status = nvme_submit_user_cmd(ns ? ns->queue : ctrl->admin_q, &c, cmd.addr, cmd.data_len, nvme_to_user_ptr(cmd.metadata), - cmd.metadata_len, 0, &result, timeout, NULL); + cmd.metadata_len, 0, &result, timeout, NULL, rq_flags); if (status >= 0) { if (put_user(result, &ucmd->result)) @@ -335,6 +338,7 @@ static int nvme_user_cmd64(struct nvme_ctrl *ctrl, struct nvme_ns *ns, { struct nvme_passthru_cmd64 cmd; struct nvme_command c; + unsigned int rq_flags = 0; unsigned timeout = 0; int status; @@ -342,14 +346,15 @@ static int nvme_user_cmd64(struct nvme_ctrl *ctrl, struct nvme_ns *ns, return -EACCES; if (copy_from_user(&cmd, ucmd, sizeof(cmd))) return -EFAULT; - if (cmd.flags) - return -EINVAL; + if (cmd.flags & NVME_HIPRI) + rq_flags |= REQ_POLLED; + if (!nvme_validate_passthru_nsid(ctrl, ns, cmd.nsid)) return -EINVAL; memset(&c, 0, sizeof(c)); c.common.opcode = cmd.opcode; - c.common.flags = cmd.flags; + c.common.flags = 0; c.common.nsid = cpu_to_le32(cmd.nsid); c.common.cdw2[0] = cpu_to_le32(cmd.cdw2); c.common.cdw2[1] = cpu_to_le32(cmd.cdw3); @@ -365,7 +370,7 @@ static int nvme_user_cmd64(struct nvme_ctrl *ctrl, struct nvme_ns *ns, status = nvme_submit_user_cmd(ns ? ns->queue : ctrl->admin_q, &c, cmd.addr, cmd.data_len, nvme_to_user_ptr(cmd.metadata), - cmd.metadata_len, 0, &cmd.result, timeout, ioucmd); + cmd.metadata_len, 0, &cmd.result, timeout, ioucmd, rq_flags); if (!ioucmd && status >= 0) { if (put_user(cmd.result, &ucmd->result)) diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 6bbb8ed868eb..56a7cc8421fc 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -696,7 +696,8 @@ void nvme_start_freeze(struct nvme_ctrl *ctrl); #define NVME_QID_ANY -1 struct request *nvme_alloc_request(struct request_queue *q, - struct nvme_command *cmd, blk_mq_req_flags_t flags); + struct nvme_command *cmd, blk_mq_req_flags_t flags, + unsigned int rq_flags); void nvme_cleanup_cmd(struct request *req); blk_status_t nvme_setup_cmd(struct nvme_ns *ns, struct request *req); blk_status_t nvme_fail_nonready_command(struct nvme_ctrl *ctrl, diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 50deb8b69c40..3d013a88af9d 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -1429,7 +1429,7 @@ static enum blk_eh_timer_return nvme_timeout(struct request *req, bool reserved) req->tag, nvmeq->qid); abort_req = nvme_alloc_request(dev->ctrl.admin_q, &cmd, - BLK_MQ_REQ_NOWAIT); + BLK_MQ_REQ_NOWAIT, 0); if (IS_ERR(abort_req)) { atomic_inc(&dev->ctrl.abort_limit); return BLK_EH_RESET_TIMER; @@ -2475,7 +2475,7 @@ static int nvme_delete_queue(struct nvme_queue *nvmeq, u8 opcode) cmd.delete_queue.opcode = opcode; cmd.delete_queue.qid = cpu_to_le16(nvmeq->qid); - req = nvme_alloc_request(q, &cmd, BLK_MQ_REQ_NOWAIT); + req = nvme_alloc_request(q, &cmd, BLK_MQ_REQ_NOWAIT, 0); if (IS_ERR(req)) return PTR_ERR(req); diff --git a/drivers/nvme/target/passthru.c b/drivers/nvme/target/passthru.c index 9e5b89ae29df..2a9e2fd3b137 100644 --- a/drivers/nvme/target/passthru.c +++ b/drivers/nvme/target/passthru.c @@ -253,7 +253,7 @@ static void nvmet_passthru_execute_cmd(struct nvmet_req *req) timeout = nvmet_req_subsys(req)->admin_timeout; } - rq = nvme_alloc_request(q, req->cmd, 0); + rq = nvme_alloc_request(q, req->cmd, 0, 0); if (IS_ERR(rq)) { status = NVME_SC_INTERNAL; goto out_put_ns; diff --git a/include/uapi/linux/nvme_ioctl.h b/include/uapi/linux/nvme_ioctl.h index d99b5a772698..df2c138c38d9 100644 --- a/include/uapi/linux/nvme_ioctl.h +++ b/include/uapi/linux/nvme_ioctl.h @@ -9,6 +9,10 @@ #include +enum nvme_io_flags { + NVME_HIPRI = 1 << 0, /* use polling queue if available */ +}; + struct nvme_user_io { __u8 opcode; __u8 flags; From patchwork Mon Dec 20 14:17:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12688963 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38A29C433FE for ; Tue, 21 Dec 2021 02:57:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234187AbhLUC5U (ORCPT ); Mon, 20 Dec 2021 21:57:20 -0500 Received: from mailout1.samsung.com ([203.254.224.24]:13612 "EHLO mailout1.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234126AbhLUC5U (ORCPT ); Mon, 20 Dec 2021 21:57:20 -0500 Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20211221025718epoutp010a962b7f406c0446bd5d84d94ff51b03~CpbtiQge_2321523215epoutp01J for ; Tue, 21 Dec 2021 02:57:18 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20211221025718epoutp010a962b7f406c0446bd5d84d94ff51b03~CpbtiQge_2321523215epoutp01J DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1640055438; bh=iaD7YXlIXUvfbDK5g8qCuc/nFndLZ6rb6BPryMOSuAo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IxS1Qa6tLleObWW6dvFlFbxk+fANwcDk4suicD8uKkXqXiBZk8ahYL4Kd1pOyw4Sm VgkD3TtBdWC4oiTfrcZUSbKdjAMIrezb0Ot6QLdMTHZIlpmlu0tx86tyA017O98AVP hpXC59zCTqABeobrsGioElC1Vh7807+tTU/3B50A= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20211221025718epcas5p2137b561b71e9c09be0487c3788d26ba7~CpbtDdnMO1195011950epcas5p2b; Tue, 21 Dec 2021 02:57:18 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.174]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4JJ1MT2xR1z4x9Q3; Tue, 21 Dec 2021 02:57:09 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 1B.86.06423.58241C16; Tue, 21 Dec 2021 11:57:09 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20211220142256epcas5p49e0804ff8b075e8063259f94ccc9ced0~CfJEBW6Wz0645306453epcas5p47; Mon, 20 Dec 2021 14:22:56 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20211220142256epsmtrp1e0dc23ef6074dbf3b6e113f618108991~CfJEAn5082445924459epsmtrp1n; Mon, 20 Dec 2021 14:22:56 +0000 (GMT) X-AuditID: b6c32a49-b13ff70000001917-f4-61c142850431 Received: from epsmtip1.samsung.com ( [182.195.34.30]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id F9.85.29871.0C190C16; Mon, 20 Dec 2021 23:22:56 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20211220142252epsmtip13724171da317beeb7bbdc9819b527bf6~CfJAGtPet0040400404epsmtip1k; Mon, 20 Dec 2021 14:22:52 +0000 (GMT) From: Kanchan Joshi To: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, javier@javigon.com, anuj20.g@samsung.com, joshiiitr@gmail.com, pankydev8@gmail.com Subject: [RFC 13/13] nvme: Add async passthru polling support Date: Mon, 20 Dec 2021 19:47:34 +0530 Message-Id: <20211220141734.12206-14-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211220141734.12206-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrJJsWRmVeSWpSXmKPExsWy7bCmum6r08FEg7f/+S2aJvxltlh9t5/N YuXqo0wW71rPsVh0nr7AZHH+7WEmi0mHrjFa7L2lbTF/2VN2izU3n7I4cHnsnHWX3aN5wR0W j8tnSz02repk89i8pN5j980GNo++LasYPT5vkgvgiMq2yUhNTEktUkjNS85PycxLt1XyDo53 jjc1MzDUNbS0MFdSyEvMTbVVcvEJ0HXLzAG6UEmhLDGnFCgUkFhcrKRvZ1OUX1qSqpCRX1xi q5RakJJTYFKgV5yYW1yal66Xl1piZWhgYGQKVJiQnXF5c03B5fCKS4vfMTUwLnHvYuTkkBAw kXh55gdTFyMXh5DAbkaJH+emsEA4nxglLuy8yQJSJSTwjVHi7Zs6mI6Hu66wQxTtZZTo+/ES qv0zo8T2ztusXYwcHGwCmhIXJpeCNIgIREtceH6NDcRmFuhglNjZbQtiCwvYSsybP5cJxGYR UJX4fGwLI4jNK2ApcbfhNyvEMnmJmZe+s4PYnEDxw7OXsUHUCEqcnPmEBWKmvETz1tnMIDdI CEzlkHi1+gYLyA0SAi4Spx7oQcwRlnh1fAs7hC0l8bK/Dcoulvh15yhUL9Bt1xtmskAk7CUu 7vnLBDKHGeiX9bv0IcKyElNPrWOC2Msn0fv7CRNEnFdixzwYW1Hi3qSnUPeLSzycsQTK9pA4 t7UVGro9jBKXb3czT2BUmIXkn1lI/pmFsHoBI/MqRsnUguLc9NRi0wLDvNRyeBwn5+duYgSn WS3PHYx3H3zQO8TIxMF4iFGCg1lJhHfL7P2JQrwpiZVVqUX58UWlOanFhxhNgQE+kVlKNDkf mOjzSuINTSwNTMzMzEwsjc0MlcR5T6dvSBQSSE8sSc1OTS1ILYLpY+LglGpg4staY9Hcztt3 bkF2Zmv46eOS4jq/n30TKFE6K5iy+ncw5+3pG1p6ft88vSJr4wTf2k2qf8z2Gjz96vffXKV3 LatJAf+hhxnHNdi1HS/PtYvae0zx2uHrbzoi1lv3SvU2JnnavZy5uUeYfV+hhW5g8f2d/ooX qjNdvfTk3//u4Eu72cXiXt8ZZJ3mE3V9afTcoM0ZvD6P53w7E6CVUpBhpHAuja1VZrPUhq69 7yZG810tnsi0Pasxzu7T+19psWk//bjumZRfWXg3eXG41uTZ206Jn5slc+596LrwKwkrGd/8 4A2SO/Sz6J3vRssZS30rb9+6avGm4Hxf+tJasw7FT59mn2i2SjB9ZHH5HsdvJZbijERDLeai 4kQAvuxsGTwEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrILMWRmVeSWpSXmKPExsWy7bCSnO6BiQcSDXYckbBomvCX2WL13X42 i5WrjzJZvGs9x2LRefoCk8X5t4eZLCYdusZosfeWtsX8ZU/ZLdbcfMriwOWxc9Zddo/mBXdY PC6fLfXYtKqTzWPzknqP3Tcb2Dz6tqxi9Pi8SS6AI4rLJiU1J7MstUjfLoEr4/LmmoLL4RWX Fr9jamBc4t7FyMkhIWAi8XDXFfYuRi4OIYHdjBJ7LkxlhkiISzRf+8EOYQtLrPz3HKroI6PE jM/vGbsYOTjYBDQlLkwuBakREYiV+PDrGBNIDbPAJEaJDf0PwJqFBWwl5s2fywRiswioSnw+ toURxOYVsJS42/CbFWKBvMTMS9/B6jmB4odnL2MDsYUELCROfPjCAlEvKHFy5hMwmxmovnnr bOYJjAKzkKRmIUktYGRaxSiZWlCcm55bbFhgmJdarlecmFtcmpeul5yfu4kRHAtamjsYt6/6 oHeIkYmD8RCjBAezkgjvltn7E4V4UxIrq1KL8uOLSnNSiw8xSnOwKInzXug6GS8kkJ5Ykpqd mlqQWgSTZeLglGpgijp7/vuH4x2izj0ztV6bnProIrRv/9fT95ZfYojwP/fV+2/+10zDys1L elfECb7Y8S4p+QXjGsHl+aoLzl1j+ZfHw3qrul7IPsh5+YnTa5+J37qWyHi8I2ux0q+W3U8D nHZPuFmru8LAuy1Qc7eA7sRvcYfLjkZOu+h4Mmdr/QyLdedX6N5tYI7y8szOLVbMl8iKfqyz bP5U2aUzStXXf55y78kW330HVp6Um9lWoer1Xrb0+qqd0hYH1P5ce7N4Ssbic+ejYoxcrRJn Zdox/rRZ/nX3Aj5Vtr68pKZku4OLH2+K3rDd6Ej1a8f2VeHsh22F981NWe94OjNsBde/1XcC GNuTY5P+cz99Wv6P44wSS3FGoqEWc1FxIgBDf7Ek9AIAAA== X-CMS-MailID: 20211220142256epcas5p49e0804ff8b075e8063259f94ccc9ced0 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20211220142256epcas5p49e0804ff8b075e8063259f94ccc9ced0 References: <20211220141734.12206-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Pankaj Raghav IO_URING already has polling support for read and write. This patch extends that support for uring cmd passthu. The unused flag in uring_cmd struct is used to indicate if the completion should be polled. If device side polling is not enabled, then the submission request will fallback to a non-polled request. Signed-off-by: Pankaj Raghav --- block/blk-mq.c | 3 +- drivers/nvme/host/core.c | 1 + drivers/nvme/host/ioctl.c | 79 ++++++++++++++++++++++++++++++++++- drivers/nvme/host/multipath.c | 1 + drivers/nvme/host/nvme.h | 4 ++ fs/io_uring.c | 45 ++++++++++++++++++-- include/linux/blk-mq.h | 1 + include/linux/io_uring.h | 10 ++++- 8 files changed, 135 insertions(+), 9 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index c77991688bfd..acfa55c96a43 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1193,7 +1193,7 @@ void blk_execute_rq_nowait(struct request *rq, bool at_head, rq_end_io_fn *done) } EXPORT_SYMBOL_GPL(blk_execute_rq_nowait); -static bool blk_rq_is_poll(struct request *rq) +bool blk_rq_is_poll(struct request *rq) { if (!rq->mq_hctx) return false; @@ -1203,6 +1203,7 @@ static bool blk_rq_is_poll(struct request *rq) return false; return true; } +EXPORT_SYMBOL_GPL(blk_rq_is_poll); static void blk_rq_poll_completion(struct request *rq, struct completion *wait) { diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 5199adf7ae92..f0697cbe2bf1 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -3676,6 +3676,7 @@ static const struct file_operations nvme_ns_chr_fops = { .unlocked_ioctl = nvme_ns_chr_ioctl, .compat_ioctl = compat_ptr_ioctl, .async_cmd = nvme_ns_chr_async_cmd, + .iopoll = nvme_iopoll, }; static int nvme_add_ns_cdev(struct nvme_ns *ns) diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index bdaf8f317aa8..ce2fe94df3ad 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -31,6 +31,12 @@ struct nvme_uring_cmd { void __user *meta_buffer; }; +static inline bool is_polling_enabled(struct io_uring_cmd *ioucmd, + struct request *req) +{ + return (ioucmd->flags & URING_CMD_POLLED) && blk_rq_is_poll(req); +} + static struct nvme_uring_cmd *nvme_uring_cmd(struct io_uring_cmd *ioucmd) { return (struct nvme_uring_cmd *)&ioucmd->pdu; @@ -76,8 +82,16 @@ static void nvme_end_async_pt(struct request *req, blk_status_t err) cmd->req = req; req->bio = bio; - /* this takes care of setting up task-work */ - io_uring_cmd_complete_in_task(ioucmd, nvme_pt_task_cb); + + /*IO can be completed immediately when the callback + * is in the same task context + */ + if (is_polling_enabled(ioucmd, req)) { + nvme_pt_task_cb(ioucmd); + } else { + /* this takes care of setting up task-work */ + io_uring_cmd_complete_in_task(ioucmd, nvme_pt_task_cb); + } } static void nvme_setup_uring_cmd_data(struct request *rq, @@ -183,6 +197,12 @@ static int nvme_submit_user_cmd(struct request_queue *q, } } if (ioucmd) { /* async dispatch */ + + if (bio && is_polling_enabled(ioucmd, req)) { + ioucmd->bio = bio; + bio->bi_opf |= REQ_POLLED; + } + nvme_setup_uring_cmd_data(req, ioucmd, meta, meta_buffer, meta_len, write); blk_execute_rq_nowait(req, 0, nvme_end_async_pt); @@ -496,6 +516,32 @@ int nvme_ns_chr_async_cmd(struct io_uring_cmd *ioucmd, return nvme_ns_async_ioctl(ns, ioucmd); } +int nvme_iopoll(struct kiocb *kiocb, struct io_comp_batch *iob, + unsigned int flags) +{ + struct bio *bio = NULL; + struct nvme_ns *ns = NULL; + struct request_queue *q = NULL; + int ret = 0; + + rcu_read_lock(); + bio = READ_ONCE(kiocb->private); + ns = container_of(file_inode(kiocb->ki_filp)->i_cdev, struct nvme_ns, + cdev); + q = ns->queue; + + /* bio and driver_cb are a part of the same union type in io_uring_cmd + * struct. When there are no poll queues, driver_cb is used for IRQ cb + * but polling is performed from the io_uring side. To avoid unnecessary + * polling, a check is added to see if it is a polled queue and return 0 + * if it is not. + */ + if ((test_bit(QUEUE_FLAG_POLL, &q->queue_flags)) && bio && bio->bi_bdev) + ret = bio_poll(bio, iob, flags); + rcu_read_unlock(); + return ret; +} + #ifdef CONFIG_NVME_MULTIPATH static int nvme_ns_head_ctrl_ioctl(struct nvme_ns *ns, unsigned int cmd, void __user *argp, struct nvme_ns_head *head, int srcu_idx) @@ -577,6 +623,35 @@ long nvme_ns_head_chr_ioctl(struct file *file, unsigned int cmd, srcu_read_unlock(&head->srcu, srcu_idx); return ret; } + +int nvme_ns_head_iopoll(struct kiocb *kiocb, struct io_comp_batch *iob, + unsigned int flags) +{ + struct bio *bio = NULL; + struct request_queue *q = NULL; + struct cdev *cdev = file_inode(kiocb->ki_filp)->i_cdev; + struct nvme_ns_head *head = container_of(cdev, struct nvme_ns_head, cdev); + int srcu_idx = srcu_read_lock(&head->srcu); + struct nvme_ns *ns = nvme_find_path(head); + int ret = -EWOULDBLOCK; + + if (ns) { + bio = READ_ONCE(kiocb->private); + q = ns->queue; + /* bio and driver_cb are a part of the same union type in io_uring_cmd + * struct. When there are no poll queues, driver_cb is used for IRQ cb + * but polling is performed from the io_uring side. To avoid unnecessary + * polling, a check is added to see if it is a polled queue and return 0 + * if it is not. + */ + if ((test_bit(QUEUE_FLAG_POLL, &q->queue_flags)) && bio && + bio->bi_bdev) + ret = bio_poll(bio, iob, flags); + } + + srcu_read_unlock(&head->srcu, srcu_idx); + return ret; +} #endif /* CONFIG_NVME_MULTIPATH */ static int nvme_dev_user_cmd(struct nvme_ctrl *ctrl, void __user *argp) diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index 1e59c8e06622..df91b2953932 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -424,6 +424,7 @@ static const struct file_operations nvme_ns_head_chr_fops = { .unlocked_ioctl = nvme_ns_head_chr_ioctl, .compat_ioctl = compat_ptr_ioctl, .async_cmd = nvme_ns_head_chr_async_cmd, + .iopoll = nvme_ns_head_iopoll, }; static int nvme_add_ns_head_cdev(struct nvme_ns_head *head) diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 56a7cc8421fc..730ada8a3e8e 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -752,8 +752,12 @@ long nvme_dev_ioctl(struct file *file, unsigned int cmd, unsigned long arg); int nvme_ns_chr_async_cmd(struct io_uring_cmd *ucmd, enum io_uring_cmd_flags flags); +int nvme_iopoll(struct kiocb *kiocb, struct io_comp_batch *iob, + unsigned int flags); int nvme_ns_head_chr_async_cmd(struct io_uring_cmd *ucmd, enum io_uring_cmd_flags flags); +int nvme_ns_head_iopoll(struct kiocb *kiocb, struct io_comp_batch *iob, + unsigned int flags); int nvme_getgeo(struct block_device *bdev, struct hd_geometry *geo); extern const struct attribute_group *nvme_ns_id_attr_groups[]; diff --git a/fs/io_uring.c b/fs/io_uring.c index f77dde1bdc75..ae2e7666622e 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -2655,7 +2655,20 @@ static int io_do_iopoll(struct io_ring_ctx *ctx, bool force_nonspin) if (READ_ONCE(req->iopoll_completed)) break; - ret = kiocb->ki_filp->f_op->iopoll(kiocb, &iob, poll_flags); + if (req->opcode == IORING_OP_URING_CMD || + req->opcode == IORING_OP_URING_CMD_FIXED) { + /* uring_cmd structure does not contain kiocb struct */ + struct kiocb kiocb_uring_cmd; + + kiocb_uring_cmd.private = req->uring_cmd.bio; + kiocb_uring_cmd.ki_filp = req->uring_cmd.file; + ret = req->uring_cmd.file->f_op->iopoll(&kiocb_uring_cmd, + &iob, poll_flags); + } else { + ret = kiocb->ki_filp->f_op->iopoll(kiocb, &iob, + poll_flags); + } + if (unlikely(ret < 0)) return ret; else if (ret) @@ -2768,6 +2781,15 @@ static int io_iopoll_check(struct io_ring_ctx *ctx, long min) wq_list_empty(&ctx->iopoll_list)) break; } + + /* + * In some scenarios, completion callback has been queued up to be + * completed in-task context but polling happens in the same task + * not giving a chance for the completion callback to complete. + */ + if (current->task_works) + io_run_task_work(); + ret = io_do_iopoll(ctx, !min); if (ret < 0) break; @@ -4122,6 +4144,14 @@ static int io_linkat(struct io_kiocb *req, unsigned int issue_flags) return 0; } +static void io_complete_uring_cmd_iopoll(struct io_kiocb *req, long res) +{ + WRITE_ONCE(req->result, res); + /* order with io_iopoll_complete() checking ->result */ + smp_wmb(); + WRITE_ONCE(req->iopoll_completed, 1); +} + /* * Called by consumers of io_uring_cmd, if they originally returned * -EIOCBQUEUED upon receiving the command. @@ -4132,7 +4162,11 @@ void io_uring_cmd_done(struct io_uring_cmd *cmd, ssize_t ret) if (ret < 0) req_set_fail(req); - io_req_complete(req, ret); + + if (req->uring_cmd.flags & URING_CMD_POLLED) + io_complete_uring_cmd_iopoll(req, ret); + else + io_req_complete(req, ret); } EXPORT_SYMBOL_GPL(io_uring_cmd_done); @@ -4147,8 +4181,11 @@ static int io_uring_cmd_prep(struct io_kiocb *req, return -EOPNOTSUPP; if (req->ctx->flags & IORING_SETUP_IOPOLL) { - printk_once(KERN_WARNING "io_uring: iopoll not supported!\n"); - return -EOPNOTSUPP; + req->uring_cmd.flags = URING_CMD_POLLED; + req->uring_cmd.bio = NULL; + req->iopoll_completed = 0; + } else { + req->uring_cmd.flags = 0; } cmd->op = READ_ONCE(csqe->op); diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index e35a5d835b1f..2233ccf41c19 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -933,6 +933,7 @@ int blk_rq_map_kern(struct request_queue *, struct request *, void *, int blk_rq_append_bio(struct request *rq, struct bio *bio); void blk_execute_rq_nowait(struct request *rq, bool at_head, rq_end_io_fn *end_io); +bool blk_rq_is_poll(struct request *rq); blk_status_t blk_execute_rq(struct request *rq, bool at_head); struct req_iterator { diff --git a/include/linux/io_uring.h b/include/linux/io_uring.h index 07732bc850af..bbc9c4ea19c3 100644 --- a/include/linux/io_uring.h +++ b/include/linux/io_uring.h @@ -6,6 +6,7 @@ #include enum { + URING_CMD_POLLED = (1 << 0), URING_CMD_FIXEDBUFS = (1 << 1), }; /* @@ -17,8 +18,13 @@ struct io_uring_cmd { __u16 op; __u16 flags; __u32 len; - /* used if driver requires update in task context*/ - void (*driver_cb)(struct io_uring_cmd *cmd); + union { + void *bio; // Used for polling based completion + + /* used if driver requires update in task context for IRQ based completion*/ + void (*driver_cb)(struct io_uring_cmd *cmd); + }; + __u64 pdu[5]; /* 40 bytes available inline for free use */ };