From patchwork Tue Aug 23 16:14:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12952443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA3F4C3F6B0 for ; Tue, 23 Aug 2022 18:13:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233828AbiHWSND (ORCPT ); Tue, 23 Aug 2022 14:13:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232509AbiHWSMo (ORCPT ); Tue, 23 Aug 2022 14:12:44 -0400 Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C53EF9C2F1 for ; Tue, 23 Aug 2022 09:25:14 -0700 (PDT) Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20220823162513epoutp0320b1520e66b4e5c1f3c948ffa66b839b~OBgDdOAxN1550415504epoutp036 for ; Tue, 23 Aug 2022 16:25:13 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20220823162513epoutp0320b1520e66b4e5c1f3c948ffa66b839b~OBgDdOAxN1550415504epoutp036 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1661271913; bh=Vlg7RjwsSm838OJBc8ZvpdV0ePSO3S5EX14/Jkn77YA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VwRaGtrGd9mjEJqk8GJCaoRWG4V97ksKrzNEeag0WS3Cl363K8GFR/4QcwlHN7j5z pSLnh3IVytG8JhDIRw48EU8pQUN1jiT/gOw8ln0/QK2X0Zu55zKQirCHP9e3PlPEZw VKFKCJVKLUCqBuRwX9cLCAMCxAEbKPlXjRAG6d70= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20220823162512epcas5p2399704c8759e8de5eb9dba160180c223~OBgCuW6N_1154711547epcas5p2C; Tue, 23 Aug 2022 16:25:12 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.179]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4MBvhj1Bq7z4x9Pw; Tue, 23 Aug 2022 16:25:09 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 69.15.18001.56FF4036; Wed, 24 Aug 2022 01:25:09 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20220823162508epcas5p3ae39903d3ee1079134fb70ed675159fc~OBf-RSsx52674026740epcas5p37; Tue, 23 Aug 2022 16:25:08 +0000 (GMT) Received: from epsmgms1p2.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20220823162508epsmtrp1f5707276f5118361bd3228cdf2339b3c~OBf-QWshH2270522705epsmtrp1H; Tue, 23 Aug 2022 16:25:08 +0000 (GMT) X-AuditID: b6c32a4a-2c3ff70000004651-f0-6304ff65fc0d Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2.samsung.com (Symantec Messaging Gateway) with SMTP id 91.A2.18644.46FF4036; Wed, 24 Aug 2022 01:25:08 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220823162507epsmtip2cc532be7b3229f2cf757628788163b72~OBf93tghM3113431134epsmtip2V; Tue, 23 Aug 2022 16:25:07 +0000 (GMT) From: Kanchan Joshi To: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, asml.silence@gmail.com Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, ming.lei@redhat.com, gost.dev@samsung.com, Kanchan Joshi Subject: [PATCH for-next v3 1/4] fs: add file_operations->uring_cmd_iopoll Date: Tue, 23 Aug 2022 21:44:40 +0530 Message-Id: <20220823161443.49436-2-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220823161443.49436-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprHJsWRmVeSWpSXmKPExsWy7bCmhm7qf5Zkg6NP2SzmrNrGaLH6bj+b xc0DO5ksVq4+ymTxrvUci8XR/2/ZLCYdusZosfeWtsX8ZU/ZLQ5NbmZy4PLYOesuu8fls6Ue m1Z1snlsXlLvsftmA5vH+31X2Tz6tqxi9Pi8SS6AIyrbJiM1MSW1SCE1Lzk/JTMv3VbJOzje Od7UzMBQ19DSwlxJIS8xN9VWycUnQNctMwfoQiWFssScUqBQQGJxsZK+nU1RfmlJqkJGfnGJ rVJqQUpOgUmBXnFibnFpXrpeXmqJlaGBgZEpUGFCdsb/9ltsBXtYK15dPcHUwHiUpYuRk0NC wESiZXsHYxcjF4eQwG5GiVfXV7FCOJ8YJZ4sfc8M4XxmlDjzfSpcy+XN69ghErsYJf6v7mWH q7py6ypQPwcHm4CmxIXJpSANIgJeEvdvvwcbyyywllHi9N4vTCAJYaDEwcXLwWwWAVWJaTMu s4LYvAIWEv9eb2CE2CYvMfPSd3aQmZwClhKnD8pBlAhKnJz5BOwgZqCS5q2zwS6VEJjIIdHf sp8ZotdF4vnhuWwQtrDEq+Nb2CFsKYmX/W1QdrLEpZnnmCDsEonHew5C2fYSraf6mUH2MgP9 sn6XPsQuPone30+YQMISArwSHW1CENWKEvcmPWWFsMUlHs5YAmV7SFy5sQcaoj2MEg/X9LNO YJSfheSFWUhemIWwbQEj8ypGydSC4tz01GLTAqO81HJ4xCbn525iBCdULa8djA8ffNA7xMjE wXiIUYKDWUmE1+oYS7IQb0piZVVqUX58UWlOavEhRlNgEE9klhJNzgem9LySeEMTSwMTMzMz E0tjM0Mlcd4p2ozJQgLpiSWp2ampBalFMH1MHJxSDUzTDnyPfDhn8a6/D/Keifz8N7uryyfp Q6bAQZGWjXYWxRd5Jxd0v7YMOPy97oDJctO/W/XM5K4oryiOljO4Yl/lGDtBWfS8pHDE7/W5 v5e6sEyet+bHqTauSENxx7nREj27FgUoPZK9JSb7mXnthCK+9D/zvVdrr3X3mHn9xQztvj23 fqpPr41X7ZM8oLJ3f0PFh5SrTHx+n7RzGxLecNedUGX3DHx+6tXvr3HTalWuLVJImF7GPbNv 0zyv1+rJLja+0yMzyyYYBTx5N6N81rvUj9P8VE/vzLnInLCSO1Dl6rnpkzk2CcZ8mic4y6ey ZaHiGw7W/YaX3CYy2VbyBdZkZfKdW/v/2LlZYqc/Cv9QYinOSDTUYi4qTgQAducbzTEEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrILMWRmVeSWpSXmKPExsWy7bCSvG7Kf5Zkg6ZGPYs5q7YxWqy+289m cfPATiaLlauPMlm8az3HYnH0/1s2i0mHrjFa7L2lbTF/2VN2i0OTm5kcuDx2zrrL7nH5bKnH plWdbB6bl9R77L7ZwObxft9VNo++LasYPT5vkgvgiOKySUnNySxLLdK3S+DK+N9+i61gD2vF q6snmBoYj7J0MXJySAiYSFzevI69i5GLQ0hgB6PEk427mCAS4hLN136wQ9jCEiv/PYcq+sgo cfv+J+YuRg4ONgFNiQuTS0FqRAQCJA42XgarYRbYzCjx6fQxZpCEsICXxMHFy8GGsgioSkyb cZkVxOYVsJD493oDI8QCeYmZl76zg8zkFLCUOH1QDiQsBFTyZ1MDG0S5oMTJmU/AjmYGKm/e Opt5AqPALCSpWUhSCxiZVjFKphYU56bnFhsWGOWllusVJ+YWl+al6yXn525iBMeCltYOxj2r PugdYmTiYDzEKMHBrCTCa3WMJVmINyWxsiq1KD++qDQntfgQozQHi5I474Wuk/FCAumJJanZ qakFqUUwWSYOTqkGpviVd4KK04ujNJeZsgctl188/1tqALN9v8rcsBkqZ1SWsp9s5RO3DJ2w tt1mv+ShrhXzJi8NenZS/NP7EnnXcCGr27cex15lar3Bf69XTLp6EYPr2v/xQfddp7YzTfrp HrqKk1t4ic7pDzmfuU3+vvqwdU/q7CivzDUfLt1ztbB1vfT/3YNzR813N+VsznmSmlS8uYEr 96+NHFNDLRN/Bl/lXdP9+6++y2jsCY7+byCou2TtjNiT55dv3q6z/JJ5hbCRto/c4mtCqwX6 zko8zNR6vcNAIeFgJN9VdqYclukxa2pubtmU0elXoKN1Mros4fqq4PO5Blmx+4rLvh9YUnb3 CKesddWUYzHyLNteK7EUZyQaajEXFScCAO91dEb0AgAA X-CMS-MailID: 20220823162508epcas5p3ae39903d3ee1079134fb70ed675159fc X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220823162508epcas5p3ae39903d3ee1079134fb70ed675159fc References: <20220823161443.49436-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org io_uring will invoke this to do completion polling on uring-cmd operations. Signed-off-by: Kanchan Joshi --- include/linux/fs.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/linux/fs.h b/include/linux/fs.h index 9eced4cc286e..d6badd19784f 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2132,6 +2132,7 @@ struct file_operations { loff_t len, unsigned int remap_flags); int (*fadvise)(struct file *, loff_t, loff_t, int); int (*uring_cmd)(struct io_uring_cmd *ioucmd, unsigned int issue_flags); + int (*uring_cmd_iopoll)(struct io_uring_cmd *ioucmd); } __randomize_layout; struct inode_operations { From patchwork Tue Aug 23 16:14:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12952444 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C07FDC32772 for ; Tue, 23 Aug 2022 18:13:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230451AbiHWSNG (ORCPT ); Tue, 23 Aug 2022 14:13:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232593AbiHWSMq (ORCPT ); Tue, 23 Aug 2022 14:12:46 -0400 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0BC509C2FF for ; Tue, 23 Aug 2022 09:25:18 -0700 (PDT) Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20220823162515epoutp026dbb570d92bcfed3198b24e35306a89f~OBgFmUVj52693526935epoutp02s for ; Tue, 23 Aug 2022 16:25:15 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20220823162515epoutp026dbb570d92bcfed3198b24e35306a89f~OBgFmUVj52693526935epoutp02s DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1661271915; bh=7uxpW2z8PgV6QmCh80XCC5JE20bOhAeRfZs6FnbP8ws=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CVdMOmlv10ByaXWOAPzBVCZN89aO+dSCyENBbRFe52cCoU/3HTkHdaoRfMWkQYaAW LUfNbgZ37CNZgyUFcJ7kFsH5RtVz8l1h5a+gnZ4//fRh3PJ3MlzCNjmPz+pVol6TZC /TYep2mm3pha9OWXtVA0Um+w1svoz1Q7V31/3Tvk= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p4.samsung.com (KnoxPortal) with ESMTP id 20220823162514epcas5p4bab52d83513af7624c72983dc6483d77~OBgEwqAQt1443914439epcas5p4D; Tue, 23 Aug 2022 16:25:14 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.182]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4MBvhm1wwMz4x9Pv; Tue, 23 Aug 2022 16:25:12 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 8A.15.18001.86FF4036; Wed, 24 Aug 2022 01:25:12 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20220823162511epcas5p46fc0e384524f0a386651bc694ff21976~OBgCFk0bl1443914439epcas5p4C; Tue, 23 Aug 2022 16:25:11 +0000 (GMT) Received: from epsmgms1p2.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20220823162511epsmtrp2348517518c30dc9194b2c3270f5341d1~OBgCEx5n20268302683epsmtrp2q; Tue, 23 Aug 2022 16:25:11 +0000 (GMT) X-AuditID: b6c32a4a-2c3ff70000004651-f8-6304ff68a585 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2.samsung.com (Symantec Messaging Gateway) with SMTP id E3.A2.18644.76FF4036; Wed, 24 Aug 2022 01:25:11 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220823162510epsmtip2860888bab5114fedd5a902771318d48f~OBgAdulrt3114031140epsmtip2a; Tue, 23 Aug 2022 16:25:09 +0000 (GMT) From: Kanchan Joshi To: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, asml.silence@gmail.com Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, ming.lei@redhat.com, gost.dev@samsung.com, Kanchan Joshi , Pankaj Raghav Subject: [PATCH for-next v3 2/4] io_uring: add iopoll infrastructure for io_uring_cmd Date: Tue, 23 Aug 2022 21:44:41 +0530 Message-Id: <20220823161443.49436-3-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220823161443.49436-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrAJsWRmVeSWpSXmKPExsWy7bCmum7Gf5Zkgy+LuCzmrNrGaLH6bj+b xc0DO5ksVq4+ymTxrvUci8XR/2/ZLCYdusZosfeWtsX8ZU/ZLQ5Nbmay+Ly0hd2B22PnrLvs HpfPlnpsWtXJ5rF5Sb3H7psNbB7v911l8+jbsorR4/MmuQCOqGybjNTElNQihdS85PyUzLx0 WyXv4HjneFMzA0NdQ0sLcyWFvMTcVFslF58AXbfMHKAzlRTKEnNKgUIBicXFSvp2NkX5pSWp Chn5xSW2SqkFKTkFJgV6xYm5xaV56Xp5qSVWhgYGRqZAhQnZGcunrWUtuChX8fXQFZYGxh8S XYwcHBICJhLft6p3MXJxCAnsZpSYemgJI4TziVFi3qN3TBDOZ0aJmUtPsnUxcoJ1PH5zhRki sYtR4tKCg2xwVc+fT2UHmcsmoClxYXIpSIOIgJfE/dvvWUFqmAUuM0q0dU5hBkkIC4RLbJ17 jwnEZhFQlfh/ajoriM0rYCFxfmEvK8Q2eYmZl76DzeQUsJQ4fVAOokRQ4uTMJywgNjNQSfPW 2WAHSQjM5JBYc7+fDeI3F4kV/yMhxghLvDq+hR3ClpL4/G4v1DPJEpdmnmOCsEskHu85CGXb S7Se6mcGGcMM9Mr6XfoQq/gken8/YYKYzivR0SYEUa0ocW/SU6iDxSUezljCClHiIfH2mz8k cHoYJVafnsQ0gVF+FpIHZiF5YBbCsgWMzKsYJVMLinPTU4tNC4zyUsvhsZqcn7uJEZxUtbx2 MD588EHvECMTB+MhRgkOZiURXqtjLMlCvCmJlVWpRfnxRaU5qcWHGE2BATyRWUo0OR+Y1vNK 4g1NLA1MzMzMTCyNzQyVxHmnaDMmCwmkJ5akZqemFqQWwfQxcXBKNTDpf2LT2srKZ/p30hyj TdVvhBPnuDat4Tkmei90yY9AJiWFP7w19yIOH4zNyJA3E/iiy7FiU2lW236fsg7lN0K8RlOf H5DRu/Q0nPl0Wc/bT6mvZ7CZGPws/cF4e+3JG+dU21e+V0x1DQjzFVSQfhij3j/T6pbqxNtr /A6s3Xn56roL+vmRJQIhfWsec1U/6NmTPtfwauf3XxOm8ha4XLp4tcjSUNgw26WNh33l2aUR Vzi+2t4KbC5Y7P2S6cFXy0/C8bvW3lOPWC/gcfPInDcnN7wwzmxpL523POO7yMMXaxdt3rs9 LPvowxQv/3nvLx9+lPY7X6HCoj3ePSZFfOKOjpxSe+XsKVkJX7XF9vAqsRRnJBpqMRcVJwIA appf+jMEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrKLMWRmVeSWpSXmKPExsWy7bCSvG76f5Zkg9V7tCzmrNrGaLH6bj+b xc0DO5ksVq4+ymTxrvUci8XR/2/ZLCYdusZosfeWtsX8ZU/ZLQ5Nbmay+Ly0hd2B22PnrLvs HpfPlnpsWtXJ5rF5Sb3H7psNbB7v911l8+jbsorR4/MmuQCOKC6blNSczLLUIn27BK6M5dPW shZclKv4eugKSwPjD4kuRk4OCQETicdvrjCD2EICOxglPnzjgoiLSzRf+8EOYQtLrPz3HMjm Aqr5yCjxb/9qpi5GDg42AU2JC5NLQWpEBAIkDjZeBqthFrjNKDF3+ikWkISwQKjE7KZnTCA2 i4CqxP9T01lBbF4BC4nzC3tZIRbIS8y89J0dZCangKXE6YNyEPdYSPzZ1MAGUS4ocXLmE7CR zEDlzVtnM09gFJiFJDULSWoBI9MqRsnUguLc9NxiwwKjvNRyveLE3OLSvHS95PzcTYzgmNDS 2sG4Z9UHvUOMTByMhxglOJiVRHitjrEkC/GmJFZWpRblxxeV5qQWH2KU5mBREue90HUyXkgg PbEkNTs1tSC1CCbLxMEp1cB0wKH7kfrx9AqPfI1lRx5/DDNmnBn/e2qZqOejyEWKha59tU+u hK5LEfutefXCksX3Vf47b3ENO+c+7WSJdfx7T65O7msH75Umfd97jKtGjsn6cVu2TXLa89eG j3Sr2v8cml5dZdAuIlKsnz2TZ728tscq/u9dnC4r1JI0Zt2fZphgkdceq/s59dzqlzua+l9t 3LKIrTzbQyS2w8Fw+0rBpd9Wiaq0ZN4zuzXDZoP5U6en2vFHDZ9O4r3+zDDTvnNFZHbx17Rd bg/m7yy1MGby6Njy4WN47tLLz4Mrnu3YHP9gl06+0N5f51Kc/l/I5HnfuF5OtvWKp/2/w5On HVmQKnpMZOtZLZv+DWdyjIuUWIozEg21mIuKEwGN2Eyq+AIAAA== X-CMS-MailID: 20220823162511epcas5p46fc0e384524f0a386651bc694ff21976 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220823162511epcas5p46fc0e384524f0a386651bc694ff21976 References: <20220823161443.49436-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Put this up in the same way as iopoll is done for regular read/write IO. Make place for storing a cookie into struct io_uring_cmd on submission. Perform the completion using the ->uring_cmd_iopoll handler. Signed-off-by: Kanchan Joshi Signed-off-by: Pankaj Raghav --- include/linux/io_uring.h | 8 ++++++-- io_uring/io_uring.c | 6 ++++++ io_uring/opdef.c | 1 + io_uring/rw.c | 8 +++++++- io_uring/uring_cmd.c | 11 +++++++++-- 5 files changed, 29 insertions(+), 5 deletions(-) diff --git a/include/linux/io_uring.h b/include/linux/io_uring.h index 4a2f6cc5a492..58676c0a398f 100644 --- a/include/linux/io_uring.h +++ b/include/linux/io_uring.h @@ -20,8 +20,12 @@ enum io_uring_cmd_flags { struct io_uring_cmd { struct file *file; const void *cmd; - /* callback to defer completions to task context */ - void (*task_work_cb)(struct io_uring_cmd *cmd); + union { + /* callback to defer completions to task context */ + void (*task_work_cb)(struct io_uring_cmd *cmd); + /* used for polled completion */ + void *cookie; + }; u32 cmd_op; u32 pad; u8 pdu[32]; /* available inline for free use */ diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index ebfdb2212ec2..04abcc67648e 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -1296,6 +1296,12 @@ static int io_iopoll_check(struct io_ring_ctx *ctx, long min) wq_list_empty(&ctx->iopoll_list)) break; } + + if (task_work_pending(current)) { + mutex_unlock(&ctx->uring_lock); + io_run_task_work(); + mutex_lock(&ctx->uring_lock); + } ret = io_do_iopoll(ctx, !min); if (ret < 0) break; diff --git a/io_uring/opdef.c b/io_uring/opdef.c index 72dd2b2d8a9d..9a0df19306fe 100644 --- a/io_uring/opdef.c +++ b/io_uring/opdef.c @@ -466,6 +466,7 @@ const struct io_op_def io_op_defs[] = { .needs_file = 1, .plug = 1, .name = "URING_CMD", + .iopoll = 1, .async_size = uring_cmd_pdu_size(1), .prep = io_uring_cmd_prep, .issue = io_uring_cmd, diff --git a/io_uring/rw.c b/io_uring/rw.c index 1babd77da79c..9698a789b3d5 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -1005,7 +1005,13 @@ int io_do_iopoll(struct io_ring_ctx *ctx, bool force_nonspin) if (READ_ONCE(req->iopoll_completed)) break; - ret = rw->kiocb.ki_filp->f_op->iopoll(&rw->kiocb, &iob, poll_flags); + if (req->opcode == IORING_OP_URING_CMD) { + struct io_uring_cmd *ioucmd = (struct io_uring_cmd *)rw; + + ret = req->file->f_op->uring_cmd_iopoll(ioucmd); + } else + ret = rw->kiocb.ki_filp->f_op->iopoll(&rw->kiocb, &iob, + poll_flags); if (unlikely(ret < 0)) return ret; else if (ret) diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index 8e0cc2d9205e..b0e7feeed365 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -49,7 +49,11 @@ void io_uring_cmd_done(struct io_uring_cmd *ioucmd, ssize_t ret, ssize_t res2) io_req_set_res(req, ret, 0); if (req->ctx->flags & IORING_SETUP_CQE32) io_req_set_cqe32_extra(req, res2, 0); - __io_req_complete(req, 0); + if (req->ctx->flags & IORING_SETUP_IOPOLL) + /* order with io_iopoll_req_issued() checking ->iopoll_complete */ + smp_store_release(&req->iopoll_completed, 1); + else + __io_req_complete(req, 0); } EXPORT_SYMBOL_GPL(io_uring_cmd_done); @@ -92,8 +96,11 @@ int io_uring_cmd(struct io_kiocb *req, unsigned int issue_flags) issue_flags |= IO_URING_F_SQE128; if (ctx->flags & IORING_SETUP_CQE32) issue_flags |= IO_URING_F_CQE32; - if (ctx->flags & IORING_SETUP_IOPOLL) + if (ctx->flags & IORING_SETUP_IOPOLL) { issue_flags |= IO_URING_F_IOPOLL; + req->iopoll_completed = 0; + WRITE_ONCE(ioucmd->cookie, NULL); + } if (req_has_async_data(req)) ioucmd->cmd = req->async_data; From patchwork Tue Aug 23 16:14:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12952446 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 977CDC32789 for ; Tue, 23 Aug 2022 18:13:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232192AbiHWSNQ (ORCPT ); Tue, 23 Aug 2022 14:13:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233682AbiHWSMq (ORCPT ); Tue, 23 Aug 2022 14:12:46 -0400 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5D669C502 for ; Tue, 23 Aug 2022 09:25:19 -0700 (PDT) Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20220823162518epoutp02d3392e1d8e7092109609cca54f91ee26~OBgILUK3n2693526935epoutp02u for ; Tue, 23 Aug 2022 16:25:18 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20220823162518epoutp02d3392e1d8e7092109609cca54f91ee26~OBgILUK3n2693526935epoutp02u DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1661271918; bh=M5c5MSSpPjOFINuFSuBokw/onxM6vwEHa1nJRmz/DWI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dXw/ExN0zGDDsM7oGHvmZyVSDZdajHUTKIb1Dqll99RfqTaHDPb8J6t598sIsbvhJ 68MMKPVG+RMBkmO9XzyXyIDiIbIwcsoyBHhi8iK1SpMfNdYA3Gr5tBmS7rPygA2ji/ yXHQ/Xrk68FrVULr5nA+hGzosaTTv9YPJaFONSOI= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20220823162517epcas5p1b66853c5f5de3f7deb1a454ac896da57~OBgHjRodh1977519775epcas5p1C; Tue, 23 Aug 2022 16:25:17 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.176]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4MBvhp6BdMz4x9Pv; Tue, 23 Aug 2022 16:25:14 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id AB.15.18001.A6FF4036; Wed, 24 Aug 2022 01:25:14 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20220823162514epcas5p1a86cebaed6993eacd976b59fc2c68f29~OBgEiCaF21977519775epcas5p1-; Tue, 23 Aug 2022 16:25:14 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20220823162514epsmtrp1003f6192e2d42549c83941eefcf2b0aa~OBgEhSAsd2270522705epsmtrp1J; Tue, 23 Aug 2022 16:25:14 +0000 (GMT) X-AuditID: b6c32a4a-2c3ff70000004651-fc-6304ff6a8905 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 3E.02.14392.A6FF4036; Wed, 24 Aug 2022 01:25:14 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220823162512epsmtip28f578ee1714d17addb16531bc69f4e8e~OBgDEus_z2555425554epsmtip26; Tue, 23 Aug 2022 16:25:12 +0000 (GMT) From: Kanchan Joshi To: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, asml.silence@gmail.com Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, ming.lei@redhat.com, gost.dev@samsung.com, Kanchan Joshi Subject: [PATCH for-next v3 3/4] block: export blk_rq_is_poll Date: Tue, 23 Aug 2022 21:44:42 +0530 Message-Id: <20220823161443.49436-4-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220823161443.49436-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprFJsWRmVeSWpSXmKPExsWy7bCmpm7Wf5Zkg3WfdC3mrNrGaLH6bj+b xc0DO5ksVq4+ymTxrvUci8XR/2/ZLCYdusZosfeWtsX8ZU/ZLQ5NbmZy4PLYOesuu8fls6Ue m1Z1snlsXlLvsftmA5vH+31X2Tz6tqxi9Pi8SS6AIyrbJiM1MSW1SCE1Lzk/JTMv3VbJOzje Od7UzMBQ19DSwlxJIS8xN9VWycUnQNctMwfoQiWFssScUqBQQGJxsZK+nU1RfmlJqkJGfnGJ rVJqQUpOgUmBXnFibnFpXrpeXmqJlaGBgZEpUGFCdsakGw3MBR3cFT/nzmdvYJzL2cXIwSEh YCKxZ6JYFyMXh5DAbkaJKy27mSCcT4wSxyavYe5i5ARyvjFK3DmtAGKDNHRPb2CBKNrLKDHr 5nQ2iKLPjBK7byaDTGUT0JS4MLkUJCwi4CVx//Z7VpB6ZoG1jBKn935hAkkIC9hKfH/5mR3E ZhFQlVi0ZglYnFfAQmJZ1wV2iGXyEjMvfWcHmckpYClx+qAcRImgxMmZT1hAbGagkuats5kh yidySJxbEARhu0i8OjoBKi4s8er4FqiRUhIv+9ug7GSJSzPPMUHYJRKP9xyEsu0lWk/1M4Os ZQZ6Zf0ufYhVfBK9v58wQcKNV6KjTQiiWlHi3qSnrBC2uMTDGUugbA+JJV2LmCEh1cMo8eXx TbYJjPKzkHwwC8kHsxC2LWBkXsUomVpQnJueWmxaYJSXWg6P1OT83E2M4ESq5bWD8eGDD3qH GJk4GA8xSnAwK4nwWh1jSRbiTUmsrEotyo8vKs1JLT7EaAoM4YnMUqLJ+cBUnlcSb2hiaWBi ZmZmYmlsZqgkzjtFmzFZSCA9sSQ1OzW1ILUIpo+Jg1OqgSl+2T1HDZn1Std3TtjsdGPSMsEZ IZMsrju6f5X84Plvw70XHvu2+NaXrZi/tvSgbA931L5r60ItJa89c4jNMJSzWlgz2bYuQDRz 22WjDKsnV38Hztv2f7Ly4eLL/junBO8Xe/dvyf8ZTW8yl79pf+g0Y8H22+Ji7WueBnM0TI/l VjvwfJb3/7d3zeVfbuJdfJu36rv7Fp0oOeMqRekn133PPr64L6tvqUfQTJZ3dku31W3ZXO+h uKpv4ae6sEWhfMVCrd/iLr6ZN/m4mJnKsVv+ZUlbF1eGHHh4wjLz4t+aj5/5r6+UeLz9SaLh keffU8Rinh6rnyt8Jq3U8VX907U/so9/3rJAbrb/C/uz3seetyqxFGckGmoxFxUnAgBPGbZ8 LQQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrILMWRmVeSWpSXmKPExsWy7bCSvG7Wf5Zkg4NHRS3mrNrGaLH6bj+b xc0DO5ksVq4+ymTxrvUci8XR/2/ZLCYdusZosfeWtsX8ZU/ZLQ5NbmZy4PLYOesuu8fls6Ue m1Z1snlsXlLvsftmA5vH+31X2Tz6tqxi9Pi8SS6AI4rLJiU1J7MstUjfLoErY9KNBuaCDu6K n3PnszcwzuXsYuTkkBAwkeie3sDSxcjFISSwm1Hi29Vl7BAJcYnmaz+gbGGJlf+es0MUfWSU eLR0CmMXIwcHm4CmxIXJpSA1IgIBEgcbL4PVMAtsZpT4dPoYM0hCWMBW4vvLz2CDWARUJRat WcIEYvMKWEgs67oAtUBeYual7+wgMzkFLCVOH5QDCQsBlfzZ1MAGUS4ocXLmExYQmxmovHnr bOYJjAKzkKRmIUktYGRaxSiZWlCcm55bbFhgmJdarlecmFtcmpeul5yfu4kRHAtamjsYt6/6 oHeIkYmD8RCjBAezkgiv1TGWZCHelMTKqtSi/Pii0pzU4kOM0hwsSuK8F7pOxgsJpCeWpGan phakFsFkmTg4pRqYan8K3Vr2/r5Xwt2QrLbgA/Nn/3f76vLi3N45+oZntnlPllIKlPpmph1w l2GKvk7TItfPkdw8gsserbwoN1O/XeeB6kL3Tztj7uwTc0tYFXIo6ZRH/bPOA9d77v97UyJz fE5HIOfauz9Zf+ZqH5jxuEGoezdzCPfJU8+5c/VKY73E0hz+zPz2PXGfxqJW0ZNXen977FGu /9Yd7tCYN7GqrimKJ2dy2tfaFyfVHS+ukpVKPP8v7ImzyomiH5ZbmlgPZwcd+rEsoXvP15kc 1RWX52qG2IbfCIq6/j/esaQt88PWxc9bGXuzJ3hkLpmw7OJWzvVb2YIyFEu3lDXveK8hKSK+ o05ydvn9577Pz8faK7EUZyQaajEXFScCANszprH0AgAA X-CMS-MailID: 20220823162514epcas5p1a86cebaed6993eacd976b59fc2c68f29 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220823162514epcas5p1a86cebaed6993eacd976b59fc2c68f29 References: <20220823161443.49436-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org This is in preparation to support iopoll for nvme passthrough. Signed-off-by: Kanchan Joshi --- block/blk-mq.c | 3 ++- include/linux/blk-mq.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 4b90d2d8cfb0..4a07cab7dfb8 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1233,7 +1233,7 @@ static void blk_end_sync_rq(struct request *rq, blk_status_t ret) complete(&wait->done); } -static bool blk_rq_is_poll(struct request *rq) +bool blk_rq_is_poll(struct request *rq) { if (!rq->mq_hctx) return false; @@ -1243,6 +1243,7 @@ static bool blk_rq_is_poll(struct request *rq) return false; return true; } +EXPORT_SYMBOL_GPL(blk_rq_is_poll); static void blk_rq_poll_completion(struct request *rq, struct completion *wait) { diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 74b99d716b0b..b43c81d91892 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -980,6 +980,7 @@ int blk_rq_map_kern(struct request_queue *, struct request *, void *, int blk_rq_append_bio(struct request *rq, struct bio *bio); void blk_execute_rq_nowait(struct request *rq, bool at_head); blk_status_t blk_execute_rq(struct request *rq, bool at_head); +bool blk_rq_is_poll(struct request *rq); struct req_iterator { struct bvec_iter iter; From patchwork Tue Aug 23 16:14:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12952445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D157C32796 for ; Tue, 23 Aug 2022 18:13:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229786AbiHWSNR (ORCPT ); Tue, 23 Aug 2022 14:13:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35472 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233574AbiHWSMq (ORCPT ); Tue, 23 Aug 2022 14:12:46 -0400 Received: from mailout1.samsung.com (mailout1.samsung.com [203.254.224.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 816E76EF20 for ; Tue, 23 Aug 2022 09:25:23 -0700 (PDT) Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20220823162522epoutp01c7cb68d645ecf13fd406595b28861888~OBgLiJq1G0460004600epoutp01d for ; Tue, 23 Aug 2022 16:25:21 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20220823162522epoutp01c7cb68d645ecf13fd406595b28861888~OBgLiJq1G0460004600epoutp01d DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1661271922; bh=JUKbdFo7mf2YbJAJOW6KiaI/iC7zAWCIrdft9OfKayQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iDxejuvU17dnMYSQkjfNbXTBpaE2fKw/C0hSiyRidPAbFcwd5Ag1fnbVNtPgaUcg9 TAPesH9Qy2ccHlwpwxJTuGDdkAnI0qIF9NZmYsrF2yRjGTQpe+HAVbXYCjp3PyR1AU 4MxvmxsLuE1AVPiCjDjaqerNtfCOZlBykroEe1n4= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20220823162521epcas5p1f24bd31ad16f2804dafa5f66bdc1fc44~OBgK_rdUX2001420014epcas5p1_; Tue, 23 Aug 2022 16:25:21 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.180]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4MBvht1ygSz4x9Ps; Tue, 23 Aug 2022 16:25:18 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id FC.15.18001.E6FF4036; Wed, 24 Aug 2022 01:25:18 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20220823162517epcas5p2f1b808e60bae4bc1161b2d3a3a388534~OBgHNpxT-1190311903epcas5p2F; Tue, 23 Aug 2022 16:25:17 +0000 (GMT) Received: from epsmgms1p2.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20220823162517epsmtrp13427c76839105bda0f54348525358e9f~OBgHM390I2268222682epsmtrp1L; Tue, 23 Aug 2022 16:25:17 +0000 (GMT) X-AuditID: b6c32a4a-2c3ff70000004651-02-6304ff6ed0eb Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2.samsung.com (Symantec Messaging Gateway) with SMTP id 35.A2.18644.D6FF4036; Wed, 24 Aug 2022 01:25:17 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220823162515epsmtip20aef68abb86220f6289ad6f3f98cc481~OBgFoFOsM2555425554epsmtip27; Tue, 23 Aug 2022 16:25:15 +0000 (GMT) From: Kanchan Joshi To: axboe@kernel.dk, hch@lst.de, kbusch@kernel.org, asml.silence@gmail.com Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, ming.lei@redhat.com, gost.dev@samsung.com, Kanchan Joshi , Anuj Gupta Subject: [PATCH for-next v3 4/4] nvme: wire up async polling for io passthrough commands Date: Tue, 23 Aug 2022 21:44:43 +0530 Message-Id: <20220823161443.49436-5-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220823161443.49436-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrEJsWRmVeSWpSXmKPExsWy7bCmlm7ef5Zkg78d3BZNE/4yW8xZtY3R YvXdfjaLmwd2MlmsXH2UyeJd6zkWi6P/37JZTDp0jdFi7y1ti/nLnrJbHJrczOTA7bFz1l12 j8tnSz02repk89i8pN5j980GNo/3+66yefRtWcXo8XmTXABHVLZNRmpiSmqRQmpecn5KZl66 rZJ3cLxzvKmZgaGuoaWFuZJCXmJuqq2Si0+ArltmDtCZSgpliTmlQKGAxOJiJX07m6L80pJU hYz84hJbpdSClJwCkwK94sTc4tK8dL281BIrQwMDI1OgwoTsjH+/WtkKpptU7O5/wNjAeFq7 i5GTQ0LAROLUi4/MXYxcHEICuxkl/txrYgdJCAl8YpR4O0kFIvGZUWLJhd+MMB0Xj8xhgkjs YpRYf/woC1xV56G5QBkODjYBTYkLk0tBGkQEvCTu337PClLDLHCBUeLezpssIAlhgSiJ1g3/ wNaxCKhKnPy+AizOK2Ahcap7HyvENnmJmZe+s4PM5BSwlDh9UA6iRFDi5MwnYOXMQCXNW2eD vSAhMJVDov/tShaIXheJrys2MUPYwhKvjm9hh7ClJF72t0HZyRKXZp5jgrBLJB7vOQhl20u0 nupnBtnLDPTL+l36ELv4JHp/PwF7UUKAV6KjTQiiWlHi3qSnUBeLSzycsQTK9pDoPDyJFRI8 PYwSDQ+6mCcwys9C8sIsJC/MQti2gJF5FaNkakFxbnpqsWmBUV5qOTxek/NzNzGCE6uW1w7G hw8+6B1iZOJgPMQowcGsJMJrdYwlWYg3JbGyKrUoP76oNCe1+BCjKTCIJzJLiSbnA1N7Xkm8 oYmlgYmZmZmJpbGZoZI47xRtxmQhgfTEktTs1NSC1CKYPiYOTqkGJpel9hpv5AP0yjT6ly7z uqfvWfttefSSd5lXmtgvvTl+hz3m69aPQS9esnpd/3z1XSRrte7tBS0ZJ9Q2JhfOFd68yD2u xPZy6bSV99aYLr4u28eVL3VhUlv9+etS+1z5s2P+a932dmviFQx2npHbefi+V9zUU7ePCqV8 ttV94Xq06Hu08Satlu0nGmR+akw743WlszxD7rXDybWLbq9t5Hu99PG02yfqNH7ksJctlgl6 c7BmpcaLivci1UK+flPfPRHtO89ok7Dk++IeuySO3qr5O3TjOs1+/fDnWeT475ZGksBKhuT1 b6bsS5HJ2rT+TPWatjvpc15/DFh05cd0jjV8cvMSXt+tn7xCcU6CaKMSS3FGoqEWc1FxIgCU YcOINQQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrKLMWRmVeSWpSXmKPExsWy7bCSvG7uf5Zkg10n+S2aJvxltpizahuj xeq7/WwWNw/sZLJYufook8W71nMsFkf/v2WzmHToGqPF3lvaFvOXPWW3ODS5mcmB22PnrLvs HpfPlnpsWtXJ5rF5Sb3H7psNbB7v911l8+jbsorR4/MmuQCOKC6blNSczLLUIn27BK6Mf79a 2Qqmm1Ts7n/A2MB4WruLkZNDQsBE4uKROUwgtpDADkaJ24usIeLiEs3XfrBD2MISK/89B7K5 gGo+MkpsubGCtYuRg4NNQFPiwuRSkBoRgQCJg42XwWqYBW4wSuzrnQI2VFggQmLzxvOsIDaL gKrEye8rWEBsXgELiVPd+1ghFshLzLz0nR1kJqeApcTpg3IQ91hI/NnUwAZRLihxcuYTsFZm oPLmrbOZJzAKzEKSmoUktYCRaRWjZGpBcW56brFhgVFearlecWJucWleul5yfu4mRnBMaGnt YNyz6oPeIUYmDsZDjBIczEoivFbHWJKFeFMSK6tSi/Lji0pzUosPMUpzsCiJ817oOhkvJJCe WJKanZpakFoEk2Xi4JRqYIr9bXaPw+qCxrNzPqwyjbenH9BxXnA0xtC5u0q7rsh11+2aazKL m/gbizJ+XBXy3254YdOHH5oSXf+OfJXRuJ5xdNXVq0fjr99kX3bcoSlDfuJin+6dtyz6PUwO v5Gtq/3/++LHo5lvG+/rGq17YjW/a5rd3v7N4lOEXG/YbZ/ANfvSVtPgzliPZMZTkv+ilRtl W/cHTnv7xPHBjLoNZ6JaHoW3PWwQLNieudS5rJuf4WlH3Elf1QKhmM0vVWOY9Lesnui8XSMv 8Jf1omQeqSjddRuMvFUntz0LZFj35dRyBfmrm3NjXCMrRDg0ltxiK9zvYHdkzmX21L1vTued PTZBzPtJ1KKHylP1A0MNxJRYijMSDbWYi4oTAQTShw34AgAA X-CMS-MailID: 20220823162517epcas5p2f1b808e60bae4bc1161b2d3a3a388534 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220823162517epcas5p2f1b808e60bae4bc1161b2d3a3a388534 References: <20220823161443.49436-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Store a cookie during submission, and use that to implement completion-polling inside the ->uring_cmd_iopoll handler. This handler makes use of existing bio poll facility. Signed-off-by: Kanchan Joshi Signed-off-by: Anuj Gupta --- drivers/nvme/host/core.c | 1 + drivers/nvme/host/ioctl.c | 73 ++++++++++++++++++++++++++++++++--- drivers/nvme/host/multipath.c | 1 + drivers/nvme/host/nvme.h | 2 + 4 files changed, 72 insertions(+), 5 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index af367b22871b..7ac0deb8bbf8 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -3976,6 +3976,7 @@ static const struct file_operations nvme_ns_chr_fops = { .unlocked_ioctl = nvme_ns_chr_ioctl, .compat_ioctl = compat_ptr_ioctl, .uring_cmd = nvme_ns_chr_uring_cmd, + .uring_cmd_iopoll = nvme_ns_chr_uring_cmd_iopoll, }; static int nvme_add_ns_cdev(struct nvme_ns *ns) diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index 27614bee7380..7756b439a688 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -391,11 +391,19 @@ static void nvme_uring_cmd_end_io(struct request *req, blk_status_t err) struct nvme_uring_cmd_pdu *pdu = nvme_uring_cmd_pdu(ioucmd); /* extract bio before reusing the same field for request */ struct bio *bio = pdu->bio; + void *cookie = READ_ONCE(ioucmd->cookie); pdu->req = req; req->bio = bio; - /* this takes care of moving rest of completion-work to task context */ - io_uring_cmd_complete_in_task(ioucmd, nvme_uring_task_cb); + + /* + * For iopoll, complete it directly. + * Otherwise, move the completion to task work. + */ + if (cookie != NULL && blk_rq_is_poll(req)) + nvme_uring_task_cb(ioucmd); + else + io_uring_cmd_complete_in_task(ioucmd, nvme_uring_task_cb); } static int nvme_uring_cmd_io(struct nvme_ctrl *ctrl, struct nvme_ns *ns, @@ -445,7 +453,10 @@ static int nvme_uring_cmd_io(struct nvme_ctrl *ctrl, struct nvme_ns *ns, rq_flags = REQ_NOWAIT; blk_flags = BLK_MQ_REQ_NOWAIT; } + if (issue_flags & IO_URING_F_IOPOLL) + rq_flags |= REQ_POLLED; +retry: req = nvme_alloc_user_request(q, &c, nvme_to_user_ptr(d.addr), d.data_len, nvme_to_user_ptr(d.metadata), d.metadata_len, 0, &meta, d.timeout_ms ? @@ -456,6 +467,17 @@ static int nvme_uring_cmd_io(struct nvme_ctrl *ctrl, struct nvme_ns *ns, req->end_io = nvme_uring_cmd_end_io; req->end_io_data = ioucmd; + if (issue_flags & IO_URING_F_IOPOLL && rq_flags & REQ_POLLED) { + if (unlikely(!req->bio)) { + /* we can't poll this, so alloc regular req instead */ + blk_mq_free_request(req); + rq_flags &= ~REQ_POLLED; + goto retry; + } else { + WRITE_ONCE(ioucmd->cookie, req->bio); + req->bio->bi_opf |= REQ_POLLED; + } + } /* to free bio on completion, as req->bio will be null at that time */ pdu->bio = req->bio; pdu->meta = meta; @@ -559,9 +581,6 @@ long nvme_ns_chr_ioctl(struct file *file, unsigned int cmd, unsigned long arg) static int nvme_uring_cmd_checks(unsigned int issue_flags) { - /* IOPOLL not supported yet */ - if (issue_flags & IO_URING_F_IOPOLL) - return -EOPNOTSUPP; /* NVMe passthrough requires big SQE/CQE support */ if ((issue_flags & (IO_URING_F_SQE128|IO_URING_F_CQE32)) != @@ -604,6 +623,23 @@ int nvme_ns_chr_uring_cmd(struct io_uring_cmd *ioucmd, unsigned int issue_flags) return nvme_ns_uring_cmd(ns, ioucmd, issue_flags); } +int nvme_ns_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd) +{ + struct bio *bio; + int ret = 0; + struct nvme_ns *ns; + struct request_queue *q; + + rcu_read_lock(); + bio = READ_ONCE(ioucmd->cookie); + ns = container_of(file_inode(ioucmd->file)->i_cdev, + struct nvme_ns, cdev); + q = ns->queue; + if (test_bit(QUEUE_FLAG_POLL, &q->queue_flags) && bio && bio->bi_bdev) + ret = bio_poll(bio, NULL, 0); + rcu_read_unlock(); + return ret; +} #ifdef CONFIG_NVME_MULTIPATH static int nvme_ns_head_ctrl_ioctl(struct nvme_ns *ns, unsigned int cmd, void __user *argp, struct nvme_ns_head *head, int srcu_idx) @@ -685,6 +721,29 @@ int nvme_ns_head_chr_uring_cmd(struct io_uring_cmd *ioucmd, srcu_read_unlock(&head->srcu, srcu_idx); return ret; } + +int nvme_ns_head_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd) +{ + struct cdev *cdev = file_inode(ioucmd->file)->i_cdev; + struct nvme_ns_head *head = container_of(cdev, struct nvme_ns_head, cdev); + int srcu_idx = srcu_read_lock(&head->srcu); + struct nvme_ns *ns = nvme_find_path(head); + struct bio *bio; + int ret = 0; + struct request_queue *q; + + if (ns) { + rcu_read_lock(); + bio = READ_ONCE(ioucmd->cookie); + q = ns->queue; + if (test_bit(QUEUE_FLAG_POLL, &q->queue_flags) && bio + && bio->bi_bdev) + ret = bio_poll(bio, NULL, 0); + rcu_read_unlock(); + } + srcu_read_unlock(&head->srcu, srcu_idx); + return ret; +} #endif /* CONFIG_NVME_MULTIPATH */ int nvme_dev_uring_cmd(struct io_uring_cmd *ioucmd, unsigned int issue_flags) @@ -692,6 +751,10 @@ int nvme_dev_uring_cmd(struct io_uring_cmd *ioucmd, unsigned int issue_flags) struct nvme_ctrl *ctrl = ioucmd->file->private_data; int ret; + /* IOPOLL not supported yet */ + if (issue_flags & IO_URING_F_IOPOLL) + return -EOPNOTSUPP; + ret = nvme_uring_cmd_checks(issue_flags); if (ret) return ret; diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index 6ef497c75a16..00f2f81e20fa 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -439,6 +439,7 @@ static const struct file_operations nvme_ns_head_chr_fops = { .unlocked_ioctl = nvme_ns_head_chr_ioctl, .compat_ioctl = compat_ptr_ioctl, .uring_cmd = nvme_ns_head_chr_uring_cmd, + .uring_cmd_iopoll = nvme_ns_head_chr_uring_cmd_iopoll, }; static int nvme_add_ns_head_cdev(struct nvme_ns_head *head) diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 1bdf714dcd9e..fdcbc93dea21 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -821,6 +821,8 @@ long nvme_ns_head_chr_ioctl(struct file *file, unsigned int cmd, unsigned long arg); long nvme_dev_ioctl(struct file *file, unsigned int cmd, unsigned long arg); +int nvme_ns_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd); +int nvme_ns_head_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd); int nvme_ns_chr_uring_cmd(struct io_uring_cmd *ioucmd, unsigned int issue_flags); int nvme_ns_head_chr_uring_cmd(struct io_uring_cmd *ioucmd,