From patchwork Fri Sep 2 23:00:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 12964780 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADE39C6FA82 for ; Fri, 2 Sep 2022 23:01:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229436AbiIBXBA (ORCPT ); Fri, 2 Sep 2022 19:01:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43920 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229504AbiIBXA6 (ORCPT ); Fri, 2 Sep 2022 19:00:58 -0400 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 77CC6E830A for ; Fri, 2 Sep 2022 16:00:57 -0700 (PDT) Received: by mail-pj1-x102d.google.com with SMTP id t11-20020a17090a510b00b001fac77e9d1fso6815385pjh.5 for ; Fri, 02 Sep 2022 16:00:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=14O1H6cyL6/k0HHmoAv6g1KFSTuITnmhuTbbqklDRjk=; b=xXtQbvySjBRd44CkxNkhfkdj4N1gz19UqdXxSNHzSxrjfBjlWUsLiHSCp2tDI9YZLN JFouxJuOfKiRHn9fO/2Fr/wf6Qa8M00rS1kAF1bAPikttMAfk605E4pvVmhYuI6LMTIv 1whFZb/NXnrickYRCbP3udTaS9bDJXbQQTaoKQzNKQFJW57Sv5NzTGc2f4zFSU5Y1pzt tRaH50lPZ3HyKe0IulBPrc3uNu0pB02Sn5NMb1xhG1QV8Ux5tmKPe+EB/P2txJsCgJwD Xd0lDnsb+sjW36ZSX6a9PvhqAXKBRmaPzb0sn/CxlHJh4xHVnOOyAmFwpE0ldbyVjbz6 pngg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=14O1H6cyL6/k0HHmoAv6g1KFSTuITnmhuTbbqklDRjk=; b=M4JkSeUo95Cv7dySUMOhHYJIFR+Z4bVXTSuhKjeb1x8YQKnhZAvAZyYqHocwkhvMpa /cgo0J8yBWm+0jIpG+HJnUCvq1QmW7NNyLLe0WBACfCIYTIDY+AdY5XpBxByxUrMkDzX w2p9IYvJYsb30pyujausm5M4YBILuMGjYq58bUCzeCEranW+wM2bCo659qvnXaBN04EB i7oyRxFtY3ZUxPDKMcgJqReDYon2es2DTWTbgIXW/W5Sml7efXJ4Fw8J7LwzqZHCNLyo 2xyX+5nXLP0YYXkbEielFxK46U93Mxqi33wjbhoW7MVi1gg5krRcVsBWGqCVmyWPp+fL EK1g== X-Gm-Message-State: ACgBeo3OQiL7QYP6+4Sg9MougBGJEw7ribEn8lJz0TAj4pP1jB6Uwt98 WDBjeD3sHMxqqQw+p5Y8ebORFrbH4fGeKg== X-Google-Smtp-Source: AA6agR5ioGU0Fy7S7RKzZVFAx9/1hcJfLyk2E9LuNa8hjxgnWRdRtUBSz7AyBZJWQVY+HPx1B56ZtQ== X-Received: by 2002:a17:902:e889:b0:175:3e0e:169f with SMTP id w9-20020a170902e88900b001753e0e169fmr16468634plg.114.1662159656323; Fri, 02 Sep 2022 16:00:56 -0700 (PDT) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id 12-20020a170902c10c00b0016c5306917fsm2202104pli.53.2022.09.02.16.00.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Sep 2022 16:00:55 -0700 (PDT) From: Jens Axboe To: io-uring@vger.kernel.org Cc: joshi.k@samsung.com, Jens Axboe Subject: [PATCH 1/3] io_uring: cleanly separate request types for iopoll Date: Fri, 2 Sep 2022 17:00:50 -0600 Message-Id: <20220902230052.275835-2-axboe@kernel.dk> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220902230052.275835-1-axboe@kernel.dk> References: <20220902230052.275835-1-axboe@kernel.dk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org After the addition of iopoll support for passthrough, there's a bit of a mixup here. Clean it up and get rid of the casting for the passthrough command type. Signed-off-by: Jens Axboe --- io_uring/rw.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/io_uring/rw.c b/io_uring/rw.c index 9698a789b3d5..3f03b6d2a5a3 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -994,7 +994,7 @@ int io_do_iopoll(struct io_ring_ctx *ctx, bool force_nonspin) wq_list_for_each(pos, start, &ctx->iopoll_list) { struct io_kiocb *req = container_of(pos, struct io_kiocb, comp_list); - struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw); + struct file *file = req->file; int ret; /* @@ -1006,12 +1006,15 @@ int io_do_iopoll(struct io_ring_ctx *ctx, bool force_nonspin) break; if (req->opcode == IORING_OP_URING_CMD) { - struct io_uring_cmd *ioucmd = (struct io_uring_cmd *)rw; + struct io_uring_cmd *ioucmd; - ret = req->file->f_op->uring_cmd_iopoll(ioucmd); - } else - ret = rw->kiocb.ki_filp->f_op->iopoll(&rw->kiocb, &iob, - poll_flags); + ioucmd = io_kiocb_to_cmd(req, struct io_uring_cmd); + ret = file->f_op->uring_cmd_iopoll(ioucmd, poll_flags); + } else { + struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw); + + ret = file->f_op->iopoll(&rw->kiocb, &iob, poll_flags); + } if (unlikely(ret < 0)) return ret; else if (ret) From patchwork Fri Sep 2 23:00:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 12964781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14243C6FA83 for ; Fri, 2 Sep 2022 23:01:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230224AbiIBXBB (ORCPT ); Fri, 2 Sep 2022 19:01:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43930 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229517AbiIBXA7 (ORCPT ); Fri, 2 Sep 2022 19:00:59 -0400 Received: from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com [IPv6:2607:f8b0:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B207AE8316 for ; Fri, 2 Sep 2022 16:00:58 -0700 (PDT) Received: by mail-pl1-x62d.google.com with SMTP id d12so3233156plr.6 for ; Fri, 02 Sep 2022 16:00:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=yb5YyAK5cbnH0R7fDCajHHKBEKXm4i5C7knOCXRRYOs=; b=xhh6MdL5L1er6DEOGvep5rcoYXNjyU0Y4GBZOzpf5LGfjzCBVS8knq3HVgjyS5FGyG EdZxczd2faKhoRzEPRR0/4l3FNT1tFu8EHPjAsmhCJPQpPDlXngO+NMl5ikHSC5UZVsp VIAjW90MrIkdvaJx5MspGygFiau+o6FipBmylwp5Mc2DJCxLFWGi54m+R3hIcvwWLEaC 9ZfTwW6PqGeC8np9uUkv+PXO3TiGUxG9Dq5Dv4H/UclD9+cZgkwQC2LRft2c41ThaQ2I o9FjZEh06RjZ4FYj9+igyi6tdhaKZh01M9N1FuuVOodISQ285DWxjch5AP6u83lTmfbp XAjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=yb5YyAK5cbnH0R7fDCajHHKBEKXm4i5C7knOCXRRYOs=; b=Z2/JXUvYXaoEXcaDkHJ2ZFviZhP+tUxu0PfQs6JFAyquEY6Q0O+cMTVQ6FJmtO0QPE b3bo+2/npN+ca4sMSVn8DsbOb2tO0C04q0h51LfiWRF9vS3hNBntTR+YnZBVx1rhB+O5 o+8DgOxef/oCRsgINqZlGrND5ulD0CKnVSO1lgnXPpcRlgvYa20HHh4IKewx4vyflZUs 3Bdk31bXNFmVevx0VJfsLKeHeLCnOxG6C4k0EY9MAKMKdqci8AcW3rwecuhHUJf0rQtf PTocXcAd3aCVagxawxwffvMTZ68QssfFdmprnXqQ3rKrCtd5DplT/Y4Z1gLogmASk+K7 c9YQ== X-Gm-Message-State: ACgBeo3ar/5kkiuj8j1bH1UhzHLG20r2OCCzMRybm7OEDm5G+L996CEZ Ek5U4UzvdIy39ROaAkRnH+77ZxbONcdcew== X-Google-Smtp-Source: AA6agR5VohaI+160NaSIDDOnusrfhq8Pm1CzaWfb82c2zK9JTMRrYJnmZ9Hah5CVg4nALqaIh55HIQ== X-Received: by 2002:a17:90b:3142:b0:1f7:338a:1d38 with SMTP id ip2-20020a17090b314200b001f7338a1d38mr6868031pjb.223.1662159657687; Fri, 02 Sep 2022 16:00:57 -0700 (PDT) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id 12-20020a170902c10c00b0016c5306917fsm2202104pli.53.2022.09.02.16.00.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Sep 2022 16:00:56 -0700 (PDT) From: Jens Axboe To: io-uring@vger.kernel.org Cc: joshi.k@samsung.com, Jens Axboe Subject: [PATCH 2/3] nvme: use separate end IO handler for IOPOLL Date: Fri, 2 Sep 2022 17:00:51 -0600 Message-Id: <20220902230052.275835-3-axboe@kernel.dk> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220902230052.275835-1-axboe@kernel.dk> References: <20220902230052.275835-1-axboe@kernel.dk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org Don't need to rely on the cookie or request type, set the right handler based on how we're handling the IO. Signed-off-by: Jens Axboe --- drivers/nvme/host/ioctl.c | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index 7756b439a688..f34abe95821e 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -385,25 +385,36 @@ static void nvme_uring_task_cb(struct io_uring_cmd *ioucmd) io_uring_cmd_done(ioucmd, status, result); } -static void nvme_uring_cmd_end_io(struct request *req, blk_status_t err) +static void nvme_uring_iopoll_cmd_end_io(struct request *req, blk_status_t err) { struct io_uring_cmd *ioucmd = req->end_io_data; struct nvme_uring_cmd_pdu *pdu = nvme_uring_cmd_pdu(ioucmd); /* extract bio before reusing the same field for request */ struct bio *bio = pdu->bio; - void *cookie = READ_ONCE(ioucmd->cookie); pdu->req = req; req->bio = bio; /* * For iopoll, complete it directly. - * Otherwise, move the completion to task work. */ - if (cookie != NULL && blk_rq_is_poll(req)) - nvme_uring_task_cb(ioucmd); - else - io_uring_cmd_complete_in_task(ioucmd, nvme_uring_task_cb); + nvme_uring_task_cb(ioucmd); +} + +static void nvme_uring_cmd_end_io(struct request *req, blk_status_t err) +{ + struct io_uring_cmd *ioucmd = req->end_io_data; + struct nvme_uring_cmd_pdu *pdu = nvme_uring_cmd_pdu(ioucmd); + /* extract bio before reusing the same field for request */ + struct bio *bio = pdu->bio; + + pdu->req = req; + req->bio = bio; + + /* + * Move the completion to task work. + */ + io_uring_cmd_complete_in_task(ioucmd, nvme_uring_task_cb); } static int nvme_uring_cmd_io(struct nvme_ctrl *ctrl, struct nvme_ns *ns, @@ -464,7 +475,10 @@ static int nvme_uring_cmd_io(struct nvme_ctrl *ctrl, struct nvme_ns *ns, blk_flags); if (IS_ERR(req)) return PTR_ERR(req); - req->end_io = nvme_uring_cmd_end_io; + if (issue_flags & IO_URING_F_IOPOLL) + req->end_io = nvme_uring_iopoll_cmd_end_io; + else + req->end_io = nvme_uring_cmd_end_io; req->end_io_data = ioucmd; if (issue_flags & IO_URING_F_IOPOLL && rq_flags & REQ_POLLED) { From patchwork Fri Sep 2 23:00:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 12964782 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88ED5C6FA86 for ; Fri, 2 Sep 2022 23:01:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229517AbiIBXBC (ORCPT ); Fri, 2 Sep 2022 19:01:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43938 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229504AbiIBXBA (ORCPT ); Fri, 2 Sep 2022 19:01:00 -0400 Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 69BEAE8301 for ; Fri, 2 Sep 2022 16:00:59 -0700 (PDT) Received: by mail-pj1-x1035.google.com with SMTP id i5-20020a17090a2a0500b001fd8708ffdfso6845049pjd.2 for ; Fri, 02 Sep 2022 16:00:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=18eCieYHftePoeUXpDXqQ+Nzo96yLavK1iKazpHfrQc=; b=i9MgTEVrtQyizhn15yR3MU5wi122psE0tFbhUHxxn+JQ4xROCtWxqwSaeiRs91lRod h6y3gb9HeCZr3sUlvjk7p3YM1IfN04P6AlzbKu0/mR9WBHBAcIzLs9dQ2BtUJdmhSY0y u1a5qjYPMSak0A9ZdQR5R87P+SlgQHVBo91Wmk/WzMMf1DQB5gRxfhc4HAdHKgF5eZUg 34W+yp976J7M44WlcmmNfnxaJsS+7q8BD+yPTtQozKS8++REkfwVJiIVPLkL6N9MfFbG 2AYpDKsQXDkd1IIf6kFPUOmbYG4huycszqO4INGVGGVQVcPyPkuzj8LLUUBvreHmeyEU YKbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=18eCieYHftePoeUXpDXqQ+Nzo96yLavK1iKazpHfrQc=; b=BIoa4/wv67YHIUzthEoHdiSJ4iA1prWL0VH0vG6smQlB/E1/Il8Sqo8hMDHvI2TPER /8AFVtEc7dm5Hi60ZpwB1eMENNcZOOXY2U+e6X9ORu2hyttBXybflX0GuUQDC/Zp2ZQ9 8rMnIJ7rT1DZlLG8XtGrEi0YOnAFqh3qOz2297g/bYENhT6P/vNukEBTkVQI1VXTRoDd 2pYvlqmxSA5kYK9ZW1SuES8Pf8WCy4AwuPUJf1jlF2yt6sQmB3b1rHJEAjUXkc1lY8Cl ApoKNC/rGkfkHxOJBhbN/pQ/8U2oYO3THVm+0kaZ1LBXNrroU6oWXLk7DoWKOopGDSSW kt/w== X-Gm-Message-State: ACgBeo3Sv8VSYU8KbZBfrk2KeXT3IwS64pLfGsESlxQJgpjeIkjl4ur1 6JyABVaMLyFecn2oOTgD2sNZEPuWm91vlg== X-Google-Smtp-Source: AA6agR5F+ZmGQL+vJT1QE74EP8MG0cxsfzRKuyMTDRF1kq9mFzxwGHwDzT8MjsffbqabpQhyfELgzQ== X-Received: by 2002:a17:902:f54b:b0:174:f4a3:767c with SMTP id h11-20020a170902f54b00b00174f4a3767cmr22664370plf.4.1662159658665; Fri, 02 Sep 2022 16:00:58 -0700 (PDT) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id 12-20020a170902c10c00b0016c5306917fsm2202104pli.53.2022.09.02.16.00.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Sep 2022 16:00:58 -0700 (PDT) From: Jens Axboe To: io-uring@vger.kernel.org Cc: joshi.k@samsung.com, Jens Axboe Subject: [PATCH 3/3] fs: add batch and poll flags to the uring_cmd_iopoll() handler Date: Fri, 2 Sep 2022 17:00:52 -0600 Message-Id: <20220902230052.275835-4-axboe@kernel.dk> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220902230052.275835-1-axboe@kernel.dk> References: <20220902230052.275835-1-axboe@kernel.dk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org We need the poll_flags to know how to poll for the IO, and we should have the batch structure in preparation for supporting batched completions with iopoll. Signed-off-by: Jens Axboe --- drivers/nvme/host/ioctl.c | 12 ++++++++---- drivers/nvme/host/nvme.h | 6 ++++-- include/linux/fs.h | 3 ++- io_uring/rw.c | 3 ++- 4 files changed, 16 insertions(+), 8 deletions(-) diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index f34abe95821e..7a0b12ef49ae 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -637,7 +637,9 @@ int nvme_ns_chr_uring_cmd(struct io_uring_cmd *ioucmd, unsigned int issue_flags) return nvme_ns_uring_cmd(ns, ioucmd, issue_flags); } -int nvme_ns_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd) +int nvme_ns_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd, + struct io_comp_batch *iob, + unsigned int poll_flags) { struct bio *bio; int ret = 0; @@ -650,7 +652,7 @@ int nvme_ns_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd) struct nvme_ns, cdev); q = ns->queue; if (test_bit(QUEUE_FLAG_POLL, &q->queue_flags) && bio && bio->bi_bdev) - ret = bio_poll(bio, NULL, 0); + ret = bio_poll(bio, iob, poll_flags); rcu_read_unlock(); return ret; } @@ -736,7 +738,9 @@ int nvme_ns_head_chr_uring_cmd(struct io_uring_cmd *ioucmd, return ret; } -int nvme_ns_head_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd) +int nvme_ns_head_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd, + struct io_comp_batch *iob, + unsigned int poll_flags) { struct cdev *cdev = file_inode(ioucmd->file)->i_cdev; struct nvme_ns_head *head = container_of(cdev, struct nvme_ns_head, cdev); @@ -752,7 +756,7 @@ int nvme_ns_head_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd) q = ns->queue; if (test_bit(QUEUE_FLAG_POLL, &q->queue_flags) && bio && bio->bi_bdev) - ret = bio_poll(bio, NULL, 0); + ret = bio_poll(bio, iob, poll_flags); rcu_read_unlock(); } srcu_read_unlock(&head->srcu, srcu_idx); diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index fdcbc93dea21..216acbe953b3 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -821,8 +821,10 @@ long nvme_ns_head_chr_ioctl(struct file *file, unsigned int cmd, unsigned long arg); long nvme_dev_ioctl(struct file *file, unsigned int cmd, unsigned long arg); -int nvme_ns_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd); -int nvme_ns_head_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd); +int nvme_ns_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd, + struct io_comp_batch *iob, unsigned int poll_flags); +int nvme_ns_head_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd, + struct io_comp_batch *iob, unsigned int poll_flags); int nvme_ns_chr_uring_cmd(struct io_uring_cmd *ioucmd, unsigned int issue_flags); int nvme_ns_head_chr_uring_cmd(struct io_uring_cmd *ioucmd, diff --git a/include/linux/fs.h b/include/linux/fs.h index d6badd19784f..01681d061a6a 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2132,7 +2132,8 @@ struct file_operations { loff_t len, unsigned int remap_flags); int (*fadvise)(struct file *, loff_t, loff_t, int); int (*uring_cmd)(struct io_uring_cmd *ioucmd, unsigned int issue_flags); - int (*uring_cmd_iopoll)(struct io_uring_cmd *ioucmd); + int (*uring_cmd_iopoll)(struct io_uring_cmd *, struct io_comp_batch *, + unsigned int poll_flags); } __randomize_layout; struct inode_operations { diff --git a/io_uring/rw.c b/io_uring/rw.c index 3f03b6d2a5a3..4a061326c664 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -1009,7 +1009,8 @@ int io_do_iopoll(struct io_ring_ctx *ctx, bool force_nonspin) struct io_uring_cmd *ioucmd; ioucmd = io_kiocb_to_cmd(req, struct io_uring_cmd); - ret = file->f_op->uring_cmd_iopoll(ioucmd, poll_flags); + ret = file->f_op->uring_cmd_iopoll(ioucmd, &iob, + poll_flags); } else { struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw);