From patchwork Fri Oct 15 15:14:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 12562357 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FFEBC4332F for ; Fri, 15 Oct 2021 15:14:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F0E7760E0C for ; Fri, 15 Oct 2021 15:14:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240880AbhJOPQg (ORCPT ); Fri, 15 Oct 2021 11:16:36 -0400 Received: from Galois.linutronix.de ([193.142.43.55]:50758 "EHLO galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240877AbhJOPQf (ORCPT ); Fri, 15 Oct 2021 11:16:35 -0400 From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1634310867; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t2vyjTQFjQd4X++czmFyYar/C0EbLUZWpU7OKsZ1+o4=; b=YoMS8BrxVWyprwlIDvdgAX07ZHkyn7LV9i7lrcTZv1H1KlLCjiPqw26FXHTsqIMoYF4Mcg sw8Ib832X02v6rb+EwGX7NWjmNlyTC58l2JGS8IxwXoypwciw3qLXcWzu8DKUBicE2vqe2 nhWxYTa9DTBIASVUe/L8FWortjCr3X0Dgipi2UxPsgm3BFVQV3xIsUHASiv0dD70OEG77c X7dY09UImMrKyNMIaeI0qrv6kWwATLjDCgakjAW80vVDFscmTyyRPxndINQplCeFT0qjwY OOiFG4Uqr1B1wzA6A5JYUY2rrp0i8ySJQMkCHy+DU0zki9EQNDQETCYku4XqPA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1634310867; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t2vyjTQFjQd4X++czmFyYar/C0EbLUZWpU7OKsZ1+o4=; b=Z5HO8+NB/9UEy3ZvH2v7lAQZ8BvQB7KY4BTcCiTpNx+p7JkxN1x82YUpHVhE3pkYcyCesx jgcdePyLL+/KgVCA== To: linux-block@vger.kernel.org, linux-mmc@vger.kernel.org, linux-scsi@vger.kernel.org, linux-usb@vger.kernel.org, usb-storage@lists.one-eyed-alien.net Cc: Jens Axboe , Ulf Hansson , "James E.J. Bottomley" , "Martin K. Petersen" , Alan Stern , Greg Kroah-Hartman , Christoph Hellwig , Thomas Gleixner , Sebastian Andrzej Siewior Subject: [RFC PATCH 1/3] blk-mq: Add blk_mq_complete_request_direct() Date: Fri, 15 Oct 2021 17:14:10 +0200 Message-Id: <20211015151412.3229037-2-bigeasy@linutronix.de> In-Reply-To: <20211015151412.3229037-1-bigeasy@linutronix.de> References: <20211015151412.3229037-1-bigeasy@linutronix.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Add blk_mq_complete_request_direct() which completes the block request directly instead deferring it to softirq for single queue devices. This is useful for devices which complete the requests in preemptible context and raising softirq from means scheduling ksoftirqd. Signed-off-by: Sebastian Andrzej Siewior --- block/blk-mq.c | 6 ++++++ include/linux/blk-mq.h | 1 + 2 files changed, 7 insertions(+) diff --git a/block/blk-mq.c b/block/blk-mq.c index 108a352051be5..44582aef3c32c 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -667,6 +667,12 @@ bool blk_mq_complete_request_remote(struct request *rq) } EXPORT_SYMBOL_GPL(blk_mq_complete_request_remote); +void blk_mq_complete_request_direct(struct request *rq) +{ + WRITE_ONCE(rq->state, MQ_RQ_COMPLETE); + rq->q->mq_ops->complete(rq); +} + /** * blk_mq_complete_request - end I/O on a request * @rq: the request being processed diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 13ba1861e688f..df9ea4c5d91c9 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -528,6 +528,7 @@ void __blk_mq_end_request(struct request *rq, blk_status_t error); void blk_mq_requeue_request(struct request *rq, bool kick_requeue_list); void blk_mq_kick_requeue_list(struct request_queue *q); void blk_mq_delay_kick_requeue_list(struct request_queue *q, unsigned long msecs); +void blk_mq_complete_request_direct(struct request *rq); void blk_mq_complete_request(struct request *rq); bool blk_mq_complete_request_remote(struct request *rq); bool blk_mq_queue_stopped(struct request_queue *q); From patchwork Fri Oct 15 15:14:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 12562359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6A66C4167B for ; Fri, 15 Oct 2021 15:14:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B553F6120A for ; Fri, 15 Oct 2021 15:14:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240886AbhJOPQh (ORCPT ); Fri, 15 Oct 2021 11:16:37 -0400 Received: from Galois.linutronix.de ([193.142.43.55]:50778 "EHLO galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240840AbhJOPQg (ORCPT ); Fri, 15 Oct 2021 11:16:36 -0400 From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1634310868; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w48GmUtrByyn7yHGg7JhSjj9SDZrPX8KgOLSu7dqDTM=; b=EzbhwnV8RyW75gNK2HAUscKuXOCZZPQNOwZaYAVjP0XyY1AlTUdNbKDmZkOLqwaeI5EXJe 1VdUJvQ6wh3Oi5PWK7J/QEVr2YeSsP2jTY5AsMUmFYmZls41idZDdd4PvcIIC5bh2XR+DX sZKPN5uR+ae2YQgGwP3BV7gfeMcTbv9ziKhFa7Jb7LCToKtmDvPGQIMQUcSvykCICVeuG0 9FlVL+KOWI4Q0l0sDKnxahXJfWsfAGRNB1cLMMh64ggIThcyDe9Z9iO6G11c5E4CFGmpRB mHq6qA5ZedHzFSOuwDYsdZ4Hl8sHZrtEv5gd0hfY0Ac1Fbo5b7htaM1kiO/KOA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1634310868; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w48GmUtrByyn7yHGg7JhSjj9SDZrPX8KgOLSu7dqDTM=; b=5sFeCDNOTqoUw5PGCg7hYbUv1xXyRpcer3II/ar1BKldck5BlKSOQy/ut4BED0W0gI6mNM EFmC0P5oAXY1lcDg== To: linux-block@vger.kernel.org, linux-mmc@vger.kernel.org, linux-scsi@vger.kernel.org, linux-usb@vger.kernel.org, usb-storage@lists.one-eyed-alien.net Cc: Jens Axboe , Ulf Hansson , "James E.J. Bottomley" , "Martin K. Petersen" , Alan Stern , Greg Kroah-Hartman , Christoph Hellwig , Thomas Gleixner , Sebastian Andrzej Siewior Subject: [RFC PATCH 2/3] mmc: core: Use blk_mq_complete_request_direct(). Date: Fri, 15 Oct 2021 17:14:11 +0200 Message-Id: <20211015151412.3229037-3-bigeasy@linutronix.de> In-Reply-To: <20211015151412.3229037-1-bigeasy@linutronix.de> References: <20211015151412.3229037-1-bigeasy@linutronix.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org The completion callback for the sdhci-pci device is invoked from a kworker. I couldn't identify in which context is mmc_blk_mq_req_done() invoke but the remaining caller are from invoked from preemptible context. Here it would make sense to complete the request directly instead scheduling ksoftirqd for its completion. Signed-off-by: Sebastian Andrzej Siewior --- drivers/mmc/core/block.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/drivers/mmc/core/block.c b/drivers/mmc/core/block.c index 431af5e8be2f8..16b4ea0e92c4b 100644 --- a/drivers/mmc/core/block.c +++ b/drivers/mmc/core/block.c @@ -2051,7 +2051,8 @@ static void mmc_blk_mq_dec_in_flight(struct mmc_queue *mq, struct request *req) mmc_put_card(mq->card, &mq->ctx); } -static void mmc_blk_mq_post_req(struct mmc_queue *mq, struct request *req) +static void mmc_blk_mq_post_req(struct mmc_queue *mq, struct request *req, + bool can_sleep) { struct mmc_queue_req *mqrq = req_to_mmc_queue_req(req); struct mmc_request *mrq = &mqrq->brq.mrq; @@ -2065,8 +2066,12 @@ static void mmc_blk_mq_post_req(struct mmc_queue *mq, struct request *req) */ if (mq->in_recovery) mmc_blk_mq_complete_rq(mq, req); - else if (likely(!blk_should_fake_timeout(req->q))) - blk_mq_complete_request(req); + else if (likely(!blk_should_fake_timeout(req->q))) { + if (can_sleep) + blk_mq_complete_request_direct(req); + else + blk_mq_complete_request(req); + } mmc_blk_mq_dec_in_flight(mq, req); } @@ -2087,7 +2092,7 @@ void mmc_blk_mq_recovery(struct mmc_queue *mq) mmc_blk_urgent_bkops(mq, mqrq); - mmc_blk_mq_post_req(mq, req); + mmc_blk_mq_post_req(mq, req, true); } static void mmc_blk_mq_complete_prev_req(struct mmc_queue *mq, @@ -2106,7 +2111,7 @@ static void mmc_blk_mq_complete_prev_req(struct mmc_queue *mq, if (prev_req) *prev_req = mq->complete_req; else - mmc_blk_mq_post_req(mq, mq->complete_req); + mmc_blk_mq_post_req(mq, mq->complete_req, true); mq->complete_req = NULL; @@ -2178,7 +2183,8 @@ static void mmc_blk_mq_req_done(struct mmc_request *mrq) mq->rw_wait = false; wake_up(&mq->wait); - mmc_blk_mq_post_req(mq, req); + /* context unknown */ + mmc_blk_mq_post_req(mq, req, false); } static bool mmc_blk_rw_wait_cond(struct mmc_queue *mq, int *err) @@ -2238,7 +2244,7 @@ static int mmc_blk_mq_issue_rw_rq(struct mmc_queue *mq, err = mmc_start_request(host, &mqrq->brq.mrq); if (prev_req) - mmc_blk_mq_post_req(mq, prev_req); + mmc_blk_mq_post_req(mq, prev_req, true); if (err) mq->rw_wait = false; From patchwork Fri Oct 15 15:14:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 12562361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC2ACC4167D for ; Fri, 15 Oct 2021 15:14:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D1F4D611C1 for ; Fri, 15 Oct 2021 15:14:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240893AbhJOPQi (ORCPT ); Fri, 15 Oct 2021 11:16:38 -0400 Received: from Galois.linutronix.de ([193.142.43.55]:50798 "EHLO galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240877AbhJOPQh (ORCPT ); Fri, 15 Oct 2021 11:16:37 -0400 From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1634310869; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xuQ9dlseb8QSm1Z1qmlP/P61CRYzuxuhzN+uy8YQa4c=; b=VbRC6g75nsEgTp52bSLZb1bvKlYsvT0DUQhp1nvTPv2mvoJplXXFrkUa92NGhfnlofU43n NGagCYRNmkgvONMRTAO96OD8D+5kLEE8qvx25X7eWFIawKriuSGnZ+t2BdzlelL6bYDvn1 7x6qHhY2NreivGd4RN0r2Gh2+Om8wzHN28E9IsMni1tpBzSsT7ZwHqQ3kgmzsLORMxc8Du x25+8lPDdmtMW0PBynZqucU69zjr2I4DF4g03QmuRtiecj4fK9Ubs0/wkLbnn33i6Zwd4F TQxdT0o/2LESVyw3Xfxh7MjiW0DVf7Zg6oTAQ0kOMVZN7GJraBVMJUF1iwq6Ug== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1634310869; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xuQ9dlseb8QSm1Z1qmlP/P61CRYzuxuhzN+uy8YQa4c=; b=8/q4h8aY2eTLusekixdaiKwfo4eGZ7DKaSQD54mcFKcjcfWh0qDtQn4ySYovt4Iu0f2WvW JSPKlU4ShcrIwtAg== To: linux-block@vger.kernel.org, linux-mmc@vger.kernel.org, linux-scsi@vger.kernel.org, linux-usb@vger.kernel.org, usb-storage@lists.one-eyed-alien.net Cc: Jens Axboe , Ulf Hansson , "James E.J. Bottomley" , "Martin K. Petersen" , Alan Stern , Greg Kroah-Hartman , Christoph Hellwig , Thomas Gleixner , Sebastian Andrzej Siewior Subject: [RFC PATCH 3/3] scsi, usb: storage: Complete the blk-request directly. Date: Fri, 15 Oct 2021 17:14:12 +0200 Message-Id: <20211015151412.3229037-4-bigeasy@linutronix.de> In-Reply-To: <20211015151412.3229037-1-bigeasy@linutronix.de> References: <20211015151412.3229037-1-bigeasy@linutronix.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org The usb-storage driver runs in a thread and completes its request from that thread. Since it is a single queued device it always schedules ksoftirqd for its completion. The completion is performed in the SCSI stack. Add scsi_done_preemptible() which inlines most of scsi_mq_done() and completes the request directly via blk_mq_complete_request_direct(). Signed-off-by: Sebastian Andrzej Siewior --- drivers/scsi/scsi_lib.c | 17 ++++++++++++++++- drivers/usb/storage/usb.c | 2 +- include/scsi/scsi_cmnd.h | 7 +++++++ 3 files changed, 24 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index 572673873ddf8..f0eeedce6b081 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -1575,16 +1575,31 @@ static blk_status_t scsi_prepare_cmd(struct request *req) return scsi_cmd_to_driver(cmd)->init_command(cmd); } -static void scsi_mq_done(struct scsi_cmnd *cmd) +static void _scsi_mq_done(struct scsi_cmnd *cmd) { if (unlikely(blk_should_fake_timeout(scsi_cmd_to_rq(cmd)->q))) return; if (unlikely(test_and_set_bit(SCMD_STATE_COMPLETE, &cmd->state))) return; trace_scsi_dispatch_cmd_done(cmd); +} + +static void scsi_mq_done(struct scsi_cmnd *cmd) +{ + _scsi_mq_done(cmd); blk_mq_complete_request(scsi_cmd_to_rq(cmd)); } +void scsi_done_preemptible(struct scsi_cmnd *scmd) +{ + if (scmd->scsi_done != scsi_mq_done) { + scmd->scsi_done(scmd); + return; + } + _scsi_mq_done(scmd); + blk_mq_complete_request_direct(scsi_cmd_to_rq(scmd)); +} + static void scsi_mq_put_budget(struct request_queue *q, int budget_token) { struct scsi_device *sdev = q->queuedata; diff --git a/drivers/usb/storage/usb.c b/drivers/usb/storage/usb.c index 90aa9c12ffac5..6ceedd1e14ce7 100644 --- a/drivers/usb/storage/usb.c +++ b/drivers/usb/storage/usb.c @@ -417,7 +417,7 @@ static int usb_stor_control_thread(void * __us) if (srb) { usb_stor_dbg(us, "scsi cmd done, result=0x%x\n", srb->result); - srb->scsi_done(srb); + scsi_done_preemptible(srb); } } /* for (;;) */ diff --git a/include/scsi/scsi_cmnd.h b/include/scsi/scsi_cmnd.h index eaf04c9a1dfcb..e992f2f74dd69 100644 --- a/include/scsi/scsi_cmnd.h +++ b/include/scsi/scsi_cmnd.h @@ -396,4 +396,11 @@ static inline unsigned scsi_transfer_length(struct scsi_cmnd *scmd) extern void scsi_build_sense(struct scsi_cmnd *scmd, int desc, u8 key, u8 asc, u8 ascq); +static inline void scsi_done(struct scsi_cmnd *scmd) +{ + scmd->scsi_done(scmd); +} + +extern void scsi_done_preemptible(struct scsi_cmnd *scmd); + #endif /* _SCSI_SCSI_CMND_H */