From patchwork Tue Dec 12 19:01:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 10108177 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E5234602C2 for ; Tue, 12 Dec 2017 19:02:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D85D423B24 for ; Tue, 12 Dec 2017 19:02:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CD64B27BA5; Tue, 12 Dec 2017 19:02:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4E89B23B24 for ; Tue, 12 Dec 2017 19:02:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752705AbdLLTCc (ORCPT ); Tue, 12 Dec 2017 14:02:32 -0500 Received: from mail-qt0-f193.google.com ([209.85.216.193]:39115 "EHLO mail-qt0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752414AbdLLTB5 (ORCPT ); Tue, 12 Dec 2017 14:01:57 -0500 Received: by mail-qt0-f193.google.com with SMTP id k19so49968915qtj.6; Tue, 12 Dec 2017 11:01:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=IoXI/zb2+uN+l+8LZm1Z2yiJu5l0Ty0JVMCLBmeZWmQ=; b=f68EfFLIatEfAnBZYdJJQ6MGAhcVLxzpuxKnV2lagrvngwJoO2nAHVS303/vfYqtqz fyG/BZwWL3y/xwAuTRTX1i/XtNZ4UNVqtPwvcyfSojIZ7sT27MwT90XRacQpYcF4XzOv X8BjjVB/HWUeW0rxsbCFNcBE9Y/2DpAJ0ctoFTjLFXdkLWFaz1iya1DJ22zjqDSExwYa SjuS+Ggiaav8RwLdrvnd+ft7T2VkOT/hG3QwCsWLWgn6yaR1cnuef160vwu2UFd5AwgH Yksz+Gf/Q/yOjEU8jknk9tVJTq1onN3JhnrmovKtFkm4W/3DY2zl0WV5ePsd4nDYwvCp o2uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=IoXI/zb2+uN+l+8LZm1Z2yiJu5l0Ty0JVMCLBmeZWmQ=; b=P1JoLDuG3NJNLKsDDOpCGR8f9YN8DjDbU+L4WXX4Is26W8IeIwRm/fiqaytR7NqaPY X56ZQf6HuApss3cakq5P3/52a2k01q/5gMC/E3T0nbk/kJUEXjPsi2+TO/UDlach48ZR LQ2HiIRPxdTR+55p83FcBbUlHGJKsqoWbyvOcYgL/HkI/N/H/FpP26mPg25qaPDpeBqi pprF2+z+BB+YXgF6DHhrn1QKfEWB9w7Wyxk73dDh5qmazhKCDlStFgo8Dg7FOYZtrZ6S 0XhHloUMK8FMdf7/mgiKq0Di86NItzx8G2Z635dd8QahGBDpDQsr7m1hqoyqjERHb0S+ VPXw== X-Gm-Message-State: AKGB3mJSSTI5bsz/00RrBB+ME3ODWk5L1JUY4QF3rqsPkHXdcKu8Lu1B dgkunQ9EFfWyn6trMWN6EVA= X-Google-Smtp-Source: ACJfBoub6daiydBzrVMrSinnqpw3/MximyBWAEegefpE8tQjdNjq3GoYKJi9NsqJYmDBOeeURqHGTA== X-Received: by 10.200.50.78 with SMTP id y14mr7444875qta.84.1513105316224; Tue, 12 Dec 2017 11:01:56 -0800 (PST) Received: from localhost (dhcp-ec-8-6b-ed-7a-cf.cpe.echoes.net. [72.28.5.223]) by smtp.gmail.com with ESMTPSA id j4sm6128020qtk.15.2017.12.12.11.01.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Dec 2017 11:01:55 -0800 (PST) From: Tejun Heo To: axboe@kernel.dk Cc: linux-kernel@vger.kernel.org, oleg@redhat.com, peterz@infradead.org, kernel-team@fb.com, osandov@fb.com, linux-block@vger.kernel.org, hch@lst.de, Tejun Heo , "jianchao.wang" Subject: [PATCH 5/6] blk-mq: remove REQ_ATOM_COMPLETE usages from blk-mq Date: Tue, 12 Dec 2017 11:01:33 -0800 Message-Id: <20171212190134.535941-6-tj@kernel.org> X-Mailer: git-send-email 2.9.5 In-Reply-To: <20171212190134.535941-1-tj@kernel.org> References: <20171212190134.535941-1-tj@kernel.org> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP After the recent updates to use generation number and state based synchronization, blk-mq no longer depends on REQ_ATOM_COMPLETE for anything. Remove all REQ_ATOM_COMPLETE usages. This removes atomic bitops from hot paths too. v2: Removed blk_clear_rq_complete() from blk_mq_rq_timed_out(). Signed-off-by: Tejun Heo Cc: "jianchao.wang" --- block/blk-mq.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 73d6444..7269552 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -596,14 +596,12 @@ void blk_mq_complete_request(struct request *rq) */ if (!(hctx->flags & BLK_MQ_F_BLOCKING)) { rcu_read_lock(); - if (blk_mq_rq_aborted_gstate(rq) != rq->gstate && - !blk_mark_rq_complete(rq)) + if (blk_mq_rq_aborted_gstate(rq) != rq->gstate) __blk_mq_complete_request(rq); rcu_read_unlock(); } else { srcu_idx = srcu_read_lock(hctx->queue_rq_srcu); - if (blk_mq_rq_aborted_gstate(rq) != rq->gstate && - !blk_mark_rq_complete(rq)) + if (blk_mq_rq_aborted_gstate(rq) != rq->gstate) __blk_mq_complete_request(rq); srcu_read_unlock(hctx->queue_rq_srcu, srcu_idx); } @@ -650,8 +648,6 @@ void blk_mq_start_request(struct request *rq) write_seqcount_end(&rq->gstate_seq); set_bit(REQ_ATOM_STARTED, &rq->atomic_flags); - if (test_bit(REQ_ATOM_COMPLETE, &rq->atomic_flags)) - clear_bit(REQ_ATOM_COMPLETE, &rq->atomic_flags); if (q->dma_drain_size && blk_rq_bytes(rq)) { /* @@ -819,7 +815,6 @@ static void blk_mq_rq_timed_out(struct request *req, bool reserved) req->aborted_gstate = 0; u64_stats_update_end(&req->aborted_gstate_sync); blk_add_timer(req); - blk_clear_rq_complete(req); break; case BLK_EH_NOT_HANDLED: break; @@ -870,8 +865,7 @@ static void blk_mq_terminate_expired(struct blk_mq_hw_ctx *hctx, * now guaranteed to see @rq->aborted_gstate and yield. If * @rq->aborted_gstate still matches @rq->gstate, @rq is ours. */ - if (READ_ONCE(rq->gstate) == rq->aborted_gstate && - !blk_mark_rq_complete(rq)) + if (READ_ONCE(rq->gstate) == rq->aborted_gstate) blk_mq_rq_timed_out(rq, reserved); }