From patchwork Sun Mar 18 20:59:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Douglas Gilbert X-Patchwork-Id: 10291405 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1691660291 for ; Sun, 18 Mar 2018 20:59:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0744728EE6 for ; Sun, 18 Mar 2018 20:59:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F013B28EF0; Sun, 18 Mar 2018 20:59:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 894D428EE6 for ; Sun, 18 Mar 2018 20:59:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754489AbeCRU7O (ORCPT ); Sun, 18 Mar 2018 16:59:14 -0400 Received: from smtp.infotech.no ([82.134.31.41]:45551 "EHLO smtp.infotech.no" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754537AbeCRU7J (ORCPT ); Sun, 18 Mar 2018 16:59:09 -0400 Received: from localhost (localhost [127.0.0.1]) by smtp.infotech.no (Postfix) with ESMTP id B4B50204170; Sun, 18 Mar 2018 21:59:05 +0100 (CET) X-Virus-Scanned: by amavisd-new-2.6.6 (20110518) (Debian) at infotech.no Received: from smtp.infotech.no ([127.0.0.1]) by localhost (smtp.infotech.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jQtHQFnO2SLn; Sun, 18 Mar 2018 21:59:05 +0100 (CET) Received: from xtwo70.infotech.no (unknown [82.134.31.163]) by smtp.infotech.no (Postfix) with ESMTPA id 79CE32041C0; Sun, 18 Mar 2018 21:59:05 +0100 (CET) From: Douglas Gilbert To: linux-scsi@vger.kernel.org Cc: martin.petersen@oracle.com, jejb@linux.vnet.ibm.com, hare@suse.de, bart.vanassche@wdc.com, jthumshirn@suse.de Subject: [PATCH v2 5/6] scsi_io_completion hints on fastpatch Date: Sun, 18 Mar 2018 21:59:03 +0100 Message-Id: <20180318205904.28541-6-dgilbert@interlog.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180318205904.28541-1-dgilbert@interlog.com> References: <20180318205904.28541-1-dgilbert@interlog.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add likely() and unlikely() hints to conditionals on or near the fastpath. Signed-off-by: Douglas Gilbert Reviewed-by: Johannes Thumshirn Reviewed-by: Bart Van Assche --- A reviewer wanted any performance improvement (or otherwise) quantified. The improvement was so small, that ftrace ignored it. Inline timing code suggests the improvement from this whole patchset is around 7 nanoseconds per invocation (tested on a Lenovo X270 (i5-7200U)). Not exactly huge. Another win might be the smaller size of scsi_io_completion() after the refactoring; this should allow more other code to fit in the instruction cache. drivers/scsi/scsi_lib.c | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index 0e2dcdb63fcd..63f79b0dbbf3 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -1020,7 +1020,7 @@ void scsi_io_completion(struct scsi_cmnd *cmd, unsigned int good_bytes) struct request *req = cmd->request; blk_status_t blk_stat = BLK_STS_OK; /* u8: BLK_STS_OK is only 0 */ - if (result) { /* does not necessarily mean there is an error */ + if (unlikely(result)) { /* a nz result may or may not be an error */ blk_stat = scsi_io_completion_nz_result(cmd, result); if (blk_stat == BLK_STS_OK) result = 0; /* suppress error processing now */ @@ -1033,14 +1033,14 @@ void scsi_io_completion(struct scsi_cmnd *cmd, unsigned int good_bytes) } - if (blk_rq_is_passthrough(req)) { + if (unlikely(blk_rq_is_passthrough(req))) { /* * __scsi_error_from_host_byte may have reset the host_byte */ scsi_req(req)->result = cmd->result; scsi_req(req)->resid_len = scsi_get_resid(cmd); - if (scsi_bidi_cmnd(cmd)) { + if (unlikely(scsi_bidi_cmnd(cmd))) { /* * Bidi commands Must be complete as a whole, * both sides at once. @@ -1053,7 +1053,7 @@ void scsi_io_completion(struct scsi_cmnd *cmd, unsigned int good_bytes) } } - /* no bidi support for !blk_rq_is_passthrough yet */ + /* no bidi support yet, other than in pass-through */ BUG_ON(blk_bidi_rq(req)); /* @@ -1069,15 +1069,13 @@ void scsi_io_completion(struct scsi_cmnd *cmd, unsigned int good_bytes) * handle. Failed, zero length commands always need to drop down * to retry code. Fast path should return in this block. */ - if (blk_rq_bytes(req) > 0 || blk_stat == BLK_STS_OK) { - if (!scsi_end_request(req, blk_stat, good_bytes, 0)) - return; /* no bytes remaining */ + if (likely(blk_rq_bytes(req) > 0 || blk_stat == BLK_STS_OK)) { + if (likely(!scsi_end_request(req, blk_stat, good_bytes, 0))) + return; /* no bytes remaining */ } - /* - * Kill remainder if no retrys. - */ - if (blk_stat && scsi_noretry_cmd(cmd)) { + /* Kill remainder if no retrys */ + if (unlikely(blk_stat && scsi_noretry_cmd(cmd))) { if (scsi_end_request(req, blk_stat, blk_rq_bytes(req), 0)) BUG(); return; @@ -1087,14 +1085,14 @@ void scsi_io_completion(struct scsi_cmnd *cmd, unsigned int good_bytes) * If there had been no error, but we have leftover bytes in the * requeues just queue the command up again. */ - if (result == 0) { + if (likely(result == 0)) { /* * Unprep the request and put it back at the head of the * queue. A new command will be prepared and issued. * This block is the same as case ACTION_REPREP in * scsi_io_completion_action() above. */ - if (q->mq_ops) + if (likely(q->mq_ops)) scsi_mq_requeue_cmd(cmd); else { scsi_release_buffers(cmd);