From patchwork Tue Nov 28 03:36:59 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mikulas Patocka X-Patchwork-Id: 10078591 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0489C6056A for ; Tue, 28 Nov 2017 03:37:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E5A5228E8F for ; Tue, 28 Nov 2017 03:37:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D770B290A4; Tue, 28 Nov 2017 03:37:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 84E6F28E8F for ; Tue, 28 Nov 2017 03:37:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751943AbdK1DhB (ORCPT ); Mon, 27 Nov 2017 22:37:01 -0500 Received: from mx1.redhat.com ([209.132.183.28]:39446 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751822AbdK1DhA (ORCPT ); Mon, 27 Nov 2017 22:37:00 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A8CDF81DE4; Tue, 28 Nov 2017 03:37:00 +0000 (UTC) Received: from file01.intranet.prod.int.rdu2.redhat.com (file01.intranet.prod.int.rdu2.redhat.com [10.11.5.7]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3EBDD5FCA0; Tue, 28 Nov 2017 03:37:00 +0000 (UTC) Received: from file01.intranet.prod.int.rdu2.redhat.com (localhost [127.0.0.1]) by file01.intranet.prod.int.rdu2.redhat.com (8.14.4/8.14.4) with ESMTP id vAS3axsD013674; Mon, 27 Nov 2017 22:36:59 -0500 Received: from localhost (mpatocka@localhost) by file01.intranet.prod.int.rdu2.redhat.com (8.14.4/8.14.4/Submit) with ESMTP id vAS3axmJ013666; Mon, 27 Nov 2017 22:36:59 -0500 X-Authentication-Warning: file01.intranet.prod.int.rdu2.redhat.com: mpatocka owned process doing -bs Date: Mon, 27 Nov 2017 22:36:59 -0500 (EST) From: Mikulas Patocka X-X-Sender: mpatocka@file01.intranet.prod.int.rdu2.redhat.com To: Ingo Molnar cc: Ingo Molnar , Peter Zijlstra , linux-kernel@vger.kernel.org, Jens Axboe , linux-block@vger.kernel.org Subject: Re: [PATCH] schedule: use unlikely() In-Reply-To: <20171125081613.damtbce2pelywx2k@gmail.com> Message-ID: References: <20171124073819.ejig4akxinmugbnr@gmail.com> <20171125081613.damtbce2pelywx2k@gmail.com> User-Agent: Alpine 2.02 (LRH 1266 2009-07-14) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 28 Nov 2017 03:37:00 +0000 (UTC) Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Sat, 25 Nov 2017, Ingo Molnar wrote: > > * Mikulas Patocka wrote: > > > On Fri, 24 Nov 2017, Ingo Molnar wrote: > > > > > > + return unlikely(plug != NULL) && > > > > (!list_empty(&plug->list) || > > > > !list_empty(&plug->mq_list) || > > > > !list_empty(&plug->cb_list)); > > > > > > That's an unrelated change. > > > > It is related, because this function gets inlined into schedule(). > > That needs to be in the changelog and the patch needs to be Cc:-ed to the > affected maintainer as well. > > > The static branch prediction in gcc assumes that pointers are usually not > > NULL, but in this case tsk->pi_blocked_on and tsk->plug are usually NULL. > > That too should be in the changelog. > > Thanks, > > Ingo > OK - here I resend it with more verbose message: Mikulas From: Mikulas Patocka Subject: [PATCH] schedule: add unlikely() A small patch for schedule(), so that the code goes straght in the common case. This patch saves two jumps in the assembler listing. The static branch prediction in GCC assumes that pointers are usually non-NULL when they are compared against NULL, but in theis case, tsk->plug is usually NULL (the function blk_needs_flush_plug gets inlined into schedule()) and tsk->pi_blocked_on is also usually NULL (that is checked by tsk_is_pi_blocked). This patch adds the macro unlikely() to override the static prediction. Signed-off-by: Mikulas Patocka --- include/linux/blkdev.h | 2 +- kernel/sched/core.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) Index: linux-2.6/include/linux/blkdev.h =================================================================== --- linux-2.6.orig/include/linux/blkdev.h +++ linux-2.6/include/linux/blkdev.h @@ -1308,7 +1308,7 @@ static inline bool blk_needs_flush_plug( { struct blk_plug *plug = tsk->plug; - return plug && + return unlikely(plug != NULL) && (!list_empty(&plug->list) || !list_empty(&plug->mq_list) || !list_empty(&plug->cb_list)); Index: linux-2.6/kernel/sched/core.c =================================================================== --- linux-2.6.orig/kernel/sched/core.c +++ linux-2.6/kernel/sched/core.c @@ -3405,7 +3405,7 @@ void __noreturn do_task_dead(void) static inline void sched_submit_work(struct task_struct *tsk) { - if (!tsk->state || tsk_is_pi_blocked(tsk)) + if (!tsk->state || unlikely(tsk_is_pi_blocked(tsk))) return; /* * If we are going to sleep and we have plugged IO queued,