From patchwork Thu Jun 30 14:27:24 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Shreyas B. Prabhu" X-Patchwork-Id: 9208161 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0DDC860752 for ; Thu, 30 Jun 2016 14:31:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F21A92848A for ; Thu, 30 Jun 2016 14:31:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E6EAC2867E; Thu, 30 Jun 2016 14:31:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 37A3D2848A for ; Thu, 30 Jun 2016 14:31:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752459AbcF3Ob1 (ORCPT ); Thu, 30 Jun 2016 10:31:27 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:48683 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752323AbcF3Ob0 (ORCPT ); Thu, 30 Jun 2016 10:31:26 -0400 Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.11/8.16.0.11) with SMTP id u5UETNIY129066 for ; Thu, 30 Jun 2016 10:31:25 -0400 Received: from e33.co.us.ibm.com (e33.co.us.ibm.com [32.97.110.151]) by mx0b-001b2d01.pphosted.com with ESMTP id 23vvccuas2-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Thu, 30 Jun 2016 10:31:25 -0400 Received: from localhost by e33.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 30 Jun 2016 08:31:19 -0600 Received: from d03dlp03.boulder.ibm.com (9.17.202.179) by e33.co.us.ibm.com (192.168.1.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 30 Jun 2016 08:31:04 -0600 X-IBM-Helo: d03dlp03.boulder.ibm.com X-IBM-MailFrom: shreyas@linux.vnet.ibm.com Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by d03dlp03.boulder.ibm.com (Postfix) with ESMTP id 3BDA519D82A7; Thu, 30 Jun 2016 08:27:05 -0600 (MDT) Received: from b03ledav003.gho.boulder.ibm.com (b03ledav003.gho.boulder.ibm.com [9.17.130.234]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id u5UERRkl63111230; Thu, 30 Jun 2016 07:27:27 -0700 Received: from b03ledav003.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 355FF6A047; Thu, 30 Jun 2016 08:27:27 -0600 (MDT) Received: from ltchab1.austin.ibm.com (unknown [9.40.193.138]) by b03ledav003.gho.boulder.ibm.com (Postfix) with ESMTP id 6B1886A041; Thu, 30 Jun 2016 08:27:26 -0600 (MDT) From: "Shreyas B. Prabhu" To: rjw@rjwysocki.net Cc: daniel.lezcano@linaro.org, linux-pm@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, anton@samba.org, mpe@ellerman.id.au, bsingharora@gmail.com, David.Laight@ACULAB.COM, arnd@arndb.de, nicolas.pitre@linaro.org, "Shreyas B. Prabhu" Subject: [PATCH] cpuidle: Fix last_residency division Date: Thu, 30 Jun 2016 09:27:24 -0500 X-Mailer: git-send-email 2.1.4 X-TM-AS-GCONF: 00 X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16063014-0008-0000-0000-000004F35048 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 16063014-0009-0000-0000-000038E5C308 Message-Id: <1467296844-2055-1-git-send-email-shreyas@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2016-06-30_06:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=1 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1604210000 definitions=main-1606300138 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Snooze is a poll idle state in powernv and pseries platforms. Snooze has a timeout so that if a cpu stays in snooze for more than target residency of the next available idle state, then it would exit thereby giving chance to the cpuidle governor to re-evaluate and promote the cpu to a deeper idle state. Therefore whenever snooze exits due to this timeout, its last_residency will be target_residency of next deeper state. commit e93e59ce5b85 ("cpuidle: Replace ktime_get() with local_clock()") changed the math around last_residency calculation. Specifically, while converting last_residency value from nanoseconds to microseconds it does right shift by 10. Due to this, in snooze timeout exit scenarios last_residency calculated is roughly 2.3% less than target_residency of next available state. This pattern is picked up get_typical_interval() in the menu governor and therefore expected_interval in menu_select() is frequently less than the target_residency of any state but snooze. Due to this we are entering snooze at a higher rate, thereby affecting the single thread performance. Fix this by using a better approximation for division by 1000. Reported-by: Anton Blanchard Bisected-by: Shilpasri G Bhat Suggested-by David Laight Signed-off-by: Shreyas B. Prabhu Reviewed-by: Nicolas Pitre --- Changes in v4 ============= - Increasing the threshold upto which approximation can be used. - Removed explicit cast. Instead added a comment saying why cast is safe. Changes in v3 ============= - Using approximation suggested by David Changes in v2 ============= - Fixing it in the cpuidle core code instead of driver code. drivers/cpuidle/cpuidle.c | 11 +++-------- drivers/cpuidle/cpuidle.h | 38 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+), 8 deletions(-) diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c index a4d0059..f55ad01 100644 --- a/drivers/cpuidle/cpuidle.c +++ b/drivers/cpuidle/cpuidle.c @@ -174,7 +174,6 @@ int cpuidle_enter_state(struct cpuidle_device *dev, struct cpuidle_driver *drv, struct cpuidle_state *target_state = &drv->states[index]; bool broadcast = !!(target_state->flags & CPUIDLE_FLAG_TIMER_STOP); u64 time_start, time_end; - s64 diff; /* * Tell the time framework to switch to a broadcast timer because our @@ -218,14 +217,10 @@ int cpuidle_enter_state(struct cpuidle_device *dev, struct cpuidle_driver *drv, local_irq_enable(); /* - * local_clock() returns the time in nanosecond, let's shift - * by 10 (divide by 1024) to have microsecond based time. + * local_clock() returns the time in nanoseconds, convert it to + * microsecond based time. */ - diff = (time_end - time_start) >> 10; - if (diff > INT_MAX) - diff = INT_MAX; - - dev->last_residency = (int) diff; + dev->last_residency = convert_nsec_to_usec(time_end - time_start); if (entered_state >= 0) { /* Update cpuidle counters */ diff --git a/drivers/cpuidle/cpuidle.h b/drivers/cpuidle/cpuidle.h index f87f399..a027b35 100644 --- a/drivers/cpuidle/cpuidle.h +++ b/drivers/cpuidle/cpuidle.h @@ -68,4 +68,42 @@ static inline void cpuidle_coupled_unregister_device(struct cpuidle_device *dev) } #endif +/* + * To ensure that there is no overflow while approximation + * for dividing val by 1000, we must respect - + * val + (val >> 5) <= 0xFFFFFFFF + * val + val/32 <= 0xFFFFFFFF + * val <= (0xFFFFFFFF * 32) / 33 + * val <= 0xF83E0F82 + * Hence the threshold for val below which we can use the + * approximation is 0xF83E0F82 + */ +#define DIV_APPROXIMATION_THRESHOLD 0xF83E0F82UL + +/* + * Used for calculating last_residency in usec. Optimized for case + * where last_residency in nsecs is < DIV_APPROXIMATION_THRESHOLD + * Approximated value has less than 1% error. + */ +static inline int convert_nsec_to_usec(u64 nsec) +{ + if (likely(nsec < DIV_APPROXIMATION_THRESHOLD)) { + u32 usec = nsec; + + usec += usec >> 5; + usec = usec >> 10; + + /* Can safely cast to int since usec is < INT_MAX */ + return usec; + } else { + u64 usec = div_u64(nsec, 1000); + + if (usec > INT_MAX) + usec = INT_MAX; + + /* Can safely cast to int since usec is < INT_MAX */ + return usec; + } +} + #endif /* __DRIVER_CPUIDLE_H */