From patchwork Thu Dec 12 01:57:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sultan Alsawaf X-Patchwork-Id: 13904593 Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2F8C7FBAC; Thu, 12 Dec 2024 01:57:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733968661; cv=none; b=ayAjEcsbSh6Rpynvos59STA81JR2rJsYktr3K4geqIZGE2v58FQJYIOuekIHsw/a/tBfmNrfBAgfiXlcU46kL7+LzdE1yu6mc760VGC4KvUqdgigptONgaA5xi/g3qm3aREV/dHkIc1pNXB0vemoxTfLN7fLijFoUnK/vjHwdyY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733968661; c=relaxed/simple; bh=Bz9SD0S4ih/0WGt3sKvD75BXOA7xvusCYRJ6XvQZeBM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MYlPmPAYohOp/dSUAwFbvgP0S0jHTg0ZqgBOMBFNO7EweI87bZJp6UoirNcpZIC2Ro+C5PTjozOvCORx6OImPwdND6QY3TOHoG3Jx4YUs7LSVrbKSepYEI2ENJYugOvm4Q/6/cdEV1WM9UAK7+AfiCYQoEnZFnT4tXLdnFSLDnw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kerneltoast.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=VscK2q5O; arc=none smtp.client-ip=209.85.215.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kerneltoast.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="VscK2q5O" Received: by mail-pg1-f176.google.com with SMTP id 41be03b00d2f7-7fd2ff40782so128795a12.2; Wed, 11 Dec 2024 17:57:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733968659; x=1734573459; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=UIIEH4/iFa3/TdDh0MQgQz7IigAr1CH+cv3VD+qL854=; b=VscK2q5O3gKynQgxuHJwU5ftgwy9zi6Fjw1zBrA7a5Ngw3uDcZZczlDCreNvk4BsMV 0VSbDLQpFUKqd4QvFGOEnvIm3JwWG61j0hjAF/EGYmts5uHX7Ci5VUECg4FyWXJNZXN/ P2UkAAWel0fTA8BKaFfxTBnf0bHE4rGjwryIxsTZxWUdeAVqp8nxgNfLa3nVYD398Wrq RnANLtQN6W5F8R3ilTodDfa819UOYmeSPlDeP/zoEtkgbmDbx9ljYge39Z/7M0CffxdX WPyX0xWmr5UIJLhIy8+n0ud7Er1keCPCY3ZkAqtuCYhLYUlhQ/vCVBITfrAxtw/hp9DT w60g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733968659; x=1734573459; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=UIIEH4/iFa3/TdDh0MQgQz7IigAr1CH+cv3VD+qL854=; b=MnOvFfvvFMjfuSFKAPzAE+EfvqTGLOWN29ZoMvJwyobNW34pn2wII7CQCH/pQPdWtb eDL4jH5qLSkHnee1A6nMiMlb8kWLbkwqCmP/IKERyrtnMcLEFDKAQ1QgsZ3AGzwr18fy ZEmFiDp+Qo+5jhdt/+s0H5S7M3QQGUUP1R7Zzw8o21JX36fyIUsUIpGuiTGl7+AGT3Tf LCShXRk95bhHSnRTbGPSkpPEtwPAxFWspN+FAbKqg42a37EETrMaqq2QFVBgs7MPq8KJ zUEvTejIjiR6aw7if1fZlPJ0+R4GEC8Bxoa8HUMNafFT3nqFyN8A7DokLOKXm6xssmuz MRuQ== X-Forwarded-Encrypted: i=1; AJvYcCUyudaP5fqcWrKBWiVJ8sDWj+U+7TjYEvILHiV/ovg9adEj0s1XFo7Mkr8Lk+7v1g63CT0pM4wyceQ=@vger.kernel.org, AJvYcCXXo76xGX71JTLD03xPU6gSZ2SwyEEHtrfpkQ7CaXmoh3Hc+xcZZvJsf08YNT9XLnjy+lWQfqA2nU1xxa4=@vger.kernel.org X-Gm-Message-State: AOJu0YyLb+k+ty270Xi7yGqwN2TnVZNrcs2mrQ4Qc6nAJaPPMZNqqueC cgYSa7DC6YU9h6tFFt5wXCopU5HyUZISp2RB5mdbUjfx9He911o4 X-Gm-Gg: ASbGnctHHNJHiJwq3KXFHT7iZ9OtkPjNCwQDaMaRzZSqjqEMi4igHqrXxZWhU+7NOms ldws5n/rAur/QsJvEg7D9B87MNtUNV5xmg/SQ2+84FuqxwK0Y8q8vS6ZD3C7ShXZ3G7xW5dNj9N ID/gGezZ7U2WMqFqd2zvbOvbFB+abFXtFzZqc7K3une8mHUYTVikYitErGZr8B8HDKaxWlBC7Oy X/afZCGqys/kqHYnZfz9CH9YF9gJU6poQQ= X-Google-Smtp-Source: AGHT+IE9I1p8dQJuK436AUBm2uffVmP97hAVngfmOadxV6DnkxeR2VumK0NurxR6A846OfPDvOn9Ag== X-Received: by 2002:a05:6a21:32a5:b0:1e0:d867:c875 with SMTP id adf61e73a8af0-1e1cebdeb94mr2059856637.36.1733968659061; Wed, 11 Dec 2024 17:57:39 -0800 (PST) Received: from sultan-box.lan ([142.147.89.231]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7fd41a128ebsm7086456a12.85.2024.12.11.17.57.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Dec 2024 17:57:38 -0800 (PST) Sender: Sultan Alsawaf From: Sultan Alsawaf X-Google-Original-From: Sultan Alsawaf (unemployed) To: Cc: "Sultan Alsawaf (unemployed)" , "Rafael J. Wysocki" , Viresh Kumar , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/2] cpufreq: schedutil: Fix superfluous updates caused by need_freq_update Date: Wed, 11 Dec 2024 17:57:32 -0800 Message-ID: <20241212015734.41241-2-sultan@kerneltoast.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241212015734.41241-1-sultan@kerneltoast.com> References: <20241212015734.41241-1-sultan@kerneltoast.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Sultan Alsawaf (unemployed)" A redundant frequency update is only truly needed when there is a policy limits change with a driver that specifies CPUFREQ_NEED_UPDATE_LIMITS. In spite of that, drivers specifying CPUFREQ_NEED_UPDATE_LIMITS receive a frequency update _all the time_, not just for a policy limits change, because need_freq_update is never cleared. Furthermore, ignore_dl_rate_limit()'s usage of need_freq_update also leads to a redundant frequency update, regardless of whether or not the driver specifies CPUFREQ_NEED_UPDATE_LIMITS, when the next chosen frequency is the same as the current one. Fix the superfluous updates by only honoring CPUFREQ_NEED_UPDATE_LIMITS when there's a policy limits change, and clearing need_freq_update when a requisite redundant update occurs. This is neatly achieved by moving up the CPUFREQ_NEED_UPDATE_LIMITS test and instead setting need_freq_update to false in sugov_update_next_freq(). Signed-off-by: Sultan Alsawaf (unemployed) --- kernel/sched/cpufreq_schedutil.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index 28c77904ea74..e51d5ce730be 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -83,7 +83,7 @@ static bool sugov_should_update_freq(struct sugov_policy *sg_policy, u64 time) if (unlikely(sg_policy->limits_changed)) { sg_policy->limits_changed = false; - sg_policy->need_freq_update = true; + sg_policy->need_freq_update = cpufreq_driver_test_flags(CPUFREQ_NEED_UPDATE_LIMITS); return true; } @@ -96,7 +96,7 @@ static bool sugov_update_next_freq(struct sugov_policy *sg_policy, u64 time, unsigned int next_freq) { if (sg_policy->need_freq_update) - sg_policy->need_freq_update = cpufreq_driver_test_flags(CPUFREQ_NEED_UPDATE_LIMITS); + sg_policy->need_freq_update = false; else if (sg_policy->next_freq == next_freq) return false; From patchwork Thu Dec 12 01:57:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sultan Alsawaf X-Patchwork-Id: 13904592 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 05091B640; Thu, 12 Dec 2024 01:57:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733968659; cv=none; b=t5inB0rNeCtlE8pjqTmAEQuAJdnB1ahMslI0ZU4HvgFi7xXKcdYnz9PCszWziS63ITpp6UXWNbrJM/6HN2FT0PVHVbquRlMv2dhlYPa/FB+2ybk4MF8eIRsNz9NBQNA1oLAPAh/LdnJFO+UDijQA1zBWMXEToTMfQx/fM31vBaM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733968659; c=relaxed/simple; bh=V6oh0JB1tyWMyrCwjgseP1Lzp1BH76pDoodTXJ0bb9w=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=CU0XkzadtnqjInvTY3rE04b79ZeTnxltoH7HN+/EG3TkoOLUh9+KKj7n4B+unitSP3U+WW0PQBPrD4H8ICbSRJ9HkBunf3IOV9egRBwjDoeNk2amW1c1KY+qHAyWw5pu+ARtSrLvmlioG25uWD4dqCthnPWlmrUS0ahUCXFT3jM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kerneltoast.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XOFVZkBM; arc=none smtp.client-ip=209.85.210.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kerneltoast.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XOFVZkBM" Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-723f37dd76cso88657b3a.0; Wed, 11 Dec 2024 17:57:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733968657; x=1734573457; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:from:to:cc:subject:date:message-id:reply-to; bh=nwG3dseg3dFXG3OHkltH5qxLvvZzBHAhdNLBlAWWT6Q=; b=XOFVZkBMGC9NcpH5OB31OEvwvLIeSP8TxzDl+UAVmrjbql5Du5rG4udYQZ3HD0fuch 0jp8V12hOdoKjdSZUpr8MOkiVzgq8LCnR3vhpGgv1OXIBnWd1pkiIEAIh2YJRraoEIJy esjxoIrQIKhZ9x4y41qaZDwjTgDL6LNmgKX3dskElFR/5sdYpsY5kN5HZU2xd7KEonY+ e5O5nl5B08DNhynW+UV6gPG2/KNBm9EhnYsPQqV03sHsYldPS2FwPL/lf2zcpnmcJu/k nR6jrSarwbVtyqXc6254DO4wZ8XU6UQjkC/RMRj8sYlfuSyUsG6iVabWcZAFJT0yvmoW DNHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733968657; x=1734573457; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=nwG3dseg3dFXG3OHkltH5qxLvvZzBHAhdNLBlAWWT6Q=; b=cbi0DIKgO0V5Ce8PoEQGYi0LD5kixQOYbwXEByAb+c0ay4t5yIuIFstTxe1UCgXjHH f4dkI9nrwdgzKPj+EjDPw9HzOJO6Jx3xllEm08CdB4myQFw8WXKE1mRpEy68TQaC9QyZ 03cLjAg8OKyil7ZhjWsQX73rOmSYW3Z5fs6fjtuNgjTwvbgEwbjaKnhIqJ6lLU23LpyR UGpv8Aj8T0T8tt972rtYsNgPtfMPw2BLCCUu35XTZDuf0HRfwlsApEd+CNJ9/QmSKM5j EQz+5SIH0X1vgWBrn3L8E7/1GT8Ybfw+tK3lSIwbq7ZddeAwYnOnKW2IuCGsTK6r9jsO mNlQ== X-Forwarded-Encrypted: i=1; AJvYcCUOUIhW9iMgl4ehFnSr1FGLdgj4G/2yG0b+pTH/8c5LU406sF0SA4q5kJy/pDTgYMYY+c2Gz8U0gyU=@vger.kernel.org, AJvYcCWROZYX0rOsb9ofUHGqlqTReOgYzZgFi7CiOHxlHiUtHJu+VaCfWGKdrogu2+f/aC92jl/XAbS/PPVP4Uo=@vger.kernel.org X-Gm-Message-State: AOJu0YyanmlRQcld8TTOLZf7ogNEYe0n/pEjfqebNdF6FmX3isDchpnP dVibHEMH9maOFORbvKkle9MHAXbip+0VzoizpdQYMRP7c5yYQC6S X-Gm-Gg: ASbGnctP1wvrszL86salweISkT3ynfi+dBzlAX671ZjZAOLzKtDjVKR65Qt/hSC51P1 hwYOtzEkQKuMu+Ehz+9oxQrdiDB39YTk+ld6bt+qQHMYqynEJVC1d16s39mBnjQpi1doHrCULmf tECm3j/b8GugbrirMDuIlC7aw3NrDW97GqAuBNWKKxlDH1l+NhOUyiLp8lEBblAgTJlgzDDXVhP c/KgBXw66ZWPVSl3C4NUKKz9LAe63O50bI= X-Google-Smtp-Source: AGHT+IHFz1p986CrolQn3NjI/x8hFPs8BhwhQZregchadiI4OpGY1ZYBH+sFDLZjfFGdCFGPNpGw1g== X-Received: by 2002:a05:6a20:72a4:b0:1e1:a932:4a3f with SMTP id adf61e73a8af0-1e1cea7ff33mr2400400637.3.1733968657226; Wed, 11 Dec 2024 17:57:37 -0800 (PST) Received: from sultan-box.lan ([142.147.89.231]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7fd41a128ebsm7086456a12.85.2024.12.11.17.57.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Dec 2024 17:57:36 -0800 (PST) Sender: Sultan Alsawaf From: Sultan Alsawaf X-Google-Original-From: Sultan Alsawaf (unemployed) To: Cc: "Sultan Alsawaf (unemployed)" , "Rafael J. Wysocki" , Viresh Kumar , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/2] cpufreq: schedutil: Ignore rate limit when scaling up with FIE present Date: Wed, 11 Dec 2024 17:57:31 -0800 Message-ID: <20241212015734.41241-1-sultan@kerneltoast.com> X-Mailer: git-send-email 2.47.1 Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: "Sultan Alsawaf (unemployed)" When schedutil disregards a frequency transition due to the transition rate limit, there is no guaranteed deadline as to when the frequency transition will actually occur after the rate limit expires. For instance, depending on how long a CPU spends in a preempt/IRQs disabled context, a rate-limited frequency transition may be delayed indefinitely, until said CPU reaches the scheduler again. This also hurts tasks boosted via UCLAMP_MIN. For frequency transitions _down_, this only poses a theoretical loss of energy savings since a CPU may remain at a higher frequency than necessary for an indefinite period beyond the rate limit expiry. For frequency transitions _up_, however, this poses a significant hit to performance when a CPU is stuck at an insufficient frequency for an indefinitely long time. In latency-sensitive and bursty workloads especially, a missed frequency transition up can result in a significant performance loss due to a CPU operating at an insufficient frequency for too long. When support for the Frequency Invariant Engine (FIE) _isn't_ present, a rate limit is always required for the scheduler to compute CPU utilization with some semblance of accuracy: any frequency transition that occurs before the previous transition latches would result in the scheduler not knowing the frequency a CPU is actually operating at, thereby trashing the computed CPU utilization. However, when FIE support _is_ present, there's no technical requirement to rate limit all frequency transitions to a cpufreq driver's reported transition latency. With FIE, the scheduler's CPU utilization tracking is unaffected by any frequency transitions that occur before the previous frequency is latched. Therefore, ignore the frequency transition rate limit when scaling up on systems where FIE is present. This guarantees that transitions to a higher frequency cannot be indefinitely delayed, since they simply cannot be delayed at all. Signed-off-by: Sultan Alsawaf (unemployed) --- kernel/sched/cpufreq_schedutil.c | 35 +++++++++++++++++++++++++++----- 1 file changed, 30 insertions(+), 5 deletions(-) diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index e51d5ce730be..563baab89a24 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -59,10 +59,15 @@ static DEFINE_PER_CPU(struct sugov_cpu, sugov_cpu); /************************ Governor internals ***********************/ -static bool sugov_should_update_freq(struct sugov_policy *sg_policy, u64 time) +static bool sugov_should_rate_limit(struct sugov_policy *sg_policy, u64 time) { - s64 delta_ns; + s64 delta_ns = time - sg_policy->last_freq_update_time; + + return delta_ns < sg_policy->freq_update_delay_ns; +} +static bool sugov_should_update_freq(struct sugov_policy *sg_policy, u64 time) +{ /* * Since cpufreq_update_util() is called with rq->lock held for * the @target_cpu, our per-CPU data is fully serialized. @@ -87,17 +92,37 @@ static bool sugov_should_update_freq(struct sugov_policy *sg_policy, u64 time) return true; } - delta_ns = time - sg_policy->last_freq_update_time; + /* + * When frequency-invariant utilization tracking is present, there's no + * rate limit when increasing frequency. Therefore, the next frequency + * must be determined before a decision can be made to rate limit the + * frequency change, hence the rate limit check is bypassed here. + */ + if (arch_scale_freq_invariant()) + return true; - return delta_ns >= sg_policy->freq_update_delay_ns; + return !sugov_should_rate_limit(sg_policy, time); } static bool sugov_update_next_freq(struct sugov_policy *sg_policy, u64 time, unsigned int next_freq) { + /* + * When a frequency update isn't mandatory (!need_freq_update), the rate + * limit is checked again upon frequency reduction because systems with + * frequency-invariant utilization bypass the rate limit check entirely + * in sugov_should_update_freq(). This is done so that the rate limit + * can be applied only for frequency reduction when frequency-invariant + * utilization is present. Now that the next frequency is known, the + * rate limit can be selectively applied to frequency reduction on such + * systems. A check for arch_scale_freq_invariant() is omitted here + * because unconditionally rechecking the rate limit is cheaper. + */ if (sg_policy->need_freq_update) sg_policy->need_freq_update = false; - else if (sg_policy->next_freq == next_freq) + else if (next_freq == sg_policy->next_freq || + (next_freq < sg_policy->next_freq && + sugov_should_rate_limit(sg_policy, time))) return false; sg_policy->next_freq = next_freq;