From patchwork Thu Jul 9 10:13:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viresh Kumar X-Patchwork-Id: 11654003 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 955C214DD for ; Thu, 9 Jul 2020 10:15:59 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6E1E220674 for ; Thu, 9 Jul 2020 10:15:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="J3oe3hOG"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="OmDlpJ0f" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6E1E220674 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=QyNx3aU9ledWm9yavXcDhzvxS2nLt2gnFFC1OCUrLys=; b=J3oe3hOGbCDLWank0U+26JUZ0 xpaKZZ93NHjaLFhRjEcMGo1KCFXTkriIoHPBM7lYHRR/kpTD2vEeEqJIGRC320KyBCMDxIOmUGxKG ZmwtD+c2knhBuLPcKBByhIURf6h+5ZgNeaE//gSYiFOnVzMyXryK6bbU9MdeRxoXL7pziLiQYQ/kD 8EM8jsn60SLVygPva571sS5XgW/XconB+ebr2QdByGDVdW4p6O+ZDiakiWZSZLz8pRy33fV9YF24a 8Zhx4t60ekayGqyxTx2tEOt10Ze4Vna2GDYgNOwHcguHjGTrSVsFpidwgLvUAr9niL6a+ZzrFXt83 HBc36lqZQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jtTZM-0004ZM-4x; Thu, 09 Jul 2020 10:14:20 +0000 Received: from mail-pl1-x644.google.com ([2607:f8b0:4864:20::644]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jtTZF-0004WO-Bt for linux-arm-kernel@lists.infradead.org; Thu, 09 Jul 2020 10:14:14 +0000 Received: by mail-pl1-x644.google.com with SMTP id b9so652329plx.6 for ; Thu, 09 Jul 2020 03:14:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yaA0NjXT9ejZD1re0oAP+uDPoFeFqxhvljIg3jQh5Fg=; b=OmDlpJ0f/w2TVYbbJkMRYc+V1bqmuarLLa/EookrkuDVPdoxPOaoTwW2FdIo6o18Qq Y+f7Ph0V8W36eKNfyxE7Un2XDMm3T84cN37TqFgz7ABiKCscy8ezDUBDx4fQupfS91nf Y3q6n/AVJfmt1rtTTKaHkRVv0fpTAeqPdSwvSaMxnvC4ei7816Eobex9wc6X9aqOjHH6 rc1avRejayCXlWqTvHDnp0tg6NUjMBqXqbAvvUhLapddHReM7Q53MLxnz+9iqJQSItsQ 5EohM8Mt8Pof2Qtt4SNLRxX4EFpkKCHnlxYifo5cuHuohDjqF+frQPpsGDDJh5htpIhv RZPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yaA0NjXT9ejZD1re0oAP+uDPoFeFqxhvljIg3jQh5Fg=; b=FdTaj8QGJWkbVxX8Sd1FOQ14XpuvT0i1KlcJa3g7oaYkcf8+xPsFRRvsdNWifnzb5j AunBWrCRDVfX1iw/mOHdN7iO3763jMGtzVum3CN8C4D8F5Qdqbc8tCcC+1xq5SnIIaZv EiHbtgxHTQB2MhDcFJJjkSArnomPQG12fpTxrt8BMbAbvW433EIYcGbia6rrYPlNz7T7 8JcO5/ZmHTd00gpF2u8B1yyUmUPxhn6qOIT5ML6rNHwV11mmRcBSZG7fI7k6D7Uel6Wa u+qj9G33gOQFVCCucgpXodvaQw4X/l4qk/Klv0JZhgdg75GeoQbx066loW+lNzsK1ZRi +UxQ== X-Gm-Message-State: AOAM533sbUm0z+kSLvMhsxeFHdkmV1NuGKGGyalKiqqV+DgvCKCs+Wh1 knoIH9UxfMrNdVABSLi3pSjFfQ== X-Google-Smtp-Source: ABdhPJy8HyrcoQ6VzKHy2wCN8zHGO7OHZaG7NF7HUVcRbgP1JwrDTgqUnW0Scs6eadbC6ftvWS1FLQ== X-Received: by 2002:a17:902:a40c:: with SMTP id p12mr43077013plq.51.1594289651367; Thu, 09 Jul 2020 03:14:11 -0700 (PDT) Received: from localhost ([122.172.40.201]) by smtp.gmail.com with ESMTPSA id w1sm2135716pfq.53.2020.07.09.03.14.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Jul 2020 03:14:10 -0700 (PDT) From: Viresh Kumar To: Catalin Marinas , Will Deacon , Sudeep Holla , Greg Kroah-Hartman , "Rafael J. Wysocki" Subject: [RFC 2/3] topology: Provide generic implementation of arch_freq_counters_available() Date: Thu, 9 Jul 2020 15:43:34 +0530 Message-Id: X-Mailer: git-send-email 2.25.0.rc1.19.g042ed3e048af In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200709_061413_421087_E0DE9E24 X-CRM114-Status: GOOD ( 19.36 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:644 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vincent Guittot , Viresh Kumar , linux-kernel@vger.kernel.org, Peter Puhov , Ionela Voinescu , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org arch_freq_counters_available() is implemented only for ARM AMU hardware right now and this patch attempts to make it generic enough so other parts of the kernel can also register their own implementation of topology_scale_freq_tick() routine. Signed-off-by: Viresh Kumar --- arch/arm64/include/asm/topology.h | 7 --- arch/arm64/kernel/topology.c | 81 +++++++++++++++---------------- drivers/base/arch_topology.c | 43 ++++++++++++++-- include/linux/arch_topology.h | 5 +- 4 files changed, 81 insertions(+), 55 deletions(-) diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h index 0cc835ddfcd1..93cca6a8cde3 100644 --- a/arch/arm64/include/asm/topology.h +++ b/arch/arm64/include/asm/topology.h @@ -16,14 +16,7 @@ int pcibus_to_node(struct pci_bus *bus); #include -#ifdef CONFIG_ARM64_AMU_EXTN -/* - * Replace task scheduler's default counter-based - * frequency-invariance scale factor setting. - */ -void topology_scale_freq_tick(void); #define arch_scale_freq_tick topology_scale_freq_tick -#endif /* CONFIG_ARM64_AMU_EXTN */ /* Replace task scheduler's default frequency-invariant accounting */ #define arch_scale_freq_capacity topology_get_freq_scale diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index 74fde35b56ef..97741da31b6d 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -188,6 +188,41 @@ bool amu_counters_supported(void) cpumask_equal(valid_cpus, cpu_present_mask); } +void amu_scale_freq_tick(void) +{ + u64 prev_core_cnt, prev_const_cnt; + u64 core_cnt, const_cnt, scale; + + const_cnt = read_sysreg_s(SYS_AMEVCNTR0_CONST_EL0); + core_cnt = read_sysreg_s(SYS_AMEVCNTR0_CORE_EL0); + prev_const_cnt = this_cpu_read(arch_const_cycles_prev); + prev_core_cnt = this_cpu_read(arch_core_cycles_prev); + + if (unlikely(core_cnt <= prev_core_cnt || + const_cnt <= prev_const_cnt)) + goto store_and_exit; + + /* + * /\core arch_max_freq_scale + * scale = ------- * -------------------- + * /\const SCHED_CAPACITY_SCALE + * + * See setup_freq_invariance() for details on + * arch_max_freq_scale and the use of SCHED_CAPACITY_SHIFT. + */ + scale = core_cnt - prev_core_cnt; + scale *= this_cpu_read(arch_max_freq_scale); + scale = div64_u64(scale >> SCHED_CAPACITY_SHIFT, + const_cnt - prev_const_cnt); + + scale = min_t(unsigned long, scale, SCHED_CAPACITY_SCALE); + this_cpu_write(freq_scale, (unsigned long)scale); + +store_and_exit: + this_cpu_write(arch_core_cycles_prev, core_cnt); + this_cpu_write(arch_const_cycles_prev, const_cnt); +} + static int __init early_init_amu_fie(void) { int cpu; @@ -230,9 +265,11 @@ static int __init late_init_amu_fie(void) if (!cpumask_empty(amu_fie_cpus)) { pr_info("CPUs[%*pbl]: counters will be used for FIE.", cpumask_pr_args(amu_fie_cpus)); - static_branch_enable(&amu_fie_key); + if (!topology_set_scale_freq_tick(amu_scale_freq_tick, amu_fie_cpus)) + pr_info("Registered amu_scale_freq_tick()\n"); } + free_cpumask_var(amu_fie_cpus); return 0; } late_initcall_sync(late_init_amu_fie); @@ -242,48 +279,6 @@ bool arch_freq_counters_available(struct cpumask *cpus) return amu_freq_invariant() && cpumask_subset(cpus, amu_fie_cpus); } - -void topology_scale_freq_tick(void) -{ - u64 prev_core_cnt, prev_const_cnt; - u64 core_cnt, const_cnt, scale; - int cpu = smp_processor_id(); - - if (!amu_freq_invariant()) - return; - - if (!cpumask_test_cpu(cpu, amu_fie_cpus)) - return; - - const_cnt = read_sysreg_s(SYS_AMEVCNTR0_CONST_EL0); - core_cnt = read_sysreg_s(SYS_AMEVCNTR0_CORE_EL0); - prev_const_cnt = this_cpu_read(arch_const_cycles_prev); - prev_core_cnt = this_cpu_read(arch_core_cycles_prev); - - if (unlikely(core_cnt <= prev_core_cnt || - const_cnt <= prev_const_cnt)) - goto store_and_exit; - - /* - * /\core arch_max_freq_scale - * scale = ------- * -------------------- - * /\const SCHED_CAPACITY_SCALE - * - * See setup_freq_invariance() for details on - * arch_max_freq_scale and the use of SCHED_CAPACITY_SHIFT. - */ - scale = core_cnt - prev_core_cnt; - scale *= this_cpu_read(arch_max_freq_scale); - scale = div64_u64(scale >> SCHED_CAPACITY_SHIFT, - const_cnt - prev_const_cnt); - - scale = min_t(unsigned long, scale, SCHED_CAPACITY_SCALE); - this_cpu_write(freq_scale, (unsigned long)scale); - -store_and_exit: - this_cpu_write(arch_core_cycles_prev, core_cnt); - this_cpu_write(arch_const_cycles_prev, const_cnt); -} #else bool amu_counters_supported(void) { diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c index 4d0a0038b476..3820109864c1 100644 --- a/drivers/base/arch_topology.c +++ b/drivers/base/arch_topology.c @@ -21,11 +21,48 @@ #include #include -__weak bool arch_freq_counters_available(struct cpumask *cpus) +static void (*scale_freq_tick)(void); +static cpumask_var_t freq_tick_cpus; + +int topology_set_scale_freq_tick(void *ptr, const struct cpumask *cpus) { - return false; + if (scale_freq_tick) + return -EBUSY; + + if (!zalloc_cpumask_var(&freq_tick_cpus, GFP_KERNEL)) + return -ENOMEM; + + cpumask_copy(freq_tick_cpus, cpus); + scale_freq_tick = ptr; + return 0; } +EXPORT_SYMBOL_GPL(topology_set_scale_freq_tick); + +void topology_remove_scale_freq_tick(void *ptr) +{ + if (scale_freq_tick == ptr) { + free_cpumask_var(freq_tick_cpus); + scale_freq_tick = NULL; + } else { + pr_err("%s: Invalid argument\n", __func__); + } +} +EXPORT_SYMBOL_GPL(topology_remove_scale_freq_tick); + +void topology_scale_freq_tick(void) +{ + if (scale_freq_tick && + cpumask_test_cpu(raw_smp_processor_id(), freq_tick_cpus)) + scale_freq_tick(); +} + +static bool support_scale_freq_tick(struct cpumask *cpus) +{ + return scale_freq_tick && cpumask_subset(cpus, freq_tick_cpus); +} + DEFINE_PER_CPU(unsigned long, freq_scale) = SCHED_CAPACITY_SCALE; +EXPORT_SYMBOL_GPL(freq_scale); void arch_set_freq_scale(struct cpumask *cpus, unsigned long cur_freq, unsigned long max_freq) @@ -38,7 +75,7 @@ void arch_set_freq_scale(struct cpumask *cpus, unsigned long cur_freq, * want to update the scale factor with information from CPUFREQ. * Instead the scale factor will be updated from arch_scale_freq_tick. */ - if (arch_freq_counters_available(cpus)) + if (support_scale_freq_tick(cpus)) return; scale = (cur_freq << SCHED_CAPACITY_SHIFT) / max_freq; diff --git a/include/linux/arch_topology.h b/include/linux/arch_topology.h index 0566cb3314ef..6305148c8aa0 100644 --- a/include/linux/arch_topology.h +++ b/include/linux/arch_topology.h @@ -10,6 +10,9 @@ void topology_normalize_cpu_scale(void); int topology_update_cpu_topology(void); +void topology_scale_freq_tick(void); +int topology_set_scale_freq_tick(void *ptr, const struct cpumask *cpus); +void topology_remove_scale_freq_tick(void *ptr); struct device_node; bool topology_parse_cpu_capacity(struct device_node *cpu_node, int cpu); @@ -30,8 +33,6 @@ static inline unsigned long topology_get_freq_scale(int cpu) return per_cpu(freq_scale, cpu); } -bool arch_freq_counters_available(struct cpumask *cpus); - DECLARE_PER_CPU(unsigned long, thermal_pressure); static inline unsigned long topology_get_thermal_pressure(int cpu)