From patchwork Thu Feb 29 23:21:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FEC4C54798 for ; Thu, 29 Feb 2024 23:22:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D4306B009B; Thu, 29 Feb 2024 18:22:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 85E036B009C; Thu, 29 Feb 2024 18:22:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D6076B009D; Thu, 29 Feb 2024 18:22:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5DA2A6B009B for ; Thu, 29 Feb 2024 18:22:20 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 0EA3BA15CC for ; Thu, 29 Feb 2024 23:22:20 +0000 (UTC) X-FDA: 81846417240.20.8D71317 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf07.hostedemail.com (Postfix) with ESMTP id 664E840006 for ; Thu, 29 Feb 2024 23:22:18 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=TUnIFIhB; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf07.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709248938; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1/3iO4NfkALka4Buc8Img0QDYdUTnf5/4SNG0fcN5rQ=; b=SKGgge5U5sSIXu5gVRDK6RbvhFP/C7jHeStPar6ICNjE67Y5/5FZYa7j17JXIIGr2cg2q4 tG6Tu9aY1Mx+s27Cdwq6wEeJ2pSsvuaoJix4S+0nhh5/Go1rpBlY/U2OUleFhpWQ2ucAgh NXFgOT+1Il51V2twxw1rC77sSQHcIko= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=TUnIFIhB; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf07.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709248938; a=rsa-sha256; cv=none; b=Y8qFzPwS9/bQ/Ln/j9mT8mmbHBkBxpfZpzA3Feiuz4GbROLiITUJW6EjCsgpRtwwxX44Sw Wn7x7+ts7nOLtbb7NxKJA/vQ4CDFIKjPHlPMbUg/jH6xaAmSMs7eRHxshCPgxiIm6en4Fm QqTILjUomEBYr2WOSTRfhmPX+Wd5tUs= Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-6e46dcd8feaso903331b3a.2 for ; Thu, 29 Feb 2024 15:22:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248937; x=1709853737; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1/3iO4NfkALka4Buc8Img0QDYdUTnf5/4SNG0fcN5rQ=; b=TUnIFIhBW10F4zpw6+FCBFW82zSfHJO4zq/BBuhLIN1OlKrwONZeUqoPvqX8Kf682I eFwLgwjrk9SflZXI67b85FIc7s9E/vdz4IN7LYRvE7/HCoyhjCQ0FyA0D1NxJb296E6U rt/OYwi7x3A8sZpufpzEySko1Wj/pVlcXY/3V9nwIFJ4I7M5R6fip3DDmzUQTD+wfcGV zp1touQn/Jz1JDeBM8r/9nYY+hcorYmiIzrnzUIn0sgGuO7JJn485cHiFY1VeCZ+8XVg MkxcMs6H2ruWFDKIYK7O6ias3VJi00X/FCJsZvPD0tcK4f1iS6jY7mhz75YXyv/bYUVS 4Aww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248937; x=1709853737; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1/3iO4NfkALka4Buc8Img0QDYdUTnf5/4SNG0fcN5rQ=; b=itmdT3WHgiEK5ZbP2TT7jhkxi0FW7CYj4527VFCyK+XakEx6dQoP/1UmHbhpkmilHE Qol/m5Oe5+6FA/U9wlYYq3Yk+VsxkC4Y96PvNsQFH4J/DLOeFbdlb3V1AZmhcSF+xAqR 4vSDPdJXZ5PL0xg7LiBYNj/57jOIIsV/ZR4aAtai/yg32UGLdNXFRsT+O+0CCODFpgqr 5q9E9cOz/CQSmIqXTbNh9K2zJg2y5K0eZY5z6ghDwLZ4Wbt2L6jHkWJnH1cI5MnA9lfY xlZOqHsIG+1R+7tsBJe0HxdOEOIxooBDEqFMX/A7+f0D/cLp3MFRjyWcVvn3jntv8rP1 WxQg== X-Forwarded-Encrypted: i=1; AJvYcCW5JZu7YV2mwRpg3KcSnuuQGhHI7Y7uT0dleXk2Cr3tjj7dMp9d/EFJyh7KxnfLaivvw1De1ii0cbr1jgfXIRMfimg= X-Gm-Message-State: AOJu0YzKKJHq7H0H1D8tm8YW3Xm2INGMFr3/UgfomMRtWtcQQ6HX27GV JFdEh4+TvyWhVpOCFfFl6ZEM3M2Y4JBIiMFYc2SgA2zMzn/KM40pBKi7GnLljuw= X-Google-Smtp-Source: AGHT+IHEWuHXVZ/LoNCuQjq9ikFwqlpYM8ivEnjNJqMVh5lQ2hbDE0NnP51hssPDjMbEUOy86+sdSg== X-Received: by 2002:a05:6a00:14cf:b0:6e5:a996:5201 with SMTP id w15-20020a056a0014cf00b006e5a9965201mr172471pfu.10.1709248937282; Thu, 29 Feb 2024 15:22:17 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:16 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 04/13] riscv: mm: Broadcast kernel TLB flushes only when needed Date: Thu, 29 Feb 2024 15:21:45 -0800 Message-ID: <20240229232211.161961-5-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 664E840006 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: xmaf4ufxwwd48cr5exo1ymbjjr7qqfio X-HE-Tag: 1709248938-941850 X-HE-Meta: U2FsdGVkX19Zt2basboWfXrau6Gj1fctC6qGDmcMs9OWHKHKH/yGyvZMStQbRWg1R91JVZCL2548FG3MvoIoLAbb6eE+4KUw5An34daOzxrDVRCk/apsM8Y8lwP4TiK/0yLuSuybpMl5Lkn+rnAPm5Uqwf9G3pycn8Mlxm/kZQbyVB9wqnPshiUqY0o7Qbj9bFPOPWAREz8UE5Za5cidxP2lIousp4cXA427mMqZ+n6fhJ0EP5HErREU/sTtnx745K+m6DWLUZAt84Bm8Gd70rkeqYlEvxpjMoFdqB7AAgVK+YJVusD5bCYn+wxQPamNxDoLNKd0Pbd9TWi5quvkrynjLA0DglFjd7NQ1Bjl9eVWOj8zwZ9KXZ5inVxnEdMjn+DrX7uZzz/+tY5iM5HEc4FJG4gXQw8y9k0JeBHe8jHHz97Xo3P32n1pCQybsJVIGRtULlnGkcKHP/Xa1rk8IAJEyVAwgKTan2RK1kgR+XQR5YclNO8oHVOHilrLnJorWMJDczuuJOYkJIJflQMdjLOEA9ozpcBR8GBylc5EGL9iENPBcBV1ztIlPpGlAprFCuJwPb10vZmP46yx/uJGwLc/hJAQQgrK79VZjovOw/VnJ3KJH9zlDmknZhYh0T1Mg2zzD0qex4NQeH6tbtECNw3SWd6ZLQsBc3ZO50WLpFzV1nDXblNLkXxchxcUDHVRBCBAR+vafnxSp1WjAx6BTQ20FWWP3DGfneDS6fMdKMAUycCnZXgoqoFRtdWIFSzu1rrQk57uZV0+MFhlBFZYAw4UGemCGuPAyWMIyEZJmzNGl5OrfM1cjVVy+5Xz9u5DUqXha3Nd+SM2d33efsBFZlAoRIstPv07zUt5f7KFUk6yeep526d5hDqVJdn/HIe4wq8VfS5wQt9TbRks5T6ceeZqvYzDH4acXKZuz18HG0qvT8GPQvoa69xH/A2Osv6M7SIquz9FxhJusdPQUqp v+Q6fEZL WOEIUS9OPeYmvHBGNNimRXdlLZLLRFubrKe7OFR2klUezsCyADEZR6OaIQY0ZA62nrmwg/kWjVsc8oAEC+xDhoIPj6lI5wGfTuKdm7iwM+oNuTtQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: __flush_tlb_range() avoids broadcasting TLB flushes when an mm context is only active on the local CPU. Apply this same optimization to TLB flushes of kernel memory when only one CPU is online. This check can be constant-folded when SMP is disabled. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v4) Changes in v4: - New patch for v4 arch/riscv/mm/tlbflush.c | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 0373661bd1c4..8cdb082f00ca 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -102,22 +102,15 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, unsigned long start, unsigned long size, unsigned long stride) { - bool broadcast; + unsigned int cpu; if (cpumask_empty(cmask)) return; - if (cmask != cpu_online_mask) { - unsigned int cpuid; + cpu = get_cpu(); - cpuid = get_cpu(); - /* check if the tlbflush needs to be sent to other CPUs */ - broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids; - } else { - broadcast = true; - } - - if (!broadcast) { + /* Check if the TLB flush needs to be sent to other CPUs. */ + if (cpumask_any_but(cmask, cpu) >= nr_cpu_ids) { local_flush_tlb_range_asid(start, size, stride, asid); } else if (riscv_use_sbi_for_rfence()) { sbi_remote_sfence_vma_asid(cmask, start, size, asid); @@ -131,8 +124,7 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, on_each_cpu_mask(cmask, __ipi_flush_tlb_range_asid, &ftd, 1); } - if (cmask != cpu_online_mask) - put_cpu(); + put_cpu(); } static inline unsigned long get_mm_asid(struct mm_struct *mm)