From patchwork Wed Mar 27 04:49:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605548 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34A1CCD11DD for ; Wed, 27 Mar 2024 04:50:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CD5316B0098; Wed, 27 Mar 2024 00:50:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C94A76B0099; Wed, 27 Mar 2024 00:50:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB1396B009B; Wed, 27 Mar 2024 00:50:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 93B636B0098 for ; Wed, 27 Mar 2024 00:50:49 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5F2DE1606B4 for ; Wed, 27 Mar 2024 04:50:49 +0000 (UTC) X-FDA: 81941593818.07.6E5F18E Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) by imf29.hostedemail.com (Postfix) with ESMTP id 862C612000B for ; Wed, 27 Mar 2024 04:50:47 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="J1l3/XvX"; spf=pass (imf29.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515047; a=rsa-sha256; cv=none; b=62uoAoUE8ACbTuIUyCCHDtdxgRRFhdyDhdMp/Dap8d92mbnSNLP7Al+uH50mn5cg15nIca Vr2LTkvsl9KkF+1u0tXjQFFg2SzJzg0Ls45aN2tz2AkfsKrCyJ383ROFc8WR1t6WSGlaZp 9yoQc8qNBtIrwgvT7qNKlxGHAWIbyrA= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="J1l3/XvX"; spf=pass (imf29.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515047; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2fwmOl0CxtBfQxlaR9NQyd7IGBnsfsK+MLDyKM1aSvU=; b=KpnH72bMpJklXOaVwWWiYeurXCtZeH+uFSU0oZOr54nFB3Ecs/PoB4KJ0usZ1GFxNkgEJo ZkvyeArkjaQ+cbGlr+3KYaj5h8J3s/g305Ov2BejUMOw/gT1l8PTKfEUkLPnvb5qTzYYCG hDvtjetpAiIYuPXAHSE8SMpUqXFJalw= Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-6e6ee9e3cffso363724b3a.1 for ; Tue, 26 Mar 2024 21:50:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515046; x=1712119846; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2fwmOl0CxtBfQxlaR9NQyd7IGBnsfsK+MLDyKM1aSvU=; b=J1l3/XvXFYxCwlDbvmmJQ8kENx29feo+qSrpnp4nFuY2Cy3Ft4B1UGIl/mS6+N2v4r eF5BWYyxKqomnFqRRqvIyLDnbz8NOBqcKVOCntuBVEhMIHyI+JtwOi9X6SIr5lHXtusv zLS7F90aFfu+JhEbUbu5HX6Z1JNxWetdD6Ulc8WqxGY/dgUH429a4bv3C41ZmflK4dmh ME5q8OlR0loM/+yUjXAEoPNSOqdBoTs0qnPZQR74hmtP45LDONm7ncYc7hxEnGdODzo7 qVGsKVKCRLGZOEFFC2TzQUh0ytmOjczY9QT9P1N93lQDEhugAwtHdBDU2Lk0yvi54yWJ +oaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515046; x=1712119846; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2fwmOl0CxtBfQxlaR9NQyd7IGBnsfsK+MLDyKM1aSvU=; b=hv/SPOYEGwfyJGQpPKS0b23cXk+pL3WZPfyf1pCpSr3w6DvbglWYuXlVOcHKnQk+ym W4/ile7LHQGe9klDBnRxFl+E/iFSmo8YtRc7U9595xexG/EbJA7hS1+oZc4HKX4i/G11 klc00B5eJ9upQH+dyoXxKnr+IBr6q0psJyLIIXxed4hj3DyR3BM/nGjxE6Y/G+5EFEz6 1N0ZkzTn8WOPBHSXZhJRV3J2x4uoWIzh9gWXIOwHfDT0WY13afBlznLx92peWpg3TS96 AIGi4ftRrl3tqKwCA8CkSXeY0KiuL72Lw3s9dT9ZgZCxNddygCtbQ8KM++UAj67Hu1mW ikeg== X-Forwarded-Encrypted: i=1; AJvYcCX5gIFYemDMmzMNblcBYdFoIYiejB5AmpScfls+d+tifTpDd6AC0VDn6PbwLatQ88oJJFCoE0eOSnUxxfR6n4vB3Oc= X-Gm-Message-State: AOJu0YwsjrHE0AnGIjoKY+HtWXOrsvvNhkwghmsFwZeYtHqGxp2LmiIs DCAykWTAIIU1O+SVgIHW29+NbybeQbtAsPj1m0Ox+Tg08sy7Qq0bBY3TyQ4RdfY= X-Google-Smtp-Source: AGHT+IGHAmpkwghveoQCxqBYVJp1RVR15SSZ2ypXvD8LXUp94lm0FkTd5l29fFm2kWdFZwCld1auyw== X-Received: by 2002:a05:6a20:3d11:b0:1a3:6a11:4898 with SMTP id y17-20020a056a203d1100b001a36a114898mr4924839pzi.3.1711515046438; Tue, 26 Mar 2024 21:50:46 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.50.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:50:45 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v6 04/13] riscv: mm: Broadcast kernel TLB flushes only when needed Date: Tue, 26 Mar 2024 21:49:45 -0700 Message-ID: <20240327045035.368512-5-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 862C612000B X-Stat-Signature: bkpbbx86943nb8bssc7gducjdkkp5dgm X-Rspam-User: X-HE-Tag: 1711515047-875262 X-HE-Meta: U2FsdGVkX182nEs2mz7G8j3Gf7TJm4A+rv0cuviL4/oKjoyBsSlv1ewNnMC/SEtV2XWz+aEKkNZRnuW/U9sW6M/xibkqZQWUIEHpXOCZoEnz5mTSRY6a0Z33Pe4UZlgfHRcEGV+JoeKx1epKZgEmyRFchAk3TpE3cKZDRMeyp83EuS7E2htvuKQrrcPByIL3FjtLG5V7WZwCfBMeY88WMV4euQxzdq1kG//IEjfLNKQjnq+7FHqxVr2/m8Q/Q2bCA0bNuHXzef9ubfAhclIAgxop9E9Fa02J8+nQVqLpXICxX89yJRre0hq4VrRam4X6VU7+oGgG3sn/RU+go7/vsV2NOZYFZiK46fo4SnBUVCnh2gUjpZhTI8ee0D6Zqo5S0bgyQuQZQhEKb1Xj8/o+9AGLBJUZeXg6j3azmOu+gyV65qM8B8cmvEFzZMlck78CudZO3QQ2pDlcRNeJJJy4hkz8xJmT3RYyKhTEX/rh16roHh20zNOAg83w8T37lln0DCERjhkt8JSnRINvR7TqjTilpzQFXy71CuhO4iMo87JXGHffNXKERTEbQVVjnZGMyn5tr9gVlt4ZhUAxlGIG3SUrMdGtQwv9ozcdRlyEF7fieSrY+b1XL9DMtoSEqYLpjFyt8MmoVIeiIXp9m02PMNAnD+eYjiSVMoq8M40rXsiGMNk0358879dQ4nkHv6BHT9IOYmis+Eft61unSPOu2NrGAIlr/0QMwfmxF0aCkDRx/Mej8ED1C2Mrr9/+wqvpxgVcagosdOlPMERTjudsTtskGnaF4pj2wiCym4OlkmMOMIvjg2lnd++LZtANMIFZKt+wIpEjuPZY8GEuRYl9QplvPYFMC+8sPm4yifE/k2fQHv2azYv6G8iOG+jEm5RaE1lbfBngjJPv3WO5pOZ1ZKr8kwq3ZxLVDQQ0UrzvOnmDrhfJr5Fqv9kIfxBF1yQnc5TMPZNqud6nFOjGAnb H19RGHBL Pc2riEbHbgDLletcj1iNqvwoqf5Y2VPCvS6WWqHqzqIqkiG+ACMsgDMne0THZQG0p70SBbf00jC0RMKx2QRVeRVoYm2wCW/hFsGPQvZnsr/dMmX8ttkAYnOy8JboSqioHc4O1EncEkcQvKLdMPfJzt61hKighLFqvz/W9867xnyx8T6rmB8fc1W9iBMXFK9xZcqZ3eyZEdsmCzqfWfd79pqif999dSeWDVhP2cHAWBJ5WENVWpqDXlsmy4iOLey4arWVoiZKmsS2PYDz51vDr0oGIGyS2Sdosc2f5+j8IyR3JnOls0PF4K7ok4nRmXn/VviXp6P5SNmNo+q4eMvhye9g0SSJ2DfWawMsPYTx3MKDHI2Bj46Kyw7H2/OSU9Nby5AcrZTA59cv2m2XcTldkbY8q9yLIOGVBLOh718qrUcA4m5M1UCfAYS/Pl7MyA46lZYU81snHSUrYHyo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: __flush_tlb_range() avoids broadcasting TLB flushes when an mm context is only active on the local CPU. Apply this same optimization to TLB flushes of kernel memory when only one CPU is online. This check can be constant-folded when SMP is disabled. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v4) Changes in v4: - New patch for v4 arch/riscv/mm/tlbflush.c | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 0435605b07d0..da821315d43e 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -103,22 +103,15 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, unsigned long start, unsigned long size, unsigned long stride) { - bool broadcast; + unsigned int cpu; if (cpumask_empty(cmask)) return; - if (cmask != cpu_online_mask) { - unsigned int cpuid; + cpu = get_cpu(); - cpuid = get_cpu(); - /* check if the tlbflush needs to be sent to other CPUs */ - broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids; - } else { - broadcast = true; - } - - if (!broadcast) { + /* Check if the TLB flush needs to be sent to other CPUs. */ + if (cpumask_any_but(cmask, cpu) >= nr_cpu_ids) { local_flush_tlb_range_asid(start, size, stride, asid); } else if (riscv_use_sbi_for_rfence()) { sbi_remote_sfence_vma_asid(cmask, start, size, asid); @@ -132,8 +125,7 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, on_each_cpu_mask(cmask, __ipi_flush_tlb_range_asid, &ftd, 1); } - if (cmask != cpu_online_mask) - put_cpu(); + put_cpu(); } static inline unsigned long get_mm_asid(struct mm_struct *mm)