From patchwork Tue Jan 2 22:00:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13509530 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6BBFC46CD2 for ; Tue, 2 Jan 2024 22:01:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B178C6B0071; Tue, 2 Jan 2024 17:01:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AA0906B0072; Tue, 2 Jan 2024 17:01:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 91BBC6B0078; Tue, 2 Jan 2024 17:01:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7C9596B0071 for ; Tue, 2 Jan 2024 17:01:43 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 5130BA1788 for ; Tue, 2 Jan 2024 22:01:43 +0000 (UTC) X-FDA: 81635743686.14.A75252A Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by imf13.hostedemail.com (Postfix) with ESMTP id 71EA120027 for ; Tue, 2 Jan 2024 22:01:41 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=gCqnm4il; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf13.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704232901; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XA8TOHyyeKTc95PFXJRpYatfRR5oPHMuobG+iziT3HE=; b=QqWLVs1qlxuxt2tS8ivKuzsn8x0kdXQ2soFC1DkTf0LxQY54ufrG55QAe5JG7o+JoD9T6u gsCq8y2DfjMlVyu3PJw/A1kbn6H/u8KLAt2HTjT+cAxtNsOyPpN12vonL0WdgozEbJNVgI avme46BbtHzgIxTR3ID6Go5KAa1Vr/E= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=gCqnm4il; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf13.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704232901; a=rsa-sha256; cv=none; b=hRnjUd5ifV5NGu95H9+xV2qGLjh2pgbQ0q0nDqZPXvCn3gxb2ETCuSAr/6c4ErO8hAVySN 6eZ28aCZg9CwOjDTt0jEMcz5kgL9Eoe+ytn685DE6HuH4rSEAq6yJUHWxcqI7npDfzQwtg mwQw4RPY4ShPUyIj1fAFhb0Bmo8cJGU= Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-28bfdf3cd12so3960751a91.3 for ; Tue, 02 Jan 2024 14:01:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704232900; x=1704837700; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XA8TOHyyeKTc95PFXJRpYatfRR5oPHMuobG+iziT3HE=; b=gCqnm4ilMPvZFHQ9kC7J4ek2IovZhrVpa0gHIuSmf7Rzi1oexpz2j5Eb8JtzqSFSkd SxAXpzKQ1Zh/vMn4VdrCffxM+UYVgwjgud41MsNc+qn/lCnNe+3T1AwVonrvAXvsDfsz KMl3M3mAP5RAoU/mGL+DwaTrZVFjimuktbFaM3FmSzF66cNJjReryKICl/JEJsDdsJF3 b9+r/5jppXwC8uaa4ExBw6oHUkU4+Gr0HPEMQY/41KlBi8t9bmJs0oVLZ6DIUA/VzYnF Bso5OJ+NbhSdZgtp6gV1uyJC+bqvNplSxr9cPrJu/DnRIdtnYkjZ3ra7I84ySqECStgq A8Yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704232900; x=1704837700; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XA8TOHyyeKTc95PFXJRpYatfRR5oPHMuobG+iziT3HE=; b=jMP17b5BsQhH4J3GomR8uhZFrmKu2YAh3fQ5VN3TJ04zYsJoq9rB6XyVT57qfJlHxG NVP/ECbcaybp1Fwn22I6+xw1nsLPxjugmIzOoTaVWOx+zaCPh9yiWl7ztP2E6VM2zT+Q qNJt8uT87ChdwvswjnbwnxoDRxG6cZsJFKZqDeDJ9L2lZwRT7jEki/nsD/KM+ECBlv3k RUCHndJGq8f1rxG3g6F7LEFOmw5OoMLHmo50oylplwIwFH0/LZUTgz2UI9Qs2y3UVPzN 1zjDsfIY/fjg6Q6SrToujoqiWsg363tU5U4uRp8SlkwmFButwLQLgcFlYO3sQkoeSnZR 0hiA== X-Gm-Message-State: AOJu0Yxh6DEWtBCTnc0qDciWC/UW9hvDxgJ4D/lRIx7ASmG+H9b4sCUp 3yopfI1nTuvBHHD4nSo0lwpZ241A6BfG2A== X-Google-Smtp-Source: AGHT+IGXoPWHR5PafwwvCfsK03qnPYeIgtaQPhGwO0vZvHcHemW0jssRCcGF3figFCmbRtQxmnrgGQ== X-Received: by 2002:a17:90a:974c:b0:28c:accc:b1db with SMTP id i12-20020a17090a974c00b0028cacccb1dbmr2272863pjw.44.1704232900428; Tue, 02 Jan 2024 14:01:40 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id r59-20020a17090a43c100b0028ce507cd7dsm101724pjg.55.2024.01.02.14.01.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:01:40 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v4 03/12] riscv: mm: Broadcast kernel TLB flushes only when needed Date: Tue, 2 Jan 2024 14:00:40 -0800 Message-ID: <20240102220134.3229156-4-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240102220134.3229156-1-samuel.holland@sifive.com> References: <20240102220134.3229156-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 71EA120027 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 7t4y5iq6zykxetgac8pui3ijgddfbw9w X-HE-Tag: 1704232901-624241 X-HE-Meta: U2FsdGVkX19T0547pafWRVsGauK+v8RrEWxEWijcIyz20zR95/sB9iy4xoWvHjKUT4hip+R0EmFbkNtXcXBIdwVELn/PPlwo76ZPqjMpLEtfeOIIUSzAdPWzTEx5K8FmuYKEhAxynIItICEFJ9MK5qSQIfHqDR76ASj8MMNlwn4QUEfWZwkfUwURsm1ZVdsa3q3cgSt7tvgF4P8cAaDYlJilbxbwFcq/A5bEGP0dElCo0WeH73H0JLuVAfdnRM++LpJTlAM+zzZe2hGPU5rfd3snN75yxEJPfnw/PVVl/dlelanKAlLrjhqnOSQ4HLpgfDkB/rZjiPuv9P/bPhJXfTt++b4gt6n9uOTdygkrsi1r5O5UBUjfoAG+UgedxMFrJBkmwxiVKS+REgdMgp1BjVyhQBzajN+pft814DwY/wQcxf21dUqt2/6eboKgliSGEwWCg8s9k6pl/zOkzQbNkrm4QOGwAh7QDN7gDJmA308Vq90/3O4GLOJKWqIzndQ+vnQQLYzTUjrzWt2Kb4T5jhBY2lCeIn9UKXcrMkPqDNG8JGX/trircM0MA3jZCTd4BoyUZIwHpqLIh666eB946g//kZrCR2K3n92SKmEr6+rwB/hRKUEjcS5ejff2ERQgTfWvRHdk+nOxKpvlhm3w/lcK8AWTv2IiQjmi2s2eYMnaqUiqq2ytDNkRxtd7H+RMnWuf6U6vJ6TjELoIHQaCXal073G13ISdbSF1pj0TPvU9s8kYI4XZ6cXc38dW7p5wClo+H2thd+Sz8+taN3M6haQVWuMdDDFLVA30qgVcCBNedojGqucGAP9BDzyzVu7u+cC4irxGWu8tUBDW7kdLW7BZOxuZTprhgEtB7PAz8feImxTp25CPpH6NbCr1GcnvXbF3fWTlWh8rkbdwu0bdr0SkjKij5ZMqzQUrJJ/LN2HpeIhMvG6M+vfLjifP6OtzYbG0WMFzYVIlRRb8FHh d/Dj1WZR UcAxduLyKOuKRF28kDTE403JJzr7ZhHaQzoZic90VOyvCqFEZzKOm0rCqJHbLjsO5Z4506lrfljMS9xDIDmZiDi2ZIywzlPXA6qZOK23D4f7lvwqRQRSJ9GBjhQmSlStAWgCKBHzTwWhMjFYOqRP1J/IEmR/PV9p/O2kpE7InWEH4IrhOQqI0X7tYN9ZJKkqsNUAaDH1DTWVNgZH8NSjx8nr/il6lJHFubjEUZFlYpurUXyN2IJMEhv65y0bBeOyXOiA7tP3uhcB+CDCAqqYU8FkveIqhLMCxXe+lPG2/9RZY5HD4b2KruIRRaqa8ek3hp+5hazzU/3lPRckZyLDVxgbCZYkrZOXPrunW1l7Yx+6XxZxdMM4rPGu7OEWhfQGGy6DkGC09O42b353AqaCM3169CnzpjBhi3swqVViPbBi+oTk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: __flush_tlb_range() avoids broadcasting TLB flushes when an mm context is only active on the local CPU. Apply this same optimization to TLB flushes of kernel memory when only one CPU is online. This check can be constant-folded when SMP is disabled. Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti --- Changes in v4: - New patch for v4 arch/riscv/mm/tlbflush.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 09b03bf71e6a..2f18fe6fc4f3 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -98,27 +98,23 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, { const struct cpumask *cmask; unsigned long asid = FLUSH_TLB_NO_ASID; - bool broadcast; + unsigned int cpu; if (mm) { - unsigned int cpuid; - cmask = mm_cpumask(mm); if (cpumask_empty(cmask)) return; - cpuid = get_cpu(); - /* check if the tlbflush needs to be sent to other CPUs */ - broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids; - if (static_branch_unlikely(&use_asid_allocator)) asid = atomic_long_read(&mm->context.id) & asid_mask; } else { cmask = cpu_online_mask; - broadcast = true; } - if (!broadcast) { + cpu = get_cpu(); + + /* Check if the TLB flush needs to be sent to other CPUs. */ + if (cpumask_any_but(cmask, cpu) >= nr_cpu_ids) { local_flush_tlb_range_asid(start, size, stride, asid); } else if (riscv_use_sbi_for_rfence()) { sbi_remote_sfence_vma_asid(cmask, start, size, asid); @@ -132,8 +128,7 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, on_each_cpu_mask(cmask, __ipi_flush_tlb_range_asid, &ftd, 1); } - if (mm) - put_cpu(); + put_cpu(); } void flush_tlb_mm(struct mm_struct *mm)