From patchwork Wed Nov 22 01:07:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13463788 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40C69C61D85 for ; Wed, 22 Nov 2023 01:08:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B9B8C6B0523; Tue, 21 Nov 2023 20:08:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AFCCA6B0525; Tue, 21 Nov 2023 20:08:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C5476B0526; Tue, 21 Nov 2023 20:08:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 8A3906B0523 for ; Tue, 21 Nov 2023 20:08:21 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6352CB5D6C for ; Wed, 22 Nov 2023 01:08:21 +0000 (UTC) X-FDA: 81483804402.26.62374A6 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf27.hostedemail.com (Postfix) with ESMTP id 9064F40010 for ; Wed, 22 Nov 2023 01:08:19 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="mQy/FkxA"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf27.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700615299; a=rsa-sha256; cv=none; b=4MmsP/3ScNJJNPbEAVIopC+Hl1riKzjuJ0dzQ9+FXDD3gz3KhjMPILj8G7VVvcQmfOea1I hGmt2dq4YQVyjG6/fzKdKoEnmmAoeh59O33uzNAvjywQgwH//ZE3wHex7i8kG1FYRKEk3X Cbw3gTybuBVbL42WYpgQQEpqLdv51Q4= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="mQy/FkxA"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf27.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700615299; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/6X+mOMZoGJuf/KhFjronMx0sFEsDU8bO3mA0YdbfIg=; b=c2gMZgnFoO/vngeDYLzJzEvatDxIVJGzwdMkdvEX4zIDp0eiSBLRv1rlrJuAoI9h3wals0 3VuloG7XGYbJfyLMFi5HosPgz5JKNtbr6nSOA2jrMEDzjyZgp5RZ38BWoMNLSdBJXUia98 sC8E+FHRNIayC/wrSt8dXlRthtsEkMg= Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-6c39ad730aaso5094507b3a.0 for ; Tue, 21 Nov 2023 17:08:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1700615298; x=1701220098; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/6X+mOMZoGJuf/KhFjronMx0sFEsDU8bO3mA0YdbfIg=; b=mQy/FkxAiftK/O9lFSeGaJzSklfNbVj59N0GfHgioeHp0iuZpeolNfQ0lJsjCJFk2j RxSWKnhXsWcC0aPinb4C/dItnvAetuNgAUvSpQEPoEu6u+c58F/6jfr/4cXtLRwYR1Ku iuXi4cu+R7JEYD33hn8fXY8YvZKwAzEL0nBJ8VcRMZ/dKJRodjAOkhTtv1NOaJkB0V1f O+dbTwXbxVyg8rg3aIi4MiSzClNTHDDWDJNE634tP87hw+cebsLlbp2T2+IWZyoJ6n1k GFZsnm/GUrPVE5ZcLvR7RdnP7wYKM6NLmMmb1M/qwR62PWEcKpqlGzpr2uga9A2dSa++ UKTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700615298; x=1701220098; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/6X+mOMZoGJuf/KhFjronMx0sFEsDU8bO3mA0YdbfIg=; b=U2BrcwnF/kStpiKSplaPXiwbau2uSckYk93MV1ZmflMbT0eusWMzhFjxlQyydwa5Ih 8GazFqF1e65JpI4zfgD358pE8bXoF1ylIS3kE6WX9V6C0VdKHF3slp3QgJv45INjGXNB vwMQJG90oRXI1jQfU5bgb/mx5pzgA9oOQN85C7XmQCvtl0/qAAE6WRz1jH96zdrt9bhW DSf8LrFK+PD6erJ6dtJFR6QSRNfaGjzSu6wC6MpxsNt6xGSV94R0rsFqLOPY3gqRr0GZ +KuMLdTuY2K2lPtQe8VTF5QiFMrXd/BvgQGxACIOz7YG+5yMOquwEWglvU0TzQh1VXQL ibMQ== X-Gm-Message-State: AOJu0Yxh8e2my1iiOEz83y7yCmaSBW30lETFq+Vl71+B480Hr9Y9s8rG 5d1N7Sqq/FM6ip/aCvPCrf/WgBz2pccpfYtbkmM= X-Google-Smtp-Source: AGHT+IF0MPSUCqiTsnl1eQc+NOEkZdfylg4Cinzvui2qig7vBCcLaky4/GlXwa1fL7VF+JjfOGRcVA== X-Received: by 2002:a05:6a20:431a:b0:18a:da5a:3825 with SMTP id h26-20020a056a20431a00b0018ada5a3825mr948843pzk.34.1700615298368; Tue, 21 Nov 2023 17:08:18 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id bn5-20020a056a00324500b006be047268d5sm8713961pfb.174.2023.11.21.17.08.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Nov 2023 17:08:18 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v3 1/8] riscv: mm: Combine the SMP and UP TLB flush code Date: Tue, 21 Nov 2023 17:07:12 -0800 Message-ID: <20231122010815.3545294-2-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231122010815.3545294-1-samuel.holland@sifive.com> References: <20231122010815.3545294-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 9064F40010 X-Stat-Signature: 7zakapz85rjgqayadybe7iiimfppa7x1 X-HE-Tag: 1700615299-707683 X-HE-Meta: U2FsdGVkX1/uoZwuEszcnn549q57eR2vlmJUxVfZg+nKwuENnWUQ0c7idP+TPOZqYi6cevmnWSGbJvv7INEjA2/ccNkeIVrILxDdGQpm9FBWKU+Xq6aBN3d5pynsqIEWbDuJ78tJYM5JW60x//3+ko2qjsbXk72CoF/pWBF+zzSL+YV4Uh84vPo2L/8vof6x+nQby06jq3yKmmour4TSxnJ+j7Y71rkvjIWMt5ikCgPtwFjApBpKbN1TPDIgH+pCVXBpgQuJBj6Ne3Nk5ZRj4UiuZ08zrghbyrPp2/LRbipH39dnv/BXByDr2gXAyGdjkKDrhTWobq6t/7OckqyYmX30BqI7Vkgzca4KbTdM4LengEUNw9Be1quA0ynVbOvDZEF/ye3QHimYIFHNSGtbWWMBiyo9U+46TdN+67a0UD4dl9bMTg9VHYkwGmM4HmHFVPQkHeAtsBDcGMc+PXEtGynGQucvU68Im8ClZTDX9m59XfQPcoPmGUk1OC3fIVkrpQhigkY0yhKsG0JFjnx8ySr59itlMYxNjcIrWktTtiCTw+EB+g+qSi0QY/utIuhWb2CPhiDv/M+3sXIq7RonZ53xZVQUBPLE7GSKtTOCjN5LdSI3BMw3l/2gcWyGpQXETrBZjvF/vPKWx2ancgC7SQmmnfpVFoLWelyiz6MuLofu+CDWSxQXjQOe5PI74W5o4PvptakA0IroX/S56mv/AKX+1t57OabYtIaE+GQO6CRXMoNB3ib5kJryPk7iyI9gYYzBtiac6CdRQ4XSJCmk2k6Fvy/1V6IyLzoq8zD2zrFXyZ8b4Ls9XBdd3UMDaDstzdgqFvS26EDMXVDcaL8NN1w7WJdqe2bPkiDSi3g8sA+uzNFsy92FQeULXssB/UF1f0rNBFH2kZ53Y8vJ4pe+m+2bFc9Vi9zJe7ntazIYWr8VITxhwrnwstXrpqr+6ybu+Qs+W8FYDjL2BN68cae rgw/GJI+ ovpu3Q8MIHZHvCQlqCM50OtuoxVUCJEB5Ll4ZOacUs6daECz8rL8RDf17g9oBn6D/iHBjXdjbKARm0CVT51rrb6vn3jZtjktWCN5yByzkpSSUts/ELnMcGD/Ahmntj14OeEjNoDPkkt9p0L7qF0wbsOp2T+3HfBhz29sBN0c10OuuWlx+kK+QGSTS5LZYKm9P4sKbNksSIf9afG9JK2RUVSD7UrxTkRg5pgZXeUhSh+LFhcXERXyjB2DxbP0UTMV/hZZux+ZkLIZHuNq9ZmN9zlW7EZ9sb7iYG5vehN129XMoC7D/f1rIFI4J3Dwx0ZzQyihE6241TydAYtI7AtlfaH9MBk74vAltpRO9hfcVaxK7h5LkfrZhjycg+kgAyXni3hQwrpXRC/l/tjU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In SMP configurations, all TLB flushing narrower than flush_tlb_all() goes through __flush_tlb_range(). Do the same in UP configurations. This allows UP configurations to take advantage of recent improvements to the code in tlbflush.c, such as support for huge pages and flushing multiple-page ranges. Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Move the SMP/UP merge earlier in the series to avoid build issues - Make a copy of __flush_tlb_range() instead of adding ifdefs inside - local_flush_tlb_all() is the only function used on !MMU (smpboot.c) arch/riscv/include/asm/tlbflush.h | 33 +++++++------------------------ arch/riscv/mm/Makefile | 5 +---- arch/riscv/mm/tlbflush.c | 13 ++++++++++++ 3 files changed, 21 insertions(+), 30 deletions(-) diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 8f3418c5f172..317a1811aa51 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -27,13 +27,12 @@ static inline void local_flush_tlb_page(unsigned long addr) { ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); } -#else /* CONFIG_MMU */ -#define local_flush_tlb_all() do { } while (0) -#define local_flush_tlb_page(addr) do { } while (0) -#endif /* CONFIG_MMU */ -#if defined(CONFIG_SMP) && defined(CONFIG_MMU) +#ifdef CONFIG_SMP void flush_tlb_all(void); +#else +#define flush_tlb_all() local_flush_tlb_all() +#endif void flush_tlb_mm(struct mm_struct *mm); void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, unsigned long end, unsigned int page_size); @@ -46,26 +45,8 @@ void flush_tlb_kernel_range(unsigned long start, unsigned long end); void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); #endif -#else /* CONFIG_SMP && CONFIG_MMU */ - -#define flush_tlb_all() local_flush_tlb_all() -#define flush_tlb_page(vma, addr) local_flush_tlb_page(addr) - -static inline void flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) -{ - local_flush_tlb_all(); -} - -/* Flush a range of kernel pages */ -static inline void flush_tlb_kernel_range(unsigned long start, - unsigned long end) -{ - local_flush_tlb_all(); -} - -#define flush_tlb_mm(mm) flush_tlb_all() -#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() -#endif /* !CONFIG_SMP || !CONFIG_MMU */ +#else /* CONFIG_MMU */ +#define local_flush_tlb_all() do { } while (0) +#endif /* CONFIG_MMU */ #endif /* _ASM_RISCV_TLBFLUSH_H */ diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index 3a4dfc8babcf..96e65c571ce8 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -13,15 +13,12 @@ endif KCOV_INSTRUMENT_init.o := n obj-y += init.o -obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o +obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o tlbflush.o obj-y += cacheflush.o obj-y += context.o obj-y += pgtable.o obj-y += pmem.o -ifeq ($(CONFIG_MMU),y) -obj-$(CONFIG_SMP) += tlbflush.o -endif obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_KASAN) += kasan_init.o diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index e6659d7368b3..22d7ed5abf8e 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -66,6 +66,7 @@ static inline void local_flush_tlb_range_asid(unsigned long start, local_flush_tlb_range_threshold_asid(start, size, stride, asid); } +#ifdef CONFIG_SMP static void __ipi_flush_tlb_all(void *info) { local_flush_tlb_all(); @@ -138,6 +139,18 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, if (mm) put_cpu(); } +#else +static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, + unsigned long size, unsigned long stride) +{ + unsigned long asid = FLUSH_TLB_NO_ASID; + + if (mm && static_branch_unlikely(&use_asid_allocator)) + asid = atomic_long_read(&mm->context.id) & asid_mask; + + local_flush_tlb_range_asid(start, size, stride, asid); +} +#endif void flush_tlb_mm(struct mm_struct *mm) { From patchwork Wed Nov 22 01:07:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13463789 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11DEAC61D97 for ; Wed, 22 Nov 2023 01:08:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AFF466B0526; Tue, 21 Nov 2023 20:08:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AAF506B0527; Tue, 21 Nov 2023 20:08:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DB3B6B0528; Tue, 21 Nov 2023 20:08:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 73AFD6B0526 for ; Tue, 21 Nov 2023 20:08:22 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 5777A1CAC95 for ; Wed, 22 Nov 2023 01:08:22 +0000 (UTC) X-FDA: 81483804444.16.D401890 Received: from mail-oa1-f54.google.com (mail-oa1-f54.google.com [209.85.160.54]) by imf16.hostedemail.com (Postfix) with ESMTP id 89C8C180012 for ; Wed, 22 Nov 2023 01:08:20 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=YMvYOThq; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf16.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.160.54 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700615300; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4E3lnavwhWkbsPt/8jollKuN1Wp7270MVQq6Nf32IZI=; b=dLdRKJD+Hd7+hdliZIUbM62ysYcMXe9o4AP2OKl2aJEGwu8rTbwMkPk79DaT5QpWdlkJFo wlutQY5UJIjF4aiF6oQf2dkGcfxUtURBVBZlU6f7wL7cFm+pkCC22eBXOc/t9B58IZV1TB WDREFYFDlnbNpXJjYjOVC8w/o5WHXSw= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=YMvYOThq; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf16.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.160.54 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700615300; a=rsa-sha256; cv=none; b=rN37Sax8XHJNKPCyG59SF/+XqG4lqMHkZIzM24S5QdMSaPY+oz0OeXXGFLKMSnG4awhwYF mCNQTq8NvltFGa0J6pn8PfW8nrYhgpsvowOOtVGpeBkFuWk4kWjmyP2JA+DAtB39bdAaWr 5hRtGRBAnb9S+kFKMXAoG4OGCszvHzE= Received: by mail-oa1-f54.google.com with SMTP id 586e51a60fabf-1f94a097b5eso1436905fac.1 for ; Tue, 21 Nov 2023 17:08:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1700615300; x=1701220100; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4E3lnavwhWkbsPt/8jollKuN1Wp7270MVQq6Nf32IZI=; b=YMvYOThqZGmSbK2oUqCiG3xcF8vIzKRZPwhCJmLAmGIf7GVYnLWHbtZEkJ2tt6UXDx XSZ9dF0y30czxDWjwL0k3bbB30g5X7snjD5mRDaQVE6ootMVuWQ/LsaBa8ux4FP83NDo 4vdT41WVHynvByLH3Zh8L9rl5z45RqDUzjSIU36GODs2iMwWwJOknGyxIpAzwSTHpLNw WC6h/Pq9cXPt/rjvEtKbQfR2dAyC+AgG5r3cuBV5diUIF55ES50YM5Pr/6Y038wV+NGY ktMwkcF2qdl1gmuHeICkT4yRPXKnqncypQVTLETcDI3QoTOJi7aXXrwAiSOce4BM2r48 KoeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700615300; x=1701220100; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4E3lnavwhWkbsPt/8jollKuN1Wp7270MVQq6Nf32IZI=; b=G1qjGjcAzcGLJ6eLHX41H+x+nweWN9ilJNvPGCGpfazHeaRZsUYSi/z+IMbocREX2r 04NlZJDH+2zyspsROeUywY+ZJymS62GI86Y9A2pXbw3qPXfbixxgxxBfZy8avdIZqF12 +1QiT5PuDGtm7FymSUkv0Qv0soocCRXno+78Un1qLKTlXUo68ACh0ENMYM9l6bzN8oay 9D/k8PeGERaJvP4SqnyfhKt8ztqsbio4QtBz/MownoQqbSY0dF4C+zBnvPqN73YQcXBq NuvCZj+4N9EvWAmdOmPZA/ADH9wpisqpH9h7R9gyf7DiGwQJnmzzApaVbLPXknVJx5jX 4Lkg== X-Gm-Message-State: AOJu0YzE5Hq4ZZrlt6i1F9D+foa81XcfJwzJKTXTg7qDe56EhqUJm5/r fvtwuJ9YWhu3SXfTMzIRlHgKqQ== X-Google-Smtp-Source: AGHT+IFhS1aXruOMSeKbF/kr742mxFr/4ERJTn5q1RSxs6qFscJwCL7KnIKBT8DwBL+aJBuJSXtucQ== X-Received: by 2002:a05:6871:e805:b0:1d6:8b7f:ce89 with SMTP id qd5-20020a056871e80500b001d68b7fce89mr1213468oac.51.1700615299719; Tue, 21 Nov 2023 17:08:19 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id bn5-20020a056a00324500b006be047268d5sm8713961pfb.174.2023.11.21.17.08.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Nov 2023 17:08:19 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v3 2/8] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Date: Tue, 21 Nov 2023 17:07:13 -0800 Message-ID: <20231122010815.3545294-3-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231122010815.3545294-1-samuel.holland@sifive.com> References: <20231122010815.3545294-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 89C8C180012 X-Stat-Signature: 1dxrbzpg4916zbrthjkrt99dsdb4cryj X-HE-Tag: 1700615300-399477 X-HE-Meta: U2FsdGVkX1+6DZvy4e5a3Bk4P3nQbXuDS1WN8LLTKfL8k6noAl+gBxlGXs23qAPVafj8jkCP8g9HOxN67APU1urGn9GL5JVPDLAbLGUJCFHFseA56WVgiwoSDhoTel80uXvgOtgV8HRdxYm1BcHnvYNgflobuKwI67xERlMu8dtd+DwaehnoI/oKm6cL0s03tf+SDxm5uptkhRt6ifvZs29i/16bI1s0vyCiZdYEK9vWedD8pLEV+2u3KNqsvLvqEN1ulS6zmsASL4+1Q15yDultE76K50MXGO7eLbRjgzdmcPFZYbKpPQUO0mRLKtL1T9wpP2+/Sj1xsKnKKvq2tAmv3IdvMkr/m8uRWQU0eE4m9SZhlPNtXE4/cyfYCJTbgkKTbXN94bgtWSAkcO/GD9bHEptgcCZqcnJBMM2joCB3D55kfn50ZKA22OGJegUbmK8dMtmaJu05DdBrE3MR6I2BHWsd5H/3uaebrdMGGwyRQFS5a4jT4I2jGN3TWB8nke0BwLxeDH7KvTvR7TXcAZJUuzbitd6bjekpcaqBNRvsEIBANsxVD4kru6vJUEINQ8jXoYU9aE1pgIk77QSc8WqUpQsudRs0K8Ubd+ggmEwo6d84CWKrk4UkejQbEiIVtar+jqWz+sfa23iSADWl7GE/87HUIKUisnM0PQrOEB2KHQQemaMIaiwrEjWeXYNea4QRQIEvgXQZwm2HxXCQ+1CEF9XwxOQQjNwCuXFaPTO5NV8kxjt0Iepnyg+ngky1mh86ksPD4SuN+Yp5JbwJBPyiFlt8pba8VUymAjNgnNGJd4oLSSRF6rhf5kM8kq31vCpvwwrAnr+cBQKvFTK+Ubm2/hf9OdaPXRW5Ehnv6fFncCj4sMonY7ztdac1J5nTQzuSwNI1YH1NRf4htd9kriQ4nB9GdbEv7hf+PziWjXZNJiVZjonjlmvWZOj3c8ieyJoO8hRILcqskf+OPU0 /QHYRAyj TJRc39FFrRhyuzmpMQ1dhS85/NIFk3CXkRiiK4uSchNhMpysnx/xEZVTF67mTFRYPfYKRTYGQE5w2X+BzJrhhOOZ/Crq25amY0Ns8ywpHNkrmhZsJ9XcExaoGoBXMossXilBQMn/8vOkuWv5K/mFKbbZWNYAy9geKDnMtijckbVF99j6Q47+GPZ7lpGzENayxC6TlQw65vQC+NbUhg9fpl7Zcgc54QK8UECxIVxvFR0WqvbiMnl3dqfR/5yoVntW8p/RUAbfx4C/r+aViZ/rF71vMBxdm3GJakSF8WfNRRSd8miZ8wWaDDUWRzDRs3S8BelquVr6aIjqit5BX5acJB9i/MT44DhT3POftD6456HWnvj15610NppHkHwgfI8eWtOz5rLKVrrU5eAk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: commit 3f1e782998cd ("riscv: add ASID-based tlbflushing methods") added calls to the sfence.vma instruction with rs2 != x0. These single-ASID instruction variants are also affected by SiFive errata CIP-1200. Until now, the errata workaround was not needed for the single-ASID sfence.vma variants, because they were only used when the ASID allocator was enabled, and the affected SiFive platforms do not support multiple ASIDs. However, we are going to start using those sfence.vma variants regardless of ASID support, so now we need alternatives covering them. Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Rebase on Alexandre's "riscv: tlb flush improvements" series v5 arch/riscv/include/asm/errata_list.h | 12 +++++++++++- arch/riscv/include/asm/tlbflush.h | 19 ++++++++++++++++++- arch/riscv/mm/tlbflush.c | 23 ----------------------- 3 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index 83ed25e43553..6781460ae564 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -44,11 +44,21 @@ ALTERNATIVE(__stringify(RISCV_PTR do_page_fault), \ CONFIG_ERRATA_SIFIVE_CIP_453) #else /* !__ASSEMBLY__ */ -#define ALT_FLUSH_TLB_PAGE(x) \ +#define ALT_SFENCE_VMA_ASID(asid) \ +asm(ALTERNATIVE("sfence.vma x0, %0", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (asid) : "memory") + +#define ALT_SFENCE_VMA_ADDR(addr) \ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \ ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ : : "r" (addr) : "memory") +#define ALT_SFENCE_VMA_ADDR_ASID(addr, asid) \ +asm(ALTERNATIVE("sfence.vma %0, %1", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (addr), "r" (asid) : "memory") + /* * _val is marked as "will be overwritten", so need to set it to 0 * in the default case. diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 317a1811aa51..e529a643be17 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -22,10 +22,27 @@ static inline void local_flush_tlb_all(void) __asm__ __volatile__ ("sfence.vma" : : : "memory"); } +static inline void local_flush_tlb_all_asid(unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ASID(asid); + else + local_flush_tlb_all(); +} + /* Flush one page from local TLB */ static inline void local_flush_tlb_page(unsigned long addr) { - ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); + ALT_SFENCE_VMA_ADDR(addr); +} + +static inline void local_flush_tlb_page_asid(unsigned long addr, + unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ADDR_ASID(addr, asid); + else + local_flush_tlb_page(addr); } #ifdef CONFIG_SMP diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 22d7ed5abf8e..0feccb8932d2 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -7,29 +7,6 @@ #include #include -static inline void local_flush_tlb_all_asid(unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma x0, %0" - : - : "r" (asid) - : "memory"); - else - local_flush_tlb_all(); -} - -static inline void local_flush_tlb_page_asid(unsigned long addr, - unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma %0, %1" - : - : "r" (addr), "r" (asid) - : "memory"); - else - local_flush_tlb_page(addr); -} - /* * Flush entire TLB if number of entries to be flushed is greater * than the threshold below. From patchwork Wed Nov 22 01:07:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13463790 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37F8FC61D98 for ; Wed, 22 Nov 2023 01:08:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C12DE6B0527; Tue, 21 Nov 2023 20:08:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B7BC66B0528; Tue, 21 Nov 2023 20:08:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C80E6B0529; Tue, 21 Nov 2023 20:08:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 869B66B0527 for ; Tue, 21 Nov 2023 20:08:24 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6523FC0750 for ; Wed, 22 Nov 2023 01:08:24 +0000 (UTC) X-FDA: 81483804528.02.5645D43 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by imf24.hostedemail.com (Postfix) with ESMTP id 80997180008 for ; Wed, 22 Nov 2023 01:08:22 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=EanuiLGW; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf24.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700615302; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VvpOAl9p0Aiam4sGuchXKDW+3q/tf43lzqxId4c/moI=; b=JV+5m7hp9Crc26oX3OcKnxLRVeXXtxfkJwiOXkNLKgrd54Mp/Pcv3wPnUeIEfVOPGgZV1P 2NtSvuI6C/zcLvbiueZ1586ta7YOh9608rNBG/2sjXyJhDptPkK3PMAnPUOksW52DK2lLM y5zYcq9Icrhvj4Y9FCz3IYNPGXP8pAg= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=EanuiLGW; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf24.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700615302; a=rsa-sha256; cv=none; b=79UDprlLXk6k2lgsGpRboR3lK9jTJe91APijIVkzy14B8hoqnttnJSCfDSGlxSvIez7J5l Y1rcvwUhg1P3Lhv2/AE1UsUY1e1t9Vcora+LPuAE14JvfuTfhyQPPiEwaOwbh7TfvE6INE /a9MrzJT5aywjdwSEkSTEWzwEyudBnw= Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-6c320a821c4so5514155b3a.2 for ; Tue, 21 Nov 2023 17:08:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1700615301; x=1701220101; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VvpOAl9p0Aiam4sGuchXKDW+3q/tf43lzqxId4c/moI=; b=EanuiLGWmZNFZj+NTmSQXagq+gy4Z24dirLuDbLayJVyXwcTG3pqRJEA5+K0T8n34p xZp6z4TDt/IVuliLILO04KKcPv0LQ3XGkxoM1FrvEP+CZ71czbb3oXFJ3dh5SpPuCuW3 a3zOdQ23ftciLL2Zoa4rf8zwrx/XiNifyPBki72bE8VMoGyufUbhU9vqikweVMjbINBm aW/uXpNP8rBC+ZrrsH+qdxB4YRQOqU1jz71keEqtMXpWi4gaZbDlYFhIAUHp9LhGh14K RWWwAnaNEWrKByHZ88vb7pblQE4TVu1YI1/ZUSGSOQNOuUyUID+Q4L8eDzjkGLsyiO0E 0zGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700615301; x=1701220101; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VvpOAl9p0Aiam4sGuchXKDW+3q/tf43lzqxId4c/moI=; b=DEXa3uc0ugtbf8BTi2YPheoSOhdWodrlX38JqTRpL74wy13S0ZAk37hQRF27yV4KUn ar+mxn4buZXdk/0kuMZym9zP2UEtGfxW85XCJV/Vw8RPa67JuwJ6GjueKcJyKFgxgrV1 OeZEbdJ0WJqB+WjT5ymKUZsLCN/skd+DQEZppTkVJ9FLfrhh5LQsdOopqXDydkU+4zHG ru8IscGLD0+mhuV4e7vZs8nyZwqlZTaLM4Ohugivpk5P3amPQGQbU+XNtOA7xETXLo/e nU7DzINqXljgifeoEBTkNt7xoxC8q189wgITPjW87AekQ/Ndn/2NKgrhhN3EVO2RLhVy SPIg== X-Gm-Message-State: AOJu0YwGyJUrOsY0nAnL+Wbo+uIxZPW1J+UICzSLFZa0SAsmB+egZk5o 9kfIWDe5vclhuZ73njxxrhjwz37+2rGYqim9+t0= X-Google-Smtp-Source: AGHT+IF9d9fQIS/asTDDunhMGjFlgWHJ5Uekq1PrtWCEzSD057jeC559ik7d8wrAC6IaVqDr4/Nfcw== X-Received: by 2002:a05:6a00:3927:b0:6c6:aaa4:5d68 with SMTP id fh39-20020a056a00392700b006c6aaa45d68mr871292pfb.27.1700615301365; Tue, 21 Nov 2023 17:08:21 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id bn5-20020a056a00324500b006be047268d5sm8713961pfb.174.2023.11.21.17.08.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Nov 2023 17:08:20 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v3 3/8] riscv: Avoid TLB flush loops when affected by SiFive CIP-1200 Date: Tue, 21 Nov 2023 17:07:14 -0800 Message-ID: <20231122010815.3545294-4-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231122010815.3545294-1-samuel.holland@sifive.com> References: <20231122010815.3545294-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 80997180008 X-Stat-Signature: p44c6ae3bo13uzkmc5xzu1adi1yrrgub X-HE-Tag: 1700615302-529068 X-HE-Meta: U2FsdGVkX19PRNo68tU7SHR51HWap2TcPPtCIz0fGE3MvKEbM/5Cc4QbvozAJ/T4VH9Qz4Da9BAubBkXiR0y8adJ0fU8dAX4CBJ75hBxbSydNuLIzU6/c7UiMmPirZMA/i58Xl4Omp7ZAHtZ0XBCD5vFIMVkhLuQRZH35AqIh5PFBqtL2lY0S+lEJpgKjVzQhZzshWGuujXBDxA8VYMQnC2TDFo8y0DUsoH51/erg7brugotbZqKXkrw/nC+YoTNs5W+iimXVFcNfJOiRliMQbjbsc+PIZnnSUodN+CwQWo93gSHxUNfDr7KhOU2Au7+7nvQGVxEh8lmNi0iYQfMS0apRhUYiDsyVa/DMZ3RO2mv+6YLASroWCYtSK1ohFwZSAwTDR63j5khmCJs19k1KeQc2e5boHf9m0YvQCn2/aDuQmnGDAxz1IGuCbesTWqgZTL2Tkn6+7efE5dC57pmYI4SQ/MGt0b224bBBKysq88rsYoTcrvQdzIx5ue6GJVSwp0GXvWFUD5K8N8ai5Scbx0Qj+DWVt3lby3cjmvqxGjM4ub/qiqkEXR8+VCEBI2LJbhiPw1eRTKGsByOZfEUoEzijMR4pnB6+PNTlJ1oWxtaO6IVZnhWPNvcw37sPIadeKJzAB6+SQE3/7TIlC9VkF3HWrroOvH/9OO7h3zp3bC5+z296MvG2iX6YrJ+8iWL/gMyVwaP58v+feNuylDza4w5f3WknbpzDi9gLoCWxmBUcGEZxItGEOFeo/ROLoo+66uuwvoBEeyA+LYfNZKqRHuw2BLHkO6ykw2JTINdznM6ius7CExeb/hEJj44ISmQj/PnBrIzUJ+elqUF6exaeN3KIfzRHMjIiqoNxz0DJ1x7AK9DoTVLKnUAJqjtDARxW8NpU5EEheGQcpWTbMb7ncpfvjorvXco01mHCnQ3FjWgLRDfYg1hQTQ3yc97uwnDBcQ7uySYyHe8kD5xj7M KB7L84hN d5KBh9gUtvuTlPiC1Q61bzqPzcF9bQJTiwo3ER4p0X5xFVd0a3gNjMSVCc0y27B94y0wQDUGDYAyfuHcKG9XSXyorOGDUCdwHuDiAyn8nU/1ANMCs1DW4r43BMHhYkIXRksAx/jJl0dML6WgBCvB/Pj+ovPzcp5L8BP0euPI69DvZ+76816GiPnM5ZnZLxuRlGi+Vjyyy0aGVDdYx8aLdrSHZ1SHABBbwoFJ4/cY/HrkJkBbv4xFeCLSI6HS5AuqxJDJC1XTve2S3T9xHSHKCbQCm4wsuOR51E9PtIC15q+JJIXnIsILQymPYTu7ijlbbMXM7WFHSzoqaJmIcqgIBh8w4z9zmGMAhnoeaLLVJouZXIj00y2lz3kwQ99J5lfrKd4Rduxw01siIMBk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.003160, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since implementations affected by SiFive errata CIP-1200 always use the global variant of the sfence.vma instruction, they only need to execute the instruction once. The range-based loop only hurts performance. Signed-off-by: Samuel Holland --- Changes in v3: - New patch for v3 arch/riscv/errata/sifive/errata.c | 3 +++ arch/riscv/include/asm/tlbflush.h | 2 ++ arch/riscv/mm/tlbflush.c | 2 +- 3 files changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c index 3d9a32d791f7..00e011d78866 100644 --- a/arch/riscv/errata/sifive/errata.c +++ b/arch/riscv/errata/sifive/errata.c @@ -42,6 +42,9 @@ static bool errata_cip_1200_check_func(unsigned long arch_id, unsigned long imp return false; if ((impid & 0xffffff) > 0x200630 || impid == 0x1200626) return false; + + tlb_flush_all_threshold = 0; + return true; } diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index e529a643be17..3b393f765805 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -62,6 +62,8 @@ void flush_tlb_kernel_range(unsigned long start, unsigned long end); void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); #endif + +extern unsigned long tlb_flush_all_threshold; #else /* CONFIG_MMU */ #define local_flush_tlb_all() do { } while (0) #endif /* CONFIG_MMU */ diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 0feccb8932d2..27b3744b5673 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -11,7 +11,7 @@ * Flush entire TLB if number of entries to be flushed is greater * than the threshold below. */ -static unsigned long tlb_flush_all_threshold __read_mostly = 64; +unsigned long tlb_flush_all_threshold __read_mostly = 64; static void local_flush_tlb_range_threshold_asid(unsigned long start, unsigned long size, From patchwork Wed Nov 22 01:07:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13463791 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6834AC61D97 for ; Wed, 22 Nov 2023 01:08:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4AE36B0528; Tue, 21 Nov 2023 20:08:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CF99D6B0529; Tue, 21 Nov 2023 20:08:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B99886B052A; Tue, 21 Nov 2023 20:08:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9E8556B0528 for ; Tue, 21 Nov 2023 20:08:25 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 7FB6F140B45 for ; Wed, 22 Nov 2023 01:08:25 +0000 (UTC) X-FDA: 81483804570.28.F89F294 Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) by imf14.hostedemail.com (Postfix) with ESMTP id AA06D100013 for ; Wed, 22 Nov 2023 01:08:23 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=Yhjvns4V; spf=pass (imf14.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700615303; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qsEPuyPCg+yq6nUonOiNDjvoYO+XNuK3/GzQIQmVNGs=; b=PqKnraBvHtQ24LZ9+QSXiGCb3PzakGzJ0mIPrdtHkrRSw+5ClvIr+QA4Gt8bwsMnzyA51W dqT4hPom673A/OERt1H5YCCFDccj/Z/Y00PJto/tM4TcNRMa0oP4rVU5Lc+3nomXTOe/8p lRYNoR5CvGgxCjlh5tNmGb6z5lVL4r8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700615303; a=rsa-sha256; cv=none; b=J9rk4Oxi2C/UkLBabV8aYYfoRDnPuAnv2PQyQqyGqNSlNWboiK+DVHMTzHXhnGnDeSuWV9 bCl2H1L74BoK5EGRsEcTHsH+X9iMYtsX22yDHtRscZXtQZLsESWajXj+5Gp26yl77G+whG M4Nf6E2dDjdhZyqrDVNw6zlEZlQiffg= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=Yhjvns4V; spf=pass (imf14.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com Received: by mail-pg1-f182.google.com with SMTP id 41be03b00d2f7-517ab9a4a13so4764228a12.1 for ; Tue, 21 Nov 2023 17:08:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1700615302; x=1701220102; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qsEPuyPCg+yq6nUonOiNDjvoYO+XNuK3/GzQIQmVNGs=; b=Yhjvns4VFhTvnzcqFwcGs4etQlpSiammofkF6DaRG9NkNEk0ubuMMN2msFekRMuSlr 2KZ11b/wAm/FH5+RwwgdcTd5gGYvxAiM13/a2U2BFOw3NLj30s2P3tm6rL7bFIAajJcY 6AW7tfWTKvRoQ452+xC+HLI4/e2I9kXA8CDKdl8RP2snKsogJcjukBWxTiams/prHslh BQUNEkOQw+6ulUYbYdaMMBoTsd8BQp+SxIrt1EtKbJhs3D4KcJDJ3CfkUdRSGbHqrvTs lWieWvB3w24mHGepZm2rS1cHSiq55EHNET/RfAbcvC8OOORWWgAp30BeqhmL+0OSekn+ EN1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700615302; x=1701220102; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qsEPuyPCg+yq6nUonOiNDjvoYO+XNuK3/GzQIQmVNGs=; b=CFSdOovuaq/rk4GGKAZFV6YlkIx2wR6jQK5xxfrC2v9GShYKnBofnWcQ0e285slJBz D2JqcqyH0suFid7PkjL0VYUnrUk6OBoG74xhTH6cfhn1RTA5iwotm9t5jZyclD8fa0kh qo4dFiUhqdcoKUFriQ5I7yP58FR0biyt879NxeDJjTiCtEIpH47feol6xU59j3ehjNqA QVBKpQPy22o077QmHDHfkimZ5EKQ4i24GuQXZ3KP2x6U2kq+vvmMz7i6OwDqXvc8PLm6 1j+MSImpwHOMvxtS7B6+UTEm+fdqlv/S+7hiwGyc3JUjoQCV6hIYnVHi5kdIMEe/wAVQ B60A== X-Gm-Message-State: AOJu0YyhSvWJxC542e49K99SVjAnSNIv6oTn8Acrtyoj96anj8Ev7RW6 oUyxfhEsB08htTXY5+DKIEoCdg== X-Google-Smtp-Source: AGHT+IF7ribPLGqSLRhKn+k/5RoFarRyyKm41YCW2vHm9TG9JHEkBRqiRle7Sp7LVXO54oVTBXbYlQ== X-Received: by 2002:a05:6a20:c6cd:b0:188:75c:e69a with SMTP id gw13-20020a056a20c6cd00b00188075ce69amr779809pzb.4.1700615302563; Tue, 21 Nov 2023 17:08:22 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id bn5-20020a056a00324500b006be047268d5sm8713961pfb.174.2023.11.21.17.08.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Nov 2023 17:08:22 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v3 4/8] riscv: mm: Introduce cntx2asid/cntx2version helper macros Date: Tue, 21 Nov 2023 17:07:15 -0800 Message-ID: <20231122010815.3545294-5-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231122010815.3545294-1-samuel.holland@sifive.com> References: <20231122010815.3545294-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: AA06D100013 X-Rspam-User: X-Stat-Signature: xcgzf6jghbsjmjfk5za3qh4xdyamr4hd X-Rspamd-Server: rspam03 X-HE-Tag: 1700615303-141739 X-HE-Meta: U2FsdGVkX18wbTWypqKnWlK5fbX5yffSD5CSnbOZSI2SOUoz/Aqz8mMMmbo3SRnoGzKTqNjT1zMEEBKTQhZSLd2DVJjy1YRBBJkrogUWr4U1f/Cu2A0ZHr8+8yry3dBcaEQdxt71PgAONywDgYlxXTjGpci5WmzabQdwxwxxGEuI/4ixMXCyZ6tELcBTRDlMsAvuQnMUK32rKFePBz4+Ym/I84qmaXICvF5j57NgJHMTWKkFo1FEkf1OnLGNG1DSbjBRd1r0h8fwG2M64FkL9azhxlsrwnSbKTvDz1YAwaZFqNIMWm1afuj88ZiaRpBWh3o2+orxaPm+ucXCrCHobUwYHa2FATWDhoUyZDJhwFtvASV3uly5uppN3sVzN2CSdRpxTzDpQxjmN7mgIwGjRXD12+a8v6ZM59k3gsYVEqslZgFLoXrOb1EHzQi3Nh2981MOSt+Byteta5VxALjfwZm+iKMIPjrJ6AX68ge/kXTwKcPxIRtQhdHfwceZSzNO/a9X0qlo92biu0EpBCJxV90S4NgKWEV+H+O/uOsJDI5430qmLHKYOVshFJWBeNVFdNkFdsl6U4WPLSDbX+nMglA3I3KbJqT3S6ABKH4tiA2sEihNtae/CSCaCMvbNJMzz/A3u+fuUa0ZKDWOc4VRQdOEC79MSU5Q0vJN/+6n5w8w3RmauyjYz+dfEcJLoNGt9wWvSEYYNOcXpo00UrALmkn7bhMCPVnxsYv8F8gppZcPGJKiJBBrOFV5LmJ8RAY+pnNWyx5Ia/NyI+P945VjzPHwsXMZ8GRH0ZuKWs4CdHr89cNPN7QzcRKbVtuMHwW6GZbZAeoI/4O1pEphQ6wREY8LCpOJCT7KsAq1prAp4hz7un+ehzyIeJ0Xw509THuNBqDXlxDe0lyLBuTnhc+VFpdeHC2vHNam+YvrimM2Qm1LKf2CrT9Q41rCxENexkJBUdhAOCA5lU8j4XVBYUQ VkdxR31q NV/wvyl+5wIQuHBnSneaA1eVfoLqF2tSuc2YGlkRgACLwUNOZcXq+xYatI6RCjnO+Vgw6sx3Dku+E7lq22wVGMl5xLWT787lZyep4gP9qIXpPmjbGLCblcgrj1sU3AlRFZkXrQ92IlEmdBAKextg9bibCSDTtd2oRFInTlTJhb1a9+9O06xofUThaNEcD7bWAqG/5CUybmV32Mf1kRSHNFwZZylxfyw45Q7SAmvcs+nvhhrYN/FTzYKOuJYjehf2FI+mSb3dZoZx3G1p5HbkAxelHUJXZfBLr1FgW9c36b+Gfvku3dzk5EFaXyu0ak7FCLe1iAjHskRebJmBy47OCz3SS8V/UwnF3FvNdDSMrtU5brn5XBf6akQsheQbjbFZneti9d1/TCuk5yDA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000005, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When using the ASID allocator, the MM context ID contains two values: the ASID in the lower bits, and the allocator version number in the remaining bits. Use macros to make this separation more obvious. Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/include/asm/mmu.h | 3 +++ arch/riscv/mm/context.c | 12 ++++++------ arch/riscv/mm/tlbflush.c | 4 ++-- 3 files changed, 11 insertions(+), 8 deletions(-) diff --git a/arch/riscv/include/asm/mmu.h b/arch/riscv/include/asm/mmu.h index 355504b37f8e..a550fbf770be 100644 --- a/arch/riscv/include/asm/mmu.h +++ b/arch/riscv/include/asm/mmu.h @@ -26,6 +26,9 @@ typedef struct { #endif } mm_context_t; +#define cntx2asid(cntx) ((cntx) & asid_mask) +#define cntx2version(cntx) ((cntx) & ~asid_mask) + void __init create_pgd_mapping(pgd_t *pgdp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, pgprot_t prot); #endif /* __ASSEMBLY__ */ diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 217fd4de6134..43d005f63253 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -81,7 +81,7 @@ static void __flush_context(void) if (cntx == 0) cntx = per_cpu(reserved_context, i); - __set_bit(cntx & asid_mask, context_asid_map); + __set_bit(cntx2asid(cntx), context_asid_map); per_cpu(reserved_context, i) = cntx; } @@ -102,7 +102,7 @@ static unsigned long __new_context(struct mm_struct *mm) lockdep_assert_held(&context_lock); if (cntx != 0) { - unsigned long newcntx = ver | (cntx & asid_mask); + unsigned long newcntx = ver | cntx2asid(cntx); /* * If our current CONTEXT was active during a rollover, we @@ -115,7 +115,7 @@ static unsigned long __new_context(struct mm_struct *mm) * We had a valid CONTEXT in a previous life, so try to * re-use it if possible. */ - if (!__test_and_set_bit(cntx & asid_mask, context_asid_map)) + if (!__test_and_set_bit(cntx2asid(cntx), context_asid_map)) return newcntx; } @@ -168,7 +168,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) */ old_active_cntx = atomic_long_read(&per_cpu(active_context, cpu)); if (old_active_cntx && - ((cntx & ~asid_mask) == atomic_long_read(¤t_version)) && + (cntx2version(cntx) == atomic_long_read(¤t_version)) && atomic_long_cmpxchg_relaxed(&per_cpu(active_context, cpu), old_active_cntx, cntx)) goto switch_mm_fast; @@ -177,7 +177,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) /* Check that our ASID belongs to the current_version. */ cntx = atomic_long_read(&mm->context.id); - if ((cntx & ~asid_mask) != atomic_long_read(¤t_version)) { + if (cntx2version(cntx) != atomic_long_read(¤t_version)) { cntx = __new_context(mm); atomic_long_set(&mm->context.id, cntx); } @@ -191,7 +191,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) switch_mm_fast: csr_write(CSR_SATP, virt_to_pfn(mm->pgd) | - ((cntx & asid_mask) << SATP_ASID_SHIFT) | + (cntx2asid(cntx) << SATP_ASID_SHIFT) | satp_mode); if (need_flush_tlb) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 27b3744b5673..23409d70440f 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -91,7 +91,7 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids; if (static_branch_unlikely(&use_asid_allocator)) - asid = atomic_long_read(&mm->context.id) & asid_mask; + asid = cntx2asid(atomic_long_read(&mm->context.id)); } else { cmask = cpu_online_mask; broadcast = true; @@ -123,7 +123,7 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, unsigned long asid = FLUSH_TLB_NO_ASID; if (mm && static_branch_unlikely(&use_asid_allocator)) - asid = atomic_long_read(&mm->context.id) & asid_mask; + asid = cntx2asid(atomic_long_read(&mm->context.id)); local_flush_tlb_range_asid(start, size, stride, asid); } From patchwork Wed Nov 22 01:07:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13463792 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B90BC61D9A for ; Wed, 22 Nov 2023 01:08:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E82106B052A; Tue, 21 Nov 2023 20:08:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E33336B052B; Tue, 21 Nov 2023 20:08:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C0E896B052C; Tue, 21 Nov 2023 20:08:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id AC7016B052A for ; Tue, 21 Nov 2023 20:08:26 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 8DBB4A02EA for ; Wed, 22 Nov 2023 01:08:26 +0000 (UTC) X-FDA: 81483804612.17.380D79B Received: from mail-il1-f181.google.com (mail-il1-f181.google.com [209.85.166.181]) by imf24.hostedemail.com (Postfix) with ESMTP id BA7B3180011 for ; Wed, 22 Nov 2023 01:08:24 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=NHXz72V4; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf24.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.166.181 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700615304; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=b39ZSDVnn3/TpDDYcLNRgBYhWVyekPNc8OuxKxdgheo=; b=B1yqDy2kjYbVPFPpmXPWUhbmYy0NOf07BqL88/5hxtkEQaA39Wjp6tkkC3GLgTjNft36l7 qI6b6kYGD6dX+aWsejaia5uDr8leEGwXRn63PGnDOwXP+F5ESb0zUGq2N8LBRGgBya6uYr 8U6VdBDsLyaYQOweK06dBDZ3S0+y6X8= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=NHXz72V4; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf24.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.166.181 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700615304; a=rsa-sha256; cv=none; b=3hGskMHwcux6eDlet8CxCzDoB8ArLaHTUZhzfSSPRDDuRqmCN/DTG3C8+/TTeknpW/P4RP 18f5TI2fl8HbcxUcc5fVbKVfkkxtCTltjBiKKfLDxgPk7bpUIqa63uJZN+nR7zwCinCHZS nXJvpy86cKUVpMSDFZzFcVYp/jxZwcg= Received: by mail-il1-f181.google.com with SMTP id e9e14a558f8ab-359d27f6d46so19882915ab.3 for ; Tue, 21 Nov 2023 17:08:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1700615304; x=1701220104; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=b39ZSDVnn3/TpDDYcLNRgBYhWVyekPNc8OuxKxdgheo=; b=NHXz72V4AvAntZornk6h1HyMLuBrvCCFq2Up1liyt0cvGV7sF+yOAnJaLgUcR0HUD5 nXUfkiyrBa6S7jY7b3XYV2K1rbp9KmAg10R+Vqx+BDZlLLZuXk+LE5WNS6i77fgWLuhm QFuytjJyzrrYtFXfQM/Y9zGBXowezA7stNGJLknIvcSrF9n/XtYV8Ckx9UWW10fS1tfR Z9oXtZPYope/yltPM6TJo+Y+fDnI/MrQY/Esoad7BJ8/atnmZpI5ITskPPbvQffO+3V/ QujrGzrqYbBZWTe6X+HAekErrygMWKStHxdPtW4bC1IMgjEBVz0TPtEBzFNjYtqVLpXn QSsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700615304; x=1701220104; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=b39ZSDVnn3/TpDDYcLNRgBYhWVyekPNc8OuxKxdgheo=; b=NduQ8aKwjchfg/p/SyjLxMDy5p2OqYV1qOtBKXXwiJxuipSdb+gO1XKgtr6bHmblBQ M35XasdaqE8wUy9ct7YCLkY/mV5MLiWiUdyRwqINU/0QxowV+4R1jS4Qvan+usS7842K nYblBOPJOXDMI+v8vRl737sgLbgEq/2ERaF5LClpUN+hV5wz+src9r4OiI1BqL14Qp5R bmfzoZYrOFC5MrMzVY33A9NIbccxT7ftFLk4ejA3sqWSVvu7RXDLCiw40BmlgoqUlZBt l+1kE72Br2nkc7HkAyrMkNppUAPGQXxCYcSFtrT1S7MiLhl7p9UFpXwtM9Afa5mCXzRS MegQ== X-Gm-Message-State: AOJu0YwiQHBKDXxP1SExK2tRraddIuqYfUZ1jW7EZ7tW0BpvgQFgRCZh gFobYxdaM29JtwyMsSGkkivQlg== X-Google-Smtp-Source: AGHT+IGi2JpVuCgk8T+HgcOfCC/VHmmgf8iSnDwMNXS0Bc2add3cL906GsPfWT4iX9QyNn2xttBJig== X-Received: by 2002:a05:6e02:b48:b0:35b:2b49:d816 with SMTP id f8-20020a056e020b4800b0035b2b49d816mr721861ilu.13.1700615303918; Tue, 21 Nov 2023 17:08:23 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id bn5-20020a056a00324500b006be047268d5sm8713961pfb.174.2023.11.21.17.08.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Nov 2023 17:08:23 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v3 5/8] riscv: mm: Use a fixed layout for the MM context ID Date: Tue, 21 Nov 2023 17:07:16 -0800 Message-ID: <20231122010815.3545294-6-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231122010815.3545294-1-samuel.holland@sifive.com> References: <20231122010815.3545294-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: BA7B3180011 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: myk7rxqmbo73qcnmfeejxu7hmtc98tm8 X-HE-Tag: 1700615304-751293 X-HE-Meta: U2FsdGVkX1+RbJYM2ApdiQBH6rHtmSuHnh8lkKn6lozfOcnGRXQJr7sAUtHokrHvcmcK0/CaOdgImOEHHiq3jIosssLXnolaar7PoJglL8/xTV/4tRROC4Avnb2FoNVkqgdq0j/RBmNDqkszy3NjlyR3EWfC9JHHzU+szCJ9pM+9HJly14jbpWCNNmDFqhh7+OlZEuRqcobQShBWxTlTYR8tCNf0NLqcEn8HYDS6xy9mgwiuaEEei/rOS2YcocXUmQskIPFgkKOk6Ju353Fe4K59Po+t4uVebXbRnj2aM/roocDZ6YJq8hVWjQtXvMM/LNcUdijSQsKBABJhX5p9HndLdANJ+ftprzL3mxPBPK71at2SjyzKau6xzCYZ75N9GDZyzlu3W83q8nwIjgBB6eevioelHRHv46twieo+2vmge8BcqAp+GyawFfg0abnJsqW+f5KYqcppbIl9szC/3bMzFohtgM1lNvWD3hLsgThS44I0/liMgHdUcxXt8YROZJUs9e4J3A8wgaehzQ/b2FkT/sq3JaztdFKhQCPWVZW5ghzigFDnnag8hX4gwSBVYJDXCCkXki16PaLvwykA5Sb25YqHzWj0bKE5zfbtnX7FbqzDMkDuxS7nFujpuL4X7Tnj9OI8r5kSmtTx9WbIM19pzIDC6i2sY4x6hh1AZMwB6VXW0amq5Fl6IH+cNiiO6SZ+dLMVWS16YeQBe8hrvS7csRxQSwgAde/UcyH5ABxbGWox77Ie+QKUS5w1HFV4ehKB0dBW/nGXHXCcpja6ZBgJcvXX40EgcB2vY+clJD5kc2UP8xB9OwNNTSSHsSoUFx6krJg0PXI+vd6ePx+ydLrurrF5Gg/NzGf8ckM3xnqGk5aiOzX/oumrSpAReea2u3fHgEdJFvDO/3ol5XOhiLbOwhkBFVP99R/imI+GWAbl1P6rtI9jD+W10bnqWiAauk65z7tVjhe1kDnMkAf c4PUs9xz sSssqxfAxm5YIg4XPOPTUOlj9Lj1Ar7gcOgr6h2C41v/lIuXYCyr6a3Zv1SurSQHCd59R6s+hcSzyxv+IhXAuUgr1c39uCW4qToZQmmKRLHEAEdKwXJlc8S0SQULesXfDmEM/V5J6Ph3YckK6h/moKYgOpb8pSCGmc1a1j02APGd1MTpINMa6ptP0BiBTLZ1ZPbn3teYFhzp6ZmOxgKOkvkOyBJUMUr9V2yctUUsGxiVpTacIGwfHW4O6WJbO5T+7FgU6gL3rGFPhuo5ED+yVcfUpfkoPF0mnpoE7vD+PJSjgb8jG7tuZOXCBqzCyqggRV3sc3XyXbz0AMxOnGKc/h2Ov1/Cb7stfkgiA4rXIQGKxq3Cobyp//K/0UYfclZt1H3kOBQdMSi9JnCI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, the size of the ASID field in the MM context ID dynamically depends on the number of hardware-supported ASID bits. This requires reading a global variable to extract either field from the context ID. Instead, allocate the maximum possible number of bits to the ASID field, so the layout of the context ID is known at compile-time. Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/include/asm/mmu.h | 4 ++-- arch/riscv/include/asm/tlbflush.h | 2 -- arch/riscv/mm/context.c | 6 ++---- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/arch/riscv/include/asm/mmu.h b/arch/riscv/include/asm/mmu.h index a550fbf770be..dc0273f7905f 100644 --- a/arch/riscv/include/asm/mmu.h +++ b/arch/riscv/include/asm/mmu.h @@ -26,8 +26,8 @@ typedef struct { #endif } mm_context_t; -#define cntx2asid(cntx) ((cntx) & asid_mask) -#define cntx2version(cntx) ((cntx) & ~asid_mask) +#define cntx2asid(cntx) ((cntx) & SATP_ASID_MASK) +#define cntx2version(cntx) ((cntx) & ~SATP_ASID_MASK) void __init create_pgd_mapping(pgd_t *pgdp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, pgprot_t prot); diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 3b393f765805..4448d907f2c9 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -15,8 +15,6 @@ #define FLUSH_TLB_NO_ASID ((unsigned long)-1) #ifdef CONFIG_MMU -extern unsigned long asid_mask; - static inline void local_flush_tlb_all(void) { __asm__ __volatile__ ("sfence.vma" : : : "memory"); diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 43d005f63253..b5170ac1b742 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -22,7 +22,6 @@ DEFINE_STATIC_KEY_FALSE(use_asid_allocator); static unsigned long asid_bits; static unsigned long num_asids; -unsigned long asid_mask; static atomic_long_t current_version; @@ -128,7 +127,7 @@ static unsigned long __new_context(struct mm_struct *mm) goto set_asid; /* We're out of ASIDs, so increment current_version */ - ver = atomic_long_add_return_relaxed(num_asids, ¤t_version); + ver = atomic_long_add_return_relaxed(BIT(SATP_ASID_BITS), ¤t_version); /* Flush everything */ __flush_context(); @@ -247,7 +246,6 @@ static int __init asids_init(void) /* Pre-compute ASID details */ if (asid_bits) { num_asids = 1 << asid_bits; - asid_mask = num_asids - 1; } /* @@ -255,7 +253,7 @@ static int __init asids_init(void) * at-least twice more than CPUs */ if (num_asids > (2 * num_possible_cpus())) { - atomic_long_set(¤t_version, num_asids); + atomic_long_set(¤t_version, BIT(SATP_ASID_BITS)); context_asid_map = bitmap_zalloc(num_asids, GFP_KERNEL); if (!context_asid_map) From patchwork Wed Nov 22 01:07:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13463793 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D6A8C61DAB for ; Wed, 22 Nov 2023 01:08:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 24EEA6B052B; Tue, 21 Nov 2023 20:08:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0EB886B052C; Tue, 21 Nov 2023 20:08:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E7F246B052D; Tue, 21 Nov 2023 20:08:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D43186B052B for ; Tue, 21 Nov 2023 20:08:27 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 9BD33160130 for ; Wed, 22 Nov 2023 01:08:27 +0000 (UTC) X-FDA: 81483804654.11.B2440FF Received: from mail-oa1-f45.google.com (mail-oa1-f45.google.com [209.85.160.45]) by imf15.hostedemail.com (Postfix) with ESMTP id CE435A000C for ; Wed, 22 Nov 2023 01:08:25 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=mtpOu7ub; spf=pass (imf15.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.160.45 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700615305; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=O8uQuAx5HbeGrqX4Ip/qQY7nv6lrxw4qacCJFbpKbko=; b=4H4CZR2JEm6cTPczZqR2RPpFUWSHUxYeANzgmOkoe1LQOicwg9vAez2Jt45XFOgjWpw9b3 54sfye3uq3XQC7BATi17QPo5oZXlZORg87KaUa0ZX5iR7+ZWGN3Fkoq/MrkElxku+mbUKc pc8whX+5SzpxjSfuwAvkCgtbem57c/A= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700615305; a=rsa-sha256; cv=none; b=B7z9GiEjlaS/F9oMXo5GLjrwrK2oen4tzLatEmPpoR71hKmd/z0NJ981ISxfS8xGLdoN0l ZZ5kBzBXMufmRk8oX+8ILKk/Sf/T5XCmbAKH8puRG3b1prFrt53Nf5eofAxltCjovIDBWt AE3qBH1pdHLXA8V7tQszie8j8R/3/hU= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=mtpOu7ub; spf=pass (imf15.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.160.45 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com Received: by mail-oa1-f45.google.com with SMTP id 586e51a60fabf-1f938410f92so1553570fac.3 for ; Tue, 21 Nov 2023 17:08:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1700615305; x=1701220105; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=O8uQuAx5HbeGrqX4Ip/qQY7nv6lrxw4qacCJFbpKbko=; b=mtpOu7ub6+Z+r9/pDGuSeP4cLfnJFA0o4mfTzLG4llzK8mBrcDCHvBaqgKamK5tlqh frfDxb4Iuz9A3cukPlbglmbkGYiwDNCvlFFaGuQCQVLueed5baSRornz2SKmaTJjtzP1 AKzZ96ryrc5hF1QISFvI2o+tcS9Z8PcydO6dCBKk+j+bbFq0xnIifEkch8vXUPUme7qt HpQutugIITuf4KeAT2Xw8VL+hkQe/Qb0AHt5EEyl+Epz0X+1UT9x3Wbf7GUJkZ+TQmfq u5asxz5FA6yoJbA1sHuDyCtjApHXJgM0spuwG13gqedIhF6OCnqthbv3J6gnnPmm5pzO bM4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700615305; x=1701220105; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=O8uQuAx5HbeGrqX4Ip/qQY7nv6lrxw4qacCJFbpKbko=; b=i/J8SSg0ou1A+opbuC59xMVcDMX2xI2srIiR+CfmQszGfCzYgNiNjErG1pBqC42tse uNQV+mtszBKmEIvcU5tesTaMrxBFURAUAiSi9qbqJbx5wF9P+cWtokYXDTkQ7yiJ10B4 R9Q49v/+Hr+Q6db+xezgknzvqWetnEJcqKLtfYCsxlhQABsH8eMnc42e5db2JRb7qyQt zD54ywtFDXYej10cnKB6N8LDAKEEL/UdtmG5n11BdhuaGxotXLJfLenfdalDsHOKn6Ik sev0t5/VQR4EvDp08JiXVkB3dPoxK4Q1IJbvkDjAy7EDG/OmgI2cGgpceROG029wfNCn dhGg== X-Gm-Message-State: AOJu0YxMRy0MjO5HGnoJw40GZ9+IK3Ghe2xFDXf0dEwMZsOC8gUQfYUM GbrmekWIRDnlnGGTVIDIxInLQA== X-Google-Smtp-Source: AGHT+IHzY35P/bmGNlxsboL33Yxa5yN9S+7CxJ9PS0wbWCSnN9yRamcO2jnn4/Cd8sidyrAKu3XrJw== X-Received: by 2002:a05:6870:9b0c:b0:1f5:b5ca:435e with SMTP id hq12-20020a0568709b0c00b001f5b5ca435emr1104182oab.52.1700615304951; Tue, 21 Nov 2023 17:08:24 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id bn5-20020a056a00324500b006be047268d5sm8713961pfb.174.2023.11.21.17.08.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Nov 2023 17:08:24 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v3 6/8] riscv: mm: Make asid_bits a local variable Date: Tue, 21 Nov 2023 17:07:17 -0800 Message-ID: <20231122010815.3545294-7-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231122010815.3545294-1-samuel.holland@sifive.com> References: <20231122010815.3545294-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: CE435A000C X-Rspam-User: X-Stat-Signature: 7rqc5y9ryommesnfwwd4rwfqobczajr4 X-Rspamd-Server: rspam03 X-HE-Tag: 1700615305-238755 X-HE-Meta: U2FsdGVkX19owbEmWpjrNWz9jwgw0bAU/r4BNnN7c2if3LyF7ODAP5ooWPLRxiR19/bLlfMZWKhwZW2Dw+Kzk0rJkFg/Cdz318I7RvaKH4h7I0CVyhkqAWuB/lZ5f5T0N9LOK51CVAYWskpGPLSEiKsTwBwDgV2PJOwdBzq4FJRUbLS/EIeR5nyqUm2CAZaGoq8Lf6tHj1apyehP2zK/0MB6Wpq1I+2AF+s0wMKeH4f29bx1apbsexa7PEWVNcNBGVXwrUKJAFVHeLSGygNQA4j2psZ2YuczlfDaB8Sn1bmzoUJzvevNb7LPNnrWZY++TEw9U6vqBR+vRgXuH0HcumKJ1+u/k4usqfAfwo1/d1BU5idFmARCvgVA3KdzZxnsLC140f0y9qggkK3abAV4ivWo1o8D/stChv6l2BGXYqH2HGz/Zh/sbilS/uEZl+sY0U3tB6ZmKc4ov1Kl5OGK0rIFwkRV6ioXKCpVXrE6y5y2SPChiJls4BhW3SntrEYblzpuiEaQrWkHR7TS4vlXPWnGBAbuWeZdXVTbe2746lr+yJFEqjRO6TOzrWaq/in2c+5X6c14n9Fq6bNcahwaf7g1ktRClNTM/8JpgKQTN/Mbn1vmO240L09cyah9KXlvFMkE6yNAusO0U4OZLDNvZkJa8SqrQ71/GYRMYaDdI3UWEA0fOdZFdecvaO6Xbvh8eohtAS8kkvs6hsZOcmJeHtLKaLA+RD6D2TVeBSStx9AkHcEiJys3VaIOEkq2cabIdMcgnJr+C8ctUSqRlhqk8zVc3g4VbMXTCyhUx2abKW+46282Ij5aU6io0CU9CwD90k32MCODk3PK3s6aefEv313CXpzS/n/SLWn9GjHTQ065ApsmyXpMWqEPT1HCx4NoLCvUoe+BMOoGgaxcmNwAVH2vxmzAJHU6nJTnm9BSYVgoCnLoT2WQcGFb3O7C5PIUtiHsc0P9JUbI07Yg/6G OQoSPwBh fy4dZrh6Js57DGLoLG6+w6s0tdrthgjBD8MGaPzFQj+fID8C2HIWj7yuX1Du+6/KqYlljMyX5Vz3ZxjSgw4x1dxJ/BpNFRurnX2X8VYq8Lbmd7n7OqHxYcFlAbQJ3WonTo8wrXRdq3wDMdAL+Xp7NdEImnfkc3aRplGf9mVR9EzuO6jm93ewsAMJFBdH9+TjnklPOQZcCqUqCd30VsPx03XecIORiPZI0MHcOLIkjjqmYQ/Ju0LSg5I0TknP/Rw4Sq/6A3y8wndt7uVKWopmltnTIfli1Q3Mb/SMkVjW7KrYcn3x9Glg19JoXu0Gaj8C58tlFwXm0OCzvxE+LjbGW0QDrhjVckgMDicxeXv0DF1dCWGuUBztHAr9GvB3Hc2ilgH5zyK1FJyoSTgpftclnFRZgD2uqJOGnZHf9Eo85iNovcQC8rAocJbtNm5tlXXNsxQrI X-Bogosity: Ham, tests=bogofilter, spamicity=0.003389, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This variable is only used inside asids_init(). Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/mm/context.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index b5170ac1b742..43a8bc2d5af4 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -20,7 +20,6 @@ DEFINE_STATIC_KEY_FALSE(use_asid_allocator); -static unsigned long asid_bits; static unsigned long num_asids; static atomic_long_t current_version; @@ -226,7 +225,7 @@ static inline void set_mm(struct mm_struct *prev, static int __init asids_init(void) { - unsigned long old; + unsigned long asid_bits, old; /* Figure-out number of ASID bits in HW */ old = csr_read(CSR_SATP); From patchwork Wed Nov 22 01:07:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13463794 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0166C61D9A for ; Wed, 22 Nov 2023 01:08:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3D6226B052C; Tue, 21 Nov 2023 20:08:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 35DF76B052E; Tue, 21 Nov 2023 20:08:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 162146B052F; Tue, 21 Nov 2023 20:08:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id F073A6B052C for ; Tue, 21 Nov 2023 20:08:28 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id CADD7809C2 for ; Wed, 22 Nov 2023 01:08:28 +0000 (UTC) X-FDA: 81483804696.29.717E31E Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by imf25.hostedemail.com (Postfix) with ESMTP id EC97BA0019 for ; Wed, 22 Nov 2023 01:08:26 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=GwSehgox; spf=pass (imf25.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700615307; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MROdnZxFJAn7brM6ye/jIKPJcvVCbRqlgsjUSxtP0PY=; b=tKzrruAdoyc0xLX9ZKIgM8y+uprCsE9dl9cxX2XlYt/w4uLVzdseNdPjk47Gmg1h1k/BBg VwzWzlu5fEnwMRU9UcNGnUvBhhDILxiQtBWEObBWv1US41NToqjkz5Kzm4pyR/BdeDJbEO PBC8uVqitUeiM4H+5QAQ+qoMFyYOdok= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700615307; a=rsa-sha256; cv=none; b=4E3BHRGGvUptW/knf+2UgmHjJY2xHQ+ztBze416g9pRv+pHBk0bExdpiPfGVpDwqcGeNiQ BqrUtd19zDlARNh9mZJvsUkZ5jYPk5xfygBG/cFD2MydtJKwQJG57WghdRUerozwKdz6nl tj7Ef2IJ1nNUNd/BXiRnOB0ncYCxZHc= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=GwSehgox; spf=pass (imf25.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-6be1bc5aa1cso6299850b3a.3 for ; Tue, 21 Nov 2023 17:08:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1700615306; x=1701220106; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MROdnZxFJAn7brM6ye/jIKPJcvVCbRqlgsjUSxtP0PY=; b=GwSehgoxUoyJ0r8MANoeR9L8D36WIS1kkWTLiYaT2Zo78X12kPIb5vIqHzYTr2yaxm mv5giWVyoiJBhnhJr63cFxHsM9Jlo2QH+BHEDcJ58hnQLbCXuWu+ICj3slAFR7043HBp fragPRR+pvfxge80gPMjVJ+c0mZBhSKqRQu+YQt1h1rO6XbLxTb4pEg9g9mtp1ov2o9X 4KimVQOJXlpe5Dxb1d8X+p7y8H+/cinhdK5gVAt5ISRVzeRTqKPVJHEa9mqJvvEgtPaT +Q7dwhjYU9sNf7Gxp/np3LEqsuQK3lKJ3QMpnUUfADCpyjcMfbMjOFa1nloOL/lgGfjh tpfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700615306; x=1701220106; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MROdnZxFJAn7brM6ye/jIKPJcvVCbRqlgsjUSxtP0PY=; b=tU0YGf316BkJjqzYm/Pwbod8PheWnkfcO5wzI8KAYCR58T1f3AsmjYRPii+wx7YRMJ GZ29PVUHCiHd+sqRHTNFCPm357/e7rklE+pZ22SuXjouXJsclLM9GE6zStZs30vfxniG 8jVES0kwGJxwDbZIE/HvGFzvWvrMmyT8G0lnoXaZdnLVdtZ/k2SQcpwgYI3AiMgwLTWL RVWUgDH6fZCLZMWSiVKn7lcLS5PzRYhYtJIZZLzuRCX4ikrO4DLmj1c/sFAcHpLIDEJ5 Lsu7JZdM1U2d9WS9qRu6JrrJSw+2u7w33UEOykVSccNAtZ1xPALdB7HJcdSg5Uv7JDVN fG5A== X-Gm-Message-State: AOJu0Yyk82egodoTaHJfaw7fi6D+/SdlReuo1XW9v+6qj12OUj4Xsgnd c2Na+f7HvBobPglqWeqM/dac9A== X-Google-Smtp-Source: AGHT+IFVZu+XEimHfZXOsjvtj11LrkIBAYyBlYwN/UfpJ7aqLGYZgxb4tCN+mMPanboANiNPoJtK3w== X-Received: by 2002:a05:6a00:b54:b0:6cb:a1fe:5217 with SMTP id p20-20020a056a000b5400b006cba1fe5217mr1041980pfo.16.1700615306009; Tue, 21 Nov 2023 17:08:26 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id bn5-20020a056a00324500b006be047268d5sm8713961pfb.174.2023.11.21.17.08.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Nov 2023 17:08:25 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v3 7/8] riscv: mm: Preserve global TLB entries when switching contexts Date: Tue, 21 Nov 2023 17:07:18 -0800 Message-ID: <20231122010815.3545294-8-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231122010815.3545294-1-samuel.holland@sifive.com> References: <20231122010815.3545294-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: EC97BA0019 X-Rspam-User: X-Stat-Signature: 5n93dk14qzdgjg5czwcw4yo1nrgdagrz X-Rspamd-Server: rspam03 X-HE-Tag: 1700615306-293805 X-HE-Meta: U2FsdGVkX18pR/kD+N53d0ONDMaKyhxIiOp2R1B8rgg3bhTyl4qym4VCfYoEXMsxmzghc3r8P0tvwtIRE0oNg6XZt6HS0Elk6kXS1e/7XpPuiNrdkvjaQcZSN5bADY3v9aZgPZOZvhg+335m+cGgmnlNpGcytcPEl3EYD4CXHPGAKn2zEX4Cv3tJ8bRptr4UTfSeLxXqYDv5j6HgmAiirBDXRb/cyAV49/kYC+HKbpyOh2AIEDVjKeJCxUAxoNfYxa3Ezl8mJ2eOzaDGtlLpsLp7PQTOuT3qI9IL0GJUmZF3BitpoST81HfC20GQlwDqnNzmD0Ph5M2QCZxqRlSc7PzuQICGlgJJkZL2g8vdWzmSSKSSvjXhm2WhsGTw4Wl27SVCWF358I4kW05/GL0aYjUZAnEivSEwHLPh1yPKaa6U3MTPo9mJDZEBVYBUbaTnLPteaC0uGrAD+qdc3f4yIuSdq03JH62pIcAzJBtHQ8lgURvt2jLpWNK0uxJTS90NIqnYlACQ6yRn3gvMptcLWwAdsy/jH3OoCTysERKPIBn0f6X9B4yitT67aHBLBHng9AZ5ZJQdkRVWBBVz6mWSfIqgC/6j3CPmyWuwW5tn+b8qDlXn69s1PwQTVquTN5SIhp5aD/tgGmX/50iXmeFzyNp6Brdnn/mYBlrdMPay6W6Hf1y9fwmAWkTxE+F1SbsCq6kgRW947I//hlF2mOlYxVc1BjRayGyAjlt28en372Zt7UKyi9Ny5zQaRBTKXjN9nmFQm5uk9fD8YhFgfdx9wjqN6jHDTrw0u2DnAK5o5NFJX3DZJARuvBwzXupXCi+LLMdO9w+te9MSnJhfhqtVFGf6vCejfxvwafV03XWbfuq9D/6Q0vQ8LDbrgt9NZQNI/VM0RnixSeJmKPLLwDIhOflZjA4atP4FriMRm596cYO4BslmS9+u5uWcN4qEUTiEVGJt74MCxJaqXm8woZC sitpCv4e 1h+9REw+wSO7n7bWWEdKvBHQn5wbDa8KOaiqRlDF8cA7H718XFIiMKYKB7ceDQnAvUJHqnVpi7NLIERkEGXAL+iJwjZGDiPcH6MzRndA0Ei32nX2rTyFqoy0uZDryzv+x2SZj6CjiB3XYkkHF9SlqNHsDEVOWZj9kecFUv1PtQYkzXsEHZLgB294sPmbng4IZttfepmaedmRq2QAZKzjvBsFU6Vc784aCFWw3qimKV2JkYAst7iv2OcDMDJg0Qs3vFwlDGKMwS2BcKiOeUjh3xpCXNjdO3QfXT5JC/0ay8q/q4U+pY7mzUEQmZSvqroJafzDEmUgpEnZbaRUH81KLdtaLtg8RKcclhxPtMxMDJ5qWSHOQ99RAcK3gy5vi2WgN70xt9Gqkm96mf+4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000030, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If the CPU does not support multiple ASIDs, all MM contexts use ASID 0. In this case, it is still beneficial to flush the TLB by ASID, as the single-ASID variant of the sfence.vma instruction preserves TLB entries for global (kernel) pages. This optimization is recommended by the RISC-V privileged specification: If the implementation does not provide ASIDs, or software chooses to always use ASID 0, then after every satp write, software should execute SFENCE.VMA with rs1=x0. In the common case that no global translations have been modified, rs2 should be set to a register other than x0 but which contains the value zero, so that global translations are not flushed. It is not possible to apply this optimization when using the ASID allocator, because that code must flush the TLB for all ASIDs at once when incrementing the version number. Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/mm/context.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 43a8bc2d5af4..3ca9b653df7d 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -200,7 +200,7 @@ static void set_mm_noasid(struct mm_struct *mm) { /* Switch the page table and blindly nuke entire local TLB */ csr_write(CSR_SATP, virt_to_pfn(mm->pgd) | satp_mode); - local_flush_tlb_all(); + local_flush_tlb_all_asid(0); } static inline void set_mm(struct mm_struct *prev, From patchwork Wed Nov 22 01:07:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13463795 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E195C61D97 for ; Wed, 22 Nov 2023 01:08:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 189716B052E; Tue, 21 Nov 2023 20:08:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 13AAC6B052F; Tue, 21 Nov 2023 20:08:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ECF3F6B0530; Tue, 21 Nov 2023 20:08:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D630E6B052E for ; Tue, 21 Nov 2023 20:08:29 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B2E9116040B for ; Wed, 22 Nov 2023 01:08:29 +0000 (UTC) X-FDA: 81483804738.20.DF107F1 Received: from mail-oi1-f174.google.com (mail-oi1-f174.google.com [209.85.167.174]) by imf07.hostedemail.com (Postfix) with ESMTP id DEAC240013 for ; Wed, 22 Nov 2023 01:08:27 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=mp0GnDnd; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf07.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.167.174 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700615308; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Wj2ICW1Fptg7hPvDVbuVBoYZZibiQ7SyAcM7RK2jyBY=; b=5fmO70WnmfWZuxw32ayQxvHmqv6fX5uFqnFmie3PKAeAvC7mRRz+B4KoldON9gQ3KwT561 hDxMISfz+xO9mJvNT2heql2O9T3yzTU43WLzOoVx9fxvbA4cRRt7UL/r24S4WHqWUfKIkp d6KiiK9WBtoh9sUDNfB5wjY4l0Xa6FE= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=mp0GnDnd; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf07.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.167.174 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700615308; a=rsa-sha256; cv=none; b=7tz12YGezNUhZQfrxnLtOQbpJkxyDne9B1yzhpaGJ2P4LKhuy+ycSFmb/tcG7lR59cBi6D tZJjN0p9jdBbbg4efREzV/nQyXbpI1O9wf8VvzWC/aTYp/r7NFJyVo0Xkx0LYFJiAY+tqF jyafKNFrhSbwSXuMfoJUAFFwmtIl4QY= Received: by mail-oi1-f174.google.com with SMTP id 5614622812f47-3b2f507c03cso3625798b6e.2 for ; Tue, 21 Nov 2023 17:08:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1700615307; x=1701220107; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Wj2ICW1Fptg7hPvDVbuVBoYZZibiQ7SyAcM7RK2jyBY=; b=mp0GnDndrzNR/NftyalIXOK3XrbX6bCD0G+PLEBLFr1n5RbI7Oe+V7pRKt+OiEvv5g EyGkVL2BUh24ORnUSjFcoUE8vjPIuY9IsetCLCb1jZt1Z3kLnPbNHY4dgp/SA/J6kYd3 6U1ZM4gEFqIhlXI6oQBJJDfLLMupDmHw8mMGgxo8LQ2XKj0lFjEDAfh6LEpsRLE2krk2 ByO+3ViuJHK1dPPl/SUPAvLQQs2p+iX/bfzGDlu9j8cUGYuLgBTNS85m5prOz+g4EvHY UPq+7ToHHrBpqdv+UUpyC5Bw2LprQ0zP9p0d2GwWf7lwkA9xbuEVq5Vk5oVQcqYQTZ3G PdGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700615307; x=1701220107; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Wj2ICW1Fptg7hPvDVbuVBoYZZibiQ7SyAcM7RK2jyBY=; b=USLs63Y+07M4pY8+Jm8921pKsKKAHU2ZdyHvb43xuFpaCGDPFHVjv+R3g7aPQum5DY /pMmJd8JJ4qHD0ASE/8LfrpU3qCA2bWqLUGvDJuWDbfIA27+bTwOKz/rLmkW9nGdvI0P LwYdbt+302bNcCY7Ta52w7kntcgWUhPa3gBjKhEBRBFO0wLnZaEDteqo64QNmWHALitr bLhLfb4NO3X22DYc+A5gazly+5gOD5Kf0aB/t0Uw752zQqpjxtPbcwxuDnvSwbPxnJd2 EY2b949TOa/igq95Pd+yJzrhOX2QAiN9e0qzMx1MCXlnUAanaK7daAtzMxNsDLbf9fuc Gb5g== X-Gm-Message-State: AOJu0YwFOFxtELmVX2+dKHNN5n4/3lHrOvga/1bCe9rsrOhi7De6WTKL CjhWva5Kl3aW4Bxu4xrC2jfcKg== X-Google-Smtp-Source: AGHT+IE4b2JAkpIlmqFLR7/QsgdBYQzfOQCASKlXtamU7cDqbXpB+e41Rju/TG/nF3J4VXpKvwiJIw== X-Received: by 2002:a05:6808:699:b0:3b7:673:8705 with SMTP id k25-20020a056808069900b003b706738705mr885196oig.18.1700615307054; Tue, 21 Nov 2023 17:08:27 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id bn5-20020a056a00324500b006be047268d5sm8713961pfb.174.2023.11.21.17.08.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Nov 2023 17:08:26 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v3 8/8] riscv: mm: Always use ASID to flush MM contexts Date: Tue, 21 Nov 2023 17:07:19 -0800 Message-ID: <20231122010815.3545294-9-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231122010815.3545294-1-samuel.holland@sifive.com> References: <20231122010815.3545294-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: DEAC240013 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: y4fpagfdnff7zfimd84eyo8fzn76t8ec X-HE-Tag: 1700615307-265292 X-HE-Meta: U2FsdGVkX18bR0BgzgSeUhDvjFVevAHt4YBSf5AyLf9MOG0RdyiRxJ4GydJr74ZOsY+9bzxx94q+eZHAQZvJcPflu124XZAiCuy7JEcn0mSpXUqGc8cTfiVOgp8vsjTCoRaLwjLlPN2uuiqm5doo7zOKs3q3k6AXWYfW39RYGyeTpI16VWNuQZHThhpH0jMo5LqlUnxwEhWWJmsFWDwVP3OVjdWDrqsl4F5vsRPxUXiCwrpJ/Q202f1akODic0EYvzTakaamB9hMgCr3qDIVHy+8o8jyodREFT8imv3tk5U6LSkJSl3xSpOxADJ8fcJdxsSH/RsS/nxvOyXStD59mccv2ex9sX84JZ3i9In4/XdYw6VuPjXK2d5gFpssoIx/EE9fxtg8MuQHBGbMvUydyDDjYBiFVdDyS/CnVcC97GZ5DzNGSqYMeUcMf6D+LcuuUQOTgWzijj5oqSnQ3tvKfJHe8Q75Ube2r8EZKBPXzMq6y00FG65kj1E9TNxfFUbLYHyRTqVkgi84lkCb3AYs5WLzdDWwvpvx8ZC/P/hDBsaDxQvgOtDib2zE97GY3Jfw7MMJ1R1YkEoGXrbvrXbE1lPrSj10u6yr3UU1fyD65S7gRAX9ETXPiXpsA5yNKoaXq1AAiWmt0FAh57i+bb9zWkYmHzdFLIOeGFCzzQhjX3JSnU40VZcw7QU8/Cys6AmiZN1i/w9lbrwvKZo/6eDb4Ex4c8OdyJ4trSmTo+L1FhO2G2CfW6Zh6jqZ44ruy3h2QEdG8H5qIfIz0/VjqKT2ZT/u86vTqqIEcUoxUkc5jQaWSzSkabBUsf74WbbF0Fe567ZQYu5MdYgjGdOc6d0ykIvPzN8y/4Sk2QByAkPd6AHCoJIPSAoDsBo9OZhOD7tq7z0GhetTN333RKgvwzloZHmosVMHSt8Xo4/g+cCRnOqkGzN2l24cn55xKoEWg5tluI0jh13x15y9OC91vtd 6/R6ViBP Nl8crEIrpTipjSSJDYlv4j3cxdCa7sTDf3i8J/rOp9/FLbGHjPs32bfHU8f09PRrCDOPPo+X3H+zkmsI3yUAlQEuTMA57r2OW2x9pbfuIzybvw2vGY2XWn+mF6AZItELMgNDqEPVb12phlibP6OoHvMHP/8KVrBYlckU0TfNYN7DfduYEZkX3wqsdn87/eGaAvoZV/H5ozLo+r4YIdw4WGc+5cgRM1vMEqTtBkZsQ0IbSfJfXE/qOdF5k9zgLswqItSkmzHKbTogm7/Ldu1vle1Ol3LJtnO1aucN+6pI0ox0WNxiKhppaYzVWibIwpn27gSpgYsMJ644TXSnI0gtsWyYM8ltc33jUmcMLxJNdqOF7Dd7fEXlk5M0rHrYmmbR+bG5ijcxh4hZ/gBbCTdTfvqRJ8i6i8YTtxra4u1AROprtr6dkT5Oo3yt7Zg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.001400, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Even if multiple ASIDs are not supported, using the single-ASID variant of the sfence.vma instruction preserves TLB entries for global (kernel) pages. So it is always more efficient to use the single-ASID code path. Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Update both copies of __flush_tlb_range() arch/riscv/include/asm/mmu_context.h | 2 -- arch/riscv/mm/context.c | 3 +-- arch/riscv/mm/tlbflush.c | 5 ++--- 3 files changed, 3 insertions(+), 7 deletions(-) diff --git a/arch/riscv/include/asm/mmu_context.h b/arch/riscv/include/asm/mmu_context.h index 7030837adc1a..b0659413a080 100644 --- a/arch/riscv/include/asm/mmu_context.h +++ b/arch/riscv/include/asm/mmu_context.h @@ -33,8 +33,6 @@ static inline int init_new_context(struct task_struct *tsk, return 0; } -DECLARE_STATIC_KEY_FALSE(use_asid_allocator); - #include #endif /* _ASM_RISCV_MMU_CONTEXT_H */ diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 3ca9b653df7d..20057085ab8a 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -18,8 +18,7 @@ #ifdef CONFIG_MMU -DEFINE_STATIC_KEY_FALSE(use_asid_allocator); - +static DEFINE_STATIC_KEY_FALSE(use_asid_allocator); static unsigned long num_asids; static atomic_long_t current_version; diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 23409d70440f..d6619be10341 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -90,8 +90,7 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, /* check if the tlbflush needs to be sent to other CPUs */ broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids; - if (static_branch_unlikely(&use_asid_allocator)) - asid = cntx2asid(atomic_long_read(&mm->context.id)); + asid = cntx2asid(atomic_long_read(&mm->context.id)); } else { cmask = cpu_online_mask; broadcast = true; @@ -122,7 +121,7 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, { unsigned long asid = FLUSH_TLB_NO_ASID; - if (mm && static_branch_unlikely(&use_asid_allocator)) + if (mm) asid = cntx2asid(atomic_long_read(&mm->context.id)); local_flush_tlb_range_asid(start, size, stride, asid);