From patchwork Wed Mar 27 04:49:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4003CD1288 for ; Wed, 27 Mar 2024 04:50:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0A2AC6B009D; Wed, 27 Mar 2024 00:50:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0263E6B009E; Wed, 27 Mar 2024 00:50:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE5476B009F; Wed, 27 Mar 2024 00:50:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C9BD26B009D for ; Wed, 27 Mar 2024 00:50:54 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A177FA0673 for ; Wed, 27 Mar 2024 04:50:54 +0000 (UTC) X-FDA: 81941594028.24.6E915C7 Received: from mail-ot1-f49.google.com (mail-ot1-f49.google.com [209.85.210.49]) by imf14.hostedemail.com (Postfix) with ESMTP id CEB3A100007 for ; Wed, 27 Mar 2024 04:50:52 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="Yz1JOL4/"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf14.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.49 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515052; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kHzKH2DTKf6Vc5BlA0YNBJTz56l+llAyGgYGMd65KTs=; b=cFSXAU+iAVthXsHCayLO090EjwZ1uxBBQvfPpjVDd1TCHqdxVdkv8S12a5MwEdQMNNTt6O elVoCYNk5XbVPT/uyNz0VI5xEE9+28eIDojCHXLq54EXEz09sNG+DEYRIWo1RJ0s3NQTtw zbuE7F3KJNXp+TZcl0b5Y8JN1bGeMdw= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="Yz1JOL4/"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf14.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.49 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515052; a=rsa-sha256; cv=none; b=hFvTKtiMbP1T6YjL/Cn0wFdUWQbOfGM2X+CpbygcVINTY8qiPdTO77as25t7+6+/NB3mj1 MCah09cvD864qyIR0U/mkL/TmN9Q5ZXRVH4mx7Ow5/xqAmYXKjP9gmlbbPeaAeZng2lEqC VFp3o80ELQbx9If8Rz/nFJxjgWKHmeI= Received: by mail-ot1-f49.google.com with SMTP id 46e09a7af769-6e6f72b63e2so16253a34.3 for ; Tue, 26 Mar 2024 21:50:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515052; x=1712119852; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kHzKH2DTKf6Vc5BlA0YNBJTz56l+llAyGgYGMd65KTs=; b=Yz1JOL4/UCYmV6S4j5D6e1yCSp3yxcxQS25Aw571p8emF9JChB2k/3fp37ZzJbfp4Y zIlTNUmCi7s/rq8mgXe2o9TZuwLraXPCrwwmjg80mqtPOqLZNtu4twLmDHZrQiNctddV FknlfUhNaLgmFNY0RvxV+Q4sNuLfMYlC71SMUnX5JN34Xl/zITaHG5ByBAoAmE3tzkBO aJBtfT0FXHUFxKxlcJJyK0s5V0shm8RnhScCEyNpz6wZEMBDhaoHKkYVgSg9182/W34I OwQRTWWUdY9Z1FJN78yVoqBOOSoKMDG/v+f0VFg6Q5EUHN9OXnkajri+J68ju31hhIKj TQBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515052; x=1712119852; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kHzKH2DTKf6Vc5BlA0YNBJTz56l+llAyGgYGMd65KTs=; b=CgbM+49j+fKXbTVrNMYacZ7v1BIC0dwQF+FtCuRwLX2t4JrU2s24rRL7ShQ20IV4a3 LKf9kFox+U+UfWXF1vMcqfxGLlX4s8jydVl8S44Z+BO40/PCltHh7sBEh+UNiAA4fSIS xumBvRfJ72OQbBoKuYxSxDuC/1J0QkNIEK42iFpkRDG55TikOKDsA83TpxDhOUR4sTmL cgMnoPxFviUZDElG9NxARPG5vDzIj0uXXb4fyTd42mZ6qGH5LxlJlsR4w1rSTvk4CJF7 09+EeF7wS+PAmAL/IWe0VmapxTdvLAYN3DU0vQY58ug5sOOVBB9B1wlmbrImwbjP67dS Hufg== X-Forwarded-Encrypted: i=1; AJvYcCUhD9hdwIU1sHtAK+PuLTrdO7Kmpw5ynnM1PUTx+gR1yl7fx3FHXSx+FIMsHLjHYclhHiidudduT34+1iqk7L/Zj1M= X-Gm-Message-State: AOJu0Yx+ZpnOHMiSqXCWaNh0emXwlDrgp5dV/osiIxj4zZyt9Y8wFrV+ ptYf3nqmjbnegBc0q94rodzmEiS+iaE+meotSlAIgfpFfnGwBXwcA991Ztd3FWo= X-Google-Smtp-Source: AGHT+IFNZ9hbtEq1CmgyYTIu4vUTxuEiR33JD4yc+xyNKXepJuE48avCWWfj7WZgftYdmFVpAMAO4A== X-Received: by 2002:a9d:6e0c:0:b0:6e6:b2ed:f12a with SMTP id e12-20020a9d6e0c000000b006e6b2edf12amr1764968otr.30.1711515052040; Tue, 26 Mar 2024 21:50:52 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.50.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:50:51 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v6 07/13] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Date: Tue, 26 Mar 2024 21:49:48 -0700 Message-ID: <20240327045035.368512-8-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: hcdy1dtw4juh41rf9jwend5z6r6g573t X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: CEB3A100007 X-HE-Tag: 1711515052-114508 X-HE-Meta: U2FsdGVkX1/lCN/AdSOYLEeT6ue7/BWhgTx7QpNHF11aBQjdEvJ+u98YPLEcS9WRJfoTDgTFjeYU6B2wZs3r7qqpoYrGgY3WC7fUnLh3N4vct7EwmwlpG6rvvcRlJnqe3ut8AjN9s+SW1D76Uv9/LQ8AWpHg2b0tKq8p6rIniTm2KJI/LFuvuinBn10cLYftvlkfT1Eh/0PWWs9cCPw9YFwBcSa6gLlOdMDeiqB2vHILK1OgnnpMR/2yKHkK0/56iVyM/oW9nec26mNBmPK5+IK0igCzFyVH2UCinU7BI5V7pfUgh9IXsyWpx/msuJwoTmfHSEujbrzuCvjklMssHFMbUs/DOJq60wJoEf2BINlpADdWyOV4d/4Ca7Iccubbc/wwRM2nnX+eu5yrm17jdSWul2ecwN+sR/GkVzAKDlHcoJT8JvFYzvRpSqZtfhsBV60oKWnsTokuIqU+ZVuXS1lVBWbe1yW3rConD3a+BkLhPr2zAhqWQ0+NTQEZvI5br93wtVRrzLmFdvAap0dztUyPU5XtF3aPealNG1HoMx9JI12AFDyqq0+zzdhLewfM//e2QnyymrmYhsn3ztGhZKKxzisW11F1u+3xHFRc1bNa3131q0NuyryBttDq+PB6DKpIrqObsrgJm21aM8uuvHACbrNZ3HPT8aJ71uCbFyiQQ/8LxKv6c5LrMNnF+1gWw0WkEN/bSKIc4o/TlSO+eqb8ejvDsq1XfUpivb5b9Weu3HHtDn7YYjmritsalBz6svn0tDsB3D9RzzS2FavbpuWXzDzAdADVT7fSY2MunJpIi9cI+nYDmtoTGn/ZU9LsrGKcvWvLQ172iI/oGzhkeMWbE1/X5AAj8Q9NvwrIXSVTUjQR3WtskRfs54K44g0af6oYlvbu44+e88hkiNlq7rz6LKVngpQW/nY61cbo6SJawxlnu2cqHRb4Xs+3RmQ3keQKAlQOgryl7HadRcV gmVvaAuB Q1BQiXKS4yC/uEG+IItzbemObNOU8xKtzChWYbeOD9cf1c7mjomHigJq75/Yg9aE6uT6NhKrs6eZBTeSQyCWi6d29pj8gu8/qV+/L0XWxtWJZHFWwC5MGk39CIIFpjsTnPIYwgXR4eEIwMC4UPuTecexlKjq6+WiEzaqCELSw4SIfXl6x1oqzJ3uzczdkdeRKkbAGQhT5bT1FQJJU32e66lK28lW10vANcbk0kJnnFJQskcGDSNCcp9HqUikEI7z5/7gJPlJOrviLSril8K4CGP7hcIWapHTlwzkKNhcS2pX6oRQ9hUhSxwO3i9MC9LSPa4P/Tgp1VhiQdnkejP/hqKO5I0E0h11cUN3mLTcm1bhYBbu19izJXkQ13Uy3fjW+SO9LzNF4Yo6zKdU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: commit 3f1e782998cd ("riscv: add ASID-based tlbflushing methods") added calls to the sfence.vma instruction with rs2 != x0. These single-ASID instruction variants are also affected by SiFive errata CIP-1200. Until now, the errata workaround was not needed for the single-ASID sfence.vma variants, because they were only used when the ASID allocator was enabled, and the affected SiFive platforms do not support multiple ASIDs. However, we are going to start using those sfence.vma variants regardless of ASID support, so now we need alternatives covering them. Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Rebase on Alexandre's "riscv: tlb flush improvements" series v5 arch/riscv/include/asm/errata_list.h | 12 +++++++++++- arch/riscv/include/asm/tlbflush.h | 19 ++++++++++++++++++- arch/riscv/mm/tlbflush.c | 23 ----------------------- 3 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index 1f2dbfb8a8bf..35ce26899960 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -43,11 +43,21 @@ ALTERNATIVE(__stringify(RISCV_PTR do_page_fault), \ CONFIG_ERRATA_SIFIVE_CIP_453) #else /* !__ASSEMBLY__ */ -#define ALT_FLUSH_TLB_PAGE(x) \ +#define ALT_SFENCE_VMA_ASID(asid) \ +asm(ALTERNATIVE("sfence.vma x0, %0", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (asid) : "memory") + +#define ALT_SFENCE_VMA_ADDR(addr) \ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \ ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ : : "r" (addr) : "memory") +#define ALT_SFENCE_VMA_ADDR_ASID(addr, asid) \ +asm(ALTERNATIVE("sfence.vma %0, %1", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (addr), "r" (asid) : "memory") + /* * _val is marked as "will be overwritten", so need to set it to 0 * in the default case. diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 4f86424b1ba5..463b615d7728 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -22,10 +22,27 @@ static inline void local_flush_tlb_all(void) __asm__ __volatile__ ("sfence.vma" : : : "memory"); } +static inline void local_flush_tlb_all_asid(unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ASID(asid); + else + local_flush_tlb_all(); +} + /* Flush one page from local TLB */ static inline void local_flush_tlb_page(unsigned long addr) { - ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); + ALT_SFENCE_VMA_ADDR(addr); +} + +static inline void local_flush_tlb_page_asid(unsigned long addr, + unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ADDR_ASID(addr, asid); + else + local_flush_tlb_page(addr); } void flush_tlb_all(void); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 0901aa47b58f..ad7bdcfcc219 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -7,29 +7,6 @@ #include #include -static inline void local_flush_tlb_all_asid(unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma x0, %0" - : - : "r" (asid) - : "memory"); - else - local_flush_tlb_all(); -} - -static inline void local_flush_tlb_page_asid(unsigned long addr, - unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma %0, %1" - : - : "r" (addr), "r" (asid) - : "memory"); - else - local_flush_tlb_page(addr); -} - /* * Flush entire TLB if number of entries to be flushed is greater * than the threshold below.