From patchwork Wed Nov 22 01:07:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13463789 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11DEAC61D97 for ; Wed, 22 Nov 2023 01:08:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AFF466B0526; Tue, 21 Nov 2023 20:08:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AAF506B0527; Tue, 21 Nov 2023 20:08:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DB3B6B0528; Tue, 21 Nov 2023 20:08:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 73AFD6B0526 for ; Tue, 21 Nov 2023 20:08:22 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 5777A1CAC95 for ; Wed, 22 Nov 2023 01:08:22 +0000 (UTC) X-FDA: 81483804444.16.D401890 Received: from mail-oa1-f54.google.com (mail-oa1-f54.google.com [209.85.160.54]) by imf16.hostedemail.com (Postfix) with ESMTP id 89C8C180012 for ; Wed, 22 Nov 2023 01:08:20 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=YMvYOThq; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf16.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.160.54 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700615300; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4E3lnavwhWkbsPt/8jollKuN1Wp7270MVQq6Nf32IZI=; b=dLdRKJD+Hd7+hdliZIUbM62ysYcMXe9o4AP2OKl2aJEGwu8rTbwMkPk79DaT5QpWdlkJFo wlutQY5UJIjF4aiF6oQf2dkGcfxUtURBVBZlU6f7wL7cFm+pkCC22eBXOc/t9B58IZV1TB WDREFYFDlnbNpXJjYjOVC8w/o5WHXSw= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=YMvYOThq; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf16.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.160.54 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700615300; a=rsa-sha256; cv=none; b=rN37Sax8XHJNKPCyG59SF/+XqG4lqMHkZIzM24S5QdMSaPY+oz0OeXXGFLKMSnG4awhwYF mCNQTq8NvltFGa0J6pn8PfW8nrYhgpsvowOOtVGpeBkFuWk4kWjmyP2JA+DAtB39bdAaWr 5hRtGRBAnb9S+kFKMXAoG4OGCszvHzE= Received: by mail-oa1-f54.google.com with SMTP id 586e51a60fabf-1f94a097b5eso1436905fac.1 for ; Tue, 21 Nov 2023 17:08:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1700615300; x=1701220100; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4E3lnavwhWkbsPt/8jollKuN1Wp7270MVQq6Nf32IZI=; b=YMvYOThqZGmSbK2oUqCiG3xcF8vIzKRZPwhCJmLAmGIf7GVYnLWHbtZEkJ2tt6UXDx XSZ9dF0y30czxDWjwL0k3bbB30g5X7snjD5mRDaQVE6ootMVuWQ/LsaBa8ux4FP83NDo 4vdT41WVHynvByLH3Zh8L9rl5z45RqDUzjSIU36GODs2iMwWwJOknGyxIpAzwSTHpLNw WC6h/Pq9cXPt/rjvEtKbQfR2dAyC+AgG5r3cuBV5diUIF55ES50YM5Pr/6Y038wV+NGY ktMwkcF2qdl1gmuHeICkT4yRPXKnqncypQVTLETcDI3QoTOJi7aXXrwAiSOce4BM2r48 KoeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700615300; x=1701220100; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4E3lnavwhWkbsPt/8jollKuN1Wp7270MVQq6Nf32IZI=; b=G1qjGjcAzcGLJ6eLHX41H+x+nweWN9ilJNvPGCGpfazHeaRZsUYSi/z+IMbocREX2r 04NlZJDH+2zyspsROeUywY+ZJymS62GI86Y9A2pXbw3qPXfbixxgxxBfZy8avdIZqF12 +1QiT5PuDGtm7FymSUkv0Qv0soocCRXno+78Un1qLKTlXUo68ACh0ENMYM9l6bzN8oay 9D/k8PeGERaJvP4SqnyfhKt8ztqsbio4QtBz/MownoQqbSY0dF4C+zBnvPqN73YQcXBq NuvCZj+4N9EvWAmdOmPZA/ADH9wpisqpH9h7R9gyf7DiGwQJnmzzApaVbLPXknVJx5jX 4Lkg== X-Gm-Message-State: AOJu0YzE5Hq4ZZrlt6i1F9D+foa81XcfJwzJKTXTg7qDe56EhqUJm5/r fvtwuJ9YWhu3SXfTMzIRlHgKqQ== X-Google-Smtp-Source: AGHT+IFhS1aXruOMSeKbF/kr742mxFr/4ERJTn5q1RSxs6qFscJwCL7KnIKBT8DwBL+aJBuJSXtucQ== X-Received: by 2002:a05:6871:e805:b0:1d6:8b7f:ce89 with SMTP id qd5-20020a056871e80500b001d68b7fce89mr1213468oac.51.1700615299719; Tue, 21 Nov 2023 17:08:19 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id bn5-20020a056a00324500b006be047268d5sm8713961pfb.174.2023.11.21.17.08.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Nov 2023 17:08:19 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v3 2/8] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Date: Tue, 21 Nov 2023 17:07:13 -0800 Message-ID: <20231122010815.3545294-3-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231122010815.3545294-1-samuel.holland@sifive.com> References: <20231122010815.3545294-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 89C8C180012 X-Stat-Signature: 1dxrbzpg4916zbrthjkrt99dsdb4cryj X-HE-Tag: 1700615300-399477 X-HE-Meta: U2FsdGVkX1+6DZvy4e5a3Bk4P3nQbXuDS1WN8LLTKfL8k6noAl+gBxlGXs23qAPVafj8jkCP8g9HOxN67APU1urGn9GL5JVPDLAbLGUJCFHFseA56WVgiwoSDhoTel80uXvgOtgV8HRdxYm1BcHnvYNgflobuKwI67xERlMu8dtd+DwaehnoI/oKm6cL0s03tf+SDxm5uptkhRt6ifvZs29i/16bI1s0vyCiZdYEK9vWedD8pLEV+2u3KNqsvLvqEN1ulS6zmsASL4+1Q15yDultE76K50MXGO7eLbRjgzdmcPFZYbKpPQUO0mRLKtL1T9wpP2+/Sj1xsKnKKvq2tAmv3IdvMkr/m8uRWQU0eE4m9SZhlPNtXE4/cyfYCJTbgkKTbXN94bgtWSAkcO/GD9bHEptgcCZqcnJBMM2joCB3D55kfn50ZKA22OGJegUbmK8dMtmaJu05DdBrE3MR6I2BHWsd5H/3uaebrdMGGwyRQFS5a4jT4I2jGN3TWB8nke0BwLxeDH7KvTvR7TXcAZJUuzbitd6bjekpcaqBNRvsEIBANsxVD4kru6vJUEINQ8jXoYU9aE1pgIk77QSc8WqUpQsudRs0K8Ubd+ggmEwo6d84CWKrk4UkejQbEiIVtar+jqWz+sfa23iSADWl7GE/87HUIKUisnM0PQrOEB2KHQQemaMIaiwrEjWeXYNea4QRQIEvgXQZwm2HxXCQ+1CEF9XwxOQQjNwCuXFaPTO5NV8kxjt0Iepnyg+ngky1mh86ksPD4SuN+Yp5JbwJBPyiFlt8pba8VUymAjNgnNGJd4oLSSRF6rhf5kM8kq31vCpvwwrAnr+cBQKvFTK+Ubm2/hf9OdaPXRW5Ehnv6fFncCj4sMonY7ztdac1J5nTQzuSwNI1YH1NRf4htd9kriQ4nB9GdbEv7hf+PziWjXZNJiVZjonjlmvWZOj3c8ieyJoO8hRILcqskf+OPU0 /QHYRAyj TJRc39FFrRhyuzmpMQ1dhS85/NIFk3CXkRiiK4uSchNhMpysnx/xEZVTF67mTFRYPfYKRTYGQE5w2X+BzJrhhOOZ/Crq25amY0Ns8ywpHNkrmhZsJ9XcExaoGoBXMossXilBQMn/8vOkuWv5K/mFKbbZWNYAy9geKDnMtijckbVF99j6Q47+GPZ7lpGzENayxC6TlQw65vQC+NbUhg9fpl7Zcgc54QK8UECxIVxvFR0WqvbiMnl3dqfR/5yoVntW8p/RUAbfx4C/r+aViZ/rF71vMBxdm3GJakSF8WfNRRSd8miZ8wWaDDUWRzDRs3S8BelquVr6aIjqit5BX5acJB9i/MT44DhT3POftD6456HWnvj15610NppHkHwgfI8eWtOz5rLKVrrU5eAk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: commit 3f1e782998cd ("riscv: add ASID-based tlbflushing methods") added calls to the sfence.vma instruction with rs2 != x0. These single-ASID instruction variants are also affected by SiFive errata CIP-1200. Until now, the errata workaround was not needed for the single-ASID sfence.vma variants, because they were only used when the ASID allocator was enabled, and the affected SiFive platforms do not support multiple ASIDs. However, we are going to start using those sfence.vma variants regardless of ASID support, so now we need alternatives covering them. Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Rebase on Alexandre's "riscv: tlb flush improvements" series v5 arch/riscv/include/asm/errata_list.h | 12 +++++++++++- arch/riscv/include/asm/tlbflush.h | 19 ++++++++++++++++++- arch/riscv/mm/tlbflush.c | 23 ----------------------- 3 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index 83ed25e43553..6781460ae564 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -44,11 +44,21 @@ ALTERNATIVE(__stringify(RISCV_PTR do_page_fault), \ CONFIG_ERRATA_SIFIVE_CIP_453) #else /* !__ASSEMBLY__ */ -#define ALT_FLUSH_TLB_PAGE(x) \ +#define ALT_SFENCE_VMA_ASID(asid) \ +asm(ALTERNATIVE("sfence.vma x0, %0", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (asid) : "memory") + +#define ALT_SFENCE_VMA_ADDR(addr) \ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \ ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ : : "r" (addr) : "memory") +#define ALT_SFENCE_VMA_ADDR_ASID(addr, asid) \ +asm(ALTERNATIVE("sfence.vma %0, %1", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (addr), "r" (asid) : "memory") + /* * _val is marked as "will be overwritten", so need to set it to 0 * in the default case. diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 317a1811aa51..e529a643be17 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -22,10 +22,27 @@ static inline void local_flush_tlb_all(void) __asm__ __volatile__ ("sfence.vma" : : : "memory"); } +static inline void local_flush_tlb_all_asid(unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ASID(asid); + else + local_flush_tlb_all(); +} + /* Flush one page from local TLB */ static inline void local_flush_tlb_page(unsigned long addr) { - ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); + ALT_SFENCE_VMA_ADDR(addr); +} + +static inline void local_flush_tlb_page_asid(unsigned long addr, + unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ADDR_ASID(addr, asid); + else + local_flush_tlb_page(addr); } #ifdef CONFIG_SMP diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 22d7ed5abf8e..0feccb8932d2 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -7,29 +7,6 @@ #include #include -static inline void local_flush_tlb_all_asid(unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma x0, %0" - : - : "r" (asid) - : "memory"); - else - local_flush_tlb_all(); -} - -static inline void local_flush_tlb_page_asid(unsigned long addr, - unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma %0, %1" - : - : "r" (addr), "r" (asid) - : "memory"); - else - local_flush_tlb_page(addr); -} - /* * Flush entire TLB if number of entries to be flushed is greater * than the threshold below.