From patchwork Sat Oct 28 23:12:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13439614 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32C6EC0018A for ; Sat, 28 Oct 2023 23:14:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5895A6B02D7; Sat, 28 Oct 2023 19:13:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 50FA06B02D9; Sat, 28 Oct 2023 19:13:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 33CD66B02DA; Sat, 28 Oct 2023 19:13:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 1E0586B02D7 for ; Sat, 28 Oct 2023 19:13:53 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id EABF34014F for ; Sat, 28 Oct 2023 23:13:52 +0000 (UTC) X-FDA: 81396424704.01.E6E4D23 Received: from mail-pg1-f177.google.com (mail-pg1-f177.google.com [209.85.215.177]) by imf30.hostedemail.com (Postfix) with ESMTP id 23AF780002 for ; Sat, 28 Oct 2023 23:13:50 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=QvmlC+7Z; spf=pass (imf30.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.215.177 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698534831; a=rsa-sha256; cv=none; b=MGiFfOGPUVujhZzRjiFov6SjFvzJVZ64voC6Zc4a0+NkxL7GfU49mbuy1ZmVvSGREyfITM ZS9ViBzv3dVk1cKYIj/FzxtdChvgxXNyusoIFgCUN/NRzI/Ba9LGjCqZtJmM/xFoGtyMRx S29Mwk9CLHfwykDZuTxA61rTje6xKk0= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=QvmlC+7Z; spf=pass (imf30.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.215.177 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698534831; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UP8DHxv66mJzZKknfJMFXXbwrunx+Hl9Ii6vLteV5ng=; b=yTpWAfPtatPhNDxxCIzoyFq3qZI9FwBH8MvQR7p9LLSv7O+B8Zmu5P8mSJAFYqzA4cPPz3 orXw65Z9kacF47fFIdF6mzGHZyBHitauGHpzOGkx9i3yxJ8NJDQr+yRzsAWWFH9LUpbf3k 29Ne4ZWzcCSjMuMbIc13HtISshqklbA= Received: by mail-pg1-f177.google.com with SMTP id 41be03b00d2f7-5b93ddb10b8so1388368a12.0 for ; Sat, 28 Oct 2023 16:13:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1698534830; x=1699139630; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UP8DHxv66mJzZKknfJMFXXbwrunx+Hl9Ii6vLteV5ng=; b=QvmlC+7Z0MNKqGfzzc49vfhTdGnvigC+l2UyHfq0S5PY+pZQURm2604EHlV6Jz2W5t 5T7zt8hB9BWpEfQALxCgHnAnzE8/ZGdgQCybHXCWS3g53GrqD0OtvWbIIOo2TnPJ4B1i oh4fmojjUBow0v/KjyWMoU4UQkuzviapRaz3gLXUsIIpz0Myd8NyNvsBLml4xL7gpr/k 8KZkwQoHs2p8205S+6MMwGynNqT2VXN09dMif7v62JQ1mMX+uJyAv4VyfipCfQAzjL7Y GHEy6avTiJqV/Ksn8o2IU6xylz2Pq9Oessg/sOSQNxo6pUteXs2vsauH5zXmdn3IqfXy Bf6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698534830; x=1699139630; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UP8DHxv66mJzZKknfJMFXXbwrunx+Hl9Ii6vLteV5ng=; b=bYPlbPbXCuAdzv16SDfPn4cbmHz0ZUaKbc4Hv4QXzArjMQP0MX5Js2t3zrxZfm6pI/ SQvmIwEmxLxn7Na+hWWR1iYzNna2B8wHrxNuVy+nfDo2q+X+dUdAgmJf0g3GCxTtnunU CCBQKxa956VNr/qcQBC/RGZ8GAhD3+4+O1XTIwTLX7H8b+cqBnNGrqp57hGQbcq8Y0ll 0i7EpGAt1aZZ4eVuJUkmn0TXqdM/JIlugXUa6RvjkYrFDnoAq2iqFHteRQf9u9UF5c9r Sum9PTiUb1ydL/sk2vvbxornOJ/QwIT/IMyVCwKefKG8ccGj3SsVlsKil5IIo5CU+AJW Df+g== X-Gm-Message-State: AOJu0YwcrjEoygRgbl42/567tyKvqo3SY05S6mVmEmrf3jyZWOuSBpcP qrEq5by89ewrGw8LGHfKmoAmiA== X-Google-Smtp-Source: AGHT+IGnkbRXw2gehdqQUYovjjCjlvZO0f8kuIzIEbkVVgufqh5Fzfqz26Y3uN2f14MpzFpb+9CFvQ== X-Received: by 2002:a17:90a:1506:b0:280:2a3f:9938 with SMTP id l6-20020a17090a150600b002802a3f9938mr2800689pja.12.1698534830039; Sat, 28 Oct 2023 16:13:50 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id u17-20020a17090341d100b001b8622c1ad2sm3679345ple.130.2023.10.28.16.13.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 28 Oct 2023 16:13:49 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , Alexandre Ghiti , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Samuel Holland Subject: [PATCH v2 06/11] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Date: Sat, 28 Oct 2023 16:12:04 -0700 Message-ID: <20231028231339.3116618-7-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231028231339.3116618-1-samuel.holland@sifive.com> References: <20231028231339.3116618-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 23AF780002 X-Stat-Signature: emtise6m83bjihmtmj31zxmeod1oz39k X-Rspam-User: X-HE-Tag: 1698534830-86322 X-HE-Meta: U2FsdGVkX1/sozyrG29uBygEsgGn+GVlPHewQ46ySZaCz61MHakW+uPLDgzddOFVW38lXBoGqf7FgaoTvukERkZ/pztfTuHjJ+Qo0pphq0Ygp4l2ztt65P3m+rIAquJ4qTuIxUx6ZDsO5DxNMd80pWOkDU+tR+2gBhl1fCrjwELzER2rfbsr2NHpyfMyEA0JBb4SZJHEviVCEP0OwXvQwM2tMiF1R3V12MthL24KZJbyIogu0YLUiq7TtqrrT+es1DTrF3XhFYU+127pWQskjrapkkg5WZovqTaQGUI863Zq8GBFQu3rCUavf/o1N4z4jbM6BQwMcN25E0P9sHKcyR23jxCGg9p0JV3HDdtW8wlaqcj1P1Ctt2C0VupMI6Y5/5Yl09qdhmNuvA5Es0aic3GJrYCl8y36IQ1Gh8IDKAryN5r0bXrCIJX/zc/fCWBcHlvwgHV49bqwy5MPwsgZS+yvyqpX6J2bXpOIkSzeJA//X1NvY0nOjsDnV8w2wufV3vkX9fyVYbaZavDxcMdET8JW1h+h7aLZQz3/5YWJr91iKk6++K0bY14nFt4a1a7Mz29iKAHiwf7Cu/tTFakjHCKgPqmrALkCTzfwRK6Y/7Ca3mQQ331ix5EfzGETBnIEpawKP+ZvjndcU71xe8DJ2sGYsa6sP1RyI5jHY0Pu5LP4JcbqWQ/qa5/pNhBNKx1e78umhmF6StxsPjEvdnTbWznybHtDzy5l3n1ta53k+v32TGaFuU2RZifr47/dPkAMIF4/NWtXOMUm2cTuthtWEfzjoRjrRdo58Engz+T/2J4N10+8XjaJ0ojqNIQZeMFOgxIC9jbgNOjQU6S2Bvbwp4/XiokOumsqnTqPCxdu2Ibq9x2FmGRac6IFU3VsAYeyKH45I54zuqpu6kGOXrsUH/bRRzO2nRsgc1SIuDa38Wb7gu/FC1lD3FbGbWB+rlm7OUdna4PqdJoyckA+xCa UeqZGhih tw92VG69gvoyCjUqarUQBSGWboLwanD8eYanC0Q+eHLIblJrd1c6aSM741V7c/MhnKXI1EKBtW0614nRm4WL4fyukXBH8+gKjQipSL6Q2cqHtwlCQhxXsD8gNorr/qJnlB58eKTBYv0Hs/cU/+3Pf8rTzitbm3A06rlClfWydHdSh344mbwxleJDe3WQp2FJTQ9Sf3DbzjX6uLowKWAoV2elPk1KcLuZI8bOxqArD+IwgVTTfLPM3xMh79gjqqTioMjhj2l8/TMZUGuWzn6J7Z2HwNnenzQdVn94dUYukkKwmn7QF9TUvYhmu2mHVkTOMS5yltZoHR93bXEGJzu0toqik0Rd0NwYGiXC4MiPz985hcfLsaTtSJaz/Q7uUMr1kqhBBcyH/84SoFgQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: commit 3f1e782998cd ("riscv: add ASID-based tlbflushing methods") added calls to the sfence.vma instruction with rs2 != x0. These single-ASID instruction variants are also affected by SiFive errata CIP-1200. Until now, the errata workaround was not needed for the single-ASID sfence.vma variants, because they were only used when the ASID allocator was enabled, and the affected SiFive platforms do not support multiple ASIDs. However, we are going to start using those sfence.vma variants regardless of ASID support, so now we need alternatives covering them. Signed-off-by: Samuel Holland --- Changes in v2: - Rebase on Alexandre's "riscv: tlb flush improvements" series v5 arch/riscv/include/asm/errata_list.h | 12 +++++++++++- arch/riscv/include/asm/tlbflush.h | 19 ++++++++++++++++++- arch/riscv/mm/tlbflush.c | 23 ----------------------- 3 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index b55b434f0059..d3f3c237adad 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -44,11 +44,21 @@ ALTERNATIVE(__stringify(RISCV_PTR do_page_fault), \ CONFIG_ERRATA_SIFIVE_CIP_453) #else /* !__ASSEMBLY__ */ -#define ALT_FLUSH_TLB_PAGE(x) \ +#define ALT_SFENCE_VMA_ASID(asid) \ +asm(ALTERNATIVE("sfence.vma x0, %0", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (asid) : "memory") + +#define ALT_SFENCE_VMA_ADDR(addr) \ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \ ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ : : "r" (addr) : "memory") +#define ALT_SFENCE_VMA_ADDR_ASID(addr, asid) \ +asm(ALTERNATIVE("sfence.vma %0, %1", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (addr), "r" (asid) : "memory") + /* * _val is marked as "will be overwritten", so need to set it to 0 * in the default case. diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 317a1811aa51..e529a643be17 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -22,10 +22,27 @@ static inline void local_flush_tlb_all(void) __asm__ __volatile__ ("sfence.vma" : : : "memory"); } +static inline void local_flush_tlb_all_asid(unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ASID(asid); + else + local_flush_tlb_all(); +} + /* Flush one page from local TLB */ static inline void local_flush_tlb_page(unsigned long addr) { - ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); + ALT_SFENCE_VMA_ADDR(addr); +} + +static inline void local_flush_tlb_page_asid(unsigned long addr, + unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ADDR_ASID(addr, asid); + else + local_flush_tlb_page(addr); } #ifdef CONFIG_SMP diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 22d7ed5abf8e..0feccb8932d2 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -7,29 +7,6 @@ #include #include -static inline void local_flush_tlb_all_asid(unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma x0, %0" - : - : "r" (asid) - : "memory"); - else - local_flush_tlb_all(); -} - -static inline void local_flush_tlb_page_asid(unsigned long addr, - unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma %0, %1" - : - : "r" (addr), "r" (asid) - : "memory"); - else - local_flush_tlb_page(addr); -} - /* * Flush entire TLB if number of entries to be flushed is greater * than the threshold below.