From patchwork Thu Feb 29 23:21:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13577712 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E1ADC54798 for ; Thu, 29 Feb 2024 23:22:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 94E576B00A2; Thu, 29 Feb 2024 18:22:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8863F6B00A1; Thu, 29 Feb 2024 18:22:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68B196B00A2; Thu, 29 Feb 2024 18:22:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 54B856B009F for ; Thu, 29 Feb 2024 18:22:23 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 3485C1202A8 for ; Thu, 29 Feb 2024 23:22:23 +0000 (UTC) X-FDA: 81846417366.26.A2B085B Received: from mail-oa1-f49.google.com (mail-oa1-f49.google.com [209.85.160.49]) by imf17.hostedemail.com (Postfix) with ESMTP id 6F44440014 for ; Thu, 29 Feb 2024 23:22:21 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=i5HnTCVo; spf=pass (imf17.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.160.49 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709248941; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4jEC1X5CZdJa4ryFg9hWmhSsIovSPBi+ZbblNH/6mwk=; b=ssHpx3tq+gUhV1YD7FlN9N8sOfqT5vJovtrckJl+JVGvCyyqm+U+gHJhRg2ZNjTBA2Ob+w nkU8YicrQeDINXRoAE8HO5QfZ33NARpw7WBPcIk3oshPnhcCE0/XLJkwZiZnk9R5meHDOt setSsHCF1oSaMsvmO6CUGKG2mERU2vg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709248941; a=rsa-sha256; cv=none; b=6fEsdoKmdCxWIf0hd+AMuawKAepf4m6bz9eISHd+5Uo1kjM0ve4/wCKl3n1x+2TFSOQlZZ Uf7Xehrw8gn2E+e4lBcAA5PQ7OsY7vpPl3qEw94WsXBubX6Obc/miDe2Jnx6m9nNGT4FPD TaOVj+RqvzHZduT58WGwW+hLfEhcjS8= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=i5HnTCVo; spf=pass (imf17.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.160.49 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com Received: by mail-oa1-f49.google.com with SMTP id 586e51a60fabf-21f996af9bdso615127fac.2 for ; Thu, 29 Feb 2024 15:22:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1709248940; x=1709853740; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4jEC1X5CZdJa4ryFg9hWmhSsIovSPBi+ZbblNH/6mwk=; b=i5HnTCVou5yeicsyMtoB3sjEDcciIyI6/A/CsK585vuhUOctJZTBs43SZXSlj/Ma9u D8OGqv1d6AtqHO614sIM47TaNPAd7KXnyGkrLIiXUkjgJC0b/9Df6RWoC//5N5CAv863 CRtPVedO08YwZ05kj1kshKYHSYPozcLGUmMI81VsyT/tQAljaQT90UtfIk8cIAAFhhNS WGAIrYDusI9X5yp2dFPwRqwYBNCiHBvpHpby70uU/UuU9PUo6/nsrDCoVW7nrOOD9pQu K2fnDMsEqMYMQsQYaPogOiMYPHQf4OsPXO09FMDNhYRDsHJmQzD2UfFhmJ/ujdMkdJKa XLIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709248940; x=1709853740; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4jEC1X5CZdJa4ryFg9hWmhSsIovSPBi+ZbblNH/6mwk=; b=kyTMvhNaDjw4ZRvQPFh+JPdHygipTqNgq8AH6a+5cQtTioGEBS+GJ6ARZeNtLFmW8n ppPRK5QYhjJO9nZkixQNxsbQ5iFDAj9WFpUr2itvHguyEhchbIjF9uv+Hy8iZojMq8f1 V888nX5huWNHcv/8q7KaukpFJW1Zc2QuSFbP2GI2GS/WZjhSeMDJVNOoGS0QbfYkufcx LzTz9jaO63ukDXykS4ilx32qk9Cwao4iithsR9abOHMTXEWy3EplvY1bbH56UH3Q00/z gMBW7tOvOHuAlcYDTiMn/hfdjIzAh4habJt9XbW64HmiBIphjwz6k/w7nbBwwhcRfi6O OeSg== X-Forwarded-Encrypted: i=1; AJvYcCVHFCBTEYW2cRy1dbV4eA0hZK3sChBEiEFEW3HSyYySQhb6aufQTDG0skLUfV6oHPyB/k6qW7H21rOkOvxHS18k7m8= X-Gm-Message-State: AOJu0Yyac39SNfYS0kRe2zyZ7nNipHb0m1Xn1XDMSHMbkGKng+oBUh62 KEWywMY/C4KbKM5wypwUEw4DeEWL/5S6r8QFRbY+gDsEL3AbvVP2GKARNGqyXZo= X-Google-Smtp-Source: AGHT+IGX4/HV5W58fetZh8Yw57gYr75gd/4aFWyaezvIp/BsV5LakI2yn+mW+sbVnuOSeLnU1moubA== X-Received: by 2002:a05:6870:ac22:b0:21e:dd7a:2d3e with SMTP id kw34-20020a056870ac2200b0021edd7a2d3emr4104469oab.22.1709248940592; Thu, 29 Feb 2024 15:22:20 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id c6-20020aa78806000000b006e55aa75d6csm1779719pfo.122.2024.02.29.15.22.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 15:22:20 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v5 07/13] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Date: Thu, 29 Feb 2024 15:21:48 -0800 Message-ID: <20240229232211.161961-8-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240229232211.161961-1-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6F44440014 X-Rspam-User: X-Stat-Signature: 8zkdp5gd7xqirhz7tq3bj6wsyzh8cshx X-Rspamd-Server: rspam03 X-HE-Tag: 1709248941-754001 X-HE-Meta: U2FsdGVkX184KFET5FDYrNPJun6UixvdkiO38yZodmzTMv9zWWFlO9w/AbKov4Rh7Yjpkv2X84dSv5NZgRoIY44wy/lJmxU6vDLaL72SXDpctI8lHftPgGil89dMHEZNVqfWRPoLDdsi0BC+JnGHP6LMwz4YSiOX9WTXY+PevNusFowOs/xg+Zz1O5upnR/nWc1AI1OE+cNkKFEOssvpGEZZX+t+U5jbXkczaeJ23Ko4vubITtbCeA/oT7S9nV8tYAyTDdvO/b3SuR4hLc59oLa0gsvsD1iE0qqXpEmX0h54JKMxWvwhboX6A1AewD3bcZPKt4b7gAO5IgQ8usW9NRjpjOAWveT9u2CVKR85nwUAj8fDg2T/Hbzi2AkMx6mk0146PNptxNsc69hUU06urY11G8WerY8uHRRsDKuzagccH5QJ4esL8w95oioMpyNMeNJ6lES4DBPV+zfpQZTylSufFDRwXZta47sbwyaRpt0k2ZO0r0iU7kSIYKieHBTJl2ng3GEewMK7Md+B3h9LidQHoY1+3MVKt1B/Y9SQPyrX2goxwsofXOXcEHQmEfvP77rY9tg+fDWjrXeXJzlXl5Lp5b9HcnvgChCtX+bcoq5XoQRkMscMPEbUEQ9CcSLhOUarGv0KlLFkH4HpeN7mylMpxltDm3lkGdj2JsyWyv+6K92SXEyYGAPJxUlPt+QuZWIkyzPw7DqT0ctuQRtEpuy8kGQp/CZhlGjEueY/c8KxIISZ71afizbyhl2a933P6yJ9AMxNQf5Z3Mq6moGlrgR0eMYHqgKFw8bQMWrONtRXYuL9dwnzrHyuJBsdZB+tDY5YCR5wiNMXq/K7ITcBK9SkRQ3XtDAZDcyO4+2+BuxBS+5GJnuvHOu7amBZOx1C2Wg3I/gPYXB8X1A6eZeCc2tvkH9a8Ry14nmO6uB4Yz58OBSduY8FsJArNyQXgzoIXyA0uKkZh0QEzlhtVR0 A/tczNcY 0lyGWWKCX+ZiDLXgjFQNU+PbAhQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: commit 3f1e782998cd ("riscv: add ASID-based tlbflushing methods") added calls to the sfence.vma instruction with rs2 != x0. These single-ASID instruction variants are also affected by SiFive errata CIP-1200. Until now, the errata workaround was not needed for the single-ASID sfence.vma variants, because they were only used when the ASID allocator was enabled, and the affected SiFive platforms do not support multiple ASIDs. However, we are going to start using those sfence.vma variants regardless of ASID support, so now we need alternatives covering them. Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Rebase on Alexandre's "riscv: tlb flush improvements" series v5 arch/riscv/include/asm/errata_list.h | 12 +++++++++++- arch/riscv/include/asm/tlbflush.h | 19 ++++++++++++++++++- arch/riscv/mm/tlbflush.c | 23 ----------------------- 3 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index ea33288f8a25..6e2762101968 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -43,11 +43,21 @@ ALTERNATIVE(__stringify(RISCV_PTR do_page_fault), \ CONFIG_ERRATA_SIFIVE_CIP_453) #else /* !__ASSEMBLY__ */ -#define ALT_FLUSH_TLB_PAGE(x) \ +#define ALT_SFENCE_VMA_ASID(asid) \ +asm(ALTERNATIVE("sfence.vma x0, %0", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (asid) : "memory") + +#define ALT_SFENCE_VMA_ADDR(addr) \ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \ ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ : : "r" (addr) : "memory") +#define ALT_SFENCE_VMA_ADDR_ASID(addr, asid) \ +asm(ALTERNATIVE("sfence.vma %0, %1", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (addr), "r" (asid) : "memory") + /* * _val is marked as "will be overwritten", so need to set it to 0 * in the default case. diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 4f86424b1ba5..463b615d7728 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -22,10 +22,27 @@ static inline void local_flush_tlb_all(void) __asm__ __volatile__ ("sfence.vma" : : : "memory"); } +static inline void local_flush_tlb_all_asid(unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ASID(asid); + else + local_flush_tlb_all(); +} + /* Flush one page from local TLB */ static inline void local_flush_tlb_page(unsigned long addr) { - ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); + ALT_SFENCE_VMA_ADDR(addr); +} + +static inline void local_flush_tlb_page_asid(unsigned long addr, + unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ADDR_ASID(addr, asid); + else + local_flush_tlb_page(addr); } void flush_tlb_all(void); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 69402c260a89..365e0a0e4725 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -7,29 +7,6 @@ #include #include -static inline void local_flush_tlb_all_asid(unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma x0, %0" - : - : "r" (asid) - : "memory"); - else - local_flush_tlb_all(); -} - -static inline void local_flush_tlb_page_asid(unsigned long addr, - unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma %0, %1" - : - : "r" (addr), "r" (asid) - : "memory"); - else - local_flush_tlb_page(addr); -} - /* * Flush entire TLB if number of entries to be flushed is greater * than the threshold below.