From patchwork Tue Jan 2 22:00:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13509533 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5984CC47079 for ; Tue, 2 Jan 2024 22:01:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CC3966B02BA; Tue, 2 Jan 2024 17:01:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C739F6B02BB; Tue, 2 Jan 2024 17:01:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACAE46B02BC; Tue, 2 Jan 2024 17:01:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 9500C6B02BA for ; Tue, 2 Jan 2024 17:01:46 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 5BBA9807B2 for ; Tue, 2 Jan 2024 22:01:46 +0000 (UTC) X-FDA: 81635743812.07.FB275A9 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf28.hostedemail.com (Postfix) with ESMTP id 882ABC000F for ; Tue, 2 Jan 2024 22:01:44 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="mqWS/RZk"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf28.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704232904; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6c+wYTacIgINHvokNTIzxqzbFxSPL5G6VDrdebR45nU=; b=YECNGlXChk1NQK+pNB02IlUH9WYgRiJO0H/7AQVnMSE2o8K3yMKLkhj+wt/rGDwfaKU6Su ZFgTm/VwrIubEZtEirkgrjCQIr+txdh65hiH8pB76PoRY/c5SLSQXs0HgEqZSmn+DaVNrx ew06C1LelnDy4zhJvouusLKUVIBkOqc= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="mqWS/RZk"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf28.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704232904; a=rsa-sha256; cv=none; b=4uYm+3RXif4ZwuqsLa9SD4w2rc3czAfDMhfBu/r6Hgeaxfu/u9uYExhaMqcK+YyxJ9ASVR QBwDSnywnzq18b2+BVRxEAyjtEv2RWVu9LYwOlDiDiJaAqWksF5FtK071NCL77TGeIUcbt 46j//cvepMLIFA0t80GMJJcG3AUw4vQ= Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-1d3e416f303so24955855ad.0 for ; Tue, 02 Jan 2024 14:01:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704232903; x=1704837703; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6c+wYTacIgINHvokNTIzxqzbFxSPL5G6VDrdebR45nU=; b=mqWS/RZkWBFHASn1szrVdQ2ZAdDGV3IDvcXDcch7wXArBB4uNZo//Koh4Yg0fzQjKX GMmzDnmvEhgumIA9vCNMM6USSRk1llvC/971caXgZldGDKLHFmP4nDfh42+uWvmMBVUz JLRKd9S43/sBqkdoadFVRGSAtJAB3t6u4wz6BjhB8xEctO/lC/WG5EXIoJOSypin2c6y OaASkXzBSokHn4yL6mFkOWnMlqO4IgR2rcIxoEDBTFUaKhybd2ZdXlQNa7o+ePU2Fe8K XO7L+2TiYyWaj8cr0UXf9tapfxWWWwCDsXfMwqjkG1Yx///TsRYdQq6LVierYdcVsSoY t5Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704232903; x=1704837703; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6c+wYTacIgINHvokNTIzxqzbFxSPL5G6VDrdebR45nU=; b=Cytwevv6SYSv2GSWT1/ym9sZ/ZfZyew8si8k+Py3aXWCe8MMK+M4fll7S+wF7rN0Pi GRXyhIHAAEHrPKjybc5i3mfjocQYTSAj5U/Fl+Dxx8ar8IozgEglEIwboj9JRg/csOWQ 2qjjIo6AunXhM3+iz8ObOfZa2+/RDY08KhGLj4OnACN47cyqXVlReMkIZam8jjRKcCaF HNaxBgVJx2bxACAu3V9roPTXBDGQya05z0VrxMJS6i9f7RhgAI4vRjSdMQeWYdKSxW6Q JsrX28rBlxccylZXUdg/bEWNTasgglf7oRPLs0YEl2MwrcL6rSZKL1ZWkzFYpTC9ddHX MZaQ== X-Gm-Message-State: AOJu0YwoNx3vQViV22WlM3Bta1eSWv4HN4d/bnCrveVxTQeTsWo5Z4Gu zhZJ9TWqPhrHENkNHRb0C6dnkXC6Ew9ogg== X-Google-Smtp-Source: AGHT+IE2GdHLRUcW4ROKdXldqRiWcx2M7fdl7/RqRT5d/m4h5bCpV21xGk3IklfQQqw3IhSZ5sLqag== X-Received: by 2002:a17:902:d584:b0:1d4:5f9:d3d7 with SMTP id k4-20020a170902d58400b001d405f9d3d7mr8348439plh.127.1704232903384; Tue, 02 Jan 2024 14:01:43 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id r59-20020a17090a43c100b0028ce507cd7dsm101724pjg.55.2024.01.02.14.01.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:01:43 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v4 06/12] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Date: Tue, 2 Jan 2024 14:00:43 -0800 Message-ID: <20240102220134.3229156-7-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240102220134.3229156-1-samuel.holland@sifive.com> References: <20240102220134.3229156-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: bqkxgfh48owzcq7jth3h7u7edsfm6qwf X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 882ABC000F X-HE-Tag: 1704232904-707802 X-HE-Meta: U2FsdGVkX1/yp1ZcchS7EZlI2pfy4n+EIUjJxIS03GOGoPJ5Jx7rLQzTaaJM8i5ZtMbi098D+MLT0vJogJdS1r6CvM0j064beG0l8hCosNPh0S9pHgnI/t/cW4QOcAug95ujtU+/Jerzjb70sPHOfPonbzm/cqWJoFbcoMfDB3lELRwJL7vM9OncE66k7mEAVRCXKixY3riBTmoH2NTs2dySnZSAGn4yEIiaDZ8Ir+yMqMT0ZwScuQN0dnlKMkC4EGYql7pj5AbSsOplo/TrXe0/+oE6oLZB9To47pPOttddPAdxWZ1paXiXNGBQfSZYvZJAKKGANInvtaG/xn+QQrCLWTcLZ1IlFceJg4JJ/3/C3dkZQCsdstMyxmVl/KjW+LvatI3qbbI3ju/CSBvzPh0beLbV6ec4UlSC5CTJCp/LSMWTHizgWxsK8OlkGS24Y0+VPMsa/0Rf6ED994WDN4ROVzD5pgxK4riohNnLWNh5cn/+f1lQD7UIA3qfA0jBY9FvKKSTIQuAopzBcbGJok/zRzUWJ0F7WD97VPhEcouc3Q/vgDsPqJwUeQTLSoWqSiDOs97CTzIaEP1f/PVQFPqV3XgD6DeRE4AJ2A8uoX2taJQQYa28kD+DnGCEg0fA8vZV7N3W2RzhR8M5uQVzUa51yHR3k2MTcMpFaSNwkHpfiajD5PasvFEt+chIwQwG6Bixb4kHZt/kYuEcS3au5e1f70jDtLXyB4I7nE5BNQI4ySmWO4VAHmwPpGZtB9wiCbzxMX74hwom82n2gJTnQCcifyR4ZxnzliqH3MhcDygN1bdi/rkGrnNNOwqHNB8fkD6pjuMK4vW1cBiqxgXVM6LZ8ioZYR8NxvpHweDFnv6+nbM1Os03cuKE9e8IX6qo5BchW8NBigwXHRBTebztgxAKUn6+KYU7f/csClxuiNPbyqchxOD45KrErvCbFKSTqSMnrOyCiDZa4OXU2dF iGDm5xZ+ 6UQ8HmrZmCTwGZ7weQQb3CGm+r+rpPK+g2F7UHJwOkeIhoB0GQxPxwhJdDf04DKgoxPV0DSyFhXbuGSYCSfC7U7yJ+E5i7fY2iAVoQ8lLWqVeGN4DTmIpoBlmy9+Uqr/jTtWYLBWOfXpc2MdlzvDiPpdV5RWOrfqNwTWUw8jvw8XqrELa8ETHZuaDyrIaq8wQtp1rBAXwbzusUKpJyriclzwPgZ9Aw67BUfNyOp9Hx4CENGb/vglXv3CQIsipcXKy4iVBl1/X4w5+5ViQGdBg2rD2smw/i5iAU/i+WrvJagNvNTRCTdMys2x8FiTins9OBhXlZyw9NQCGtSQn9vHRoj97p1V/2Qp2QoHEOxayCEJz1viddrZr7aRWdQTwp7jdFB3+gXRMJLlSLpg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: commit 3f1e782998cd ("riscv: add ASID-based tlbflushing methods") added calls to the sfence.vma instruction with rs2 != x0. These single-ASID instruction variants are also affected by SiFive errata CIP-1200. Until now, the errata workaround was not needed for the single-ASID sfence.vma variants, because they were only used when the ASID allocator was enabled, and the affected SiFive platforms do not support multiple ASIDs. However, we are going to start using those sfence.vma variants regardless of ASID support, so now we need alternatives covering them. Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Rebase on Alexandre's "riscv: tlb flush improvements" series v5 arch/riscv/include/asm/errata_list.h | 12 +++++++++++- arch/riscv/include/asm/tlbflush.h | 19 ++++++++++++++++++- arch/riscv/mm/tlbflush.c | 23 ----------------------- 3 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index 83ed25e43553..6781460ae564 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -44,11 +44,21 @@ ALTERNATIVE(__stringify(RISCV_PTR do_page_fault), \ CONFIG_ERRATA_SIFIVE_CIP_453) #else /* !__ASSEMBLY__ */ -#define ALT_FLUSH_TLB_PAGE(x) \ +#define ALT_SFENCE_VMA_ASID(asid) \ +asm(ALTERNATIVE("sfence.vma x0, %0", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (asid) : "memory") + +#define ALT_SFENCE_VMA_ADDR(addr) \ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \ ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ : : "r" (addr) : "memory") +#define ALT_SFENCE_VMA_ADDR_ASID(addr, asid) \ +asm(ALTERNATIVE("sfence.vma %0, %1", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (addr), "r" (asid) : "memory") + /* * _val is marked as "will be overwritten", so need to set it to 0 * in the default case. diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 7712ffe2f6c4..002c4c2620f3 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -22,10 +22,27 @@ static inline void local_flush_tlb_all(void) __asm__ __volatile__ ("sfence.vma" : : : "memory"); } +static inline void local_flush_tlb_all_asid(unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ASID(asid); + else + local_flush_tlb_all(); +} + /* Flush one page from local TLB */ static inline void local_flush_tlb_page(unsigned long addr) { - ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); + ALT_SFENCE_VMA_ADDR(addr); +} + +static inline void local_flush_tlb_page_asid(unsigned long addr, + unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ADDR_ASID(addr, asid); + else + local_flush_tlb_page(addr); } void flush_tlb_all(void); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 37b3c93e3c30..292d7cf3c4f6 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -7,29 +7,6 @@ #include #include -static inline void local_flush_tlb_all_asid(unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma x0, %0" - : - : "r" (asid) - : "memory"); - else - local_flush_tlb_all(); -} - -static inline void local_flush_tlb_page_asid(unsigned long addr, - unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma %0, %1" - : - : "r" (addr), "r" (asid) - : "memory"); - else - local_flush_tlb_page(addr); -} - /* * Flush entire TLB if number of entries to be flushed is greater * than the threshold below.