From patchwork Mon Sep 11 13:12:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13379326 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25329EE57DF for ; Mon, 11 Sep 2023 13:14:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A1B326B0074; Mon, 11 Sep 2023 09:14:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9CAD86B007E; Mon, 11 Sep 2023 09:14:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 86C3C6B0080; Mon, 11 Sep 2023 09:14:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 73FFB6B0074 for ; Mon, 11 Sep 2023 09:14:33 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 485A3B3B17 for ; Mon, 11 Sep 2023 13:14:33 +0000 (UTC) X-FDA: 81224360826.23.8E95992 Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) by imf05.hostedemail.com (Postfix) with ESMTP id 5B9C410001F for ; Mon, 11 Sep 2023 13:14:31 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=QHShZum3; spf=pass (imf05.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.42 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694438071; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jTDHLs2EOQtTQ3uvgmHng/nVHNHrVDMEJR/1UpGvkHQ=; b=G7rmEZophibnajLDnPWN1/sxq16rYZuNNjGH0w/VtB7CX/jsV9x0m59Jbe2t1ZTTa9Q4sH BExH2ZKzXM5F5/c9aMXuiMvMMtVHvPIWeMJJT18ItDGqqiNvi9jdRHRV00JhpVIItYVe2Z SFojDWlBmC2mrPpeiObSj4kB0aHBc7I= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=QHShZum3; spf=pass (imf05.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.42 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694438071; a=rsa-sha256; cv=none; b=FAtTI3SVTn8v4J6XvJj3+CLW7dh59eGrzc4OQBPPREUEeRfu00n4W83lO8trvFW2yf9x5d 1Fyv0h/vk08+v7GlvkORNBbFf6WQ9tjCpb2Xw44krQBCieeHITtTgCBUpnHavgfHOHC3U/ HbSO1hwOKn5GuoSM7Ndd3g1UifFy+0g= Received: by mail-wr1-f42.google.com with SMTP id ffacd0b85a97d-31dcf18f9e2so4448283f8f.0 for ; Mon, 11 Sep 2023 06:14:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1694438070; x=1695042870; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jTDHLs2EOQtTQ3uvgmHng/nVHNHrVDMEJR/1UpGvkHQ=; b=QHShZum3XI/lL30ln9fhFq88dCc6q4FWrucEDTt125R6hfsmeVBRC7rwsCzvHKX8WK veNwJWr2SlKKCVsFdJmnjc890PDGQgm25ePCN9fFZlYTFG9TpNBJ02kGH0YhbU6lboE/ gsJRXXv3hVMXni94Nl7VAMPL2e8NNlrMnBlGChRHqIEF9fV9xTU5G3vkwVjFpwcP0oDq hKTxMlYlg92eEn37xEzxm1fG16jmjncHcL9MVF4TThhr4gCUncQFg49V3dfn3tMbLoCM 1q0NcntLLBg3cdd3J0Q6+Cco410a5x0PmEaqstvSRTpkqheZIV8A4SxrA5dKO35k38g0 AwvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694438070; x=1695042870; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jTDHLs2EOQtTQ3uvgmHng/nVHNHrVDMEJR/1UpGvkHQ=; b=i8sMiCp0giN1hB4iHsOsAkShaLraxBT0yaaTBUfYDRtSWX1BYHZK6UNgyd5QIAk+TH gY0pp1l4xQssGP+IcrMnl2s+q5fSCoT4l1/IcKCUhE0/SKz0l8fHl3wUsaoOsMMZp6zK e+hCALAszDrOsIXUhRgx6b8tg+/zvUKHeIbVmc1+ly6iW1jA8f6MwIR58PwerBYKquNY z+8kAgbnWzDWQImcCr9VfW26AIDAyfpiiHdxovmug4bqUOmXmko/c941YS5jCqdlpAA3 Yj1FYO8B0IZNiwu7v2WN0t13tAHH0lvbg8bStAxPs1RBe+ExJed3zJp3zblkbtWhmo9K DBnA== X-Gm-Message-State: AOJu0Yy6FX4pPCS8B9xKMEQ/+Bcq3cIjlwC+R1YXEyybWtq8ppaTHCjX d6N//iW6WOZTbrRAlFPmIdXXBg== X-Google-Smtp-Source: AGHT+IGYcFnpNiP+FLLZHYSCC5nHXUyn1lvxXozVLynaGkKKRW3cVE8D6L3lkbbcr2jik100ZVPpZw== X-Received: by 2002:a5d:6510:0:b0:317:ec04:ee0c with SMTP id x16-20020a5d6510000000b00317ec04ee0cmr8022431wru.47.1694438069661; Mon, 11 Sep 2023 06:14:29 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id v11-20020a1cf70b000000b00401d8810c8bsm13230128wmh.15.2023.09.11.06.14.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Sep 2023 06:14:29 -0700 (PDT) From: Alexandre Ghiti To: Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra , Mayuresh Chitale , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Samuel Holland , Lad Prabhakar Cc: Alexandre Ghiti , Andrew Jones Subject: [PATCH v4 2/4] riscv: Improve flush_tlb_range() for hugetlb pages Date: Mon, 11 Sep 2023 15:12:22 +0200 Message-Id: <20230911131224.61924-3-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230911131224.61924-1-alexghiti@rivosinc.com> References: <20230911131224.61924-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5B9C410001F X-Rspam-User: X-Stat-Signature: rpzknaknddr3waurhhpkn9wn8paakkdq X-Rspamd-Server: rspam01 X-HE-Tag: 1694438071-809451 X-HE-Meta: U2FsdGVkX1+9xR5FSxffknD9dWcNevrFLZS4n48KCE7ILjrMuW3qVW7E9cVdh1WYjB70Yl+oivQMk2hvlTg40g7PyTzQy8/+zpdGsUyM4UiTN+q+IKF3mNG0Ag/mZ6c5fNU45HCCB3stOKKQIhtNvfgHghN9JxQ9SM5wzh6Mdezeyxd0WP4vL6cTFmHncCklukZEvr0gjK82ouHsRwKzpa0phzOSJVbnH+NRplu3WajIlNp5QVb8qRTCt4ajg0nO9AZiR4NKr3UjRK/GsUM7sT2Y+BwUceX/6hWNB/lzduAG34aGlkBm9fhWWPFhp+Ctm0X0/bxAXDzUwkxnEIMKeLJWqnaBucYea2AJOv39GsyIw0o98oLhggbuFHekcvPDCFVJCkWTPIOym23/2SZns7d5KvkzJyYXbqSg3N2gSx39gPuRpZlXix1AggkYdiRLSk09BaSWDj/VJ3oYlpO8VBuUdkTmW+o3JLErJBYR0wAwhbMrghWcMARirJkE2XaPV+F3DuiG68jRz6/8PHBkDL3IzdtBwbc/wQ7ZKNyX2JnCoXuCmsArzRMCl1BpWj4U4svpDthFBBkpT6dGWsJm/663wq37zEe/LPGIO9PJRNWCdWbdlCIKY2+/cM3St+GlUNFlhyh91nBNfz1wU/9Fa8YvpvGPSZyjWPz79su76x3gaDG7lmmpKOE2WFnj/ZwjxkDS6toMonXYM+YJOFfdHxUXtO6Lch+NhLLwaNZl0C65oEymf1/JKJrEnQYlWvDB4j3pBPsOAK4EGYyOXXMDErAuNNasXZQWKr1gehNff44GKDtn/Cx39TrHTgZo/5makGPw8nRY4efx55YLj5ckiFOAVt399plLFbk/t5L4SZ2NI35xY6pabBV1xZhEOWFCF4NDe88OdsSmKzvMie+yZt3gI5ZT+kkJckxffTWWD6klwty5NLN+Z9lcQQB+TVmsyVUej4uHbIHPN8iqMDa +624V040 aoXxJm7fwIuVKYRRJFUQAlsRSJBFxZ/Wh31Ze11qG2fqylj6pKgmwBtPa7Muo4h02SI8JpZIpP284tgvWH9VDlVVR2yDLfmCoor3PMO02ctVnYaCx0uj1Ly9wKYQ9ZNOeGIo2Ef1FWfhSBGv5YrTzGpO4D1N/dU2dBAikdhPzchhTGzRAzP1ekflYrewXJvAFhlUk++aV/r5OjeQ/AtfO+8RCpkqsjcJItlYbc4UgMx11/SSEWnmR6czl6id74vAiGvH9Wib3WmnK2nf2REJr2ehvbHDFvFLelqrwCMsacWZETnzFwmco8VjqghjIkAt6OWKxI3yom5wGZ7RtyUwDpXVqyrgqNdalPeU315K/ZntCj3xCgXCHhYkQVoCQuZKtLo8CWXC4XozPfNr4OyY6EbrbIRqvEyu8Rdlntumn5Qwyz298uMUThsKqduscf9GwKApPsj1tyOtW01XU8hycQIKUi3LGjyc5gRy/aD6H4eQ6K64G7vPlOXXb4DzKIQn2EFQGbQTJ/8q25kKGlftn58ExTeAmb7JDpgO9NEVydPGJZ+y9WeZUGkJxw2xHxdnR/8013Ss4ffBcojOX6JOyQDEFivqWz+J4Mxi+uFSg0oPh9JoWDu6gj5QqVw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: flush_tlb_range() uses a fixed stride of PAGE_SIZE and in its current form, when a hugetlb mapping needs to be flushed, flush_tlb_range() flushes the whole tlb: so set a stride of the size of the hugetlb mapping in order to only flush the hugetlb mapping. However, if the hugepage is a NAPOT region, all PTEs that constitute this mapping must be invalidated, so the stride size must actually be the size of the PTE. Note that THPs are directly handled by flush_pmd_tlb_range(). Signed-off-by: Alexandre Ghiti Reviewed-by: Andrew Jones Tested-by: Lad Prabhakar # --- arch/riscv/mm/tlbflush.c | 39 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index fa03289853d8..5bda6d4fed90 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include @@ -147,7 +148,43 @@ void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr) void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end) { - __flush_tlb_range(vma->vm_mm, start, end - start, PAGE_SIZE); + unsigned long stride_size; + + stride_size = is_vm_hugetlb_page(vma) ? + huge_page_size(hstate_vma(vma)) : + PAGE_SIZE; + +#ifdef CONFIG_RISCV_ISA_SVNAPOT + /* + * As stated in the privileged specification, every PTE in a NAPOT + * region must be invalidated, so reset the stride in that case. + */ + if (has_svnapot()) { + unsigned long order, napot_size; + + for_each_napot_order(order) { + napot_size = napot_cont_size(order); + + if (stride_size != napot_size) + continue; + + if (napot_size >= PGDIR_SIZE) + stride_size = PGDIR_SIZE; + else if (napot_size >= P4D_SIZE) + stride_size = P4D_SIZE; + else if (napot_size >= PUD_SIZE) + stride_size = PUD_SIZE; + else if (napot_size >= PMD_SIZE) + stride_size = PMD_SIZE; + else + stride_size = PAGE_SIZE; + + break; + } + } +#endif + + __flush_tlb_range(vma->vm_mm, start, end - start, stride_size); } #ifdef CONFIG_TRANSPARENT_HUGEPAGE void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start,