From patchwork Mon Sep 11 13:12:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13379315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 383B3EE7FF4 for ; Mon, 11 Sep 2023 13:12:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 97D2C6B0291; Mon, 11 Sep 2023 09:12:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 92D0F6B0292; Mon, 11 Sep 2023 09:12:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7CE5B6B0293; Mon, 11 Sep 2023 09:12:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 653996B0291 for ; Mon, 11 Sep 2023 09:12:31 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 352FD80A45 for ; Mon, 11 Sep 2023 13:12:31 +0000 (UTC) X-FDA: 81224355702.24.83955D6 Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) by imf13.hostedemail.com (Postfix) with ESMTP id 4A29B20026 for ; Mon, 11 Sep 2023 13:12:29 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=jvyK8tNs; spf=pass (imf13.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.51 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694437949; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=YAch9/9+1yvRN3ndGGQdZRoEnDN1q2q/b1WTFSVL2jQ=; b=Jhcq9V3w+4fX0E70jfOgGWjPEQkcsWFeLCPP09jYWXdjybgxDZEGgSqOzUtze1JFnAUQjS ViP0Gb6ZjmTQiKBrhcRPIJMupAR5GSIWrsaTQp5JlPpwECBL/IoncHk0uaun5FnQb+V70k roLpIny9NCd43GbYV5n4DSJpnC8oDVk= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=jvyK8tNs; spf=pass (imf13.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.51 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694437949; a=rsa-sha256; cv=none; b=kHJIOcP7JvvLsK+vVgRxewX3FcUrSaR86lPsleHF3/zId46Hg3QvaKLMb970x1Qh2ro8kH TRQ+a+LClLpY7y2K/IEYDPd8uDKFhD8SO3sHaJAhFBwNFV4mjNsCLufj7ttjxm+Cyxq1vy yZHEoKexmMEUaPEwOAYAKS/QHX+jsB8= Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-31f7400cb74so3587665f8f.2 for ; Mon, 11 Sep 2023 06:12:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1694437947; x=1695042747; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=YAch9/9+1yvRN3ndGGQdZRoEnDN1q2q/b1WTFSVL2jQ=; b=jvyK8tNsfe6tp6uT9qMkTJWEhAiOtztwVMKwZQ4uJ8RIpc5OSgO1kS98S5gloTaQc3 qS0rJ67yaNJJYArAAWsuxA36iVrdHjwVmook75/4MjAj9MUCLeMtuoE+sv8HxXO1gz8T vMt4TbkM0ZMNyM6ABnurN3qjNzpTBkFgX7ghHUrjI+N0dNn9OwriURYiXjnzYpAZCagO cBXosAUNjwirwDQO/i8xEHXvV3pKSfLs2nPQoRVQH2ARX290h3OkR/+cqrSoHDIEQPGi w6mdiWM2vZsWsTC4GJvSK4+Ql+J5cUz0Cn+UnA7X5CTjjTFmIS5EpOoJiQ8rD3mFnTS/ LHZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694437947; x=1695042747; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YAch9/9+1yvRN3ndGGQdZRoEnDN1q2q/b1WTFSVL2jQ=; b=UZyK2gysVX8zS0zTxQH6cEYV9gRHebaQrNlDynnHna7AVT9Lsk3Y3Xu2ZP4d4DPx+K kEY+b06GXu43Cca+I+v596MOQjwxX5NBF53YpfsdEc5CozK1dge+N5h1X1rcjPuNATtj tMPAHStQjaBfsz9wGsY3IG41FpcdmpV8FYuPkcmaQorlHpvybAvt0FFOTzwtYemTZO+q EGBxljwo/oz3vX1m886YEFpmcqjOpelTUeSfcQ8NxQjKafmyb0r/RTuzvkTjHo76Mp5Y d1aKDyVMnqaskIsPNq/FIK1GGxFDqye0pcTGqmlqCoFOpmySPwiJDyk5P4rIhDulzLTI rR3A== X-Gm-Message-State: AOJu0Yzen9H2l4oIie+m4lh6AftWTBTq1PH7bjOquKa0aPSfihpDRJ0Z DVb/8pzlAA24fEC2Vw5I+W6Bkw== X-Google-Smtp-Source: AGHT+IEo8vfSE/53LIIg4K2AfasH/DaM1jP5FLSKYkUJRorFSpro0cfH84Iowld11wYazbHz32ri/Q== X-Received: by 2002:a05:6000:1101:b0:317:650e:9030 with SMTP id z1-20020a056000110100b00317650e9030mr7641721wrw.57.1694437946629; Mon, 11 Sep 2023 06:12:26 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id r17-20020adfce91000000b003198a9d758dsm10104895wrn.78.2023.09.11.06.12.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Sep 2023 06:12:26 -0700 (PDT) From: Alexandre Ghiti To: Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra , Mayuresh Chitale , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Samuel Holland , Lad Prabhakar Cc: Alexandre Ghiti Subject: [PATCH v4 0/4] riscv: tlb flush improvements Date: Mon, 11 Sep 2023 15:12:20 +0200 Message-Id: <20230911131224.61924-1-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 X-Rspamd-Queue-Id: 4A29B20026 X-Rspam-User: X-Stat-Signature: yf7hwwgx9c941tqmm797tffxag83jxep X-Rspamd-Server: rspam01 X-HE-Tag: 1694437949-10973 X-HE-Meta: U2FsdGVkX18KzdMQNz6DUVHv/lVSEDCsjy/BHjWfwpdTFI27fLO6db7Nvog9krYff4SY/nq7C662qzwqG4PzwyJ1BMV35g1xLTarw/sBhI1CwrJECxOAUWTxtl35WGwYnjQ3eazB9AhlkoFXkM3HP6Ztp74Fe2G0N0cn2zOjZV7Bfw5GImkFPs36K0rIeeGVE+VG94/stL5R4VfdFhO89rClSaV155cQzQ5St9sKtiH4pcca+eoq1/VlPNNekNYcGjKuLYOC8imgCMXAyn44up4eUyWiYinhg491GRT6Jgms8IPrzv8OBBkb27g2yBkGnJPanP9pLS5hfBcRf2U2etkthqZ65zfRYsinhCZNL2lfe1gDQdMx3l0lt0EfN1HDki0sUChEhLeDC2LygeyfFiBVC37smqK2jTcbLxhDJloNshBRr5Grm0dlaFu1aTqAt00UR0s+xAqBfcJc6TQjY/Qkek3GMZLSFtk+bNzrzEsAl33KyGCaNIERq80txRHvsLx+xVLDiz5SVHAH7JKuu/OcL259SwtZamBhc70tcknImv+f8rSYiEoEzjns1SuiKCAiN3uV3+x2usNHYhMZS4drHB2TP4pfvzosO4FP9BQDtBUswGFuzix7IlMwjQsRHcRYDbKveqU/ImdL9/0BnK5Uf7LpLaGEVbit1d7K6Hc7seMftKdoBB/iT+sPj+Z3c+0uf/Ed2rd9dbdep/bT74gC/adIb7Oo4WNkyDk1iVi+zxLG2nZjp1Ipn4aZhH/55I9sNjUcqqECrlFcSZstX5LQBS/UGEcPV924ONu1D2rM7NTNsD0wMn+fDvy/L6iQxUmHDpsuQpUga9tpvtBLI6N+iQUMT4FCWJkhFPx/w2yN5/muqTP193at3Eo3CiY5EJ9/7SFDoGRsEL15jrQe7dVVcjUp3hS0qdlzbxBKVeayLu2u97ct83g0gM7TCihoFEtJDcKX7BnNzj5GrZ9 QRxamcV3 KOaUOJa+J5Vc7dvijJNFRZKABBAFZpv3Knuab2OYkoF7LhYCUtXV9PMA6zbxt0fG9AfSyq+V7THB3Y5milvXRkNAfRPmtOI87qcvnGETzlghHPO04eiUevFjZ4gWbB77QxV2IvkRqAd/diAdERD8aR3+FNPjZHuQc22R9NTjeCfC1rn1q9jIqBmQThVq5ClXjFcTAKDCqWUV7yNM6lp+z1icVlX46w/TzXO9QPqGOEy50xYIrwOWK0R33NfUYiqMB0WQ2XC6x0/CESyV4F5UnQek2N0m+fdUjcta/u2qt0qwabiMGFsieD9ptHBI8k3UZQBD/F/bppKNiWyEwQt2I0m12sEiZAjpRrYOHqmW1Q8LBzqGAnubPpYVz+vX/PpItVTQ+tV3wDFoZxm7A8QrqADRJB9D3VYpgxZ1oYgMnUc9zcGYvb2YUEbHdQwekb1Qa8Kd1SbfUPsR043YlSLfVzSYe8DfITVPGCgNek0b6aqmlASncv/YyUahWKCuDTQsduQ4XUaMTP2zXQGqKwZohTi2SxwxCjxb+IWz6J8z/HYJMzF8U5LWD+mfx5vhDMOU5Pu/3g9XvG+7/FpI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This series optimizes the tlb flushes on riscv which used to simply flush the whole tlb whatever the size of the range to flush or the size of the stride. Patch 3 introduces a threshold that is microarchitecture specific and will very likely be modified by vendors, not sure though which mechanism we'll use to do that (dt? alternatives? vendor initialization code?). Next steps would be to implement: - svinval extension as Mayuresh did here [1] - BATCHED_UNMAP_TLB_FLUSH (I'll wait for arm64 patchset to land) - MMU_GATHER_RCU_TABLE_FREE - MMU_GATHER_MERGE_VMAS Any other idea welcome. [1] https://lore.kernel.org/linux-riscv/20230623123849.1425805-1-mchitale@ventanamicro.com/ Changes in v4: - Correctly handle the stride size for a NAPOT hugepage, thanks to Aaron Durbin! - Fix flush_tlb_kernel_range() which passed a wrong argument to __flush_tlb_range() - Factorize code to handle asid/no asid flushes - Fix kernel flush bug where I used to pass 0 instead of x0, big thanks to Samuel for finding that! Changes in v3: - Add RB from Andrew, thanks! - Unwrap a few lines, as suggested by Andrew - Introduce defines for -1 constants used in tlbflush.c, as suggested by Andrew and Conor - Use huge_page_size() directly instead of using the shift, as suggested by Andrew - Remove misleading comments as suggested by Conor Changes in v2: - Make static tlb_flush_all_threshold, we'll figure out later how to override this value on a vendor basis, as suggested by Conor and Palmer - Fix nommu build, as reported by Conor Alexandre Ghiti (4): riscv: Improve flush_tlb() riscv: Improve flush_tlb_range() for hugetlb pages riscv: Make __flush_tlb_range() loop over pte instead of flushing the whole tlb riscv: Improve flush_tlb_kernel_range() arch/riscv/include/asm/sbi.h | 3 - arch/riscv/include/asm/tlb.h | 8 +- arch/riscv/include/asm/tlbflush.h | 15 ++- arch/riscv/kernel/sbi.c | 32 ++--- arch/riscv/mm/tlbflush.c | 192 ++++++++++++++++++++---------- 5 files changed, 155 insertions(+), 95 deletions(-)