From patchwork Thu Oct 19 14:01:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13429254 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13864CDB482 for ; Thu, 19 Oct 2023 14:02:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6B1756B00D7; Thu, 19 Oct 2023 10:02:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 65FD76B00D9; Thu, 19 Oct 2023 10:02:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 500846B00DE; Thu, 19 Oct 2023 10:02:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 38DD56B00D7 for ; Thu, 19 Oct 2023 10:02:22 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id CE38A1405A1 for ; Thu, 19 Oct 2023 14:02:21 +0000 (UTC) X-FDA: 81362375682.20.9B5E288 Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) by imf11.hostedemail.com (Postfix) with ESMTP id 82C0E40077 for ; Thu, 19 Oct 2023 14:02:03 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=LyJ6vxwz; dmarc=none; spf=pass (imf11.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.51 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697724123; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=dYapP8IUUxaJFtFuPCmZusI+AtIHAeMa/0OE5NMEm0Y=; b=lA7vw11C2geijjaAE0bwF4oH3CgMN4J6uo0LDUUL4oo4SGlPYi+DkNMlev/7+2c1wulA3o oj+GHRQeU5vuM2negR2vcFbShaDjEPyVK1f8JW42crkOEpkCwI6CEOEc8kR6SAg5AH+rCj Nldv0cSOEKl0unZxg71QDmGyUGB72+k= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=LyJ6vxwz; dmarc=none; spf=pass (imf11.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.51 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697724123; a=rsa-sha256; cv=none; b=eDLbeU/K1hAovRpslY16b7P88YleGSUGGE1AYOFZ13AI8VH9+wl6gUb+QVaEjn9jeFflYp /QdSHZTO/Xgm6smOD0wEmXFa1rSUPNubMrNGRJYwCSdQ2hyUDHCDzsf9p7VdwX7WZDSf77 bOt+jmHBOmS0B9RpbDzZeaKOi5cjTQM= Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-32dd70c5401so725090f8f.0 for ; Thu, 19 Oct 2023 07:02:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1697724122; x=1698328922; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=dYapP8IUUxaJFtFuPCmZusI+AtIHAeMa/0OE5NMEm0Y=; b=LyJ6vxwzRsiJ2qwvzdZDQu0WKTo7CwE+R2JjUfDzi9oUF2eliJMVbyR5ny+8dan7Ez S/GP8g72NhcD9LYOM9AHnqEeAxZ/yEk/Ziw5Qf4FLEmkEM/UB8Dl8Ji6X9YwhxZ9+1TI vLUQPJoe74zoPPjWMwwrwocwD6eFld2Z5XpRODl7zkHqPGI2tyLAGtHqXdBRwnjyx3ic Qjf9Z2rnusa83Y8qHPzo604ZAMqheAvjFOUdNMNIGD0VXoSd8J+FR9KmdQuwUzSE9611 nSH5RaNsYPkr5K9iya56UQAdoji6HdC/ObhJqFWE+lsWFxklN0c/lyX3ajt8uhRvfgyu 15pQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697724122; x=1698328922; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=dYapP8IUUxaJFtFuPCmZusI+AtIHAeMa/0OE5NMEm0Y=; b=QO0zHXOvmZRSy3dCNsRmA2dNlu0MetOu8CrPnNS9YI4dmFui4dUfHjIQx6pbdKRmVi C1tPvXCdFYRN4fFk5hXXKAtdHys0aNcjgASP9Wb+p4ATcJRijKeRQiUGZRfAZNUd+mlN 6QOG9O/On5qsFMC5C0wMVaoFImjEnZzUVpuMuRidoxnG+l07xxcl2zk8MNie0st2cjSe EDqKGoAmskKekB2Z59vxnqmqBESI69tWa5j+gGFjUSJOZ3u9PQ+EoK0cm4KEiD9/kYbQ W56N60e0KYTAkyh0eq3aVOYnjOOyG6EC52F8tt7c0UtAd7ny97ty5g05tcJLj5zeS5bH nlfQ== X-Gm-Message-State: AOJu0YxU2IFSOv6KJBXBqXVbaOhUUu3l0mDRGzyqlNvE9AYp16NkpW3I q4A1PSThdwj4b4b4JcRIzBXTFA== X-Google-Smtp-Source: AGHT+IHxfbIAtPpTWfqvyIaAWohPN03476Ru2wycW4vXZu6twctXJ/42I8ftrwVedoleNr+FF1TXMA== X-Received: by 2002:a5d:4561:0:b0:32d:a827:d0fb with SMTP id a1-20020a5d4561000000b0032da827d0fbmr1817267wrc.27.1697724121113; Thu, 19 Oct 2023 07:02:01 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id b16-20020a056000055000b00326f5d0ce0asm4598711wrf.21.2023.10.19.07.01.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Oct 2023 07:02:00 -0700 (PDT) From: Alexandre Ghiti To: Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra , Mayuresh Chitale , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Samuel Holland , Lad Prabhakar Cc: Alexandre Ghiti Subject: [PATCH v5 0/4] riscv: tlb flush improvements Date: Thu, 19 Oct 2023 16:01:47 +0200 Message-Id: <20231019140151.21629-1-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 82C0E40077 X-Stat-Signature: ufr8o9jsubnw6ddj6k93f4k4adzzpk57 X-HE-Tag: 1697724123-78496 X-HE-Meta: U2FsdGVkX1/vG3WNc4PzRd0sg4/A6HZR44wLxu5Aolakkq2m1e5RdTVZym+g1+f9ba+P7FlI3A2v/v7ODwctsFuaLrDAVkyIAPagoXUpUEw7LofVWdVZBhMdHU5ghH2rmbFPGGiAU7D/Kl4qSQvXkF9DudKwHSIZ/WeNSrACijfwRqg0b4t/5k1y4xT7FHz/u/7reV7HBmk2EKsMcN9HaBf5h0PjgtGFjzA4KnSdEPnb5C393vAlK43OUyiDQBIZBJlVSc/K6PyFURV9GkETV8IuofvmlkofVh4BubvCiOIksoWPfYGP1vAizJatdRFj/i6wRXLi9AJ5XIDFekzHMV9TvFwDFp+X2pxF5gUOWpWlaPb59ptt2kL7Z/GOmbPtz/MXqBr9xahwuN6Ssg/Ag0iPpMtgyt18qPXMg4pDV1uHBGdSrZRVvi3wwDX9s1VqAsb5koyua9L6GLQzkvPL+qeqpSAyZk2D1KsraCaB7UrHJVJi4QWBhcPyDYbNAtED2Xxs22FS9MWvol4q0xhEDRi/+487cPnSEBK1jWnQuzgJkOGdu56XD4LKB71ZovMBIdeSkNeCtdVjXK3kFyiT1c1UdeettNksfPcUTq5prYxhPFW8erq3PuH2wO5I5Ffv27HdyudNJLlF+RDPl8fvjK83owipNF8wJW+ijJ5RXvg3+YiZxkVYd4Uvg/ZUQY8qAivL5vKYaVkNopBVU261BLk8s4wfwyuafc66aTtHkRXD2Nj5ovxtrDs397awCc6EsyjxH60lKdah5LfbgWx65PrFr1PA1R1H5eZb6kf1R6BdTd5iFKsQqC9W6F8AVQMSpyk9rLoGn65RC8sJYk4RqB4n1qD6ftQA1NqgPLIUaDjK92sZoEIV2Sto/FNMzYibdOmgkB17rm48RK9tW3c9svh/E/AmP/wyHCSf9Iwh6WtXMp/McP1XQLXipXrBegMe303VUClEwGn2z5cC3g3 L6lImcja JtEZiO4sPN1j9iQQiWw315aeXc/4thYpLzcU12nFaVVePa3YO+GENzTlApyfSnKlvNlpCI2mwtmRCti67uLRSryRNgZk6pfOGisKnUrasINasP8B665Z8Z923g0SJYILq+knvo/Z65bYm9XZi1P1rhiyeTqpvaDer0tC8qBU+Qp61QUknTsnaaA4ZEps61Ix0Ec3sRGM+n9vMfEZpDmnAEGBtNut/lKfhG8QAAc6iYxQPFNnf5ljD+6iCsFjaj4C5vKqfVqToZAv0P3mK4XMgPw4t/CrygTYfBm5DkyXISjcKypUmKSJiOOM7b3RBsZr+8dW9mlaxm0hF7wyk1mB2zWsLV3gSqvGhaImIlRDzTyiI4waNvattoQURmGyC95kQ4IXP2nsHebwCAAJPUoukHqVAzV9oblKwLp9RUkuo1HO+8aEHnkLs8icbXqIA6Em8zECA9Igj/vftwQSau0rg90EcQiBpUif+sojyHyE0MyTHcOecPVNpC0Z4BhV3OzRSKnD2iiPva5VG9HKxNxQRwD9g3wq4Nm/p8kwNS9zugGItl53ccahzn+PXARACapmZrtzV53KOSXCL17E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This series optimizes the tlb flushes on riscv which used to simply flush the whole tlb whatever the size of the range to flush or the size of the stride. Patch 3 introduces a threshold that is microarchitecture specific and will very likely be modified by vendors, not sure though which mechanism we'll use to do that (dt? alternatives? vendor initialization code?). Next steps would be to implement: - svinval extension as Mayuresh did here [1] - BATCHED_UNMAP_TLB_FLUSH (I'll wait for arm64 patchset to land) - MMU_GATHER_RCU_TABLE_FREE - MMU_GATHER_MERGE_VMAS Any other idea welcome. [1] https://lore.kernel.org/linux-riscv/20230623123849.1425805-1-mchitale@ventanamicro.com/ Changes in v5: - Fix commit message s/flush_tlb/tlb_flush thanks to Samuel - Simplify NAPOT mapping stride size handling, as suggested by Samuel - Add TB from Prabhakar - Add RB from Samuel - Remove TB/RB from patch 2 as it changed enough Changes in v4: - Correctly handle the stride size for a NAPOT hugepage, thanks to Aaron Durbin! - Fix flush_tlb_kernel_range() which passed a wrong argument to __flush_tlb_range() - Factorize code to handle asid/no asid flushes - Fix kernel flush bug where I used to pass 0 instead of x0, big thanks to Samuel for finding that! Changes in v3: - Add RB from Andrew, thanks! - Unwrap a few lines, as suggested by Andrew - Introduce defines for -1 constants used in tlbflush.c, as suggested by Andrew and Conor - Use huge_page_size() directly instead of using the shift, as suggested by Andrew - Remove misleading comments as suggested by Conor Changes in v2: - Make static tlb_flush_all_threshold, we'll figure out later how to override this value on a vendor basis, as suggested by Conor and Palmer - Fix nommu build, as reported by Conor Alexandre Ghiti (4): riscv: Improve tlb_flush() riscv: Improve flush_tlb_range() for hugetlb pages riscv: Make __flush_tlb_range() loop over pte instead of flushing the whole tlb riscv: Improve flush_tlb_kernel_range() arch/riscv/include/asm/sbi.h | 3 - arch/riscv/include/asm/tlb.h | 8 +- arch/riscv/include/asm/tlbflush.h | 15 ++- arch/riscv/kernel/sbi.c | 32 ++---- arch/riscv/mm/tlbflush.c | 184 +++++++++++++++++++----------- 5 files changed, 147 insertions(+), 95 deletions(-)