From patchwork Mon Oct 30 13:30:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13440586 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3D00C4332F for ; Mon, 30 Oct 2023 13:32:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 604266B0183; Mon, 30 Oct 2023 09:32:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 58DEC6B01A2; Mon, 30 Oct 2023 09:32:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 42F486B01A4; Mon, 30 Oct 2023 09:32:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 2E9E26B0183 for ; Mon, 30 Oct 2023 09:32:08 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id F26071408CD for ; Mon, 30 Oct 2023 13:32:07 +0000 (UTC) X-FDA: 81402216294.06.C38B240 Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) by imf21.hostedemail.com (Postfix) with ESMTP id 061111C000C for ; Mon, 30 Oct 2023 13:32:04 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=MPEK4tDy; dmarc=none; spf=pass (imf21.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.52 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698672725; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hd/FLevx18XhR5UIn9smwlv93JRL3lzcOo0hNfovrwU=; b=hvdQSVyqGbfuN4DmlZ91tRNb1hx72Fm/ovDl8T+QEWlk5DjibEunF3JKNLeuQPhTnAacKM AF3FM1/0Ec76RHT2V8P51jasVtX0XqCosxfKicrOxr9C3uG41xvm6/nj0c+9Kvlhvex5e2 RYg+SzaWF+BkSrMQwOXLwonsxI/Ph6c= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=MPEK4tDy; dmarc=none; spf=pass (imf21.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.52 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698672725; a=rsa-sha256; cv=none; b=LqoSIP2xIpIycuh+8ITFUY4hKuUgLMx2qx5uA9Ujj9AZ5GzuzhPHG8nZMbuyLY2tXNBZST tP5s+FoJDyzvI+7Q0muCGS3o9NBYSOlj4zhS0tgAP2uBjR/luIRHkEWikAft4Hw+gp18uH 0NDH8xkPpIyJPhj7/GvBLlea3P5PdKc= Received: by mail-wr1-f52.google.com with SMTP id ffacd0b85a97d-32f7c80ab33so1215143f8f.0 for ; Mon, 30 Oct 2023 06:32:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1698672723; x=1699277523; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hd/FLevx18XhR5UIn9smwlv93JRL3lzcOo0hNfovrwU=; b=MPEK4tDyPZrJQxYlihIj0q2/PHgXKr8FgiPU9dn6eMMQ5VXGMTleJKHqBhgZ8LouBC BIj7C2zV9QAIwz38LThlyeeMbiCCsrB7k7cBeeM3uRNYd34Ozu3sjgYxf+neENzkw/jW FWFUHqMPM3O8SU12ugPxvsGQF5c4GK9GLIHtLTjZUHY+yx3I6SzPC3EVu6rcivNhbiKD ANCCjjW/OWkATMBwrbVyBOGAUJX0qw39SaRQw2rWNVaX4OMVuJrM4Xo0EQdFA9id0J4x F78LI8lROYfiSpXpfckTnUMBwyuReTGUoX/Zrnm0X+ii4O8iODvXH7q0qt/w25SPdt9d YHEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698672723; x=1699277523; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hd/FLevx18XhR5UIn9smwlv93JRL3lzcOo0hNfovrwU=; b=QDPRsp/fxrbobgSVZsnxGzZgFf7q62PBQ65NYIeFEZwHNFYkL3v/MqZlBe7XyYDhuY MKdKgLKQuZE2wYXOw3EVtkXEjy8yh/m9tTNk8Yt2xN287LzKMoYmrGAKUsss0vVUyK9b sSFhT20OTDbRBkYv+VgutUiGTIEW7uXMjXyeDP0JRDGrGtAAiowgLOmKMrmv9tFTbcIX hJQ/gcStQlSPB9jTwqHbRSmjSBZ3oaRSeQKqhYKOS1rL5BaVcMq6MIxf/iosv9WeSQsR ErIU8PzjTf/MzjtCDPjfu+gCReVFEGtUy9fnNpjsWvM1xa1rtWWRxMMEfbgHBPwfnjk+ SNHw== X-Gm-Message-State: AOJu0Yw3WHjmPmti4TlaTk5bzKoQ7TpcC7VHfSuVQsn8vO6vH9bHk0H5 8jI93lwpOxy0hsQ6RsCpocLaZw== X-Google-Smtp-Source: AGHT+IGz8B0HEGjTPMhdhfOE7+2f/YykM3hv3LZ+YFlrU3T3FZRfdHfdZDwT6aaKJLc5fO13EcZA5Q== X-Received: by 2002:a5d:4b0c:0:b0:32c:f8cb:f908 with SMTP id v12-20020a5d4b0c000000b0032cf8cbf908mr8512441wrq.59.1698672723353; Mon, 30 Oct 2023 06:32:03 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id a15-20020adfed0f000000b0032d88e370basm8262398wro.34.2023.10.30.06.32.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Oct 2023 06:32:02 -0700 (PDT) From: Alexandre Ghiti To: Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra , Mayuresh Chitale , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Samuel Holland , Lad Prabhakar Cc: Alexandre Ghiti , Andrew Jones , Samuel Holland , Lad Prabhakar Subject: [PATCH v6 1/4] riscv: Improve tlb_flush() Date: Mon, 30 Oct 2023 14:30:25 +0100 Message-Id: <20231030133027.19542-2-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231030133027.19542-1-alexghiti@rivosinc.com> References: <20231030133027.19542-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 061111C000C X-Stat-Signature: hjtordhychzdk7a3aqyk7dka49gst6gn X-Rspam-User: X-HE-Tag: 1698672724-372919 X-HE-Meta: U2FsdGVkX1/d3uyINCu4PZFP5qPGtKOImf1QRjOtvPgS60mZeo+4/jilS20XJVeSC5f5mkp6pnmCDjC6ifQsE0pbs1IjOpJMIPZCxwvZEO+6+vizktbX96/LMhgSohQ686RKNeBDRjKVFgQKQoCsB5eRuS94wDBMsaoLf7y8gvBN1f/8cFCv6Q1gK5Fol2kIvRMhhQZ6irsDfQ9C31VbmviweNsSwE923pRwpGNnXW7cZxB6Ml/BuqlNcE8b5EJoKLR9+HzIvNsXjY/oUKZTjxmXjb2gQpx6YSRBlF0wvhEJtznex0Td2LyOR74QBneQQzw8j/WxllQrgD7DQWOfyX7ut2MNvey9DvqiJ6rRlAZit+J+RabTfr4xcyhAZ9v847M4CdxBanSz/pqXgtk40YAIW82ddQUoA804kMNQcm0Q/I7o9g2xF6od764LJy5pMl7lfrCcZCk747Bu6YjZKzb7OMRpDMYGihk2pRrVGhZr+C+TN2qLMVHFGs2/FZwTOwNpMH2HsKWxTnJKuy41ZlAzkbkaaahCGz8uQ+QnUhRRiIMZ27EOKp48WLPlxgQTtZ2IllbsixoGcJNfZviKvvpL/nTquTlFyD35UqShkzI0oTBqwHtZkqDqqIBODhKR46VzZDx47hIcc2y7QQOV68hXDnwvC0ZwDBPPvlDULtX52lK1HP93S/V81unVIuNFwetheb72uTGcGeq0/gdeTtxH+EwtOkxzFnIOPUuHtPtfrlOS3/W+EXlihzuGRYdryv22L9JQqOQQ2BN6lG+Pzx4rLBiQqXLPhwUuVswzkRCThtGVD8VqUsuOKHtTt6gw1EWjcuwSynLj9VwcY/0N3felnSPbFG/cq0qFikd3kfx5q7yGCLoztS9IU/gdqVn6TDNzlQGLWAfr8wp9CmKizGALd5SyrdQoBcoHUFWue398vs055E2vUopAX/7NLzyVGYuCMyY6l1EHFbA6hsG VIRIl9/T 5RyP0nGG71wXd6KfGujmynZpuowY/5Jl1gXOFMTHSvjWviBUF/dDZrsra0G1cVMWy27UQIKoXtiuEvoW5/DLblhGCQLAQAB7+AWsMIC0rUoLaIH9GUNCyBsBbef2vASfflFpMguftwzc7seKDQcX7xfeXiKMC6mZlmnNiCrJpXiRTXJxhYr49SyIncamQ++60CngN+wlYB/zwMWtIpdZc6MlmYafn19ZbLbA1jA6ibuivSt5t1ZQ+RvMriDtF38ME3iEasx+wyVrINXc9LfGeNqNJd4J1KobkFswRtpgOoSJ1jYnMgbN9SlWEGocOEMzrBqixx6Zf7s0AYffeJsH8EwRterrnYWbjL8AOJ6VcOSBpjO27o9F8BAEgDXMxeiRe58WDVbW1e/Ri8hqH4wZbIGGuiV6qS5Rule4+cnwNSkjax+vMDlj3GqUo6iB9Fv2arBDXhi0neeIPn+1jJT6XFyEHm+03nxD05GriIfP/CAv1AmpXBD0NDfLjoUS6Eut9lkyIG/JZHgQ6mMb8YUtF2VE9pfNa8tsh/5wZ4rfPH8+THtrwZ9OIUidIGEY4NPZ5UwDlZsZ6LrQhSo31STXSegeyrsq5JuHtNAfGfXjWW+Jz4EPvd2WJdI8w+60XDMAamulHVdBGit4KlUqgdZCOi3gyHYgqyDfOGVtVLMHvRAdJrtNJ3et751V/ptSYT5QsALFrQCzqnDocpVs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For now, tlb_flush() simply calls flush_tlb_mm() which results in a flush of the whole TLB. So let's use mmu_gather fields to provide a more fine-grained flush of the TLB. Signed-off-by: Alexandre Ghiti Reviewed-by: Andrew Jones Reviewed-by: Samuel Holland Tested-by: Lad Prabhakar # On RZ/Five SMARC --- arch/riscv/include/asm/tlb.h | 8 +++++++- arch/riscv/include/asm/tlbflush.h | 3 +++ arch/riscv/mm/tlbflush.c | 7 +++++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/arch/riscv/include/asm/tlb.h b/arch/riscv/include/asm/tlb.h index 120bcf2ed8a8..1eb5682b2af6 100644 --- a/arch/riscv/include/asm/tlb.h +++ b/arch/riscv/include/asm/tlb.h @@ -15,7 +15,13 @@ static void tlb_flush(struct mmu_gather *tlb); static inline void tlb_flush(struct mmu_gather *tlb) { - flush_tlb_mm(tlb->mm); +#ifdef CONFIG_MMU + if (tlb->fullmm || tlb->need_flush_all) + flush_tlb_mm(tlb->mm); + else + flush_tlb_mm_range(tlb->mm, tlb->start, tlb->end, + tlb_get_unmap_size(tlb)); +#endif } #endif /* _ASM_RISCV_TLB_H */ diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index a09196f8de68..f5c4fb0ae642 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -32,6 +32,8 @@ static inline void local_flush_tlb_page(unsigned long addr) #if defined(CONFIG_SMP) && defined(CONFIG_MMU) void flush_tlb_all(void); void flush_tlb_mm(struct mm_struct *mm); +void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, + unsigned long end, unsigned int page_size); void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr); void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); @@ -52,6 +54,7 @@ static inline void flush_tlb_range(struct vm_area_struct *vma, } #define flush_tlb_mm(mm) flush_tlb_all() +#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() #endif /* !CONFIG_SMP || !CONFIG_MMU */ /* Flush a range of kernel pages */ diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 77be59aadc73..fa03289853d8 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -132,6 +132,13 @@ void flush_tlb_mm(struct mm_struct *mm) __flush_tlb_range(mm, 0, -1, PAGE_SIZE); } +void flush_tlb_mm_range(struct mm_struct *mm, + unsigned long start, unsigned long end, + unsigned int page_size) +{ + __flush_tlb_range(mm, start, end - start, page_size); +} + void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr) { __flush_tlb_range(vma->vm_mm, addr, PAGE_SIZE, PAGE_SIZE);