From patchwork Fri Nov 10 14:07:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13452443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DB8AC4332F for ; Fri, 10 Nov 2023 14:08:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 735F0280026; Fri, 10 Nov 2023 09:08:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 69611280022; Fri, 10 Nov 2023 09:08:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 55F69280026; Fri, 10 Nov 2023 09:08:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 45A6D280022 for ; Fri, 10 Nov 2023 09:08:29 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 98669C0ED3 for ; Fri, 10 Nov 2023 14:08:28 +0000 (UTC) X-FDA: 81442224696.18.E4982F9 Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com [209.85.167.43]) by imf10.hostedemail.com (Postfix) with ESMTP id 88551C000D for ; Fri, 10 Nov 2023 14:08:26 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=g+IxDWSj; dmarc=none; spf=pass (imf10.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.167.43 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699625306; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2AMKJOgOBU+36iIxGMD9Trqw8IVdjZaTyhAA3F26Egw=; b=oBo3Iq7Rw3dC7j71Ou2b9qV5k2WhbIOOeCwqM4BaPN+KCi3K3V+/TeXlUddm0DPumhLZ+h i9q4jsYrYDiqPHIB0Ul7eEJg4uQHQBO9dxMh2aONrsmtTsR6RFH1MJYTqoGNMZ2YMoOxE0 GyYFNFZYje9Nqt0Ehwg5csO4ZxV+Gjo= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=g+IxDWSj; dmarc=none; spf=pass (imf10.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.167.43 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699625306; a=rsa-sha256; cv=none; b=JVYedP3Sp4z61h/1Oz1J2xktFzyN5eHYvDhgaIVHNTwWFRbNLuSHMyvA0qpZylP6B6Clxb V8e4bsMGxeq0T0fjzrebS5P8Soi8iSYSiwoJzRss/bLfF/OvkG6gG/Bu6U0cOQlBz7HhRy uXDQDEA13XgH/6mYXZk/YvqblnBwL2M= Received: by mail-lf1-f43.google.com with SMTP id 2adb3069b0e04-50a71aac023so653407e87.3 for ; Fri, 10 Nov 2023 06:08:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1699625304; x=1700230104; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2AMKJOgOBU+36iIxGMD9Trqw8IVdjZaTyhAA3F26Egw=; b=g+IxDWSjwiapiG9dWmnScQobJMlgkPv1sK3iljbBpXqkB6GSlu6roQtkVTDW7wv3zN p2EpjdToLeolR03wZ4pHxrQ/Be8VjjdWs/1hoc0H9hJCqhH2guBp/JxChotTWCu+C4rg VIkorkGBrahJ0mFaOecyrgMjg0ABwYj6kT6dfxn0i14O1j0beeAPOGWYPtNg6DPrrFC+ SAai8YPI0eOBjiCY5amKLM3Oj/XI0YIBLL+NNjup97Cu9jLEIFivFBF2dBMG1DNOnaUS azpAx40xDrnUcJJ5JFW6qOyLThiSNK8ErWhaj5pbj5qSXag4eNqzBSxuidhdJ/UXe7Lv 0Nyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699625304; x=1700230104; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2AMKJOgOBU+36iIxGMD9Trqw8IVdjZaTyhAA3F26Egw=; b=SYF/p0l6XjeLjp4770wwPJPFERdocicvKfFScUy8Kpl2YOv6dHj4EOJG66hQfZhtc2 oX4i+mTbT3UJd8Gbk+aMTF2K8PfA6MyinBl1BbRIhCa6xSZajMS1YiwNXkhS7Q+zdFu3 pJiiLormnxefXTLC9XoxPAn4lYh3agp/Gj4b1suVXte/QLuVyYeevDfkfOvK74zLHQYR Rm7J9S0lFppdKN6JHAuHB625sHsXY6xK5TzPUHUlscwNE1IW20sbVZoPq/gqyWXO88YQ F7/CxHGYa9KtSMud2K0BaxhNZMBFw7MgBEjggi6xhfnXgplFsaA7CJ8aarfAp+C0tWGm 9DDw== X-Gm-Message-State: AOJu0Yz/cRfXLej/zkMrqnpAJlmaGSsyRSQNG+O6EYkSr05ZzNexVuq2 zCdQ1m/Mkw6Vsk4Caz1qp5RlwA== X-Google-Smtp-Source: AGHT+IEGn96XLIR0ZFAsjYB2eJfL/m69l1crDYV4Ee9Qc9cnFj8KCnXUpx20enm/StvHsssXdR1xaw== X-Received: by 2002:ac2:5209:0:b0:503:1783:d5a9 with SMTP id a9-20020ac25209000000b005031783d5a9mr3895555lfl.3.1699625304483; Fri, 10 Nov 2023 06:08:24 -0800 (PST) Received: from alex-rivos.home (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id t18-20020a05600c451200b004076f522058sm5366438wmo.0.2023.11.10.06.08.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Nov 2023 06:08:24 -0800 (PST) From: Alexandre Ghiti To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , Arnd Bergmann , Dennis Zhou , Tejun Heo , Christoph Lameter , Andrew Morton , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-arch@vger.kernel.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH 1/2] mm: Introduce flush_cache_vmap_early() and its riscv implementation Date: Fri, 10 Nov 2023 15:07:20 +0100 Message-Id: <20231110140721.114235-2-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231110140721.114235-1-alexghiti@rivosinc.com> References: <20231110140721.114235-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 88551C000D X-Stat-Signature: 63fgjqfgqnax96xsycq8b9pe7ek6fj19 X-HE-Tag: 1699625306-901817 X-HE-Meta: U2FsdGVkX19uB3qpNIK2vDyOxAy0sK7Mb4kyeRSDQQTdpI0r2eCeiRC5BvxVBfMLbFlCcV0KV5U3/jqsADjapq6v4Wq9n1UIPecH1HWIQpooPF4TxvY+goPVgX4Ri3NXyq3grH+rkQHxPI6GU0J2j9cDZKcZ2aDe/ek5ghVUKWp9pgUtJWJb2FAJUi98dYHDb4Cl49pmB2d7sjqZNxUg04cWmHCTTQo3KiOJjVRGN+oXY+afyNaoyL3Z/GMIpijYohLOH93iMJTgPM32bRx/f+dkO4se46luvpVAGANE+aSUAso6CRKe+nzJ9tVA3HCRR9Msr9OJzVxh2+CeVZ2cRT3mSLa/yDI0NSqphc8+iIEH6QfUzNiWgsGr2A2ZFudT2Qf4vySz5FJZfa76pVnuqL5hea3yfXFAjPBRzep0o04WjiK1h53k0tGPI1OlgXZAAZG78ovlA9ebGCVmyXNeWHQ3PTS4fE9uZXrAWVHBw5CjBhTfOn3lRryPFrpw5W3BnLxNEG2i+QsZ5yMLg7uJQb3b1CuJv3bKhdpGfS2PZe0mMwp2FhPZKaSlrr1rXq30Ubo8kT4/fUsC2j+vddAQ/36rTLVZutJTa+nsxW4/FCMxfj7v6VRwWA6yTw+0UMeHdF5H73GKIdO6UtfVj46aDwpkDiNk+w9y34yFDnMfxwrg0rAEQuJtg7iEnoQvce3k0YZpwXKF90cymQIrdHU4OeKS/EsnTY33qaKidkGjZ77+tU9xK18Pd2k1wbYdKI8B9RRKIi3i9wEcca2IRVVZKESz0Txs+PBZLUhJ8KAB0ZybxXLzAd5zKwyPNxp5rceQb5l/kphXhBBRQ6wTK0kNVV4x2rob+fWmtJZC4D1D27auC8BjFkqqOwXKo6/XoEyLVrEL903guv6c7AHA5Lr8zUjFif/P9rAMpPmdsqqIxmCw45IOtAjT59i4VHZn9UwQnP4XUTPSkE7+fOueyOP 3l2b02X+ 5hhx43TtJI302R5G0rG8OK5qfe7z/95sQNA2ejweNRHSOj8+22j379GjFR89LfT7z+hktUWXV0csAk1NacCclsjgr0aueCtDwAwsdVzgKmQZacMecLor8TyufH6ZkRJaS1VyFN+xj+0oxnz9YzugqyEiXDr6EKPUgTONwS3gDqlj9iqDOFS/ZF8MhHsAQLANw2blaIuo9n666EFYLpW2p5ayMtH+mIBJjHFJcCenjVBAZ4brHw+TRv4tM6UHaE5Kr3BznltuXpLhTCPJ264jx+6n0IdIspgi8P8AU+BrluXRuEYTOynwbVs0Ecm6rRiLFKb99+cH8LlLCNAj0xX1RauhsDa5KYd6+X1nrBKeHJm1NQUZ/tlENvhIt4rtCS2yTN0WfKarslrjzQwc7LpeGPrfhijNCS9ZIKbSRAYpmB1LhykmigeouId0tfNq8ntOPu6nO4zOZ4Bpu2mYNLI6tsf50ZoxuQ1FQ8STvn/XlirboOHLCavYvNQj2GMgZmoIi5qCLS+PdlAv4P/pPQLXslGlIrU+2vcw6AWKD6lcVXdhsNswrAYi15EQG1x2zWoa7S9pdw9HyKt998vVb/34sAFhSb2SYpmZWpnaxUjXv15SuOY4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The pcpu setup when using the page allocator sets up a new vmalloc mapping very early in the boot process, so early that it cannot use the flush_cache_vmap() function which may depend on structures not yet initialized (for example in riscv, we currently send an IPI to flush other cpus TLB). But on some architectures, we must call flush_cache_vmap(): for example, in riscv, some uarchs can cache invalid TLB entries so we need to flush the new established mapping to avoid taking an exception. So fix this by introducing a new function flush_cache_vmap_early() which is called right after setting the new page table entry and before accessing this new mapping. This new function implements a local flush tlb on riscv and is no-op for other architectures (same as today). Signed-off-by: Alexandre Ghiti --- arch/riscv/include/asm/cacheflush.h | 3 ++- arch/riscv/include/asm/tlbflush.h | 2 ++ arch/riscv/mm/tlbflush.c | 5 +++++ include/asm-generic/cacheflush.h | 6 ++++++ mm/percpu.c | 8 +------- 5 files changed, 16 insertions(+), 8 deletions(-) diff --git a/arch/riscv/include/asm/cacheflush.h b/arch/riscv/include/asm/cacheflush.h index 3cb53c4df27c..a129dac4521d 100644 --- a/arch/riscv/include/asm/cacheflush.h +++ b/arch/riscv/include/asm/cacheflush.h @@ -37,7 +37,8 @@ static inline void flush_dcache_page(struct page *page) flush_icache_mm(vma->vm_mm, 0) #ifdef CONFIG_64BIT -#define flush_cache_vmap(start, end) flush_tlb_kernel_range(start, end) +#define flush_cache_vmap(start, end) flush_tlb_kernel_range(start, end) +#define flush_cache_vmap_early(start, end) local_flush_tlb_kernel_range(start, end) #endif #ifndef CONFIG_SMP diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 8f3418c5f172..f0d6328076b6 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -41,6 +41,7 @@ void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr); void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); void flush_tlb_kernel_range(unsigned long start, unsigned long end); +void local_flush_tlb_kernel_range(unsigned long start, unsigned long end); #ifdef CONFIG_TRANSPARENT_HUGEPAGE #define __HAVE_ARCH_FLUSH_PMD_TLB_RANGE void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, @@ -64,6 +65,7 @@ static inline void flush_tlb_kernel_range(unsigned long start, local_flush_tlb_all(); } +#define local_flush_tlb_kernel_range(start, end) flush_tlb_kernel_range(start, end) #define flush_tlb_mm(mm) flush_tlb_all() #define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() #endif /* !CONFIG_SMP || !CONFIG_MMU */ diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index e6659d7368b3..8aadc5f71c93 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -66,6 +66,11 @@ static inline void local_flush_tlb_range_asid(unsigned long start, local_flush_tlb_range_threshold_asid(start, size, stride, asid); } +void local_flush_tlb_kernel_range(unsigned long start, unsigned long end) +{ + local_flush_tlb_range_asid(start, end, PAGE_SIZE, FLUSH_TLB_NO_ASID); +} + static void __ipi_flush_tlb_all(void *info) { local_flush_tlb_all(); diff --git a/include/asm-generic/cacheflush.h b/include/asm-generic/cacheflush.h index 84ec53ccc450..7ee8a179d103 100644 --- a/include/asm-generic/cacheflush.h +++ b/include/asm-generic/cacheflush.h @@ -91,6 +91,12 @@ static inline void flush_cache_vmap(unsigned long start, unsigned long end) } #endif +#ifndef flush_cache_vmap_early +static inline void flush_cache_vmap_early(unsigned long start, unsigned long end) +{ +} +#endif + #ifndef flush_cache_vunmap static inline void flush_cache_vunmap(unsigned long start, unsigned long end) { diff --git a/mm/percpu.c b/mm/percpu.c index a7665de8485f..d287cebd58ca 100644 --- a/mm/percpu.c +++ b/mm/percpu.c @@ -3306,13 +3306,7 @@ int __init pcpu_page_first_chunk(size_t reserved_size, pcpu_fc_cpu_to_node_fn_t if (rc < 0) panic("failed to map percpu area, err=%d\n", rc); - /* - * FIXME: Archs with virtual cache should flush local - * cache for the linear mapping here - something - * equivalent to flush_cache_vmap() on the local cpu. - * flush_cache_vmap() can't be used as most supporting - * data structures are not set up yet. - */ + flush_cache_vmap_early(unit_addr, unit_addr + ai->unit_size); /* copy static data */ memcpy((void *)unit_addr, __per_cpu_load, ai->static_size); From patchwork Fri Nov 10 14:07:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13452444 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 296C7C4332F for ; Fri, 10 Nov 2023 14:09:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A1B66280028; Fri, 10 Nov 2023 09:09:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9A47F280022; Fri, 10 Nov 2023 09:09:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81E20280028; Fri, 10 Nov 2023 09:09:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6A602280022 for ; Fri, 10 Nov 2023 09:09:29 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 40B01B4C0A for ; Fri, 10 Nov 2023 14:09:29 +0000 (UTC) X-FDA: 81442227258.18.561AA92 Received: from mail-lj1-f173.google.com (mail-lj1-f173.google.com [209.85.208.173]) by imf06.hostedemail.com (Postfix) with ESMTP id 3FEA6180024 for ; Fri, 10 Nov 2023 14:09:26 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=jiXz5Fh0; dmarc=none; spf=pass (imf06.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.173 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699625367; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TNncknImGYd0G7MADnDhbFKl7Y37Hs80keyCtMwuCbk=; b=1NJboyd366y4EIRBzYAM3yllKxlO3zCJippj0p0p5VfE94q/i+alZ0oPkupDIdPGJwX2+g FXQsE41GW27fVZcTXb5SCefrNXB693Fye9c+0mwQjtzW9AUHX3zKy0yZObTcOQMwoQZX3u Pwr1cVQPNc1r/py5Ial+OzpDoNcoifI= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=jiXz5Fh0; dmarc=none; spf=pass (imf06.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.173 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699625367; a=rsa-sha256; cv=none; b=JgM2hhGSaIwJ9XiIj4gMJ4KCBHidbt5hodmC013S+I+tyGhy/tRugC0MSiZmm+10Eh82SZ DxHgS2rmCIEVBKbXDJl106/eGOrAZea5tdnXswTmQQzRdCVN8SPd946fhcBgvyS/NRUpMO KjPEiidyvYUZG/7w4Zu5EXRNDriI9EU= Received: by mail-lj1-f173.google.com with SMTP id 38308e7fff4ca-2c501bd6ff1so28348371fa.3 for ; Fri, 10 Nov 2023 06:09:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1699625365; x=1700230165; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TNncknImGYd0G7MADnDhbFKl7Y37Hs80keyCtMwuCbk=; b=jiXz5Fh0UUe0/KpSzjDXOyvFY8EtNoMXwGx9HhhHBUdFvg/M+SZ43tUCGQCTYU2DpE m8vgoVTrpWwzErtfCEUrTYKd2J1yDCUYhYtj20uKthcz1/QagNejRmKXZTrFYvO7cH8W 6RnQJdujrGbY+LFOwIc8InuKiqeECFSJczCndT4dzXy3TNGfOad3GTYsARHucRD/yHde kpGaVOx0/0YOHAT4pgJ62EIguhEuGWJy/Vml5xADzj8Ftmz6nZ0YDXQlA2p0xcJWTH8i WxEP8NkWfdLVMwWjzFPt+rKknRb+TcfqM8Zwj0UK53r81ngUx2B7Oq3YwPo7thZY636K Wftw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699625365; x=1700230165; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TNncknImGYd0G7MADnDhbFKl7Y37Hs80keyCtMwuCbk=; b=Qe0wQcqiHRgFRDv2MNob+cQ+r0L+BJldLvsvcr+c3m+7bnelfexzfzx0BQK4wkM4cl KerqbDkyxW9yi2JbZzQZgDDnncI0vAEq7wfbkmuN5WstW0bnSymWnvZGNNVIuKW6sSBc mP5qilqZDfyJdronaJYpVvqDCRTGVlLKegh6BCm8Bea0ujrbHaynjJ3ueKrybNVitbdq Li+N7ty4EhAiKsn+DVIpvFPVGhb/JQW2YlwlY7uhTA19tCbAhZOAxnAlmRmstVBLc+DM D0yUCJ2BYo+jIXrK/4FVPp7VmsG6AqTVbTxHmOdW2qYt7Wj5vpnd0jOXguiclBPGeCML DKIQ== X-Gm-Message-State: AOJu0YztGKPll5/OiYY2GkIVBxzA252aBIild7ZQ3KYqA/o++k4+duuI bI+mRxHfbRWG027PammjMu+33A== X-Google-Smtp-Source: AGHT+IFnc5DDn6eDvAmwHfvD11R9Doa6XtjEu4mYLiLBs82TOIbZPRsbYbDAWR9qLMG/ZVUIcvzhnw== X-Received: by 2002:a2e:2c15:0:b0:2c6:f51f:c96d with SMTP id s21-20020a2e2c15000000b002c6f51fc96dmr6506404ljs.13.1699625365568; Fri, 10 Nov 2023 06:09:25 -0800 (PST) Received: from alex-rivos.home (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id b12-20020a05600c150c00b004083a105f27sm5173099wmg.26.2023.11.10.06.09.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Nov 2023 06:09:25 -0800 (PST) From: Alexandre Ghiti To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , Arnd Bergmann , Dennis Zhou , Tejun Heo , Christoph Lameter , Andrew Morton , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-arch@vger.kernel.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH 2/2] riscv: Enable pcpu page first chunk allocator Date: Fri, 10 Nov 2023 15:07:21 +0100 Message-Id: <20231110140721.114235-3-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231110140721.114235-1-alexghiti@rivosinc.com> References: <20231110140721.114235-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 3FEA6180024 X-Stat-Signature: 46gj4pap6ypaczyiakbkicn3hjie1xpp X-Rspam-User: X-HE-Tag: 1699625366-170396 X-HE-Meta: U2FsdGVkX195/ANg+fvHuC0Di3fRye8KZBNEtDje6XRvVnhpxd3A/pnzt1kYyk6G+ICVzcygSGTq2sU4iZOGb75STNkKlbwKokD5zBBxSvdH0U7BnE4RqaimVWWInFoxCdf6D9DlEn7EPnfbNe8fzME2eGXmPMgpwjj4aVjBLQ4zRJc/x3se1A9j2mmgxZmwhWTQbD6C20biwvK025VfQuFyYdVXW9saPoEVac/nKttRuI/FrQqIsJAJfKBlDV94scYTcYSaKwzDlV9CGLfyifz4PGHd2wYUzFl+lwQ8JlQ1bcIKwKqRzF+qGUBYOqdFJzkBYOp/oZwGllSh/z0JVL+1rSpODmyD1dFT0E8mS/6KzKSNfCijA5LDghA5I1vCQ5IQsM6NCUZdMEZcGOAIQum8N7UsZSaGtMqglSuqmPxkXciiRkV0yRf9yOGgJiMbBXj1rUD5RR8erivHROnaCJ2BgqtxhMiC2Q1qziUZIpL/5aQ97wcUP1LDZ0dGTe89pOv0Wvne7tYTsURiehN7/+Ig+rGfsv+2qOlYrDDz1BCKcmPHrhKyR4JPBeGnCjHmLGkoIhmRyNXskpQDzvmwCnsqTmEm89l6ZzNJ8DMJ1LKnM+xyJbyJbIlTWBKie4IBWv8m2piZ07htWwu6hnbdvD4vkM9obc9fRjRsL0fRyxb70eXEKYdCcNwq8OJMozaftHfgFHsUI2woHftUEi8d5VHO5NC9N68T0MGeFDIqK5RrRekI9lpIbTqTfdZktA/x2O6fRpgYDVRKrXfDO/NbanLZABEMRmHpDO2rcHHF8UOZMXQSYZiuKydn4VGajkKbnsRpY72WVgDO6067DTrndmr5eNC2dI+yw3xKz7oyPMX/FgdqLWpSPBmieNRymgcef2/9LYZch56/isz4R4MhEm4qHYBg6mpSOUwn7OXTVW/xotsFAeg06SD+IFGYMSWUWJWdhp3/buqb6OJtP2f xIruiLKK znshzYcBBHltQVgxjN8umtH4DMtvKSogqNKDldn+TbVMEYSIC3+qmkkFWy7G88WhG/7Ioy7uQZAxZBesiW5FtNFR4nE6tQzVmYH2AZQD0tXYJaii+rI1i4/j1Fn+22k2fLYkUBCWFpsvKDxcGIuDT9ftQT+QbtXplIdgWeAzy+53+w5P2XqfqChNh1EslqZTKT3k3fyV5JrZN9N8p/hNnw6FOtadhmbxz/Jccad7RvoBKU7o3GbzvE7UY5KbEln87Q3GzYUbQIQyMCE++PBV9V94bhcExLllySZOAzffTRcCZq1solFjiAvbPUy4+Y+QwvH0P7+Bc1r4WLx0tTxJbQrdGbfAUaNVjxN6Kskoir7b/RpyedYEXEXwcS3Bfl+WuPFXk2BB0FTGlFOWDQLntuB/ffzQS+RUGx7bcBcDAXWGxVDiU5VpPL6WsnF5SHKcXm0IZAgPtPv9xlYcJAlOToZlvb7qWi9MSiXEVxNJ7iEAzVkgu2J42b5/6CP7aguiUcO5G2EKy+QXFJ1E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: As explained in commit 6ea529a2037c ("percpu: make embedding first chunk allocator check vmalloc space size"), the embedding first chunk allocator needs the vmalloc space to be larger than the maximum distance between units which are grouped into NUMA nodes. On a very sparse NUMA configurations and a small vmalloc area (for example, it is 64GB in sv39), the allocation of dynamic percpu data in the vmalloc area could fail. So provide the pcpu page allocator as a fallback in case we fall into such a sparse configuration (which happened in arm64 as shown by commit 09cea6195073 ("arm64: support page mapping percpu first chunk allocator")). Signed-off-by: Alexandre Ghiti --- arch/riscv/Kconfig | 2 ++ arch/riscv/mm/kasan_init.c | 8 ++++++++ 2 files changed, 10 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 5b1e61aca6cf..7b82d8301e42 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -416,7 +416,9 @@ config NUMA depends on SMP && MMU select ARCH_SUPPORTS_NUMA_BALANCING select GENERIC_ARCH_NUMA + select HAVE_SETUP_PER_CPU_AREA select NEED_PER_CPU_EMBED_FIRST_CHUNK + select NEED_PER_CPU_PAGE_FIRST_CHUNK select OF_NUMA select USE_PERCPU_NUMA_NODE_ID help diff --git a/arch/riscv/mm/kasan_init.c b/arch/riscv/mm/kasan_init.c index 5e39dcf23fdb..4c9a2c527f08 100644 --- a/arch/riscv/mm/kasan_init.c +++ b/arch/riscv/mm/kasan_init.c @@ -438,6 +438,14 @@ static void __init kasan_shallow_populate(void *start, void *end) kasan_shallow_populate_pgd(vaddr, vend); } +#ifdef CONFIG_KASAN_VMALLOC +void __init kasan_populate_early_vm_area_shadow(void *start, unsigned long size) +{ + kasan_populate(kasan_mem_to_shadow(start), + kasan_mem_to_shadow(start + size)); +} +#endif + static void __init create_tmp_mapping(void) { void *ptr;