From patchwork Fri Jan 10 18:40:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brendan Jackman X-Patchwork-Id: 13935238 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3854DE7719D for ; Fri, 10 Jan 2025 18:41:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1BADB6B0092; Fri, 10 Jan 2025 13:41:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 112806B00C7; Fri, 10 Jan 2025 13:41:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DEF926B00C9; Fri, 10 Jan 2025 13:41:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B98566B0092 for ; Fri, 10 Jan 2025 13:41:09 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 71FADB0496 for ; Fri, 10 Jan 2025 18:41:09 +0000 (UTC) X-FDA: 82992409458.26.7959B4E Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf12.hostedemail.com (Postfix) with ESMTP id 9576740004 for ; Fri, 10 Jan 2025 18:41:07 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=qZYQVmZ0; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of 3wWmBZwgKCNwH8AIK8L9EMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--jackmanb.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3wWmBZwgKCNwH8AIK8L9EMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--jackmanb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736534467; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WaFp0TOYaWSuGPhKgfBnenMecO/UsWYRxmgXZ26ETaU=; b=ytDQJAB0soSf0Tx4WWQrUBPQ8eiMpaWrQbaoJsDsVkM0ajBQgvQjvYVn+eBjQueEP083/Z xd+M/ZIP4z8HwEu4LUcKZTVVNjVwwYkwqBEZMWg+sap5SQoaprQpe7JE6UH/1DCE5rxu+7 CFORaY1QYBEK7a40HN486/bv6pt5lGc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736534467; a=rsa-sha256; cv=none; b=lLkXtNWkOwgwYFjKSsIG5liXxTrP/lLmpCKNkenHV9APk+0ItMHUwbYz6IV63ayddkC3a5 neGmwKhjYA7/QfI57Dzbd/MzSjd1hNLs2GYowqVVEe0RinqJlzf21X0hZVfIMyCvSt/u2M udoAHrucolnVhJKjuXF8gVQlgXsn0/k= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=qZYQVmZ0; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of 3wWmBZwgKCNwH8AIK8L9EMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--jackmanb.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3wWmBZwgKCNwH8AIK8L9EMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--jackmanb.bounces.google.com Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-4359206e1e4so19288725e9.2 for ; Fri, 10 Jan 2025 10:41:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736534466; x=1737139266; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=WaFp0TOYaWSuGPhKgfBnenMecO/UsWYRxmgXZ26ETaU=; b=qZYQVmZ0p9hX2/pNRV9dKT4B5r4ArLoVoTp0IW3ab+yiNRNnHGmskqQip6v0jGS/mt hmCTUFchYKJMT6VQnc05NYw5fI8m7xqSM+P3AmichOGKovdBluk9sE5QjpkPoNxvaiSt ya2RdFRNSaylj7fSVVB9p26+/8yDbYhRlJLVWK1ZdOhCY6ALUhTwq83X2pmsG+0dFELq rTLl6arYmafdb0mGENkWs8m4Vsazlj3pttzIdk1ijHreEpZx8reeoEfllz77hTgXo+hu V/84RD+LCZ9WC7Tn1Ng5yHb9fI8LhBhrGtvuSmxPuHjVchHQICWajazKn+FFsSXdrIed Ajhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736534466; x=1737139266; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WaFp0TOYaWSuGPhKgfBnenMecO/UsWYRxmgXZ26ETaU=; b=vB775fVS9drOqpqTPuBdoe7bU8SgQy1CsTjV/tOOxvJgEeWX1v/3tCPfLdCMWWGW6I x2yc/zvzSUZDgexpomjdkvYsfxPJubONhxymxaBT2uQaFJExQCcZ33zYbHpbhC3jb8Xw /jkgVzWfRdtUY5sRedggPR1VpTKnVc6OPiSPhRHSc8YZ1uWX8tNOLmB6nqiP3EuKFGCz bMHH5OVmWn12OaUFXfASVMGVEYa4CV+Q/bO4ve/+LGhKpqJcGqFwiXuqGloJAIzRFUXd QO3J1B6E7gVbeLjvynVFofV0wZyGqrIlumgpF2RRP7Z6bfB8odWAFXiOyCDKujbKZzcH 1Kuw== X-Forwarded-Encrypted: i=1; AJvYcCWE5nAotOfVVjsq2+dlLZVddlaUbSwNhA6hYkUUnhary2HE0ooWaM0CMLBO53z3w+TAimg4OS7Fyg==@kvack.org X-Gm-Message-State: AOJu0Yy2q5eJCHPRPi8sSJq2xzakmvLYxR0xEOA+XEh/ofwt9mBWDDpL RNzENXAs5lXArQ4ZAFldqp9QcF2BSrWLyFqGHUVkcHvHm9UqVSmOUq3CtFSxBy7d1gq7zbXHwoR GO8rJqN+Niw== X-Google-Smtp-Source: AGHT+IEE4az+Oz6XpM1h3immNPii0BZQy1AjHZ6Oe5B/in2qx8QsaCvn5PLTDmCUbo/W1PshFIv+Soo2Sqz9XA== X-Received: from wmrn35.prod.google.com ([2002:a05:600c:5023:b0:434:f2eb:aa72]) (user=jackmanb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3ca4:b0:434:a26c:8291 with SMTP id 5b1f17b1804b1-436e26e203emr101768035e9.24.1736534465947; Fri, 10 Jan 2025 10:41:05 -0800 (PST) Date: Fri, 10 Jan 2025 18:40:35 +0000 In-Reply-To: <20250110-asi-rfc-v2-v2-0-8419288bc805@google.com> Mime-Version: 1.0 References: <20250110-asi-rfc-v2-v2-0-8419288bc805@google.com> X-Mailer: b4 0.15-dev Message-ID: <20250110-asi-rfc-v2-v2-9-8419288bc805@google.com> Subject: [PATCH RFC v2 09/29] mm: asi: ASI page table allocation functions From: Brendan Jackman To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Richard Henderson , Matt Turner , Vineet Gupta , Russell King , Catalin Marinas , Will Deacon , Guo Ren , Brian Cain , Huacai Chen , WANG Xuerui , Geert Uytterhoeven , Michal Simek , Thomas Bogendoerfer , Dinh Nguyen , Jonas Bonn , Stefan Kristiansson , Stafford Horne , "James E.J. Bottomley" , Helge Deller , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Naveen N Rao , Madhavan Srinivasan , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Yoshinori Sato , Rich Felker , John Paul Adrian Glaubitz , "David S. Miller" , Andreas Larsson , Richard Weinberger , Anton Ivanov , Johannes Berg , Chris Zankel , Max Filippov , Arnd Bergmann , Andrew Morton , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Uladzislau Rezki , Christoph Hellwig , Masami Hiramatsu , Mathieu Desnoyers , Mike Rapoport , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , Dennis Zhou , Tejun Heo , Christoph Lameter , Sean Christopherson , Paolo Bonzini , Ard Biesheuvel , Josh Poimboeuf , Pawan Gupta Cc: x86@kernel.org, linux-kernel@vger.kernel.org, linux-alpha@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-um@lists.infradead.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, kvm@vger.kernel.org, linux-efi@vger.kernel.org, Brendan Jackman , Junaid Shahid X-Stat-Signature: pf6owxogotai5sjygoijdnsnapxb64zt X-Rspamd-Queue-Id: 9576740004 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1736534467-30985 X-HE-Meta: U2FsdGVkX19Hrhh8dFBE3/JZ78+af2RIiuD0YWXmyIUyDYCP0Nzb1fre7KlaMBNz6r/q3GVpbBPQMFdWG1G9Brp8yUR658Yi454XGqNCMM9mKVMRkNIxfE70VRhHuteUlyj8/4wx4xla6tXEUgJkutmioS2OqeJeIChIZBNPpPtGJiNrh2OgSryPOjqhyw9i4KSCzRUcAywk/LP6ray+dGzyeZhbxboUg01J1A0iZpyLAkseTNiJPEDP97Fs6jEBhEQUHpg5vM0JPDx2qIKwpgxrvsgErn7j3ZjJyBb/ZFcrxDrevNnCwUgh6iZitHSBcYDeuuxrka73PmXjKboPCbeHdFYN1hHtX3Mu/FHho7lKRV/8xozn06BG6zSukstg9ZH27drPuo4likQZUtErAu4vwPow6n6tcMX3C0ivT/JSpG/h3lzuMiR83NnQEW1cXTkUOIng2qIdQxwbIfCDjVCmUEPxf7z8MImbxtInHMy7fENymwhuxazGLyjdampZLHiJWxq8YEI4EjcY1g3ufD6Pr8WQn8SKGLTSa6b3evJTTgOZqa9M48BoEzYLasbaootm814NW6mJB07TXwRcNyKR68ALyFWTbRId8Lmra+LQeDWC1f8kt2zj6MhIchBeWrE+HVcjQ4hxeQEX1foix1q0Lg8R54Yc0ktU+74Fn/m0tSWLpmcUBe3YS2fSKNudwgz8jT1fp1sSTyCIp3KfK6P6N8dqlfOBWM1zOnJKk2IPS+eNwZtwGkkbvV+bLK9kMWqgyuBCs7CxquEz8AQT7n8Zw68rQlFIwKUKC7uU+4sLfXLp3oOBgr8nh+HES9cIHR4svPClyLFNXRx9Vaq2SVZ8u5XQKSdae8FmbJlwlR3X9iE2xfFsBP40VxMfYUCWq23JJr7P2NRAp2hgCQ65spP8Ixy1Q7jNex2Shv7HRkKN9CbZhkK+56QGvBf8yLcH1GNl/C3ivdEQqg9TJ2x Qsk1jO5n F5qExIlsi8I1FjOKIgAlY4AIzeF6wuufFgkzp3j+P1oR86QIlekEYe4Ryqpx1+nwKzKg3oO0OVdnmPaFNz+FQLg5FRjaeqRw8tdsveLtpbNmNMiz3KhxHVfWfCObmGmZYFvGMfl25P5ORdnn+wElPrHgKZm6M5B9KlwztcN7yUCRnvanRTRNBSnMqiU2UW0z9CdGD1DP0FGz5iSjINxHtgCRgrHv1hkysgwB42hmW3BTSRbG4bFFMFQFOpWdyiTsRk6XwrsMGZFLcarJDBMhnwa11NYWpF1uoK5c1hdvzmcsw0FhQczGsNrH/zfLt/QXTbjAyBkQNoFrNdCGKX0nUfMLrW5wyd1LcnM5xuIbExyQz5ikaT70BlvQY9z6PaFjSEtWyB4Zm8kWAs5nK5RxkfTW4jzPap+dEafMygfQRTipdwVX1+fhqhdOVQeMo2IRcP1vt9p3ZlyOmcgOih4dbbuDad6ksr0DzT64KqlZL8ZWu4Ylt89F22+K4kcvvw0bhXXiwGHRWQZnfttPdTyEczjFpIewGfT5gJ0qTjFeEQcE9cpIihOUn6qgBhw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Junaid Shahid This adds custom allocation and free functions for ASI page tables. The alloc functions support allocating memory using different GFP reclaim flags, in order to be able to support non-sensitive allocations from both standard and atomic contexts. They also install the page tables locklessly, which makes it slightly simpler to handle non-sensitive allocations from interrupts/exceptions. checkpatch.pl MACRO_ARG_UNUSED,SPACING is false positive. COMPLEX_MACRO - I dunno, suggestions welcome. Checkpatch-args: --ignore=MACRO_ARG_UNUSED,SPACING,COMPLEX_MACRO Signed-off-by: Junaid Shahid Signed-off-by: Brendan Jackman --- arch/x86/mm/asi.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/arch/x86/mm/asi.c b/arch/x86/mm/asi.c index 8d060c633be68b508847e2c1c111761df1da92af..b15d043acedc9f459f17e86564a15061650afc3a 100644 --- a/arch/x86/mm/asi.c +++ b/arch/x86/mm/asi.c @@ -73,6 +73,65 @@ const char *asi_class_name(enum asi_class_id class_id) return asi_class_names[class_id]; } +#ifndef mm_inc_nr_p4ds +#define mm_inc_nr_p4ds(mm) do {} while (false) +#endif + +#ifndef mm_dec_nr_p4ds +#define mm_dec_nr_p4ds(mm) do {} while (false) +#endif + +#define pte_offset pte_offset_kernel + +/* + * asi_p4d_alloc, asi_pud_alloc, asi_pmd_alloc, asi_pte_alloc. + * + * These are like the normal xxx_alloc functions, but: + * + * - They use atomic operations instead of taking a spinlock; this allows them + * to be used from interrupts. This is necessary because we use the page + * allocator from interrupts and the page allocator ultimately calls this + * code. + * - They support customizing the allocation flags. + * + * On the other hand, they do not use the normal page allocation infrastructure, + * that means that PTE pages do not have the PageTable type nor the PagePgtable + * flag and we don't increment the meminfo stat (NR_PAGETABLE) as they do. + */ +static_assert(!IS_ENABLED(CONFIG_PARAVIRT)); +#define DEFINE_ASI_PGTBL_ALLOC(base, level) \ +__maybe_unused \ +static level##_t * asi_##level##_alloc(struct asi *asi, \ + base##_t *base, ulong addr, \ + gfp_t flags) \ +{ \ + if (unlikely(base##_none(*base))) { \ + ulong pgtbl = get_zeroed_page(flags); \ + phys_addr_t pgtbl_pa; \ + \ + if (!pgtbl) \ + return NULL; \ + \ + pgtbl_pa = __pa(pgtbl); \ + \ + if (cmpxchg((ulong *)base, 0, \ + pgtbl_pa | _PAGE_TABLE) != 0) { \ + free_page(pgtbl); \ + goto out; \ + } \ + \ + mm_inc_nr_##level##s(asi->mm); \ + } \ +out: \ + VM_BUG_ON(base##_leaf(*base)); \ + return level##_offset(base, addr); \ +} + +DEFINE_ASI_PGTBL_ALLOC(pgd, p4d) +DEFINE_ASI_PGTBL_ALLOC(p4d, pud) +DEFINE_ASI_PGTBL_ALLOC(pud, pmd) +DEFINE_ASI_PGTBL_ALLOC(pmd, pte) + void __init asi_check_boottime_disable(void) { bool enabled = IS_ENABLED(CONFIG_MITIGATION_ADDRESS_SPACE_ISOLATION_DEFAULT_ON);