From patchwork Mon Jun 10 20:48:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Torvalds X-Patchwork-Id: 13692416 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E1EE6C27C55 for ; Mon, 10 Jun 2024 20:49:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=nIn8LYtuXKmfv1lpq+E8x5krCtZ5SQ2pFNjuwryZ5Go=; b=Cz2kICkS5zTjAt kqgKMlrMoB2qOIb5Szuy6iTLZF/VTD4trQHxpB3Btf6PZGDNwH05usSOws8lt1fGDMEKLF/H31I67 VU+InoFujOa6M1Ur5WSnlA7ysgRTZlbaoT/2LetlsGD3y1fg9Dinn/imB+p4bQvL66/+dmzJ9izhZ rmvgIFcP4pEWb7vZUYWdxcbgRvx8w1eGw17oAxL1xjy8fNlgJ37RU9CeBC/hZRL3sqeuboMTVej3p 6oqk9nY8xBt2v/N2Lw3HBczsskp7BslnBZUtlwWCo1gPoZ6KMQWzsNIzIUqguBVB1E63VBSdbEYBu urMYPj7M9hNNtLyJbVHA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sGlwi-00000006Qzo-2aqA; Mon, 10 Jun 2024 20:48:52 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sGlwf-00000006Qvi-2tJ4 for linux-arm-kernel@lists.infradead.org; Mon, 10 Jun 2024 20:48:51 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 1CB0360C13; Mon, 10 Jun 2024 20:48:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 86479C32789; Mon, 10 Jun 2024 20:48:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1718052525; bh=FOi86UUldJ0DWqgv3J6xhcGdiAK6DXsRyDRsaDmLflk=; h=From:To:Cc:Subject:Date:From; b=JbjctN5ukgqDo74t/mtIe3qdJp7NLV6bNy0d+r9Tl9X8W2pffb0HzTbX9bcA2WglY u3t5dDKfR3wnUKv9jy5lrQ7feHQUisP+6OvaVf+J0CmAu+oZQQoF9dp3n+rGQNrnT9 lhBQcb81GuKAGvcYAlquAO2mDYX4s1hUdkY5t/M8= From: Linus Torvalds To: Peter Anvin , Ingo Molnar , Borislav Petkov , Thomas Gleixner , Rasmus Villemoes , Josh Poimboeuf , Catalin Marinas , Will Deacon Cc: Linux Kernel Mailing List , the arch/x86 maintainers , linux-arm-kernel@lists.infradead.org, linux-arch , Linus Torvalds Subject: [PATCH 0/7] arm64 / x86-64: low-level code generation issues Date: Mon, 10 Jun 2024 13:48:14 -0700 Message-ID: <20240610204821.230388-1-torvalds@linux-foundation.org> X-Mailer: git-send-email 2.45.1.209.gc6f12300df MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240610_134849_900167_03E1009B X-CRM114-Status: GOOD ( 16.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org So this is the result of me doing some profiling on my 128-core Altra box. I've sent out versions of this before, but they've all been fairly ugly partial series. This is the full cleaned-up series with patches split up to be logical, and with fixes from some of the commentary from previous patches. The first four patches are for the 'runtime constant' code, where I did the initial implementation on x86-64 just because I was more comfy with that, and the arm64 version of it came once I had the x86-64 side working. The horror that is __d_lookup_rcu() shows up a lot more on my Altra box because of the relatively pitiful caches, but it's something that I've wanted on x86-64 before. The arm64 numbers just made me bite the bullet on the whole runtime constant thing. The last three patches are purely arm64-specific, and just fix up some nasty code generation in the user access functions. I just noticed that I will need to implement 'user_access_save()' for KCSAN now that I do the unsafe user access functions. Anyway, that 'user_access_save/restore()' issue only shows up with KCSAN. And it would be a no-op thanks to arm64 doing SMAP the right way (pet peeve: arm64 did what I told the x86 designers to do originally, but they claimed was too hard, so we ended up with that CLAC/STAC instead)... Sadly that "no-op for KCSAN" would is except for the horrid CONFIG_ARM64_SW_TTBR0_PAN case, which is why I'm not touching it. I'm hoping some hapless^Whelpful arm64 person is willing to tackle this (or maybe make KCSAN and ARM64_SW_TTBR0_PAN incompatible in the Kconfig). Note: the final access_ok() change in 7/7 is a API relaxation and cleanup, and as such much more worrisome than the other patches. It's _simpler_ than the other patches, but the others aren't intended to really change behavior. That one does. Linus Torvalds (7): vfs: dcache: move hashlen_hash() from callers into d_hash() add default dummy 'runtime constant' infrastructure x86: add 'runtime constant' support arm64: add 'runtime constant' support arm64: start using 'asm goto' for get_user() when available arm64: start using 'asm goto' for put_user() when available arm64: access_ok() optimization arch/arm64/include/asm/runtime-const.h | 75 ++++++++++ arch/arm64/include/asm/uaccess.h | 191 +++++++++++++++++-------- arch/arm64/kernel/mte.c | 12 +- arch/arm64/kernel/vmlinux.lds.S | 3 + arch/x86/include/asm/runtime-const.h | 61 ++++++++ arch/x86/kernel/vmlinux.lds.S | 3 + fs/dcache.c | 17 ++- include/asm-generic/Kbuild | 1 + include/asm-generic/runtime-const.h | 15 ++ include/asm-generic/vmlinux.lds.h | 8 ++ 10 files changed, 319 insertions(+), 67 deletions(-) create mode 100644 arch/arm64/include/asm/runtime-const.h create mode 100644 arch/x86/include/asm/runtime-const.h create mode 100644 include/asm-generic/runtime-const.h