From patchwork Wed Apr 27 17:12:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12829262 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A76C2C433F5 for ; Wed, 27 Apr 2022 17:28:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=HRe9zaDs1crEh1Z0uQjCznqh2ZknurR3SEeekfqjFqs=; b=L5opjKhdrjC8Xu +DosY4K9Obab1CCHSC+nEhMPfPEZSki3WCUts7BgdaV0jrXtttVJNWFB6x4bVlfptNpO5J7fh5Ln8 s7UxqeAvZ1HnKaC3LMAJkmvw+aroN+N+cIKAztRUh4lvdZ7wpIvsGOWaW8RhQ0VunVouO+LgjmziQ FRlpp5SHK5MSgF89wZ+sQfEyoSXretH//e/q/x/pw4qx43ZI0XHj3asQPKXUEqN9NjyiNf75WFLxe 5MdI0T3H0mRz7zIK1mfeR/9ElfveUubBldvAwJy6vThQkBRXZUochANsJBe3H4WATc2et3haTeVD0 YhcIxd6HzR/4a86vIexw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njlRZ-002epF-6M; Wed, 27 Apr 2022 17:27:13 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njlDs-002ZFh-JY for linux-arm-kernel@lists.infradead.org; Wed, 27 Apr 2022 17:13:08 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 93BB061E18; Wed, 27 Apr 2022 17:13:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DE246C385B0; Wed, 27 Apr 2022 17:13:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1651079583; bh=bQZstQBwyA5UayDt39rQFQoQ6xyTaxuR5gluVAnpbCo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mI404qdyQVYXiaRbn6/6ZPVC2ntIlSl21vTAydS/n9ncaoCJbrAi6XBF8ip35OMWJ TizpRxIsSZx0MaVroD9Ogi7Lb40W8BUC7cCdEwldLoX9TvZiDoH6pvNi3DK/++jZpO foWGK2/L6PaHVyPUMjT2tIKIwjeRVgK6ACbFjYKzk1wkgcEjLeRVp0oVv26TI7XuQ9 eRdNpvQXdy5cEmCjozQ+SU7StaHEWZbTOwjWDFw2pAZx6UhH4YN8xBwzWchaP/H69j EMxGtAKM3oO+eEA2/X3HhpylUXSizfbP3R0Fb9R0b3wB1ysmrVKnDmnexQmdlQLkLZ q/Piad9h99K6g== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: clang-built-linux@googlegroups.com, will@kernel.org, catalin.marinas@arm.com, keescook@chromium.org, mark.rutland@arm.com, nathan@kernel.org, Ard Biesheuvel , Sami Tolvanen , Nick Desaulniers Subject: [RFC PATCH 1/2] arm64: jump_label: use more precise asm constraints Date: Wed, 27 Apr 2022 19:12:40 +0200 Message-Id: <20220427171241.2426592-2-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220427171241.2426592-1-ardb@kernel.org> References: <20220427171241.2426592-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1925; h=from:subject; bh=bQZstQBwyA5UayDt39rQFQoQ6xyTaxuR5gluVAnpbCo=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBiaXmGCSeZrap2M2MThRtu4fDOfDoPj5aMO6bi5gQ1 eQuq2PWJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYml5hgAKCRDDTyI5ktmPJNofC/ 0VZ5MoKi4s9Y+GZ5c8YFJO+jHPce5blHGGtcHMZS9q+d67GFIRnul6D3Walmuh81behJNGEtGZHC7M Y7/lhDbH7uQ4bnOWezIVYDR0WhpgvMHvdVdbrB/Lwa9ov7Jrd6HWubST9cn8PSnI2muUbWRkOkMVCO n63UPa0S+f5KXgY1VWUkeGZHp5DucgdYNqx76C4WrBz9ExgmNiRxCw2ZToRxtV34RywpdFEoeYm5Dh gRc81k6i8db9zmxTgKlvUrPfXL8iqs4jwUepd9+NQt7Yz/i0xdxt5XIs2KExN0yhbcroxJWbMAEESM a3lGizpea2DK3p7Cb1Z6+NiuYN0N8N3Av9fG/dClYe7QZikg+unIu486eoLHxK0claKGNRlqZ9MDUn mLcPSHM8abPdY8TKtTIGhcAXPTgNAp7j2iQ5kSRGPqGXjA4jFUd6f5FNmljiYjGfDKocVf3xYAXNMK 7Kr3rwk3ePpwmKjh+xpYMLv8bwU1Gv0jJUozhmSohVa+w= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220427_101304_779088_752DDE19 X-CRM114-Status: GOOD ( 11.54 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In order to set bit #0 of the struct static_key pointer in the the jump label struct, we currently cast the pointer to char[], and take the address of either the 0th or 1st array member, depending on the value of 'branch'. This works but creates problems with -fpie code generation: GCC complains about the constraint being unsatisfiable, and Clang miscompiles the code in a way that causes stability issues (immediate panic on 'attempt to kill init') So instead, pass the struct static_key reference and the 'branch' immediate individually, in a way that satisfies both GCC and Clang (GCC wants the 'S' constraint, whereas Clang wants the 'i' constraint for argument %0) Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/jump_label.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/jump_label.h b/arch/arm64/include/asm/jump_label.h index cea441b6aa5d..f741bbacf175 100644 --- a/arch/arm64/include/asm/jump_label.h +++ b/arch/arm64/include/asm/jump_label.h @@ -23,9 +23,9 @@ static __always_inline bool arch_static_branch(struct static_key *key, " .pushsection __jump_table, \"aw\" \n\t" " .align 3 \n\t" " .long 1b - ., %l[l_yes] - . \n\t" - " .quad %c0 - . \n\t" + " .quad %c0 - . + %1 \n\t" " .popsection \n\t" - : : "i"(&((char *)key)[branch]) : : l_yes); + : : "Si"(key), "i"(branch) : : l_yes); return false; l_yes: @@ -40,9 +40,9 @@ static __always_inline bool arch_static_branch_jump(struct static_key *key, " .pushsection __jump_table, \"aw\" \n\t" " .align 3 \n\t" " .long 1b - ., %l[l_yes] - . \n\t" - " .quad %c0 - . \n\t" + " .quad %c0 - . + %1 \n\t" " .popsection \n\t" - : : "i"(&((char *)key)[branch]) : : l_yes); + : : "Si"(key), "i"(branch) : : l_yes); return false; l_yes: From patchwork Wed Apr 27 17:12:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12829260 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3C896C433EF for ; Wed, 27 Apr 2022 17:26:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Xtxt20/1iK22TmxNbaAjIA8t5Qejf/zyMtAjtcU99lY=; b=jCEMegkyiaP5Y2 VEelVgVN5hW6mPMc83s5exBJHCamAqlaD8N4S19YZZgQb2/2dJ9RlZREDQvTswNKv9noNn7zLSCLp 61LTdHK//zjinJfkeUoavk1nXNlMokNbMRh1zHjLhk9ghk4lHhboCZV3eH/DiAQ1bRLxNVh1LY2J0 H5ImSJIiztbh6ZdB0Pi8XT+t/VQtiM6QSpw+jXRelGnjlmyjuw/jikliC/yQm01dwgqL7ohh1TW4L zd1+yfLtfhhyKSbuYnzNFNWl7HODDONML+TLOK4/dcJ5bXVtPGfCqtocwvt7QUKsDhyxLSMSa5o6e jO7FXy8OOZp0fylADcOw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njlPm-002eCq-OE; Wed, 27 Apr 2022 17:25:23 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njlDu-002ZGM-IR for linux-arm-kernel@lists.infradead.org; Wed, 27 Apr 2022 17:13:08 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1FFB261DFC; Wed, 27 Apr 2022 17:13:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6EC0AC385A9; Wed, 27 Apr 2022 17:13:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1651079585; bh=0zQad1AzUfSHcODYLrfSNabOLPAeMX8RNzogCZtNbtY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AD/8r+7AXhFckjTI9SE86QRwjkukODcnIF+lo5y9Tj3iVIsSutIIq5QmLPxTDTLCm Yy0jJJhZxSvr7G1Wpol9U2AC6ggn/dEXBt7cvRFkdRHDHOQja3bHA5vxoRJDpvFTS6 FrDtouCNW1t/HrdNDuSGeTXai5+JZiLbKnsFD/x4IrcfKfIFRYERjVXFDodRAl4eht wJejMrLn3Az0FGuckgt9PfTWv3L1pdr2PpgIDWHhvQBUoq2dm42lp6ZgQn64M+qakL orfKEC+YQ8I7+xqe+9WrWtTjGRGSLEHYnZPbcIsN9euGLZALmLxmgsq+TX2B5IL17a VdoGKcMEQjbwA== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: clang-built-linux@googlegroups.com, will@kernel.org, catalin.marinas@arm.com, keescook@chromium.org, mark.rutland@arm.com, nathan@kernel.org, Ard Biesheuvel , Sami Tolvanen , Nick Desaulniers Subject: [RFC PATCH 2/2] arm64: kernel: switch to PIE code generation for relocatable kernels Date: Wed, 27 Apr 2022 19:12:41 +0200 Message-Id: <20220427171241.2426592-3-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220427171241.2426592-1-ardb@kernel.org> References: <20220427171241.2426592-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4534; h=from:subject; bh=0zQad1AzUfSHcODYLrfSNabOLPAeMX8RNzogCZtNbtY=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBiaXmImHHxfcHQw0Gp6ComhPU3xoA64oQsSc9QbO9i oGwpLHGJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYml5iAAKCRDDTyI5ktmPJPbhC/ 9R5hCeZkcm+5EEtGr0stIHqQ68Du8cfsdjjfZwtbXcgNBoFLKZFwGN7kHZQisTOy0x6q2qEBh/SKND GQomgfP582S/Ohi6YDdIM1VZGNgG19SvkesvwKETvDFK7auiISiTjUGuUI+AKAsTyq4cGChSvgC0B/ NGDTkoMP9ixpFZ5CItrKfFx5CxlvseJuAe3qRchGE70uI0AHd3/JZDPVZW0RiiXzr5sLykNMNTa5e5 b25i/Yt5EUxWTXArUPeL2QWU/qP/kMllmjRw8t6faVn8boNvwI3VekeIgaHU3ea6iHCcM25DoTkCjI CZH2qJItBhRkQ1zmbgwcrlB4+fmNw5Jn8fAgI3TJXF6Q+JPhqA+9zUlXCFmAmwPwGrZsPsqfzdHNwf NfAZfuzWn/CsjnU2+AHfAt9JpxIfUVVXSirPeoOmFo0CaZGoU5YTR/87xobnmy5h50VoZ6+RSgCObx 30LxnC42XW728cKqCOVeecpWVZoKx0LNo31RwVjT8dKmc= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220427_101306_741877_185ACED3 X-CRM114-Status: GOOD ( 22.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We currently use ordinary, position dependent code generation for the core kernel, which happens to default to the 'small' code model on both GCC and Clang. This is the code model that relies on ADRP/ADD or ADRP/LDR pairs for symbol references, which are PC-relative with a range of -/+ 4 GiB, and therefore happen to be position independent in practice. This means that the fact that we can link the relocatable KASLR kernel using the -pie linker flag (which generates the runtime relocations and inserts them into the binary) is somewhat of a coincidence, and not something which is explicitly supported by the toolchains. The reason we have not used -fpie for code generation so far (which is the compiler flag that should be used to generate code that is to be linked with -pie) is that by default, it generates code based on assumptions that only hold for shared libraries and PIE executables, i.e., that gathering all relocatable quantities into a Global Offset Table (GOT) is desirable because it reduces the CoW footprint, and because it permits ELF symbol preemption (which lets an executable override symbols defined in a shared library, in a way that forces the shared library to update all of its internal references as well). Ironically, this means we end up with many more absolute references that all need to be fixed up at boot. Fortunately, we can convince the compiler to handle this in a way that is a bit more suitable for freestanding binaries such as the kernel, by setting the 'hidden' visibility #pragma, which informs the compiler that symbol preemption or CoW footprint are of no concern to us, and so PC-relative references that are resolved at link time are perfectly fine. So let's enable this #pragma and build with -fpie when building a relocatable kernel. This also means that all constant data items that carry statically initialized pointer variables are now emitted into the .data.rel.ro* sections, so move these into .rodata where they belong. Code size impact (GCC): Before: text data bss total filename 16712396 18659064 534556 35906016 vmlinux After: text data bss total filename 16804400 18612876 534556 35951832 vmlinux Code size impact (Clang): Before: text data bss total filename 17194584 13335060 535268 31064912 vmlinux After: text data bss total filename 17194536 13310032 535268 31039836 vmlinux Signed-off-by: Ard Biesheuvel --- arch/arm64/Makefile | 4 ++++ arch/arm64/kernel/vmlinux.lds.S | 9 ++++----- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 2f1de88651e6..94b6c51f5de6 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -18,6 +18,10 @@ ifeq ($(CONFIG_RELOCATABLE), y) # with the relocation offsets always being zero. LDFLAGS_vmlinux += -shared -Bsymbolic -z notext \ $(call ld-option, --no-apply-dynamic-relocs) + +# Generate position independent code without relying on a Global Offset Table +KBUILD_CFLAGS_KERNEL += -fpie -include $(srctree)/include/linux/hidden.h + endif ifeq ($(CONFIG_ARM64_ERRATUM_843419),y) diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index edaf0faf766f..b1e071ac1acf 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -174,8 +174,6 @@ SECTIONS KEXEC_TEXT TRAMP_TEXT *(.gnu.warning) - . = ALIGN(16); - *(.got) /* Global offset table */ } /* @@ -192,6 +190,8 @@ SECTIONS /* everything from this point to __init_begin will be marked RO NX */ RO_DATA(PAGE_SIZE) + .data.rel.ro : ALIGN(8) { *(.got) *(.data.rel.ro*) } + HYPERVISOR_DATA_SECTIONS idmap_pg_dir = .; @@ -273,6 +273,8 @@ SECTIONS _sdata = .; RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_ALIGN) + .data.rel : ALIGN(8) { *(.data.rel*) } + /* * Data written with the MMU off but read with the MMU on requires * cache lines to be invalidated, discarding up to a Cache Writeback @@ -320,9 +322,6 @@ SECTIONS *(.plt) *(.plt.*) *(.iplt) *(.igot .igot.plt) } ASSERT(SIZEOF(.plt) == 0, "Unexpected run-time procedure linkages detected!") - - .data.rel.ro : { *(.data.rel.ro) } - ASSERT(SIZEOF(.data.rel.ro) == 0, "Unexpected RELRO detected!") } #include "image-vars.h"