From patchwork Thu Oct 26 20:29:07 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Desaulniers X-Patchwork-Id: 10028779 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 759196022E for ; Thu, 26 Oct 2017 20:30:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6564C28ECA for ; Thu, 26 Oct 2017 20:30:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5858E28ECC; Thu, 26 Oct 2017 20:30:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, DKIM_VALID, RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CD33E28ECA for ; Thu, 26 Oct 2017 20:30:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=KpD1NHL1PT2T9OMSiwY4uefO5uN60haTfPz9F+2KXDA=; b=g9n 3LMLhz21OnYVkMhAl2A6zfI+MCvDkDGVcW5YxxdhU99+nPd78qSCz5QstW1Lw7Qq216PknCndczwD 45vk1GzZe0zKDqYz9zAJhkFuLbS3CDjdtQ3+Tb2xJSa9a4EHgnnGFLav8nBNsvAbl1hUjB85D3r3C 0omcRA7m1/odhQlaC3cmjPZeojn/jJQlyw/8RmmSZ8HO0OYdDye+BT1RqOV0XsTzdN2SHllMCQENx zArLcjVmyZhOPxfpJznx/o+Sh68/GWVM7tklzffJD5L6cyB2iIm6BBH6DnOgA95BYof1rydOaA/m1 1jXpHyaqzpgjQ3iqy9kYdLv37VTnNZg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1e7on7-0002ER-Pd; Thu, 26 Oct 2017 20:30:13 +0000 Received: from mail-io0-x244.google.com ([2607:f8b0:4001:c06::244]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1e7omh-0002CU-Hf for linux-arm-kernel@lists.infradead.org; Thu, 26 Oct 2017 20:29:49 +0000 Received: by mail-io0-x244.google.com with SMTP id p186so8146865ioe.12 for ; Thu, 26 Oct 2017 13:29:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=96lZT80J1eKTgIeLB5qCvvv6+u9IU3l5fb6ZZK2qxIc=; b=Vn9/vo6c90B7ALCimrTfoYKNlO9sj6hqaTsabw552yrXS01ezAz27bVE3b9MqaAsnd hNf053jRXdbwekTSvEzjEumdNe/TvPs7tnVItNIYAoCM8zw7hK3rbCslpVXWKbjo9WuH F7lCXdh4pa+7JlOVVypSFPUHJ+gmI42x/h9wpMO7r3WL/8j9oVC9Cbfxix0/RSD1O1Se L+anu6CEN9mNkE9OVDEXaOiq7rXl5VMdEURnvRs9j1eVPOGzxErPUjFHMKVd2MLexkPi 6LyOhKUlS+J4dofIXX/lBC4h/h18WHRH85tBIifKR7b1y9NHOWiy+pX91nQWXTCRYJNw yLAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=96lZT80J1eKTgIeLB5qCvvv6+u9IU3l5fb6ZZK2qxIc=; b=RW/ghaaF72tMYDJmih3ZgGK10Rt1kDG3nKTMo7/UhZgqNu8QPjO74/BWVbTISo4hPO 4pwbneMTojJqHBOdpTYiW/J56y8jUUg7+vsIcVmcw2QIHXfWEq0UTc9ODkY2vp9qXr7O f3K0SAxouJxOKht1VzYFY5JucaHpRRRxSCa6RtS9CyUOZDzNqc0XSdvvJYzSnqS7ZcEl gHemuX1sCKTW9u/+D2c7C8D3km7QBt8R5H/KrAYZcf/Y/HrfxnGINdwvEhBo++EuDjtY pTx9N6wkXNCt1YC0wxmQTgT7rIt1pEvXyx1wfF6J0iUJP0jRQe4qAgMEz2AfTzX093rG nhxw== X-Gm-Message-State: AMCzsaURN8qY4OyhZmYzMLxd5dwuW4tme0rdGKQR1QeNFCkf1c4CtNhd gu0WKZY07cSu1PMDr9ZOevSeIQ== X-Google-Smtp-Source: ABhQp+QfiJIdHN/m/uHbpkjFq7FA7QW4tlmjxGc8/es1ZhjRbQnLOdm56eReuiOXQ9C9UjMgMjEg9w== X-Received: by 10.107.147.67 with SMTP id v64mr30715882iod.211.1509049764843; Thu, 26 Oct 2017 13:29:24 -0700 (PDT) Received: from ndesaulniers0.mtv.corp.google.com ([100.98.120.97]) by smtp.gmail.com with ESMTPSA id k66sm66621ita.6.2017.10.26.13.29.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 26 Oct 2017 13:29:23 -0700 (PDT) From: Nick Desaulniers To: Subject: [PATCH] arm64: prevent regressions in compressed kernel image size when upgrading to binutils 2.27 Date: Thu, 26 Oct 2017 13:29:07 -0700 Message-Id: <20171026202907.91852-1-ndesaulniers@google.com> X-Mailer: git-send-email 2.15.0.rc2.357.g7e34df9404-goog X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20171026_132947_631947_37B4A330 X-CRM114-Status: GOOD ( 14.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kees Cook , Catalin Marinas , Jiri Kosina , Nick Desaulniers , linux-kernel@vger.kernel.org, Will Deacon , linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Upon upgrading to binutils 2.27, we found that our lz4 compressed kernel images were significantly larger, resulting is 10ms boot time regressions. As noted by Rahul: "aarch64 binaries uses RELA relocations, where each relocation entry includes an addend value. This is similar to x86_64. On x86_64, the addend values are also stored at the relocation offset for relative relocations. This is an optimization: in the case where code does not need to be relocated, the loader can simply skip processing relative relocations. In binutils-2.25, both bfd and gold linkers did this for x86_64, but only the gold linker did this for aarch64. The kernel build here is using the bfd linker, which stored zeroes at the relocation offsets for relative relocations. Since a set of zeroes compresses better than a set of non-zero addend values, this behavior was resulting in much better lz4 compression. The bfd linker in binutils-2.27 is now storing the actual addend values at the relocation offsets. The behavior is now consistent with what it does for x86_64 and what gold linker does for both architectures. The change happened in this upstream commit: https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=1f56df9d0d5ad89806c24e71f296576d82344613 Since a bunch of zeroes got replaced by non-zero addend values, we see the side effect of lz4 compressed image being a bit bigger. To get the old behavior from the bfd linker, "--no-apply-dynamic-relocs" flag can be used: $ LDFLAGS="--no-apply-dynamic-relocs" ./build/build.sh With this flag, the compressed image size is back to what it was with binutils-2.25. If the kernel is using ASLR, there aren't additional runtime costs to --no-apply-dynamic-relocs, as the relocations will need to be applied again anyway after the kernel is relocated to a random address. If the kernel is not using ASLR, then presumably the current default behavior of the linker is better. Since the static linker performed the dynamic relocs, and the kernel is not moved to a different address at load time, it can skip applying the relocations all over again." Some measurements: $ ld -v GNU ld (binutils-2.25-f3d35cf6) 2.25.51.20141117 ^ $ ls -l vmlinux -rwxr-x--- 1 ndesaulniers eng 300652760 Oct 26 11:57 vmlinux $ ls -l Image.lz4-dtb -rw-r----- 1 ndesaulniers eng 16932627 Oct 26 11:57 Image.lz4-dtb $ ld -v GNU ld (binutils-2.27-53dd00a1) 2.27.0.20170315 ^ pre patch: $ ls -l vmlinux -rwxr-x--- 1 ndesaulniers eng 300376208 Oct 26 11:43 vmlinux $ ls -l Image.lz4-dtb -rw-r----- 1 ndesaulniers eng 18159474 Oct 26 11:43 Image.lz4-dtb post patch: $ ls -l vmlinux -rwxr-x--- 1 ndesaulniers eng 300376208 Oct 26 12:06 vmlinux $ ls -l Image.lz4-dtb -rw-r----- 1 ndesaulniers eng 16932466 Oct 26 12:06 Image.lz4-dtb 10ms boot time savings isn't anything to get excited about, but users of arm64+lz4+bfd-2.27 should not have to pay a penalty for no runtime improvement. Reported-by: Gopinath Elanchezhian Reported-by: Sindhuri Pentyala Reported-by: Wei Wang Suggested-by: Rahul Chaudhry Suggested-by: Siqi Lin Suggested-by: Stephen Hines Signed-off-by: Nick Desaulniers --- * Question to reviewers: should I be using $(and ..., ...) here rather than double equals block? grep turns up no hits in the kernel for an example. arch/arm64/Makefile | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 939b310913cf..eed3b8bdc089 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -18,6 +18,12 @@ ifneq ($(CONFIG_RELOCATABLE),) LDFLAGS_vmlinux += -pie -shared -Bsymbolic endif +ifeq ($(CONFIG_KERNEL_LZ4), y) +ifeq ($(CONFIG_RANDOMIZE_BASE), y) +LDFLAGS_vmlinux += $(call ld-option, --no-apply-dynamic-relocs) +endif +endif + ifeq ($(CONFIG_ARM64_ERRATUM_843419),y) ifeq ($(call ld-option, --fix-cortex-a53-843419),) $(warning ld does not support --fix-cortex-a53-843419; kernel may be susceptible to erratum)