From patchwork Thu Oct 26 21:43:56 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Desaulniers X-Patchwork-Id: 10028799 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 59BBA601E8 for ; Thu, 26 Oct 2017 21:45:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4B5E628606 for ; Thu, 26 Oct 2017 21:45:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 499AF28EF1; Thu, 26 Oct 2017 21:45:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, DKIM_VALID, RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 96E4428606 for ; Thu, 26 Oct 2017 21:45:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ugJOMnHz9J6yBmjEwVhpIsHDt8RDQckX1K8077OwmTI=; b=lYLn106VvGSfNntIo1XBVQCWzT 4KqgoIwhKFFPSuyEiu2lJyVtSCENjzthClRSHkDGpGop3ny0oXfvnDEd3AtoBIFlqEf75oRukj++X Pgs+PnMZBi4GqdGiS3v8VG8E88q9JaRIpxp4onARFNDLq1UUb731G+rbJU1XywAwBRx/lQiLjjuJU 1dtzGKwnNduSA8blgkehDk6iwCSISFoIipbUst1xbeygNctAT2vCu7joCNf84XFwN6D5d7HumTXVe U41Y7t246tLj/PAKPM9ApZm8Gw5+HlaI/KJE4b0cHS4Y+85KkxMJ9ZxDjBWJOD7Mg0Ut5t6P77N1I I16HMFtA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1e7pxb-0005vi-79; Thu, 26 Oct 2017 21:45:07 +0000 Received: from mail-io0-x243.google.com ([2607:f8b0:4001:c06::243]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1e7pxB-0005tV-GK for linux-arm-kernel@lists.infradead.org; Thu, 26 Oct 2017 21:44:43 +0000 Received: by mail-io0-x243.google.com with SMTP id 101so8575383ioj.3 for ; Thu, 26 Oct 2017 14:44:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dhNRZ39WZtOXbwFT7MBo3f1x3IJzvbKFs0rgwDpV6ZA=; b=RufxpueA6h/h/1g539/KPs6gBbgOy4J6Zy53ddB9MuvcPKwVPHOS0SWmAwtAbNYXxF QNpw8LNDZBsBhNzoz1Shhd44uJhJXqk/9hxIwliZApTNuCWxfFzD0GehsN0eBAIVg3nS ZXVjSVpUBrocvYnXN9KASGUxkYBSj4nNNCrmOjFUYqMEH6W8A3B2f9CQpA6AIbgNhsqm DByQaiWho3IaqLWA+wZh8ZU+UmwcO39BeFmxJpzK99PPnNiiWQ56ur3WBJ55V+B8INb/ +vXqD9vgr+5dKann5h5TckJAMCzsZoisIPHIKYcsA4B7SNMXIzZFAyOHNPoBhTP8QSnF imcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dhNRZ39WZtOXbwFT7MBo3f1x3IJzvbKFs0rgwDpV6ZA=; b=mYlhiq0oTSHITFUdymEFUUiVTeOOVV0BRKrhPQA7DIQL4B6KTYAN86qafIjA5M9N0j TPsJTG9xwcvHUmqSuy/03Q+u2UXizBcrGG0MoRYXo2SAAyLnV8eB7aFKEDARD4VI3NP5 IJWZsvYqQ34Y6McLIOAEkJ4ip7crSUgNAsjuK6zN9l/qjupRg7FJC3FaDi3o896cQ07P QneLrT0hURCElbX56+xXeY8QXz+SdMz7E24JaOlyUY93ctrXeoK0t3Be/K9wPVDpVA6S rptBVR+zAl53OqUW4+8FtrEQYOY25oTA4/ei18vtxLGHYwJp35VNDf4aThi6n/DnEZig woFQ== X-Gm-Message-State: AMCzsaX5szINJTf/Z+FP8zRe7A9eG7Y0xsvqu/27BdQUVSTGVP7DNfcy vqRhM+AOEv0jUjf9c22AOIE1kg== X-Google-Smtp-Source: ABhQp+QFGbFu8oZW0zSTOchh3kjJYyK8Lp6/EtioBlBSzLBHFk63Kps84KCiTmsPVgednYKkHz1EhQ== X-Received: by 10.36.77.198 with SMTP id l189mr530699itb.116.1509054257512; Thu, 26 Oct 2017 14:44:17 -0700 (PDT) Received: from ndesaulniers0.mtv.corp.google.com ([100.98.120.97]) by smtp.gmail.com with ESMTPSA id 137sm122327itb.42.2017.10.26.14.44.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 26 Oct 2017 14:44:16 -0700 (PDT) From: Nick Desaulniers To: Ard Biesheuvel Subject: [PATCH v2] arm64: prevent regressions in compressed kernel image size when upgrading to binutils 2.27 Date: Thu, 26 Oct 2017 14:43:56 -0700 Message-Id: <20171026214356.26963-1-ndesaulniers@google.com> X-Mailer: git-send-email 2.15.0.rc2.357.g7e34df9404-goog In-Reply-To: References: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20171026_144441_593183_A1CFA8CE X-CRM114-Status: GOOD ( 16.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jiri Kosina , Kees Cook , Rahul Chaudhry , Catalin Marinas , Gopinath Elanchezhian , Nick Desaulniers , Sindhuri Pentyala , Will Deacon , linux-kernel@vger.kernel.org, Stephen Hines , Wei Wang , linux-arm-kernel@lists.infradead.org, Siqi Lin MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Upon upgrading to binutils 2.27, we found that our lz4 compressed kernel images were significantly larger, resulting is 10ms boot time regressions. As noted by Rahul: "aarch64 binaries uses RELA relocations, where each relocation entry includes an addend value. This is similar to x86_64. On x86_64, the addend values are also stored at the relocation offset for relative relocations. This is an optimization: in the case where code does not need to be relocated, the loader can simply skip processing relative relocations. In binutils-2.25, both bfd and gold linkers did this for x86_64, but only the gold linker did this for aarch64. The kernel build here is using the bfd linker, which stored zeroes at the relocation offsets for relative relocations. Since a set of zeroes compresses better than a set of non-zero addend values, this behavior was resulting in much better lz4 compression. The bfd linker in binutils-2.27 is now storing the actual addend values at the relocation offsets. The behavior is now consistent with what it does for x86_64 and what gold linker does for both architectures. The change happened in this upstream commit: https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=1f56df9d0d5ad89806c24e71f296576d82344613 Since a bunch of zeroes got replaced by non-zero addend values, we see the side effect of lz4 compressed image being a bit bigger. To get the old behavior from the bfd linker, "--no-apply-dynamic-relocs" flag can be used: $ LDFLAGS="--no-apply-dynamic-relocs" ./build/build.sh With this flag, the compressed image size is back to what it was with binutils-2.25. If the kernel is using ASLR, there aren't additional runtime costs to --no-apply-dynamic-relocs, as the relocations will need to be applied again anyway after the kernel is relocated to a random address. If the kernel is not using ASLR, then presumably the current default behavior of the linker is better. Since the static linker performed the dynamic relocs, and the kernel is not moved to a different address at load time, it can skip applying the relocations all over again." Some measurements: $ ld -v GNU ld (binutils-2.25-f3d35cf6) 2.25.51.20141117 ^ $ ls -l vmlinux -rwxr-x--- 1 ndesaulniers eng 300652760 Oct 26 11:57 vmlinux $ ls -l Image.lz4-dtb -rw-r----- 1 ndesaulniers eng 16932627 Oct 26 11:57 Image.lz4-dtb $ ld -v GNU ld (binutils-2.27-53dd00a1) 2.27.0.20170315 ^ pre patch: $ ls -l vmlinux -rwxr-x--- 1 ndesaulniers eng 300376208 Oct 26 11:43 vmlinux $ ls -l Image.lz4-dtb -rw-r----- 1 ndesaulniers eng 18159474 Oct 26 11:43 Image.lz4-dtb post patch: $ ls -l vmlinux -rwxr-x--- 1 ndesaulniers eng 300376208 Oct 26 12:06 vmlinux $ ls -l Image.lz4-dtb -rw-r----- 1 ndesaulniers eng 16932466 Oct 26 12:06 Image.lz4-dtb We've also verified gzip improves by 0.69 MB. Any compression scheme should be able to get better results from the longer runs of zeros, not just GZIP and LZ4. 10ms boot time savings isn't anything to get excited about, but users of arm64+compression+bfd-2.27 should not have to pay a boot time penalty for no runtime improvement. Reported-by: Gopinath Elanchezhian Reported-by: Sindhuri Pentyala Reported-by: Wei Wang Suggested-by: Ard Biesheuvel Suggested-by: Rahul Chaudhry Suggested-by: Siqi Lin Suggested-by: Stephen Hines Signed-off-by: Nick Desaulniers Reviewed-by: Ard Biesheuvel --- Changes since v1: * dropped LZ4 only outer conditional, per Ard. * changed inner conditional for all RELOCATABLE, not just RANDOMIZE_BASE, per Ard. * updated commit message with findings for gzip. * added Ard to suggested by line in commit message. arch/arm64/Makefile | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 939b310913cf..9f47d4276a21 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -18,6 +18,10 @@ ifneq ($(CONFIG_RELOCATABLE),) LDFLAGS_vmlinux += -pie -shared -Bsymbolic endif +ifeq ($(CONFIG_RELOCATABLE), y) +LDFLAGS_vmlinux += $(call ld-option, --no-apply-dynamic-relocs) +endif + ifeq ($(CONFIG_ARM64_ERRATUM_843419),y) ifeq ($(call ld-option, --fix-cortex-a53-843419),) $(warning ld does not support --fix-cortex-a53-843419; kernel may be susceptible to erratum)