From patchwork Tue Sep 12 00:37:14 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konrad Rzeszutek Wilk X-Patchwork-Id: 9948237 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E473D603FB for ; Tue, 12 Sep 2017 00:41:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D63C928D33 for ; Tue, 12 Sep 2017 00:41:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CA98A28D5E; Tue, 12 Sep 2017 00:41:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,RCVD_IN_SORBS_SPAM,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2301028D5D for ; Tue, 12 Sep 2017 00:41:05 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1drZDA-0000ec-3y; Tue, 12 Sep 2017 00:37:56 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1drZD8-0000dj-Uo for xen-devel@lists.xenproject.org; Tue, 12 Sep 2017 00:37:55 +0000 Received: from [85.158.137.68] by server-7.bemta-3.messagelabs.com id 55/BE-02224-26C27B95; Tue, 12 Sep 2017 00:37:54 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrMIsWRWlGSWpSXmKPExsVyMfTOYd1Ene2 RBg03xCy+b5nM5MDocfjDFZYAxijWzLyk/IoE1ozvF10KNhpWPOw6wdbA+FSti5GLQ0hgBqPE s97Z7CAOi8AHFomDExqYuhg5OSQEprFK3LqfC2HHSTxsO80KYadJ7P+9GsqukjhwbTFYvZCAk sSWyY8ZIaYeZJK48W42I0hCWEBPYvK322A2m4C+xNO115i7GDmAmt0kPl3jAqkXEWhnlJh95R fYUGYBQ4nWt0fZQGqEBbIlvi/xAQmzCKhKXPj7CayEV8BS4tmxCWwQN8hLTOydBjaeU8BK4ve PeawQ91hKdCy4wjSBUXgBI8MqRvXi1KKy1CJdS72kosz0jJLcxMwcXUMDY73c1OLixPTUnMSk Yr3k/NxNjMDgrGdgYNzB+Pqn0yFGSQ4mJVHe38LbI4X4kvJTKjMSizPii0pzUosPMcpwcChJ8 L7QAsoJFqWmp1akZeYA4wQmLcHBoyTCmwWS5i0uSMwtzkyHSJ1itOe4cOfSHyaOA3tuAcmOm3 eB5JRbQFKIJS8/L1VKnPc3SJsASFtGaR7cUFhcX2KUlRLmZWRgYBDiKUgtys0sQZV/xSjOwag kzKuuDTSFJzOvBG73K6CzmIDO4rm0BeSskkSElFQD4wqGtpoHa5nZQipbJj9ebc0QunKtceJD 6XnHVOI6rxTlhtv5hc6RL/FNkD0e9iHymsWazABT91CGFlvzlsQL32Y7TC9vq2GN5tf9Wh0qU SHW4rCizOxxVH49c5ZY3m0lubQFRgl7Lhl0+/Ecr8i18K66KbZ03lq1bTLslx70h7SHee06NV WJpTgj0VCLuag4EQDmUeIQ5gIAAA== X-Env-Sender: ketuzsezr@gmail.com X-Msg-Ref: server-2.tower-31.messagelabs.com!1505176672!102596291!1 X-Originating-IP: [209.85.220.195] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.4.45; banners=-,-,- X-VirusChecked: Checked Received: (qmail 53805 invoked from network); 12 Sep 2017 00:37:53 -0000 Received: from mail-qk0-f195.google.com (HELO mail-qk0-f195.google.com) (209.85.220.195) by server-2.tower-31.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 12 Sep 2017 00:37:53 -0000 Received: by mail-qk0-f195.google.com with SMTP id i14so6376249qke.3 for ; Mon, 11 Sep 2017 17:37:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=Hs3YqurOLsPMLVoeQocjxqHzEJbq9/alBsPQnn/h2O8=; b=hkOzRak775qvPGXxiNCq7CVjlDREebnT+Nt/moIR6c3iPrLOSXYrNLw9MBE8vLytwy 5GbnRsC3VYpu6q3Xb2p1adwdfdLrE+XObdd0qjKxgy2CryXPL71eMETrvtRp2j3kH2O3 9ZBcRg8vpumNgoA58dfQz4lct3rAadAERlDlq5Ce6+rB3d4Jf2gCwWZT1lNmcciCqDDT JjWcurZ6DCmubLo1cbfGGlEDNCvAVqZMAOlCLiyOpvKV1h7lGB8/tzJqU5+qyAKbMw7D 7IC8CTJgzUeQesJOzvY/bjfFP0whTQV+PD/CpwL4z3jqMP75wOH1/0dSpvoK8TXxOVMz A8lQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=Hs3YqurOLsPMLVoeQocjxqHzEJbq9/alBsPQnn/h2O8=; b=al4HkJEnRhoeLAgkZ7LzLQpaQO1sH4F3UkZAvzrKSioN+t96p3fegLrQKgWv8eGiDD VUliI2TO9kJow5zxbgkZcM5JqLa/saWe++G2mFGeCeJ7p43w9dSGzWwK0FaQU1qvHb1+ ddMsQj2Sd4wgAwZbvszepzMZnQa+5u9PsqSBB1t2KkUm6FN4SXgCOonM+pYcBIJ5iLw9 1L3X7MFpX9Wavt5zJg7EwcR2KxMVSXOI8lYKeB3VLSr2Jhk328kHlS7h1ePPzdyQWG9U 5ej5EgS16IkvZtny6nRnyE6OFDi6lGm+I1KpJP6bTENboVx0VyiE2zQcaQTJn0kKGHP0 088A== X-Gm-Message-State: AHPjjUiTn6sAuMzEBQTS4s3+iXGH0khE8iX5NxQd2YrbOrZAXCw/Sozi 7F/yW5Ak3NqrA1mP X-Google-Smtp-Source: AOwi7QBIR80FYRAAl8DQSVZXLZPD0kTgY9ngyoEIoVpkyTdgiDrUYmTqtqNpvliA4aO9TRxqJ2QSaw== X-Received: by 10.233.244.72 with SMTP id z8mr18051220qkl.186.1505176671672; Mon, 11 Sep 2017 17:37:51 -0700 (PDT) Received: from localhost.localdomain (209-6-200-48.s4398.c3-0.smr-ubr2.sbo-smr.ma.cable.rcncustomer.com. [209.6.200.48]) by smtp.gmail.com with ESMTPSA id z75sm6771034qkb.71.2017.09.11.17.37.50 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 11 Sep 2017 17:37:50 -0700 (PDT) From: Konrad Rzeszutek Wilk X-Google-Original-From: Konrad Rzeszutek Wilk To: xen-devel@lists.xenproject.org, ross.lagerwall@citrix.com, konrad.wilk@oracle.com, julien.grall@arm.com, sstabellini@kernel.org Date: Mon, 11 Sep 2017 20:37:14 -0400 Message-Id: <20170912003726.368-6-konrad.wilk@oracle.com> X-Mailer: git-send-email 2.13.3 In-Reply-To: <20170912003726.368-1-konrad.wilk@oracle.com> References: <20170912003726.368-1-konrad.wilk@oracle.com> Cc: andrew.cooper3@citrix.com, jbeulich@suse.com Subject: [Xen-devel] [PATCH v3 05/17] alternative/x86/arm32: Align altinstructions (and altinstr_replacement) sections. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This is very similar to 137c59b9ff3f7a214f03b52d9c00a0a02374af1f "bug/x86/arm: Align bug_frames sections." On ARM and on x86 the C and assembler macros don't include any alignment information - hence they end up being the default byte granularity. On ARM32 it is paramount that the alignment is word-size (4) otherwise if one tries to use (uint32_t*) access (such as livepatch ELF relocations) we get a Data Abort. Specifically this issue was observed on ARM32 with a cross compiler for the livepatches. Mainly the livepatches .data section size was not padded to the section alignment: ARM32 native: Contents of section .rodata: 0000 68695f66 756e6300 63686563 6b5f666e hi_func.check_fn 0010 63000000 78656e5f 65787472 615f7665 c...xen_extra_ve 0020 7273696f 6e000000 rsion... ARM32 cross compiler: Contents of section .rodata: 0000 68695f66 756e6300 63686563 6b5f666e hi_func.check_fn 0010 63000000 78656e5f 65787472 615f7665 c...xen_extra_ve 0020 7273696f 6e00 rsion. And when we loaded it the next section would be put right behind it: native: (XEN) livepatch.c:413: livepatch: xen_hello_world: Loaded .text at 00a02000 (XEN) livepatch.c:413: livepatch: xen_hello_world: Loaded .rodata at 00a04024 (XEN) livepatch.c:413: livepatch: xen_hello_world: Loaded .altinstructions at 00a0404c cross compiler: (XEN) livepatch.c:413: livepatch: xen_hello_world: Loaded .text at 00a02000 (XEN) livepatch.c:413: livepatch: xen_hello_world: Loaded .rodata at 00a04024 (XEN) livepatch.c:413: livepatch: xen_hello_world: Loaded .altinstructions at 00a0404a (See 4a vs 4c) native readelf: [ 4] .rodata PROGBITS 00000000 000164 000028 00 A 0 0 4 [ 5] .altinstructions PROGBITS 00000000 00018c 00000c 00 A 0 0 1 cross compiler readelf --sections: [ 4] .rodata PROGBITS 00000000 000164 000026 00 A 0 0 4 [ 5] .altinstructions PROGBITS 00000000 00018a 00000c 00 A 0 0 1 And as can be seen the .altinstructions have alignment of 1 which from 'man elf' is: "Values of zero and one mean no alignment is required." which means we can ignore it. Enforcing .altinstructions (and also .altinstr_replacement for completness on ARM) to have the proper alignment across all architectures and in both C and x86 makes them all the same. On x86 the bloat-o-meter detects that with this change the file shrinks: add/remove: 1/0 grow/shrink: 0/2 up/down: 156/-367 (-211) function old new delta get_page_from_gfn - 156 +156 do_mmu_update 4578 4569 -9 do_mmuext_op 5604 5246 -358 Total: Before=3170439, After=3170228, chg -0.01% But as found adding even "#Hi!\n" will casue this optimization, so the bloat-o-meter value here is useless. While on ARM 32/64: add/remove: 0/0 grow/shrink: 0/0 up/down: 0/0 (0) function old new delta Total: Before=822563, After=822563, chg +0.00% Also since the macros have the alignment coded in them there is no need to do that for the xen.lds.S anymore. Signed-off-by: Konrad Rzeszutek Wilk --- v2: - First version v3: - Figured out the x86 bloat-o-meter results. - Removed the .ALIGN from xen.lds.S - Removed the .p2align on .altinstr_replacement per Jan's request. - Put most of the commit description for the original issue --- xen/arch/arm/xen.lds.S | 2 -- xen/arch/x86/xen.lds.S | 1 - xen/include/asm-arm/alternative.h | 4 ++++ xen/include/asm-x86/alternative.h | 1 + 4 files changed, 5 insertions(+), 3 deletions(-) diff --git a/xen/arch/arm/xen.lds.S b/xen/arch/arm/xen.lds.S index c9b9546435..447d33888f 100644 --- a/xen/arch/arm/xen.lds.S +++ b/xen/arch/arm/xen.lds.S @@ -155,11 +155,9 @@ SECTIONS __initcall_end = .; #ifdef CONFIG_HAS_ALTERNATIVE - . = ALIGN(4); __alt_instructions = .; *(.altinstructions) __alt_instructions_end = .; - . = ALIGN(4); *(.altinstr_replacement) #endif diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S index d5e8821d41..9eb42048d5 100644 --- a/xen/arch/x86/xen.lds.S +++ b/xen/arch/x86/xen.lds.S @@ -202,7 +202,6 @@ SECTIONS * "Alternative instructions for different CPU types or capabilities" * Think locking instructions on spinlocks. */ - . = ALIGN(8); __alt_instructions = .; *(.altinstructions) __alt_instructions_end = .; diff --git a/xen/include/asm-arm/alternative.h b/xen/include/asm-arm/alternative.h index 6cc9d0dc5f..cd1373fdd5 100644 --- a/xen/include/asm-arm/alternative.h +++ b/xen/include/asm-arm/alternative.h @@ -54,9 +54,11 @@ int apply_alternatives(const struct alt_instr *start, const struct alt_instr *en oldinstr "\n" \ "662:\n" \ ".pushsection .altinstructions,\"a\"\n" \ + ".p2align 2\n" \ ALTINSTR_ENTRY(feature) \ ".popsection\n" \ ".pushsection .altinstr_replacement, \"a\"\n" \ + ".p2align 2\n" \ "663:\n\t" \ newinstr "\n" \ "664:\n\t" \ @@ -84,6 +86,7 @@ int apply_alternatives(const struct alt_instr *start, const struct alt_instr *en .if \enable 661: \insn1 662: .pushsection .altinstructions, "a" + .p2align 2 altinstruction_entry 661b, 663f, \cap, 662b-661b, 664f-663f .popsection .pushsection .altinstr_replacement, "ax" @@ -103,6 +106,7 @@ int apply_alternatives(const struct alt_instr *start, const struct alt_instr *en .macro alternative_if_not cap, enable = 1 .if \enable .pushsection .altinstructions, "a" + .p2align 2 altinstruction_entry 661f, 663f, \cap, 662f-661f, 664f-663f .popsection 661: diff --git a/xen/include/asm-x86/alternative.h b/xen/include/asm-x86/alternative.h index db4f08e0e7..79430fdf05 100644 --- a/xen/include/asm-x86/alternative.h +++ b/xen/include/asm-x86/alternative.h @@ -56,6 +56,7 @@ extern void alternative_instructions(void); #define ALTERNATIVE_N(newinstr, feature, number) \ ".pushsection .altinstructions,\"a\"\n" \ + ".p2align 2\n" \ ALTINSTR_ENTRY(feature, number) \ ".section .discard,\"a\",@progbits\n" \ DISCARD_ENTRY(number) \