From patchwork Fri Jul 26 20:32:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11061805 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4EF696C5 for ; Fri, 26 Jul 2019 20:34:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 407E528B54 for ; Fri, 26 Jul 2019 20:34:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 34D3228B56; Fri, 26 Jul 2019 20:34:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A8AA528B54 for ; Fri, 26 Jul 2019 20:34:23 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hr6tF-0007Rw-0D; Fri, 26 Jul 2019 20:32:33 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hr6tD-0007Rj-9T for xen-devel@lists.xenproject.org; Fri, 26 Jul 2019 20:32:31 +0000 X-Inumbo-ID: 7bbca7d2-afe4-11e9-b547-5fdf0de33ccb Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 7bbca7d2-afe4-11e9-b547-5fdf0de33ccb; Fri, 26 Jul 2019 20:32:28 +0000 (UTC) Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: 6AisI96sse7n/WtIbi8RBGmQJ1IxUoiA/SyvQ3Kea1TYTS9u+R81hOSTQnW64LGO+HSzhf9yXO JNAFdKvzrt1vdycXW10fou28sqfwGN8FbT3o4+jdo4y9mmSBEzkq17nd68Z4B90ZcN6TfwPKnw LSn2h41/g/csT5VtHOTX7fZc+EGg2jstRhIQBcE0nbAgoXmI5DAYkgkdp35vjmRzfBbMUE3kFa HjWskFVNLT4pZupPLY/bBgtG/uw7PqpfHb354dK+j0LRMp6W7cHkYFPLB7hDF/IM0L30Oodrmk 9Og= X-SBRS: 2.7 X-MesageID: 3595793 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.64,312,1559534400"; d="scan'208";a="3595793" From: Andrew Cooper To: Xen-devel Date: Fri, 26 Jul 2019 21:32:21 +0100 Message-ID: <20190726203222.4833-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190726203222.4833-1-andrew.cooper3@citrix.com> References: <20190726203222.4833-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v2 1/2] xen/link: Introduce .bss.percpu.page_aligned X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Wei Liu , Andrew Cooper , Julien Grall , Jan Beulich , Volodymyr Babchuk , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Future changes are going to need to page align some percpu data. This means that the percpu area needs suitably aligning in the BSS so CPU0 has correctly aligned data. Shuffle the exact link order of items within the BSS to give .bss.percpu.page_aligned appropriate alignment. In addition, we need to be able to specify an alignment attribute to __DEFINE_PER_CPU(). Rework it so the caller passes in all attributes, and adjust DEFINE_PER_CPU{,_READ_MOSTLY}() to match. This has the added bonus that it is now possible to grep for .bss.percpu and find all the users. Finally, introduce DEFINE_PER_CPU_PAGE_ALIGNED() which uses both section and alignment attributes. Signed-off-by: Andrew Cooper Acked-by: Julien Grall Reviewed-by: Roger Pau Monné --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné CC: Stefano Stabellini CC: Julien Grall CC: Volodymyr Babchuk v2: * Rework __DEFINE_PER_CPU() to allow for further attributes to be passed. * Specify __aligned(PAGE_SIZE) as part of DEFINE_PER_CPU_PAGE_ALIGNED(). --- xen/arch/arm/xen.lds.S | 5 +++-- xen/arch/x86/xen.lds.S | 5 +++-- xen/include/asm-arm/percpu.h | 6 ++---- xen/include/asm-x86/percpu.h | 6 ++---- xen/include/xen/percpu.h | 10 ++++++++-- 5 files changed, 18 insertions(+), 14 deletions(-) diff --git a/xen/arch/arm/xen.lds.S b/xen/arch/arm/xen.lds.S index 12c107f45d..07cbdf2543 100644 --- a/xen/arch/arm/xen.lds.S +++ b/xen/arch/arm/xen.lds.S @@ -201,14 +201,15 @@ SECTIONS *(.bss.stack_aligned) . = ALIGN(PAGE_SIZE); *(.bss.page_aligned) - *(.bss) - . = ALIGN(SMP_CACHE_BYTES); __per_cpu_start = .; + *(.bss.percpu.page_aligned) *(.bss.percpu) . = ALIGN(SMP_CACHE_BYTES); *(.bss.percpu.read_mostly) . = ALIGN(SMP_CACHE_BYTES); __per_cpu_data_end = .; + *(.bss) + . = ALIGN(SMP_CACHE_BYTES); __bss_end = .; } :text _end = . ; diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S index a73139cd29..b8a2ea4259 100644 --- a/xen/arch/x86/xen.lds.S +++ b/xen/arch/x86/xen.lds.S @@ -293,14 +293,15 @@ SECTIONS __bss_start = .; *(.bss.stack_aligned) *(.bss.page_aligned*) - *(.bss) - . = ALIGN(SMP_CACHE_BYTES); __per_cpu_start = .; + *(.bss.percpu.page_aligned) *(.bss.percpu) . = ALIGN(SMP_CACHE_BYTES); *(.bss.percpu.read_mostly) . = ALIGN(SMP_CACHE_BYTES); __per_cpu_data_end = .; + *(.bss) + . = ALIGN(SMP_CACHE_BYTES); __bss_end = .; } :text _end = . ; diff --git a/xen/include/asm-arm/percpu.h b/xen/include/asm-arm/percpu.h index 9584b830d4..264120b192 100644 --- a/xen/include/asm-arm/percpu.h +++ b/xen/include/asm-arm/percpu.h @@ -10,10 +10,8 @@ extern char __per_cpu_start[], __per_cpu_data_end[]; extern unsigned long __per_cpu_offset[NR_CPUS]; void percpu_init_areas(void); -/* Separate out the type, so (int[3], foo) works. */ -#define __DEFINE_PER_CPU(type, name, suffix) \ - __section(".bss.percpu" #suffix) \ - __typeof__(type) per_cpu_##name +#define __DEFINE_PER_CPU(attr, type, name) \ + attr __typeof__(type) per_cpu_ ## name #define per_cpu(var, cpu) \ (*RELOC_HIDE(&per_cpu__##var, __per_cpu_offset[cpu])) diff --git a/xen/include/asm-x86/percpu.h b/xen/include/asm-x86/percpu.h index ff34dc7897..5b6cef04c4 100644 --- a/xen/include/asm-x86/percpu.h +++ b/xen/include/asm-x86/percpu.h @@ -7,10 +7,8 @@ extern unsigned long __per_cpu_offset[NR_CPUS]; void percpu_init_areas(void); #endif -/* Separate out the type, so (int[3], foo) works. */ -#define __DEFINE_PER_CPU(type, name, suffix) \ - __section(".bss.percpu" #suffix) \ - __typeof__(type) per_cpu_##name +#define __DEFINE_PER_CPU(attr, type, name) \ + attr __typeof__(type) per_cpu_ ## name /* var is in discarded region: offset to particular copy we want */ #define per_cpu(var, cpu) \ diff --git a/xen/include/xen/percpu.h b/xen/include/xen/percpu.h index aeec5c19d6..71a31cc361 100644 --- a/xen/include/xen/percpu.h +++ b/xen/include/xen/percpu.h @@ -9,9 +9,15 @@ * The _##name concatenation is being used here to prevent 'name' from getting * macro expanded, while still allowing a per-architecture symbol name prefix. */ -#define DEFINE_PER_CPU(type, name) __DEFINE_PER_CPU(type, _##name, ) +#define DEFINE_PER_CPU(type, name) \ + __DEFINE_PER_CPU(__section(".bss.percpu"), type, _ ## name) + +#define DEFINE_PER_CPU_PAGE_ALIGNED(type, name) \ + __DEFINE_PER_CPU(__section(".bss.percpu.page_aligned") \ + __aligned(PAGE_SIZE), type, _ ## name) + #define DEFINE_PER_CPU_READ_MOSTLY(type, name) \ - __DEFINE_PER_CPU(type, _##name, .read_mostly) + __DEFINE_PER_CPU(__section(".bss.percpu.read_mostly"), type, _ ## name) #define get_per_cpu_var(var) (per_cpu__##var) From patchwork Fri Jul 26 20:32:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11061803 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AC5876C5 for ; Fri, 26 Jul 2019 20:34:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9D07328B54 for ; Fri, 26 Jul 2019 20:34:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 907A628B56; Fri, 26 Jul 2019 20:34:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0DA7428B54 for ; Fri, 26 Jul 2019 20:34:21 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hr6tI-0007Sy-MV; Fri, 26 Jul 2019 20:32:36 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hr6tH-0007SW-HC for xen-devel@lists.xenproject.org; Fri, 26 Jul 2019 20:32:35 +0000 X-Inumbo-ID: 7e19cbc2-afe4-11e9-8917-c7450cf3864c Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 7e19cbc2-afe4-11e9-8917-c7450cf3864c; Fri, 26 Jul 2019 20:32:31 +0000 (UTC) Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: PVt+kn7h/xaBMPM/m/+Ceu4Jtz8PaUwqv88cYkgqzdoKuiH6vuKJH7dLvzewTR1c7puLNC7Th5 PbZCoQfZLfW6SK9eEMaar3gdIJuifCVGXPvZzI+aee0fu9Hewi98r78F7xvgw5rJ0R8N/P60gM WUX8lKaailVDLVFmb4yMiEaVFRDSPFyspYNjeZVrNA38NK4KJvGmD5pZg7OQO7m+/MJxHye1RM l//fA9VsoSGUZ+yvOKRfugOZr/I0Eq3wsYSFOyj1O/fS2V6QSy+ts4kMTVXDDrhFKkOTKTm+y/ w6k= X-SBRS: 2.7 X-MesageID: 3595794 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.64,312,1559534400"; d="scan'208";a="3595794" From: Andrew Cooper To: Xen-devel Date: Fri, 26 Jul 2019 21:32:22 +0100 Message-ID: <20190726203222.4833-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190726203222.4833-1-andrew.cooper3@citrix.com> References: <20190726203222.4833-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v2 2/2] x86/xpti: Don't leak TSS-adjacent percpu data via Meltdown X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP The XPTI work restricted the visibility of most of memory, but missed a few aspects when it came to the TSS. Given that the TSS is just an object in percpu data, the 4k mapping for it created in setup_cpu_root_pgt() maps adjacent percpu data, making it all leakable via Meltdown, even when XPTI is in use. Furthermore, no care is taken to check that the TSS doesn't cross a page boundary. As it turns out, struct tss_struct is aligned on its size which does prevent it straddling a page boundary, but this will cease to be true once CET and Shadow Stack support is added to Xen. Move the TSS into the page aligned percpu area, so no adjacent data can be leaked. Move the definition from setup.c to traps.c, which is a more appropriate place for it to live. Signed-off-by: Andrew Cooper Reviewed-by: Roger Pau Monné --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné v2: * Rebase over changes to include __aligned() within DEFINE_PER_CPU_PAGE_ALIGNED() * Drop now-unused xen/percpu.h from setup.c --- xen/arch/x86/setup.c | 3 --- xen/arch/x86/traps.c | 6 ++++++ xen/arch/x86/xen.lds.S | 2 ++ xen/include/asm-x86/processor.h | 4 ++-- 4 files changed, 10 insertions(+), 5 deletions(-) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index d2011910fa..f9d38155d3 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -16,7 +16,6 @@ #include #include #include -#include #include #include #include @@ -100,8 +99,6 @@ unsigned long __read_mostly xen_phys_start; unsigned long __read_mostly xen_virt_end; -DEFINE_PER_CPU(struct tss_struct, init_tss); - char __section(".bss.stack_aligned") __aligned(STACK_SIZE) cpu0_stack[STACK_SIZE]; diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c index 38d12013db..de3ac135f5 100644 --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -108,6 +108,12 @@ idt_entry_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) /* Pointer to the IDT of every CPU. */ idt_entry_t *idt_tables[NR_CPUS] __read_mostly; +/* + * The TSS is smaller than a page, but we give it a full page to avoid + * adjacent per-cpu data leaking via Meltdown when XPTI is in use. + */ +DEFINE_PER_CPU_PAGE_ALIGNED(struct tss_struct, init_tss); + bool (*ioemul_handle_quirk)( u8 opcode, char *io_emul_stub, struct cpu_user_regs *regs); diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S index b8a2ea4259..c82e1e504a 100644 --- a/xen/arch/x86/xen.lds.S +++ b/xen/arch/x86/xen.lds.S @@ -368,6 +368,8 @@ ASSERT(IS_ALIGNED(__2M_rwdata_end, SECTION_ALIGN), "__2M_rwdata_end misaligned ASSERT(IS_ALIGNED(cpu0_stack, STACK_SIZE), "cpu0_stack misaligned") +ASSERT(IS_ALIGNED(per_cpu__init_tss, PAGE_SIZE), "per_cpu(init_tss) misaligned") + ASSERT(IS_ALIGNED(__init_begin, PAGE_SIZE), "__init_begin misaligned") ASSERT(IS_ALIGNED(__init_end, PAGE_SIZE), "__init_end misaligned") diff --git a/xen/include/asm-x86/processor.h b/xen/include/asm-x86/processor.h index 2862321eee..b5bee94931 100644 --- a/xen/include/asm-x86/processor.h +++ b/xen/include/asm-x86/processor.h @@ -411,7 +411,7 @@ static always_inline void __mwait(unsigned long eax, unsigned long ecx) #define IOBMP_BYTES 8192 #define IOBMP_INVALID_OFFSET 0x8000 -struct __packed __cacheline_aligned tss_struct { +struct __packed tss_struct { uint32_t :32; uint64_t rsp0, rsp1, rsp2; uint64_t :64; @@ -425,6 +425,7 @@ struct __packed __cacheline_aligned tss_struct { /* Pads the TSS to be cacheline-aligned (total size is 0x80). */ uint8_t __cacheline_filler[24]; }; +DECLARE_PER_CPU(struct tss_struct, init_tss); #define IST_NONE 0UL #define IST_DF 1UL @@ -463,7 +464,6 @@ static inline void disable_each_ist(idt_entry_t *idt) extern idt_entry_t idt_table[]; extern idt_entry_t *idt_tables[]; -DECLARE_PER_CPU(struct tss_struct, init_tss); DECLARE_PER_CPU(root_pgentry_t *, root_pgt); extern void write_ptbase(struct vcpu *v);