From patchwork Fri May 1 22:58:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523395 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 64482139A for ; Fri, 1 May 2020 23:00:50 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3D472216FD for ; Fri, 1 May 2020 23:00:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="HRQXCw/9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3D472216FD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUecn-0007zf-7K; Fri, 01 May 2020 22:59:17 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUecm-0007z2-4R for xen-devel@lists.xenproject.org; Fri, 01 May 2020 22:59:16 +0000 X-Inumbo-ID: 5ce7dcf8-8bff-11ea-b9cf-bc764e2007e4 Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 5ce7dcf8-8bff-11ea-b9cf-bc764e2007e4; Fri, 01 May 2020 22:59:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588373948; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KmdiFWTUtoRQ6SF0OGgDhEv3ORD8KJHYv15kmE23ReQ=; b=HRQXCw/9UuQPkIuaP98GPgTkF0YBdIab2DH4VEdhwH9i4faNXdsIGgyF NkuISlKUyrxyGN9IzXFuPbnIc1bbYnMwE8ZdfdYkmJnQZiuPDfTBcOw0Z +QafbUqMWxnEjyYATrkQJc3PHsKTd16Qy/wX0vdgZSwkAvzdJKHjnci9k c=; Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa4.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: 7WuTT0jWBm09Mc/nbOEE+auc69OKlH3GWOmbKE2uVAaOnAmqnkYT90HxHA3xmAPWNI+BMHFQ2r QHw2wS8jFcFSMJk6tCSvXzMJibya4VQtH2wTwGkhBdEFfWkDG6fKCGbYlu5ntbh7Dsr5JO9xLN Fer8dsWTpxRAq4tmk0Yp0XzXCokf/n7WlYO6HQGVD0hhksc5wAJxM2nddEnnSLjZW9vBsgpxs6 zEEL36xfT+/7Vf5tWmR5HD6wPpCbAhCWLB/LFoVBCjAczqFaoycmS3subZlpqx8D2WiXLueYZb klg= X-SBRS: 2.7 X-MesageID: 17293962 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="17293962" From: Andrew Cooper To: Xen-devel Subject: [PATCH 01/16] x86/traps: Drop last_extable_addr Date: Fri, 1 May 2020 23:58:23 +0100 Message-ID: <20200501225838.9866-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The only user of this facility is dom_crash_sync_extable() by passing 0 into asm_domain_crash_synchronous(). The common error cases are already covered with show_page_walk(), leaving only %ss/%fs selector/segment errors in the compat case. Point at dom_crash_sync_extable in the error message, which is about as good as the error hints from other users of asm_domain_crash_synchronous(), and drop last_extable_addr. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/traps.c | 11 +---------- xen/arch/x86/x86_64/entry.S | 2 +- 2 files changed, 2 insertions(+), 11 deletions(-) diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c index 33e5d21ece..fe9457cdb6 100644 --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -96,7 +96,6 @@ static char __read_mostly opt_nmi[10] = "fatal"; string_param("nmi", opt_nmi); DEFINE_PER_CPU(uint64_t, efer); -static DEFINE_PER_CPU(unsigned long, last_extable_addr); DEFINE_PER_CPU_READ_MOSTLY(seg_desc_t *, gdt); DEFINE_PER_CPU_READ_MOSTLY(l1_pgentry_t, gdt_l1e); @@ -786,7 +785,6 @@ static void do_trap(struct cpu_user_regs *regs) { dprintk(XENLOG_ERR, "Trap %u: %p [%ps] -> %p\n", trapnr, _p(regs->rip), _p(regs->rip), _p(fixup)); - this_cpu(last_extable_addr) = regs->rip; regs->rip = fixup; return; } @@ -1099,7 +1097,6 @@ void do_invalid_op(struct cpu_user_regs *regs) die: if ( (fixup = search_exception_table(regs)) != 0 ) { - this_cpu(last_extable_addr) = regs->rip; regs->rip = fixup; return; } @@ -1122,7 +1119,6 @@ void do_int3(struct cpu_user_regs *regs) if ( (fixup = search_exception_table(regs)) != 0 ) { - this_cpu(last_extable_addr) = regs->rip; dprintk(XENLOG_DEBUG, "Trap %u: %p [%ps] -> %p\n", TRAP_int3, _p(regs->rip), _p(regs->rip), _p(fixup)); regs->rip = fixup; @@ -1461,7 +1457,6 @@ void do_page_fault(struct cpu_user_regs *regs) perfc_incr(copy_user_faults); if ( unlikely(regs->error_code & PFEC_reserved_bit) ) reserved_bit_page_fault(addr, regs); - this_cpu(last_extable_addr) = regs->rip; regs->rip = fixup; return; } @@ -1591,7 +1586,6 @@ void do_general_protection(struct cpu_user_regs *regs) { dprintk(XENLOG_INFO, "GPF (%04x): %p [%ps] -> %p\n", regs->error_code, _p(regs->rip), _p(regs->rip), _p(fixup)); - this_cpu(last_extable_addr) = regs->rip; regs->rip = fixup; return; } @@ -2085,10 +2079,7 @@ void asm_domain_crash_synchronous(unsigned long addr) */ clac(); - if ( addr == 0 ) - addr = this_cpu(last_extable_addr); - - printk("domain_crash_sync called from entry.S: fault at %p %pS\n", + printk("domain_crash_sync called from entry.S: issue around %p %pS\n", _p(addr), _p(addr)); __domain_crash(current->domain); diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S index d55453f3f3..a3ce298529 100644 --- a/xen/arch/x86/x86_64/entry.S +++ b/xen/arch/x86/x86_64/entry.S @@ -527,7 +527,7 @@ ENTRY(dom_crash_sync_extable) sete %al leal (%rax,%rax,2),%eax orb %al,UREGS_cs(%rsp) - xorl %edi,%edi + lea dom_crash_sync_extable(%rip), %rdi jmp asm_domain_crash_synchronous /* Does not return */ .popsection #endif /* CONFIG_PV */ From patchwork Fri May 1 22:58:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523403 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8EF7C81 for ; Fri, 1 May 2020 23:00:59 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6BC94208DB for ; Fri, 1 May 2020 23:00:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="CWTRh3jM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6BC94208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUech-0007wj-9B; Fri, 01 May 2020 22:59:11 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUecg-0007wT-18 for xen-devel@lists.xenproject.org; Fri, 01 May 2020 22:59:10 +0000 X-Inumbo-ID: 5cf8f43e-8bff-11ea-9b6f-12813bfff9fa Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 5cf8f43e-8bff-11ea-9b6f-12813bfff9fa; Fri, 01 May 2020 22:59:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588373949; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vfDg0gLirp3wqWcYHVf6YMOyxLiUvrLksiHPpLTiioo=; b=CWTRh3jMoBdgTb8l5JZHkZrzWJ2i0vSdsuBe5XBN8ZgsOo63lF313NXC jcrMgm4JJKaBTeMwUeVoA6dDaITrVacXQchFXaHEJeyNLRGmAFy70EYJM ReZyH0y3kedaALnaXzJwEBoVOhmM2WzXdWtUC4TuN20BTnPgajOGU/hSd s=; Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa3.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: kIYvcEtKJuBgW1dMx7Ua+jevmpknfZwyDEOZbchuAhJfHUXQJf87Ye5sg9yPemW/8UqWX4hLus St19uT9t+tWzu0d55eT5RBQKAJxbdBzfeHAW6pOK/P81MbGa14YVdTnu4e7mi0Cmk3kNQzuEzx SlaZso0FMEoWSNc5fA5n1mhN2agYO9cEcTFH5WYR8JGX8bCbIZdYciuvQoRfTxC/qQFUAi88VU +hm++7OuEAR1TbbVXSFEMtG66NL0jubThZ0LPvqy/3f0dbqp3mkKaOzHfvlBVeaovzxEm1pQ/a 5Jo= X-SBRS: 2.7 X-MesageID: 16584680 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="16584680" From: Andrew Cooper To: Xen-devel Subject: [PATCH 02/16] x86/traps: Clean up printing in do_reserved_trap()/fatal_trap() Date: Fri, 1 May 2020 23:58:24 +0100 Message-ID: <20200501225838.9866-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" For one, they render the vector in a different base. Introduce X86_EXC_* constants and vec_name() to refer to exceptions by their mnemonic, which starts bringing the code/diagnostics in line with the Intel and AMD manuals. Provide constants for every archtiecturally defined exception, even those not implemented by Xen yet, as do_reserved_trap() is a catch-all handler. Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/traps.c | 24 +++++++++++++++++++----- xen/include/asm-x86/processor.h | 6 +----- xen/include/asm-x86/x86-defns.h | 35 +++++++++++++++++++++++++++++++++++ 3 files changed, 55 insertions(+), 10 deletions(-) diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c index fe9457cdb6..e73f07f28a 100644 --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -686,6 +686,20 @@ const char *trapstr(unsigned int trapnr) return trapnr < ARRAY_SIZE(strings) ? strings[trapnr] : "???"; } +static const char *vec_name(unsigned int vec) +{ + static const char names[][4] = { +#define N(x) [X86_EXC_ ## x] = #x + N(DE), N(DB), N(NMI), N(BP), N(OF), N(BR), N(UD), N(NM), + N(DF), N(CSO), N(TS), N(NP), N(SS), N(GP), N(PF), N(SPV), + N(MF), N(AC), N(MC), N(XM), N(VE), N(CP), + N(HV), N(VC), N(SX), +#undef N + }; + + return (vec < ARRAY_SIZE(names) && names[vec][0]) ? names[vec] : "??"; +} + /* * This is called for faults at very unexpected times (e.g., when interrupts * are disabled). In such situations we can't do much that is safe. We try to @@ -743,10 +757,9 @@ void fatal_trap(const struct cpu_user_regs *regs, bool show_remote) } } - panic("FATAL TRAP: vector = %d (%s)\n" - "[error_code=%04x] %s\n", - trapnr, trapstr(trapnr), regs->error_code, - (regs->eflags & X86_EFLAGS_IF) ? "" : ", IN INTERRUPT CONTEXT"); + panic("FATAL TRAP: vec %u, #%s[%04x]%s\n", + trapnr, vec_name(trapnr), regs->error_code, + (regs->eflags & X86_EFLAGS_IF) ? "" : " IN INTERRUPT CONTEXT"); } static void do_reserved_trap(struct cpu_user_regs *regs) @@ -757,7 +770,8 @@ static void do_reserved_trap(struct cpu_user_regs *regs) return; show_execution_state(regs); - panic("FATAL RESERVED TRAP %#x: %s\n", trapnr, trapstr(trapnr)); + panic("FATAL RESERVED TRAP: vec %u, #%s[%04x]\n", + trapnr, vec_name(trapnr), regs->error_code); } static void do_trap(struct cpu_user_regs *regs) diff --git a/xen/include/asm-x86/processor.h b/xen/include/asm-x86/processor.h index 8f6f5a97dd..12b55e1022 100644 --- a/xen/include/asm-x86/processor.h +++ b/xen/include/asm-x86/processor.h @@ -43,11 +43,7 @@ #define TRAP_virtualisation 20 #define TRAP_nr 32 -#define TRAP_HAVE_EC \ - ((1u << TRAP_double_fault) | (1u << TRAP_invalid_tss) | \ - (1u << TRAP_no_segment) | (1u << TRAP_stack_error) | \ - (1u << TRAP_gp_fault) | (1u << TRAP_page_fault) | \ - (1u << TRAP_alignment_check)) +#define TRAP_HAVE_EC X86_EXC_HAVE_EC /* Set for entry via SYSCALL. Informs return code to use SYSRETQ not IRETQ. */ /* NB. Same as VGCF_in_syscall. No bits in common with any other TRAP_ defn. */ diff --git a/xen/include/asm-x86/x86-defns.h b/xen/include/asm-x86/x86-defns.h index 8bf503220a..84e15b15be 100644 --- a/xen/include/asm-x86/x86-defns.h +++ b/xen/include/asm-x86/x86-defns.h @@ -118,4 +118,39 @@ #define X86_NR_VECTORS 256 +/* Exception Vectors */ +#define X86_EXC_DE 0 /* Divide Error. */ +#define X86_EXC_DB 1 /* Debug Exception. */ +#define X86_EXC_NMI 2 /* NMI. */ +#define X86_EXC_BP 3 /* Breakpoint. */ +#define X86_EXC_OF 4 /* Overflow. */ +#define X86_EXC_BR 5 /* BOUND Range. */ +#define X86_EXC_UD 6 /* Invalid Opcode. */ +#define X86_EXC_NM 7 /* Device Not Available. */ +#define X86_EXC_DF 8 /* Double Fault. */ +#define X86_EXC_CSO 9 /* Coprocessor Segment Overrun. */ +#define X86_EXC_TS 10 /* Invalid TSS. */ +#define X86_EXC_NP 11 /* Segment Not Present. */ +#define X86_EXC_SS 12 /* Stack-Segment Fault. */ +#define X86_EXC_GP 13 /* General Porection Fault. */ +#define X86_EXC_PF 14 /* Page Fault. */ +#define X86_EXC_SPV 15 /* PIC Spurious Interrupt Vector. */ +#define X86_EXC_MF 16 /* Maths fault (x87 FPU). */ +#define X86_EXC_AC 17 /* Alignment Check. */ +#define X86_EXC_MC 18 /* Machine Check. */ +#define X86_EXC_XM 19 /* SIMD Exception. */ +#define X86_EXC_VE 20 /* Virtualisation Exception. */ +#define X86_EXC_CP 21 /* Control-flow Protection. */ +#define X86_EXC_HV 28 /* Hypervisor Injection. */ +#define X86_EXC_VC 29 /* VMM Communication. */ +#define X86_EXC_SX 30 /* Security Exception. */ + +/* Bitmap of exceptions which have error codes. */ +#define X86_EXC_HAVE_EC \ + ((1u << X86_EXC_DF) | (1u << X86_EXC_TS) | (1u << X86_EXC_NP) | \ + (1u << X86_EXC_SS) | (1u << X86_EXC_GP) | (1u << X86_EXC_PF) | \ + (1u << X86_EXC_AC) | (1u << X86_EXC_CP) | \ + (1u << X86_EXC_VC) | (1u << X86_EXC_SX)) + + #endif /* __XEN_X86_DEFNS_H__ */ From patchwork Fri May 1 22:58:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523399 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3665F14B4 for ; Fri, 1 May 2020 23:00:56 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 13166208DB for ; Fri, 1 May 2020 23:00:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="PFIorDf9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 13166208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUecx-00086e-OP; Fri, 01 May 2020 22:59:27 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUecw-00085r-4x for xen-devel@lists.xenproject.org; Fri, 01 May 2020 22:59:26 +0000 X-Inumbo-ID: 5d9ce7c4-8bff-11ea-b9cf-bc764e2007e4 Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 5d9ce7c4-8bff-11ea-b9cf-bc764e2007e4; Fri, 01 May 2020 22:59:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588373949; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yuj7W8FBuxAyknqo2wUBSETsY2vvMAq34uBhRh1QqIU=; b=PFIorDf9U2lcHF4KRelD4zKVEGGgTsqywgiKwD/Bmi46SZMqu8YuZkeW 0s2ljweIozajvWXyGYmH9q5BiJ49GUwSgmqXF+nDVrpUxXhYRJpILFmDZ H8e3SlIc1ryWSusinCueyNFzy9pZdxvTMiYmCb9oy5lHoWHBtldDK33rx 4=; Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa4.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: 1A+w2IiSnHoSp+2Fe/WtiLGlowGS5xLZFh3Kc1HlxmeFkXDESQL+ABj8ImGYUdnf88c9oSuF95 a2mnTEP9WFBGNQEjnPFutxWQz9U7Ez7a43V8iAZ0rtEJ06CQONN4B7XhJV7WDXek1nYwlUYK5c aD8KBkD6uU11lcpIelSRNn1Y0cMSTwvSdhvhe5L9Zo9aT9Zy9YNRw1M6yDMK3fix3Mc/o6574p m5HTRrk4b0vl6y0LGDEs9rEByLDJhIVH3i8BV6Df0L4cDc0L67G64vD8U8x3aBoqvuL1nUjasz uNs= X-SBRS: 2.7 X-MesageID: 17293963 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="17293963" From: Andrew Cooper To: Xen-devel Subject: [PATCH 03/16] x86/traps: Factor out exception_fixup() and make printing consistent Date: Fri, 1 May 2020 23:58:25 +0100 Message-ID: <20200501225838.9866-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" UD faults never had any diagnostics printed, and the others were inconsistent. Don't use dprintk() because identifying traps.c is actively unhelpful in the message, as it is the location of the fixup, not the fault. Use the new vec_name() infrastructure, rather than leaving raw numbers for the log. (XEN) Running stub recovery selftests... (XEN) Fixup #UD[0000]: ffff82d07fffd040 [ffff82d07fffd040] -> ffff82d0403ac9d6 (XEN) Fixup #GP[0000]: ffff82d07fffd041 [ffff82d07fffd041] -> ffff82d0403ac9d6 (XEN) Fixup #SS[0000]: ffff82d07fffd040 [ffff82d07fffd040] -> ffff82d0403ac9d6 (XEN) Fixup #BP[0000]: ffff82d07fffd041 [ffff82d07fffd041] -> ffff82d0403ac9d6 Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/traps.c | 68 ++++++++++++++++++++++++---------------------------- 1 file changed, 31 insertions(+), 37 deletions(-) diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c index e73f07f28a..737ab036d2 100644 --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -774,10 +774,27 @@ static void do_reserved_trap(struct cpu_user_regs *regs) trapnr, vec_name(trapnr), regs->error_code); } +static bool exception_fixup(struct cpu_user_regs *regs, bool print) +{ + unsigned long fixup = search_exception_table(regs); + + if ( unlikely(fixup == 0) ) + return false; + + /* Can currently be triggered by guests. Make sure we ratelimit. */ + if ( IS_ENABLED(CONFIG_DEBUG) && print ) + printk(XENLOG_GUEST XENLOG_WARNING "Fixup #%s[%04x]: %p [%ps] -> %p\n", + vec_name(regs->entry_vector), regs->error_code, + _p(regs->rip), _p(regs->rip), _p(fixup)); + + regs->rip = fixup; + + return true; +} + static void do_trap(struct cpu_user_regs *regs) { unsigned int trapnr = regs->entry_vector; - unsigned long fixup; if ( regs->error_code & X86_XEC_EXT ) goto hardware_trap; @@ -795,13 +812,8 @@ static void do_trap(struct cpu_user_regs *regs) return; } - if ( likely((fixup = search_exception_table(regs)) != 0) ) - { - dprintk(XENLOG_ERR, "Trap %u: %p [%ps] -> %p\n", - trapnr, _p(regs->rip), _p(regs->rip), _p(fixup)); - regs->rip = fixup; + if ( likely(exception_fixup(regs, true)) ) return; - } hardware_trap: if ( debugger_trap_fatal(trapnr, regs) ) @@ -1109,11 +1121,8 @@ void do_invalid_op(struct cpu_user_regs *regs) } die: - if ( (fixup = search_exception_table(regs)) != 0 ) - { - regs->rip = fixup; + if ( likely(exception_fixup(regs, true)) ) return; - } if ( debugger_trap_fatal(TRAP_invalid_op, regs) ) return; @@ -1129,15 +1138,8 @@ void do_int3(struct cpu_user_regs *regs) if ( !guest_mode(regs) ) { - unsigned long fixup; - - if ( (fixup = search_exception_table(regs)) != 0 ) - { - dprintk(XENLOG_DEBUG, "Trap %u: %p [%ps] -> %p\n", - TRAP_int3, _p(regs->rip), _p(regs->rip), _p(fixup)); - regs->rip = fixup; + if ( likely(exception_fixup(regs, true)) ) return; - } if ( !debugger_trap_fatal(TRAP_int3, regs) ) printk(XENLOG_DEBUG "Hit embedded breakpoint at %p [%ps]\n", @@ -1435,7 +1437,7 @@ static int fixup_page_fault(unsigned long addr, struct cpu_user_regs *regs) */ void do_page_fault(struct cpu_user_regs *regs) { - unsigned long addr, fixup; + unsigned long addr; unsigned int error_code; addr = read_cr2(); @@ -1466,12 +1468,11 @@ void do_page_fault(struct cpu_user_regs *regs) if ( pf_type != real_fault ) return; - if ( likely((fixup = search_exception_table(regs)) != 0) ) + if ( likely(exception_fixup(regs, false)) ) { perfc_incr(copy_user_faults); if ( unlikely(regs->error_code & PFEC_reserved_bit) ) reserved_bit_page_fault(addr, regs); - regs->rip = fixup; return; } @@ -1529,7 +1530,6 @@ void do_general_protection(struct cpu_user_regs *regs) #ifdef CONFIG_PV struct vcpu *v = current; #endif - unsigned long fixup; if ( debugger_trap_entry(TRAP_gp_fault, regs) ) return; @@ -1596,13 +1596,8 @@ void do_general_protection(struct cpu_user_regs *regs) gp_in_kernel: - if ( likely((fixup = search_exception_table(regs)) != 0) ) - { - dprintk(XENLOG_INFO, "GPF (%04x): %p [%ps] -> %p\n", - regs->error_code, _p(regs->rip), _p(regs->rip), _p(fixup)); - regs->rip = fixup; + if ( likely(exception_fixup(regs, true)) ) return; - } hardware_gp: if ( debugger_trap_fatal(TRAP_gp_fault, regs) ) @@ -1761,18 +1756,17 @@ void do_device_not_available(struct cpu_user_regs *regs) if ( !guest_mode(regs) ) { - unsigned long fixup = search_exception_table(regs); - - gprintk(XENLOG_ERR, "#NM: %p [%ps] -> %p\n", - _p(regs->rip), _p(regs->rip), _p(fixup)); /* * We shouldn't be able to reach here, but for release builds have * the recovery logic in place nevertheless. */ - ASSERT_UNREACHABLE(); - BUG_ON(!fixup); - regs->rip = fixup; - return; + if ( exception_fixup(regs, true) ) + { + ASSERT_UNREACHABLE(); + return; + } + + fatal_trap(regs, false); } #ifdef CONFIG_PV From patchwork Fri May 1 22:58:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523407 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4278D81 for ; Fri, 1 May 2020 23:01:10 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1EDEF208DB for ; Fri, 1 May 2020 23:01:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="fGi6lLAU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1EDEF208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUed2-000898-1K; Fri, 01 May 2020 22:59:32 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUed1-00088e-40 for xen-devel@lists.xenproject.org; Fri, 01 May 2020 22:59:31 +0000 X-Inumbo-ID: 5dd33824-8bff-11ea-ae69-bc764e2007e4 Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 5dd33824-8bff-11ea-ae69-bc764e2007e4; Fri, 01 May 2020 22:59:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588373949; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zvkVxgzJkr/FcspZwSx7AStaUlQ9aD0VLFmFs6qVRg0=; b=fGi6lLAUwPdBOiAVnHjWKK41Hlx2go5w8R3p2YUdlH3M2qWCCLOm/+30 CGTZ7u/y2QHMJB2jLnSBgDL3eDVMkfd8796S2IOUlQd7lXm65I+AS4zt8 0mnc6XI6GExBY2f1SmPcH3skfdtYVPkE1xl82dnU3cNsqsYlsSFSn9U/h M=; Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa1.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: uy/J8kG2T7UsumKNrlGpZrycXRrLKHkS8/gkbU9dYQkI//NE/4SNcE1Qgh87iP7E0nwcLnHD82 NqrkcX/IuKsap+7nG9do8Pmvg5rRYn+kNoArkjTxcKfm4aNlugT2LrRAe+VZLgA4aJqbeuKBhb I8P1T/h6bQGm6aw4IR62GF8X9FdbwUMVXXMgmB/4tOlS+VC7KDeWKCZofJWfAqQdV+a+H/T0FD fCueCjlYLwN7bzkITbTT5OPtNTDae3q9QDVZwbo1ytYpxTw58XrwSYQg7RIFaF1Wc3+LfNd8Qs JZk= X-SBRS: 2.7 X-MesageID: 16854948 X-Ironport-Server: esa1.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="16854948" From: Andrew Cooper To: Xen-devel Subject: [PATCH 04/16] x86/smpboot: Write the top-of-stack block in cpu_smpboot_alloc() Date: Fri, 1 May 2020 23:58:26 +0100 Message-ID: <20200501225838.9866-5-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" This allows the AP boot assembly use per-cpu variables, and brings the semantics closer to that of the BSP, which can use per-cpu variables from the start of day. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/smpboot.c | 7 ++++++- xen/include/asm-x86/current.h | 5 ----- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index 5a3786d399..f999323bc4 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -329,7 +329,6 @@ void start_secondary(void *unused) /* Critical region without IDT or TSS. Any fault is deadly! */ - set_processor_id(cpu); set_current(idle_vcpu[cpu]); this_cpu(curr_vcpu) = idle_vcpu[cpu]; rdmsrl(MSR_EFER, this_cpu(efer)); @@ -986,6 +985,7 @@ static void cpu_smpboot_free(unsigned int cpu, bool remove) static int cpu_smpboot_alloc(unsigned int cpu) { + struct cpu_info *info; unsigned int i, memflags = 0; nodeid_t node = cpu_to_node(cpu); seg_desc_t *gdt; @@ -999,6 +999,11 @@ static int cpu_smpboot_alloc(unsigned int cpu) stack_base[cpu] = alloc_xenheap_pages(STACK_ORDER, memflags); if ( stack_base[cpu] == NULL ) goto out; + + info = get_cpu_info_from_stack((unsigned long)stack_base[cpu]); + info->processor_id = cpu; + info->per_cpu_offset = __per_cpu_offset[cpu]; + memguard_guard_stack(stack_base[cpu]); gdt = per_cpu(gdt, cpu) ?: alloc_xenheap_pages(0, memflags); diff --git a/xen/include/asm-x86/current.h b/xen/include/asm-x86/current.h index 0b47485337..5b8f4dbc79 100644 --- a/xen/include/asm-x86/current.h +++ b/xen/include/asm-x86/current.h @@ -100,11 +100,6 @@ static inline struct cpu_info *get_cpu_info(void) #define current (get_current()) #define get_processor_id() (get_cpu_info()->processor_id) -#define set_processor_id(id) do { \ - struct cpu_info *ci__ = get_cpu_info(); \ - ci__->per_cpu_offset = __per_cpu_offset[ci__->processor_id = (id)]; \ -} while (0) - #define guest_cpu_user_regs() (&get_cpu_info()->guest_cpu_user_regs) /* From patchwork Fri May 1 22:58:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523401 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5AA4081 for ; Fri, 1 May 2020 23:00:57 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 37833208DB for ; Fri, 1 May 2020 23:00:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="W5hPMo+A" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 37833208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUeci-0007x5-HD; Fri, 01 May 2020 22:59:12 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUecg-0007wY-RE for xen-devel@lists.xenproject.org; Fri, 01 May 2020 22:59:10 +0000 X-Inumbo-ID: 5cf8f43f-8bff-11ea-9b6f-12813bfff9fa Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 5cf8f43f-8bff-11ea-9b6f-12813bfff9fa; Fri, 01 May 2020 22:59:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588373950; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/F+D6hsQxodq6rXf+K3jo7CsyQLXckH86K/eAndVO7A=; b=W5hPMo+AtKM9SfAZAKucnc7KqAHchiBZ94AOU8K17mfANdY7vZ/eV317 UAHuQsGMsgW5Wz819HRbTH9lOWCWK1t8DU6yctdQtd7H6DKkEAt0Rgnhr l7TDbzlRSgwpPmZxbHpou/50zPz4Jq3vazdVDmqXNeAR6WVCiH7P88OMn c=; Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa3.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: dPHGxFspW1wRPP49DXo66IYXvO9cK4Bt/EPT0HGBFSsfKiNJZIIvpKskWh2ibiJW+/hd3b0Gpv GI8k9lsx5Ne53H5tQcEAflZQetbw4YxwDZs3n/hCxlRoM5I81g4bOPMIaPjIkAs665am6ovBN7 NIMFIzVQiOuIf6qj3xP4OjemoVDAhEjWiaRPuYpGfnNgoeGG1h63VeWgfvqbyr2emvgiqV1Uy8 0M4LwaUtCneAYSqpam1lTgIpIanoB4+Bqt3AER3TLXWIarSHOJPm6/2HmyxN/zmpPgtzKZ/Vo/ N+c= X-SBRS: 2.7 X-MesageID: 16584683 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="16584683" From: Andrew Cooper To: Xen-devel Subject: [PATCH 05/16] x86/shstk: Introduce Supervisor Shadow Stack support Date: Fri, 1 May 2020 23:58:27 +0100 Message-ID: <20200501225838.9866-6-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Introduce CONFIG_HAS_AS_CET to determine whether CET instructions are supported in the assembler, and CONFIG_XEN_SHSTK as the main build option. Introduce xen={no-,}shstk to for a user to select whether or not to use shadow stacks at runtime, and X86_FEATURE_XEN_SHSTK to determine Xen's overall enablement of shadow stacks. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/Kconfig | 17 +++++++++++++++++ xen/arch/x86/setup.c | 30 ++++++++++++++++++++++++++++++ xen/include/asm-x86/cpufeature.h | 1 + xen/include/asm-x86/cpufeatures.h | 1 + xen/scripts/Kconfig.include | 4 ++++ 5 files changed, 53 insertions(+) diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig index 96432f1f69..ebd01e6893 100644 --- a/xen/arch/x86/Kconfig +++ b/xen/arch/x86/Kconfig @@ -34,6 +34,9 @@ config ARCH_DEFCONFIG config INDIRECT_THUNK def_bool $(cc-option,-mindirect-branch-register) +config HAS_AS_CET + def_bool $(as-instr,wrssq %rax$(comma)0;setssbsy;endbr64) + menu "Architecture Features" source "arch/Kconfig" @@ -97,6 +100,20 @@ config HVM If unsure, say Y. +config XEN_SHSTK + bool "Supervisor Shadow Stacks" + depends on HAS_AS_CET && EXPERT = "y" + default y + ---help--- + Control-flow Enforcement Technology (CET) is a set of features in + hardware designed to combat Return-oriented Programming (ROP, also + call/jump COP/JOP) attacks. Shadow Stacks are one CET feature + designed to provide return address protection. + + This option arranges for Xen to use CET-SS for its own protection. + When CET-SS is active, 32bit PV guests cannot be used. Backwards + compatiblity can be provided vai the PV Shim mechanism. + config SHADOW_PAGING bool "Shadow Paging" default y diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 9e9576344c..aa21201507 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -95,6 +95,36 @@ unsigned long __initdata highmem_start; size_param("highmem-start", highmem_start); #endif +static bool __initdata opt_xen_shstk = true; + +static int parse_xen(const char *s) +{ + const char *ss; + int val, rc = 0; + + do { + ss = strchr(s, ','); + if ( !ss ) + ss = strchr(s, '\0'); + + if ( (val = parse_boolean("shstk", s, ss)) >= 0 ) + { +#ifdef CONFIG_XEN_SHSTK + opt_xen_shstk = val; +#else + no_config_param("XEN_SHSTK", "xen", s, ss); +#endif + } + else + rc = -EINVAL; + + s = ss + 1; + } while ( *ss ); + + return rc; +} +custom_param("xen", parse_xen); + cpumask_t __read_mostly cpu_present_map; unsigned long __read_mostly xen_phys_start; diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h index 859970570b..6b25f61832 100644 --- a/xen/include/asm-x86/cpufeature.h +++ b/xen/include/asm-x86/cpufeature.h @@ -136,6 +136,7 @@ #define cpu_has_aperfmperf boot_cpu_has(X86_FEATURE_APERFMPERF) #define cpu_has_lfence_dispatch boot_cpu_has(X86_FEATURE_LFENCE_DISPATCH) #define cpu_has_xen_lbr boot_cpu_has(X86_FEATURE_XEN_LBR) +#define cpu_has_xen_shstk boot_cpu_has(X86_FEATURE_XEN_SHSTK) #define cpu_has_msr_tsc_aux (cpu_has_rdtscp || cpu_has_rdpid) diff --git a/xen/include/asm-x86/cpufeatures.h b/xen/include/asm-x86/cpufeatures.h index b9d3cac975..d7e42d9bb6 100644 --- a/xen/include/asm-x86/cpufeatures.h +++ b/xen/include/asm-x86/cpufeatures.h @@ -38,6 +38,7 @@ XEN_CPUFEATURE(XEN_LBR, X86_SYNTH(22)) /* Xen uses MSR_DEBUGCTL.LBR */ XEN_CPUFEATURE(SC_VERW_PV, X86_SYNTH(23)) /* VERW used by Xen for PV */ XEN_CPUFEATURE(SC_VERW_HVM, X86_SYNTH(24)) /* VERW used by Xen for HVM */ XEN_CPUFEATURE(SC_VERW_IDLE, X86_SYNTH(25)) /* VERW used by Xen for idle */ +XEN_CPUFEATURE(XEN_SHSTK, X86_SYNTH(26)) /* Xen uses CET Shadow Stacks */ /* Bug words follow the synthetic words. */ #define X86_NR_BUG 1 diff --git a/xen/scripts/Kconfig.include b/xen/scripts/Kconfig.include index 8221095ca3..e1f13e1720 100644 --- a/xen/scripts/Kconfig.include +++ b/xen/scripts/Kconfig.include @@ -31,6 +31,10 @@ cc-option = $(success,$(CC) -Werror $(CLANG_FLAGS) $(1) -E -x c /dev/null -o /de # Return y if the linker supports , n otherwise ld-option = $(success,$(LD) -v $(1)) +# $(as-instr,) +# Return y if the assembler supports , n otherwise +as-instr = $(success,printf "%b\n" "$(1)" | $(CC) $(CLANG_FLAGS) -c -x assembler -o /dev/null -) + # check if $(CC) and $(LD) exist $(error-if,$(failure,command -v $(CC)),compiler '$(CC)' not found) $(error-if,$(failure,command -v $(LD)),linker '$(LD)' not found) From patchwork Fri May 1 22:58:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523409 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DAB4C81 for ; Fri, 1 May 2020 23:01:12 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B7E24208DB for ; Fri, 1 May 2020 23:01:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="AoZkj8XW" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B7E24208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUecY-0007um-Cu; Fri, 01 May 2020 22:59:02 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUecX-0007uc-3v for xen-devel@lists.xenproject.org; Fri, 01 May 2020 22:59:01 +0000 X-Inumbo-ID: 57b00d78-8bff-11ea-9b6f-12813bfff9fa Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 57b00d78-8bff-11ea-9b6f-12813bfff9fa; Fri, 01 May 2020 22:58:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588373939; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mdKKOqqqJIKSsZmEMlIiWYVPPhe96GgpGtAJ8re5vaM=; b=AoZkj8XWAXySDVIlp67DaBa4KIy3gfhba0AtRgtQR/Vz/5hQfTsH4NnA f4DFSaUYpybfSrJ+1e7APrU4EEa0b0bsZ4E7muV8rYG8EyzerlvRBk6cE j9m7FlE2Xu9dik/pahXmWkpw7yho5jvf85uJM6DAreNkKgX53QCdJbSZw k=; Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: et/aZxFKlhLg+JTjMM/J44nwQ0ngPJ/IIJz7GCRgGRYgzmeti5B6CJG2rdhI1RBGdRwd39Wlq6 h+HTh7Oi0XBaTAI3Wl3YDd0HgoitKVBNijB0hbAUhJ8EjgnVhE9vKorVbjJ9UU3L0qrcHIvxDt z3n8egHgMpzXiCWGKWMRpyRccw6RG+Z/re9cosldoJx3OZSniZ6LrRMhx1/v13FziJZKu6qwLo w59xsYhop4A+lThdHGdBCYHBup0BCl/WOjbedOTFug7t6GXDvd7fLSyI7QqRIktI/JadpUh7WW hyE= X-SBRS: 2.7 X-MesageID: 16905916 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="16905916" From: Andrew Cooper To: Xen-devel Subject: [PATCH 06/16] x86/traps: Implement #CP handler and extend #PF for shadow stacks Date: Fri, 1 May 2020 23:58:28 +0100 Message-ID: <20200501225838.9866-7-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" For now, any #CP exception or shadow stack #PF indicate a bug in Xen, but attempt to recover if taken in guest context. Drop the comment beside do_page_fault(). It's stale (missing PFEC_prot_key), and inaccurate (PFEC_present being set means just that, not necesserily a protection violation). Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/traps.c | 55 ++++++++++++++++++++++++++++++++++------- xen/arch/x86/x86_64/entry.S | 7 +++++- xen/include/asm-x86/processor.h | 2 ++ 3 files changed, 54 insertions(+), 10 deletions(-) diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c index 737ab036d2..ddbe312f89 100644 --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -158,7 +158,9 @@ void (* const exception_table[TRAP_nr])(struct cpu_user_regs *regs) = { [TRAP_alignment_check] = do_trap, [TRAP_machine_check] = (void *)do_machine_check, [TRAP_simd_error] = do_trap, - [TRAP_virtualisation ... + [TRAP_virtualisation] = do_reserved_trap, + [X86_EXC_CP] = do_entry_CP, + [X86_EXC_CP + 1 ... (ARRAY_SIZE(exception_table) - 1)] = do_reserved_trap, }; @@ -1427,14 +1429,6 @@ static int fixup_page_fault(unsigned long addr, struct cpu_user_regs *regs) return 0; } -/* - * #PF error code: - * Bit 0: Protection violation (=1) ; Page not present (=0) - * Bit 1: Write access - * Bit 2: User mode (=1) ; Supervisor mode (=0) - * Bit 3: Reserved bit violation - * Bit 4: Instruction fetch - */ void do_page_fault(struct cpu_user_regs *regs) { unsigned long addr; @@ -1457,6 +1451,10 @@ void do_page_fault(struct cpu_user_regs *regs) { enum pf_type pf_type = spurious_page_fault(addr, regs); + /* Any fault on a shadow stack access is a bug in Xen. */ + if ( error_code & PFEC_shstk ) + goto fatal; + if ( (pf_type == smep_fault) || (pf_type == smap_fault) ) { console_start_sync(); @@ -1476,6 +1474,7 @@ void do_page_fault(struct cpu_user_regs *regs) return; } + fatal: if ( debugger_trap_fatal(TRAP_page_fault, regs) ) return; @@ -1906,6 +1905,43 @@ void do_debug(struct cpu_user_regs *regs) pv_inject_hw_exception(TRAP_debug, X86_EVENT_NO_EC); } +void do_entry_CP(struct cpu_user_regs *regs) +{ + static const char errors[][10] = { + [1] = "near ret", + [2] = "far/iret", + [3] = "endbranch", + [4] = "rstorssp", + [5] = "setssbsy", + }; + const char *err = "??"; + unsigned int ec = regs->error_code; + + if ( debugger_trap_entry(TRAP_debug, regs) ) + return; + + /* Decode ec if possible */ + if ( ec < ARRAY_SIZE(errors) && errors[ec][0] ) + err = errors[ec]; + + /* + * For now, only supervisors shadow stacks should be active. A #CP from + * guest context is probably a Xen bug, but kill the guest in an attempt + * to recover. + */ + if ( guest_mode(regs) ) + { + gprintk(XENLOG_ERR, "Hit #CP[%04x] in guest context %04x:%p\n", + ec, regs->cs, _p(regs->rip)); + ASSERT_UNREACHABLE(); + domain_crash(current->domain); + return; + } + + show_execution_state(regs); + panic("CONTROL-FLOW PROTECTION FAULT: #CP[%04x] %s\n", ec, err); +} + static void __init noinline __set_intr_gate(unsigned int n, uint32_t dpl, void *addr) { @@ -1995,6 +2031,7 @@ void __init init_idt_traps(void) set_intr_gate(TRAP_alignment_check,&alignment_check); set_intr_gate(TRAP_machine_check,&machine_check); set_intr_gate(TRAP_simd_error,&simd_coprocessor_error); + set_intr_gate(X86_EXC_CP, entry_CP); /* Specify dedicated interrupt stacks for NMI, #DF, and #MC. */ enable_each_ist(idt_table); diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S index a3ce298529..6403c0ab92 100644 --- a/xen/arch/x86/x86_64/entry.S +++ b/xen/arch/x86/x86_64/entry.S @@ -795,6 +795,10 @@ ENTRY(alignment_check) movl $TRAP_alignment_check,4(%rsp) jmp handle_exception +ENTRY(entry_CP) + movl $X86_EXC_CP, 4(%rsp) + jmp handle_exception + ENTRY(double_fault) movl $TRAP_double_fault,4(%rsp) /* Set AC to reduce chance of further SMAP faults */ @@ -940,7 +944,8 @@ autogen_stubs: /* Automatically generated stubs. */ entrypoint 1b /* Reserved exceptions, heading towards do_reserved_trap(). */ - .elseif vec == TRAP_copro_seg || vec == TRAP_spurious_int || (vec > TRAP_simd_error && vec < TRAP_nr) + .elseif vec == TRAP_copro_seg || vec == TRAP_spurious_int || \ + vec == TRAP_virtualisation || (vec > X86_EXC_CP && vec < TRAP_nr) 1: test $8,%spl /* 64bit exception frames are 16 byte aligned, but the word */ jz 2f /* size is 8 bytes. Check whether the processor gave us an */ diff --git a/xen/include/asm-x86/processor.h b/xen/include/asm-x86/processor.h index 12b55e1022..5e8a0fb649 100644 --- a/xen/include/asm-x86/processor.h +++ b/xen/include/asm-x86/processor.h @@ -68,6 +68,7 @@ #define PFEC_reserved_bit (_AC(1,U) << 3) #define PFEC_insn_fetch (_AC(1,U) << 4) #define PFEC_prot_key (_AC(1,U) << 5) +#define PFEC_shstk (_AC(1,U) << 6) #define PFEC_arch_mask (_AC(0xffff,U)) /* Architectural PFEC values. */ /* Internally used only flags. */ #define PFEC_page_paged (1U<<16) @@ -529,6 +530,7 @@ DECLARE_TRAP_HANDLER(coprocessor_error); DECLARE_TRAP_HANDLER(simd_coprocessor_error); DECLARE_TRAP_HANDLER_CONST(machine_check); DECLARE_TRAP_HANDLER(alignment_check); +DECLARE_TRAP_HANDLER(entry_CP); DECLARE_TRAP_HANDLER(entry_int82); From patchwork Fri May 1 22:58:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523391 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5978E913 for ; Fri, 1 May 2020 22:59:46 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2B642208DB for ; Fri, 1 May 2020 22:59:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="R9E45x62" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2B642208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUecZ-0007us-Kz; Fri, 01 May 2020 22:59:03 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUecX-0007ud-Gj for xen-devel@lists.xenproject.org; Fri, 01 May 2020 22:59:01 +0000 X-Inumbo-ID: 57b0d122-8bff-11ea-9887-bc764e2007e4 Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 57b0d122-8bff-11ea-9887-bc764e2007e4; Fri, 01 May 2020 22:58:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588373939; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7x5ESMNeelNN6mx92zjbIOif/mDl8/XYbLA1njk7T8g=; b=R9E45x62qjLIIIrMD3jzK2GbTFHHkI+BwqxvDmcVb70lHvfjcb3mzZ/C ghkXCb8EpC4SZSzeTtfL4uwQlKdO5KPQpNKPWrHjIlvE4gTrPZAr9/USD 1+ISsVUN2IdU39rVqsWHJNvkaCXBqRyz3J7RPXXlTYe2h6rJJJuEDwl9T 8=; Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa6.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: V89ehA/w7+CdA8L11/1s8fe0cbsS4DVzi0DwAvyQ+3YkGGg7gmWsJXV9QR5avOXlUNEoFQS9n6 8Gkk62mZSUQqmPjD3g7sISTx7m4uUmjqAVbftWuDHHvCj1CO0anil0m0yVq0Itnv+YA2QQ2jUW s5qAbhy5L3GsMo8wVxwOe+szIBB48mDxwqqJvPSEokAEDmXxXq2XAx5NcGxNIdnVWvXsnDaUSl +z8W3a83ZhXBmPRQGBGoZLvUnDFkqtwG/L+2gPxpuZ1w6nOicjY37awmlO9tdJUHNM27mrC9A8 p8Y= X-SBRS: 2.7 X-MesageID: 16994843 X-Ironport-Server: esa6.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="16994843" From: Andrew Cooper To: Xen-devel Subject: [PATCH 07/16] x86/shstk: Re-layout the stack block for shadow stacks Date: Fri, 1 May 2020 23:58:29 +0100 Message-ID: <20200501225838.9866-8-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" We have two free pages in the current stack. A useful property of shadow stacks and regular stacks is that they act as each others guard pages as far as OoB writes go. Move the regular IST stacks up by one page, to allow their shadow stack page to be in slot 0. The primary shadow stack uses slot 5. As the shadow IST stacks are only 1k large, shuffle the order of IST vectors to have #DF numerically highest (so there is no chance of a shadow stack overflow clobbering the supervisor token). The XPTI code already breaks the MEMORY_GUARD abstraction for stacks by forcing it to be present. To avoid having too many configurations, do away with the concept entirely, and unconditionally unmap the pages in all cases. A later change will turn these properly into shadow stacks. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/cpu/common.c | 10 +++++----- xen/arch/x86/mm.c | 19 ++++++------------- xen/arch/x86/smpboot.c | 3 +-- xen/arch/x86/traps.c | 23 ++++++----------------- xen/include/asm-x86/current.h | 12 ++++++------ xen/include/asm-x86/mm.h | 1 - xen/include/asm-x86/processor.h | 6 +++--- 7 files changed, 27 insertions(+), 47 deletions(-) diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c index 131ff03fcf..290f9f1c30 100644 --- a/xen/arch/x86/cpu/common.c +++ b/xen/arch/x86/cpu/common.c @@ -732,14 +732,14 @@ void load_system_tables(void) .rsp2 = 0x8600111111111111ul, /* - * MCE, NMI and Double Fault handlers get their own stacks. + * #DB, NMI, DF and #MCE handlers get their own stacks. * All others poisoned. */ .ist = { - [IST_MCE - 1] = stack_top + IST_MCE * PAGE_SIZE, - [IST_DF - 1] = stack_top + IST_DF * PAGE_SIZE, - [IST_NMI - 1] = stack_top + IST_NMI * PAGE_SIZE, - [IST_DB - 1] = stack_top + IST_DB * PAGE_SIZE, + [IST_MCE - 1] = stack_top + (1 + IST_MCE) * PAGE_SIZE, + [IST_NMI - 1] = stack_top + (1 + IST_NMI) * PAGE_SIZE, + [IST_DB - 1] = stack_top + (1 + IST_DB) * PAGE_SIZE, + [IST_DF - 1] = stack_top + (1 + IST_DF) * PAGE_SIZE, [IST_MAX ... ARRAY_SIZE(tss->ist) - 1] = 0x8600111111111111ul, diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 355c50ff91..bc44d865ef 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -6002,25 +6002,18 @@ void memguard_unguard_range(void *p, unsigned long l) void memguard_guard_stack(void *p) { - /* IST_MAX IST pages + at least 1 guard page + primary stack. */ - BUILD_BUG_ON((IST_MAX + 1) * PAGE_SIZE + PRIMARY_STACK_SIZE > STACK_SIZE); + map_pages_to_xen((unsigned long)p, virt_to_mfn(p), 1, _PAGE_NONE); - memguard_guard_range(p + IST_MAX * PAGE_SIZE, - STACK_SIZE - PRIMARY_STACK_SIZE - IST_MAX * PAGE_SIZE); + p += 5 * PAGE_SIZE; + map_pages_to_xen((unsigned long)p, virt_to_mfn(p), 1, _PAGE_NONE); } void memguard_unguard_stack(void *p) { - memguard_unguard_range(p + IST_MAX * PAGE_SIZE, - STACK_SIZE - PRIMARY_STACK_SIZE - IST_MAX * PAGE_SIZE); -} - -bool memguard_is_stack_guard_page(unsigned long addr) -{ - addr &= STACK_SIZE - 1; + map_pages_to_xen((unsigned long)p, virt_to_mfn(p), 1, PAGE_HYPERVISOR_RW); - return addr >= IST_MAX * PAGE_SIZE && - addr < STACK_SIZE - PRIMARY_STACK_SIZE; + p += 5 * PAGE_SIZE; + map_pages_to_xen((unsigned long)p, virt_to_mfn(p), 1, PAGE_HYPERVISOR_RW); } void arch_dump_shared_mem_info(void) diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index f999323bc4..e0f421ca3d 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -823,8 +823,7 @@ static int setup_cpu_root_pgt(unsigned int cpu) /* Install direct map page table entries for stack, IDT, and TSS. */ for ( off = rc = 0; !rc && off < STACK_SIZE; off += PAGE_SIZE ) - if ( !memguard_is_stack_guard_page(off) ) - rc = clone_mapping(__va(__pa(stack_base[cpu])) + off, rpt); + rc = clone_mapping(__va(__pa(stack_base[cpu])) + off, rpt); if ( !rc ) rc = clone_mapping(idt_tables[cpu], rpt); diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c index ddbe312f89..1cf00c1f4a 100644 --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -369,20 +369,15 @@ static void show_guest_stack(struct vcpu *v, const struct cpu_user_regs *regs) /* * Notes for get_stack_trace_bottom() and get_stack_dump_bottom() * - * Stack pages 0 - 3: + * Stack pages 1 - 4: * These are all 1-page IST stacks. Each of these stacks have an exception * frame and saved register state at the top. The interesting bound for a * trace is the word adjacent to this, while the bound for a dump is the * very top, including the exception frame. * - * Stack pages 4 and 5: - * None of these are particularly interesting. With MEMORY_GUARD, page 5 is - * explicitly not present, so attempting to dump or trace it is - * counterproductive. Without MEMORY_GUARD, it is possible for a call chain - * to use the entire primary stack and wander into page 5. In this case, - * consider these pages an extension of the primary stack to aid debugging - * hopefully rare situations where the primary stack has effective been - * overflown. + * Stack pages 0 and 5: + * Shadow stacks. These are mapped read-only, and used by CET-SS capable + * processors. They will never contain regular stack data. * * Stack pages 6 and 7: * These form the primary stack, and have a cpu_info at the top. For a @@ -396,13 +391,10 @@ unsigned long get_stack_trace_bottom(unsigned long sp) { switch ( get_stack_page(sp) ) { - case 0 ... 3: + case 1 ... 4: return ROUNDUP(sp, PAGE_SIZE) - offsetof(struct cpu_user_regs, es) - sizeof(unsigned long); -#ifndef MEMORY_GUARD - case 4 ... 5: -#endif case 6 ... 7: return ROUNDUP(sp, STACK_SIZE) - sizeof(struct cpu_info) - sizeof(unsigned long); @@ -416,12 +408,9 @@ unsigned long get_stack_dump_bottom(unsigned long sp) { switch ( get_stack_page(sp) ) { - case 0 ... 3: + case 1 ... 4: return ROUNDUP(sp, PAGE_SIZE) - sizeof(unsigned long); -#ifndef MEMORY_GUARD - case 4 ... 5: -#endif case 6 ... 7: return ROUNDUP(sp, STACK_SIZE) - sizeof(unsigned long); diff --git a/xen/include/asm-x86/current.h b/xen/include/asm-x86/current.h index 5b8f4dbc79..99b66a0087 100644 --- a/xen/include/asm-x86/current.h +++ b/xen/include/asm-x86/current.h @@ -16,12 +16,12 @@ * * 7 - Primary stack (with a struct cpu_info at the top) * 6 - Primary stack - * 5 - Optionally not present (MEMORY_GUARD) - * 4 - Unused; optionally not present (MEMORY_GUARD) - * 3 - Unused; optionally not present (MEMORY_GUARD) - * 2 - MCE IST stack - * 1 - NMI IST stack - * 0 - Double Fault IST stack + * 5 - Primay Shadow Stack (read-only) + * 4 - #DF IST stack + * 3 - #DB IST stack + * 2 - NMI IST stack + * 1 - #MC IST stack + * 0 - IST Shadow Stacks (4x 1k, read-only) */ /* diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h index 3d3f9d49ac..7e74996053 100644 --- a/xen/include/asm-x86/mm.h +++ b/xen/include/asm-x86/mm.h @@ -536,7 +536,6 @@ void memguard_unguard_range(void *p, unsigned long l); void memguard_guard_stack(void *p); void memguard_unguard_stack(void *p); -bool __attribute_const__ memguard_is_stack_guard_page(unsigned long addr); struct mmio_ro_emulate_ctxt { unsigned long cr2; diff --git a/xen/include/asm-x86/processor.h b/xen/include/asm-x86/processor.h index 5e8a0fb649..f7e80d12e4 100644 --- a/xen/include/asm-x86/processor.h +++ b/xen/include/asm-x86/processor.h @@ -439,10 +439,10 @@ struct tss_page { DECLARE_PER_CPU(struct tss_page, tss_page); #define IST_NONE 0UL -#define IST_DF 1UL +#define IST_MCE 1UL #define IST_NMI 2UL -#define IST_MCE 3UL -#define IST_DB 4UL +#define IST_DB 3UL +#define IST_DF 4UL #define IST_MAX 4UL /* Set the Interrupt Stack Table used by a particular IDT entry. */ From patchwork Fri May 1 22:58:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523393 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8BCFA912 for ; Fri, 1 May 2020 23:00:44 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6752F208DB for ; Fri, 1 May 2020 23:00:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="Kn9cZ2YJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6752F208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUecd-0007w6-0s; Fri, 01 May 2020 22:59:07 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUecc-0007vF-31 for xen-devel@lists.xenproject.org; Fri, 01 May 2020 22:59:06 +0000 X-Inumbo-ID: 57d17eb8-8bff-11ea-b07b-bc764e2007e4 Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 57d17eb8-8bff-11ea-b07b-bc764e2007e4; Fri, 01 May 2020 22:59:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588373940; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rL/dG5kGKiStpqKmDj82L3vvn9kv06+Bf9qYwu7T+rE=; b=Kn9cZ2YJBdhJQbkjLexIclJ/3Dm+uFcV1T7SZRyiX/XxGRUW1T0CE2kJ qGXbDyvdkHwc9bEp+KTrQ2z2lJLmA+goxswBKWNgIiYa0Aq+4VMPX9WrB ISb3v7cQKDPMvqBBZY2rHigt8UD5kC3ioFYm0cMvwnub1cQjPrnWmzdhB 0=; Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa3.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: zm/Rio6/a5ZOg6ZjO8FJSzE8qGDSpCwuHsz559zCVgwOpMbAVIWa+S+s+4hHf9z7aT2FUR1wds j8hzds/+dTrTjpEaIXPADLOAMqt3yRyHmm9A0LnVkHmhpXigbIKGdELb900ETMHcpI+qyFOeGA 58gpwwWRCnlpY1OeCdntVjVcMg2aq4AC/6YmnP7hHHHxJ2IXmCoK/j4lBT7MWB0mM2EtG93O09 DkbCoFQgm5gIq7A0Q4kz3bgf1EJRMWgsE5KCpUVrZsfjWyGjgLhB3oFrNAA/yQlvLBrqvQhX/0 CVo= X-SBRS: 2.7 X-MesageID: 16584676 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="16584676" From: Andrew Cooper To: Xen-devel Subject: [PATCH 08/16] x86/shstk: Create shadow stacks Date: Fri, 1 May 2020 23:58:30 +0100 Message-ID: <20200501225838.9866-9-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Introduce HYPERVISOR_SHSTK pagetable constants, which are Read-Only + Dirty. Use these in place of _PAGE_NONE for memguard_guard_stack(). Supervisor shadow stacks need a token written at the top, which is most easily done before making the frame read only. Allocate the shadow IST stack block in struct tss_page. It doesn't strictly need to live here, but it is a convenient location (and XPTI-safe, for testing purposes). Have load_system_tables() set up the shadow IST stack table when setting up the regular IST in the TSS. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/cpu/common.c | 19 +++++++++++++++++++ xen/arch/x86/mm.c | 22 +++++++++++++++++++--- xen/include/asm-x86/page.h | 1 + xen/include/asm-x86/processor.h | 3 ++- xen/include/asm-x86/x86_64/page.h | 1 + 5 files changed, 42 insertions(+), 4 deletions(-) diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c index 290f9f1c30..3962717aa5 100644 --- a/xen/arch/x86/cpu/common.c +++ b/xen/arch/x86/cpu/common.c @@ -748,6 +748,25 @@ void load_system_tables(void) .bitmap = IOBMP_INVALID_OFFSET, }; + /* Set up the shadow stack IST. */ + if ( cpu_has_xen_shstk ) { + unsigned int i; + uint64_t *ist_ssp = this_cpu(tss_page).ist_ssp; + + /* Must point at the supervisor stack token. */ + ist_ssp[IST_MCE] = stack_top + (IST_MCE * 0x400) - 8; + ist_ssp[IST_NMI] = stack_top + (IST_NMI * 0x400) - 8; + ist_ssp[IST_DB] = stack_top + (IST_DB * 0x400) - 8; + ist_ssp[IST_DF] = stack_top + (IST_DF * 0x400) - 8; + + /* Poision unused entries. */ + for ( i = IST_MAX; + i < ARRAY_SIZE(this_cpu(tss_page).ist_ssp); ++i ) + ist_ssp[i] = 0x8600111111111111ul; + + wrmsrl(MSR_INTERRUPT_SSP_TABLE, (unsigned long)ist_ssp); + } + BUILD_BUG_ON(sizeof(*tss) <= 0x67); /* Mandated by the architecture. */ _set_tssldt_desc(gdt + TSS_ENTRY, (unsigned long)tss, diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index bc44d865ef..4e2c3c9735 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -6000,12 +6000,28 @@ void memguard_unguard_range(void *p, unsigned long l) #endif -void memguard_guard_stack(void *p) +static void write_sss_token(unsigned long *ptr) { - map_pages_to_xen((unsigned long)p, virt_to_mfn(p), 1, _PAGE_NONE); + /* + * A supervisor shadow stack token is its own linear address, with the + * busy bit (0) clear. + */ + *ptr = (unsigned long)ptr; +} +void memguard_guard_stack(void *p) +{ + /* IST Shadow stacks. 4x 1k in stack page 0. */ + write_sss_token(p + 0x3f8); + write_sss_token(p + 0x7f8); + write_sss_token(p + 0xbf8); + write_sss_token(p + 0xff8); + map_pages_to_xen((unsigned long)p, virt_to_mfn(p), 1, PAGE_HYPERVISOR_SHSTK); + + /* Primary Shadow Stack. 1x 4k in stack page 5. */ p += 5 * PAGE_SIZE; - map_pages_to_xen((unsigned long)p, virt_to_mfn(p), 1, _PAGE_NONE); + write_sss_token(p + 0xff8); + map_pages_to_xen((unsigned long)p, virt_to_mfn(p), 1, PAGE_HYPERVISOR_SHSTK); } void memguard_unguard_stack(void *p) diff --git a/xen/include/asm-x86/page.h b/xen/include/asm-x86/page.h index 5acf3d3d5a..f632affaef 100644 --- a/xen/include/asm-x86/page.h +++ b/xen/include/asm-x86/page.h @@ -364,6 +364,7 @@ void efi_update_l4_pgtable(unsigned int l4idx, l4_pgentry_t); _PAGE_DIRTY | _PAGE_RW) #define __PAGE_HYPERVISOR_UCMINUS (__PAGE_HYPERVISOR | _PAGE_PCD) #define __PAGE_HYPERVISOR_UC (__PAGE_HYPERVISOR | _PAGE_PCD | _PAGE_PWT) +#define __PAGE_HYPERVISOR_SHSTK (__PAGE_HYPERVISOR_RO | _PAGE_DIRTY) #define MAP_SMALL_PAGES _PAGE_AVAIL0 /* don't use superpages mappings */ diff --git a/xen/include/asm-x86/processor.h b/xen/include/asm-x86/processor.h index f7e80d12e4..54e1a8b605 100644 --- a/xen/include/asm-x86/processor.h +++ b/xen/include/asm-x86/processor.h @@ -434,7 +434,8 @@ struct __packed tss64 { uint16_t :16, bitmap; }; struct tss_page { - struct tss64 __aligned(PAGE_SIZE) tss; + uint64_t __aligned(PAGE_SIZE) ist_ssp[8]; + struct tss64 tss; }; DECLARE_PER_CPU(struct tss_page, tss_page); diff --git a/xen/include/asm-x86/x86_64/page.h b/xen/include/asm-x86/x86_64/page.h index 9876634881..26621f9519 100644 --- a/xen/include/asm-x86/x86_64/page.h +++ b/xen/include/asm-x86/x86_64/page.h @@ -171,6 +171,7 @@ static inline intpte_t put_pte_flags(unsigned int x) #define PAGE_HYPERVISOR_RW (__PAGE_HYPERVISOR_RW | _PAGE_GLOBAL) #define PAGE_HYPERVISOR_RX (__PAGE_HYPERVISOR_RX | _PAGE_GLOBAL) #define PAGE_HYPERVISOR_RWX (__PAGE_HYPERVISOR | _PAGE_GLOBAL) +#define PAGE_HYPERVISOR_SHSTK (__PAGE_HYPERVISOR_SHSTK | _PAGE_GLOBAL) #define PAGE_HYPERVISOR PAGE_HYPERVISOR_RW #define PAGE_HYPERVISOR_UCMINUS (__PAGE_HYPERVISOR_UCMINUS | \ From patchwork Fri May 1 22:58:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523405 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 78E4E81 for ; Fri, 1 May 2020 23:01:04 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 54DA6208DB for ; Fri, 1 May 2020 23:01:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="ErI7A2pb" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 54DA6208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUeci-0007xH-PR; Fri, 01 May 2020 22:59:12 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUech-0007wd-3P for xen-devel@lists.xenproject.org; Fri, 01 May 2020 22:59:11 +0000 X-Inumbo-ID: 58e26510-8bff-11ea-b07b-bc764e2007e4 Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 58e26510-8bff-11ea-b07b-bc764e2007e4; Fri, 01 May 2020 22:59:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588373941; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Ap0r5eSmJ8rufFjf/AMweZgfzQrJaoB/RQzTcg9JGIc=; b=ErI7A2pbb2ioIashxwuWHNLQAyklofIDa/D2N96acCgo3k+vCBkNV2gK gmAuAy5i5iA+cQnrVzmK88VQ/LTbg75b+zi+1DEK5gVd0h5Aj9ysdA2Zo R8289BthNZdqkpTON6atESa0yYzB1en2ytYN+Xhv7iVQMZrSrcDJ7Zvod Y=; Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa3.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: l2SQ2JLuOddSOfsNIBJCTSLhOtnmkBvFgvONzvDTCcY8Vkp/NtCl2khdZ9t2t6JtA7zD4GarrM BqM+kxLU7j90gPA9Mp6887De5bHXiO5NWfRNikGaWTDozSZykal5kSAwGFRXOF2TRn2G8/vW9i psvTC6stvSV5iP+ognHDjvS9c8hUo0/bVTGX/mip9oFyenMUwKfYF0xuGotxrfdCAmi16TcbUO IZ+h13PiXE4v3ZHgGZU4m+eNTts24WDvxeJeeOa5GzXBA0gSoFcgRtRKUSN69nEpr7Ozrc7ZXV m00= X-SBRS: 2.7 X-MesageID: 16584677 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="16584677" From: Andrew Cooper To: Xen-devel Subject: [PATCH 09/16] x86/cpu: Adjust enable_nmis() to be shadow stack compatible Date: Fri, 1 May 2020 23:58:31 +0100 Message-ID: <20200501225838.9866-10-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" When executing an IRET-to-self, the shadow stack must agree with the regular stack. We can't manipulate SSP directly, so have to fake a shadow IRET frame by executing 3 CALLs, then editing the result to look correct. This is not a fastpath, is called on the BSP long before CET can be set up, and may be called on the crash path after CET is disabled. Use the fact that INCSSP is allocated from the hint nop space to construct a test for CET being active which is safe on all processors. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/include/asm-x86/processor.h | 43 +++++++++++++++++++++++++++++++---------- 1 file changed, 33 insertions(+), 10 deletions(-) diff --git a/xen/include/asm-x86/processor.h b/xen/include/asm-x86/processor.h index 54e1a8b605..654d46a6f4 100644 --- a/xen/include/asm-x86/processor.h +++ b/xen/include/asm-x86/processor.h @@ -544,17 +544,40 @@ static inline void enable_nmis(void) { unsigned long tmp; - asm volatile ( "mov %%rsp, %[tmp] \n\t" - "push %[ss] \n\t" - "push %[tmp] \n\t" - "pushf \n\t" - "push %[cs] \n\t" - "lea 1f(%%rip), %[tmp] \n\t" - "push %[tmp] \n\t" - "iretq; 1: \n\t" - : [tmp] "=&r" (tmp) + asm volatile ( "mov %%rsp, %[rsp] \n\t" + "lea .Ldone(%%rip), %[rip] \n\t" +#ifdef CONFIG_XEN_SHSTK + /* Check for CET-SS being active. */ + "mov $1, %k[ssp] \n\t" + "rdsspq %[ssp] \n\t" + "cmp $1, %k[ssp] \n\t" + "je .Lshstk_done \n\t" + + /* Push 3 words on the shadow stack */ + ".rept 3 \n\t" + "call 1f; nop; 1: \n\t" + ".endr \n\t" + + /* Fixup to be an IRET shadow stack frame */ + "wrssq %q[cs], -1*8(%[ssp]) \n\t" + "wrssq %[rip], -2*8(%[ssp]) \n\t" + "wrssq %[ssp], -3*8(%[ssp]) \n\t" + + ".Lshstk_done:" +#endif + /* Write an IRET regular frame */ + "push %[ss] \n\t" + "push %[rsp] \n\t" + "pushf \n\t" + "push %q[cs] \n\t" + "push %[rip] \n\t" + "iretq \n\t" + ".Ldone: \n\t" + : [rip] "=&r" (tmp), + [rsp] "=&r" (tmp), + [ssp] "=&r" (tmp) : [ss] "i" (__HYPERVISOR_DS), - [cs] "i" (__HYPERVISOR_CS) ); + [cs] "r" (__HYPERVISOR_CS) ); } void sysenter_entry(void); From patchwork Fri May 1 22:58:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523411 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 521B881 for ; Fri, 1 May 2020 23:04:50 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2E02C208DB for ; Fri, 1 May 2020 23:04:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="ca/Y9wdg" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2E02C208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUehv-0001G4-UQ; Fri, 01 May 2020 23:04:35 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUehu-0001FM-JM for xen-devel@lists.xenproject.org; Fri, 01 May 2020 23:04:34 +0000 X-Inumbo-ID: 1aa23f5e-8c00-11ea-ae69-bc764e2007e4 Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 1aa23f5e-8c00-11ea-ae69-bc764e2007e4; Fri, 01 May 2020 23:04:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588374266; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=uwqto+nsVNLj4DX0sYxzltmriQwYE0YYvDMISJG9Vwo=; b=ca/Y9wdg9cmMQpC3zP2Ui4KH0I8NVjJXvc0kr3vkrt3YLGY2usoshxAX nhfOsQW3Cku71pAPLN/s+m5eBRf+kvHP+SBlA5zqYSN+8KYbZYor0ro4I DyI23g5Tf59uthtG8myT92ICjOsJQEXCIuhMgjH4N2KjVr/Zgt0vRVxhq 4=; Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa1.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: vyZJf3XChK3+6Biz+YWJLkqCcBFG7t+PFauyzWQfKb9zrybxorqUy3rh0tRdcOmcwU3CGgISvh AH+Bh7Ql2I3ViuWPd3lwaZ/ovOI8NPHBZEFMxWfxzDjTzc2pnXc7vFq0zTZrOcincvZo0Ph3FH tCaVyfz5hEEiOHBL60lcUiC8CQCTswwturEabnlNLXsLXQsfyZh2bRhBM6jxWb4HLI9q9hyu4N Be8ZGyyI27g3DUZr1FjC13O0uesvVRjPAJPRoplG9UjBwr3UaYkomzHNjiZJTT7JJkg6DikVbw oWI= X-SBRS: 2.7 X-MesageID: 16855103 X-Ironport-Server: esa1.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="16855103" From: Andrew Cooper To: Xen-devel Subject: [PATCH 10/16] x86/cpu: Adjust reset_stack_and_jump() to be shadow stack compatible Date: Fri, 1 May 2020 23:58:32 +0100 Message-ID: <20200501225838.9866-11-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" We need to unwind up to the supervisor token. See the comment for details. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/include/asm-x86/current.h | 42 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 39 insertions(+), 3 deletions(-) diff --git a/xen/include/asm-x86/current.h b/xen/include/asm-x86/current.h index 99b66a0087..2a7b728b1e 100644 --- a/xen/include/asm-x86/current.h +++ b/xen/include/asm-x86/current.h @@ -124,13 +124,49 @@ unsigned long get_stack_dump_bottom (unsigned long sp); # define CHECK_FOR_LIVEPATCH_WORK "" #endif +#ifdef CONFIG_XEN_SHSTK +/* + * We need to unwind the primary shadow stack to its supervisor token, located + * at 0x5ff8 from the base of the stack blocks. + * + * Read the shadow stack pointer, subtract it from 0x5ff8, divide by 8 to get + * the number of slots needing popping. + * + * INCSSPQ can't pop more than 255 entries. We shouldn't ever need to pop + * that many entries, and getting this wrong will cause us to #DF later. + */ +# define SHADOW_STACK_WORK \ + "mov $1, %[ssp];" \ + "rdsspd %[ssp];" \ + "cmp $1, %[ssp];" \ + "je 1f;" /* CET not active? Skip. */ \ + "mov $"STR(0x5ff8)", %[val];" \ + "and $"STR(STACK_SIZE - 1)", %[ssp];" \ + "sub %[ssp], %[val];" \ + "shr $3, %[val];" \ + "cmp $255, %[val];" \ + "jle 2f;" \ + "ud2a;" \ + "2: incsspq %q[val];" \ + "1:" +#else +# define SHADOW_STACK_WORK "" +#endif + #define switch_stack_and_jump(fn, instr) \ ({ \ + unsigned int tmp; \ __asm__ __volatile__ ( \ - "mov %0,%%"__OP"sp;" \ + "cmc;" \ + SHADOW_STACK_WORK \ + "mov %[stk], %%rsp;" \ instr \ - "jmp %c1" \ - : : "r" (guest_cpu_user_regs()), "i" (fn) : "memory" ); \ + "jmp %c[fun];" \ + : [val] "=&r" (tmp), \ + [ssp] "=&r" (tmp) \ + : [stk] "r" (guest_cpu_user_regs()), \ + [fun] "i" (fn) \ + : "memory" ); \ unreachable(); \ }) From patchwork Fri May 1 22:58:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523419 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B585A81 for ; Fri, 1 May 2020 23:05:26 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 92206208DB for ; Fri, 1 May 2020 23:05:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="VNJPbmkv" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 92206208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUei3-0001JS-7D; Fri, 01 May 2020 23:04:43 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUei2-0001It-1o for xen-devel@lists.xenproject.org; Fri, 01 May 2020 23:04:42 +0000 X-Inumbo-ID: 224a49b9-8c00-11ea-9b70-12813bfff9fa Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 224a49b9-8c00-11ea-9b70-12813bfff9fa; Fri, 01 May 2020 23:04:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588374281; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yRFj9eeWE6eYBW6P5Xnzqp10Fr4wYOcVKuNdQFor0Io=; b=VNJPbmkvQgE5brThFiytUokhbVXDGIEIJEUABfEjVNjoaVr+PW33VVwu NL8Ju50UGhtNhW63wQXi0NFbAzGqtKifBYAP+K77LEhN+wNYkhQOfdzp0 7L/X9lMAYrHKLboA6bkdOy8xlIADP4ehPprUHq0cVd/wRZAcvNOqqWUeE w=; Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa6.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: 2i7piaTx+JM8Ul2K6o4UR4T0PTd0cCe5iJrxEMSC3awac4iuUrQcAQeGW7Yrs3Agn+wv4pZ2wa R1CTWdRp2YrDB3jQm8kZKJWX4qQVJyqaxzSaAJ8J1SBwQEx7SX407VXgjN3bBhhspUd1+qxrPx ARkljdaBFO6B3G6gw9/4al0M8dQ7BLBCxPURI3LCxQ/MuY+MVP8rFJCIckc6hymGNNkeVPlqiq 2xUStgugFXq/EOTe2DKfhb1ajO4x5+zN8lFoBE1oul2xUeOxBmGCc3l2lt/VKSJV6pIEB59AJt BY0= X-SBRS: 2.7 X-MesageID: 16995035 X-Ironport-Server: esa6.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="16995035" From: Andrew Cooper To: Xen-devel Subject: [PATCH 11/16] x86/spec-ctrl: Adjust DO_OVERWRITE_RSB to be shadow stack compatible Date: Fri, 1 May 2020 23:58:33 +0100 Message-ID: <20200501225838.9866-12-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The 32 calls need dropping from the shadow stack as well as the regular stack. To shorten the code, we can use the 32bit forms of RDSSP/INCSSP, but need to double up the input to INCSSP to counter the operand size based multiplier. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/include/asm-x86/spec_ctrl_asm.h | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h index c60093b090..cb34299a86 100644 --- a/xen/include/asm-x86/spec_ctrl_asm.h +++ b/xen/include/asm-x86/spec_ctrl_asm.h @@ -83,9 +83,9 @@ * Requires nothing * Clobbers \tmp (%rax by default), %rcx * - * Requires 256 bytes of stack space, but %rsp has no net change. Based on - * Google's performance numbers, the loop is unrolled to 16 iterations and two - * calls per iteration. + * Requires 256 bytes of {,shadow}stack space, but %rsp/SSP has no net + * change. Based on Google's performance numbers, the loop is unrolled to 16 + * iterations and two calls per iteration. * * The call filling the RSB needs a nonzero displacement. A nop would do, but * we use "1: pause; lfence; jmp 1b" to safely contains any ret-based @@ -114,6 +114,16 @@ sub $1, %ecx jnz .L\@_fill_rsb_loop mov %\tmp, %rsp /* Restore old %rsp */ + +#ifdef CONFIG_XEN_SHSTK + mov $1, %ecx + rdsspd %ecx + cmp $1, %ecx + je .L\@_shstk_done + mov $64, %ecx /* 64 * 4 bytes, given incsspd */ + incsspd %ecx /* Restore old SSP */ +.L\@_shstk_done: +#endif .endm .macro DO_SPEC_CTRL_ENTRY_FROM_HVM From patchwork Fri May 1 22:58:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523421 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E19C214B4 for ; Fri, 1 May 2020 23:05:26 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BEE2F208DB for ; Fri, 1 May 2020 23:05:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="eOVxo4n+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BEE2F208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUeho-0001Bq-9T; Fri, 01 May 2020 23:04:28 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUehn-0001BR-50 for xen-devel@lists.xenproject.org; Fri, 01 May 2020 23:04:27 +0000 X-Inumbo-ID: 1a67054c-8c00-11ea-9b70-12813bfff9fa Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 1a67054c-8c00-11ea-9b70-12813bfff9fa; Fri, 01 May 2020 23:04:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588374266; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xNDjbEw11ojfVPqDcOxo4l2IKtjrw8Xz8xcuwlJlm7I=; b=eOVxo4n+9/viNDAsWhSpUoPBcSA59fnHUISp3LXnZfWKMoFNgl87Imud T76m4Z/2FwXqpTIKEIB76Kd1K0DumP8ZwZWBGayH4TRgwapFo6EWMKMdw JeM2v7SjYBZyB9x2dRltNYcvtrTXwIuAVyUuy//YWWhHwYLzpk8yx+ZNj 8=; Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: RQO3kNgN7pqeaRKjD9Q7Gzk0acvKAER0HyP3NHfdCg9AB4epBxfWQjxpqYwfziLU6MdX16WBTJ m7hpKTIs/06zkqPrwoy3oS5boN20Wlm8uHuxFZDZcx8vQaXXvLB2NXEYjwYsiEZVOwvvLgtwGT 9LV4M8CLgpvLmAljGbxKnZTLrcvmZLaS9N31HkYU3sWXcTzPB3wrcVwfKxrM9XG0x36zjjoDW4 VBv/6wK9MncRLIlOKplp+a8VKMEnCrxKzzAeNz3FFmHDfCXM788JCNHWizvayskqdZeJwtXzUY 4CA= X-SBRS: 2.7 X-MesageID: 16906065 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="16906065" From: Andrew Cooper To: Xen-devel Subject: [PATCH 12/16] x86/extable: Adjust extable handling to be shadow stack compatible Date: Fri, 1 May 2020 23:58:34 +0100 Message-ID: <20200501225838.9866-13-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" When adjusting an IRET frame to recover from a fault, and equivalent adjustment needs making in the shadow IRET frame. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/traps.c | 22 ++++++++++++++++++++++ xen/arch/x86/x86_64/entry.S | 11 ++++++++++- 2 files changed, 32 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c index 1cf00c1f4a..2354357cc1 100644 --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -778,6 +778,28 @@ static bool exception_fixup(struct cpu_user_regs *regs, bool print) vec_name(regs->entry_vector), regs->error_code, _p(regs->rip), _p(regs->rip), _p(fixup)); + if ( IS_ENABLED(CONFIG_XEN_SHSTK) ) + { + unsigned long ssp; + + asm ("rdsspq %0" : "=r" (ssp) : "0" (1) ); + if ( ssp != 1 ) + { + unsigned long *ptr = _p(ssp); + + /* Search for %rip in the shadow stack, ... */ + while ( *ptr != regs->rip ) + ptr++; + + ASSERT(ptr[1] == __HYPERVISOR_CS); + + /* ... and adjust to the fixup location. */ + asm ("wrssq %[fix], %[stk]" + : [stk] "=m" (*ptr) + : [fix] "r" (fixup)); + } + } + regs->rip = fixup; return true; diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S index 6403c0ab92..06da350ba0 100644 --- a/xen/arch/x86/x86_64/entry.S +++ b/xen/arch/x86/x86_64/entry.S @@ -708,7 +708,16 @@ exception_with_ints_disabled: call search_pre_exception_table testq %rax,%rax # no fixup code for faulting EIP? jz 1b - movq %rax,UREGS_rip(%rsp) + movq %rax,UREGS_rip(%rsp) # fixup regular stack + +#ifdef CONFIG_XEN_SHSTK + mov $1, %edi + rdsspq %rdi + cmp $1, %edi + je .L_exn_shstk_done + wrssq %rax, (%rdi) # fixup shadow stack +.L_exn_shstk_done: +#endif subq $8,UREGS_rsp(%rsp) # add ec/ev to previous stack frame testb $15,UREGS_rsp(%rsp) # return %rsp is now aligned? jz 1f # then there is a pad quadword already From patchwork Fri May 1 22:58:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523417 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5727313B2 for ; Fri, 1 May 2020 23:05:25 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2855F208DB for ; Fri, 1 May 2020 23:05:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="iXGR5INg" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2855F208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUei4-0001KJ-G0; Fri, 01 May 2020 23:04:44 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUei2-0001JE-Ud for xen-devel@lists.xenproject.org; Fri, 01 May 2020 23:04:42 +0000 X-Inumbo-ID: 23f6d6f0-8c00-11ea-9b70-12813bfff9fa Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 23f6d6f0-8c00-11ea-9b70-12813bfff9fa; Fri, 01 May 2020 23:04:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588374282; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VTFDnoal8uf23mjLhPOcZyRMpnFsT5lsnYXHVwdqsjQ=; b=iXGR5INgY6IsJTqgU5U0m96sGcM65YqJFlHDsJVfUDQEQT2bTBefk84h CgPr1SEbpwYsd/iSCDfjIhcLtSWDd5//0XyvXSr6RLLS7H/nrwyqQZLCS S3Va8RIJ3JHnhKYnk5nXULzc7YC3ux3hOMSjBSL2DdK8olR+Q/Gl9DUQf s=; Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa6.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: tcUAXi3qqq2QFegR7Dr5w7LqqzPZbEH3INzJFOcZ7ggbdbyDPEu2JiVPCJu7W+lB8aSaYbcjp3 ghu7Kun0k2rPVCAmuCXw/nGsEbp0MadK4aIZVUYC5cuKpbXAjmjaC+yXHXLkGMmhqwFWJkRy3W sNDSLLPX6aS5bFQevEC+M/20+8Mi/8ihFzxrB0H2f4cdL0IOzU3q70XtFxk3/bb99q7y3UbSwL nSeSenWENhksll65mhdlmACEFbjKcslcREi7707oGG37RL8HJ3bn+LPCCG/YXlSBtfvyWPewJc 8s4= X-SBRS: 2.7 X-MesageID: 16995031 X-Ironport-Server: esa6.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="16995031" From: Andrew Cooper To: Xen-devel Subject: [PATCH 13/16] x86/ioemul: Rewrite stub generation to be shadow stack compatible Date: Fri, 1 May 2020 23:58:35 +0100 Message-ID: <20200501225838.9866-14-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The logic is completely undocumented and almost impossible to follow. It actually uses return oriented programming. Rewrite it to conform to more normal call mechanics, and leave a big comment explaining thing. As well as the code being easier to follow, it will execute faster as it isn't fighting the branch predictor. Move the ioemul_handle_quirk() function pointer from traps.c to ioport_emulate.c. There is no reason for it to be in neither of the two translation units which use it. Alter the behaviour to return the number of bytes written into the stub. Access the addresses of the host/guest helpers with extern const char arrays. Nothing good will come of C thinking they are regular functions. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné Posted previously on its perf benefits alone, but here is the real reason behind the change. --- xen/arch/x86/ioport_emulate.c | 11 ++--- xen/arch/x86/pv/emul-priv-op.c | 91 +++++++++++++++++++++++++++++++----------- xen/arch/x86/pv/gpr_switch.S | 37 +++++------------ xen/arch/x86/traps.c | 3 -- xen/include/asm-x86/io.h | 3 +- 5 files changed, 85 insertions(+), 60 deletions(-) diff --git a/xen/arch/x86/ioport_emulate.c b/xen/arch/x86/ioport_emulate.c index 499c1f6056..f7511a9c49 100644 --- a/xen/arch/x86/ioport_emulate.c +++ b/xen/arch/x86/ioport_emulate.c @@ -8,7 +8,10 @@ #include #include -static bool ioemul_handle_proliant_quirk( +unsigned int (*ioemul_handle_quirk)( + u8 opcode, char *io_emul_stub, struct cpu_user_regs *regs); + +static unsigned int ioemul_handle_proliant_quirk( u8 opcode, char *io_emul_stub, struct cpu_user_regs *regs) { static const char stub[] = { @@ -19,18 +22,16 @@ static bool ioemul_handle_proliant_quirk( 0xa8, 0x80, /* test $0x80, %al */ 0x75, 0xfb, /* jnz 1b */ 0x9d, /* popf */ - 0xc3, /* ret */ }; uint16_t port = regs->dx; uint8_t value = regs->al; if ( (opcode != 0xee) || (port != 0xcd4) || !(value & 0x80) ) - return false; + return 0; memcpy(io_emul_stub, stub, sizeof(stub)); - BUILD_BUG_ON(IOEMUL_QUIRK_STUB_BYTES < sizeof(stub)); - return true; + return sizeof(stub); } /* This table is the set of system-specific I/O emulation hooks. */ diff --git a/xen/arch/x86/pv/emul-priv-op.c b/xen/arch/x86/pv/emul-priv-op.c index e24b84f46a..f150886711 100644 --- a/xen/arch/x86/pv/emul-priv-op.c +++ b/xen/arch/x86/pv/emul-priv-op.c @@ -54,51 +54,96 @@ struct priv_op_ctxt { unsigned int bpmatch; }; -/* I/O emulation support. Helper routines for, and type of, the stack stub. */ -void host_to_guest_gpr_switch(struct cpu_user_regs *); -unsigned long guest_to_host_gpr_switch(unsigned long); +/* I/O emulation helpers. Use non-standard calling conventions. */ +extern const char load_guest_gprs[], save_guest_gprs[]; typedef void io_emul_stub_t(struct cpu_user_regs *); static io_emul_stub_t *io_emul_stub_setup(struct priv_op_ctxt *ctxt, u8 opcode, unsigned int port, unsigned int bytes) { + /* + * Construct a stub for IN/OUT emulation. + * + * Some platform drivers communicate with the SMM handler using GPRs as a + * mailbox. Therefore, we must perform the emulation with the hardware + * domain's registers in view. + * + * We write a stub of the following form, using the guest load/save + * helpers (abnormal calling conventions), and one of several possible + * stubs performing the real I/O. + */ + static const char prologue[] = { + 0x53, /* push %rbx */ + 0x55, /* push %rbp */ + 0x41, 0x54, /* push %r12 */ + 0x41, 0x55, /* push %r13 */ + 0x41, 0x56, /* push %r14 */ + 0x41, 0x57, /* push %r15 */ + 0x57, /* push %rdi (param for save_guest_gprs) */ + }; /* call load_guest_gprs */ + /* */ + /* call save_guest_gprs */ + static const char epilogue[] = { + 0x5f, /* pop %rdi */ + 0x41, 0x5f, /* pop %r15 */ + 0x41, 0x5e, /* pop %r14 */ + 0x41, 0x5d, /* pop %r13 */ + 0x41, 0x5c, /* pop %r12 */ + 0x5d, /* pop %rbp */ + 0x5b, /* pop %rbx */ + 0xc3, /* ret */ + }; + struct stubs *this_stubs = &this_cpu(stubs); unsigned long stub_va = this_stubs->addr + STUB_BUF_SIZE / 2; - long disp; - bool use_quirk_stub = false; + unsigned int quirk_bytes = 0; + char *p; + + /* Helpers - Read outer scope but only modify p. */ +#define APPEND_BUFF(b) ({ memcpy(p, b, sizeof(b)); p += sizeof(b); }) +#define APPEND_CALL(f) \ + ({ \ + long disp = (long)(f) - (stub_va + p - ctxt->io_emul_stub + 5); \ + BUG_ON((int32_t)disp != disp); \ + *p++ = 0xe8; \ + *(int32_t *)p = disp; p += 4; \ + }) if ( !ctxt->io_emul_stub ) ctxt->io_emul_stub = map_domain_page(_mfn(this_stubs->mfn)) + (stub_va & ~PAGE_MASK); - /* call host_to_guest_gpr_switch */ - ctxt->io_emul_stub[0] = 0xe8; - disp = (long)host_to_guest_gpr_switch - (stub_va + 5); - BUG_ON((int32_t)disp != disp); - *(int32_t *)&ctxt->io_emul_stub[1] = disp; + p = ctxt->io_emul_stub; + + APPEND_BUFF(prologue); + APPEND_CALL(load_guest_gprs); + /* Some platforms might need to quirk the stub for specific inputs. */ if ( unlikely(ioemul_handle_quirk) ) - use_quirk_stub = ioemul_handle_quirk(opcode, &ctxt->io_emul_stub[5], - ctxt->ctxt.regs); + { + quirk_bytes = ioemul_handle_quirk(opcode, p, ctxt->ctxt.regs); + p += quirk_bytes; + } - if ( !use_quirk_stub ) + /* Default I/O stub. */ + if ( likely(!quirk_bytes) ) { - /* data16 or nop */ - ctxt->io_emul_stub[5] = (bytes != 2) ? 0x90 : 0x66; - /* */ - ctxt->io_emul_stub[6] = opcode; - /* imm8 or nop */ - ctxt->io_emul_stub[7] = !(opcode & 8) ? port : 0x90; - /* ret (jumps to guest_to_host_gpr_switch) */ - ctxt->io_emul_stub[8] = 0xc3; + *p++ = (bytes != 2) ? 0x90 : 0x66; /* data16 or nop */ + *p++ = opcode; /* */ + *p++ = !(opcode & 8) ? port : 0x90; /* imm8 or nop */ } - BUILD_BUG_ON(STUB_BUF_SIZE / 2 < MAX(9, /* Default emul stub */ - 5 + IOEMUL_QUIRK_STUB_BYTES)); + APPEND_CALL(save_guest_gprs); + APPEND_BUFF(epilogue); + + BUG_ON(STUB_BUF_SIZE / 2 < (p - ctxt->io_emul_stub)); /* Handy function-typed pointer to the stub. */ return (void *)stub_va; + +#undef APPEND_CALL +#undef APPEND_BUFF } diff --git a/xen/arch/x86/pv/gpr_switch.S b/xen/arch/x86/pv/gpr_switch.S index 6d26192c2c..e3f8037b69 100644 --- a/xen/arch/x86/pv/gpr_switch.S +++ b/xen/arch/x86/pv/gpr_switch.S @@ -9,59 +9,42 @@ #include -ENTRY(host_to_guest_gpr_switch) - movq (%rsp), %rcx - movq %rdi, (%rsp) +/* Load guest GPRs. Parameter in %rdi, clobbers all registers. */ +ENTRY(load_guest_gprs) movq UREGS_rdx(%rdi), %rdx - pushq %rbx movq UREGS_rax(%rdi), %rax movq UREGS_rbx(%rdi), %rbx - pushq %rbp movq UREGS_rsi(%rdi), %rsi movq UREGS_rbp(%rdi), %rbp - pushq %r12 - movq UREGS_r8(%rdi), %r8 + movq UREGS_r8 (%rdi), %r8 movq UREGS_r12(%rdi), %r12 - pushq %r13 - movq UREGS_r9(%rdi), %r9 + movq UREGS_r9 (%rdi), %r9 movq UREGS_r13(%rdi), %r13 - pushq %r14 movq UREGS_r10(%rdi), %r10 movq UREGS_r14(%rdi), %r14 - pushq %r15 movq UREGS_r11(%rdi), %r11 movq UREGS_r15(%rdi), %r15 - pushq %rcx /* dummy push, filled by guest_to_host_gpr_switch pointer */ - pushq %rcx - leaq guest_to_host_gpr_switch(%rip),%rcx - movq %rcx,8(%rsp) movq UREGS_rcx(%rdi), %rcx movq UREGS_rdi(%rdi), %rdi ret -ENTRY(guest_to_host_gpr_switch) +/* Save guest GPRs. Parameter on the stack above the return address. */ +ENTRY(save_guest_gprs) pushq %rdi - movq 7*8(%rsp), %rdi + movq 2*8(%rsp), %rdi movq %rax, UREGS_rax(%rdi) - popq UREGS_rdi(%rdi) + popq UREGS_rdi(%rdi) movq %r15, UREGS_r15(%rdi) movq %r11, UREGS_r11(%rdi) - popq %r15 movq %r14, UREGS_r14(%rdi) movq %r10, UREGS_r10(%rdi) - popq %r14 movq %r13, UREGS_r13(%rdi) - movq %r9, UREGS_r9(%rdi) - popq %r13 + movq %r9, UREGS_r9 (%rdi) movq %r12, UREGS_r12(%rdi) - movq %r8, UREGS_r8(%rdi) - popq %r12 + movq %r8, UREGS_r8 (%rdi) movq %rbp, UREGS_rbp(%rdi) movq %rsi, UREGS_rsi(%rdi) - popq %rbp movq %rbx, UREGS_rbx(%rdi) movq %rdx, UREGS_rdx(%rdi) - popq %rbx movq %rcx, UREGS_rcx(%rdi) - popq %rcx ret diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c index 2354357cc1..3923950df7 100644 --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -117,9 +117,6 @@ idt_entry_t *idt_tables[NR_CPUS] __read_mostly; */ DEFINE_PER_CPU_PAGE_ALIGNED(struct tss_page, tss_page); -bool (*ioemul_handle_quirk)( - u8 opcode, char *io_emul_stub, struct cpu_user_regs *regs); - static int debug_stack_lines = 20; integer_param("debug_stack_lines", debug_stack_lines); diff --git a/xen/include/asm-x86/io.h b/xen/include/asm-x86/io.h index 8708b79b99..c4ec52cba7 100644 --- a/xen/include/asm-x86/io.h +++ b/xen/include/asm-x86/io.h @@ -49,8 +49,7 @@ __OUT(w,"w",short) __OUT(l,,int) /* Function pointer used to handle platform specific I/O port emulation. */ -#define IOEMUL_QUIRK_STUB_BYTES 10 -extern bool (*ioemul_handle_quirk)( +extern unsigned int (*ioemul_handle_quirk)( u8 opcode, char *io_emul_stub, struct cpu_user_regs *regs); #endif From patchwork Fri May 1 22:58:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523413 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8983081 for ; Fri, 1 May 2020 23:05:05 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 660B6208DB for ; Fri, 1 May 2020 23:05:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="iOc9QmKx" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 660B6208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUehg-0001AP-P4; Fri, 01 May 2020 23:04:20 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUehf-0001AF-N1 for xen-devel@lists.xenproject.org; Fri, 01 May 2020 23:04:19 +0000 X-Inumbo-ID: 15fcccbc-8c00-11ea-9887-bc764e2007e4 Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 15fcccbc-8c00-11ea-9887-bc764e2007e4; Fri, 01 May 2020 23:04:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588374258; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=HTUK8m+QrV1a4/c+FcbBl6B+TxDRXiRJZVfXM92PQy0=; b=iOc9QmKxzAeTJCrxrx/MA3LkSf8Dde+9NsbXTPx7VC+uh1TNJ1r4ZSth cnMZLPIFRVYxoxolviwSMGZB/OGw3yD2TTR8r+OKo2WtgHJ3PZByi2q29 ZIJl+1fuy2gzZxNaDsCmKgG5XIW/+WaD8zePshBmg/3NsGwsBh7ekJU2B k=; Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa4.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: LwgmbsDFxdS+tPhQKNfQOdklFvxXZDioEMP9BKxTatOmGiSlPKvotKZZwqe3z+LlvdvTNSoFug NJ2mHwyf1Ao8tw+o6ofA7ZWar0nPm/qP15wYYp3kPU9d9mj5tHrECaarbawvhyN6nNML/mr2zj nV6vc007LnblHHWt4VgzZNHORmk98DXVTGnb9tKQ23wuwFb3Z69Sj6/2mDLXs0eRRopJDOE/J5 bDq1LMsTN2dNfIyG4ZSHRsaXwMf2x/ZRetoDZ9OmSoGfZ/vGglnjx+g51ZgbGV/iEsQcEvHcRj Xzk= X-SBRS: 2.7 X-MesageID: 17294144 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="17294144" From: Andrew Cooper To: Xen-devel Subject: [PATCH 14/16] x86/alt: Adjust _alternative_instructions() to not create shadow stacks Date: Fri, 1 May 2020 23:58:36 +0100 Message-ID: <20200501225838.9866-15-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The current alternatives algorithm clears CR0.WP and writes into .text. This has a side effect of the mappings becoming shadow stacks once CET is active. Adjust _alternative_instructions() to clean up after itself. This involves extending the set of bits modify_xen_mappings() to include Dirty (and Accessed for good measure). Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/alternative.c | 14 ++++++++++++++ xen/arch/x86/mm.c | 6 +++--- 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/xen/arch/x86/alternative.c b/xen/arch/x86/alternative.c index ce2b4302e6..004e9ede25 100644 --- a/xen/arch/x86/alternative.c +++ b/xen/arch/x86/alternative.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -398,6 +399,19 @@ static void __init _alternative_instructions(bool force) panic("Timed out waiting for alternatives self-NMI to hit\n"); set_nmi_callback(saved_nmi_callback); + + /* + * When Xen is using shadow stacks, the alternatives clearing CR0.WP and + * writing into the mappings set dirty bits, turning the mappings into + * shadow stack mappings. + * + * While we can execute from them, this would also permit them to be the + * target of WRSS instructions, so reset the dirty after patching. + */ + if ( cpu_has_xen_shstk ) + modify_xen_mappings(XEN_VIRT_START + MB(2), + (unsigned long)&__2M_text_end, + PAGE_HYPERVISOR_RX); } void __init alternative_instructions(void) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 4e2c3c9735..26b01cb917 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -5448,8 +5448,8 @@ int populate_pt_range(unsigned long virt, unsigned long nr_mfns) * mappings, but will shatter superpages if necessary, and will destroy * mappings if not passed _PAGE_PRESENT. * - * The only flags considered are NX, RW and PRESENT. All other input flags - * are ignored. + * The only flags considered are NX, D, A, RW and PRESENT. All other input + * flags are ignored. * * It is an error to call with present flags over an unpopulated range. */ @@ -5462,7 +5462,7 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) unsigned long v = s; /* Set of valid PTE bits which may be altered. */ -#define FLAGS_MASK (_PAGE_NX|_PAGE_RW|_PAGE_PRESENT) +#define FLAGS_MASK (_PAGE_NX|_PAGE_DIRTY|_PAGE_ACCESSED|_PAGE_RW|_PAGE_PRESENT) nf &= FLAGS_MASK; ASSERT(IS_ALIGNED(s, PAGE_SIZE)); From patchwork Fri May 1 22:58:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523415 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CF70913B2 for ; Fri, 1 May 2020 23:05:11 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ABB9B208DB for ; Fri, 1 May 2020 23:05:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="KNNp+gba" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ABB9B208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUehm-0001Av-0w; Fri, 01 May 2020 23:04:26 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUehk-0001Ad-J0 for xen-devel@lists.xenproject.org; Fri, 01 May 2020 23:04:24 +0000 X-Inumbo-ID: 1779f240-8c00-11ea-9887-bc764e2007e4 Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 1779f240-8c00-11ea-9887-bc764e2007e4; Fri, 01 May 2020 23:04:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588374261; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6HdmjgF6D28Zm4TcRg1rdSfUjLRRQBlEC1IznYRSCjQ=; b=KNNp+gbapOKdSfOCeWRD8NK8qtexsKixwsdTUuELxQEIqnupsNlkQm8B s2/ML2342gRREcHTI4Ug26K3Wqq5Bgy34C7OMTLxUolB97dCUFn6Y/yKo r5Ut1BEpY/VkrTj8DiP54+Cedw1PBRkIB/xmyZW2UTa4zPFnxdHmiTsWV g=; Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa4.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: V6SFzoHHkY51sW4Cww8TJ2Z4t6bksd+bmW8hNop1qzs5IfwqLPCX6DhPY3lmIO4x0k91u4Tl+5 Q3yoaFOFI0bDWvpc32p3mXv5uDq36txwkI9u615vvf/bN1cNXNfwcPigtV6yFhjOMFAV2pO115 n4jaEypIRuhd5AWjfYWhhCqj9LDNIec9PegiXvJibGt14RJ7+WgjbN/mSkWS5TWN3IPYSGVjlt HBlgtPNhSyM8R20u8iLCWAqHq5Ie0BNsDykjyE9S22fSt13dz4BDZ4gra6gkaC2dbpsA5IVtbj CXo= X-SBRS: 2.7 X-MesageID: 17294146 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="17294146" From: Andrew Cooper To: Xen-devel Subject: [PATCH 15/16] x86/entry: Adjust guest paths to be shadow stack compatible Date: Fri, 1 May 2020 23:58:37 +0100 Message-ID: <20200501225838.9866-16-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The SYSCALL/SYSEXIT paths need to use {SET,CLR}SSBSY. The IRET to guest paths must not, which forces us to spill a register to the stack. The IST switch onto the primary stack is not great as we have an instruction boundary with no shadow stack. This is the least bad option available. These paths are not used before shadow stacks are properly established, so can use alternatives to avoid extra runtime CET detection logic. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/x86_64/compat/entry.S | 2 +- xen/arch/x86/x86_64/entry.S | 19 ++++++++++++++++++- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/x86_64/compat/entry.S b/xen/arch/x86/x86_64/compat/entry.S index 3cd375bd48..7816d0d4ac 100644 --- a/xen/arch/x86/x86_64/compat/entry.S +++ b/xen/arch/x86/x86_64/compat/entry.S @@ -198,7 +198,7 @@ ENTRY(cr4_pv32_restore) /* See lstar_enter for entry register state. */ ENTRY(cstar_enter) - /* sti could live here when we don't switch page tables below. */ + ALTERNATIVE "", "setssbsy", X86_FEATURE_XEN_SHSTK CR4_PV32_RESTORE movq 8(%rsp),%rax /* Restore %rax. */ movq $FLAT_USER_SS32, 8(%rsp) /* Assume a 64bit domain. Compat handled lower. */ diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S index 06da350ba0..91cd8f94fd 100644 --- a/xen/arch/x86/x86_64/entry.S +++ b/xen/arch/x86/x86_64/entry.S @@ -194,6 +194,15 @@ restore_all_guest: movq 8(%rsp),%rcx # RIP ja iret_exit_to_guest + /* Clear the supervisor shadow stack token busy bit. */ +.macro rag_clrssbsy + push %rax + rdsspq %rax + clrssbsy (%rax) + pop %rax +.endm + ALTERNATIVE "", rag_clrssbsy, X86_FEATURE_XEN_SHSTK + cmpw $FLAT_USER_CS32,16(%rsp)# CS movq 32(%rsp),%rsp # RSP je 1f @@ -226,7 +235,7 @@ iret_exit_to_guest: * %ss must be saved into the space left by the trampoline. */ ENTRY(lstar_enter) - /* sti could live here when we don't switch page tables below. */ + ALTERNATIVE "", "setssbsy", X86_FEATURE_XEN_SHSTK movq 8(%rsp),%rax /* Restore %rax. */ movq $FLAT_KERNEL_SS,8(%rsp) pushq %r11 @@ -877,6 +886,14 @@ handle_ist_exception: movl $UREGS_kernel_sizeof/8,%ecx movq %rdi,%rsp rep movsq + + /* Switch Shadow Stacks */ +.macro ist_switch_shstk + rdsspq %rdi + clrssbsy (%rdi) + setssbsy +.endm + ALTERNATIVE "", ist_switch_shstk, X86_FEATURE_XEN_SHSTK 1: #else ASSERT_CONTEXT_IS_XEN From patchwork Fri May 1 22:58:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11523423 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C56E881 for ; Fri, 1 May 2020 23:05:35 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 96F19208DB for ; Fri, 1 May 2020 23:05:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="Bwi9hlB9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 96F19208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUehq-0001DA-HP; Fri, 01 May 2020 23:04:30 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jUehp-0001Cf-IW for xen-devel@lists.xenproject.org; Fri, 01 May 2020 23:04:29 +0000 X-Inumbo-ID: 18c43340-8c00-11ea-ae69-bc764e2007e4 Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 18c43340-8c00-11ea-ae69-bc764e2007e4; Fri, 01 May 2020 23:04:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1588374263; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QQTKLF5FcqWscWYTybl+ldFADN5l5S3i4DlWR7dOp0U=; b=Bwi9hlB9twrgPWza/i/YntZ7SklkzWhK0zE/g/G/SlgHFMG1/egX3gKX pNcAshrDAbplO/4jVljkNkPC1zAWbTKDzYomEKd6oZtxIpiF20Y69E8TV QOJQ6WlGJkW5YTquqzoPxHYHHf/xIEr+FhrtMG06LTnJLim37yBAiurYE Q=; Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa1.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: warOANc0prrJjWlz01fdFNaC2VNLf73T7N7GydCNmFSXCFe6EDaDu2iZOgOD68YmWQ2GEUfuTM A3HcFFK3s9Rbp+jKi7I/QHNKpiiv3hEgRrValrt6B2uexJr9r9j2RQBlv18LDcAJN6qN82CoNV QwD4q0xFbOm8t9Xqc6X/l+FMOacKZQOIj+ynNDE/6wNHVHsEfc+Zezrmp0KIqz+NGH6XtijoDo Gq0NcGs3jzdFKwVRtlo9phZJaYJargH2g+P0H+mSlZWhbZTuT02AP6LyQn/c+u6tIXqkBeE+Wm eW4= X-SBRS: 2.7 X-MesageID: 16855100 X-Ironport-Server: esa1.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,341,1583211600"; d="scan'208";a="16855100" From: Andrew Cooper To: Xen-devel Subject: [PATCH 16/16] x86/shstk: Activate Supervisor Shadow Stacks Date: Fri, 1 May 2020 23:58:38 +0100 Message-ID: <20200501225838.9866-17-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200501225838.9866-1-andrew.cooper3@citrix.com> References: <20200501225838.9866-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" With all other plumbing in place, activate shadow stacks when possible. The BSP needs to wait until alternatives have run (to avoid interaction with CR0.WP), and after the first reset_stack_and_jump() to avoid having a pristine shadow stack interact in problematic ways with an in-use regular stack. Activate shadow stack in reinit_bsp_stack(). APs have all infrastructure set up by the booting CPU, so enable shadow stacks before entering C. The S3 path needs save and restore SSP along side RSP. The crash path needs to turn CET off to avoid interfereing with the kexec kernel's environment. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/acpi/wakeup_prot.S | 56 +++++++++++++++++++++++++++++++++++++++++ xen/arch/x86/boot/x86_64.S | 30 +++++++++++++++++++++- xen/arch/x86/cpu/common.c | 5 ++++ xen/arch/x86/crash.c | 7 ++++++ xen/arch/x86/setup.c | 26 +++++++++++++++++++ xen/arch/x86/spec_ctrl.c | 8 ++++++ xen/include/asm-x86/msr-index.h | 3 +++ xen/include/asm-x86/x86-defns.h | 1 + 8 files changed, 135 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/acpi/wakeup_prot.S b/xen/arch/x86/acpi/wakeup_prot.S index 4dba6020a7..22c0f8cc79 100644 --- a/xen/arch/x86/acpi/wakeup_prot.S +++ b/xen/arch/x86/acpi/wakeup_prot.S @@ -1,3 +1,8 @@ +#include +#include +#include +#include + .file __FILE__ .text .code64 @@ -15,6 +20,12 @@ ENTRY(do_suspend_lowlevel) mov %cr0, %rax mov %rax, saved_cr0(%rip) +#ifdef CONFIG_XEN_SHSTK + mov $1, %eax + rdsspq %rax + mov %rax, saved_ssp(%rip) +#endif + /* enter sleep state physically */ mov $3, %edi call acpi_enter_sleep_state @@ -48,6 +59,48 @@ ENTRY(s3_resume) pushq %rax lretq 1: +#ifdef CONFIG_XEN_SHSTK + /* + * Restoring SSP is a little convoluted, because we are intercepting + * the middle of an in-use shadow stack. Write a temporary supervisor + * token under the stack, so SETSSBSY takes us where we want, then + * reset MSR_PL0_SSP to its usual value and pop the temporary token. + */ + mov saved_rsp(%rip), %rdi + cmpq $1, %rdi + je .L_shstk_done + + /* Write a supervisor token under SSP. */ + sub $8, %rdi + mov %rdi, (%rdi) + + /* Load it into MSR_PL0_SSP. */ + mov $MSR_PL0_SSP, %ecx + mov %rdi, %rdx + shr $32, %rdx + mov %edi, %eax + + /* Enable CET. */ + mov $MSR_S_CET, %ecx + xor %edx, %edx + mov $CET_SHSTK_EN | CET_WRSS_EN, %eax + wrmsr + + /* Activate our temporary token. */ + mov $XEN_MINIMAL_CR4 | X86_CR4_CET, %ebx + mov %rbx, %cr4 + setssbsy + + /* Reset MSR_PL0_SSP back to its expected value. */ + and $~(STACK_SIZE - 1), %eax + or $0x5ff8, %eax + wrmsr + + /* Pop the temporary token off the stack. */ + mov $2, %eax + incsspd %eax +.L_shstk_done: +#endif call load_system_tables @@ -65,6 +118,9 @@ ENTRY(s3_resume) saved_rsp: .quad 0 saved_cr0: .quad 0 +#ifdef CONFIG_XEN_SHSTK +saved_ssp: .quad 0 +#endif GLOBAL(saved_magic) .long 0x9abcdef0 diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S index 314a32a19f..59b770f955 100644 --- a/xen/arch/x86/boot/x86_64.S +++ b/xen/arch/x86/boot/x86_64.S @@ -28,8 +28,36 @@ ENTRY(__high_start) lretq 1: test %ebx,%ebx - jnz start_secondary + jz .L_bsp + /* APs. Set up shadow stacks before entering C. */ + + testl $cpufeat_mask(X86_FEATURE_XEN_SHSTK), \ + CPUINFO_FEATURE_OFFSET(X86_FEATURE_XEN_SHSTK) + boot_cpu_data(%rip) + je .L_ap_shstk_done + + mov $MSR_S_CET, %ecx + xor %edx, %edx + mov $CET_SHSTK_EN | CET_WRSS_EN, %eax + wrmsr + + mov $MSR_PL0_SSP, %ecx + mov %rsp, %rdx + shr $32, %rdx + mov %esp, %eax + and $~(STACK_SIZE - 1), %eax + or $0x5ff8, %eax + wrmsr + + mov $XEN_MINIMAL_CR4 | X86_CR4_CET, %ecx + mov %rcx, %cr4 + setssbsy + +.L_ap_shstk_done: + call start_secondary + BUG /* start_secondary() shouldn't return. */ + +.L_bsp: /* Pass off the Multiboot info structure to C land (if applicable). */ mov multiboot_ptr(%rip),%edi call __start_xen diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c index 3962717aa5..a77be36349 100644 --- a/xen/arch/x86/cpu/common.c +++ b/xen/arch/x86/cpu/common.c @@ -323,6 +323,11 @@ void __init early_cpu_init(void) x86_cpuid_vendor_to_str(c->x86_vendor), c->x86, c->x86, c->x86_model, c->x86_model, c->x86_mask, eax); + if (c->cpuid_level >= 7) { + cpuid_count(7, 0, &eax, &ebx, &ecx, &edx); + c->x86_capability[cpufeat_word(X86_FEATURE_CET_SS)] = ecx; + } + eax = cpuid_eax(0x80000000); if ((eax >> 16) == 0x8000 && eax >= 0x80000008) { eax = cpuid_eax(0x80000008); diff --git a/xen/arch/x86/crash.c b/xen/arch/x86/crash.c index 450eecd46b..0611b4fb9b 100644 --- a/xen/arch/x86/crash.c +++ b/xen/arch/x86/crash.c @@ -200,6 +200,13 @@ void machine_crash_shutdown(void) /* Reset CPUID masking and faulting to the host's default. */ ctxt_switch_levelling(NULL); + /* Disable shadow stacks. */ + if ( cpu_has_xen_shstk ) + { + wrmsrl(MSR_S_CET, 0); + write_cr4(read_cr4() & ~X86_CR4_CET); + } + info = kexec_crash_save_info(); info->xen_phys_start = xen_phys_start; info->dom0_pfn_to_mfn_frame_list_list = diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index aa21201507..5c574b2035 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -664,6 +664,13 @@ static void __init noreturn reinit_bsp_stack(void) stack_base[0] = stack; memguard_guard_stack(stack); + if ( cpu_has_xen_shstk ) + { + wrmsrl(MSR_PL0_SSP, (unsigned long)stack + 0x5ff8); + wrmsrl(MSR_S_CET, CET_SHSTK_EN | CET_WRSS_EN); + asm volatile ("setssbsy" ::: "memory"); + } + reset_stack_and_jump_nolp(init_done); } @@ -985,6 +992,21 @@ void __init noreturn __start_xen(unsigned long mbi_p) /* This must come before e820 code because it sets paddr_bits. */ early_cpu_init(); + /* Choose shadow stack early, to set infrastructure up appropriately. */ + if ( opt_xen_shstk && boot_cpu_has(X86_FEATURE_CET_SS) ) + { + printk("Enabling Supervisor Shadow Stacks\n"); + + setup_force_cpu_cap(X86_FEATURE_XEN_SHSTK); +#ifdef CONFIG_PV32 + if ( opt_pv32 ) + { + opt_pv32 = 0; + printk(" - Disabling PV32 due to Shadow Stacks\n"); + } +#endif + } + /* Sanitise the raw E820 map to produce a final clean version. */ max_page = raw_max_page = init_e820(memmap_type, &e820_raw); @@ -1721,6 +1743,10 @@ void __init noreturn __start_xen(unsigned long mbi_p) alternative_branches(); + /* Defer CR4.CET until alternatives have finished playing with CR4.WP */ + if ( cpu_has_xen_shstk ) + set_in_cr4(X86_CR4_CET); + /* * NB: when running as a PV shim VCPUOP_up/down is wired to the shim * physical cpu_add/remove functions, so launch the guest with only diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index c5d8e587a8..a94be2d594 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -882,6 +882,14 @@ void __init init_speculation_mitigations(void) hw_smt_enabled = check_smt_enabled(); /* + * First, disable the use of retpolines if Xen is using shadow stacks, as + * they are incompatible. + */ + if ( cpu_has_xen_shstk && + (opt_thunk == THUNK_DEFAULT || opt_thunk == THUNK_RETPOLINE) ) + thunk = THUNK_JMP; + + /* * Has the user specified any custom BTI mitigations? If so, follow their * instructions exactly and disable all heuristics. */ diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h index 85c5f20b76..cdfb7b047b 100644 --- a/xen/include/asm-x86/msr-index.h +++ b/xen/include/asm-x86/msr-index.h @@ -68,6 +68,9 @@ #define MSR_U_CET 0x000006a0 #define MSR_S_CET 0x000006a2 +#define CET_SHSTK_EN (_AC(1, ULL) << 0) +#define CET_WRSS_EN (_AC(1, ULL) << 1) + #define MSR_PL0_SSP 0x000006a4 #define MSR_PL1_SSP 0x000006a5 #define MSR_PL2_SSP 0x000006a6 diff --git a/xen/include/asm-x86/x86-defns.h b/xen/include/asm-x86/x86-defns.h index 84e15b15be..4051a80485 100644 --- a/xen/include/asm-x86/x86-defns.h +++ b/xen/include/asm-x86/x86-defns.h @@ -73,6 +73,7 @@ #define X86_CR4_SMEP 0x00100000 /* enable SMEP */ #define X86_CR4_SMAP 0x00200000 /* enable SMAP */ #define X86_CR4_PKE 0x00400000 /* enable PKE */ +#define X86_CR4_CET 0x00800000 /* Control-flow Enforcement Technology */ /* * XSTATE component flags in XCR0