From patchwork Mon Mar 6 22:03:48 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Garnier X-Patchwork-Id: 9607723 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5F600602B4 for ; Mon, 6 Mar 2017 22:06:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 58C4527BE5 for ; Mon, 6 Mar 2017 22:06:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4C7D828477; Mon, 6 Mar 2017 22:06:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 92C1527BE5 for ; Mon, 6 Mar 2017 22:06:32 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cl0ji-0006TJ-Vk; Mon, 06 Mar 2017 22:04:10 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cl0jh-0006T2-Rv for xen-devel@lists.xenproject.org; Mon, 06 Mar 2017 22:04:10 +0000 Received: from [85.158.143.35] by server-4.bemta-6.messagelabs.com id 20/E4-25093-9DCDDB85; Mon, 06 Mar 2017 22:04:09 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprPIsWRWlGSWpSXmKPExsXiVRtspHvjzt4 Ig2XPRSy+b5nM5MDocfjDFZYAxijWzLyk/IoE1owrX9uZC6a4VfxZ0sHSwHjYuouRi0NIYDqj xJENz1lAHBaBVywSp6bvYAVxJAT6WSVOT53M2MXICeTESCy80cYGYVdIrPnzHSwuJKAksXXDU mYIu4FJ4vBtXhCbTUBLYk/DfCYQW0TgI5/EgT6wocwCa5kktl09A5YQFnCX+LS7BWwoi4CqxK 1Nn8HivAIWEnvWPmKBWGYhcefxEqA4BwengKXEkg1hELssJCacnsM+gVFgASPDKkb14tSistQ iXVO9pKLM9IyS3MTMHF1DAzO93NTi4sT01JzEpGK95PzcTYzAwGIAgh2M0y/7H2KU5GBSEuX9 HrYnQogvKT+lMiOxOCO+qDQntfgQowwHh5IE78EJQDnBotT01Iq0zBxgiMOkJTh4lER4N4Kke YsLEnOLM9MhUqcYdTk+9B9+wyTEkpeflyolzlsFUiQAUpRRmgc3AhZvlxhlpYR5GYGOEuIpSC 3KzSxBlX/FKM7BqCTM6wIyhSczrwRu0yugI5iAjvCT2QlyREkiQkqqgbGBP4rzmAHjMpfE5Hv OMidnxNuEvkw9cv/LqaPC6ib/neeE/qmJr3pf+u1bzX+egDmnuO1FeOonbzsscjPew5GN5Wtw SGbpTh4WjcgrMwwvaNycabfpTY2e45wD7tXuZn1bApcfKqso7ahy/DCv2zx/XcOdtP6MhtZrL P13o263SGyNUDc9rMRSnJFoqMVcVJwIAKK12sOyAgAA X-Env-Sender: thgarnie@google.com X-Msg-Ref: server-4.tower-21.messagelabs.com!1488837846!54778311!1 X-Originating-IP: [74.125.83.50] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.2.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 2873 invoked from network); 6 Mar 2017 22:04:08 -0000 Received: from mail-pg0-f50.google.com (HELO mail-pg0-f50.google.com) (74.125.83.50) by server-4.tower-21.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 6 Mar 2017 22:04:08 -0000 Received: by mail-pg0-f50.google.com with SMTP id 77so26564786pgc.1 for ; Mon, 06 Mar 2017 14:04:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=uQ3mYePs7pNE2bZPQR70Hs3mk68faHtffhoJHoQNZNY=; b=AuwS/wDDiTrZ32c/2iP/p7BLzW+Xi1tk0PCs9nfDqSN+Vz2TkE3j3ap0J01VK4QchY vnh1wZET/UkbIUEfGGM1yUv0MdbpUCzIn02ccFe9o8003htdPncqDk3XoJzcyEskJRO9 juyHFre0jcd3kVHpCTccR1NWSeN3Q9CAS6IRhI7LnE2mH9N2WFwBwsaFsHPyUt5CD+XM g4XuvuakUpiwbWaRQnVGLXKJIPsXgk25qijflY0sKpdHwde98gsXfTbD20cOhVy/pWb1 ruiJZWhof2IoiYeVI0Xp/fY9dU5i3mtrz+MgHCHaJntFqPhf2ODCqMg7qq8sDoq+BJBA pFhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=uQ3mYePs7pNE2bZPQR70Hs3mk68faHtffhoJHoQNZNY=; b=ez9xgZnhjW/mwvMC3fl0WQhEnlflWlx4hZX2EuqlshjRt/ATMOQdn1psBxt0SuqZH+ dp8xrVhOuiV9LQtsrjCBYXlbjnfORKCsvrrC114quUeKfnivGFo3ffNIuDqTJ6yarGsR HBTm8r1Brb7OhK9tejw47U8unfRjAo0d10muQRlEkTPU1FQNV9VWzaihzwaHqKndhilT nlN+biBf98Po8a+DZU5vqKFfqj3XSzWOcj9V0MB6WN9bc+if41390qWEssOcwcuuWF16 Lsa4NTDFrkM1iljIQzaBbncq751EDCrIk7OtTChYcufKwxfn6XU0reQvgMNTyyJTLOaI 1XcQ== X-Gm-Message-State: AMke39lcNCYiHBvWeNHgBAyfyNfrKAT8Wg+SAL01HQ6JTjgNCOWMDszIPdWwxUSGq1ivVu99 X-Received: by 10.98.9.29 with SMTP id e29mr10157904pfd.101.1488837846202; Mon, 06 Mar 2017 14:04:06 -0800 (PST) Received: from skynet.sea.corp.google.com ([100.100.206.185]) by smtp.gmail.com with ESMTPSA id l72sm41407671pfi.93.2017.03.06.14.04.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 06 Mar 2017 14:04:05 -0800 (PST) From: Thomas Garnier To: Thomas Gleixner , Ingo Molnar , "H . Peter Anvin" , Jonathan Corbet , Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Thomas Garnier , Kees Cook , Juergen Gross , Lorenzo Stoakes , Andy Lutomirski , Paul Gortmaker , Andrew Morton , Michal Hocko , zijun_hu , Chris Wilson , Joonsoo Kim , Andy Lutomirski , "Rafael J . Wysocki" , Len Brown , Pavel Machek , Jiri Kosina , Matt Fleming , Ard Biesheuvel , Boris Ostrovsky , Rusty Russell , Paolo Bonzini , Borislav Petkov , Christian Borntraeger , "Luis R . Rodriguez" , Fenghua Yu , He Chen , Brian Gerst , Frederic Weisbecker , Stanislaw Gruszka , Arnd Bergmann , Peter Zijlstra , Dave Hansen , Josh Poimboeuf , Vitaly Kuznetsov , David Vrabel , Tim Chen , Andi Kleen , Jiri Olsa , Prarit Bhargava , Michael Ellerman , Joerg Roedel , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= Date: Mon, 6 Mar 2017 14:03:48 -0800 Message-Id: <20170306220348.79702-3-thgarnie@google.com> X-Mailer: git-send-email 2.12.0.rc1.440.g5b76565f74-goog In-Reply-To: <20170306220348.79702-1-thgarnie@google.com> References: <20170306220348.79702-1-thgarnie@google.com> Cc: linux-efi@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, x86@kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, lguest@lists.ozlabs.org, kernel-hardening@lists.openwall.com, xen-devel@lists.xenproject.org Subject: [Xen-devel] [PATCH v5 3/3] x86: Make the GDT remapping read-only on 64-bit X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This patch makes the GDT remapped pages read-only to prevent corruption. This change is done only on 64-bit. The native_load_tr_desc function was adapted to correctly handle a read-only GDT. The LTR instruction always writes to the GDT TSS entry. This generates a page fault if the GDT is read-only. This change checks if the current GDT is a remap and swap GDTs as needed. This function was tested by booting multiple machines and checking hibernation works properly. KVM SVM and VMX were adapted to use the writeable GDT. On VMX, the per-cpu variable was removed for functions to fetch the original GDT. Instead of reloading the previous GDT, VMX will reload the fixmap GDT as expected. For testing, VMs were started and restored on multiple configurations. Signed-off-by: Thomas Garnier --- Based on next-20170306 --- arch/x86/include/asm/desc.h | 106 +++++++++++++++++++++++++-------------- arch/x86/include/asm/processor.h | 1 + arch/x86/kernel/cpu/common.c | 28 ++++++++--- arch/x86/kvm/svm.c | 4 +- arch/x86/kvm/vmx.c | 11 ++-- 5 files changed, 96 insertions(+), 54 deletions(-) diff --git a/arch/x86/include/asm/desc.h b/arch/x86/include/asm/desc.h index 549393ae93a0..9b7fda6a2d73 100644 --- a/arch/x86/include/asm/desc.h +++ b/arch/x86/include/asm/desc.h @@ -247,9 +247,77 @@ static inline void native_set_ldt(const void *addr, unsigned int entries) } } +static inline void native_load_gdt(const struct desc_ptr *dtr) +{ + asm volatile("lgdt %0"::"m" (*dtr)); +} + +static inline void native_load_idt(const struct desc_ptr *dtr) +{ + asm volatile("lidt %0"::"m" (*dtr)); +} + +static inline void native_store_gdt(struct desc_ptr *dtr) +{ + asm volatile("sgdt %0":"=m" (*dtr)); +} + +static inline void native_store_idt(struct desc_ptr *dtr) +{ + asm volatile("sidt %0":"=m" (*dtr)); +} + +/* + * The LTR instruction marks the TSS GDT entry as busy. On 64-bit, the GDT is + * a read-only remapping. To prevent a page fault, the GDT is switched to the + * original writeable version when needed. + */ +#ifdef CONFIG_X86_64 static inline void native_load_tr_desc(void) { + struct desc_ptr gdt; + int cpu = raw_smp_processor_id(); + bool restore = 0; + struct desc_struct *fixmap_gdt; + + native_store_gdt(&gdt); + fixmap_gdt = get_cpu_gdt_ro(cpu); + + /* + * If the current GDT is the read-only fixmap, swap to the original + * writeable version. Swap back at the end. + */ + if (gdt.address == (unsigned long)fixmap_gdt) { + load_direct_gdt(cpu); + restore = 1; + } asm volatile("ltr %w0"::"q" (GDT_ENTRY_TSS*8)); + if (restore) + load_fixmap_gdt(cpu); +} +#else +static inline void native_load_tr_desc(void) +{ + asm volatile("ltr %w0"::"q" (GDT_ENTRY_TSS*8)); +} +#endif + +static inline unsigned long native_store_tr(void) +{ + unsigned long tr; + + asm volatile("str %0":"=r" (tr)); + + return tr; +} + +static inline void native_load_tls(struct thread_struct *t, unsigned int cpu) +{ + struct desc_struct *gdt = get_cpu_gdt_rw(cpu); + unsigned int i; + + for (i = 0; i < GDT_ENTRY_TLS_ENTRIES; i++) + gdt[GDT_ENTRY_TLS_MIN + i] = t->tls_array[i]; } DECLARE_PER_CPU(bool, __tss_limit_invalid); @@ -304,44 +372,6 @@ static inline void invalidate_tss_limit(void) this_cpu_write(__tss_limit_invalid, true); } -static inline void native_load_gdt(const struct desc_ptr *dtr) -{ - asm volatile("lgdt %0"::"m" (*dtr)); -} - -static inline void native_load_idt(const struct desc_ptr *dtr) -{ - asm volatile("lidt %0"::"m" (*dtr)); -} - -static inline void native_store_gdt(struct desc_ptr *dtr) -{ - asm volatile("sgdt %0":"=m" (*dtr)); -} - -static inline void native_store_idt(struct desc_ptr *dtr) -{ - asm volatile("sidt %0":"=m" (*dtr)); -} - -static inline unsigned long native_store_tr(void) -{ - unsigned long tr; - - asm volatile("str %0":"=r" (tr)); - - return tr; -} - -static inline void native_load_tls(struct thread_struct *t, unsigned int cpu) -{ - struct desc_struct *gdt = get_cpu_gdt_rw(cpu); - unsigned int i; - - for (i = 0; i < GDT_ENTRY_TLS_ENTRIES; i++) - gdt[GDT_ENTRY_TLS_MIN + i] = t->tls_array[i]; -} - /* This intentionally ignores lm, since 32-bit apps don't have that field. */ #define LDT_empty(info) \ ((info)->base_addr == 0 && \ diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h index 2ec4d2dc559b..28828f1f99a4 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -716,6 +716,7 @@ extern struct desc_ptr early_gdt_descr; extern void cpu_set_gdt(int); extern void switch_to_new_gdt(int); +extern void load_direct_gdt(int); extern void load_fixmap_gdt(int); extern void load_percpu_segment(int); extern void cpu_init(void); diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index a9e847da014a..bff2f8bb13b5 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -448,13 +448,31 @@ void load_percpu_segment(int cpu) load_stack_canary_segment(); } +/* On 64-bit the GDT remapping is read-only */ +#ifdef CONFIG_X86_64 +#define PAGE_FIXMAP_GDT PAGE_KERNEL_RO +#else +#define PAGE_FIXMAP_GDT PAGE_KERNEL +#endif + /* Setup the fixmap mapping only once per-processor */ static inline void setup_fixmap_gdt(int cpu) { __set_fixmap(get_cpu_gdt_ro_index(cpu), - __pa(get_cpu_gdt_rw(cpu)), PAGE_KERNEL); + __pa(get_cpu_gdt_rw(cpu)), PAGE_FIXMAP_GDT); } +/* Load the original GDT from the per-cpu structure */ +void load_direct_gdt(int cpu) +{ + struct desc_ptr gdt_descr; + + gdt_descr.address = (long)get_cpu_gdt_rw(cpu); + gdt_descr.size = GDT_SIZE - 1; + load_gdt(&gdt_descr); +} +EXPORT_SYMBOL_GPL(load_direct_gdt); + /* Load a fixmap remapping of the per-cpu GDT */ void load_fixmap_gdt(int cpu) { @@ -464,6 +482,7 @@ void load_fixmap_gdt(int cpu) gdt_descr.size = GDT_SIZE - 1; load_gdt(&gdt_descr); } +EXPORT_SYMBOL_GPL(load_fixmap_gdt); /* * Current gdt points %fs at the "master" per-cpu area: after this, @@ -471,11 +490,8 @@ void load_fixmap_gdt(int cpu) */ void switch_to_new_gdt(int cpu) { - struct desc_ptr gdt_descr; - - gdt_descr.address = (long)get_cpu_gdt_rw(cpu); - gdt_descr.size = GDT_SIZE - 1; - load_gdt(&gdt_descr); + /* Load the original GDT */ + load_direct_gdt(cpu); /* Reload the per-cpu base */ load_percpu_segment(cpu); } diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index d1efe2c62b3f..c02b9af2056a 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -741,7 +741,6 @@ static int svm_hardware_enable(void) struct svm_cpu_data *sd; uint64_t efer; - struct desc_ptr gdt_descr; struct desc_struct *gdt; int me = raw_smp_processor_id(); @@ -763,8 +762,7 @@ static int svm_hardware_enable(void) sd->max_asid = cpuid_ebx(SVM_CPUID_FUNC) - 1; sd->next_asid = sd->max_asid + 1; - native_store_gdt(&gdt_descr); - gdt = (struct desc_struct *)gdt_descr.address; + gdt = get_current_gdt_rw(); sd->tss_desc = (struct kvm_ldttss_desc *)(gdt + GDT_ENTRY_TSS); wrmsrl(MSR_EFER, efer | EFER_SVME); diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index 283aa8601833..440ba96e4dfe 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -935,7 +935,6 @@ static DEFINE_PER_CPU(struct vmcs *, current_vmcs); * when a CPU is brought down, and we need to VMCLEAR all VMCSs loaded on it. */ static DEFINE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); -static DEFINE_PER_CPU(struct desc_ptr, host_gdt); /* * We maintian a per-CPU linked-list of vCPU, so in wakeup_handler() we @@ -2059,7 +2058,7 @@ static unsigned long segment_base(u16 selector) if (!(selector & ~SEGMENT_RPL_MASK)) return 0; - table = (struct desc_struct *)gdt->address; + table = get_current_gdt_ro(); if ((selector & SEGMENT_TI_MASK) == SEGMENT_LDT) { u16 ldt_selector = kvm_read_ldt(); @@ -2164,7 +2163,7 @@ static void __vmx_load_host_state(struct vcpu_vmx *vmx) #endif if (vmx->host_state.msr_host_bndcfgs) wrmsrl(MSR_IA32_BNDCFGS, vmx->host_state.msr_host_bndcfgs); - load_gdt(this_cpu_ptr(&host_gdt)); + load_fixmap_gdt(raw_smp_processor_id()); } static void vmx_load_host_state(struct vcpu_vmx *vmx) @@ -2266,7 +2265,7 @@ static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) } if (!already_loaded) { - struct desc_ptr *gdt = this_cpu_ptr(&host_gdt); + unsigned long gdt = get_current_gdt_ro_vaddr(); unsigned long sysenter_esp; kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu); @@ -2277,7 +2276,7 @@ static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) */ vmcs_writel(HOST_TR_BASE, (unsigned long)this_cpu_ptr(&cpu_tss)); - vmcs_writel(HOST_GDTR_BASE, gdt->address); + vmcs_writel(HOST_GDTR_BASE, gdt); /* 22.2.4 */ /* * VM exits change the host TR limit to 0x67 after a VM @@ -3465,8 +3464,6 @@ static int hardware_enable(void) ept_sync_global(); } - native_store_gdt(this_cpu_ptr(&host_gdt)); - return 0; }