From patchwork Tue Nov 6 22:20:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Orr X-Patchwork-Id: 10671561 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 30544109C for ; Tue, 6 Nov 2018 22:20:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 235FF2B2F5 for ; Tue, 6 Nov 2018 22:20:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2156E2B2D7; Tue, 6 Nov 2018 22:20:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2C0E92B31A for ; Tue, 6 Nov 2018 22:20:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A95D06B049B; Tue, 6 Nov 2018 17:20:28 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A45EA6B049D; Tue, 6 Nov 2018 17:20:28 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 95C406B049E; Tue, 6 Nov 2018 17:20:28 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by kanga.kvack.org (Postfix) with ESMTP id 6B8D66B049B for ; Tue, 6 Nov 2018 17:20:28 -0500 (EST) Received: by mail-qt1-f197.google.com with SMTP id h17so2089721qto.4 for ; Tue, 06 Nov 2018 14:20:28 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=ddOQR0QLCW0j/2BbMnNWk3OM9fUkYz1ejyoozwkZ17g=; b=Y72Pb67C9bbbYIyJCy9xeW0TuavxmGbfWZ+9UU/q/LTny9wkWjWfGU9nXDIHTaB9Vs dNUxHH+cu1K9YufqVVw+h8SVDGngfJRVfXNsby5sRXfPG5AQ4AHTicORWySJz/x0mw9B +eVEhYzm2lv+ywSiCN/8Wx09KUXR0uBR+GY8tLntNl/HbsO8ElV/JHCImJTAOkTNawhP Grusw2WgBURXddiYlz72FkDOqoUrAxOUTRUwSdenK6lROdbElLd3GLI2YhPxQN+JZH0i iIcBTUAVcslfuDdqJHyyQyns5JZiWIMro5/PcbOdhgXzPWBWZBBhJoF/+G/4sWP6elfv DuLw== X-Gm-Message-State: AGRZ1gIa8kTjS3JWJUshyTrDvz38FWTYNYptnQi4HQh5Mt6VuJn7qoaE 0Y9hAOoi/ildPKflGj0h69Qc5nqWBnXw6F1ihsXhCRSaSw17PNKv+jbIca2mN+zmGw5hmlkw9j6 KUrOL9VIuD5Wdc2nwARmCY1udn0UEVU7Uk4ZlgVvI8NHsmUX7daxx7H6czLEsvO6pCO/NmI6wCD l4xJKa2zCvGDUW7/7HMthvukvmn+u8vVBHdcW4JTwQt4918AuGchvs59OUIR1a+zXAh8KOpmfK8 gq0IADlfzBMoCPnywKsVkWPbL01oMXwKozgZwYbcJx3cMxwYJMUCzbfL4jaVHSdKNV3hEuFKZg3 zm9z42o+pKqBOMLKJPVXF3aJVYRWjbdqo94LatTj5SZM9VpZNTiKaR1krtN2Xa8+r3VBIqRq/sD f X-Received: by 2002:a37:ac05:: with SMTP id e5mr27245132qkm.102.1541542828146; Tue, 06 Nov 2018 14:20:28 -0800 (PST) X-Received: by 2002:a37:ac05:: with SMTP id e5mr27245104qkm.102.1541542827190; Tue, 06 Nov 2018 14:20:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1541542827; cv=none; d=google.com; s=arc-20160816; b=EeVmUCT5VilDBs0AaCkl5TOQ9L/lqBN6tRWIEt3yJLODmOU/RnjVVTzsZJbPMZsjEx We+Pvfw7obKJJjer3Xh09Vd9z76O1NSntH+vZgndOCyEL5cgndg2cWreZ38qVT5/yuYs 9vEtcYJioUFuHmgLjbzQiFoYsjkewyScJCFEg0J2rtK3vzrwgNtW4GgcIC5UcDDuLM9T 5dO6MjUtHQZXUvb4OOuNab206khoVBMO5a8bceXKt2X2XnXHgbRTHLMYrAMjFZNr34Fy 7EkwcEn/NaUdAOcfyL90T47WHdA9jfBts6AITQZpLZ3dKfXoI9fsJ1M804LzP/SJs5L/ rmvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:dkim-signature; bh=ddOQR0QLCW0j/2BbMnNWk3OM9fUkYz1ejyoozwkZ17g=; b=wUyrAln4rwUfRR3tD3RhkASBEVpoNyJPyrbWIVkoysrGlm8uIag7mWKaMI8bwXLjjQ yuELlEDPqlpquwlD0mBzlWVGX4+U/QjHnGwqT27ukrrWlgBZdCma4V5ht+ht3Yoq5MS5 U5hTLxIWgY0lIGnOGSR5Cg54qTjaoriQBD00XFkBsMsgSZucgJpaGys6zO0J/i6O4vUQ 9dnwKHN3QnXe/O/PdoxOeDFwPoPBiwIIE/tiCM7cJFcTaONd3vnZ9Dqsbz5uFt5WD3BU YLqIZR+5JJQwwwX2PJL9gZwsBQH55C906Ms4OyXJhT45QMjYkW13vN/+vR6cNFwBFhsp zUqQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=rt5yrDgt; spf=pass (google.com: domain of 3qhpiwwckcd8nbsdpsshpphmf.dpnmjovy-nnlwbdl.psh@flex--marcorr.bounces.google.com designates 209.85.220.73 as permitted sender) smtp.mailfrom=3qhPiWwcKCD8nbsdpsshpphmf.dpnmjovy-nnlwbdl.psh@flex--marcorr.bounces.google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f73.google.com (mail-sor-f73.google.com. [209.85.220.73]) by mx.google.com with SMTPS id s63-v6sor23561356qkd.108.2018.11.06.14.20.27 for (Google Transport Security); Tue, 06 Nov 2018 14:20:27 -0800 (PST) Received-SPF: pass (google.com: domain of 3qhpiwwckcd8nbsdpsshpphmf.dpnmjovy-nnlwbdl.psh@flex--marcorr.bounces.google.com designates 209.85.220.73 as permitted sender) client-ip=209.85.220.73; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=rt5yrDgt; spf=pass (google.com: domain of 3qhpiwwckcd8nbsdpsshpphmf.dpnmjovy-nnlwbdl.psh@flex--marcorr.bounces.google.com designates 209.85.220.73 as permitted sender) smtp.mailfrom=3qhPiWwcKCD8nbsdpsshpphmf.dpnmjovy-nnlwbdl.psh@flex--marcorr.bounces.google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ddOQR0QLCW0j/2BbMnNWk3OM9fUkYz1ejyoozwkZ17g=; b=rt5yrDgt6e6kWAcGNA/TiTYCMs6q/oSAJOdZNqa7U0Nu4H42M2orWoEcSpsRgtBlWQ C2WhyeANAZb4U7aDoomRKH04YkvxCVR3dolYgRat5lM/AXoDyDwvOAdaMIGl++fCCoHe X+8ZE6YzDlWValO6GJ1a5QdczKmVVZaXcqFbiq9/yt24J5fggPFMkGAtsci8AICYrKoh sbuqODo5nZmWzj6bDtoj3aKW25vqKN5RArrc5XW9WrXHU0aSaatogATDvQLXwKrH4rRK to4aujREzNrYec15jJEaARMYjVZQzbsNpOX/c7WQPGoCw8/DhXX3HR47TMQrcdvCrRL9 4cFw== X-Google-Smtp-Source: AJdET5fqLCZOJrTv/UICoaaX1i3yEPWC2eA+6rEhhDouX7+rUZiXbrOVCd8S01Na56GFb+vdUS+k8QhC+x1U X-Received: by 2002:a37:d15c:: with SMTP id s89mr6931054qki.17.1541542826911; Tue, 06 Nov 2018 14:20:26 -0800 (PST) Date: Tue, 6 Nov 2018 14:20:09 -0800 In-Reply-To: <20181106222009.90833-1-marcorr@google.com> Message-Id: <20181106222009.90833-3-marcorr@google.com> Mime-Version: 1.0 References: <20181106222009.90833-1-marcorr@google.com> X-Mailer: git-send-email 2.19.1.930.g4563a0d9d0-goog Subject: [kvm PATCH v7 2/2] kvm: x86: Dynamically allocate guest_fpu From: Marc Orr To: kvm@vger.kernel.org, jmattson@google.com, rientjes@google.com, konrad.wilk@oracle.com, linux-mm@kvack.org, akpm@linux-foundation.org, pbonzini@redhat.com, rkrcmar@redhat.com, willy@infradead.org, sean.j.christopherson@intel.com, dave.hansen@linux.intel.com, kernellwp@gmail.com Cc: Marc Orr , Dave Hansen X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Previously, the guest_fpu field was embedded in the kvm_vcpu_arch struct. Unfortunately, the field is quite large, (e.g., 4352 bytes on my current setup). This bloats the kvm_vcpu_arch struct for x86 into an order 3 memory allocation, which can become a problem on overcommitted machines. Thus, this patch moves the fpu state outside of the kvm_vcpu_arch struct. With this patch applied, the kvm_vcpu_arch struct is reduced to 15168 bytes for vmx on my setup when building the kernel with kvmconfig. Suggested-by: Dave Hansen Signed-off-by: Marc Orr --- arch/x86/include/asm/kvm_host.h | 3 +- arch/x86/kvm/svm.c | 10 +++++++ arch/x86/kvm/vmx.c | 10 +++++++ arch/x86/kvm/x86.c | 51 ++++++++++++++++++++++++--------- 4 files changed, 60 insertions(+), 14 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index ebb1d7a755d4..c8a2a263f91f 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -610,7 +610,7 @@ struct kvm_vcpu_arch { * "guest_fpu" state here contains the guest FPU context, with the * host PRKU bits. */ - struct fpu guest_fpu; + struct fpu *guest_fpu; u64 xcr0; u64 guest_supported_xcr0; @@ -1194,6 +1194,7 @@ struct kvm_arch_async_pf { }; extern struct kvm_x86_ops *kvm_x86_ops; +extern struct kmem_cache *x86_fpu_cache; #define __KVM_HAVE_ARCH_VM_ALLOC static inline struct kvm *kvm_arch_alloc_vm(void) diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index f416f5c7f2ae..ac0c52ca22c6 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -2121,6 +2121,13 @@ static struct kvm_vcpu *svm_create_vcpu(struct kvm *kvm, unsigned int id) goto out; } + svm->vcpu.arch.guest_fpu = kmem_cache_zalloc(x86_fpu_cache, GFP_KERNEL); + if (!svm->vcpu.arch.guest_fpu) { + printk(KERN_ERR "kvm: failed to allocate vcpu's fpu\n"); + err = -ENOMEM; + goto free_partial_svm; + } + err = kvm_vcpu_init(&svm->vcpu, kvm, id); if (err) goto free_svm; @@ -2180,6 +2187,8 @@ static struct kvm_vcpu *svm_create_vcpu(struct kvm *kvm, unsigned int id) uninit: kvm_vcpu_uninit(&svm->vcpu); free_svm: + kmem_cache_free(x86_fpu_cache, svm->vcpu.arch.guest_fpu); +free_partial_svm: kmem_cache_free(kvm_vcpu_cache, svm); out: return ERR_PTR(err); @@ -2194,6 +2203,7 @@ static void svm_free_vcpu(struct kvm_vcpu *vcpu) __free_page(virt_to_page(svm->nested.hsave)); __free_pages(virt_to_page(svm->nested.msrpm), MSRPM_ALLOC_ORDER); kvm_vcpu_uninit(vcpu); + kmem_cache_free(x86_fpu_cache, svm->vcpu.arch.guest_fpu); kmem_cache_free(kvm_vcpu_cache, svm); /* * The vmcb page can be recycled, causing a false negative in diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index abeeb45d1c33..4078cf15a4b0 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -11476,6 +11476,7 @@ static void vmx_free_vcpu(struct kvm_vcpu *vcpu) free_loaded_vmcs(vmx->loaded_vmcs); kfree(vmx->guest_msrs); kvm_vcpu_uninit(vcpu); + kmem_cache_free(x86_fpu_cache, vmx->vcpu.arch.guest_fpu); kmem_cache_free(kvm_vcpu_cache, vmx); } @@ -11489,6 +11490,13 @@ static struct kvm_vcpu *vmx_create_vcpu(struct kvm *kvm, unsigned int id) if (!vmx) return ERR_PTR(-ENOMEM); + vmx->vcpu.arch.guest_fpu = kmem_cache_zalloc(x86_fpu_cache, GFP_KERNEL); + if (!vmx->vcpu.arch.guest_fpu) { + printk(KERN_ERR "kvm: failed to allocate vcpu's fpu\n"); + err = -ENOMEM; + goto free_partial_vcpu; + } + vmx->vpid = allocate_vpid(); err = kvm_vcpu_init(&vmx->vcpu, kvm, id); @@ -11576,6 +11584,8 @@ static struct kvm_vcpu *vmx_create_vcpu(struct kvm *kvm, unsigned int id) kvm_vcpu_uninit(&vmx->vcpu); free_vcpu: free_vpid(vmx->vpid); + kmem_cache_free(x86_fpu_cache, vmx->vcpu.arch.guest_fpu); +free_partial_vcpu: kmem_cache_free(kvm_vcpu_cache, vmx); return ERR_PTR(err); } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ff77514f7367..8abe058f48d9 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -213,6 +213,9 @@ struct kvm_stats_debugfs_item debugfs_entries[] = { u64 __read_mostly host_xcr0; +struct kmem_cache *x86_fpu_cache; +EXPORT_SYMBOL_GPL(x86_fpu_cache); + static int emulator_fix_hypercall(struct x86_emulate_ctxt *ctxt); static inline void kvm_async_pf_hash_reset(struct kvm_vcpu *vcpu) @@ -3635,7 +3638,7 @@ static int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu, static void fill_xsave(u8 *dest, struct kvm_vcpu *vcpu) { - struct xregs_state *xsave = &vcpu->arch.guest_fpu.state.xsave; + struct xregs_state *xsave = &vcpu->arch.guest_fpu->state.xsave; u64 xstate_bv = xsave->header.xfeatures; u64 valid; @@ -3677,7 +3680,7 @@ static void fill_xsave(u8 *dest, struct kvm_vcpu *vcpu) static void load_xsave(struct kvm_vcpu *vcpu, u8 *src) { - struct xregs_state *xsave = &vcpu->arch.guest_fpu.state.xsave; + struct xregs_state *xsave = &vcpu->arch.guest_fpu->state.xsave; u64 xstate_bv = *(u64 *)(src + XSAVE_HDR_OFFSET); u64 valid; @@ -3725,7 +3728,7 @@ static void kvm_vcpu_ioctl_x86_get_xsave(struct kvm_vcpu *vcpu, fill_xsave((u8 *) guest_xsave->region, vcpu); } else { memcpy(guest_xsave->region, - &vcpu->arch.guest_fpu.state.fxsave, + &vcpu->arch.guest_fpu->state.fxsave, sizeof(struct fxregs_state)); *(u64 *)&guest_xsave->region[XSAVE_HDR_OFFSET / sizeof(u32)] = XFEATURE_MASK_FPSSE; @@ -3755,7 +3758,7 @@ static int kvm_vcpu_ioctl_x86_set_xsave(struct kvm_vcpu *vcpu, if (xstate_bv & ~XFEATURE_MASK_FPSSE || mxcsr & ~mxcsr_feature_mask) return -EINVAL; - memcpy(&vcpu->arch.guest_fpu.state.fxsave, + memcpy(&vcpu->arch.guest_fpu->state.fxsave, guest_xsave->region, sizeof(struct fxregs_state)); } return 0; @@ -6818,11 +6821,30 @@ int kvm_arch_init(void *opaque) goto out; } + if (!boot_cpu_has(X86_FEATURE_FPU) || !boot_cpu_has(X86_FEATURE_FXSR)) { + printk(KERN_ERR "kvm: inadequate fpu\n"); + r = -EOPNOTSUPP; + goto out; + } + r = -ENOMEM; + x86_fpu_cache = kmem_cache_create_usercopy( + "x86_fpu", + fpu_kernel_xstate_size, + __alignof__(struct fpu), + SLAB_ACCOUNT, + offsetof(struct fpu, state), + fpu_kernel_xstate_size, + NULL); + if (!x86_fpu_cache) { + printk(KERN_ERR "kvm: failed to allocate cache for x86 fpu\n"); + goto out; + } + shared_msrs = alloc_percpu(struct kvm_shared_msrs); if (!shared_msrs) { printk(KERN_ERR "kvm: failed to allocate percpu kvm_shared_msrs\n"); - goto out; + goto out_free_x86_fpu_cache; } r = kvm_mmu_module_init(); @@ -6855,6 +6877,8 @@ int kvm_arch_init(void *opaque) out_free_percpu: free_percpu(shared_msrs); +out_free_x86_fpu_cache: + kmem_cache_destroy(x86_fpu_cache); out: return r; } @@ -6878,6 +6902,7 @@ void kvm_arch_exit(void) kvm_x86_ops = NULL; kvm_mmu_module_exit(); free_percpu(shared_msrs); + kmem_cache_destroy(x86_fpu_cache); } int kvm_vcpu_halt(struct kvm_vcpu *vcpu) @@ -8001,7 +8026,7 @@ static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu) preempt_disable(); copy_fpregs_to_fpstate(¤t->thread.fpu); /* PKRU is separately restored in kvm_x86_ops->run. */ - __copy_kernel_to_fpregs(&vcpu->arch.guest_fpu.state, + __copy_kernel_to_fpregs(&vcpu->arch.guest_fpu->state, ~XFEATURE_MASK_PKRU); preempt_enable(); trace_kvm_fpu(1); @@ -8011,7 +8036,7 @@ static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu) static void kvm_put_guest_fpu(struct kvm_vcpu *vcpu) { preempt_disable(); - copy_fpregs_to_fpstate(&vcpu->arch.guest_fpu); + copy_fpregs_to_fpstate(vcpu->arch.guest_fpu); copy_kernel_to_fpregs(¤t->thread.fpu.state); preempt_enable(); ++vcpu->stat.fpu_reload; @@ -8506,7 +8531,7 @@ int kvm_arch_vcpu_ioctl_get_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu) vcpu_load(vcpu); - fxsave = &vcpu->arch.guest_fpu.state.fxsave; + fxsave = &vcpu->arch.guest_fpu->state.fxsave; memcpy(fpu->fpr, fxsave->st_space, 128); fpu->fcw = fxsave->cwd; fpu->fsw = fxsave->swd; @@ -8526,7 +8551,7 @@ int kvm_arch_vcpu_ioctl_set_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu) vcpu_load(vcpu); - fxsave = &vcpu->arch.guest_fpu.state.fxsave; + fxsave = &vcpu->arch.guest_fpu->state.fxsave; memcpy(fxsave->st_space, fpu->fpr, 128); fxsave->cwd = fpu->fcw; @@ -8582,9 +8607,9 @@ static int sync_regs(struct kvm_vcpu *vcpu) static void fx_init(struct kvm_vcpu *vcpu) { - fpstate_init(&vcpu->arch.guest_fpu.state); + fpstate_init(&vcpu->arch.guest_fpu->state); if (boot_cpu_has(X86_FEATURE_XSAVES)) - vcpu->arch.guest_fpu.state.xsave.header.xcomp_bv = + vcpu->arch.guest_fpu->state.xsave.header.xcomp_bv = host_xcr0 | XSTATE_COMPACTION_ENABLED; /* @@ -8708,11 +8733,11 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) */ if (init_event) kvm_put_guest_fpu(vcpu); - mpx_state_buffer = get_xsave_addr(&vcpu->arch.guest_fpu.state.xsave, + mpx_state_buffer = get_xsave_addr(&vcpu->arch.guest_fpu->state.xsave, XFEATURE_MASK_BNDREGS); if (mpx_state_buffer) memset(mpx_state_buffer, 0, sizeof(struct mpx_bndreg_state)); - mpx_state_buffer = get_xsave_addr(&vcpu->arch.guest_fpu.state.xsave, + mpx_state_buffer = get_xsave_addr(&vcpu->arch.guest_fpu->state.xsave, XFEATURE_MASK_BNDCSR); if (mpx_state_buffer) memset(mpx_state_buffer, 0, sizeof(struct mpx_bndcsr));