From patchwork Mon Feb 17 13:55:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Wei Liu X-Patchwork-Id: 11386521 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 58CF017F0 for ; Mon, 17 Feb 2020 13:56:33 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 353882070B for ; Mon, 17 Feb 2020 13:56:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RFrglyFo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 353882070B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j3grw-0006FK-86; Mon, 17 Feb 2020 13:55:28 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j3gru-0006EV-Ut for xen-devel@lists.xenproject.org; Mon, 17 Feb 2020 13:55:26 +0000 X-Inumbo-ID: 23a4b0e4-518d-11ea-aa99-bc764e2007e4 Received: from mail-wm1-x344.google.com (unknown [2a00:1450:4864:20::344]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 23a4b0e4-518d-11ea-aa99-bc764e2007e4; Mon, 17 Feb 2020 13:55:22 +0000 (UTC) Received: by mail-wm1-x344.google.com with SMTP id p17so18560944wma.1 for ; Mon, 17 Feb 2020 05:55:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wLwrdLrf3Y5Q/7f9/LZJ0C4qbdwwFvoupOHOFtCD0qc=; b=RFrglyFoHhein7XRsb5cs7Uha4vcvBkg2AUJhyr3LPYwVkVBgDnbdb+fXz6Z1hgR6x SMfwXo83ezWoqDW0Z4VKfdYFzC9g+H/t6DEl8QN3uYia9SANWm7OVbE6gVjGVt8s4j6q yY6dvmhIuLJYHJvkYuju4Ub65xjUcN0IYIwaaJFBy1Xck/1QvryNXHLzSvRJ9QFQ/hAl Ozk1UR14mfwjX44fZYgV0G45mjjcWMD+rJ1Hm6kB5ABa3k3EvzqtkrzZpWKgaVx5++t7 dDq8I5CPHIMTJ52HV4mSIBCmYkQhVx03TrPUC7uNrMYE9WD6z2dCDCF+6ZoAZTE5Xh3W 2jOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=wLwrdLrf3Y5Q/7f9/LZJ0C4qbdwwFvoupOHOFtCD0qc=; b=cmUvfSAV8OL/xJCJ0CrhK3cTifxYokgH5UfwYaEbl6E2shbIoYGpI4dzghnYfQ6cEs EnvYVUzMN7RRh1xFKn1WRkhX2eFPYgU6Vzf1farCm01uyJOeKQafn9ZQrViSK0DbFVrq J7jNuBTsjApY08ZQzZH7OXZEgfW36olkN68xVYkLaeGpSkxP6RNAb6cjqUQShXH4dF1D VNGtk7S3rKIGcyFhOTPtPmWHjN2sYl8vwxaSp2+aU+AL4p0a8WdkwyzzAITXSeTFVOvb KjsUFDcOPrgvzFG4zJD40yKNGYKz02IHtqvt+APCNeRI4s7+szQSe+5rNi+kNmtD/jak 62Xw== X-Gm-Message-State: APjAAAWN9lZzA25jkPK432fRbRMbGVelvePcchE5SmIvTfuh0Sspg2Ky iLV+AApobHGcDJbVYv1JwFFzDxSO X-Google-Smtp-Source: APXvYqwVrlE69e4XKeGBPsQWOJRnPUqFAcI6aCJOBDW/X8p0QFE6onAQHGfq3xZ6YQgFyxGubFj2Vw== X-Received: by 2002:a1c:a381:: with SMTP id m123mr22383818wme.158.1581947721253; Mon, 17 Feb 2020 05:55:21 -0800 (PST) Received: from localhost.localdomain (41.142.6.51.dyn.plus.net. [51.6.142.41]) by smtp.gmail.com with ESMTPSA id m3sm1021533wrs.53.2020.02.17.05.55.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2020 05:55:20 -0800 (PST) From: Wei Liu X-Google-Original-From: Wei Liu To: Xen Development List Date: Mon, 17 Feb 2020 13:55:15 +0000 Message-Id: <20200217135517.5826-2-liuwe@microsoft.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200217135517.5826-1-liuwe@microsoft.com> References: <20200217135517.5826-1-liuwe@microsoft.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v3 1/3] x86/hypervisor: pass flags to hypervisor_flush_tlb X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Wei Liu , Wei Liu , Andrew Cooper , Paul Durrant , Michael Kelley , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Hyper-V's L0 assisted flush has fine-grained control over what gets flushed. We need all the flags available to make the best decisions possible. No functional change because Xen's implementation doesn't care about what is passed to it. Signed-off-by: Wei Liu Reviewed-by: Roger Pau Monné Reviewed-by: Paul Durrant --- v2: 1. Introduce FLUSH_TLB_FLAGS_MASK --- xen/arch/x86/guest/hypervisor.c | 7 +++++-- xen/arch/x86/guest/xen/xen.c | 2 +- xen/arch/x86/smp.c | 5 ++--- xen/include/asm-x86/flushtlb.h | 3 +++ xen/include/asm-x86/guest/hypervisor.h | 10 +++++----- 5 files changed, 16 insertions(+), 11 deletions(-) diff --git a/xen/arch/x86/guest/hypervisor.c b/xen/arch/x86/guest/hypervisor.c index 47e938e287..6ee28c9df1 100644 --- a/xen/arch/x86/guest/hypervisor.c +++ b/xen/arch/x86/guest/hypervisor.c @@ -75,10 +75,13 @@ void __init hypervisor_e820_fixup(struct e820map *e820) } int hypervisor_flush_tlb(const cpumask_t *mask, const void *va, - unsigned int order) + unsigned int flags) { + if ( flags & ~FLUSH_TLB_FLAGS_MASK ) + return -EINVAL; + if ( ops.flush_tlb ) - return alternative_call(ops.flush_tlb, mask, va, order); + return alternative_call(ops.flush_tlb, mask, va, flags); return -ENOSYS; } diff --git a/xen/arch/x86/guest/xen/xen.c b/xen/arch/x86/guest/xen/xen.c index 5d3427a713..0eb1115c4d 100644 --- a/xen/arch/x86/guest/xen/xen.c +++ b/xen/arch/x86/guest/xen/xen.c @@ -324,7 +324,7 @@ static void __init e820_fixup(struct e820map *e820) pv_shim_fixup_e820(e820); } -static int flush_tlb(const cpumask_t *mask, const void *va, unsigned int order) +static int flush_tlb(const cpumask_t *mask, const void *va, unsigned int flags) { return xen_hypercall_hvm_op(HVMOP_flush_tlbs, NULL); } diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c index c7caf5bc26..4dab74c0d5 100644 --- a/xen/arch/x86/smp.c +++ b/xen/arch/x86/smp.c @@ -258,9 +258,8 @@ void flush_area_mask(const cpumask_t *mask, const void *va, unsigned int flags) !cpumask_subset(mask, cpumask_of(cpu)) ) { if ( cpu_has_hypervisor && - !(flags & ~(FLUSH_TLB | FLUSH_TLB_GLOBAL | FLUSH_VA_VALID | - FLUSH_ORDER_MASK)) && - !hypervisor_flush_tlb(mask, va, flags & FLUSH_ORDER_MASK) ) + !(flags & ~FLUSH_TLB_FLAGS_MASK) && + !hypervisor_flush_tlb(mask, va, flags) ) { if ( tlb_clk_enabled ) tlb_clk_enabled = false; diff --git a/xen/include/asm-x86/flushtlb.h b/xen/include/asm-x86/flushtlb.h index 9773014320..a4de317452 100644 --- a/xen/include/asm-x86/flushtlb.h +++ b/xen/include/asm-x86/flushtlb.h @@ -123,6 +123,9 @@ void switch_cr3_cr4(unsigned long cr3, unsigned long cr4); /* Flush all HVM guests linear TLB (using ASID/VPID) */ #define FLUSH_GUESTS_TLB 0x4000 +#define FLUSH_TLB_FLAGS_MASK (FLUSH_TLB | FLUSH_TLB_GLOBAL | FLUSH_VA_VALID | \ + FLUSH_ORDER_MASK) + /* Flush local TLBs/caches. */ unsigned int flush_area_local(const void *va, unsigned int flags); #define flush_local(flags) flush_area_local(NULL, flags) diff --git a/xen/include/asm-x86/guest/hypervisor.h b/xen/include/asm-x86/guest/hypervisor.h index 432e57c2a0..48d54735d2 100644 --- a/xen/include/asm-x86/guest/hypervisor.h +++ b/xen/include/asm-x86/guest/hypervisor.h @@ -35,7 +35,7 @@ struct hypervisor_ops { /* Fix up e820 map */ void (*e820_fixup)(struct e820map *e820); /* L0 assisted TLB flush */ - int (*flush_tlb)(const cpumask_t *mask, const void *va, unsigned int order); + int (*flush_tlb)(const cpumask_t *mask, const void *va, unsigned int flags); }; #ifdef CONFIG_GUEST @@ -48,11 +48,11 @@ void hypervisor_e820_fixup(struct e820map *e820); /* * L0 assisted TLB flush. * mask: cpumask of the dirty vCPUs that should be flushed. - * va: linear address to flush, or NULL for global flushes. - * order: order of the linear address pointed by va. + * va: linear address to flush, or NULL for entire address space. + * flags: flags for flushing, including the order of va. */ int hypervisor_flush_tlb(const cpumask_t *mask, const void *va, - unsigned int order); + unsigned int flags); #else @@ -65,7 +65,7 @@ static inline int hypervisor_ap_setup(void) { return 0; } static inline void hypervisor_resume(void) { ASSERT_UNREACHABLE(); } static inline void hypervisor_e820_fixup(struct e820map *e820) {} static inline int hypervisor_flush_tlb(const cpumask_t *mask, const void *va, - unsigned int order) + unsigned int flags) { return -ENOSYS; } From patchwork Mon Feb 17 13:55:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Wei Liu X-Patchwork-Id: 11386519 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7507A14E3 for ; Mon, 17 Feb 2020 13:56:28 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 50F892070B for ; Mon, 17 Feb 2020 13:56:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mkd7kaGk" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 50F892070B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j3gs0-0006HW-J5; Mon, 17 Feb 2020 13:55:32 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j3grz-0006HK-Ve for xen-devel@lists.xenproject.org; Mon, 17 Feb 2020 13:55:32 +0000 X-Inumbo-ID: 242ba950-518d-11ea-b0fd-bc764e2007e4 Received: from mail-wr1-x441.google.com (unknown [2a00:1450:4864:20::441]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 242ba950-518d-11ea-b0fd-bc764e2007e4; Mon, 17 Feb 2020 13:55:23 +0000 (UTC) Received: by mail-wr1-x441.google.com with SMTP id y11so19826729wrt.6 for ; Mon, 17 Feb 2020 05:55:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=A9LCktLGPnxDPoPOx5N7Q/Ujame0oJVqmScNTU1P0Tg=; b=mkd7kaGkEFVcK2JgZV9Q4RiznHv3iqNQX2qKbZ8hZm1c8c2vjuWaD1c/9fMYSWpNl3 UqHyORN1EHQZhZitfGbDiN7x8vXeb7xiTTlz3FCzO9WRwE3sYDa0OkLzNPCPoNuapQwI UQjxyOrQtOmoHuru1N5d+PaAQ92gbetbaW69rvK9sMcSx+cAZLNC3/aw6ogbLzm1CAD4 Dyhs9tQHVYK99/h7FtHAGhEpA99iuld6YwSRPziDxA17VCNR5tBdJoPwnHKUoOt1gNxh 1DW+9r/25X2IK4BZb4L+LKYXMLJhwJcjTa0KE1l8ZLTZMsgq5nF0lVZNpxinZOsk1cxg bwjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=A9LCktLGPnxDPoPOx5N7Q/Ujame0oJVqmScNTU1P0Tg=; b=jvZOVZrhB4JjLRerzlNHv1aN1cO7aBEellICPswwGOEUTp+b7LDqaJ0cODYRIWdHfp gv4nrwpATSM1cvcUQqa4gmfiSndNPUA0O+5CGRR45Pvs7yxQYRjvpdbswZ2toCLmAsZI wO4nysRBtJwu4qiKv9AumGaI+hh9xo1Dcd5Hv1Nz5t0EujFjD+gYGEHDR0YypgRrZqI5 VjAg3iEAE1mYjB5rKQc1Etd5aShSgM8E6OwcQwJ6iZSCVF5W9mNa4xotaUUatRxpEIP6 nvuNfr2k+fi0lIROu8G/EeiayomndDedTYisSrLTE53vydBXpKh45b8wXv+1NlS9zGnm wIxg== X-Gm-Message-State: APjAAAWYkCikdmXcxum6qozVl5+sqne0geca/UclZF2lmmBA15y9ETuR WFwHmZlS5ylo8Y9GOLDAHAv86X8e X-Google-Smtp-Source: APXvYqyPTfVhJUd6v1Kg1ZYcMPz6Ls3FeN8Q5P7v/UbWUpJoMiPtZL8oz8hvYmnzFoDJtSk+eOhLtg== X-Received: by 2002:adf:b198:: with SMTP id q24mr23067900wra.188.1581947722218; Mon, 17 Feb 2020 05:55:22 -0800 (PST) Received: from localhost.localdomain (41.142.6.51.dyn.plus.net. [51.6.142.41]) by smtp.gmail.com with ESMTPSA id m3sm1021533wrs.53.2020.02.17.05.55.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2020 05:55:21 -0800 (PST) From: Wei Liu X-Google-Original-From: Wei Liu To: Xen Development List Date: Mon, 17 Feb 2020 13:55:16 +0000 Message-Id: <20200217135517.5826-3-liuwe@microsoft.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200217135517.5826-1-liuwe@microsoft.com> References: <20200217135517.5826-1-liuwe@microsoft.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v3 2/3] x86/hyperv: skeleton for L0 assisted TLB flush X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Wei Liu , Wei Liu , Andrew Cooper , Paul Durrant , Michael Kelley , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Implement a basic hook for L0 assisted TLB flush. The hook needs to check if prerequisites are met. If they are not met, it returns an error number to fall back to native flushes. Introduce a new variable to indicate if hypercall page is ready. Signed-off-by: Wei Liu Reviewed-by: Roger Pau Monné Reviewed-by: Paul Durrant --- v3: 1. Change hv_hcall_page_ready to hcall_page_ready --- xen/arch/x86/guest/hyperv/Makefile | 1 + xen/arch/x86/guest/hyperv/hyperv.c | 17 ++++++++++++ xen/arch/x86/guest/hyperv/private.h | 4 +++ xen/arch/x86/guest/hyperv/tlb.c | 41 +++++++++++++++++++++++++++++ 4 files changed, 63 insertions(+) create mode 100644 xen/arch/x86/guest/hyperv/tlb.c diff --git a/xen/arch/x86/guest/hyperv/Makefile b/xen/arch/x86/guest/hyperv/Makefile index 68170109a9..18902c33e9 100644 --- a/xen/arch/x86/guest/hyperv/Makefile +++ b/xen/arch/x86/guest/hyperv/Makefile @@ -1 +1,2 @@ obj-y += hyperv.o +obj-y += tlb.o diff --git a/xen/arch/x86/guest/hyperv/hyperv.c b/xen/arch/x86/guest/hyperv/hyperv.c index 70f4cd5ae0..f1b3073712 100644 --- a/xen/arch/x86/guest/hyperv/hyperv.c +++ b/xen/arch/x86/guest/hyperv/hyperv.c @@ -33,6 +33,8 @@ DEFINE_PER_CPU_READ_MOSTLY(void *, hv_input_page); DEFINE_PER_CPU_READ_MOSTLY(void *, hv_vp_assist); DEFINE_PER_CPU_READ_MOSTLY(unsigned int, hv_vp_index); +static bool __read_mostly hcall_page_ready; + static uint64_t generate_guest_id(void) { union hv_guest_os_id id = {}; @@ -119,6 +121,8 @@ static void __init setup_hypercall_page(void) BUG_ON(!hypercall_msr.enable); set_fixmap_x(FIX_X_HYPERV_HCALL, mfn << PAGE_SHIFT); + + hcall_page_ready = true; } static int setup_hypercall_pcpu_arg(void) @@ -199,11 +203,24 @@ static void __init e820_fixup(struct e820map *e820) panic("Unable to reserve Hyper-V hypercall range\n"); } +static int flush_tlb(const cpumask_t *mask, const void *va, + unsigned int flags) +{ + if ( !(ms_hyperv.hints & HV_X64_REMOTE_TLB_FLUSH_RECOMMENDED) ) + return -EOPNOTSUPP; + + if ( !hcall_page_ready || !this_cpu(hv_input_page) ) + return -ENXIO; + + return hyperv_flush_tlb(mask, va, flags); +} + static const struct hypervisor_ops __initdata ops = { .name = "Hyper-V", .setup = setup, .ap_setup = ap_setup, .e820_fixup = e820_fixup, + .flush_tlb = flush_tlb, }; /* diff --git a/xen/arch/x86/guest/hyperv/private.h b/xen/arch/x86/guest/hyperv/private.h index 956eff831f..509bedaafa 100644 --- a/xen/arch/x86/guest/hyperv/private.h +++ b/xen/arch/x86/guest/hyperv/private.h @@ -22,10 +22,14 @@ #ifndef __XEN_HYPERV_PRIVIATE_H__ #define __XEN_HYPERV_PRIVIATE_H__ +#include #include DECLARE_PER_CPU(void *, hv_input_page); DECLARE_PER_CPU(void *, hv_vp_assist); DECLARE_PER_CPU(unsigned int, hv_vp_index); +int hyperv_flush_tlb(const cpumask_t *mask, const void *va, + unsigned int flags); + #endif /* __XEN_HYPERV_PRIVIATE_H__ */ diff --git a/xen/arch/x86/guest/hyperv/tlb.c b/xen/arch/x86/guest/hyperv/tlb.c new file mode 100644 index 0000000000..48f527229e --- /dev/null +++ b/xen/arch/x86/guest/hyperv/tlb.c @@ -0,0 +1,41 @@ +/****************************************************************************** + * arch/x86/guest/hyperv/tlb.c + * + * Support for TLB management using hypercalls + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; If not, see . + * + * Copyright (c) 2020 Microsoft. + */ + +#include +#include + +#include "private.h" + +int hyperv_flush_tlb(const cpumask_t *mask, const void *va, + unsigned int flags) +{ + return -EOPNOTSUPP; +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ From patchwork Mon Feb 17 13:55:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Liu X-Patchwork-Id: 11386525 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D2316139A for ; Mon, 17 Feb 2020 13:56:52 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A27E020718 for ; Mon, 17 Feb 2020 13:56:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LWGaO59s" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A27E020718 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j3gs6-0006KR-16; Mon, 17 Feb 2020 13:55:38 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j3gs4-0006Jv-VI for xen-devel@lists.xenproject.org; Mon, 17 Feb 2020 13:55:36 +0000 X-Inumbo-ID: 24d98958-518d-11ea-ade5-bc764e2007e4 Received: from mail-wr1-x442.google.com (unknown [2a00:1450:4864:20::442]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 24d98958-518d-11ea-ade5-bc764e2007e4; Mon, 17 Feb 2020 13:55:24 +0000 (UTC) Received: by mail-wr1-x442.google.com with SMTP id g3so19813193wrs.12 for ; Mon, 17 Feb 2020 05:55:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/n18PxJCMQcDNziI7TYERKCu5xatQmIITtNR1Ioqht4=; b=LWGaO59sIJAREqIYKvNJfeuYFcBQQ4NmYWxk2sYvCARuXJwXHjGRQTYF+yW9cIe8zC eT498WIHwMtqIh8Kc/PxF+KplHddnsIsWYxK5tHQPfNkc411AL3N5uGqxRdFtVTaXMaW xA8nMpnY5wEltqAFKVFYX8p5KHhFprZkXq2N5M+3tTo2LWA7noRozYX2wJq+fdyn8MfQ MeN4XbZGjYwu1Fmi6ok1g1DKTvkugPD5xNvKuqRmj4Fi59X0XY9q/LGOS8UZl7vzli5B WdXRFzWpoqUF6yPty+2vb9iT2CAINwKq0qokqZz36A08SUDCCzFyEW6Cgct3++vuVrbU zj3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=/n18PxJCMQcDNziI7TYERKCu5xatQmIITtNR1Ioqht4=; b=hgUHMo76hes6U8UyRAPUezAQp2pyM8IIyGSfLVx5bl/9g1gTKf/4TegkEzNKe1Sz/H 1sR13zUZm2pfva91Tah/cSKXKK3JfBnYNsawE1lig3M6psuhE/bcOjUJ9uoGqxe8aV6c 1QpErqkRrWsqEp4G0Sx3XzYhOFYL23PlOwtqxOch/Ya6A1j9TxZaLl1wOizHdJRen9CS C/2xYBGNbKt9M1TEca1p/bSFGzKhcJCiSiUZXbxEh3Z5E3RiqSUmZvqApeIsjpQsH4oh yv93VUVnB1TNlvQ9lK535nzWgcRI1e/eqBVG6irmLt6BYH4+gYQfbEpPhY6kuSO7C3oL +0+g== X-Gm-Message-State: APjAAAVpXP9BQHztqtq3PxOqsk5PQgxqh48BbfyoLWtS3j/sNqyD4rZ7 pE09hNzxIo7y310Tg0Dw6Kn7s495 X-Google-Smtp-Source: APXvYqwoGaEtpzAaSbBYELOL4SpTzxl0+XGOReyr3m+XAIN6uzJC/Pq6iodaU/TcKod/y+PfyN2B2A== X-Received: by 2002:adf:f606:: with SMTP id t6mr22339738wrp.304.1581947723152; Mon, 17 Feb 2020 05:55:23 -0800 (PST) Received: from localhost.localdomain (41.142.6.51.dyn.plus.net. [51.6.142.41]) by smtp.gmail.com with ESMTPSA id m3sm1021533wrs.53.2020.02.17.05.55.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2020 05:55:22 -0800 (PST) From: Wei Liu X-Google-Original-From: Wei Liu To: Xen Development List Date: Mon, 17 Feb 2020 13:55:17 +0000 Message-Id: <20200217135517.5826-4-liuwe@microsoft.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200217135517.5826-1-liuwe@microsoft.com> References: <20200217135517.5826-1-liuwe@microsoft.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v3 3/3] x86/hyperv: L0 assisted TLB flush X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Wei Liu , Wei Liu , Andrew Cooper , Paul Durrant , Michael Kelley , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Implement L0 assisted TLB flush for Xen on Hyper-V. It takes advantage of several hypercalls: * HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST * HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX * HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE * HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX Pick the most efficient hypercalls available. Signed-off-by: Wei Liu --- v3: 1. Address more comments. 2. Fix usage of max_vp_index. 3. Use the fill_gva_list algorithm from Linux. v2: 1. Address Roger and Jan's comments re types etc. 2. Fix pointer arithmetic. 3. Misc improvement to code. --- xen/arch/x86/guest/hyperv/Makefile | 1 + xen/arch/x86/guest/hyperv/private.h | 9 ++ xen/arch/x86/guest/hyperv/tlb.c | 173 +++++++++++++++++++++++++++- xen/arch/x86/guest/hyperv/util.c | 74 ++++++++++++ 4 files changed, 256 insertions(+), 1 deletion(-) create mode 100644 xen/arch/x86/guest/hyperv/util.c diff --git a/xen/arch/x86/guest/hyperv/Makefile b/xen/arch/x86/guest/hyperv/Makefile index 18902c33e9..0e39410968 100644 --- a/xen/arch/x86/guest/hyperv/Makefile +++ b/xen/arch/x86/guest/hyperv/Makefile @@ -1,2 +1,3 @@ obj-y += hyperv.o obj-y += tlb.o +obj-y += util.o diff --git a/xen/arch/x86/guest/hyperv/private.h b/xen/arch/x86/guest/hyperv/private.h index 509bedaafa..79a77930a0 100644 --- a/xen/arch/x86/guest/hyperv/private.h +++ b/xen/arch/x86/guest/hyperv/private.h @@ -24,12 +24,21 @@ #include #include +#include DECLARE_PER_CPU(void *, hv_input_page); DECLARE_PER_CPU(void *, hv_vp_assist); DECLARE_PER_CPU(unsigned int, hv_vp_index); +static inline unsigned int hv_vp_index(unsigned int cpu) +{ + return per_cpu(hv_vp_index, cpu); +} + int hyperv_flush_tlb(const cpumask_t *mask, const void *va, unsigned int flags); +/* Returns number of banks, -ev if error */ +int cpumask_to_vpset(struct hv_vpset *vpset, const cpumask_t *mask); + #endif /* __XEN_HYPERV_PRIVIATE_H__ */ diff --git a/xen/arch/x86/guest/hyperv/tlb.c b/xen/arch/x86/guest/hyperv/tlb.c index 48f527229e..8cd1c6f0ed 100644 --- a/xen/arch/x86/guest/hyperv/tlb.c +++ b/xen/arch/x86/guest/hyperv/tlb.c @@ -19,17 +19,188 @@ * Copyright (c) 2020 Microsoft. */ +#include #include #include +#include +#include +#include + #include "private.h" +/* + * It is possible to encode up to 4096 pages using the lower 12 bits + * in an element of gva_list + */ +#define HV_TLB_FLUSH_UNIT (4096 * PAGE_SIZE) + +static unsigned int fill_gva_list(uint64_t *gva_list, const void *va, + unsigned int order) +{ + unsigned long cur = (unsigned long)va; + /* end is 1 past the range to be flushed */ + unsigned long end = cur + (PAGE_SIZE << order); + unsigned int n = 0; + + do { + unsigned long diff = end - cur; + + gva_list[n] = cur & PAGE_MASK; + + /* + * Use lower 12 bits to encode the number of additional pages + * to flush + */ + if ( diff >= HV_TLB_FLUSH_UNIT ) + { + gva_list[n] |= ~PAGE_MASK; + cur += HV_TLB_FLUSH_UNIT; + } + else + { + gva_list[n] |= (diff - 1) >> PAGE_SHIFT; + cur = end; + } + + n++; + } while ( cur < end ); + + return n; +} + +static uint64_t flush_tlb_ex(const cpumask_t *mask, const void *va, + unsigned int flags) +{ + struct hv_tlb_flush_ex *flush = this_cpu(hv_input_page); + int nr_banks; + unsigned int max_gvas, order = flags & FLUSH_ORDER_MASK; + uint64_t *gva_list; + + if ( !flush || local_irq_is_enabled() ) + { + ASSERT_UNREACHABLE(); + return ~0ULL; + } + + if ( !(ms_hyperv.hints & HV_X64_EX_PROCESSOR_MASKS_RECOMMENDED) ) + return ~0ULL; + + flush->address_space = 0; + flush->flags = HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES; + if ( !(flags & FLUSH_TLB_GLOBAL) ) + flush->flags |= HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY; + + nr_banks = cpumask_to_vpset(&flush->hv_vp_set, mask); + if ( nr_banks < 0 ) + return ~0ULL; + + max_gvas = + (PAGE_SIZE - sizeof(*flush) - nr_banks * + sizeof(flush->hv_vp_set.bank_contents[0])) / + sizeof(uint64_t); /* gva is represented as uint64_t */ + + /* + * Flush the entire address space if va is NULL or if there is not + * enough space for gva_list. + */ + if ( !va || (PAGE_SIZE << order) / HV_TLB_FLUSH_UNIT > max_gvas ) + return hv_do_rep_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX, 0, + nr_banks, virt_to_maddr(flush), 0); + + /* + * The calculation of gva_list address requires the structure to + * be 64 bits aligned. + */ + BUILD_BUG_ON(sizeof(*flush) % sizeof(uint64_t)); + gva_list = (uint64_t *)flush + sizeof(*flush) / sizeof(uint64_t) + nr_banks; + + return hv_do_rep_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX, + fill_gva_list(gva_list, va, order), + nr_banks, virt_to_maddr(flush), 0); +} + +/* Maximum number of gvas for hv_tlb_flush */ +#define MAX_GVAS ((PAGE_SIZE - sizeof(struct hv_tlb_flush)) / sizeof(uint64_t)) + int hyperv_flush_tlb(const cpumask_t *mask, const void *va, unsigned int flags) { - return -EOPNOTSUPP; + unsigned long irq_flags; + struct hv_tlb_flush *flush = this_cpu(hv_input_page); + unsigned int order = flags & FLUSH_ORDER_MASK; + uint64_t ret; + + if ( !flush || cpumask_empty(mask) ) + { + ASSERT_UNREACHABLE(); + return -EINVAL; + } + + local_irq_save(irq_flags); + + flush->address_space = 0; + flush->flags = HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES; + flush->processor_mask = 0; + if ( !(flags & FLUSH_TLB_GLOBAL) ) + flush->flags |= HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY; + + if ( cpumask_equal(mask, &cpu_online_map) ) + flush->flags |= HV_FLUSH_ALL_PROCESSORS; + else + { + unsigned int cpu; + + /* + * Normally VP indices are in ascending order and match Xen's + * idea of CPU ids. Check the last index to see if VP index is + * >= 64. If so, we can skip setting up parameters for + * non-applicable hypercalls without looking further. + */ + if ( hv_vp_index(cpumask_last(mask)) >= 64 ) + goto do_ex_hypercall; + + for_each_cpu ( cpu, mask ) + { + unsigned int vpid = hv_vp_index(cpu); + + if ( vpid >= ms_hyperv.max_vp_index ) + { + local_irq_restore(irq_flags); + return -ENXIO; + } + + if ( vpid >= 64 ) + goto do_ex_hypercall; + + __set_bit(vpid, &flush->processor_mask); + } + } + + /* + * Flush the entire address space if va is NULL or if there is not + * enough space for gva_list. + */ + if ( !va || (PAGE_SIZE << order) / HV_TLB_FLUSH_UNIT > MAX_GVAS ) + ret = hv_do_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE, + virt_to_maddr(flush), 0); + else + ret = hv_do_rep_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST, + fill_gva_list(flush->gva_list, va, order), + 0, virt_to_maddr(flush), 0); + goto done; + + do_ex_hypercall: + ret = flush_tlb_ex(mask, va, flags); + + done: + local_irq_restore(irq_flags); + + return ret & HV_HYPERCALL_RESULT_MASK ? -ENXIO : 0; } +#undef MAX_GVAS + /* * Local variables: * mode: C diff --git a/xen/arch/x86/guest/hyperv/util.c b/xen/arch/x86/guest/hyperv/util.c new file mode 100644 index 0000000000..0abb37b05f --- /dev/null +++ b/xen/arch/x86/guest/hyperv/util.c @@ -0,0 +1,74 @@ +/****************************************************************************** + * arch/x86/guest/hyperv/util.c + * + * Hyper-V utility functions + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; If not, see . + * + * Copyright (c) 2020 Microsoft. + */ + +#include +#include +#include + +#include +#include + +#include "private.h" + +int cpumask_to_vpset(struct hv_vpset *vpset, + const cpumask_t *mask) +{ + int nr = 1; + unsigned int cpu, vcpu_bank, vcpu_offset; + unsigned int max_banks = ms_hyperv.max_vp_index / 64; + + /* Up to 64 banks can be represented by valid_bank_mask */ + if ( max_banks > 64 ) + return -E2BIG; + + /* Clear all banks to avoid flushing unwanted CPUs */ + for ( vcpu_bank = 0; vcpu_bank < max_banks; vcpu_bank++ ) + vpset->bank_contents[vcpu_bank] = 0; + + vpset->valid_bank_mask = 0; + vpset->format = HV_GENERIC_SET_SPARSE_4K; + + for_each_cpu ( cpu, mask ) + { + unsigned int vcpu = hv_vp_index(cpu); + + vcpu_bank = vcpu / 64; + vcpu_offset = vcpu % 64; + + __set_bit(vcpu_offset, &vpset->bank_contents[vcpu_bank]); + __set_bit(vcpu_bank, &vpset->valid_bank_mask); + + if ( vcpu_bank >= nr ) + nr = vcpu_bank + 1; + } + + return nr; +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */