From patchwork Wed Dec 11 10:58:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xia, Hongyan" X-Patchwork-Id: 11284729 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 417E6138C for ; Wed, 11 Dec 2019 10:59:44 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1DE522073B for ; Wed, 11 Dec 2019 10:59:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="D3D+Yzud" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1DE522073B Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iezhj-0005hV-Mz; Wed, 11 Dec 2019 10:58:51 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iezhh-0005h9-UX for xen-devel@lists.xenproject.org; Wed, 11 Dec 2019 10:58:49 +0000 X-Inumbo-ID: 3276b7ce-1c05-11ea-a914-bc764e2007e4 Received: from smtp-fw-33001.amazon.com (unknown [207.171.190.10]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 3276b7ce-1c05-11ea-a914-bc764e2007e4; Wed, 11 Dec 2019 10:58:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1576061924; x=1607597924; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=JUjwXdl/GR2O0CI+PIejfJtLTWo8NOG10go5eNrSrDE=; b=D3D+YzudekhwQXkqyqfLr/oyQEaWrixitFwG+flTO3YhfvDibIdB6+1N RN+lQPH7cymB3wInyzVTvRjTqAyYaLhbWbaibIFv8qnC59KPMXtkUA/Uo N9W51UTZBchDMOcK19+BlZLomZwRNOOOQvcTtdkXKXwyl5zT5fUfrsGR2 I=; IronPort-SDR: v17kUMJY6Nz00hkbAmmflrRrW5WlN9pqJdvZ3HViXkHnPvGlz6c/RJoR8BjFaujsWZorjtorOZ Rtt/jvZ8GBig== X-IronPort-AV: E=Sophos;i="5.69,301,1571702400"; d="scan'208";a="14255745" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-2a-f14f4a47.us-west-2.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-33001.sea14.amazon.com with ESMTP; 11 Dec 2019 10:58:36 +0000 Received: from EX13MTAUEA001.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan2.pdx.amazon.com [10.170.41.162]) by email-inbound-relay-2a-f14f4a47.us-west-2.amazon.com (Postfix) with ESMTPS id F2623A21CD; Wed, 11 Dec 2019 10:58:34 +0000 (UTC) Received: from EX13D37EUA004.ant.amazon.com (10.43.165.124) by EX13MTAUEA001.ant.amazon.com (10.43.61.243) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 11 Dec 2019 10:58:34 +0000 Received: from EX13MTAUWC001.ant.amazon.com (10.43.162.135) by EX13D37EUA004.ant.amazon.com (10.43.165.124) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 11 Dec 2019 10:58:33 +0000 Received: from u60d575063b1f53.ant.amazon.com (10.125.106.64) by mail-relay.amazon.com (10.43.162.232) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Wed, 11 Dec 2019 10:58:30 +0000 From: Hongyan Xia To: Date: Wed, 11 Dec 2019 10:58:26 +0000 Message-ID: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: Bulk Subject: [Xen-devel] [PATCH v3 1/2] x86/mm: factor out the code for shattering an l3 PTE X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , jgrall@amazon.com, Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" map_pages_to_xen and modify_xen_mappings are performing almost exactly the same operations when shattering an l3 PTE, the only difference being whether we want to flush. Signed-off-by: Hongyan Xia --- Changes in v3: - style and indentation changes. - return -ENOMEM instead of -1. Changes in v2: - improve asm. - re-read pl3e from memory when taking the lock. - move the allocation of l2t inside the shatter function. --- xen/arch/x86/mm.c | 98 +++++++++++++++++++++++------------------------ 1 file changed, 49 insertions(+), 49 deletions(-) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 7d4dd80a85..97f11b6016 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -5151,6 +5151,52 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long v) flush_area_local((const void *)v, f) : \ flush_area_all((const void *)v, f)) +/* Shatter an l3 entry and populate l2. If virt is passed in, also do flush. */ +static int shatter_l3e(l3_pgentry_t *pl3e, unsigned long virt, bool locking) +{ + unsigned int i; + l3_pgentry_t ol3e = *pl3e; + l2_pgentry_t l2e = l2e_from_intpte(l3e_get_intpte(ol3e)); + l2_pgentry_t *l2t = alloc_xen_pagetable(); + + if ( !l2t ) + return -ENOMEM; + + for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ ) + { + l2e_write(l2t + i, l2e); + l2e = l2e_from_intpte( + l2e_get_intpte(l2e) + (PAGE_SIZE << PAGETABLE_ORDER)); + } + + if ( locking ) + spin_lock(&map_pgdir_lock); + if ( (l3e_get_flags(*pl3e) & _PAGE_PRESENT) && + (l3e_get_flags(*pl3e) & _PAGE_PSE) ) + { + l3e_write_atomic(pl3e, + l3e_from_paddr((paddr_t)virt_to_maddr(l2t), __PAGE_HYPERVISOR)); + l2t = NULL; + } + if ( locking ) + spin_unlock(&map_pgdir_lock); + + if ( virt ) + { + unsigned int flush_flags = + FLUSH_TLB | FLUSH_ORDER(2 * PAGETABLE_ORDER); + + if ( l3e_get_flags(ol3e) & _PAGE_GLOBAL ) + flush_flags |= FLUSH_TLB_GLOBAL; + flush_area(virt, flush_flags); + } + + if ( l2t ) + free_xen_pagetable(l2t); + + return 0; +} + int map_pages_to_xen( unsigned long virt, mfn_t mfn, @@ -5244,9 +5290,6 @@ int map_pages_to_xen( if ( (l3e_get_flags(ol3e) & _PAGE_PRESENT) && (l3e_get_flags(ol3e) & _PAGE_PSE) ) { - unsigned int flush_flags = - FLUSH_TLB | FLUSH_ORDER(2 * PAGETABLE_ORDER); - /* Skip this PTE if there is no change. */ if ( ((l3e_get_pfn(ol3e) & ~(L2_PAGETABLE_ENTRIES * L1_PAGETABLE_ENTRIES - 1)) + @@ -5267,33 +5310,9 @@ int map_pages_to_xen( continue; } - pl2e = alloc_xen_pagetable(); - if ( pl2e == NULL ) + /* Pass virt to indicate we need to flush. */ + if ( shatter_l3e(pl3e, virt, locking) ) return -ENOMEM; - - for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ ) - l2e_write(pl2e + i, - l2e_from_pfn(l3e_get_pfn(ol3e) + - (i << PAGETABLE_ORDER), - l3e_get_flags(ol3e))); - - if ( l3e_get_flags(ol3e) & _PAGE_GLOBAL ) - flush_flags |= FLUSH_TLB_GLOBAL; - - if ( locking ) - spin_lock(&map_pgdir_lock); - if ( (l3e_get_flags(*pl3e) & _PAGE_PRESENT) && - (l3e_get_flags(*pl3e) & _PAGE_PSE) ) - { - l3e_write_atomic(pl3e, l3e_from_mfn(virt_to_mfn(pl2e), - __PAGE_HYPERVISOR)); - pl2e = NULL; - } - if ( locking ) - spin_unlock(&map_pgdir_lock); - flush_area(virt, flush_flags); - if ( pl2e ) - free_xen_pagetable(pl2e); } pl2e = virt_to_xen_l2e(virt); @@ -5578,27 +5597,8 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) } /* PAGE1GB: shatter the superpage and fall through. */ - pl2e = alloc_xen_pagetable(); - if ( !pl2e ) + if ( shatter_l3e(pl3e, 0, locking) ) return -ENOMEM; - for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ ) - l2e_write(pl2e + i, - l2e_from_pfn(l3e_get_pfn(*pl3e) + - (i << PAGETABLE_ORDER), - l3e_get_flags(*pl3e))); - if ( locking ) - spin_lock(&map_pgdir_lock); - if ( (l3e_get_flags(*pl3e) & _PAGE_PRESENT) && - (l3e_get_flags(*pl3e) & _PAGE_PSE) ) - { - l3e_write_atomic(pl3e, l3e_from_mfn(virt_to_mfn(pl2e), - __PAGE_HYPERVISOR)); - pl2e = NULL; - } - if ( locking ) - spin_unlock(&map_pgdir_lock); - if ( pl2e ) - free_xen_pagetable(pl2e); } /* From patchwork Wed Dec 11 10:58:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xia, Hongyan" X-Patchwork-Id: 11284727 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9811C14E3 for ; Wed, 11 Dec 2019 10:59:42 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 74E892073B for ; Wed, 11 Dec 2019 10:59:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="Ny+AYvRB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 74E892073B Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iezhZ-0005fl-4a; Wed, 11 Dec 2019 10:58:41 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iezhX-0005fc-UZ for xen-devel@lists.xenproject.org; Wed, 11 Dec 2019 10:58:39 +0000 X-Inumbo-ID: 302cb0d6-1c05-11ea-b6f1-bc764e2007e4 Received: from smtp-fw-2101.amazon.com (unknown [72.21.196.25]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 302cb0d6-1c05-11ea-b6f1-bc764e2007e4; Wed, 11 Dec 2019 10:58:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1576061920; x=1607597920; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=rQBwh9YMPKQJWLyox234wAEmr+DqeCvTByOZ44gtSuo=; b=Ny+AYvRBsy6Ce3dGnaJuSFRNjfIxTAZdryQfBvvQj9yyX0pBYiJhPRbH BythTsZYHj6PUdoh1tni51P9/PppdWcljQGB3SZeBfkXWSIJRyvGmn3ZM gIO1EGVwP4+cO+J8Rcp5/uNqjUaWzIxE11xoOysyZ3C7oHoZYLjBiy0P9 4=; IronPort-SDR: 6wJqfiG9pmNeCDetnzeEge/WSEYusKn+7j5RBp3pkGmYW/4zNDZvrqRz+OaKdXutn2Nxl2LKEd SuFebbfM+7sQ== X-IronPort-AV: E=Sophos;i="5.69,301,1571702400"; d="scan'208";a="8076596" Received: from iad6-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-2c-87a10be6.us-west-2.amazon.com) ([10.124.125.2]) by smtp-border-fw-out-2101.iad2.amazon.com with ESMTP; 11 Dec 2019 10:58:39 +0000 Received: from EX13MTAUEA001.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan2.pdx.amazon.com [10.170.41.162]) by email-inbound-relay-2c-87a10be6.us-west-2.amazon.com (Postfix) with ESMTPS id 1F721A232D; Wed, 11 Dec 2019 10:58:38 +0000 (UTC) Received: from EX13D37EUA003.ant.amazon.com (10.43.165.7) by EX13MTAUEA001.ant.amazon.com (10.43.61.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 11 Dec 2019 10:58:37 +0000 Received: from EX13MTAUWC001.ant.amazon.com (10.43.162.135) by EX13D37EUA003.ant.amazon.com (10.43.165.7) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 11 Dec 2019 10:58:36 +0000 Received: from u60d575063b1f53.ant.amazon.com (10.125.106.64) by mail-relay.amazon.com (10.43.162.232) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Wed, 11 Dec 2019 10:58:33 +0000 From: Hongyan Xia To: Date: Wed, 11 Dec 2019 10:58:27 +0000 Message-ID: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: Bulk Subject: [Xen-devel] [PATCH v3 2/2] x86/mm: factor out the code for shattering an l2 PTE X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , jgrall@amazon.com, Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" map_pages_to_xen and modify_xen_mappings are performing almost exactly the same operations when shattering an l2 PTE, the only difference being whether we want to flush. Signed-off-by: Hongyan Xia --- Changes in v3: - style and indentation changes. - return -ENOMEM instead of -1. Changes in v2: - improve asm. - re-read pl2e from memory when taking the lock. - move the allocation of l1t inside the shatter function. --- xen/arch/x86/mm.c | 96 ++++++++++++++++++++++++----------------------- 1 file changed, 49 insertions(+), 47 deletions(-) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 97f11b6016..e5ba6b52fb 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -5151,6 +5151,52 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long v) flush_area_local((const void *)v, f) : \ flush_area_all((const void *)v, f)) +/* Shatter an l2 entry and populate l1. If virt is passed in, also do flush. */ +static int shatter_l2e(l2_pgentry_t *pl2e, unsigned long virt, bool locking) +{ + unsigned int i; + l2_pgentry_t ol2e = *pl2e; + l1_pgentry_t l1e = l1e_from_paddr(l2e_get_paddr(ol2e), + lNf_to_l1f(l2e_get_flags(ol2e))); + l1_pgentry_t *l1t = alloc_xen_pagetable(); + + if ( !l1t ) + return -ENOMEM; + + for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ ) + { + l1e_write(l1t + i, l1e); + l1e = l1e_from_intpte(l1e_get_intpte(l1e) + PAGE_SIZE); + } + + if ( locking ) + spin_lock(&map_pgdir_lock); + if ( (l2e_get_flags(*pl2e) & _PAGE_PRESENT) && + (l2e_get_flags(*pl2e) & _PAGE_PSE) ) + { + l2e_write_atomic(pl2e, + l2e_from_paddr((paddr_t)virt_to_maddr(l1t), __PAGE_HYPERVISOR)); + l1t = NULL; + } + if ( locking ) + spin_unlock(&map_pgdir_lock); + + if ( virt ) + { + unsigned int flush_flags = + FLUSH_TLB | FLUSH_ORDER(PAGETABLE_ORDER); + + if ( l2e_get_flags(ol2e) & _PAGE_GLOBAL ) + flush_flags |= FLUSH_TLB_GLOBAL; + flush_area(virt, flush_flags); + } + + if ( l1t ) + free_xen_pagetable(l1t); + + return 0; +} + /* Shatter an l3 entry and populate l2. If virt is passed in, also do flush. */ static int shatter_l3e(l3_pgentry_t *pl3e, unsigned long virt, bool locking) { @@ -5364,9 +5410,6 @@ int map_pages_to_xen( } else if ( l2e_get_flags(*pl2e) & _PAGE_PSE ) { - unsigned int flush_flags = - FLUSH_TLB | FLUSH_ORDER(PAGETABLE_ORDER); - /* Skip this PTE if there is no change. */ if ( (((l2e_get_pfn(*pl2e) & ~(L1_PAGETABLE_ENTRIES - 1)) + l1_table_offset(virt)) == mfn_x(mfn)) && @@ -5385,32 +5428,9 @@ int map_pages_to_xen( goto check_l3; } - pl1e = alloc_xen_pagetable(); - if ( pl1e == NULL ) + /* Pass virt to indicate we need to flush. */ + if ( shatter_l2e(pl2e, virt, locking) ) return -ENOMEM; - - for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ ) - l1e_write(&pl1e[i], - l1e_from_pfn(l2e_get_pfn(*pl2e) + i, - lNf_to_l1f(l2e_get_flags(*pl2e)))); - - if ( l2e_get_flags(*pl2e) & _PAGE_GLOBAL ) - flush_flags |= FLUSH_TLB_GLOBAL; - - if ( locking ) - spin_lock(&map_pgdir_lock); - if ( (l2e_get_flags(*pl2e) & _PAGE_PRESENT) && - (l2e_get_flags(*pl2e) & _PAGE_PSE) ) - { - l2e_write_atomic(pl2e, l2e_from_mfn(virt_to_mfn(pl1e), - __PAGE_HYPERVISOR)); - pl1e = NULL; - } - if ( locking ) - spin_unlock(&map_pgdir_lock); - flush_area(virt, flush_flags); - if ( pl1e ) - free_xen_pagetable(pl1e); } pl1e = l2e_to_l1e(*pl2e) + l1_table_offset(virt); @@ -5633,26 +5653,8 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) else { /* PSE: shatter the superpage and try again. */ - pl1e = alloc_xen_pagetable(); - if ( !pl1e ) + if ( shatter_l2e(pl2e, 0, locking) ) return -ENOMEM; - for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ ) - l1e_write(&pl1e[i], - l1e_from_pfn(l2e_get_pfn(*pl2e) + i, - l2e_get_flags(*pl2e) & ~_PAGE_PSE)); - if ( locking ) - spin_lock(&map_pgdir_lock); - if ( (l2e_get_flags(*pl2e) & _PAGE_PRESENT) && - (l2e_get_flags(*pl2e) & _PAGE_PSE) ) - { - l2e_write_atomic(pl2e, l2e_from_mfn(virt_to_mfn(pl1e), - __PAGE_HYPERVISOR)); - pl1e = NULL; - } - if ( locking ) - spin_unlock(&map_pgdir_lock); - if ( pl1e ) - free_xen_pagetable(pl1e); } } else