From patchwork Mon Dec 9 11:48:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xia, Hongyan" X-Patchwork-Id: 11279215 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A51413B6 for ; Mon, 9 Dec 2019 11:49:27 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 36C6820726 for ; Mon, 9 Dec 2019 11:49:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="ChyA/pQK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 36C6820726 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ieHWg-0000Xm-P0; Mon, 09 Dec 2019 11:48:30 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ieHWf-0000Xg-Lz for xen-devel@lists.xenproject.org; Mon, 09 Dec 2019 11:48:29 +0000 X-Inumbo-ID: d0eec438-1a79-11ea-87bf-12813bfff9fa Received: from smtp-fw-9102.amazon.com (unknown [207.171.184.29]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d0eec438-1a79-11ea-87bf-12813bfff9fa; Mon, 09 Dec 2019 11:48:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1575892110; x=1607428110; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=oo5sR3iAItOanlY8zK50SN3aUNRVHnZwFG+Pu4wKHCA=; b=ChyA/pQK9/HeLi29mSjh8UoB4HsYXs6z6q8EGuZsnVXoG2zvVBCermJY 74FviMMxM0XhPjWVWonkcXGfJnjvcbS6bae5ibIUXHTLkjad1pnz8CXN5 uEWs2Fzul4R0hqxiOkj0KJBVVe5oLglAhIQczncC7xQcQC+wJ+gWeSCy+ Q=; IronPort-SDR: Ulkr5Mdc7f6PDLNoqFXb3rmgXEYbSqSBGADPAGHdtrAyAchAd3CpJhMR2Y3gYWXXyQFr4SMp+J uYjrZV9ps5TQ== X-IronPort-AV: E=Sophos;i="5.69,294,1571702400"; d="scan'208";a="12415702" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-1d-37fd6b3d.us-east-1.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP; 09 Dec 2019 11:48:18 +0000 Received: from EX13MTAUEA001.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan2.iad.amazon.com [10.40.159.162]) by email-inbound-relay-1d-37fd6b3d.us-east-1.amazon.com (Postfix) with ESMTPS id A5FB5282150; Mon, 9 Dec 2019 11:48:15 +0000 (UTC) Received: from EX13D37EUB002.ant.amazon.com (10.43.166.116) by EX13MTAUEA001.ant.amazon.com (10.43.61.243) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Mon, 9 Dec 2019 11:48:15 +0000 Received: from EX13MTAUEE001.ant.amazon.com (10.43.62.200) by EX13D37EUB002.ant.amazon.com (10.43.166.116) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Mon, 9 Dec 2019 11:48:14 +0000 Received: from u60d575063b1f53.ant.amazon.com (10.125.106.64) by mail-relay.amazon.com (10.43.62.226) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Mon, 9 Dec 2019 11:48:13 +0000 From: Hongyan Xia To: Date: Mon, 9 Dec 2019 11:48:09 +0000 Message-ID: <3375af1e708b4ec3205f493a17da6e0369249096.1575891620.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: Bulk Subject: [Xen-devel] [PATCH v2 1/2] x86/mm: factor out the code for shattering an l3 PTE X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , jgrall@amazon.com, Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" map_pages_to_xen and modify_xen_mappings are performing almost exactly the same operations when shattering an l3 PTE, the only difference being whether we want to flush. Signed-off-by: Hongyan Xia --- Changes in v2: - improve asm. - re-read pl3e from memory when taking the lock. - move the allocation of l2t inside the shatter function. --- xen/arch/x86/mm.c | 97 +++++++++++++++++++++++------------------------ 1 file changed, 48 insertions(+), 49 deletions(-) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 7d4dd80a85..6188a968ff 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -5151,6 +5151,51 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long v) flush_area_local((const void *)v, f) : \ flush_area_all((const void *)v, f)) +/* Shatter an l3 entry and populate l2. If virt is passed in, also do flush. */ +static int shatter_l3e(l3_pgentry_t *pl3e, unsigned long virt, bool locking) +{ + unsigned int i; + l3_pgentry_t ol3e; + l2_pgentry_t ol2e, *l2t = alloc_xen_pagetable(); + + if ( l2t == NULL ) + return -1; + + ol3e = *pl3e; + ol2e = l2e_from_intpte(l3e_get_intpte(ol3e)); + + for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ ) + { + l2e_write(l2t + i, ol2e); + ol2e = l2e_from_intpte( + l2e_get_intpte(ol2e) + (1 << (PAGETABLE_ORDER + PAGE_SHIFT))); + } + if ( locking ) + spin_lock(&map_pgdir_lock); + if ( (l3e_get_flags(*pl3e) & _PAGE_PRESENT) && + (l3e_get_flags(*pl3e) & _PAGE_PSE) ) + { + l3e_write_atomic(pl3e, + l3e_from_paddr((paddr_t)virt_to_maddr(l2t), __PAGE_HYPERVISOR)); + l2t = NULL; + } + if ( locking ) + spin_unlock(&map_pgdir_lock); + if ( virt ) + { + unsigned int flush_flags = + FLUSH_TLB | FLUSH_ORDER(2 * PAGETABLE_ORDER); + + if ( (l3e_get_flags(ol3e) & _PAGE_GLOBAL) ) + flush_flags |= FLUSH_TLB_GLOBAL; + flush_area(virt, flush_flags); + } + if ( l2t ) + free_xen_pagetable(l2t); + + return 0; +} + int map_pages_to_xen( unsigned long virt, mfn_t mfn, @@ -5244,9 +5289,6 @@ int map_pages_to_xen( if ( (l3e_get_flags(ol3e) & _PAGE_PRESENT) && (l3e_get_flags(ol3e) & _PAGE_PSE) ) { - unsigned int flush_flags = - FLUSH_TLB | FLUSH_ORDER(2 * PAGETABLE_ORDER); - /* Skip this PTE if there is no change. */ if ( ((l3e_get_pfn(ol3e) & ~(L2_PAGETABLE_ENTRIES * L1_PAGETABLE_ENTRIES - 1)) + @@ -5267,33 +5309,9 @@ int map_pages_to_xen( continue; } - pl2e = alloc_xen_pagetable(); - if ( pl2e == NULL ) + /* Pass virt to indicate we need to flush. */ + if ( shatter_l3e(pl3e, virt, locking) ) return -ENOMEM; - - for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ ) - l2e_write(pl2e + i, - l2e_from_pfn(l3e_get_pfn(ol3e) + - (i << PAGETABLE_ORDER), - l3e_get_flags(ol3e))); - - if ( l3e_get_flags(ol3e) & _PAGE_GLOBAL ) - flush_flags |= FLUSH_TLB_GLOBAL; - - if ( locking ) - spin_lock(&map_pgdir_lock); - if ( (l3e_get_flags(*pl3e) & _PAGE_PRESENT) && - (l3e_get_flags(*pl3e) & _PAGE_PSE) ) - { - l3e_write_atomic(pl3e, l3e_from_mfn(virt_to_mfn(pl2e), - __PAGE_HYPERVISOR)); - pl2e = NULL; - } - if ( locking ) - spin_unlock(&map_pgdir_lock); - flush_area(virt, flush_flags); - if ( pl2e ) - free_xen_pagetable(pl2e); } pl2e = virt_to_xen_l2e(virt); @@ -5578,27 +5596,8 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) } /* PAGE1GB: shatter the superpage and fall through. */ - pl2e = alloc_xen_pagetable(); - if ( !pl2e ) + if ( shatter_l3e(pl3e, 0, locking) ) return -ENOMEM; - for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ ) - l2e_write(pl2e + i, - l2e_from_pfn(l3e_get_pfn(*pl3e) + - (i << PAGETABLE_ORDER), - l3e_get_flags(*pl3e))); - if ( locking ) - spin_lock(&map_pgdir_lock); - if ( (l3e_get_flags(*pl3e) & _PAGE_PRESENT) && - (l3e_get_flags(*pl3e) & _PAGE_PSE) ) - { - l3e_write_atomic(pl3e, l3e_from_mfn(virt_to_mfn(pl2e), - __PAGE_HYPERVISOR)); - pl2e = NULL; - } - if ( locking ) - spin_unlock(&map_pgdir_lock); - if ( pl2e ) - free_xen_pagetable(pl2e); } /* From patchwork Mon Dec 9 11:48:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xia, Hongyan" X-Patchwork-Id: 11279213 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E2C3A930 for ; Mon, 9 Dec 2019 11:49:25 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BCEFF20726 for ; Mon, 9 Dec 2019 11:49:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="eNdVNJqD" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BCEFF20726 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ieHWm-0000YX-0p; Mon, 09 Dec 2019 11:48:36 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ieHWk-0000YM-LF for xen-devel@lists.xenproject.org; Mon, 09 Dec 2019 11:48:34 +0000 X-Inumbo-ID: d159763e-1a79-11ea-87bf-12813bfff9fa Received: from smtp-fw-2101.amazon.com (unknown [72.21.196.25]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d159763e-1a79-11ea-87bf-12813bfff9fa; Mon, 09 Dec 2019 11:48:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1575892109; x=1607428109; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=e4KvUPuat3fhtWxWLcqB9Dso2chveuQ7wueZkCtCGzg=; b=eNdVNJqDFH/zzjmpx0QaULin1HdKBWTNTje8DTT0ymjbSUCcDRMmyHbF HZsudR0NxrN7rZkfwWYr6tBtfIr8JGwouxvUtRqroWSrXcdpnSkeCaU/J 31fA9Th1dfhWmM0sR8NwSJ5ImXX/GmNKwvRgN1b/fmxurZVN0QS5/MVNZ U=; IronPort-SDR: lPfUEaEyFRG1kly5WBotmk8F+c3uSrTM4nLRLcabpYKJuIDetQC5O0m9XrnaeNQzpbT72FOhoC 1MbmpPE0BWDA== X-IronPort-AV: E=Sophos;i="5.69,294,1571702400"; d="scan'208";a="7726518" Received: from iad6-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-1d-98acfc19.us-east-1.amazon.com) ([10.124.125.2]) by smtp-border-fw-out-2101.iad2.amazon.com with ESMTP; 09 Dec 2019 11:48:21 +0000 Received: from EX13MTAUEA001.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan3.iad.amazon.com [10.40.159.166]) by email-inbound-relay-1d-98acfc19.us-east-1.amazon.com (Postfix) with ESMTPS id 5B19DA1E72; Mon, 9 Dec 2019 11:48:19 +0000 (UTC) Received: from EX13D37EUA003.ant.amazon.com (10.43.165.7) by EX13MTAUEA001.ant.amazon.com (10.43.61.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Mon, 9 Dec 2019 11:48:18 +0000 Received: from EX13MTAUEE001.ant.amazon.com (10.43.62.200) by EX13D37EUA003.ant.amazon.com (10.43.165.7) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Mon, 9 Dec 2019 11:48:16 +0000 Received: from u60d575063b1f53.ant.amazon.com (10.125.106.64) by mail-relay.amazon.com (10.43.62.226) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Mon, 9 Dec 2019 11:48:14 +0000 From: Hongyan Xia To: Date: Mon, 9 Dec 2019 11:48:10 +0000 Message-ID: <46ca4f3baff325a2aa558783a8dc3de215286d2e.1575891620.git.hongyxia@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: Bulk Subject: [Xen-devel] [PATCH v2 2/2] x86/mm: factor out the code for shattering an l2 PTE X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , jgrall@amazon.com, Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_?= =?utf-8?q?Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" map_pages_to_xen and modify_xen_mappings are performing almost exactly the same operations when shattering an l2 PTE, the only difference being whether we want to flush. Signed-off-by: Hongyan Xia --- Changes in v2: - improve asm. - re-read pl2e from memory when taking the lock. - move the allocation of l1t inside the shatter function. --- xen/arch/x86/mm.c | 95 ++++++++++++++++++++++++----------------------- 1 file changed, 48 insertions(+), 47 deletions(-) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 6188a968ff..103c97b903 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -5151,6 +5151,51 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long v) flush_area_local((const void *)v, f) : \ flush_area_all((const void *)v, f)) +/* Shatter an l2 entry and populate l1. If virt is passed in, also do flush. */ +static int shatter_l2e(l2_pgentry_t *pl2e, unsigned long virt, bool locking) +{ + unsigned int i; + l2_pgentry_t ol2e; + l1_pgentry_t ol1e, *l1t = alloc_xen_pagetable(); + + if ( l1t == NULL ) + return -1; + + ol2e = *pl2e; + ol1e = l1e_from_paddr(l2e_get_paddr(ol2e), lNf_to_l1f(l2e_get_flags(ol2e))); + + for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ ) + { + l1e_write(l1t + i, ol1e); + ol1e = l1e_from_intpte( + l1e_get_intpte(ol1e) + (1 << PAGE_SHIFT)); + } + if ( locking ) + spin_lock(&map_pgdir_lock); + if ( (l2e_get_flags(*pl2e) & _PAGE_PRESENT) && + (l2e_get_flags(*pl2e) & _PAGE_PSE) ) + { + l2e_write_atomic(pl2e, + l2e_from_paddr((paddr_t)virt_to_maddr(l1t), __PAGE_HYPERVISOR)); + l1t = NULL; + } + if ( locking ) + spin_unlock(&map_pgdir_lock); + if ( virt ) + { + unsigned int flush_flags = + FLUSH_TLB | FLUSH_ORDER(PAGETABLE_ORDER); + + if ( l2e_get_flags(ol2e) & _PAGE_GLOBAL ) + flush_flags |= FLUSH_TLB_GLOBAL; + flush_area(virt, flush_flags); + } + if ( l1t ) + free_xen_pagetable(l1t); + + return 0; +} + /* Shatter an l3 entry and populate l2. If virt is passed in, also do flush. */ static int shatter_l3e(l3_pgentry_t *pl3e, unsigned long virt, bool locking) { @@ -5363,9 +5408,6 @@ int map_pages_to_xen( } else if ( l2e_get_flags(*pl2e) & _PAGE_PSE ) { - unsigned int flush_flags = - FLUSH_TLB | FLUSH_ORDER(PAGETABLE_ORDER); - /* Skip this PTE if there is no change. */ if ( (((l2e_get_pfn(*pl2e) & ~(L1_PAGETABLE_ENTRIES - 1)) + l1_table_offset(virt)) == mfn_x(mfn)) && @@ -5384,32 +5426,9 @@ int map_pages_to_xen( goto check_l3; } - pl1e = alloc_xen_pagetable(); - if ( pl1e == NULL ) + /* Pass virt to indicate we need to flush. */ + if ( shatter_l2e(pl2e, virt, locking) ) return -ENOMEM; - - for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ ) - l1e_write(&pl1e[i], - l1e_from_pfn(l2e_get_pfn(*pl2e) + i, - lNf_to_l1f(l2e_get_flags(*pl2e)))); - - if ( l2e_get_flags(*pl2e) & _PAGE_GLOBAL ) - flush_flags |= FLUSH_TLB_GLOBAL; - - if ( locking ) - spin_lock(&map_pgdir_lock); - if ( (l2e_get_flags(*pl2e) & _PAGE_PRESENT) && - (l2e_get_flags(*pl2e) & _PAGE_PSE) ) - { - l2e_write_atomic(pl2e, l2e_from_mfn(virt_to_mfn(pl1e), - __PAGE_HYPERVISOR)); - pl1e = NULL; - } - if ( locking ) - spin_unlock(&map_pgdir_lock); - flush_area(virt, flush_flags); - if ( pl1e ) - free_xen_pagetable(pl1e); } pl1e = l2e_to_l1e(*pl2e) + l1_table_offset(virt); @@ -5632,26 +5651,8 @@ int modify_xen_mappings(unsigned long s, unsigned long e, unsigned int nf) else { /* PSE: shatter the superpage and try again. */ - pl1e = alloc_xen_pagetable(); - if ( !pl1e ) + if ( shatter_l2e(pl2e, 0, locking) ) return -ENOMEM; - for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ ) - l1e_write(&pl1e[i], - l1e_from_pfn(l2e_get_pfn(*pl2e) + i, - l2e_get_flags(*pl2e) & ~_PAGE_PSE)); - if ( locking ) - spin_lock(&map_pgdir_lock); - if ( (l2e_get_flags(*pl2e) & _PAGE_PRESENT) && - (l2e_get_flags(*pl2e) & _PAGE_PSE) ) - { - l2e_write_atomic(pl2e, l2e_from_mfn(virt_to_mfn(pl1e), - __PAGE_HYPERVISOR)); - pl1e = NULL; - } - if ( locking ) - spin_unlock(&map_pgdir_lock); - if ( pl1e ) - free_xen_pagetable(pl1e); } } else