From patchwork Fri Dec 16 11:48:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1E9FBC4332F for ; Fri, 16 Dec 2022 12:17:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464566.723010 (Exim 4.92) (envelope-from ) id 1p69em-0007fz-Mf; Fri, 16 Dec 2022 12:17:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464566.723010; Fri, 16 Dec 2022 12:17:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69em-0007eX-Eq; Fri, 16 Dec 2022 12:17:40 +0000 Received: by outflank-mailman (input) for mailman id 464566; Fri, 16 Dec 2022 12:17:37 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-0007LM-RB for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 12:17:37 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-00034C-MR; Fri, 16 Dec 2022 12:17:37 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DL-0004sN-Rc; Fri, 16 Dec 2022 11:49:20 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=C7J11PdwQAWvXzpUfaG/t5NjyOIfJIe9L9M+iZjsZ88=; b=SjoRoA2TbEg5ZKtGJx/cWUhwP1 OxQuQW4yTon8/KGxTK021Utu+jR4S/+tJec27zz7oMMUVsmUOlBHP+EbpLSALUsVqW9xIMpdHODH/ jPZfchwzSmpEg2nmk67/EWmD1nbmKRmZVNzgRYuyxdPzNfjjmabszsvWgf6eK4h8qmuc=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Hongyan Xia , Andrew Cooper , George Dunlap , Jan Beulich , Stefano Stabellini , Wei Liu , Julien Grall Subject: [PATCH 17/22] x86/setup: vmap heap nodes when they are outside the direct map Date: Fri, 16 Dec 2022 11:48:48 +0000 Message-Id: <20221216114853.8227-18-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Hongyan Xia When we do not have a direct map, archs_mfn_in_direct_map() will always return false, thus init_node_heap() will allocate xenheap pages from an existing node for the metadata of a new node. This means that the metadata of a new node is in a different node, slowing down heap allocation. Since we now have early vmap, vmap the metadata locally in the new node. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall ---- Changes from Hongyan's version: * arch_mfn_in_direct_map() was renamed to arch_mfns_in_direct_map() * Use vmap_contig_pages() rather than __vmap(...). * Add missing include (xen/vmap.h) so it compiles on Arm Reviewed-by: Stefano Stabellini --- xen/common/page_alloc.c | 42 +++++++++++++++++++++++++++++++---------- 1 file changed, 32 insertions(+), 10 deletions(-) diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 0c4af5a71407..581c15d74dfb 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -136,6 +136,7 @@ #include #include #include +#include #include #include @@ -597,22 +598,43 @@ static unsigned long init_node_heap(int node, unsigned long mfn, needed = 0; } else if ( *use_tail && nr >= needed && - arch_mfns_in_directmap(mfn + nr - needed, needed) && (!xenheap_bits || !((mfn + nr - 1) >> (xenheap_bits - PAGE_SHIFT))) ) { - _heap[node] = mfn_to_virt(mfn + nr - needed); - avail[node] = mfn_to_virt(mfn + nr - 1) + - PAGE_SIZE - sizeof(**avail) * NR_ZONES; - } - else if ( nr >= needed && - arch_mfns_in_directmap(mfn, needed) && + if ( arch_mfns_in_directmap(mfn + nr - needed, needed) ) + { + _heap[node] = mfn_to_virt(mfn + nr - needed); + avail[node] = mfn_to_virt(mfn + nr - 1) + + PAGE_SIZE - sizeof(**avail) * NR_ZONES; + } + else + { + mfn_t needed_start = _mfn(mfn + nr - needed); + + _heap[node] = vmap_contig_pages(needed_start, needed); + BUG_ON(!_heap[node]); + avail[node] = (void *)(_heap[node]) + (needed << PAGE_SHIFT) - + sizeof(**avail) * NR_ZONES; + } + } else if ( nr >= needed && (!xenheap_bits || !((mfn + needed - 1) >> (xenheap_bits - PAGE_SHIFT))) ) { - _heap[node] = mfn_to_virt(mfn); - avail[node] = mfn_to_virt(mfn + needed - 1) + - PAGE_SIZE - sizeof(**avail) * NR_ZONES; + if ( arch_mfns_in_directmap(mfn, needed) ) + { + _heap[node] = mfn_to_virt(mfn); + avail[node] = mfn_to_virt(mfn + needed - 1) + + PAGE_SIZE - sizeof(**avail) * NR_ZONES; + } + else + { + mfn_t needed_start = _mfn(mfn); + + _heap[node] = vmap_contig_pages(needed_start, needed); + BUG_ON(!_heap[node]); + avail[node] = (void *)(_heap[node]) + (needed << PAGE_SHIFT) - + sizeof(**avail) * NR_ZONES; + } *use_tail = false; } else if ( get_order_from_bytes(sizeof(**_heap)) ==