From patchwork Fri Dec 16 11:48:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074951 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E700CC4332F for ; Fri, 16 Dec 2022 11:49:09 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464425.722822 (Exim 4.92) (envelope-from ) id 1p69D3-0003kq-EQ; Fri, 16 Dec 2022 11:49:01 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464425.722822; Fri, 16 Dec 2022 11:49:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D3-0003ke-B5; Fri, 16 Dec 2022 11:49:01 +0000 Received: by outflank-mailman (input) for mailman id 464425; Fri, 16 Dec 2022 11:49:00 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D1-0003V6-Vy for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 11:48:59 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D1-0002Ck-IE; Fri, 16 Dec 2022 11:48:59 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69D1-0004sN-Ac; Fri, 16 Dec 2022 11:48:59 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=X62OB5zEH/tDpewvEhcpco/a600UQZqosHxduopbkeM=; b=HPVAWQs2Y10cVfCLowvI46rn4J Rk7SZE68vWXFpoVTrvipRP1OoMIpQftR34TIzujyKous+0/kj9v44h+lVFnSmLeVgl+5bjWuUBA/6 SO8fPL6Zzkj2xrK43va6J4T8CBABVMYS5MLTcfS2wJyZ2YYu3acL9yx/f982YCkmSJ7k=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Julien Grall , Andrew Cooper , George Dunlap , Jan Beulich , Stefano Stabellini , Wei Liu Subject: [PATCH 01/22] xen/common: page_alloc: Re-order includes Date: Fri, 16 Dec 2022 11:48:32 +0000 Message-Id: <20221216114853.8227-2-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Julien Grall Order the includes with the xen headers first, then asm headers and last public headers. Within each category, they are sorted alphabetically. Note that the includes in protected by CONFIG_X86 hasn't been sorted to avoid adding multiple #ifdef. Signed-off-by: Julien Grall ---- I am open to add sort the includes protected by CONFIG_X86 and add multiple #ifdef if this is preferred. Acked-by: Jan Beulich Acked-by: Stefano Stabellini --- xen/common/page_alloc.c | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 0c93a1078702..0a950288e241 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -120,27 +120,30 @@ * regions within it. */ +#include +#include #include -#include +#include +#include #include -#include -#include #include +#include +#include #include -#include -#include -#include -#include #include #include -#include -#include -#include +#include +#include +#include +#include + +#include +#include +#include + #include #include -#include -#include -#include + #ifdef CONFIG_X86 #include #include From patchwork Fri Dec 16 11:48:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074953 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 40038C4167B for ; Fri, 16 Dec 2022 11:49:12 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464426.722833 (Exim 4.92) (envelope-from ) id 1p69D4-00041M-Nh; Fri, 16 Dec 2022 11:49:02 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464426.722833; Fri, 16 Dec 2022 11:49:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D4-00041D-K5; Fri, 16 Dec 2022 11:49:02 +0000 Received: by outflank-mailman (input) for mailman id 464426; Fri, 16 Dec 2022 11:49:01 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D3-0003kh-B9 for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 11:49:01 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D3-0002Cw-AN for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 11:49:01 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69D3-0004sN-2Y; Fri, 16 Dec 2022 11:49:01 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=7G8OCEg9k6ZwbQFlkMaRu/1/MXF29psY53uz8Xg2bDg=; b=D3FSa7zJj1nOHbPE0uzqpEjkc6 XLjSia5eZs8j94llmVkRW73ZJx+jEIZKSyCazN2zcYyAJ2AgNp4DEM+wQ1d2Ko8Qy190hjbUwEzXW 98NvrzF9sBvk+pL0hwIH4vbCJH/OZ8KtVprS7qh3pYfUHoBMxfDV26AzelEHcIASXdVQ=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Wei Liu , Stefano Stabellini , Bertrand Marquis , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Wei Liu , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , David Woodhouse , Hongyan Xia , Julien Grall Subject: [PATCH 02/22] x86/setup: move vm_init() before acpi calls Date: Fri, 16 Dec 2022 11:48:33 +0000 Message-Id: <20221216114853.8227-3-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Wei Liu After the direct map removal, pages from the boot allocator are not mapped at all in the direct map. Although we have map_domain_page, they are ephemeral and are less helpful for mappings that are more than a page, so we want a mechanism to globally map a range of pages, which is what vmap is for. Therefore, we bring vm_init into early boot stage. To allow vmap to be initialised and used in early boot, we need to modify vmap to receive pages from the boot allocator during early boot stage. Signed-off-by: Wei Liu Signed-off-by: David Woodhouse Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall Reviewed-by: Stefano Stabellini --- xen/arch/arm/setup.c | 4 ++-- xen/arch/x86/setup.c | 31 ++++++++++++++++++++----------- xen/common/vmap.c | 37 +++++++++++++++++++++++++++++-------- 3 files changed, 51 insertions(+), 21 deletions(-) diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index 1f26f67b90e3..2311726f5ddd 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -1028,6 +1028,8 @@ void __init start_xen(unsigned long boot_phys_offset, setup_mm(); + vm_init(); + /* Parse the ACPI tables for possible boot-time configuration */ acpi_boot_table_init(); @@ -1039,8 +1041,6 @@ void __init start_xen(unsigned long boot_phys_offset, */ system_state = SYS_STATE_boot; - vm_init(); - if ( acpi_disabled ) { printk("Booting using Device Tree\n"); diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 6bb5bc7c84be..1c2e09711eb0 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -870,6 +870,7 @@ void __init noreturn __start_xen(unsigned long mbi_p) unsigned long eb_start, eb_end; bool acpi_boot_table_init_done = false, relocated = false; int ret; + bool vm_init_done = false; struct ns16550_defaults ns16550 = { .data_bits = 8, .parity = 'n', @@ -1442,12 +1443,23 @@ void __init noreturn __start_xen(unsigned long mbi_p) continue; if ( !acpi_boot_table_init_done && - s >= (1ULL << 32) && - !acpi_boot_table_init() ) + s >= (1ULL << 32) ) { - acpi_boot_table_init_done = true; - srat_parse_regions(s); - setup_max_pdx(raw_max_page); + /* + * We only initialise vmap and acpi after going through the bottom + * 4GiB, so that we have enough pages in the boot allocator. + */ + if ( !vm_init_done ) + { + vm_init(); + vm_init_done = true; + } + if ( !acpi_boot_table_init() ) + { + acpi_boot_table_init_done = true; + srat_parse_regions(s); + setup_max_pdx(raw_max_page); + } } if ( pfn_to_pdx((e - 1) >> PAGE_SHIFT) >= max_pdx ) @@ -1624,6 +1636,9 @@ void __init noreturn __start_xen(unsigned long mbi_p) init_frametable(); + if ( !vm_init_done ) + vm_init(); + if ( !acpi_boot_table_init_done ) acpi_boot_table_init(); @@ -1661,12 +1676,6 @@ void __init noreturn __start_xen(unsigned long mbi_p) end_boot_allocator(); system_state = SYS_STATE_boot; - /* - * No calls involving ACPI code should go between the setting of - * SYS_STATE_boot and vm_init() (or else acpi_os_{,un}map_memory() - * will break). - */ - vm_init(); bsp_stack = cpu_alloc_stack(0); if ( !bsp_stack ) diff --git a/xen/common/vmap.c b/xen/common/vmap.c index 4fd6b3067ec1..1340c7c6faf6 100644 --- a/xen/common/vmap.c +++ b/xen/common/vmap.c @@ -34,9 +34,20 @@ void __init vm_init_type(enum vmap_region type, void *start, void *end) for ( i = 0, va = (unsigned long)vm_bitmap(type); i < nr; ++i, va += PAGE_SIZE ) { - struct page_info *pg = alloc_domheap_page(NULL, 0); + mfn_t mfn; + int rc; - map_pages_to_xen(va, page_to_mfn(pg), 1, PAGE_HYPERVISOR); + if ( system_state == SYS_STATE_early_boot ) + mfn = alloc_boot_pages(1, 1); + else + { + struct page_info *pg = alloc_domheap_page(NULL, 0); + + BUG_ON(!pg); + mfn = page_to_mfn(pg); + } + rc = map_pages_to_xen(va, mfn, 1, PAGE_HYPERVISOR); + BUG_ON(rc); clear_page((void *)va); } bitmap_fill(vm_bitmap(type), vm_low[type]); @@ -62,7 +73,7 @@ static void *vm_alloc(unsigned int nr, unsigned int align, spin_lock(&vm_lock); for ( ; ; ) { - struct page_info *pg; + mfn_t mfn; ASSERT(vm_low[t] == vm_top[t] || !test_bit(vm_low[t], vm_bitmap(t))); for ( start = vm_low[t]; start < vm_top[t]; ) @@ -97,9 +108,16 @@ static void *vm_alloc(unsigned int nr, unsigned int align, if ( vm_top[t] >= vm_end[t] ) return NULL; - pg = alloc_domheap_page(NULL, 0); - if ( !pg ) - return NULL; + if ( system_state == SYS_STATE_early_boot ) + mfn = alloc_boot_pages(1, 1); + else + { + struct page_info *pg = alloc_domheap_page(NULL, 0); + + if ( !pg ) + return NULL; + mfn = page_to_mfn(pg); + } spin_lock(&vm_lock); @@ -107,7 +125,7 @@ static void *vm_alloc(unsigned int nr, unsigned int align, { unsigned long va = (unsigned long)vm_bitmap(t) + vm_top[t] / 8; - if ( !map_pages_to_xen(va, page_to_mfn(pg), 1, PAGE_HYPERVISOR) ) + if ( !map_pages_to_xen(va, mfn, 1, PAGE_HYPERVISOR) ) { clear_page((void *)va); vm_top[t] += PAGE_SIZE * 8; @@ -117,7 +135,10 @@ static void *vm_alloc(unsigned int nr, unsigned int align, } } - free_domheap_page(pg); + if ( system_state == SYS_STATE_early_boot ) + init_boot_pages(mfn_to_maddr(mfn), mfn_to_maddr(mfn) + PAGE_SIZE); + else + free_domheap_page(mfn_to_page(mfn)); if ( start >= vm_top[t] ) { From patchwork Fri Dec 16 11:48:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074954 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E861DC4332F for ; Fri, 16 Dec 2022 11:49:13 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464427.722844 (Exim 4.92) (envelope-from ) id 1p69D7-0004Jp-0a; Fri, 16 Dec 2022 11:49:05 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464427.722844; Fri, 16 Dec 2022 11:49:04 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D6-0004JZ-Si; Fri, 16 Dec 2022 11:49:04 +0000 Received: by outflank-mailman (input) for mailman id 464427; Fri, 16 Dec 2022 11:49:02 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D4-000442-S3 for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 11:49:02 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D4-0002DR-KG; Fri, 16 Dec 2022 11:49:02 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69D4-0004sN-B6; Fri, 16 Dec 2022 11:49:02 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=E2zSWaWXXVlSavKpNugaiSnQ/IiR0BYoL62Rayvr/MY=; b=hKNcTKSNOrOaCq5PJc51xa0GS0 7Z2F/tvGmQk737a6n+KjVlBXyrhVnOGfdgSVgQcX38/YimhTrbQlJEYqXoJ5mj8PtGFkckLYkOIj5 PSYkSyi0xhPCap+6jsN0Iu5iVOgSimT7ZiDWjmuJS4k1QGzJtrbo4+AKBSySghMl8mU0=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Hongyan Xia , Andrew Cooper , George Dunlap , Jan Beulich , Stefano Stabellini , Wei Liu , Julien Grall Subject: [PATCH 03/22] acpi: vmap pages in acpi_os_alloc_memory Date: Fri, 16 Dec 2022 11:48:34 +0000 Message-Id: <20221216114853.8227-4-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Hongyan Xia Also, introduce a wrapper around vmap that maps a contiguous range for boot allocations. Unfortunately, the new helper cannot be a static inline because the dependences are a mess. We would need to re-include asm/page.h (was removed in aa4b9d1ee653 "include: don't use asm/page.h from common headers") and it doesn't look to be enough anymore because bits from asm/cpufeature.h is used in the definition of PAGE_NX. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall ---- Changes since Hongyan's version: * Rename vmap_boot_pages() to vmap_contig_pages() * Move the new helper in vmap.c to avoid compilation issue * Don't use __pa() to translate the virtual address Reviewed-by: Stefano Stabellini --- xen/common/vmap.c | 5 +++++ xen/drivers/acpi/osl.c | 13 +++++++++++-- xen/include/xen/vmap.h | 2 ++ 3 files changed, 18 insertions(+), 2 deletions(-) diff --git a/xen/common/vmap.c b/xen/common/vmap.c index 1340c7c6faf6..78f051a67682 100644 --- a/xen/common/vmap.c +++ b/xen/common/vmap.c @@ -244,6 +244,11 @@ void *vmap(const mfn_t *mfn, unsigned int nr) return __vmap(mfn, 1, nr, 1, PAGE_HYPERVISOR, VMAP_DEFAULT); } +void *vmap_contig_pages(mfn_t mfn, unsigned int nr_pages) +{ + return __vmap(&mfn, nr_pages, 1, 1, PAGE_HYPERVISOR, VMAP_DEFAULT); +} + void vunmap(const void *va) { unsigned long addr = (unsigned long)va; diff --git a/xen/drivers/acpi/osl.c b/xen/drivers/acpi/osl.c index 389505f78666..44a9719b0dcf 100644 --- a/xen/drivers/acpi/osl.c +++ b/xen/drivers/acpi/osl.c @@ -221,7 +221,11 @@ void *__init acpi_os_alloc_memory(size_t sz) void *ptr; if (system_state == SYS_STATE_early_boot) - return mfn_to_virt(mfn_x(alloc_boot_pages(PFN_UP(sz), 1))); + { + mfn_t mfn = alloc_boot_pages(PFN_UP(sz), 1); + + return vmap_contig_pages(mfn, PFN_UP(sz)); + } ptr = xmalloc_bytes(sz); ASSERT(!ptr || is_xmalloc_memory(ptr)); @@ -246,5 +250,10 @@ void __init acpi_os_free_memory(void *ptr) if (is_xmalloc_memory(ptr)) xfree(ptr); else if (ptr && system_state == SYS_STATE_early_boot) - init_boot_pages(__pa(ptr), __pa(ptr) + PAGE_SIZE); + { + paddr_t addr = mfn_to_maddr(vmap_to_mfn(ptr)); + + vunmap(ptr); + init_boot_pages(addr, addr + PAGE_SIZE); + } } diff --git a/xen/include/xen/vmap.h b/xen/include/xen/vmap.h index b0f7632e8985..3c06c7c3ba30 100644 --- a/xen/include/xen/vmap.h +++ b/xen/include/xen/vmap.h @@ -23,6 +23,8 @@ void *vmalloc_xen(size_t size); void *vzalloc(size_t size); void vfree(void *va); +void *vmap_contig_pages(mfn_t mfn, unsigned int nr_pages); + void __iomem *ioremap(paddr_t, size_t); static inline void iounmap(void __iomem *va) From patchwork Fri Dec 16 11:48:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074956 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E2243C3DA78 for ; Fri, 16 Dec 2022 11:49:14 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464428.722849 (Exim 4.92) (envelope-from ) id 1p69D7-0004ND-DD; Fri, 16 Dec 2022 11:49:05 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464428.722849; Fri, 16 Dec 2022 11:49:05 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D7-0004Mg-6T; Fri, 16 Dec 2022 11:49:05 +0000 Received: by outflank-mailman (input) for mailman id 464428; Fri, 16 Dec 2022 11:49:04 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D6-0004IQ-1V for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 11:49:04 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D5-0002Dj-RC; Fri, 16 Dec 2022 11:49:03 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69D5-0004sN-Je; Fri, 16 Dec 2022 11:49:03 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=ZLdocJInFCY/LVnGw1j5HFKUJpFld4Nm6D1TB3zFoFs=; b=kpUoDQTpOqZ2oeUr+0YruDlA5R +9X+I0qjhfLukkQQwY8weaTAUl2Nl3fQGJ/YKVV/I3ZlZyl+1nxWkn3fLTr/G6T++f27r113LV0au nZHlDrqQQUXHfrnkzyPOGxy/WpLSpu5Qh+pYRcl7gixXkluuHZ4DDeZz4ybaJ9tFWa/M=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Hongyan Xia , Andrew Cooper , George Dunlap , Jan Beulich , Stefano Stabellini , Wei Liu , Julien Grall Subject: [PATCH 04/22] xen/numa: vmap the pages for memnodemap Date: Fri, 16 Dec 2022 11:48:35 +0000 Message-Id: <20221216114853.8227-5-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Hongyan Xia This avoids the assumption that there is a direct map and boot pages fall inside the direct map. Clean up the variables so that mfn actually stores a type-safe mfn. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall ---- Changes compare to Hongyan's version: * The function modified was moved to common code. So rebase it * vmap_boot_pages() was renamed to vmap_contig_pages() Reviewed-by: Jan Beulich --- xen/common/numa.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/xen/common/numa.c b/xen/common/numa.c index 4948b21fbe66..2040b3d974e5 100644 --- a/xen/common/numa.c +++ b/xen/common/numa.c @@ -424,13 +424,13 @@ static int __init populate_memnodemap(const struct node *nodes, static int __init allocate_cachealigned_memnodemap(void) { unsigned long size = PFN_UP(memnodemapsize * sizeof(*memnodemap)); - unsigned long mfn = mfn_x(alloc_boot_pages(size, 1)); + mfn_t mfn = alloc_boot_pages(size, 1); - memnodemap = mfn_to_virt(mfn); - mfn <<= PAGE_SHIFT; + memnodemap = vmap_contig_pages(mfn, size); + BUG_ON(!memnodemap); size <<= PAGE_SHIFT; printk(KERN_DEBUG "NUMA: Allocated memnodemap from %lx - %lx\n", - mfn, mfn + size); + mfn_to_maddr(mfn), mfn_to_maddr(mfn) + size); memnodemapsize = size / sizeof(*memnodemap); return 0; From patchwork Fri Dec 16 11:48:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074955 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CBD02C4167B for ; Fri, 16 Dec 2022 11:49:15 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464430.722865 (Exim 4.92) (envelope-from ) id 1p69D9-0004oz-8C; Fri, 16 Dec 2022 11:49:07 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464430.722865; Fri, 16 Dec 2022 11:49:07 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D9-0004nY-1i; Fri, 16 Dec 2022 11:49:07 +0000 Received: by outflank-mailman (input) for mailman id 464430; Fri, 16 Dec 2022 11:49:05 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D7-0004Mt-6w for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 11:49:05 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D7-0002E2-0I; Fri, 16 Dec 2022 11:49:05 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69D6-0004sN-P3; Fri, 16 Dec 2022 11:49:04 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=5oXMYhVrHa49EzlJyprSVvBAT8KPYfOFRwrMrA+Sn68=; b=cozlDAIvwTjD7N7KnGv9QUb1tK RwpSbCKYNRGaLSAwm1RH81K6v1OQRWDNz50vS5jiFMSZ7gaBC4IJSX72+xq8eQSN74nGX7o4FXLis vgi2fp7BbNeQO032Dw7wgX1vazlY4k215QZO1/kmKWTxmVfNhSW5Q2afmlqeB49atO/8=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Hongyan Xia , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , Julien Grall Subject: [PATCH 05/22] x86/srat: vmap the pages for acpi_slit Date: Fri, 16 Dec 2022 11:48:36 +0000 Message-Id: <20221216114853.8227-6-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Hongyan Xia This avoids the assumption that boot pages are in the direct map. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall ---- Changes since Hongyan's version: * vmap_boot_pages() was renamed to vmap_contig_pages() Reviewed-by: Jan Beulich --- xen/arch/x86/srat.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c index 56749ddca526..1fd178e89d28 100644 --- a/xen/arch/x86/srat.c +++ b/xen/arch/x86/srat.c @@ -139,7 +139,8 @@ void __init acpi_numa_slit_init(struct acpi_table_slit *slit) return; } mfn = alloc_boot_pages(PFN_UP(slit->header.length), 1); - acpi_slit = mfn_to_virt(mfn_x(mfn)); + acpi_slit = vmap_contig_pages(mfn, PFN_UP(slit->header.length)); + BUG_ON(!acpi_slit); memcpy(acpi_slit, slit, slit->header.length); } From patchwork Fri Dec 16 11:48:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074958 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EAC6FC3DA6E for ; Fri, 16 Dec 2022 11:49:16 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464432.722871 (Exim 4.92) (envelope-from ) id 1p69D9-0004uN-Li; Fri, 16 Dec 2022 11:49:07 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464432.722871; Fri, 16 Dec 2022 11:49:07 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D9-0004tC-E6; Fri, 16 Dec 2022 11:49:07 +0000 Received: by outflank-mailman (input) for mailman id 464432; Fri, 16 Dec 2022 11:49:06 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D8-0004b9-7Y for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 11:49:06 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D8-0002ET-5Z; Fri, 16 Dec 2022 11:49:06 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69D7-0004sN-US; Fri, 16 Dec 2022 11:49:06 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=FDnn4j9oos7o0Yst//etF30Yzyr3SCCSdb3Qh8o9mlo=; b=fHd51aNXlHLmKnuzS89FCVrVMv 6yxymS4djo7spaS62B+LYP5kO4zqErtz5FVtozvvW0CftDT7ElgEMFpbhc3CRx2lNd74u823ed3Y0 2ygOzpZZ9Hbci4Q7zxwpwOG4K/JG5ibUf88FCbaP7uRZ5jBeVRf2nw8X3shrWO8zo+Mk=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Hongyan Xia , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , Julien Grall Subject: [PATCH 06/22] x86: map/unmap pages in restore_all_guests Date: Fri, 16 Dec 2022 11:48:37 +0000 Message-Id: <20221216114853.8227-7-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Hongyan Xia Before, it assumed the pv cr3 could be accessed via a direct map. This is no longer true. Note that we do not map and unmap root_pgt for now since it is still a xenheap page. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall ---- Changes since Hongyan's version: * Remove the final dot in the commit title --- xen/arch/x86/x86_64/entry.S | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S index ae012851819a..b72abf923d9c 100644 --- a/xen/arch/x86/x86_64/entry.S +++ b/xen/arch/x86/x86_64/entry.S @@ -165,7 +165,24 @@ restore_all_guest: and %rsi, %rdi and %r9, %rsi add %rcx, %rdi - add %rcx, %rsi + + /* + * Without a direct map, we have to map first before copying. We only + * need to map the guest root table but not the per-CPU root_pgt, + * because the latter is still a xenheap page. + */ + pushq %r9 + pushq %rdx + pushq %rax + pushq %rdi + mov %rsi, %rdi + shr $PAGE_SHIFT, %rdi + callq map_domain_page + mov %rax, %rsi + popq %rdi + /* Stash the pointer for unmapping later. */ + pushq %rax + mov $ROOT_PAGETABLE_FIRST_XEN_SLOT, %ecx mov root_table_offset(SH_LINEAR_PT_VIRT_START)*8(%rsi), %r8 mov %r8, root_table_offset(SH_LINEAR_PT_VIRT_START)*8(%rdi) @@ -177,6 +194,14 @@ restore_all_guest: sub $(ROOT_PAGETABLE_FIRST_XEN_SLOT - \ ROOT_PAGETABLE_LAST_XEN_SLOT - 1) * 8, %rdi rep movsq + + /* Unmap the page. */ + popq %rdi + callq unmap_domain_page + popq %rax + popq %rdx + popq %r9 + .Lrag_copy_done: mov %r9, STACK_CPUINFO_FIELD(xen_cr3)(%rdx) movb $1, STACK_CPUINFO_FIELD(use_pv_cr3)(%rdx) From patchwork Fri Dec 16 11:48:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074957 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C71FEC4332F for ; Fri, 16 Dec 2022 11:49:16 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464434.722886 (Exim 4.92) (envelope-from ) id 1p69DB-0005Ng-7O; Fri, 16 Dec 2022 11:49:09 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464434.722886; Fri, 16 Dec 2022 11:49:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69DB-0005MW-1B; Fri, 16 Dec 2022 11:49:09 +0000 Received: by outflank-mailman (input) for mailman id 464434; Fri, 16 Dec 2022 11:49:07 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D9-0004u8-Fl for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 11:49:07 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69D9-0002Er-E8; Fri, 16 Dec 2022 11:49:07 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69D9-0004sN-6k; Fri, 16 Dec 2022 11:49:07 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=vmtNA9XdH0srZ9asaGnjFiuLx3yBtd0BVtqXIukQgjM=; b=SUa3z5qrtpvwTyqxU4268rECt/ 19/D/bc/kCdXMUWX4leTRpnP8ZIRVdrg+c0wyUEWzO4e01rpd3ocb58eTi0CIcxumiM/fF0YWiub2 bRuT7xnU51KnTe1WDHfThSfMNJSBM/D9tJmjWmL+WcPOSEKJq2t4N1gUDMIuuMFD2oA0=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Wei Liu , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , Wei Wang , Julien Grall Subject: [PATCH 07/22] x86/pv: domheap pages should be mapped while relocating initrd Date: Fri, 16 Dec 2022 11:48:38 +0000 Message-Id: <20221216114853.8227-8-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Wei Liu Xen shouldn't use domheap page as if they were xenheap pages. Map and unmap pages accordingly. Signed-off-by: Wei Liu Signed-off-by: Wei Wang Signed-off-by: Julien Grall ---- Changes since Hongyan's version: * Add missing newline after the variable declaration --- xen/arch/x86/pv/dom0_build.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c index a62f0fa2ef29..c837b2d96f89 100644 --- a/xen/arch/x86/pv/dom0_build.c +++ b/xen/arch/x86/pv/dom0_build.c @@ -611,18 +611,32 @@ int __init dom0_construct_pv(struct domain *d, if ( d->arch.physaddr_bitsize && ((mfn + count - 1) >> (d->arch.physaddr_bitsize - PAGE_SHIFT)) ) { + unsigned long nr_pages; + unsigned long len = initrd_len; + order = get_order_from_pages(count); page = alloc_domheap_pages(d, order, MEMF_no_scrub); if ( !page ) panic("Not enough RAM for domain 0 initrd\n"); + + nr_pages = 1UL << order; for ( count = -count; order--; ) if ( count & (1UL << order) ) { free_domheap_pages(page, order); page += 1UL << order; + nr_pages -= 1UL << order; } - memcpy(page_to_virt(page), mfn_to_virt(initrd->mod_start), - initrd_len); + + for ( i = 0; i < nr_pages; i++, len -= PAGE_SIZE ) + { + void *p = __map_domain_page(page + i); + + memcpy(p, mfn_to_virt(initrd_mfn + i), + min(len, (unsigned long)PAGE_SIZE)); + unmap_domain_page(p); + } + mpt_alloc = (paddr_t)initrd->mod_start << PAGE_SHIFT; init_domheap_pages(mpt_alloc, mpt_alloc + PAGE_ALIGN(initrd_len)); From patchwork Fri Dec 16 11:48:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074959 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C229EC4332F for ; Fri, 16 Dec 2022 11:49:18 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464436.722892 (Exim 4.92) (envelope-from ) id 1p69DB-0005UT-Qy; Fri, 16 Dec 2022 11:49:09 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464436.722892; Fri, 16 Dec 2022 11:49:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69DB-0005Rw-Hn; Fri, 16 Dec 2022 11:49:09 +0000 Received: by outflank-mailman (input) for mailman id 464436; Fri, 16 Dec 2022 11:49:08 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69DA-0005H5-K8 for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 11:49:08 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69DA-0002FH-JS for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 11:49:08 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DA-0004sN-CE; Fri, 16 Dec 2022 11:49:08 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=I796InQwp8GJ7wl8yn2SnJLasp1DLu8CI/8eIrqjbEo=; b=G3ocwq2f+UdE70Au4tGaHIKKiL UnMPqJfwBzJtirFdxcw08AvdAHC3MGXpc/BIoRFsd9HNxPyUXfOY+KR/lP0J0XTcXjmwACt5RlQ8B FyOqw5evHPepZDsub1Qd0y8p80o66DUg3nZY2gRwrtyO8L3lFgmUutzzNu334GKoHo1w=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Hongyan Xia , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , Julien Grall Subject: [PATCH 08/22] x86/pv: rewrite how building PV dom0 handles domheap mappings Date: Fri, 16 Dec 2022 11:48:39 +0000 Message-Id: <20221216114853.8227-9-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Hongyan Xia Building a PV dom0 is allocating from the domheap but uses it like the xenheap. This is clearly wrong. Fix. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall ---- Changes since Hongyan's version: * Rebase * Remove spurious newline --- xen/arch/x86/pv/dom0_build.c | 56 +++++++++++++++++++++++++++--------- 1 file changed, 42 insertions(+), 14 deletions(-) diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c index c837b2d96f89..cd60f259d1b7 100644 --- a/xen/arch/x86/pv/dom0_build.c +++ b/xen/arch/x86/pv/dom0_build.c @@ -383,6 +383,10 @@ int __init dom0_construct_pv(struct domain *d, l3_pgentry_t *l3tab = NULL, *l3start = NULL; l2_pgentry_t *l2tab = NULL, *l2start = NULL; l1_pgentry_t *l1tab = NULL, *l1start = NULL; + mfn_t l4start_mfn = INVALID_MFN; + mfn_t l3start_mfn = INVALID_MFN; + mfn_t l2start_mfn = INVALID_MFN; + mfn_t l1start_mfn = INVALID_MFN; /* * This fully describes the memory layout of the initial domain. All @@ -711,22 +715,32 @@ int __init dom0_construct_pv(struct domain *d, v->arch.pv.event_callback_cs = FLAT_COMPAT_KERNEL_CS; } +#define UNMAP_MAP_AND_ADVANCE(mfn_var, virt_var, maddr) \ +do { \ + UNMAP_DOMAIN_PAGE(virt_var); \ + mfn_var = maddr_to_mfn(maddr); \ + maddr += PAGE_SIZE; \ + virt_var = map_domain_page(mfn_var); \ +} while ( false ) + if ( !compat ) { maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l4_page_table; - l4start = l4tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; + UNMAP_MAP_AND_ADVANCE(l4start_mfn, l4start, mpt_alloc); + l4tab = l4start; clear_page(l4tab); - init_xen_l4_slots(l4tab, _mfn(virt_to_mfn(l4start)), - d, INVALID_MFN, true); - v->arch.guest_table = pagetable_from_paddr(__pa(l4start)); + init_xen_l4_slots(l4tab, l4start_mfn, d, INVALID_MFN, true); + v->arch.guest_table = pagetable_from_mfn(l4start_mfn); } else { /* Monitor table already created by switch_compat(). */ - l4start = l4tab = __va(pagetable_get_paddr(v->arch.guest_table)); + l4start_mfn = pagetable_get_mfn(v->arch.guest_table); + l4start = l4tab = map_domain_page(l4start_mfn); /* See public/xen.h on why the following is needed. */ maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l3_page_table; l3start = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; + UNMAP_MAP_AND_ADVANCE(l3start_mfn, l3start, mpt_alloc); } l4tab += l4_table_offset(v_start); @@ -736,14 +750,16 @@ int __init dom0_construct_pv(struct domain *d, if ( !((unsigned long)l1tab & (PAGE_SIZE-1)) ) { maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l1_page_table; - l1start = l1tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; + UNMAP_MAP_AND_ADVANCE(l1start_mfn, l1start, mpt_alloc); + l1tab = l1start; clear_page(l1tab); if ( count == 0 ) l1tab += l1_table_offset(v_start); if ( !((unsigned long)l2tab & (PAGE_SIZE-1)) ) { maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l2_page_table; - l2start = l2tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; + UNMAP_MAP_AND_ADVANCE(l2start_mfn, l2start, mpt_alloc); + l2tab = l2start; clear_page(l2tab); if ( count == 0 ) l2tab += l2_table_offset(v_start); @@ -753,19 +769,19 @@ int __init dom0_construct_pv(struct domain *d, { maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l3_page_table; - l3start = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; + UNMAP_MAP_AND_ADVANCE(l3start_mfn, l3start, mpt_alloc); } l3tab = l3start; clear_page(l3tab); if ( count == 0 ) l3tab += l3_table_offset(v_start); - *l4tab = l4e_from_paddr(__pa(l3start), L4_PROT); + *l4tab = l4e_from_mfn(l3start_mfn, L4_PROT); l4tab++; } - *l3tab = l3e_from_paddr(__pa(l2start), L3_PROT); + *l3tab = l3e_from_mfn(l2start_mfn, L3_PROT); l3tab++; } - *l2tab = l2e_from_paddr(__pa(l1start), L2_PROT); + *l2tab = l2e_from_mfn(l1start_mfn, L2_PROT); l2tab++; } if ( count < initrd_pfn || count >= initrd_pfn + PFN_UP(initrd_len) ) @@ -792,9 +808,9 @@ int __init dom0_construct_pv(struct domain *d, if ( !l3e_get_intpte(*l3tab) ) { maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l2_page_table; - l2tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE; - clear_page(l2tab); - *l3tab = l3e_from_paddr(__pa(l2tab), L3_PROT); + UNMAP_MAP_AND_ADVANCE(l2start_mfn, l2start, mpt_alloc); + clear_page(l2start); + *l3tab = l3e_from_mfn(l2start_mfn, L3_PROT); } if ( i == 3 ) l3e_get_page(*l3tab)->u.inuse.type_info |= PGT_pae_xen_l2; @@ -805,9 +821,17 @@ int __init dom0_construct_pv(struct domain *d, unmap_domain_page(l2t); } +#undef UNMAP_MAP_AND_ADVANCE + + UNMAP_DOMAIN_PAGE(l1start); + UNMAP_DOMAIN_PAGE(l2start); + UNMAP_DOMAIN_PAGE(l3start); + /* Pages that are part of page tables must be read only. */ mark_pv_pt_pages_rdonly(d, l4start, vpt_start, nr_pt_pages, &flush_flags); + UNMAP_DOMAIN_PAGE(l4start); + /* Mask all upcalls... */ for ( i = 0; i < XEN_LEGACY_MAX_VCPUS; i++ ) shared_info(d, vcpu_info[i].evtchn_upcall_mask) = 1; @@ -977,8 +1001,12 @@ int __init dom0_construct_pv(struct domain *d, * !CONFIG_VIDEO case so the logic here can be simplified. */ if ( pv_shim ) + { + l4start = map_domain_page(l4start_mfn); pv_shim_setup_dom(d, l4start, v_start, vxenstore_start, vconsole_start, vphysmap_start, si); + UNMAP_DOMAIN_PAGE(l4start); + } #ifdef CONFIG_COMPAT if ( compat ) From patchwork Fri Dec 16 11:48:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074960 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EC6A3C4332F for ; Fri, 16 Dec 2022 11:49:21 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464437.722910 (Exim 4.92) (envelope-from ) id 1p69DE-00068y-Ci; Fri, 16 Dec 2022 11:49:12 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464437.722910; Fri, 16 Dec 2022 11:49:12 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69DE-00067z-5E; Fri, 16 Dec 2022 11:49:12 +0000 Received: by outflank-mailman (input) for mailman id 464437; Fri, 16 Dec 2022 11:49:10 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69DC-0005eC-8L for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 11:49:10 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69DB-0002Fb-VD; Fri, 16 Dec 2022 11:49:09 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DB-0004sN-Nv; Fri, 16 Dec 2022 11:49:09 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=jcJSzXwZRVoOaBWM+hzG4J8upiEWzEuSM93VyqeBx/0=; b=o85LFLbRuZU3OMGlbhFEtltkQN xQpfoMDz0l9IiFoY5MDxE/T+Ir+eoSLmshFsqAmRWlm5CIX5m2J6YWsriPoW4L0vTOnhKfv6tkWsR bMyF2to9Zl3/o2IaFVNp3oC7mjAY2K6AbWbpySC0oucxSjfXXoSdrMzUa2sX8xvzFa3A=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Wei Liu , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , Wei Wang , Hongyan Xia , Julien Grall Subject: [PATCH 09/22] x86: lift mapcache variable to the arch level Date: Fri, 16 Dec 2022 11:48:40 +0000 Message-Id: <20221216114853.8227-10-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Wei Liu It is going to be needed by HVM and idle domain as well, because without the direct map, both need a mapcache to map pages. This only lifts the mapcache variable up. Whether we populate the mapcache for a domain is unchanged in this patch. Signed-off-by: Wei Liu Signed-off-by: Wei Wang Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall Reviewed-by: Jan Beulich --- xen/arch/x86/domain.c | 4 ++-- xen/arch/x86/domain_page.c | 22 ++++++++++------------ xen/arch/x86/include/asm/domain.h | 12 ++++++------ 3 files changed, 18 insertions(+), 20 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index d7a8237f01ab..069b7d2af330 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -829,6 +829,8 @@ int arch_domain_create(struct domain *d, psr_domain_init(d); + mapcache_domain_init(d); + if ( is_hvm_domain(d) ) { if ( (rc = hvm_domain_initialise(d, config)) != 0 ) @@ -836,8 +838,6 @@ int arch_domain_create(struct domain *d, } else if ( is_pv_domain(d) ) { - mapcache_domain_init(d); - if ( (rc = pv_domain_initialise(d)) != 0 ) goto fail; } diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index eac5e3304fb8..55e337aaf703 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -82,11 +82,11 @@ void *map_domain_page(mfn_t mfn) #endif v = mapcache_current_vcpu(); - if ( !v || !is_pv_vcpu(v) ) + if ( !v ) return mfn_to_virt(mfn_x(mfn)); - dcache = &v->domain->arch.pv.mapcache; - vcache = &v->arch.pv.mapcache; + dcache = &v->domain->arch.mapcache; + vcache = &v->arch.mapcache; if ( !dcache->inuse ) return mfn_to_virt(mfn_x(mfn)); @@ -187,14 +187,14 @@ void unmap_domain_page(const void *ptr) ASSERT(va >= MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END); v = mapcache_current_vcpu(); - ASSERT(v && is_pv_vcpu(v)); + ASSERT(v); - dcache = &v->domain->arch.pv.mapcache; + dcache = &v->domain->arch.mapcache; ASSERT(dcache->inuse); idx = PFN_DOWN(va - MAPCACHE_VIRT_START); mfn = l1e_get_pfn(MAPCACHE_L1ENT(idx)); - hashent = &v->arch.pv.mapcache.hash[MAPHASH_HASHFN(mfn)]; + hashent = &v->arch.mapcache.hash[MAPHASH_HASHFN(mfn)]; local_irq_save(flags); @@ -233,11 +233,9 @@ void unmap_domain_page(const void *ptr) int mapcache_domain_init(struct domain *d) { - struct mapcache_domain *dcache = &d->arch.pv.mapcache; + struct mapcache_domain *dcache = &d->arch.mapcache; unsigned int bitmap_pages; - ASSERT(is_pv_domain(d)); - #ifdef NDEBUG if ( !mem_hotplug && max_page <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) ) return 0; @@ -261,12 +259,12 @@ int mapcache_domain_init(struct domain *d) int mapcache_vcpu_init(struct vcpu *v) { struct domain *d = v->domain; - struct mapcache_domain *dcache = &d->arch.pv.mapcache; + struct mapcache_domain *dcache = &d->arch.mapcache; unsigned long i; unsigned int ents = d->max_vcpus * MAPCACHE_VCPU_ENTRIES; unsigned int nr = PFN_UP(BITS_TO_LONGS(ents) * sizeof(long)); - if ( !is_pv_vcpu(v) || !dcache->inuse ) + if ( !dcache->inuse ) return 0; if ( ents > dcache->entries ) @@ -293,7 +291,7 @@ int mapcache_vcpu_init(struct vcpu *v) BUILD_BUG_ON(MAPHASHENT_NOTINUSE < MAPCACHE_ENTRIES); for ( i = 0; i < MAPHASH_ENTRIES; i++ ) { - struct vcpu_maphash_entry *hashent = &v->arch.pv.mapcache.hash[i]; + struct vcpu_maphash_entry *hashent = &v->arch.mapcache.hash[i]; hashent->mfn = ~0UL; /* never valid to map */ hashent->idx = MAPHASHENT_NOTINUSE; diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/domain.h index 43ace233d75e..eb548eb10efe 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -285,9 +285,6 @@ struct pv_domain /* Mitigate L1TF with shadow/crashing? */ bool check_l1tf; - /* map_domain_page() mapping cache. */ - struct mapcache_domain mapcache; - struct cpuidmasks *cpuidmasks; }; @@ -326,6 +323,9 @@ struct arch_domain uint8_t spec_ctrl_flags; /* See SCF_DOM_MASK */ + /* map_domain_page() mapping cache. */ + struct mapcache_domain mapcache; + union { struct pv_domain pv; struct hvm_domain hvm; @@ -508,9 +508,6 @@ struct arch_domain struct pv_vcpu { - /* map_domain_page() mapping cache. */ - struct mapcache_vcpu mapcache; - unsigned int vgc_flags; struct trap_info *trap_ctxt; @@ -610,6 +607,9 @@ struct arch_vcpu #define async_exception_state(t) async_exception_state[(t)-1] uint8_t async_exception_mask; + /* map_domain_page() mapping cache. */ + struct mapcache_vcpu mapcache; + /* Virtual Machine Extensions */ union { struct pv_vcpu pv; From patchwork Fri Dec 16 11:48:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074972 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 48ADCC3DA78 for ; Fri, 16 Dec 2022 12:17:54 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464564.722995 (Exim 4.92) (envelope-from ) id 1p69el-0007Pr-WA; Fri, 16 Dec 2022 12:17:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464564.722995; Fri, 16 Dec 2022 12:17:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69el-0007Oq-QS; Fri, 16 Dec 2022 12:17:39 +0000 Received: by outflank-mailman (input) for mailman id 464564; Fri, 16 Dec 2022 12:17:37 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-0007LA-Kw for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 12:17:37 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-00033v-IY; Fri, 16 Dec 2022 12:17:37 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DD-0004sN-0C; Fri, 16 Dec 2022 11:49:11 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=rWiHN0PK4M1dxBlf4cLIlk8Mw8XPqON+Q0yvmpeDRiA=; b=5y2veP8n0mNPBioUEfM2I9JzxR a9/cOqrzsY2PqAPWw+z8PhCJOQK9CNylt1oihhCZTQOgbv0IGwvWeFRP/KV9pdvqAKhQQ39KA+vja UYWWGo5ovvJC5DtXjKEV0rRiB4C7edFjMVDcnetHgaxt/j382uz9MBH9PCKrdIRCP4C8=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Hongyan Xia , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , Wei Wang , Julien Grall Subject: [PATCH 10/22] x86/mapcache: initialise the mapcache for the idle domain Date: Fri, 16 Dec 2022 11:48:41 +0000 Message-Id: <20221216114853.8227-11-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Hongyan Xia In order to use the mapcache in the idle domain, we also have to populate its page tables in the PERDOMAIN region, and we need to move mapcache_domain_init() earlier in arch_domain_create(). Note, commit 'x86: lift mapcache variable to the arch level' has initialised the mapcache for HVM domains. With this patch, PV, HVM, idle domains now all initialise the mapcache. Signed-off-by: Wei Wang Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall --- xen/arch/x86/domain.c | 4 ++-- xen/arch/x86/mm.c | 3 +++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 069b7d2af330..ec150f4fd144 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -732,6 +732,8 @@ int arch_domain_create(struct domain *d, spin_lock_init(&d->arch.e820_lock); + mapcache_domain_init(d); + /* Minimal initialisation for the idle domain. */ if ( unlikely(is_idle_domain(d)) ) { @@ -829,8 +831,6 @@ int arch_domain_create(struct domain *d, psr_domain_init(d); - mapcache_domain_init(d); - if ( is_hvm_domain(d) ) { if ( (rc = hvm_domain_initialise(d, config)) != 0 ) diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 8b9740f57519..041bd4cfde17 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -5963,6 +5963,9 @@ int create_perdomain_mapping(struct domain *d, unsigned long va, l3tab = __map_domain_page(pg); clear_page(l3tab); d->arch.perdomain_l3_pg = pg; + if ( is_idle_domain(d) ) + idle_pg_table[l4_table_offset(PERDOMAIN_VIRT_START)] = + l4e_from_page(pg, __PAGE_HYPERVISOR_RW); if ( !nr ) { unmap_domain_page(l3tab); From patchwork Fri Dec 16 11:48:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074979 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E1281C4332F for ; Fri, 16 Dec 2022 12:17:56 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464572.723062 (Exim 4.92) (envelope-from ) id 1p69eq-0000MG-5x; Fri, 16 Dec 2022 12:17:44 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464572.723062; Fri, 16 Dec 2022 12:17:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ep-0000Gd-JP; Fri, 16 Dec 2022 12:17:43 +0000 Received: by outflank-mailman (input) for mailman id 464572; Fri, 16 Dec 2022 12:17:38 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ek-0007Lt-2H for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 12:17:38 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-00034G-OC; Fri, 16 Dec 2022 12:17:37 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DE-0004sN-I6; Fri, 16 Dec 2022 11:49:12 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=9j1baKWOTDAmSEXXRPz5vlDNPs8x5yzHk136j/q4JZg=; b=x83k0fXRB6Jd3aoaqIjCeiTXkZ 7qIX6P/MiqvL54Or2V6+svj4lDwUZZCmBuqxYrvv3ob8rabbm6oXC3e1YeRuKe+CMfYZghExbqb80 kDwukK5Zdqncp5eH8pALQ1iPRGFB4ocn6dI8WO4JRZ75DOghEq/3Qh1IyTnBGkN+9KEc=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Hongyan Xia , Andrew Cooper , George Dunlap , Jan Beulich , Stefano Stabellini , Wei Liu , Bertrand Marquis , Volodymyr Babchuk , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= , Julien Grall Subject: [PATCH 11/22] x86: add a boot option to enable and disable the direct map Date: Fri, 16 Dec 2022 11:48:42 +0000 Message-Id: <20221216114853.8227-12-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Hongyan Xia Also add a helper function to retrieve it. Change arch_mfns_in_direct_map to check this option before returning. This is added as a boot command line option, not a Kconfig to allow the user to experiment the feature without rebuild the hypervisor. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall ---- TODO: * Do we also want to provide a Kconfig option? Changes since Hongyan's version: * Reword the commit message * opt_directmap is only modified during boot so mark it as __ro_after_init --- docs/misc/xen-command-line.pandoc | 12 ++++++++++++ xen/arch/arm/include/asm/mm.h | 5 +++++ xen/arch/x86/include/asm/mm.h | 17 ++++++++++++++++- xen/arch/x86/mm.c | 3 +++ xen/arch/x86/setup.c | 2 ++ 5 files changed, 38 insertions(+), 1 deletion(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index b7ee97be762e..a63e4612acac 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -760,6 +760,18 @@ Specify the size of the console debug trace buffer. By specifying `cpu:` additionally a trace buffer of the specified size is allocated per cpu. The debug trace feature is only enabled in debugging builds of Xen. +### directmap (x86) +> `= ` + +> Default: `true` + +Enable or disable the direct map region in Xen. + +By default, Xen creates the direct map region which maps physical memory +in that region. Setting this to no will remove the direct map, blocking +exploits that leak secrets via speculative memory access in the direct +map. + ### dma_bits > `= ` diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index 68adcac9fa8d..2366928d71aa 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -406,6 +406,11 @@ static inline void page_set_xenheap_gfn(struct page_info *p, gfn_t gfn) } while ( (y = cmpxchg(&p->u.inuse.type_info, x, nx)) != x ); } +static inline bool arch_has_directmap(void) +{ + return true; +} + #endif /* __ARCH_ARM_MM__ */ /* * Local variables: diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index db29e3e2059f..cf8b20817c6c 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -464,6 +464,8 @@ static inline int get_page_and_type(struct page_info *page, ASSERT(((_p)->count_info & PGC_count_mask) != 0); \ ASSERT(page_get_owner(_p) == (_d)) +extern bool opt_directmap; + /****************************************************************************** * With shadow pagetables, the different kinds of address start * to get get confusing. @@ -620,13 +622,26 @@ extern const char zero_page[]; /* Build a 32bit PSE page table using 4MB pages. */ void write_32bit_pse_identmap(uint32_t *l2); +static inline bool arch_has_directmap(void) +{ + return opt_directmap; +} + /* * x86 maps part of physical memory via the directmap region. * Return whether the range of MFN falls in the directmap region. + * + * When boot command line sets directmap=no, we will not have a direct map at + * all so this will always return false. */ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr) { - unsigned long eva = min(DIRECTMAP_VIRT_END, HYPERVISOR_VIRT_END); + unsigned long eva; + + if ( !arch_has_directmap() ) + return false; + + eva = min(DIRECTMAP_VIRT_END, HYPERVISOR_VIRT_END); return (mfn + nr) <= (virt_to_mfn(eva - 1) + 1); } diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c index 041bd4cfde17..e76e135b96fc 100644 --- a/xen/arch/x86/mm.c +++ b/xen/arch/x86/mm.c @@ -157,6 +157,9 @@ l1_pgentry_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) l1_pgentry_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) l1_fixmap_x[L1_PAGETABLE_ENTRIES]; +bool __ro_after_init opt_directmap = true; +boolean_param("directmap", opt_directmap); + /* Frame table size in pages. */ unsigned long max_page; unsigned long total_pages; diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 1c2e09711eb0..2cb051c6e4e7 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1423,6 +1423,8 @@ void __init noreturn __start_xen(unsigned long mbi_p) if ( highmem_start ) xenheap_max_mfn(PFN_DOWN(highmem_start - 1)); + printk("Booting with directmap %s\n", arch_has_directmap() ? "on" : "off"); + /* * Walk every RAM region and map it in its entirety (on x86/64, at least) * and notify it to the boot allocator. From patchwork Fri Dec 16 11:48:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074980 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B1568C001B2 for ; Fri, 16 Dec 2022 12:17:57 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464575.723091 (Exim 4.92) (envelope-from ) id 1p69et-00019I-Pb; Fri, 16 Dec 2022 12:17:47 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464575.723091; Fri, 16 Dec 2022 12:17:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69es-00013S-Mw; Fri, 16 Dec 2022 12:17:46 +0000 Received: by outflank-mailman (input) for mailman id 464575; Fri, 16 Dec 2022 12:17:38 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ek-0007ME-8w for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 12:17:38 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ek-000352-1s; Fri, 16 Dec 2022 12:17:38 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DF-0004sN-Tm; Fri, 16 Dec 2022 11:49:14 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=0Z58+ljccBpyu6+vCHwEkzyLj9W+Yww30i/0Y5Sz5B8=; b=Zd4d/HSmX9uz8oxK0Qhb4IZghT JBZ/9u+4SV77VeU8gypa9BTxRue7jjePjDTAdHMsiYEZTVC7eBcRuAztSNx8zYZUfdiG/Cvc1YGIt NzlET4hvhXzN/H35oWT2f3uCWF5LomvsdWaDIttUt0F683e3F3OlljrWKZOAhE1099QA=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Julien Grall , Stefano Stabellini , Bertrand Marquis , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Wei Liu Subject: [PATCH 12/22] xen/arm: fixmap: Rename the fixmap slots to follow the x86 convention Date: Fri, 16 Dec 2022 11:48:43 +0000 Message-Id: <20221216114853.8227-13-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Julien Grall At the moment the fixmap slots are prefixed differently between arm and x86. Some of them (e.g. the PMAP slots) are used in common code. So it would be better if they are named the same way to avoid having to create aliases. I have decided to use the x86 naming because they are less change. So all the Arm fixmap slots will now be prefixed with FIX rather than FIXMAP. Signed-off-by: Julien Grall ---- Note that potentially more renaming that could be done to share more code in future. I have decided to not do that to avoid going down a rabbit hole. Reviewed-by: Jan Beulich Reviewed-by: Henry Wang Reviewed-by: Stefano Stabellini --- xen/arch/arm/acpi/lib.c | 18 +++++++++--------- xen/arch/arm/include/asm/early_printk.h | 2 +- xen/arch/arm/include/asm/fixmap.h | 16 ++++++++-------- xen/arch/arm/kernel.c | 6 +++--- xen/common/pmap.c | 8 ++++---- 5 files changed, 25 insertions(+), 25 deletions(-) diff --git a/xen/arch/arm/acpi/lib.c b/xen/arch/arm/acpi/lib.c index 41d521f720ac..736cf09ecaa8 100644 --- a/xen/arch/arm/acpi/lib.c +++ b/xen/arch/arm/acpi/lib.c @@ -40,10 +40,10 @@ char *__acpi_map_table(paddr_t phys, unsigned long size) return NULL; offset = phys & (PAGE_SIZE - 1); - base = FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) + offset; + base = FIXMAP_ADDR(FIX_ACPI_BEGIN) + offset; /* Check the fixmap is big enough to map the region */ - if ( (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE - base) < size ) + if ( (FIXMAP_ADDR(FIX_ACPI_END) + PAGE_SIZE - base) < size ) return NULL; /* With the fixmap, we can only map one region at the time */ @@ -54,7 +54,7 @@ char *__acpi_map_table(paddr_t phys, unsigned long size) size += offset; mfn = maddr_to_mfn(phys); - idx = FIXMAP_ACPI_BEGIN; + idx = FIX_ACPI_BEGIN; do { set_fixmap(idx, mfn, PAGE_HYPERVISOR); @@ -72,8 +72,8 @@ bool __acpi_unmap_table(const void *ptr, unsigned long size) unsigned int idx; /* We are only handling fixmap address in the arch code */ - if ( (vaddr < FIXMAP_ADDR(FIXMAP_ACPI_BEGIN)) || - (vaddr >= (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE)) ) + if ( (vaddr < FIXMAP_ADDR(FIX_ACPI_BEGIN)) || + (vaddr >= (FIXMAP_ADDR(FIX_ACPI_END) + PAGE_SIZE)) ) return false; /* @@ -81,16 +81,16 @@ bool __acpi_unmap_table(const void *ptr, unsigned long size) * for the ACPI fixmap region. The caller is expected to free with * the same address. */ - ASSERT((vaddr & PAGE_MASK) == FIXMAP_ADDR(FIXMAP_ACPI_BEGIN)); + ASSERT((vaddr & PAGE_MASK) == FIXMAP_ADDR(FIX_ACPI_BEGIN)); /* The region allocated fit in the ACPI fixmap region. */ - ASSERT(size < (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE - vaddr)); + ASSERT(size < (FIXMAP_ADDR(FIX_ACPI_END) + PAGE_SIZE - vaddr)); ASSERT(fixmap_inuse); fixmap_inuse = false; - size += vaddr - FIXMAP_ADDR(FIXMAP_ACPI_BEGIN); - idx = FIXMAP_ACPI_BEGIN; + size += vaddr - FIXMAP_ADDR(FIX_ACPI_BEGIN); + idx = FIX_ACPI_BEGIN; do { diff --git a/xen/arch/arm/include/asm/early_printk.h b/xen/arch/arm/include/asm/early_printk.h index c5149b2976da..a5f48801f476 100644 --- a/xen/arch/arm/include/asm/early_printk.h +++ b/xen/arch/arm/include/asm/early_printk.h @@ -17,7 +17,7 @@ /* need to add the uart address offset in page to the fixmap address */ #define EARLY_UART_VIRTUAL_ADDRESS \ - (FIXMAP_ADDR(FIXMAP_CONSOLE) + (CONFIG_EARLY_UART_BASE_ADDRESS & ~PAGE_MASK)) + (FIXMAP_ADDR(FIX_CONSOLE) + (CONFIG_EARLY_UART_BASE_ADDRESS & ~PAGE_MASK)) #endif /* !CONFIG_EARLY_PRINTK */ diff --git a/xen/arch/arm/include/asm/fixmap.h b/xen/arch/arm/include/asm/fixmap.h index d0c9a52c8c28..154db85686c2 100644 --- a/xen/arch/arm/include/asm/fixmap.h +++ b/xen/arch/arm/include/asm/fixmap.h @@ -8,17 +8,17 @@ #include /* Fixmap slots */ -#define FIXMAP_CONSOLE 0 /* The primary UART */ -#define FIXMAP_MISC 1 /* Ephemeral mappings of hardware */ -#define FIXMAP_ACPI_BEGIN 2 /* Start mappings of ACPI tables */ -#define FIXMAP_ACPI_END (FIXMAP_ACPI_BEGIN + NUM_FIXMAP_ACPI_PAGES - 1) /* End mappings of ACPI tables */ -#define FIXMAP_PMAP_BEGIN (FIXMAP_ACPI_END + 1) /* Start of PMAP */ -#define FIXMAP_PMAP_END (FIXMAP_PMAP_BEGIN + NUM_FIX_PMAP - 1) /* End of PMAP */ +#define FIX_CONSOLE 0 /* The primary UART */ +#define FIX_MISC 1 /* Ephemeral mappings of hardware */ +#define FIX_ACPI_BEGIN 2 /* Start mappings of ACPI tables */ +#define FIX_ACPI_END (FIX_ACPI_BEGIN + NUM_FIXMAP_ACPI_PAGES - 1) /* End mappings of ACPI tables */ +#define FIX_PMAP_BEGIN (FIX_ACPI_END + 1) /* Start of PMAP */ +#define FIX_PMAP_END (FIX_PMAP_BEGIN + NUM_FIX_PMAP - 1) /* End of PMAP */ -#define FIXMAP_LAST FIXMAP_PMAP_END +#define FIX_LAST FIX_PMAP_END #define FIXADDR_START FIXMAP_ADDR(0) -#define FIXADDR_TOP FIXMAP_ADDR(FIXMAP_LAST) +#define FIXADDR_TOP FIXMAP_ADDR(FIX_LAST) #ifndef __ASSEMBLY__ diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c index 23b840ea9ea8..56800750fd9c 100644 --- a/xen/arch/arm/kernel.c +++ b/xen/arch/arm/kernel.c @@ -49,7 +49,7 @@ struct minimal_dtb_header { */ void __init copy_from_paddr(void *dst, paddr_t paddr, unsigned long len) { - void *src = (void *)FIXMAP_ADDR(FIXMAP_MISC); + void *src = (void *)FIXMAP_ADDR(FIX_MISC); while (len) { unsigned long l, s; @@ -57,10 +57,10 @@ void __init copy_from_paddr(void *dst, paddr_t paddr, unsigned long len) s = paddr & (PAGE_SIZE-1); l = min(PAGE_SIZE - s, len); - set_fixmap(FIXMAP_MISC, maddr_to_mfn(paddr), PAGE_HYPERVISOR_WC); + set_fixmap(FIX_MISC, maddr_to_mfn(paddr), PAGE_HYPERVISOR_WC); memcpy(dst, src + s, l); clean_dcache_va_range(dst, l); - clear_fixmap(FIXMAP_MISC); + clear_fixmap(FIX_MISC); paddr += l; dst += l; diff --git a/xen/common/pmap.c b/xen/common/pmap.c index 14517198aae3..6e3ba9298df4 100644 --- a/xen/common/pmap.c +++ b/xen/common/pmap.c @@ -32,8 +32,8 @@ void *__init pmap_map(mfn_t mfn) __set_bit(idx, inuse); - slot = idx + FIXMAP_PMAP_BEGIN; - ASSERT(slot >= FIXMAP_PMAP_BEGIN && slot <= FIXMAP_PMAP_END); + slot = idx + FIX_PMAP_BEGIN; + ASSERT(slot >= FIX_PMAP_BEGIN && slot <= FIX_PMAP_END); /* * We cannot use set_fixmap() here. We use PMAP when the domain map @@ -53,10 +53,10 @@ void __init pmap_unmap(const void *p) unsigned int slot = virt_to_fix((unsigned long)p); ASSERT(system_state < SYS_STATE_smp_boot); - ASSERT(slot >= FIXMAP_PMAP_BEGIN && slot <= FIXMAP_PMAP_END); + ASSERT(slot >= FIX_PMAP_BEGIN && slot <= FIX_PMAP_END); ASSERT(!in_irq()); - idx = slot - FIXMAP_PMAP_BEGIN; + idx = slot - FIX_PMAP_BEGIN; __clear_bit(idx, inuse); arch_pmap_unmap(slot); From patchwork Fri Dec 16 11:48:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074973 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3A958C3DA6E for ; Fri, 16 Dec 2022 12:17:54 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464571.723052 (Exim 4.92) (envelope-from ) id 1p69ep-0008Uf-92; Fri, 16 Dec 2022 12:17:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464571.723052; Fri, 16 Dec 2022 12:17:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69eo-0008Rs-Nx; Fri, 16 Dec 2022 12:17:42 +0000 Received: by outflank-mailman (input) for mailman id 464571; Fri, 16 Dec 2022 12:17:38 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ek-0007Ln-0x for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 12:17:38 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ek-00034r-00 for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 12:17:38 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DG-0004sN-W3; Fri, 16 Dec 2022 11:49:15 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=2F2AKkVZIKUNaII54+BMfbbp6bI/EbvUjX6WZTvPuCA=; b=pte1Z/1XDpQMQlfRUwiXYayWq2 HF0IzSsuxpL3838A0DBY5yXDlB/7Mb8IGw4Wef/ufOHj4AQyRKjLmrroZDbS0eg2DKFb4gBde7toe OXPktBAdS5mp7+oJDanXCl/uSJsdwxvTDSFzyQ8EIDOq/zTqJQZMLNUSg8f6etE/OSY0=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Julien Grall , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 13/22] xen/x86: Add support for the PMAP Date: Fri, 16 Dec 2022 11:48:44 +0000 Message-Id: <20221216114853.8227-14-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Julien Grall PMAP will be used in a follow-up patch to bootstap map domain page infrastructure -- we need some way to map pages to setup the mapcache without a direct map. Signed-off-by: Julien Grall ---- The PMAP infrastructure was upstream separately for Arm since Hongyan sent the secret-free hypervisor series. So this is a new patch to plumb the feature on x86. --- xen/arch/x86/Kconfig | 1 + xen/arch/x86/include/asm/fixmap.h | 4 ++++ xen/arch/x86/include/asm/pmap.h | 25 +++++++++++++++++++++++++ 3 files changed, 30 insertions(+) create mode 100644 xen/arch/x86/include/asm/pmap.h diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig index 6a7825f4ba3c..47b120f18497 100644 --- a/xen/arch/x86/Kconfig +++ b/xen/arch/x86/Kconfig @@ -24,6 +24,7 @@ config X86 select HAS_PCI select HAS_PCI_MSI select HAS_PDX + select HAS_PMAP select HAS_SCHED_GRANULARITY select HAS_UBSAN select HAS_VPCI if HVM diff --git a/xen/arch/x86/include/asm/fixmap.h b/xen/arch/x86/include/asm/fixmap.h index 516ec3fa6c95..38f079873418 100644 --- a/xen/arch/x86/include/asm/fixmap.h +++ b/xen/arch/x86/include/asm/fixmap.h @@ -21,6 +21,8 @@ #include #include +#include + #include #include #include @@ -54,6 +56,8 @@ enum fixed_addresses { FIX_XEN_SHARED_INFO, #endif /* CONFIG_XEN_GUEST */ /* Everything else should go further down. */ + FIX_PMAP_BEGIN, + FIX_PMAP_END = FIX_PMAP_BEGIN + NUM_FIX_PMAP, FIX_APIC_BASE, FIX_IO_APIC_BASE_0, FIX_IO_APIC_BASE_END = FIX_IO_APIC_BASE_0 + MAX_IO_APICS-1, diff --git a/xen/arch/x86/include/asm/pmap.h b/xen/arch/x86/include/asm/pmap.h new file mode 100644 index 000000000000..62746e191d03 --- /dev/null +++ b/xen/arch/x86/include/asm/pmap.h @@ -0,0 +1,25 @@ +#ifndef __ASM_PMAP_H__ +#define __ASM_PMAP_H__ + +#include + +static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) +{ + unsigned long linear = (unsigned long)fix_to_virt(slot); + l1_pgentry_t *pl1e = &l1_fixmap[l1_table_offset(linear)]; + + ASSERT(!(l1e_get_flags(*pl1e) & _PAGE_PRESENT)); + + l1e_write_atomic(pl1e, l1e_from_mfn(mfn, PAGE_HYPERVISOR)); +} + +static inline void arch_pmap_unmap(unsigned int slot) +{ + unsigned long linear = (unsigned long)fix_to_virt(slot); + l1_pgentry_t *pl1e = &l1_fixmap[l1_table_offset(linear)]; + + l1e_write_atomic(pl1e, l1e_empty()); + flush_tlb_one_local(linear); +} + +#endif /* __ASM_PMAP_H__ */ From patchwork Fri Dec 16 11:48:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074978 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A843CC46467 for ; Fri, 16 Dec 2022 12:17:55 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464568.723026 (Exim 4.92) (envelope-from ) id 1p69en-0007tM-H1; Fri, 16 Dec 2022 12:17:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464568.723026; Fri, 16 Dec 2022 12:17:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69en-0007rW-35; Fri, 16 Dec 2022 12:17:41 +0000 Received: by outflank-mailman (input) for mailman id 464568; Fri, 16 Dec 2022 12:17:37 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-0007LX-Sv for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 12:17:37 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-00034K-Pm; Fri, 16 Dec 2022 12:17:37 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DI-0004sN-5A; Fri, 16 Dec 2022 11:49:16 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=jom7H6x2BCLQ18RtIk9fRFrou3zZPb9ESzutK4ktU4s=; b=2RyXYNdTUX810VURPQvYrG93Xp 8Whz/cVeKcjuD7pbldeGloSA2ysSekiJM9OjXO77IdzmxUwxtEXNwq6XqXjaDQZhp9d4eDFBE6ole LBNuWArzLaEXMv1bennufkSv8TDnBYdMsYTqzFtbPDkLrBtYidoYd2g0XlI6hKpw61aI=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Hongyan Xia , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , Julien Grall Subject: [PATCH 14/22] x86/domain_page: remove the fast paths when mfn is not in the directmap Date: Fri, 16 Dec 2022 11:48:45 +0000 Message-Id: <20221216114853.8227-15-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Hongyan Xia When mfn is not in direct map, never use mfn_to_virt for any mappings. We replace mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) with arch_mfns_in_direct_map(mfn, 1) because these two are equivalent. The extra comparison in arch_mfns_in_direct_map() looks different but because DIRECTMAP_VIRT_END is always higher, it does not make any difference. Lastly, domain_page_map_to_mfn() needs to gain to a special case for the PMAP. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall ---- Changes since Hongyan's version: * arch_mfn_in_direct_map() was renamed to arch_mfns_in_directmap() * add a special case for the PMAP in domain_page_map_to_mfn() --- xen/arch/x86/domain_page.c | 50 +++++++++++++++++++++++++++++++------- 1 file changed, 41 insertions(+), 9 deletions(-) diff --git a/xen/arch/x86/domain_page.c b/xen/arch/x86/domain_page.c index 55e337aaf703..89caefc8a210 100644 --- a/xen/arch/x86/domain_page.c +++ b/xen/arch/x86/domain_page.c @@ -14,8 +14,10 @@ #include #include #include +#include #include #include +#include #include static DEFINE_PER_CPU(struct vcpu *, override); @@ -35,10 +37,11 @@ static inline struct vcpu *mapcache_current_vcpu(void) /* * When using efi runtime page tables, we have the equivalent of the idle * domain's page tables but current may point at another domain's VCPU. - * Return NULL as though current is not properly set up yet. + * Return the idle domains's vcpu on that core because the efi per-domain + * region (where the mapcache is) is in-sync with the idle domain. */ if ( efi_rs_using_pgtables() ) - return NULL; + return idle_vcpu[smp_processor_id()]; /* * If guest_table is NULL, and we are running a paravirtualised guest, @@ -77,18 +80,24 @@ void *map_domain_page(mfn_t mfn) struct vcpu_maphash_entry *hashent; #ifdef NDEBUG - if ( mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) ) + if ( arch_mfns_in_directmap(mfn_x(mfn), 1) ) return mfn_to_virt(mfn_x(mfn)); #endif v = mapcache_current_vcpu(); - if ( !v ) - return mfn_to_virt(mfn_x(mfn)); + if ( !v || !v->domain->arch.mapcache.inuse ) + { + if ( arch_mfns_in_directmap(mfn_x(mfn), 1) ) + return mfn_to_virt(mfn_x(mfn)); + else + { + BUG_ON(system_state >= SYS_STATE_smp_boot); + return pmap_map(mfn); + } + } dcache = &v->domain->arch.mapcache; vcache = &v->arch.mapcache; - if ( !dcache->inuse ) - return mfn_to_virt(mfn_x(mfn)); perfc_incr(map_domain_page_count); @@ -184,6 +193,12 @@ void unmap_domain_page(const void *ptr) if ( !va || va >= DIRECTMAP_VIRT_START ) return; + if ( va >= FIXADDR_START && va < FIXADDR_TOP ) + { + pmap_unmap((void *)ptr); + return; + } + ASSERT(va >= MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END); v = mapcache_current_vcpu(); @@ -237,7 +252,7 @@ int mapcache_domain_init(struct domain *d) unsigned int bitmap_pages; #ifdef NDEBUG - if ( !mem_hotplug && max_page <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) ) + if ( !mem_hotplug && arch_mfn_in_directmap(0, max_page) ) return 0; #endif @@ -308,7 +323,7 @@ void *map_domain_page_global(mfn_t mfn) local_irq_is_enabled())); #ifdef NDEBUG - if ( mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) ) + if ( arch_mfn_in_directmap(mfn_x(mfn, 1)) ) return mfn_to_virt(mfn_x(mfn)); #endif @@ -335,6 +350,23 @@ mfn_t domain_page_map_to_mfn(const void *ptr) if ( va >= DIRECTMAP_VIRT_START ) return _mfn(virt_to_mfn(ptr)); + /* + * The fixmap is stealing the top-end of the VMAP. So the check for + * the PMAP *must* happen first. + * + * Also, the fixmap translate a slot to an address backwards. The + * logic will rely on it to avoid any complexity. So check at + * compile time this will always hold. + */ + BUILD_BUG_ON(fix_to_virt(FIX_PMAP_BEGIN) < fix_to_virt(FIX_PMAP_END)); + + if ( ((unsigned long)fix_to_virt(FIX_PMAP_END) <= va) && + ((va & PAGE_MASK) <= (unsigned long)fix_to_virt(FIX_PMAP_BEGIN)) ) + { + BUG_ON(system_state >= SYS_STATE_smp_boot); + return l1e_get_mfn(l1_fixmap[l1_table_offset(va)]); + } + if ( va >= VMAP_VIRT_START && va < VMAP_VIRT_END ) return vmap_to_mfn(va); From patchwork Fri Dec 16 11:48:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074976 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5821BC3DA7D for ; Fri, 16 Dec 2022 12:17:55 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464569.723032 (Exim 4.92) (envelope-from ) id 1p69en-00086G-Ve; Fri, 16 Dec 2022 12:17:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464569.723032; Fri, 16 Dec 2022 12:17:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69en-000816-HE; Fri, 16 Dec 2022 12:17:41 +0000 Received: by outflank-mailman (input) for mailman id 464569; Fri, 16 Dec 2022 12:17:38 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-0007Lc-Vo for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 12:17:37 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-00034Y-Sl; Fri, 16 Dec 2022 12:17:37 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DJ-0004sN-Dg; Fri, 16 Dec 2022 11:49:17 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=UCQfX8iMvnpIt+Q9gV/nKjf8GUqEBVsJAQhHBiS7Ykk=; b=jEszd2vIHifnpJMEMVuX8p5kmR qYfa3fiRP+5C70UMHtfog9ySnwE3LVN9RbTWolgSzo4ZPz+gbwXBqNA7frqhrYsXSLHZnHU11Cp75 4cZAuMWYr3CNX/4btJCn1A7sBfNu0PBSJNDACFYRzlX3m3Ph8bF90N91n3El1tYg4L5E=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Hongyan Xia , Andrew Cooper , George Dunlap , Jan Beulich , Stefano Stabellini , Wei Liu , Julien Grall Subject: [PATCH 15/22] xen/page_alloc: add a path for xenheap when there is no direct map Date: Fri, 16 Dec 2022 11:48:46 +0000 Message-Id: <20221216114853.8227-16-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Hongyan Xia When there is not an always-mapped direct map, xenheap allocations need to be mapped and unmapped on-demand. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall ---- I have left the call to map_pages_to_xen() and destroy_xen_mappings() in the split heap for now. I am not entirely convinced this is necessary because in that setup only the xenheap would be always mapped and this doesn't contain any guest memory (aside the grant-table). So map/unmapping for every allocation seems unnecessary. Changes since Hongyan's version: * Rebase * Fix indentation in alloc_xenheap_pages() * Fix build for arm32 --- xen/common/page_alloc.c | 41 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 40 insertions(+), 1 deletion(-) diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 0a950288e241..0c4af5a71407 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -2222,6 +2222,7 @@ void init_xenheap_pages(paddr_t ps, paddr_t pe) void *alloc_xenheap_pages(unsigned int order, unsigned int memflags) { struct page_info *pg; + void *ret; ASSERT_ALLOC_CONTEXT(); @@ -2230,17 +2231,36 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags) if ( unlikely(pg == NULL) ) return NULL; + ret = page_to_virt(pg); + + if ( !arch_has_directmap() && + map_pages_to_xen((unsigned long)ret, page_to_mfn(pg), 1UL << order, + PAGE_HYPERVISOR) ) + { + /* Failed to map xenheap pages. */ + free_heap_pages(pg, order, false); + return NULL; + } + return page_to_virt(pg); } void free_xenheap_pages(void *v, unsigned int order) { + unsigned long va = (unsigned long)v & PAGE_MASK; + ASSERT_ALLOC_CONTEXT(); if ( v == NULL ) return; + if ( !arch_has_directmap() && + destroy_xen_mappings(va, va + (1UL << (order + PAGE_SHIFT))) ) + dprintk(XENLOG_WARNING, + "Error while destroying xenheap mappings at %p, order %u\n", + v, order); + free_heap_pages(virt_to_page(v), order, false); } @@ -2264,6 +2284,7 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags) { struct page_info *pg; unsigned int i; + void *ret; ASSERT_ALLOC_CONTEXT(); @@ -2276,16 +2297,28 @@ void *alloc_xenheap_pages(unsigned int order, unsigned int memflags) if ( unlikely(pg == NULL) ) return NULL; + ret = page_to_virt(pg); + + if ( !arch_has_directmap() && + map_pages_to_xen((unsigned long)ret, page_to_mfn(pg), 1UL << order, + PAGE_HYPERVISOR) ) + { + /* Failed to map xenheap pages. */ + free_domheap_pages(pg, order); + return NULL; + } + for ( i = 0; i < (1u << order); i++ ) pg[i].count_info |= PGC_xen_heap; - return page_to_virt(pg); + return ret; } void free_xenheap_pages(void *v, unsigned int order) { struct page_info *pg; unsigned int i; + unsigned long va = (unsigned long)v & PAGE_MASK; ASSERT_ALLOC_CONTEXT(); @@ -2297,6 +2330,12 @@ void free_xenheap_pages(void *v, unsigned int order) for ( i = 0; i < (1u << order); i++ ) pg[i].count_info &= ~PGC_xen_heap; + if ( !arch_has_directmap() && + destroy_xen_mappings(va, va + (1UL << (order + PAGE_SHIFT))) ) + dprintk(XENLOG_WARNING, + "Error while destroying xenheap mappings at %p, order %u\n", + v, order); + free_heap_pages(pg, order, true); } From patchwork Fri Dec 16 11:48:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074981 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 07DF6C3DA6E for ; Fri, 16 Dec 2022 12:17:58 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464574.723081 (Exim 4.92) (envelope-from ) id 1p69es-0000ot-BN; Fri, 16 Dec 2022 12:17:46 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464574.723081; Fri, 16 Dec 2022 12:17:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69er-0000jM-DP; Fri, 16 Dec 2022 12:17:45 +0000 Received: by outflank-mailman (input) for mailman id 464574; Fri, 16 Dec 2022 12:17:38 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ek-0007M6-5n for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 12:17:38 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ek-00035J-3O; Fri, 16 Dec 2022 12:17:38 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DK-0004sN-J4; Fri, 16 Dec 2022 11:49:18 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=Fh/+Yu2IVhcS+oFNJd/xfdw1K28l8x0te4dRw3BaaIo=; b=NXmxWhqOxfDRQJ/rW5WuU3OJ5s vMl2K4A1hRxKwHrWg3G08Rp4Qg4ldVys3xXHuuAxC6LgWtm76+LLv248VGkTcSG3zdqk2n6zZzcNF +boDbASPWG/EK7ni00T4BnJUbLmgMquxER9VoZRGv/lP/F5XzH3s1wXypmXAKinjwa2s=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Hongyan Xia , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , Julien Grall Subject: [PATCH 16/22] x86/setup: leave early boot slightly earlier Date: Fri, 16 Dec 2022 11:48:47 +0000 Message-Id: <20221216114853.8227-17-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Hongyan Xia When we do not have a direct map, memory for metadata of heap nodes in init_node_heap() is allocated from xenheap, which needs to be mapped and unmapped on demand. However, we cannot just take memory from the boot allocator to create the PTEs while we are passing memory to the heap allocator. To solve this race, we leave early boot slightly sooner so that Xen PTE pages are allocated from the heap instead of the boot allocator. We can do this because the metadata for the 1st node is statically allocated, and by the time we need memory to create mappings for the 2nd node, we already have enough memory in the heap allocator in the 1st node. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall --- xen/arch/x86/setup.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 2cb051c6e4e7..ec5a7448a225 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1648,6 +1648,22 @@ void __init noreturn __start_xen(unsigned long mbi_p) numa_initmem_init(0, raw_max_page); + /* + * When we do not have a direct map, memory for metadata of heap nodes in + * init_node_heap() is allocated from xenheap, which needs to be mapped and + * unmapped on demand. However, we cannot just take memory from the boot + * allocator to create the PTEs while we are passing memory to the heap + * allocator during end_boot_allocator(). + * + * To solve this race, we need to leave early boot before + * end_boot_allocator() so that Xen PTE pages are allocated from the heap + * instead of the boot allocator. We can do this because the metadata for + * the 1st node is statically allocated, and by the time we need memory to + * create mappings for the 2nd node, we already have enough memory in the + * heap allocator in the 1st node. + */ + system_state = SYS_STATE_boot; + if ( max_page - 1 > virt_to_mfn(HYPERVISOR_VIRT_END - 1) ) { unsigned long limit = virt_to_mfn(HYPERVISOR_VIRT_END - 1); @@ -1677,8 +1693,6 @@ void __init noreturn __start_xen(unsigned long mbi_p) else end_boot_allocator(); - system_state = SYS_STATE_boot; - bsp_stack = cpu_alloc_stack(0); if ( !bsp_stack ) panic("No memory for BSP stack\n"); From patchwork Fri Dec 16 11:48:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1E9FBC4332F for ; Fri, 16 Dec 2022 12:17:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464566.723010 (Exim 4.92) (envelope-from ) id 1p69em-0007fz-Mf; Fri, 16 Dec 2022 12:17:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464566.723010; Fri, 16 Dec 2022 12:17:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69em-0007eX-Eq; Fri, 16 Dec 2022 12:17:40 +0000 Received: by outflank-mailman (input) for mailman id 464566; Fri, 16 Dec 2022 12:17:37 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-0007LM-RB for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 12:17:37 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-00034C-MR; Fri, 16 Dec 2022 12:17:37 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DL-0004sN-Rc; Fri, 16 Dec 2022 11:49:20 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=C7J11PdwQAWvXzpUfaG/t5NjyOIfJIe9L9M+iZjsZ88=; b=SjoRoA2TbEg5ZKtGJx/cWUhwP1 OxQuQW4yTon8/KGxTK021Utu+jR4S/+tJec27zz7oMMUVsmUOlBHP+EbpLSALUsVqW9xIMpdHODH/ jPZfchwzSmpEg2nmk67/EWmD1nbmKRmZVNzgRYuyxdPzNfjjmabszsvWgf6eK4h8qmuc=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Hongyan Xia , Andrew Cooper , George Dunlap , Jan Beulich , Stefano Stabellini , Wei Liu , Julien Grall Subject: [PATCH 17/22] x86/setup: vmap heap nodes when they are outside the direct map Date: Fri, 16 Dec 2022 11:48:48 +0000 Message-Id: <20221216114853.8227-18-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Hongyan Xia When we do not have a direct map, archs_mfn_in_direct_map() will always return false, thus init_node_heap() will allocate xenheap pages from an existing node for the metadata of a new node. This means that the metadata of a new node is in a different node, slowing down heap allocation. Since we now have early vmap, vmap the metadata locally in the new node. Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall ---- Changes from Hongyan's version: * arch_mfn_in_direct_map() was renamed to arch_mfns_in_direct_map() * Use vmap_contig_pages() rather than __vmap(...). * Add missing include (xen/vmap.h) so it compiles on Arm Reviewed-by: Stefano Stabellini --- xen/common/page_alloc.c | 42 +++++++++++++++++++++++++++++++---------- 1 file changed, 32 insertions(+), 10 deletions(-) diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 0c4af5a71407..581c15d74dfb 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -136,6 +136,7 @@ #include #include #include +#include #include #include @@ -597,22 +598,43 @@ static unsigned long init_node_heap(int node, unsigned long mfn, needed = 0; } else if ( *use_tail && nr >= needed && - arch_mfns_in_directmap(mfn + nr - needed, needed) && (!xenheap_bits || !((mfn + nr - 1) >> (xenheap_bits - PAGE_SHIFT))) ) { - _heap[node] = mfn_to_virt(mfn + nr - needed); - avail[node] = mfn_to_virt(mfn + nr - 1) + - PAGE_SIZE - sizeof(**avail) * NR_ZONES; - } - else if ( nr >= needed && - arch_mfns_in_directmap(mfn, needed) && + if ( arch_mfns_in_directmap(mfn + nr - needed, needed) ) + { + _heap[node] = mfn_to_virt(mfn + nr - needed); + avail[node] = mfn_to_virt(mfn + nr - 1) + + PAGE_SIZE - sizeof(**avail) * NR_ZONES; + } + else + { + mfn_t needed_start = _mfn(mfn + nr - needed); + + _heap[node] = vmap_contig_pages(needed_start, needed); + BUG_ON(!_heap[node]); + avail[node] = (void *)(_heap[node]) + (needed << PAGE_SHIFT) - + sizeof(**avail) * NR_ZONES; + } + } else if ( nr >= needed && (!xenheap_bits || !((mfn + needed - 1) >> (xenheap_bits - PAGE_SHIFT))) ) { - _heap[node] = mfn_to_virt(mfn); - avail[node] = mfn_to_virt(mfn + needed - 1) + - PAGE_SIZE - sizeof(**avail) * NR_ZONES; + if ( arch_mfns_in_directmap(mfn, needed) ) + { + _heap[node] = mfn_to_virt(mfn); + avail[node] = mfn_to_virt(mfn + needed - 1) + + PAGE_SIZE - sizeof(**avail) * NR_ZONES; + } + else + { + mfn_t needed_start = _mfn(mfn); + + _heap[node] = vmap_contig_pages(needed_start, needed); + BUG_ON(!_heap[node]); + avail[node] = (void *)(_heap[node]) + (needed << PAGE_SHIFT) - + sizeof(**avail) * NR_ZONES; + } *use_tail = false; } else if ( get_order_from_bytes(sizeof(**_heap)) == From patchwork Fri Dec 16 11:48:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074974 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7B65BC3DA7B for ; Fri, 16 Dec 2022 12:17:54 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464563.722990 (Exim 4.92) (envelope-from ) id 1p69el-0007MU-MK; Fri, 16 Dec 2022 12:17:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464563.722990; Fri, 16 Dec 2022 12:17:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69el-0007MN-Il; Fri, 16 Dec 2022 12:17:39 +0000 Received: by outflank-mailman (input) for mailman id 464563; Fri, 16 Dec 2022 12:17:37 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-0007L5-H5 for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 12:17:37 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-00033p-CF; Fri, 16 Dec 2022 12:17:37 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DN-0004sN-0j; Fri, 16 Dec 2022 11:49:21 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=nUSS981XXVbzFIos+o7hpfIBcVywhSrr+zfV2HwfzJo=; b=ZSvLchk7+kKe/9gZN0LrxVyCaU Tfv40eywb1F8m3wsqq/uNlQeyknwb4iQ03wIEb+jcYcjJo0uDhi1jQamB1rrgJze/me2W6QOBGgoO xgB+6qBC/BOKHTWMDZEJEl4bEa7rlZHTRItgDtjsmRbAYq23QVXfdBaI7clBeWnKBkOY=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Hongyan Xia , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , Julien Grall Subject: [PATCH 18/22] x86/setup: do not create valid mappings when directmap=no Date: Fri, 16 Dec 2022 11:48:49 +0000 Message-Id: <20221216114853.8227-19-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Hongyan Xia Create empty mappings in the second e820 pass. Also, destroy existing direct map mappings created in the first pass. To make xenheap pages visible in guests, it is necessary to create empty L3 tables in the direct map even when directmap=no, since guest cr3s copy idle domain's L4 entries, which means they will share mappings in the direct map if we pre-populate idle domain's L4 entries and L3 tables. A helper is introduced for this. Also, after the direct map is actually gone, we need to stop updating the direct map in update_xen_mappings(). Signed-off-by: Hongyan Xia Signed-off-by: Julien Grall --- xen/arch/x86/setup.c | 74 +++++++++++++++++++++++++++++++++++++++----- 1 file changed, 67 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index ec5a7448a225..87967abb00cb 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -856,6 +856,57 @@ static struct domain *__init create_dom0(const module_t *image, /* How much of the directmap is prebuilt at compile time. */ #define PREBUILT_MAP_LIMIT (1 << L2_PAGETABLE_SHIFT) +/* + * This either populates a valid direct map, or allocates empty L3 tables and + * creates the L4 entries for virtual address between [start, end) in the + * direct map depending on arch_has_directmap(); + * + * When directmap=no, we still need to populate empty L3 tables in the + * direct map region. The reason is that on-demand xenheap mappings are + * created in the idle domain's page table but must be seen by + * everyone. Since all domains share the direct map L4 entries, they + * will share xenheap mappings if we pre-populate the L4 entries and L3 + * tables in the direct map region for all RAM. We also rely on the fact + * that L3 tables are never freed. + */ +static void __init populate_directmap(uint64_t pstart, uint64_t pend, + unsigned int flags) +{ + unsigned long vstart = (unsigned long)__va(pstart); + unsigned long vend = (unsigned long)__va(pend); + + if ( pstart >= pend ) + return; + + BUG_ON(vstart < DIRECTMAP_VIRT_START); + BUG_ON(vend > DIRECTMAP_VIRT_END); + + if ( arch_has_directmap() ) + /* Populate valid direct map. */ + BUG_ON(map_pages_to_xen(vstart, maddr_to_mfn(pstart), + PFN_DOWN(pend - pstart), flags)); + else + { + /* Create empty L3 tables. */ + unsigned long vaddr = vstart & ~((1UL << L4_PAGETABLE_SHIFT) - 1); + + for ( ; vaddr < vend; vaddr += (1UL << L4_PAGETABLE_SHIFT) ) + { + l4_pgentry_t *pl4e = &idle_pg_table[l4_table_offset(vaddr)]; + + if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) ) + { + mfn_t mfn = alloc_boot_pages(1, 1); + void *v = map_domain_page(mfn); + + clear_page(v); + UNMAP_DOMAIN_PAGE(v); + l4e_write(pl4e, l4e_from_mfn(mfn, __PAGE_HYPERVISOR)); + } + } + } +} + void __init noreturn __start_xen(unsigned long mbi_p) { char *memmap_type = NULL; @@ -1507,8 +1558,17 @@ void __init noreturn __start_xen(unsigned long mbi_p) map_e = min_t(uint64_t, e, ARRAY_SIZE(l2_directmap) << L2_PAGETABLE_SHIFT); - /* Pass mapped memory to allocator /before/ creating new mappings. */ + /* + * Pass mapped memory to allocator /before/ creating new mappings. + * The direct map for the bottom 4GiB has been populated in the first + * e820 pass. In the second pass, we make sure those existing mappings + * are destroyed when directmap=no. + */ init_boot_pages(s, min(map_s, e)); + if ( !arch_has_directmap() ) + destroy_xen_mappings((unsigned long)__va(s), + (unsigned long)__va(min(map_s, e))); + s = map_s; if ( s < map_e ) { @@ -1517,6 +1577,9 @@ void __init noreturn __start_xen(unsigned long mbi_p) map_s = (s + mask) & ~mask; map_e &= ~mask; init_boot_pages(map_s, map_e); + if ( !arch_has_directmap() ) + destroy_xen_mappings((unsigned long)__va(map_s), + (unsigned long)__va(map_e)); } if ( map_s > map_e ) @@ -1530,8 +1593,7 @@ void __init noreturn __start_xen(unsigned long mbi_p) if ( map_e < end ) { - map_pages_to_xen((unsigned long)__va(map_e), maddr_to_mfn(map_e), - PFN_DOWN(end - map_e), PAGE_HYPERVISOR); + populate_directmap(map_e, end, PAGE_HYPERVISOR); init_boot_pages(map_e, end); map_e = end; } @@ -1540,13 +1602,11 @@ void __init noreturn __start_xen(unsigned long mbi_p) { /* This range must not be passed to the boot allocator and * must also not be mapped with _PAGE_GLOBAL. */ - map_pages_to_xen((unsigned long)__va(map_e), maddr_to_mfn(map_e), - PFN_DOWN(e - map_e), __PAGE_HYPERVISOR_RW); + populate_directmap(map_e, e, __PAGE_HYPERVISOR_RW); } if ( s < map_s ) { - map_pages_to_xen((unsigned long)__va(s), maddr_to_mfn(s), - PFN_DOWN(map_s - s), PAGE_HYPERVISOR); + populate_directmap(s, map_s, PAGE_HYPERVISOR); init_boot_pages(s, map_s); } } From patchwork Fri Dec 16 11:48:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074971 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A0A08C001B2 for ; Fri, 16 Dec 2022 12:17:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464570.723040 (Exim 4.92) (envelope-from ) id 1p69eo-0008Kq-HJ; Fri, 16 Dec 2022 12:17:42 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464570.723040; Fri, 16 Dec 2022 12:17:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69eo-0008Ha-8s; Fri, 16 Dec 2022 12:17:42 +0000 Received: by outflank-mailman (input) for mailman id 464570; Fri, 16 Dec 2022 12:17:38 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-0007Ld-W0 for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 12:17:37 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-00034R-RI; Fri, 16 Dec 2022 12:17:37 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DN-0004sN-W7; Fri, 16 Dec 2022 11:49:22 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=NkfwiZGHGUT9aDZPQ52XaRpLx0pc2E+yXE3Z0GACfGA=; b=tL1r1SSswrEWuqPOR8MX6COqIE aQZQIFzsUy4F/Gqk6/uu6XGHjPee0wjsy6yrgb+G+9EPS5xeXh9aZlTksVJX7NXw5KDlr8C3lq5jW spLEzj5WxpO3H+RBDjdqcWGJQImjavd0tGHBgzfadMFnTThQ644wQUFsXx2ZZF5Q1MwE=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Julien Grall , Stefano Stabellini , Bertrand Marquis , Volodymyr Babchuk Subject: [PATCH 19/22] xen/arm32: mm: Rename 'first' to 'root' in init_secondary_pagetables() Date: Fri, 16 Dec 2022 11:48:50 +0000 Message-Id: <20221216114853.8227-20-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Julien Grall The arm32 version of init_secondary_pagetables() will soon be re-used for arm64 as well where the root table start at level 0 rather than level 1. So rename 'first' to 'root'. Signed-off-by: Julien Grall Reviewed-by: Henry Wang Reviewed-by: Stefano Stabellini --- xen/arch/arm/mm.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index 0fc6f2992dd1..4e208f7d20c8 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -571,32 +571,30 @@ int init_secondary_pagetables(int cpu) #else int init_secondary_pagetables(int cpu) { - lpae_t *first; + lpae_t *root = alloc_xenheap_page(); - first = alloc_xenheap_page(); /* root == first level on 32-bit 3-level trie */ - - if ( !first ) + if ( !root ) { - printk("CPU%u: Unable to allocate the first page-table\n", cpu); + printk("CPU%u: Unable to allocate the root page-table\n", cpu); return -ENOMEM; } /* Initialise root pagetable from root of boot tables */ - memcpy(first, cpu0_pgtable, PAGE_SIZE); - per_cpu(xen_pgtable, cpu) = first; + memcpy(root, cpu0_pgtable, PAGE_SIZE); + per_cpu(xen_pgtable, cpu) = root; if ( !init_domheap_mappings(cpu) ) { printk("CPU%u: Unable to prepare the domheap page-tables\n", cpu); per_cpu(xen_pgtable, cpu) = NULL; - free_xenheap_page(first); + free_xenheap_page(root); return -ENOMEM; } clear_boot_pagetables(); /* Set init_ttbr for this CPU coming up */ - init_ttbr = __pa(first); + init_ttbr = __pa(root); clean_dcache(init_ttbr); return 0; From patchwork Fri Dec 16 11:48:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074982 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 43F15C4167B for ; Fri, 16 Dec 2022 12:17:58 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464573.723071 (Exim 4.92) (envelope-from ) id 1p69er-0000bJ-7S; Fri, 16 Dec 2022 12:17:45 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464573.723071; Fri, 16 Dec 2022 12:17:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69eq-0000Xg-GC; Fri, 16 Dec 2022 12:17:44 +0000 Received: by outflank-mailman (input) for mailman id 464573; Fri, 16 Dec 2022 12:17:38 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ek-0007Lv-2J for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 12:17:38 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-00034i-Ua; Fri, 16 Dec 2022 12:17:37 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DO-0004sN-VG; Fri, 16 Dec 2022 11:49:23 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=4AXKHSNyisU3EQXQfz0D6DbkivTeO0uHrc0fyYfL7V4=; b=lDKyHO+sTxwjLuN3Q6Kc8Xa7S7 +YPdzM8WDKXk8C46CCwsBPRoKnDPB+zal//UAvoNsug6nRAoEtycDPqEveWdDVnBcXVlfn9kBBVaq LopHfeGX2eiI/VGGvQLhRv/FR88fmuPeOdGDkvE4xwpOm64d/RridrcBKcMLZWHKp6iE=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Julien Grall , Stefano Stabellini , Bertrand Marquis , Volodymyr Babchuk Subject: [PATCH 20/22] xen/arm64: mm: Use per-pCPU page-tables Date: Fri, 16 Dec 2022 11:48:51 +0000 Message-Id: <20221216114853.8227-21-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Julien Grall At the moment, on Arm64, every pCPU are sharing the same page-tables. In a follow-up patch, we will allow the possibility to remove the direct map and therefore it will be necessary to have a mapcache. While we have plenty of spare virtual address space to have to reserve part for each pCPU, it means that temporary mappings (e.g. guest memory) could be accessible by every pCPU. In order to increase our security posture, it would be better if those mappings are only accessible by the pCPU doing the temporary mapping. In addition to that, a per-pCPU page-tables opens the way to have per-domain mapping area. Arm32 is already using per-pCPU page-tables so most of the code can be re-used. Arm64 doesn't yet have support for the mapcache, so a stub is provided (moved to its own header asm/domain_page.h). Take the opportunity to fix a typo in a comment that is modified. Signed-off-by: Julien Grall Reviewed-by: Henry Wang Reviewed-by: Stefano Stabellini --- xen/arch/arm/domain_page.c | 2 ++ xen/arch/arm/include/asm/arm32/mm.h | 8 ----- xen/arch/arm/include/asm/domain_page.h | 13 ++++++++ xen/arch/arm/include/asm/mm.h | 5 +++ xen/arch/arm/mm.c | 42 +++++++------------------- xen/arch/arm/setup.c | 1 + 6 files changed, 32 insertions(+), 39 deletions(-) create mode 100644 xen/arch/arm/include/asm/domain_page.h diff --git a/xen/arch/arm/domain_page.c b/xen/arch/arm/domain_page.c index b7c02c919064..4540b3c5f24c 100644 --- a/xen/arch/arm/domain_page.c +++ b/xen/arch/arm/domain_page.c @@ -3,6 +3,8 @@ #include #include +#include + /* Override macros from asm/page.h to make them work with mfn_t */ #undef virt_to_mfn #define virt_to_mfn(va) _mfn(__virt_to_mfn(va)) diff --git a/xen/arch/arm/include/asm/arm32/mm.h b/xen/arch/arm/include/asm/arm32/mm.h index 8bfc906e7178..6b039d9ceaa2 100644 --- a/xen/arch/arm/include/asm/arm32/mm.h +++ b/xen/arch/arm/include/asm/arm32/mm.h @@ -1,12 +1,6 @@ #ifndef __ARM_ARM32_MM_H__ #define __ARM_ARM32_MM_H__ -#include - -#include - -DECLARE_PER_CPU(lpae_t *, xen_pgtable); - /* * Only a limited amount of RAM, called xenheap, is always mapped on ARM32. * For convenience always return false. @@ -16,8 +10,6 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr) return false; } -bool init_domheap_mappings(unsigned int cpu); - #endif /* __ARM_ARM32_MM_H__ */ /* diff --git a/xen/arch/arm/include/asm/domain_page.h b/xen/arch/arm/include/asm/domain_page.h new file mode 100644 index 000000000000..e9f52685e2ec --- /dev/null +++ b/xen/arch/arm/include/asm/domain_page.h @@ -0,0 +1,13 @@ +#ifndef __ASM_ARM_DOMAIN_PAGE_H__ +#define __ASM_ARM_DOMAIN_PAGE_H__ + +#ifdef CONFIG_ARCH_MAP_DOMAIN_PAGE +bool init_domheap_mappings(unsigned int cpu); +#else +static inline bool init_domheap_mappings(unsigned int cpu) +{ + return true; +} +#endif + +#endif /* __ASM_ARM_DOMAIN_PAGE_H__ */ diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index 2366928d71aa..7a2c775f9562 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -2,6 +2,9 @@ #define __ARCH_ARM_MM__ #include +#include + +#include #include #include #include @@ -14,6 +17,8 @@ # error "unknown ARM variant" #endif +DECLARE_PER_CPU(lpae_t *, xen_pgtable); + /* Align Xen to a 2 MiB boundary. */ #define XEN_PADDR_ALIGN (1 << 21) diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index 4e208f7d20c8..2af751af9003 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -24,6 +24,7 @@ #include +#include #include #include @@ -90,20 +91,19 @@ DEFINE_BOOT_PAGE_TABLE(boot_third); * xen_second, xen_fixmap and xen_xenmap are always shared between all * PCPUs. */ +/* Per-CPU pagetable pages */ +/* xen_pgtable == root of the trie (zeroeth level on 64-bit, first on 32-bit) */ +DEFINE_PER_CPU(lpae_t *, xen_pgtable); + +/* Root of the trie for cpu0, other CPU's PTs are dynamically allocated */ +static DEFINE_PAGE_TABLE(cpu0_pgtable); +#define THIS_CPU_PGTABLE this_cpu(xen_pgtable) #ifdef CONFIG_ARM_64 #define HYP_PT_ROOT_LEVEL 0 -static DEFINE_PAGE_TABLE(xen_pgtable); static DEFINE_PAGE_TABLE(xen_first); -#define THIS_CPU_PGTABLE xen_pgtable #else #define HYP_PT_ROOT_LEVEL 1 -/* Per-CPU pagetable pages */ -/* xen_pgtable == root of the trie (zeroeth level on 64-bit, first on 32-bit) */ -DEFINE_PER_CPU(lpae_t *, xen_pgtable); -#define THIS_CPU_PGTABLE this_cpu(xen_pgtable) -/* Root of the trie for cpu0, other CPU's PTs are dynamically allocated */ -static DEFINE_PAGE_TABLE(cpu0_pgtable); #endif /* Common pagetable leaves */ @@ -481,14 +481,13 @@ void __init setup_pagetables(unsigned long boot_phys_offset) phys_offset = boot_phys_offset; + p = cpu0_pgtable; + #ifdef CONFIG_ARM_64 - p = (void *) xen_pgtable; p[0] = pte_of_xenaddr((uintptr_t)xen_first); p[0].pt.table = 1; p[0].pt.xn = 0; p = (void *) xen_first; -#else - p = (void *) cpu0_pgtable; #endif /* Map xen second level page-table */ @@ -527,19 +526,13 @@ void __init setup_pagetables(unsigned long boot_phys_offset) pte.pt.table = 1; xen_second[second_table_offset(FIXMAP_ADDR(0))] = pte; -#ifdef CONFIG_ARM_64 - ttbr = (uintptr_t) xen_pgtable + phys_offset; -#else ttbr = (uintptr_t) cpu0_pgtable + phys_offset; -#endif switch_ttbr(ttbr); xen_pt_enforce_wnx(); -#ifdef CONFIG_ARM_32 per_cpu(xen_pgtable, 0) = cpu0_pgtable; -#endif } static void clear_boot_pagetables(void) @@ -557,18 +550,6 @@ static void clear_boot_pagetables(void) clear_table(boot_third); } -#ifdef CONFIG_ARM_64 -int init_secondary_pagetables(int cpu) -{ - clear_boot_pagetables(); - - /* Set init_ttbr for this CPU coming up. All CPus share a single setof - * pagetables, but rewrite it each time for consistency with 32 bit. */ - init_ttbr = (uintptr_t) xen_pgtable + phys_offset; - clean_dcache(init_ttbr); - return 0; -} -#else int init_secondary_pagetables(int cpu) { lpae_t *root = alloc_xenheap_page(); @@ -599,7 +580,6 @@ int init_secondary_pagetables(int cpu) return 0; } -#endif /* MMU setup for secondary CPUS (which already have paging enabled) */ void mmu_init_secondary_cpu(void) @@ -1089,7 +1069,7 @@ static int xen_pt_update(unsigned long virt, unsigned long left = nr_mfns; /* - * For arm32, page-tables are different on each CPUs. Yet, they share + * Page-tables are different on each CPU. Yet, they share * some common mappings. It is assumed that only common mappings * will be modified with this function. * diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index 2311726f5ddd..88d9d90fb5ad 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -39,6 +39,7 @@ #include #include #include +#include #include #include #include From patchwork Fri Dec 16 11:48:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074975 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D0E68C4167B for ; Fri, 16 Dec 2022 12:17:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464567.723016 (Exim 4.92) (envelope-from ) id 1p69en-0007ni-0v; Fri, 16 Dec 2022 12:17:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464567.723016; Fri, 16 Dec 2022 12:17:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69em-0007mS-O4; Fri, 16 Dec 2022 12:17:40 +0000 Received: by outflank-mailman (input) for mailman id 464567; Fri, 16 Dec 2022 12:17:37 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-0007LR-S6 for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 12:17:37 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-000346-KU; Fri, 16 Dec 2022 12:17:37 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DP-0004sN-UP; Fri, 16 Dec 2022 11:49:24 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=CKARFD/JuIFH+12h1mCsTef+zC+DdBGXMOxyO40T+VE=; b=DfWQTxI833VghcPVaObhkv8mmj IGlaC4xXPYVQOlW5mgr9i3cecgEpJG6KgM+TqUULclYbbruYehFkAaxttk1VBqd9IpNXbQvcU5iDS KY2GIMtKOV5aouN6JJA9munAIlS5nul2zEW98NC+ZzAxnAvvqmb1qI4aohAAh50zE+tg=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Julien Grall , Stefano Stabellini , Bertrand Marquis , Volodymyr Babchuk Subject: [PATCH 21/22] xen/arm64: Implement a mapcache for arm64 Date: Fri, 16 Dec 2022 11:48:52 +0000 Message-Id: <20221216114853.8227-22-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Julien Grall At the moment, on arm64, map_domain_page() is implemented using virt_to_mfn(). Therefore it is relying on the directmap. In a follow-up patch, we will allow the admin to remove the directmap. Therefore we want to implement a mapcache. Thanksfully there is already one for arm32. So select ARCH_ARM_DOMAIN_PAGE and add the necessary boiler plate to support 64-bit: - The page-table start at level 0, so we need to allocate the level 1 page-table - map_domain_page() should check if the page is in the directmap. If yes, then use virt_to_mfn() to limit the performance impact when the directmap is still enabled (this will be selectable on the command line). Take the opportunity to replace first_table_offset(...) with offsets[...]. Note that, so far, arch_mfns_in_directmap() always return true on arm64. So the mapcache is not yet used. This will change in a follow-up patch. Signed-off-by: Julien Grall ---- There are a few TODOs: - It is becoming more critical to fix the mapcache implementation (this is not compliant with the Arm Arm) - Evaluate the performance Reviewed-by: Henry Wang Reviewed-by: Stefano Stabellini --- xen/arch/arm/Kconfig | 1 + xen/arch/arm/domain_page.c | 47 +++++++++++++++++++++++++++---- xen/arch/arm/include/asm/config.h | 7 +++++ xen/arch/arm/include/asm/mm.h | 5 ++++ xen/arch/arm/mm.c | 6 ++-- xen/arch/arm/setup.c | 4 +++ 6 files changed, 62 insertions(+), 8 deletions(-) diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index 239d3aed3c7f..9c58b2d5c3aa 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -9,6 +9,7 @@ config ARM_64 select 64BIT select ARM_EFI select HAS_FAST_MULTIPLY + select ARCH_MAP_DOMAIN_PAGE config ARM def_bool y diff --git a/xen/arch/arm/domain_page.c b/xen/arch/arm/domain_page.c index 4540b3c5f24c..f3547dc853ef 100644 --- a/xen/arch/arm/domain_page.c +++ b/xen/arch/arm/domain_page.c @@ -1,4 +1,5 @@ /* SPDX-License-Identifier: GPL-2.0-or-later */ +#include #include #include #include @@ -8,6 +9,8 @@ /* Override macros from asm/page.h to make them work with mfn_t */ #undef virt_to_mfn #define virt_to_mfn(va) _mfn(__virt_to_mfn(va)) +#undef mfn_to_virt +#define mfn_to_virt(va) __mfn_to_virt(mfn_x(mfn)) /* cpu0's domheap page tables */ static DEFINE_PAGE_TABLES(cpu0_dommap, DOMHEAP_SECOND_PAGES); @@ -31,13 +34,30 @@ bool init_domheap_mappings(unsigned int cpu) { unsigned int order = get_order_from_pages(DOMHEAP_SECOND_PAGES); lpae_t *root = per_cpu(xen_pgtable, cpu); + lpae_t *first; unsigned int i, first_idx; lpae_t *domheap; mfn_t mfn; + /* Convenience aliases */ + DECLARE_OFFSETS(offsets, DOMHEAP_VIRT_START); + ASSERT(root); ASSERT(!per_cpu(xen_dommap, cpu)); + /* + * On Arm64, the root is at level 0. Therefore we need an extra step + * to allocate the first level page-table. + */ +#ifdef CONFIG_ARM_64 + if ( create_xen_table(&root[offsets[0]]) ) + return false; + + first = xen_map_table(lpae_get_mfn(root[offsets[0]])); +#else + first = root; +#endif + /* * The domheap for cpu0 is initialized before the heap is initialized. * So we need to use pre-allocated pages. @@ -58,16 +78,20 @@ bool init_domheap_mappings(unsigned int cpu) * domheap mapping pages. */ mfn = virt_to_mfn(domheap); - first_idx = first_table_offset(DOMHEAP_VIRT_START); + first_idx = offsets[1]; for ( i = 0; i < DOMHEAP_SECOND_PAGES; i++ ) { lpae_t pte = mfn_to_xen_entry(mfn_add(mfn, i), MT_NORMAL); pte.pt.table = 1; - write_pte(&root[first_idx + i], pte); + write_pte(&first[first_idx + i], pte); } per_cpu(xen_dommap, cpu) = domheap; +#ifdef CONFIG_ARM_64 + xen_unmap_table(first); +#endif + return true; } @@ -91,6 +115,10 @@ void *map_domain_page(mfn_t mfn) lpae_t pte; int i, slot; + /* Bypass the mapcache if the page is in the directmap */ + if ( arch_mfns_in_directmap(mfn_x(mfn), 1) ) + return mfn_to_virt(mfn); + local_irq_save(flags); /* The map is laid out as an open-addressed hash table where each @@ -151,15 +179,24 @@ void *map_domain_page(mfn_t mfn) } /* Release a mapping taken with map_domain_page() */ -void unmap_domain_page(const void *va) +void unmap_domain_page(const void *ptr) { + unsigned long va = (unsigned long)ptr; unsigned long flags; lpae_t *map = this_cpu(xen_dommap); - int slot = ((unsigned long) va - DOMHEAP_VIRT_START) >> SECOND_SHIFT; + unsigned int slot; - if ( !va ) + /* + * map_domain_page() may not have mapped anything if the address + * is part of the directmap. So ignore anything outside of the + * domheap. + */ + if ( (va < DOMHEAP_VIRT_START) || + ((va - DOMHEAP_VIRT_START) >= DOMHEAP_VIRT_SIZE) ) return; + slot = (va - DOMHEAP_VIRT_START) >> SECOND_SHIFT; + local_irq_save(flags); ASSERT(slot >= 0 && slot < DOMHEAP_ENTRIES); diff --git a/xen/arch/arm/include/asm/config.h b/xen/arch/arm/include/asm/config.h index 0fefed1b8aa9..12b7f1f1b9ea 100644 --- a/xen/arch/arm/include/asm/config.h +++ b/xen/arch/arm/include/asm/config.h @@ -156,6 +156,13 @@ #define FRAMETABLE_SIZE GB(32) #define FRAMETABLE_NR (FRAMETABLE_SIZE / sizeof(*frame_table)) +#define DOMHEAP_VIRT_START SLOT0(255) +#define DOMHEAP_VIRT_SIZE GB(2) + +#define DOMHEAP_ENTRIES 1024 /* 1024 2MB mapping slots */ +/* Number of domheap pagetable pages required at the second level (2MB mappings) */ +#define DOMHEAP_SECOND_PAGES (DOMHEAP_VIRT_SIZE >> FIRST_SHIFT) + #define DIRECTMAP_VIRT_START SLOT0(256) #define DIRECTMAP_SIZE (SLOT0_ENTRY_SIZE * (265-256)) #define DIRECTMAP_VIRT_END (DIRECTMAP_VIRT_START + DIRECTMAP_SIZE - 1) diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index 7a2c775f9562..d73abf1bf763 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -416,6 +416,11 @@ static inline bool arch_has_directmap(void) return true; } +/* Helpers to allocate, map and unmap a Xen page-table */ +int create_xen_table(lpae_t *entry); +lpae_t *xen_map_table(mfn_t mfn); +void xen_unmap_table(const lpae_t *table); + #endif /* __ARCH_ARM_MM__ */ /* * Local variables: diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index 2af751af9003..f5fb957554a5 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -177,7 +177,7 @@ static void __init __maybe_unused build_assertions(void) #undef CHECK_SAME_SLOT } -static lpae_t *xen_map_table(mfn_t mfn) +lpae_t *xen_map_table(mfn_t mfn) { /* * During early boot, map_domain_page() may be unusable. Use the @@ -189,7 +189,7 @@ static lpae_t *xen_map_table(mfn_t mfn) return map_domain_page(mfn); } -static void xen_unmap_table(const lpae_t *table) +void xen_unmap_table(const lpae_t *table) { /* * During early boot, xen_map_table() will not use map_domain_page() @@ -699,7 +699,7 @@ void *ioremap(paddr_t pa, size_t len) return ioremap_attr(pa, len, PAGE_HYPERVISOR_NOCACHE); } -static int create_xen_table(lpae_t *entry) +int create_xen_table(lpae_t *entry) { mfn_t mfn; void *p; diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index 88d9d90fb5ad..b1a8f91bb385 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -923,6 +923,10 @@ static void __init setup_mm(void) */ populate_boot_allocator(); + if ( !init_domheap_mappings(smp_processor_id()) ) + panic("CPU%u: Unable to prepare the domheap page-tables\n", + smp_processor_id()); + total_pages = 0; for ( i = 0; i < banks->nr_banks; i++ ) From patchwork Fri Dec 16 11:48:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13074977 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 45703C3DA7C for ; Fri, 16 Dec 2022 12:17:55 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.464565.723001 (Exim 4.92) (envelope-from ) id 1p69em-0007UE-7u; Fri, 16 Dec 2022 12:17:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 464565.723001; Fri, 16 Dec 2022 12:17:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69em-0007S5-2a; Fri, 16 Dec 2022 12:17:40 +0000 Received: by outflank-mailman (input) for mailman id 464565; Fri, 16 Dec 2022 12:17:37 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-0007LG-NZ for xen-devel@lists.xenproject.org; Fri, 16 Dec 2022 12:17:37 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p69ej-00033s-Em; Fri, 16 Dec 2022 12:17:37 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1p69DR-0004sN-Cv; Fri, 16 Dec 2022 11:49:25 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=dh6R0xvYO+Uecym5XhNbTlIrTn3W2pIvTvhO9tTqCTc=; b=Mmd0gMpbkfRWybrVgpKrSIOnad zRsL/sM8qC+NAsevmoXMv4/2NxlERhkQFHWlVllESfoPqg7z2K9bG8KBFDlSDTw7azkRp2u5bL9Rq rfmwu//83oRrJKri8ugLzZNwEavEct/vhRf/9a9LciBMt4qkp5XqzefhDMA0CyKlXYHQ=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: julien@xen.org, Julien Grall , Andrew Cooper , George Dunlap , Jan Beulich , Stefano Stabellini , Wei Liu , Bertrand Marquis , Volodymyr Babchuk Subject: [PATCH 22/22] xen/arm64: Allow the admin to enable/disable the directmap Date: Fri, 16 Dec 2022 11:48:53 +0000 Message-Id: <20221216114853.8227-23-julien@xen.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221216114853.8227-1-julien@xen.org> References: <20221216114853.8227-1-julien@xen.org> MIME-Version: 1.0 From: Julien Grall Implement the same command line option as x86 to enable/disable the directmap. By default this is kept enabled. Also modify setup_directmap_mappings() to populate the L0 entries related to the directmap area. Signed-off-by: Julien Grall ---- This patch is in an RFC state we need to decide what to do for arm32. Also, this is moving code that was introduced in this series. So this will need to be fix in the next version (assuming Arm64 will be ready). This was sent early as PoC to enable secret-free hypervisor on Arm64. Reviewed-by: Henry Wang --- docs/misc/xen-command-line.pandoc | 2 +- xen/arch/arm/include/asm/arm64/mm.h | 2 +- xen/arch/arm/include/asm/mm.h | 12 +++++---- xen/arch/arm/mm.c | 40 +++++++++++++++++++++++++++-- xen/arch/arm/setup.c | 1 + 5 files changed, 48 insertions(+), 9 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index a63e4612acac..948035286acc 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -760,7 +760,7 @@ Specify the size of the console debug trace buffer. By specifying `cpu:` additionally a trace buffer of the specified size is allocated per cpu. The debug trace feature is only enabled in debugging builds of Xen. -### directmap (x86) +### directmap (arm64, x86) > `= ` > Default: `true` diff --git a/xen/arch/arm/include/asm/arm64/mm.h b/xen/arch/arm/include/asm/arm64/mm.h index aa2adac63189..8b5dcb091750 100644 --- a/xen/arch/arm/include/asm/arm64/mm.h +++ b/xen/arch/arm/include/asm/arm64/mm.h @@ -7,7 +7,7 @@ */ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr) { - return true; + return opt_directmap; } #endif /* __ARM_ARM64_MM_H__ */ diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index d73abf1bf763..ef9ad3b366e3 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -9,6 +9,13 @@ #include #include +extern bool opt_directmap; + +static inline bool arch_has_directmap(void) +{ + return opt_directmap; +} + #if defined(CONFIG_ARM_32) # include #elif defined(CONFIG_ARM_64) @@ -411,11 +418,6 @@ static inline void page_set_xenheap_gfn(struct page_info *p, gfn_t gfn) } while ( (y = cmpxchg(&p->u.inuse.type_info, x, nx)) != x ); } -static inline bool arch_has_directmap(void) -{ - return true; -} - /* Helpers to allocate, map and unmap a Xen page-table */ int create_xen_table(lpae_t *entry); lpae_t *xen_map_table(mfn_t mfn); diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index f5fb957554a5..925d81c450e8 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -131,6 +132,12 @@ vaddr_t directmap_virt_start __read_mostly; unsigned long directmap_base_pdx __read_mostly; #endif +bool __ro_after_init opt_directmap = true; +/* TODO: Decide what to do for arm32. */ +#ifdef CONFIG_ARM_64 +boolean_param("directmap", opt_directmap); +#endif + unsigned long frametable_base_pdx __read_mostly; unsigned long frametable_virt_end __read_mostly; @@ -606,16 +613,27 @@ void __init setup_directmap_mappings(unsigned long base_mfn, directmap_virt_end = XENHEAP_VIRT_START + nr_mfns * PAGE_SIZE; } #else /* CONFIG_ARM_64 */ -/* Map the region in the directmap area. */ +/* + * This either populate a valid fdirect map, or allocates empty L1 tables + * and creates the L0 entries for the given region in the direct map + * depending on arch_has_directmap(). + * + * When directmap=no, we still need to populate empty L1 tables in the + * directmap region. The reason is that the root page-table (i.e. L0) + * is per-CPU and secondary CPUs will initialize their root page-table + * based on the pCPU0 one. So L0 entries will be shared if they are + * pre-populated. We also rely on the fact that L1 tables are never + * freed. + */ void __init setup_directmap_mappings(unsigned long base_mfn, unsigned long nr_mfns) { + unsigned long mfn_gb = base_mfn & ~((FIRST_SIZE >> PAGE_SHIFT) - 1); int rc; /* First call sets the directmap physical and virtual offset. */ if ( mfn_eq(directmap_mfn_start, INVALID_MFN) ) { - unsigned long mfn_gb = base_mfn & ~((FIRST_SIZE >> PAGE_SHIFT) - 1); directmap_mfn_start = _mfn(base_mfn); directmap_base_pdx = mfn_to_pdx(_mfn(base_mfn)); @@ -636,6 +654,24 @@ void __init setup_directmap_mappings(unsigned long base_mfn, panic("cannot add directmap mapping at %lx below heap start %lx\n", base_mfn, mfn_x(directmap_mfn_start)); + + if ( !arch_has_directmap() ) + { + vaddr_t vaddr = (vaddr_t)__mfn_to_virt(base_mfn); + unsigned int i, slot; + + slot = first_table_offset(vaddr); + nr_mfns += base_mfn - mfn_gb; + for ( i = 0; i < nr_mfns; i += BIT(XEN_PT_LEVEL_ORDER(0), UL), slot++ ) + { + lpae_t *entry = &cpu0_pgtable[slot]; + + if ( !lpae_is_valid(*entry) && !create_xen_table(entry) ) + panic("Unable to populate zeroeth slot %u\n", slot); + } + return; + } + rc = map_pages_to_xen((vaddr_t)__mfn_to_virt(base_mfn), _mfn(base_mfn), nr_mfns, PAGE_HYPERVISOR_RW | _PAGE_BLOCK); diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index b1a8f91bb385..83ded03c7b1f 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -1032,6 +1032,7 @@ void __init start_xen(unsigned long boot_phys_offset, cmdline_parse(cmdline); setup_mm(); + printk("Booting with directmap %s\n", arch_has_directmap() ? "on" : "off"); vm_init();