| Message ID | 20240116192611.41112-1-eliasely@amazon.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Remove the directmap | expand |
On 16.01.2024 20:25, Elias El Yandouzi wrote: > Elias El Yandouzi (3): > xen/x86: Add build assertion for fixmap entries > Rename mfn_to_virt() calls > Rename maddr_to_virt() calls > > Hongyan Xia (13): > acpi: vmap pages in acpi_os_alloc_memory > xen/numa: vmap the pages for memnodemap > x86/srat: vmap the pages for acpi_slit > x86: Map/unmap pages in restore_all_guests > x86/pv: Rewrite how building PV dom0 handles domheap mappings > x86/pv: Map L4 page table for shim domain > x86/mapcache: Initialise the mapcache for the idle domain > x86: Add a boot option to enable and disable the direct map > x86/domain_page: Remove the fast paths when mfn is not in the > directmap > xen/page_alloc: Add a path for xenheap when there is no direct map > x86/setup: Leave early boot slightly earlier > x86/setup: vmap heap nodes when they are outside the direct map > x86/setup: Do not create valid mappings when directmap=no > > Julien Grall (8): > xen/vmap: Check the page has been mapped in vm_init_type() > xen/vmap: Introduce vmap_size() and use it > xen/arm: fixmap: Rename the fixmap slots to follow the x86 convention Btw, if there was clear indication that e.g. this patch (fully ack-ed afaict) can go in ahead of earlier patches, I probably would have put it in already. Considering it sits half way through the series, I don't want to blindly chance it, though. Jan > xen/x86: Add support for the PMAP > xen/arm32: mm: Rename 'first' to 'root' in init_secondary_pagetables() > xen/arm64: mm: Use per-pCPU page-tables > xen/arm64: Implement a mapcache for arm64 > xen/arm64: Allow the admin to enable/disable the directmap > > Wei Liu (3): > x86/setup: Move vm_init() before acpi calls > x86/pv: Domheap pages should be mapped while relocating initrd > x86: Lift mapcache variable to the arch level
Hi Jan, On 29/01/2024 08:28, Jan Beulich wrote: > On 16.01.2024 20:25, Elias El Yandouzi wrote: >> Julien Grall (8): >> xen/vmap: Check the page has been mapped in vm_init_type() >> xen/vmap: Introduce vmap_size() and use it >> xen/arm: fixmap: Rename the fixmap slots to follow the x86 convention > > Btw, if there was clear indication that e.g. this patch (fully ack-ed > afaict) can go in ahead of earlier patches, I probably would have put > it in already. Considering it sits half way through the series, I don't > want to blindly chance it, though. > I just forgot to strip off those ack tags. The patch got approved quite a while ago and I thought it would be better to submit it again. That being said, moving it earlier in the series would still work. Elias
Hi, On 05/02/2024 11:11, Elias El Yandouzi wrote: > Hi Jan, > > On 29/01/2024 08:28, Jan Beulich wrote: > >> On 16.01.2024 20:25, Elias El Yandouzi wrote: >>> Julien Grall (8): >>> xen/vmap: Check the page has been mapped in vm_init_type() >>> xen/vmap: Introduce vmap_size() and use it >>> xen/arm: fixmap: Rename the fixmap slots to follow the x86 convention >> >> Btw, if there was clear indication that e.g. this patch (fully ack-ed >> afaict) can go in ahead of earlier patches, I probably would have put >> it in already. Considering it sits half way through the series, I don't >> want to blindly chance it, though. >> > > I just forgot to strip off those ack tags. The patch got approved quite > a while ago and I thought it would be better to submit it again. I spoke with Stefano on Matrix and he was fine with keeping the ack. I assume that Jan is also ok as he asked if we could commit. So I have committed the patch. Cheers,
On 16.01.2024 20:25, Elias El Yandouzi wrote: > Elias El Yandouzi (3): > xen/x86: Add build assertion for fixmap entries > Rename mfn_to_virt() calls > Rename maddr_to_virt() calls > > Hongyan Xia (13): > acpi: vmap pages in acpi_os_alloc_memory > xen/numa: vmap the pages for memnodemap > x86/srat: vmap the pages for acpi_slit > x86: Map/unmap pages in restore_all_guests > x86/pv: Rewrite how building PV dom0 handles domheap mappings > x86/pv: Map L4 page table for shim domain > x86/mapcache: Initialise the mapcache for the idle domain > x86: Add a boot option to enable and disable the direct map > x86/domain_page: Remove the fast paths when mfn is not in the > directmap > xen/page_alloc: Add a path for xenheap when there is no direct map > x86/setup: Leave early boot slightly earlier > x86/setup: vmap heap nodes when they are outside the direct map > x86/setup: Do not create valid mappings when directmap=no > > Julien Grall (8): > xen/vmap: Check the page has been mapped in vm_init_type() > xen/vmap: Introduce vmap_size() and use it > xen/arm: fixmap: Rename the fixmap slots to follow the x86 convention > xen/x86: Add support for the PMAP > xen/arm32: mm: Rename 'first' to 'root' in init_secondary_pagetables() > xen/arm64: mm: Use per-pCPU page-tables > xen/arm64: Implement a mapcache for arm64 > xen/arm64: Allow the admin to enable/disable the directmap > > Wei Liu (3): > x86/setup: Move vm_init() before acpi calls > x86/pv: Domheap pages should be mapped while relocating initrd > x86: Lift mapcache variable to the arch level Just to mention it explicitly: Before hearing back on the comments given up to patch 13 I'm not intending to put time into looking at the remaining patches in this series. Jan
Hi all, A few years ago, Wei Liu implemented a PoC to remove the directmap from Xen. The last version was sent by Hongyan Xia [1]. I will start with thanking both Wei and Hongyan for the initial work to upstream the feature. A lot of patches already went in and this is the last few patches missing to effectively enable the feature. === What is the directmap? === At the moment, on both arm64 and x86, most of the RAM is mapped in Xen address space. This means that domain memory is easily accessible in Xen. === Why do we want to remove the directmap? === (Summarizing my understanding of the previous discussion) Speculation attacks (like Spectre SP1) rely on loading piece of memory in the cache. If the region is not mapped then it can't be loaded. So removing reducing the amount of memory mapped in Xen will also reduce the surface attack. === What's the performance impact? === As the guest memory is not always mapped, then the cost of mapping will increase. I haven't done the numbers with this new version, but some measurement were provided in the previous version for x86. === Improvement possible === The known area to improve on x86 are: * Mapcache: There was a patch sent by Hongyan: https://lore.kernel.org/xen-devel/4058e92ce21627731c49b588a95809dc0affd83a.1581015491.git.hongyxia@amazon.com/ * EPT: At the moment an guest page-tabel walk requires about 20 map/unmap. This will have an very high impact on the performance. We need to decide whether keep the EPT always mapped is a problem The original series didn't have support for Arm64. But as there were some interest, I have provided a PoC. There are more extra work for Arm64: * The mapcache is quite simple. We would investigate the performance * The mapcache should be made compliant to the Arm Arm (this is now more critical). * We will likely have the same problem as for the EPT. * We have no support for merging table to a superpage, neither free empty page-tables. (See more below) === Implementation === The subject is probably a misnomer. The directmap is still present but the RAM is not mapped by default. Instead, the region will still be used to map pages allocate via alloc_xenheap_pages(). The advantage is the solution is simple (so IHMO good enough for been merged as a tech preview). The disadvantage is the page allocator is not trying to keep all the xenheap pages together. So we may end up to have an increase of page table usage. In the longer term, we should consider to remove the direct map completely and switch to vmap(). The main problem with this approach is it is frequent to use mfn_to_virt() in the code. So we would need to cache the mapping (maybe in the struct page_info). === Why arm32 is not covered? === On Arm32, the domheap and xenheap is always separated. So by design the guest memory is not mapped by default. At this stage, it seems unnecessary to have to map/unmap xenheap pages every time they are allocated. === Why not using a separate domheap and xenheap? === While a separate xenheap/domheap reduce the page-table usage (all xenheap pages are contiguous and could be always mapped), it is also currently less scalable because the split is fixed at boot time (XXX: Can this be dynamic?). === Future of secret-free hypervisor === There are some information in an e-mail from Andrew a few years ago: https://lore.kernel.org/xen-devel/e3219697-0759-39fc-2486-715cdec1ca9e@citrix.com/ Cheers, [1] https://lore.kernel.org/xen-devel/cover.1588278317.git.hongyxia@amazon.com/ *** BLURB HERE *** Elias El Yandouzi (3): xen/x86: Add build assertion for fixmap entries Rename mfn_to_virt() calls Rename maddr_to_virt() calls Hongyan Xia (13): acpi: vmap pages in acpi_os_alloc_memory xen/numa: vmap the pages for memnodemap x86/srat: vmap the pages for acpi_slit x86: Map/unmap pages in restore_all_guests x86/pv: Rewrite how building PV dom0 handles domheap mappings x86/pv: Map L4 page table for shim domain x86/mapcache: Initialise the mapcache for the idle domain x86: Add a boot option to enable and disable the direct map x86/domain_page: Remove the fast paths when mfn is not in the directmap xen/page_alloc: Add a path for xenheap when there is no direct map x86/setup: Leave early boot slightly earlier x86/setup: vmap heap nodes when they are outside the direct map x86/setup: Do not create valid mappings when directmap=no Julien Grall (8): xen/vmap: Check the page has been mapped in vm_init_type() xen/vmap: Introduce vmap_size() and use it xen/arm: fixmap: Rename the fixmap slots to follow the x86 convention xen/x86: Add support for the PMAP xen/arm32: mm: Rename 'first' to 'root' in init_secondary_pagetables() xen/arm64: mm: Use per-pCPU page-tables xen/arm64: Implement a mapcache for arm64 xen/arm64: Allow the admin to enable/disable the directmap Wei Liu (3): x86/setup: Move vm_init() before acpi calls x86/pv: Domheap pages should be mapped while relocating initrd x86: Lift mapcache variable to the arch level docs/misc/xen-command-line.pandoc | 12 ++ xen/arch/arm/Kconfig | 3 +- xen/arch/arm/acpi/lib.c | 18 +-- xen/arch/arm/arm64/mmu/mm.c | 45 +++++- xen/arch/arm/domain_page.c | 50 ++++++- xen/arch/arm/include/asm/arm32/mm.h | 8 -- xen/arch/arm/include/asm/arm64/mm.h | 7 +- xen/arch/arm/include/asm/domain_page.h | 13 ++ xen/arch/arm/include/asm/early_printk.h | 2 +- xen/arch/arm/include/asm/fixmap.h | 16 +-- xen/arch/arm/include/asm/mm.h | 9 ++ xen/arch/arm/include/asm/mmu/layout.h | 13 +- xen/arch/arm/include/asm/mmu/mm.h | 2 + xen/arch/arm/mm.c | 1 + xen/arch/arm/mmu/pt.c | 12 +- xen/arch/arm/mmu/setup.c | 33 ++--- xen/arch/arm/mmu/smpboot.c | 32 ++--- xen/arch/arm/setup.c | 6 +- xen/arch/x86/Kconfig | 2 + xen/arch/x86/dmi_scan.c | 4 +- xen/arch/x86/domain.c | 12 +- xen/arch/x86/domain_page.c | 74 +++++++--- xen/arch/x86/hvm/dom0_build.c | 4 +- xen/arch/x86/include/asm/config.h | 12 +- xen/arch/x86/include/asm/domain.h | 13 +- xen/arch/x86/include/asm/fixmap.h | 9 ++ .../x86/include/asm/mach-default/bios_ebda.h | 2 +- xen/arch/x86/include/asm/mm.h | 10 +- xen/arch/x86/include/asm/page.h | 8 +- xen/arch/x86/include/asm/pmap.h | 25 ++++ xen/arch/x86/include/asm/x86_64/page.h | 2 +- xen/arch/x86/mm.c | 18 ++- xen/arch/x86/mpparse.c | 2 +- xen/arch/x86/pv/dom0_build.c | 73 +++++++--- xen/arch/x86/pv/domain.c | 34 +++++ xen/arch/x86/setup.c | 129 +++++++++++++++--- xen/arch/x86/srat.c | 4 +- xen/arch/x86/tboot.c | 2 +- xen/arch/x86/x86_64/asm-offsets.c | 1 + xen/arch/x86/x86_64/entry.S | 8 ++ xen/arch/x86/x86_64/mm.c | 26 ++-- xen/common/Kconfig | 17 +++ xen/common/efi/boot.c | 23 ++-- xen/common/numa.c | 9 +- xen/common/page_alloc.c | 89 ++++++++++-- xen/common/pmap.c | 8 +- xen/common/trace.c | 8 +- xen/common/vmap.c | 62 ++++++--- xen/drivers/acpi/osl.c | 14 +- xen/include/xen/mm.h | 7 + xen/include/xen/vmap.h | 4 + 51 files changed, 753 insertions(+), 244 deletions(-) create mode 100644 xen/arch/arm/include/asm/domain_page.h create mode 100644 xen/arch/x86/include/asm/pmap.h