mbox series

[RFC,00/84] Remove direct map from Xen

Message ID cover.1569489002.git.hongyax@amazon.com (mailing list archive)
Headers show
Series Remove direct map from Xen | expand

Message

Xia, Hongyan Sept. 26, 2019, 9:45 a.m. UTC 
From: Hongyan Xia <hongyax@amazon.com>

Previously, Wei Liu has created the patch series to switch to
alloc/map/unmap/free APIs for Xen page tables so that Xen page tables
are mapped with map_domain_page(). This is the first step towards
removing the direct map. I implemented patches on top of Wei's to remove
the direct map completely.

In addition to Wei's, my patches:
- (Wei) implement a proper PMAP infrastructure for bootstrapping
- remove direct map code in map_domain_page() and use mapcache, use PMAP
 when mapcache not ready
- map/unmap xenheap allocations to the direct map region so that va<->pa
 macros still work
- install dummy mappings (no permissions, INVALID_MFN) instead of actual
 1:1 mappings during boot, so the direct map no longer exists
- fix many places where code incorrectly assumes a direct map

I have moderately tested the patches. I am able to boot into dom0,
create/destroy PV/HVM guests and run XTF tests. Personally, I find it
difficult to discover all places where it incorrectly assumes a direct
map, so reviews and further testing are more than welcome.

Hongyan Xia (24):
  x86/mm: fix leaks in map_xen_pagetable.
  x86/pmap: break the loop in pmap APIs.
  x86/domain_page: use PMAP when d/vcache is not ready.
  x86/domain_page: mapcache is no longer tied to pv.
  mapcache: initialise the mapcache even for the idle domain.
  x86: fix some wrong assumptions on direct map. Increase PMAP slots to
    8.
  x86/domain_page: remove direct map code and initialise idle mapcache.
  page_alloc: actually do the mapping and unmapping on xenheap.
  page_alloc: comments on (un)mapping pages in xenheap allocations.
  x86/smpboot: use xenheap pages for rpts in smpboot.
  x86/setup: start tearing down the direct map.
  acpi: don't assume an always-mapped direct map in acpi allocations.
  x86/mm: Move vmap_to_mfn() to mm.c and rename to virt_to_mfn_walk().
  x86/pv: refactor how building dom0 in PV handles domheap mappings.
  x86/mm: handle PSE early termination cases in virt_to_mfn_walk().
  x86/setup: also clear the permission bits in the dummy 1:1 mapping.
  x86: properly (un)map pages in restore_all_guests.
  Revert "x86/smpboot: use xenheap pages for rpts in smpboot."
  Don't assume bootmem_region_list is mapped. Also fix a double unmap
    bug.
  x86/setup: Install dummy 1:1 mappings for all mem passed to
    allocators.
  x86/mm: optimise and properly unmap pages in virt_to_mfn_walk().
  x86: deduplicate code a bit and fix an unmapping bug.
  x86/pmap: rewrite logic for locking.
  x86/pv: fix a couple of direct map assumptions in dom0 building.

Wei Liu (60):
  x86/mm: defer clearing page in virt_to_xen_lXe
  x86: move some xen mm function declarations
  x86: introduce a new set of APIs to manage Xen page tables
  x86/mm: introduce l{1,2}t local variables to map_pages_to_xen
  x86/mm: introduce l{1,2}t local variables to modify_xen_mappings
  x86/mm: map_pages_to_xen should have one exit path
  x86/mm: add an end_of_loop label in map_pages_to_xen
  x86/mm: make sure there is one exit path for modify_xen_mappings
  x86/mm: add an end_of_loop label in modify_xen_mappings
  x86/mm: change pl2e to l2t in virt_to_xen_l2e
  x86/mm: change pl1e to l1t in virt_to_xen_l1e
  x86/mm: change pl3e to l3t in virt_to_xen_l3e
  x86/mm: rewrite virt_to_xen_l3e
  x86/mm: rewrite xen_to_virt_l2e
  x86/mm: rewrite virt_to_xen_l1e
  x86/mm: switch to new APIs in map_pages_to_xen
  xxx fixup: avoid shadowing mfn
  x86/mm: drop lXe_to_lYe invocations in map_pages_to_xen
  x86/mm: switch to new APIs in modify_xen_mappings
  x86/mm: drop lXe_to_lYe invocations from modify_xen_mappings
  x86/mm: switch to new APIs in arch_init_memory
  x86_64/mm: introduce pl2e in paging_init
  x86_64/mm: switch to new APIs in paging_init
  x86_64/mm: drop l4e_to_l3e invocation from paging_init
  x86_64/mm.c: remove code that serves no purpose in setup_m2p_table
  x86_64/mm: introduce pl2e in setup_m2p_table
  x86_64/mm: switch to new APIs in setup_m2p_table
  x86_64/mm: drop lXe_to_lYe invocations from setup_m2p_table
  efi: use new page table APIs in copy_mapping
  efi: avoid using global variable in copy_mapping
  efi: use new page table APIs in efi_init_memory
  efi: add emacs block to boot.c
  efi: switch EFI L4 table to use new APIs
  x86/smpboot: add emacs block
  x86/smpboot: clone_mapping should have one exit path
  x86/smpboot: switch pl3e to use new APIs in clone_mapping
  x86/smpboot: switch pl2e to use new APIs in clone_mapping
  x86/smpboot: switch pl1e to use new APIs in clone_mapping
  x86/smpboot: drop lXe_to_lYe invocations from cleanup_cpu_root_pgt
  x86: switch root_pgt to mfn_t and use new APIs
  x86/shim: map and unmap page tables in replace_va_mapping
  x86_64/mm: map and unmap page tables in m2p_mapped
  x86_64/mm: map and unmap page tables in share_hotadd_m2p_table
  x86_64/mm: map and unmap page tables in destroy_compat_m2p_mapping
  x86_64/mm: map and unmap page tables in destroy_m2p_mapping
  x86_64/mm: map and unmap page tables in setup_compat_m2p_table
  x86_64/mm: map and unmap page tables in cleanup_frame_table
  x86_64/mm: map and unmap page tables in subarch_init_memory
  x86_64/mm: map and unmap page tables in subarch_memory_op
  x86/smpboot: remove lXe_to_lYe in cleanup_cpu_root_pgt
  x86/pv: properly map and unmap page tables in mark_pv_pt_pages_rdonly
  x86/pv: properly map and unmap page table in dom0_construct_pv
  x86: remove lXe_to_lYe in __start_xen
  x86/mm: drop old page table APIs
  x86: switch to use domheap page for page tables
  x86/mm: drop _new suffix for page table APIs
  x86: add Persistent Map (PMAP) infrastructure
  x86: lift domain mapcache to arch_domain
  x86: lift vcpu mapcache to arch_vcpu
  x86/pv: domheap pages should be mapped while relocating initrd

 xen/arch/x86/Makefile           |   1 +
 xen/arch/x86/domain.c           |  19 +-
 xen/arch/x86/domain_page.c      |  71 ++---
 xen/arch/x86/efi/runtime.h      |  12 +-
 xen/arch/x86/mm.c               | 549 ++++++++++++++++++++++++--------
 xen/arch/x86/pmap.c             | 121 +++++++
 xen/arch/x86/pv/dom0_build.c    | 107 +++++--
 xen/arch/x86/pv/domain.c        |   2 +-
 xen/arch/x86/pv/shim.c          |  20 +-
 xen/arch/x86/setup.c            |  28 +-
 xen/arch/x86/smpboot.c          | 171 +++++++---
 xen/arch/x86/x86_64/entry.S     |  32 +-
 xen/arch/x86/x86_64/mm.c        | 268 +++++++++++-----
 xen/common/efi/boot.c           |  84 +++--
 xen/common/efi/efi.h            |   3 +-
 xen/common/efi/runtime.c        |   8 +-
 xen/common/page_alloc.c         |  37 ++-
 xen/common/schedule.c           |   1 +
 xen/common/vmap.c               |   2 +-
 xen/drivers/acpi/osl.c          |  41 ++-
 xen/include/asm-arm/mm.h        |   2 -
 xen/include/asm-x86/domain.h    |  12 +-
 xen/include/asm-x86/fixmap.h    |   3 +
 xen/include/asm-x86/mm.h        |  20 ++
 xen/include/asm-x86/page.h      |  12 -
 xen/include/asm-x86/pmap.h      |  10 +
 xen/include/asm-x86/processor.h |   2 -
 27 files changed, 1211 insertions(+), 427 deletions(-)
 create mode 100644 xen/arch/x86/pmap.c
 create mode 100644 xen/include/asm-x86/pmap.h

Comments

Julien Grall Sept. 26, 2019, 10:21 a.m. UTC | #1 
Hi,

Thank you for posting the series.

On 9/26/19 10:45 AM, hongyax@amazon.com wrote:
> From: Hongyan Xia <hongyax@amazon.com>
> 
> Previously, Wei Liu has created the patch series to switch to
> alloc/map/unmap/free APIs for Xen page tables so that Xen page tables
> are mapped with map_domain_page(). This is the first step towards
> removing the direct map. I implemented patches on top of Wei's to remove
> the direct map completely.
> 
> In addition to Wei's, my patches:
> - (Wei) implement a proper PMAP infrastructure for bootstrapping
> - remove direct map code in map_domain_page() and use mapcache, use PMAP
>   when mapcache not ready
> - map/unmap xenheap allocations to the direct map region so that va<->pa
>   macros still work
> - install dummy mappings (no permissions, INVALID_MFN) instead of actual
>   1:1 mappings during boot, so the direct map no longer exists
> - fix many places where code incorrectly assumes a direct map

I am a bit confused. Are your patches fixing bug from Wei's series?

> 
> I have moderately tested the patches. I am able to boot into dom0,
> create/destroy PV/HVM guests and run XTF tests. Personally, I find it
> difficult to discover all places where it incorrectly assumes a direct
> map, so reviews and further testing are more than welcome.

Do you have a branch with this series applied?

Cheers,
Xia, Hongyan Sept. 26, 2019, 10:33 a.m. UTC | #2 
On 26/09/2019 11:21, Julien Grall wrote:
> Hi,
> 
> Thank you for posting the series.
> 
> On 9/26/19 10:45 AM, hongyax@amazon.com wrote:
>> From: Hongyan Xia <hongyax@amazon.com>
>>
>> Previously, Wei Liu has created the patch series to switch to
>> alloc/map/unmap/free APIs for Xen page tables so that Xen page tables
>> are mapped with map_domain_page(). This is the first step towards
>> removing the direct map. I implemented patches on top of Wei's to remove
>> the direct map completely.
>>
>> In addition to Wei's, my patches:
>> - (Wei) implement a proper PMAP infrastructure for bootstrapping
>> - remove direct map code in map_domain_page() and use mapcache, use PMAP
>>   when mapcache not ready
>> - map/unmap xenheap allocations to the direct map region so that va<->pa
>>   macros still work
>> - install dummy mappings (no permissions, INVALID_MFN) instead of actual
>>   1:1 mappings during boot, so the direct map no longer exists
>> - fix many places where code incorrectly assumes a direct map
> 
> I am a bit confused. Are your patches fixing bug from Wei's series?
> 

The goal is to remove the direct map completely from Xen. Wei's series has done 
the first step but was not complete. My additional patches finished the 
remaining steps and removed the direct map.

>>
>> I have moderately tested the patches. I am able to boot into dom0,
>> create/destroy PV/HVM guests and run XTF tests. Personally, I find it
>> difficult to discover all places where it incorrectly assumes a direct
>> map, so reviews and further testing are more than welcome.
> 
> Do you have a branch with this series applied?
> 

I am setting this up. Will send out the link to the branch soon.

> Cheers,
>
Julien Grall Sept. 26, 2019, 12:02 p.m. UTC | #3 
Hi,

On 9/26/19 11:33 AM, hongyax@amazon.com wrote:
> On 26/09/2019 11:21, Julien Grall wrote:
>> Hi,
>>
>> Thank you for posting the series.
>>
>> On 9/26/19 10:45 AM, hongyax@amazon.com wrote:
>>> From: Hongyan Xia <hongyax@amazon.com>
>>>
>>> Previously, Wei Liu has created the patch series to switch to
>>> alloc/map/unmap/free APIs for Xen page tables so that Xen page tables
>>> are mapped with map_domain_page(). This is the first step towards
>>> removing the direct map. I implemented patches on top of Wei's to remove
>>> the direct map completely.
>>>
>>> In addition to Wei's, my patches:
>>> - (Wei) implement a proper PMAP infrastructure for bootstrapping
>>> - remove direct map code in map_domain_page() and use mapcache, use PMAP
>>>   when mapcache not ready
>>> - map/unmap xenheap allocations to the direct map region so that va<->pa
>>>   macros still work
>>> - install dummy mappings (no permissions, INVALID_MFN) instead of actual
>>>   1:1 mappings during boot, so the direct map no longer exists
>>> - fix many places where code incorrectly assumes a direct map
>>
>> I am a bit confused. Are your patches fixing bug from Wei's series?
>>
> 
> The goal is to remove the direct map completely from Xen. Wei's series 
> has done the first step but was not complete. My additional patches 
> finished the remaining steps and removed the direct map.

So I understand this correctly, Wei's series has no bug in it and could 
be committed as is without breaking Xen. Am I correct?

The reason I am asking that if you have a few patches that say fix 
leak/bug. If they are bug in Wei's series, then they should be squashed 
in patches introducing them.

> 
>>>
>>> I have moderately tested the patches. I am able to boot into dom0,
>>> create/destroy PV/HVM guests and run XTF tests. Personally, I find it
>>> difficult to discover all places where it incorrectly assumes a direct
>>> map, so reviews and further testing are more than welcome.
>>
>> Do you have a branch with this series applied?
>>
> 
> I am setting this up. Will send out the link to the branch soon.

Thank you.

Cheers,
Xia, Hongyan Sept. 26, 2019, 4:06 p.m. UTC | #4 
On 26/09/2019 13:02, Julien Grall wrote:
> 
> So I understand this correctly, Wei's series has no bug in it and could be 
> committed as is without breaking Xen. Am I correct?
> 
> The reason I am asking that if you have a few patches that say fix leak/bug. If 
> they are bug in Wei's series, then they should be squashed in patches 
> introducing them.
> 

My additional patches fixed several bugs found in Wei's series. If those fixes 
are folded in, Wei's series can be committed without breaking Xen.

So, it might make sense to commit Wei's series first with the fixes, then my 
extra series to actually remove the direct map.

Hongyan