| Message ID | 573B3484.1080603@citrix.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 05/17/2016 09:11 AM, David Vrabel wrote: > On 11/05/16 11:16, David Vrabel wrote: >> Why don't we get the RW bits correct when making the pteval when we >> already have the pfn, instead trying to fix it up afterwards. > Kevin, can you try this patch. Yes :D. The patch is working fine. I only got this warning while compiling: WARNING: arch/x86/xen/built-in.o(.text+0x257d): Section mismatch in reference from the variable __raw_callee_save_xen_make_pte_init to the function .init.text:xen_make_pte_init() The function __raw_callee_save_xen_make_pte_init() references the function __init xen_make_pte_init(). This is often because __raw_callee_save_xen_make_pte_init lacks a __init annotation or the annotation of xen_make_pte_init is wrong. > > David > > 8<----------------- > x86/xen: avoid m2p lookup when setting early page table entries > > When page tables entries are set using xen_set_pte_init() during early > boot there is no page fault handler that could handle a fault when > performing an M2P lookup. > > In 64 guest (usually dom0) early_ioremap() would fault in > xen_set_pte_init() because an M2P lookup faults because the MFN is in > MMIO space and not mapped in the M2P. This lookup is done to see if > the PFN in in the range used for the initial page table pages, so that > the PTE may be set as read-only. > > The M2P lookup can be avoided by moving the check (and clear of RW) > earlier when the PFN is still available. > > [ Not entirely happy with this as the 32/64 bit paths diverge even > more. Is there some way to unify them instead? ] > > Signed-off-by: David Vrabel <david.vrabel@citrix.com> > --- > arch/x86/xen/mmu.c | 28 +++++++++++++++++++++------- > 1 file changed, 21 insertions(+), 7 deletions(-) > > diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c > index 478a2de..897fad4 100644 > --- a/arch/x86/xen/mmu.c > +++ b/arch/x86/xen/mmu.c > @@ -1562,7 +1562,7 @@ static pte_t __init mask_rw_pte(pte_t *ptep, pte_t > pte) > return pte; > } > #else /* CONFIG_X86_64 */ > -static pte_t __init mask_rw_pte(pte_t *ptep, pte_t pte) > +static pteval_t __init mask_rw_pte(pteval_t pte) > { > unsigned long pfn; > > @@ -1577,10 +1577,10 @@ static pte_t __init mask_rw_pte(pte_t *ptep, > pte_t pte) > * page tables for mapping the p2m list, too, and page tables MUST be > * mapped read-only. > */ > - pfn = pte_pfn(pte); > + pfn = (pte & PTE_PFN_MASK) >> PAGE_SHIFT; > if (pfn >= xen_start_info->first_p2m_pfn && > pfn < xen_start_info->first_p2m_pfn + xen_start_info->nr_p2m_frames) > - pte = __pte_ma(pte_val_ma(pte) & ~_PAGE_RW); > + pte &= ~_PAGE_RW; > > return pte; > } > @@ -1600,13 +1600,26 @@ static pte_t __init mask_rw_pte(pte_t *ptep, > pte_t pte) > * so always write the PTE directly and rely on Xen trapping and > * emulating any updates as necessary. > */ > +__visible __init pte_t xen_make_pte_init(pteval_t pte) > +{ > +#ifdef CONFIG_X86_64 > + pte = mask_rw_pte(pte); > +#endif > + pte = pte_pfn_to_mfn(pte); > + > + if ((pte & PTE_PFN_MASK) >> PAGE_SHIFT == INVALID_P2M_ENTRY) > + pte = 0; > + > + return native_make_pte(pte); > +} > +PV_CALLEE_SAVE_REGS_THUNK(xen_make_pte_init); > + > static void __init xen_set_pte_init(pte_t *ptep, pte_t pte) > { > +#ifdef CONFIG_X86_32 > if (pte_mfn(pte) != INVALID_P2M_ENTRY) > pte = mask_rw_pte(ptep, pte); > - else > - pte = __pte_ma(0); > - > +#endif > native_set_pte(ptep, pte); > } > > @@ -2407,6 +2420,7 @@ static void __init xen_post_allocator_init(void) > pv_mmu_ops.alloc_pud = xen_alloc_pud; > pv_mmu_ops.release_pud = xen_release_pud; > #endif > + pv_mmu_ops.make_pte = PV_CALLEE_SAVE(xen_make_pte); > > #ifdef CONFIG_X86_64 > pv_mmu_ops.write_cr3 = &xen_write_cr3; > @@ -2455,7 +2469,7 @@ static const struct pv_mmu_ops xen_mmu_ops > __initconst = { > .pte_val = PV_CALLEE_SAVE(xen_pte_val), > .pgd_val = PV_CALLEE_SAVE(xen_pgd_val), > > - .make_pte = PV_CALLEE_SAVE(xen_make_pte), > + .make_pte = PV_CALLEE_SAVE(xen_make_pte_init), > .make_pgd = PV_CALLEE_SAVE(xen_make_pgd), > > #ifdef CONFIG_X86_PAE
On 17/05/16 16:11, David Vrabel wrote: > On 11/05/16 11:16, David Vrabel wrote: >> >> Why don't we get the RW bits correct when making the pteval when we >> already have the pfn, instead trying to fix it up afterwards. > > Kevin, can you try this patch. > > David > > 8<----------------- > x86/xen: avoid m2p lookup when setting early page table entries > > When page tables entries are set using xen_set_pte_init() during early > boot there is no page fault handler that could handle a fault when > performing an M2P lookup. > > In 64 guest (usually dom0) early_ioremap() would fault in > xen_set_pte_init() because an M2P lookup faults because the MFN is in > MMIO space and not mapped in the M2P. This lookup is done to see if > the PFN in in the range used for the initial page table pages, so that > the PTE may be set as read-only. > > The M2P lookup can be avoided by moving the check (and clear of RW) > earlier when the PFN is still available. > > [ Not entirely happy with this as the 32/64 bit paths diverge even > more. Is there some way to unify them instead? ] Boris, Juergen, any opinion on this patch? David > --- a/arch/x86/xen/mmu.c > +++ b/arch/x86/xen/mmu.c > @@ -1562,7 +1562,7 @@ static pte_t __init mask_rw_pte(pte_t *ptep, pte_t > pte) > return pte; > } > #else /* CONFIG_X86_64 */ > -static pte_t __init mask_rw_pte(pte_t *ptep, pte_t pte) > +static pteval_t __init mask_rw_pte(pteval_t pte) > { > unsigned long pfn; > > @@ -1577,10 +1577,10 @@ static pte_t __init mask_rw_pte(pte_t *ptep, > pte_t pte) > * page tables for mapping the p2m list, too, and page tables MUST be > * mapped read-only. > */ > - pfn = pte_pfn(pte); > + pfn = (pte & PTE_PFN_MASK) >> PAGE_SHIFT; > if (pfn >= xen_start_info->first_p2m_pfn && > pfn < xen_start_info->first_p2m_pfn + xen_start_info->nr_p2m_frames) > - pte = __pte_ma(pte_val_ma(pte) & ~_PAGE_RW); > + pte &= ~_PAGE_RW; > > return pte; > } > @@ -1600,13 +1600,26 @@ static pte_t __init mask_rw_pte(pte_t *ptep, > pte_t pte) > * so always write the PTE directly and rely on Xen trapping and > * emulating any updates as necessary. > */ > +__visible __init pte_t xen_make_pte_init(pteval_t pte) > +{ > +#ifdef CONFIG_X86_64 > + pte = mask_rw_pte(pte); > +#endif > + pte = pte_pfn_to_mfn(pte); > + > + if ((pte & PTE_PFN_MASK) >> PAGE_SHIFT == INVALID_P2M_ENTRY) > + pte = 0; > + > + return native_make_pte(pte); > +} > +PV_CALLEE_SAVE_REGS_THUNK(xen_make_pte_init); > + > static void __init xen_set_pte_init(pte_t *ptep, pte_t pte) > { > +#ifdef CONFIG_X86_32 > if (pte_mfn(pte) != INVALID_P2M_ENTRY) > pte = mask_rw_pte(ptep, pte); > - else > - pte = __pte_ma(0); > - > +#endif > native_set_pte(ptep, pte); > } > > @@ -2407,6 +2420,7 @@ static void __init xen_post_allocator_init(void) > pv_mmu_ops.alloc_pud = xen_alloc_pud; > pv_mmu_ops.release_pud = xen_release_pud; > #endif > + pv_mmu_ops.make_pte = PV_CALLEE_SAVE(xen_make_pte); > > #ifdef CONFIG_X86_64 > pv_mmu_ops.write_cr3 = &xen_write_cr3; > @@ -2455,7 +2469,7 @@ static const struct pv_mmu_ops xen_mmu_ops > __initconst = { > .pte_val = PV_CALLEE_SAVE(xen_pte_val), > .pgd_val = PV_CALLEE_SAVE(xen_pgd_val), > > - .make_pte = PV_CALLEE_SAVE(xen_make_pte), > + .make_pte = PV_CALLEE_SAVE(xen_make_pte_init), > .make_pgd = PV_CALLEE_SAVE(xen_make_pgd), > > #ifdef CONFIG_X86_PAE >
On 05/26/2016 06:24 AM, David Vrabel wrote: >> @@ -1577,10 +1577,10 @@ static pte_t __init mask_rw_pte(pte_t *ptep, >> pte_t pte) >> * page tables for mapping the p2m list, too, and page tables MUST be >> * mapped read-only. >> */ >> - pfn = pte_pfn(pte); >> + pfn = (pte & PTE_PFN_MASK) >> PAGE_SHIFT; >> if (pfn >= xen_start_info->first_p2m_pfn && >> pfn < xen_start_info->first_p2m_pfn + xen_start_info->nr_p2m_frames) >> - pte = __pte_ma(pte_val_ma(pte) & ~_PAGE_RW); >> + pte &= ~_PAGE_RW; >> >> return pte; >> } >> @@ -1600,13 +1600,26 @@ static pte_t __init mask_rw_pte(pte_t *ptep, >> pte_t pte) >> * so always write the PTE directly and rely on Xen trapping and >> * emulating any updates as necessary. >> */ >> +__visible __init pte_t xen_make_pte_init(pteval_t pte) >> +{ >> +#ifdef CONFIG_X86_64 >> + pte = mask_rw_pte(pte); >> +#endif Won't make_pte() be called on 32-bit as well? (And if yes then we can get rid of xen_set_pte_init()) (Also there were build warnings about xen_make_pte_init() being in wrong section because PV_CALLEE_SAVE is not __init). -boris >> + pte = pte_pfn_to_mfn(pte); >> + >> + if ((pte & PTE_PFN_MASK) >> PAGE_SHIFT == INVALID_P2M_ENTRY) >> + pte = 0; >> + >> + return native_make_pte(pte); >> +} >> +PV_CALLEE_SAVE_REGS_THUNK(xen_make_pte_init); >> + >> static void __init xen_set_pte_init(pte_t *ptep, pte_t pte) >> { >> +#ifdef CONFIG_X86_32 >> if (pte_mfn(pte) != INVALID_P2M_ENTRY) >> pte = mask_rw_pte(ptep, pte); >> - else >> - pte = __pte_ma(0); >> - >> +#endif >> native_set_pte(ptep, pte); >> } >> >> @@ -2407,6 +2420,7 @@ static void __init xen_post_allocator_init(void) >> pv_mmu_ops.alloc_pud = xen_alloc_pud; >> pv_mmu_ops.release_pud = xen_release_pud; >> #endif >> + pv_mmu_ops.make_pte = PV_CALLEE_SAVE(xen_make_pte); >> >> #ifdef CONFIG_X86_64 >> pv_mmu_ops.write_cr3 = &xen_write_cr3; >> @@ -2455,7 +2469,7 @@ static const struct pv_mmu_ops xen_mmu_ops >> __initconst = { >> .pte_val = PV_CALLEE_SAVE(xen_pte_val), >> .pgd_val = PV_CALLEE_SAVE(xen_pgd_val), >> >> - .make_pte = PV_CALLEE_SAVE(xen_make_pte), >> + .make_pte = PV_CALLEE_SAVE(xen_make_pte_init), >> .make_pgd = PV_CALLEE_SAVE(xen_make_pgd), >> >> #ifdef CONFIG_X86_PAE >> > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
On 26/05/16 15:05, Boris Ostrovsky wrote: > On 05/26/2016 06:24 AM, David Vrabel wrote: >>> @@ -1577,10 +1577,10 @@ static pte_t __init mask_rw_pte(pte_t *ptep, >>> pte_t pte) >>> * page tables for mapping the p2m list, too, and page tables MUST be >>> * mapped read-only. >>> */ >>> - pfn = pte_pfn(pte); >>> + pfn = (pte & PTE_PFN_MASK) >> PAGE_SHIFT; >>> if (pfn >= xen_start_info->first_p2m_pfn && >>> pfn < xen_start_info->first_p2m_pfn + xen_start_info->nr_p2m_frames) >>> - pte = __pte_ma(pte_val_ma(pte) & ~_PAGE_RW); >>> + pte &= ~_PAGE_RW; >>> >>> return pte; >>> } >>> @@ -1600,13 +1600,26 @@ static pte_t __init mask_rw_pte(pte_t *ptep, >>> pte_t pte) >>> * so always write the PTE directly and rely on Xen trapping and >>> * emulating any updates as necessary. >>> */ >>> +__visible __init pte_t xen_make_pte_init(pteval_t pte) >>> +{ >>> +#ifdef CONFIG_X86_64 >>> + pte = mask_rw_pte(pte); >>> +#endif > > > Won't make_pte() be called on 32-bit as well? (And if yes then we can > get rid of xen_set_pte_init()) Yes, but the 32-bit check needs the pointer to the PTE to see if it is currently read-only, this isn't available in make_pte(). > (Also there were build warnings about xen_make_pte_init() being in wrong > section because PV_CALLEE_SAVE is not __init). I intent to fix this up before posting a v2. David
Hello. I hit probably the same error with released "XenServer 7.0". - I have Xen4.6.1 (commit d77bac5c064ffb9dbb5b89b55b89853f1b784ebf - update Xen version to 4.6.1) - XS7 (Dundee) beta3 (kernel-3.10.96-479.383024.x86_64.rpm) work OK - XS7 release (kernel-3.10.96-484.383030.x86_64.rpm) crash - patch does not work, arch/x86/xen/mmu.c is very old in 3.10 - Can someone verify error ? Thanks, Martin Cerveny Crash (kernel-3.10.96-479.383024.x86_64.rpm): about to get started... (XEN) d0v0: unhandled page fault (ec=0000) (XEN) Pagetable walk from ffff88010278b080: (XEN) L4[0x110] = 0000000439a0d067 0000000000001a0d (XEN) L3[0x004] = 0000000000000000 ffffffffffffffff (XEN) domain_crash_sync called from entry.S: fault at ffff82d08022b2c3 create_bounce_frame+0x12b/0x13a (XEN) Domain 0 (vcpu#0) crashed on cpu#0: (XEN) ----[ Xen-4.6.1-vgpu x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e033:[<ffffffff81005dea>] (XEN) RFLAGS: 0000000000000282 EM: 1 CONTEXT: pv guest (d0v0) (XEN) rax: ffff88010278b080 rbx: ffffffff81a10000 rcx: ffff880000000080 (XEN) rdx: 00003ffffffff000 rsi: ffffffff81a01de4 rdi: 000000043a95c067 (XEN) rbp: ffffffff81a01df8 rsp: ffffffff81a01da0 r8: 00003ffffffff000 (XEN) r9: ffff880000000000 r10: 0000000000000001 r11: 0000000000000001 (XEN) r12: ffffffff80000000 r13: ffffffff81a10000 r14: 0000000000000000 (XEN) r15: 0000000000000082 cr0: 000000008005003b cr4: 00000000001526e0 (XEN) cr3: 0000000439a0c000 cr2: ffff88010278b080 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e02b cs: e033 (XEN) Guest stack trace from rsp=ffffffff81a01da0: (XEN) ffff880000000080 0000000000000001 0000000000000000 ffffffff81005dea (XEN) 000000010000e030 0000000000010082 ffffffff81a01de0 000000000000e02b (XEN) 0000000181a10000 ffffffff81a10000 ffffffff80000000 ffffffff81a01e40 (XEN) ffffffff810067f6 0000000000000001 0000000000000001 ffffffff81a10000 (XEN) ffffffff80000000 ffffffff83d7a000 0000000000000000 ffffffff81dfffff (XEN) ffffffff81a01e78 ffffffff81aedf2d 000000000114b000 0000000001000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffffffff81a01ef0 (XEN) ffffffff81add76b 0000000000000000 0000000000000000 ffffffff81a01ef0 (XEN) ffffffff81a01f08 ffffffff00000010 ffffffff81a01f00 ffffffff81a01ec0 (XEN) 0000000000000000 ffffffffffffffff ffffffff81b69900 0000000000000000 (XEN) 0000000000000000 0000000000000000 ffffffff81a01f30 ffffffff81ad5bb9 (XEN) 0000000000000000 ffffffff81b732c0 ffffffff81a01f60 00000000ffffffff (XEN) 0000000000000000 0000000000000000 ffffffff81a01f40 ffffffff81ad55ee (XEN) ffffffff81a01ff8 ffffffff81ad8b48 000306e400000000 0000000100200800 (XEN) 0300000100000032 0000000000000005 0000000000000020 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0f00000060c0c748 ccccccccccccc305 cccccccccccccccc cccccccccccccccc (XEN) Hardware Dom0 crashed: rebooting machine in 5 seconds. (XEN) Resetting with ACPI MEMORY or I/O RESET_REG. On Thu, 26 May 2016, David Vrabel wrote: > On 17/05/16 16:11, David Vrabel wrote: >> On 11/05/16 11:16, David Vrabel wrote: >>> >>> Why don't we get the RW bits correct when making the pteval when we >>> already have the pfn, instead trying to fix it up afterwards. >> >> Kevin, can you try this patch. >> >> David >> >> 8<----------------- >> x86/xen: avoid m2p lookup when setting early page table entries >> >> When page tables entries are set using xen_set_pte_init() during early >> boot there is no page fault handler that could handle a fault when >> performing an M2P lookup. >> >> In 64 guest (usually dom0) early_ioremap() would fault in >> xen_set_pte_init() because an M2P lookup faults because the MFN is in >> MMIO space and not mapped in the M2P. This lookup is done to see if >> the PFN in in the range used for the initial page table pages, so that >> the PTE may be set as read-only. >> >> The M2P lookup can be avoided by moving the check (and clear of RW) >> earlier when the PFN is still available. >> >> [ Not entirely happy with this as the 32/64 bit paths diverge even >> more. Is there some way to unify them instead? ] > > Boris, Juergen, any opinion on this patch? > > David > >> --- a/arch/x86/xen/mmu.c >> +++ b/arch/x86/xen/mmu.c >> @@ -1562,7 +1562,7 @@ static pte_t __init mask_rw_pte(pte_t *ptep, pte_t >> pte) >> return pte; >> } >> #else /* CONFIG_X86_64 */ >> -static pte_t __init mask_rw_pte(pte_t *ptep, pte_t pte) >> +static pteval_t __init mask_rw_pte(pteval_t pte) >> { >> unsigned long pfn; >> >> @@ -1577,10 +1577,10 @@ static pte_t __init mask_rw_pte(pte_t *ptep, >> pte_t pte) >> * page tables for mapping the p2m list, too, and page tables MUST be >> * mapped read-only. >> */ >> - pfn = pte_pfn(pte); >> + pfn = (pte & PTE_PFN_MASK) >> PAGE_SHIFT; >> if (pfn >= xen_start_info->first_p2m_pfn && >> pfn < xen_start_info->first_p2m_pfn + xen_start_info->nr_p2m_frames) >> - pte = __pte_ma(pte_val_ma(pte) & ~_PAGE_RW); >> + pte &= ~_PAGE_RW; >> >> return pte; >> } >> @@ -1600,13 +1600,26 @@ static pte_t __init mask_rw_pte(pte_t *ptep, >> pte_t pte) >> * so always write the PTE directly and rely on Xen trapping and >> * emulating any updates as necessary. >> */ >> +__visible __init pte_t xen_make_pte_init(pteval_t pte) >> +{ >> +#ifdef CONFIG_X86_64 >> + pte = mask_rw_pte(pte); >> +#endif >> + pte = pte_pfn_to_mfn(pte); >> + >> + if ((pte & PTE_PFN_MASK) >> PAGE_SHIFT == INVALID_P2M_ENTRY) >> + pte = 0; >> + >> + return native_make_pte(pte); >> +} >> +PV_CALLEE_SAVE_REGS_THUNK(xen_make_pte_init); >> + >> static void __init xen_set_pte_init(pte_t *ptep, pte_t pte) >> { >> +#ifdef CONFIG_X86_32 >> if (pte_mfn(pte) != INVALID_P2M_ENTRY) >> pte = mask_rw_pte(ptep, pte); >> - else >> - pte = __pte_ma(0); >> - >> +#endif >> native_set_pte(ptep, pte); >> } >> >> @@ -2407,6 +2420,7 @@ static void __init xen_post_allocator_init(void) >> pv_mmu_ops.alloc_pud = xen_alloc_pud; >> pv_mmu_ops.release_pud = xen_release_pud; >> #endif >> + pv_mmu_ops.make_pte = PV_CALLEE_SAVE(xen_make_pte); >> >> #ifdef CONFIG_X86_64 >> pv_mmu_ops.write_cr3 = &xen_write_cr3; >> @@ -2455,7 +2469,7 @@ static const struct pv_mmu_ops xen_mmu_ops >> __initconst = { >> .pte_val = PV_CALLEE_SAVE(xen_pte_val), >> .pgd_val = PV_CALLEE_SAVE(xen_pgd_val), >> >> - .make_pte = PV_CALLEE_SAVE(xen_make_pte), >> + .make_pte = PV_CALLEE_SAVE(xen_make_pte_init), >> .make_pgd = PV_CALLEE_SAVE(xen_make_pgd), >> >> #ifdef CONFIG_X86_PAE >> > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >
:-( On Wed, 1 Jun 2016, Martin Cerveny wrote: > I hit probably the same error with released "XenServer 7.0". > - I have Xen4.6.1 (commit d77bac5c064ffb9dbb5b89b55b89853f1b784ebf - update > Xen version to 4.6.1) > - XS7 (Dundee) beta3 (kernel-3.10.96-479.383024.x86_64.rpm) work OK > - XS7 release (kernel-3.10.96-484.383030.x86_64.rpm) crash > - patch does not work, arch/x86/xen/mmu.c is very old in 3.10 > - Can someone verify error ? > > Thanks, Martin Cerveny > > Crash (kernel-3.10.96-479.383024.x86_64.rpm): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ correction: kernel-3.10.96-484.383030.x86_64.rpm > about to get started... > (XEN) d0v0: unhandled page fault (ec=0000) > (XEN) Pagetable walk from ffff88010278b080: > (XEN) L4[0x110] = 0000000439a0d067 0000000000001a0d > (XEN) L3[0x004] = 0000000000000000 ffffffffffffffff > (XEN) domain_crash_sync called from entry.S: fault at ffff82d08022b2c3 > create_bounce_frame+0x12b/0x13a > (XEN) Domain 0 (vcpu#0) crashed on cpu#0: > (XEN) ----[ Xen-4.6.1-vgpu x86_64 debug=n Not tainted ]---- > (XEN) CPU: 0 > (XEN) RIP: e033:[<ffffffff81005dea>] > (XEN) RFLAGS: 0000000000000282 EM: 1 CONTEXT: pv guest (d0v0) > (XEN) rax: ffff88010278b080 rbx: ffffffff81a10000 rcx: ffff880000000080 > (XEN) rdx: 00003ffffffff000 rsi: ffffffff81a01de4 rdi: 000000043a95c067 > (XEN) rbp: ffffffff81a01df8 rsp: ffffffff81a01da0 r8: 00003ffffffff000 > (XEN) r9: ffff880000000000 r10: 0000000000000001 r11: 0000000000000001 > (XEN) r12: ffffffff80000000 r13: ffffffff81a10000 r14: 0000000000000000 > (XEN) r15: 0000000000000082 cr0: 000000008005003b cr4: 00000000001526e0 > (XEN) cr3: 0000000439a0c000 cr2: ffff88010278b080 > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e02b cs: e033 > (XEN) Guest stack trace from rsp=ffffffff81a01da0: > (XEN) ffff880000000080 0000000000000001 0000000000000000 ffffffff81005dea > (XEN) 000000010000e030 0000000000010082 ffffffff81a01de0 000000000000e02b > (XEN) 0000000181a10000 ffffffff81a10000 ffffffff80000000 ffffffff81a01e40 > (XEN) ffffffff810067f6 0000000000000001 0000000000000001 ffffffff81a10000 > (XEN) ffffffff80000000 ffffffff83d7a000 0000000000000000 ffffffff81dfffff > (XEN) ffffffff81a01e78 ffffffff81aedf2d 000000000114b000 0000000001000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 ffffffff81a01ef0 > (XEN) ffffffff81add76b 0000000000000000 0000000000000000 ffffffff81a01ef0 > (XEN) ffffffff81a01f08 ffffffff00000010 ffffffff81a01f00 ffffffff81a01ec0 > (XEN) 0000000000000000 ffffffffffffffff ffffffff81b69900 0000000000000000 > (XEN) 0000000000000000 0000000000000000 ffffffff81a01f30 ffffffff81ad5bb9 > (XEN) 0000000000000000 ffffffff81b732c0 ffffffff81a01f60 00000000ffffffff > (XEN) 0000000000000000 0000000000000000 ffffffff81a01f40 ffffffff81ad55ee > (XEN) ffffffff81a01ff8 ffffffff81ad8b48 000306e400000000 0000000100200800 > (XEN) 0300000100000032 0000000000000005 0000000000000020 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 > (XEN) 0f00000060c0c748 ccccccccccccc305 cccccccccccccccc cccccccccccccccc > (XEN) Hardware Dom0 crashed: rebooting machine in 5 seconds. > (XEN) Resetting with ACPI MEMORY or I/O RESET_REG. > > > On Thu, 26 May 2016, David Vrabel wrote: > >> On 17/05/16 16:11, David Vrabel wrote: >>> On 11/05/16 11:16, David Vrabel wrote: >>>> >>>> Why don't we get the RW bits correct when making the pteval when we >>>> already have the pfn, instead trying to fix it up afterwards. >>> >>> Kevin, can you try this patch. >>> >>> David >>> >>> 8<----------------- >>> x86/xen: avoid m2p lookup when setting early page table entries >>> >>> When page tables entries are set using xen_set_pte_init() during early >>> boot there is no page fault handler that could handle a fault when >>> performing an M2P lookup. >>> >>> In 64 guest (usually dom0) early_ioremap() would fault in >>> xen_set_pte_init() because an M2P lookup faults because the MFN is in >>> MMIO space and not mapped in the M2P. This lookup is done to see if >>> the PFN in in the range used for the initial page table pages, so that >>> the PTE may be set as read-only. >>> >>> The M2P lookup can be avoided by moving the check (and clear of RW) >>> earlier when the PFN is still available. >>> >>> [ Not entirely happy with this as the 32/64 bit paths diverge even >>> more. Is there some way to unify them instead? ] >> >> Boris, Juergen, any opinion on this patch? >> >> David >> >>> --- a/arch/x86/xen/mmu.c >>> +++ b/arch/x86/xen/mmu.c >>> @@ -1562,7 +1562,7 @@ static pte_t __init mask_rw_pte(pte_t *ptep, pte_t >>> pte) >>> return pte; >>> } >>> #else /* CONFIG_X86_64 */ >>> -static pte_t __init mask_rw_pte(pte_t *ptep, pte_t pte) >>> +static pteval_t __init mask_rw_pte(pteval_t pte) >>> { >>> unsigned long pfn; >>> >>> @@ -1577,10 +1577,10 @@ static pte_t __init mask_rw_pte(pte_t *ptep, >>> pte_t pte) >>> * page tables for mapping the p2m list, too, and page tables MUST be >>> * mapped read-only. >>> */ >>> - pfn = pte_pfn(pte); >>> + pfn = (pte & PTE_PFN_MASK) >> PAGE_SHIFT; >>> if (pfn >= xen_start_info->first_p2m_pfn && >>> pfn < xen_start_info->first_p2m_pfn + >>> xen_start_info->nr_p2m_frames) >>> - pte = __pte_ma(pte_val_ma(pte) & ~_PAGE_RW); >>> + pte &= ~_PAGE_RW; >>> >>> return pte; >>> } >>> @@ -1600,13 +1600,26 @@ static pte_t __init mask_rw_pte(pte_t *ptep, >>> pte_t pte) >>> * so always write the PTE directly and rely on Xen trapping and >>> * emulating any updates as necessary. >>> */ >>> +__visible __init pte_t xen_make_pte_init(pteval_t pte) >>> +{ >>> +#ifdef CONFIG_X86_64 >>> + pte = mask_rw_pte(pte); >>> +#endif >>> + pte = pte_pfn_to_mfn(pte); >>> + >>> + if ((pte & PTE_PFN_MASK) >> PAGE_SHIFT == INVALID_P2M_ENTRY) >>> + pte = 0; >>> + >>> + return native_make_pte(pte); >>> +} >>> +PV_CALLEE_SAVE_REGS_THUNK(xen_make_pte_init); >>> + >>> static void __init xen_set_pte_init(pte_t *ptep, pte_t pte) >>> { >>> +#ifdef CONFIG_X86_32 >>> if (pte_mfn(pte) != INVALID_P2M_ENTRY) >>> pte = mask_rw_pte(ptep, pte); >>> - else >>> - pte = __pte_ma(0); >>> - >>> +#endif >>> native_set_pte(ptep, pte); >>> } >>> >>> @@ -2407,6 +2420,7 @@ static void __init xen_post_allocator_init(void) >>> pv_mmu_ops.alloc_pud = xen_alloc_pud; >>> pv_mmu_ops.release_pud = xen_release_pud; >>> #endif >>> + pv_mmu_ops.make_pte = PV_CALLEE_SAVE(xen_make_pte); >>> >>> #ifdef CONFIG_X86_64 >>> pv_mmu_ops.write_cr3 = &xen_write_cr3; >>> @@ -2455,7 +2469,7 @@ static const struct pv_mmu_ops xen_mmu_ops >>> __initconst = { >>> .pte_val = PV_CALLEE_SAVE(xen_pte_val), >>> .pgd_val = PV_CALLEE_SAVE(xen_pgd_val), >>> >>> - .make_pte = PV_CALLEE_SAVE(xen_make_pte), >>> + .make_pte = PV_CALLEE_SAVE(xen_make_pte_init), >>> .make_pgd = PV_CALLEE_SAVE(xen_make_pgd), >>> >>> #ifdef CONFIG_X86_PAE >>> >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> http://lists.xen.org/xen-devel >> > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >
On 06/01/2016 12:23 PM, Martin Cerveny wrote: > :-( > > On Wed, 1 Jun 2016, Martin Cerveny wrote: >> I hit probably the same error with released "XenServer 7.0". >> - I have Xen4.6.1 (commit d77bac5c064ffb9dbb5b89b55b89853f1b784ebf - >> update Xen version to 4.6.1) >> - XS7 (Dundee) beta3 (kernel-3.10.96-479.383024.x86_64.rpm) work OK >> - XS7 release (kernel-3.10.96-484.383030.x86_64.rpm) crash >> - patch does not work, arch/x86/xen/mmu.c is very old in 3.10 >> - Can someone verify error ? >> >> Thanks, Martin Cerveny >> >> Crash (kernel-3.10.96-479.383024.x86_64.rpm): > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > correction: kernel-3.10.96-484.383030.x86_64.rpm If you can provide vmlinux (better) or System.map we can probably see whether it's the same signature. -boris
Hello. On Wed, 1 Jun 2016, Boris Ostrovsky wrote: > On 06/01/2016 12:23 PM, Martin Cerveny wrote: >> :-( >> >> On Wed, 1 Jun 2016, Martin Cerveny wrote: >>> I hit probably the same error with released "XenServer 7.0". >>> - I have Xen4.6.1 (commit d77bac5c064ffb9dbb5b89b55b89853f1b784ebf - >>> update Xen version to 4.6.1) >>> - XS7 (Dundee) beta3 (kernel-3.10.96-479.383024.x86_64.rpm) work OK >>> - XS7 release (kernel-3.10.96-484.383030.x86_64.rpm) crash >>> - patch does not work, arch/x86/xen/mmu.c is very old in 3.10 >>> - Can someone verify error ? >>> >>> Thanks, Martin Cerveny >>> >>> Crash (kernel-3.10.96-479.383024.x86_64.rpm): >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> correction: kernel-3.10.96-484.383030.x86_64.rpm > If you can provide vmlinux (better) or System.map we can probably see > whether it's the same signature. http://xenserver.org/open-source-virtualization-download.html -> XenServer-7.0.0-main.iso or XenServer-7.0.0-binpkg.iso -> kernel-3.10.96-484.383030.x86_64.rpm -> System.map-3.10.0+10 vmlinuz-3.10.0+10 -> http://s000.tinyupload.com/index.php?file_id=30528714656973136220 Thanks for analyzing, Martin > -boris > > >
On 06/01/2016 05:01 PM, Martin Cerveny wrote: > Hello. > > On Wed, 1 Jun 2016, Boris Ostrovsky wrote: >> On 06/01/2016 12:23 PM, Martin Cerveny wrote: >>> :-( >>> >>> On Wed, 1 Jun 2016, Martin Cerveny wrote: >>>> I hit probably the same error with released "XenServer 7.0". >>>> - I have Xen4.6.1 (commit d77bac5c064ffb9dbb5b89b55b89853f1b784ebf - >>>> update Xen version to 4.6.1) >>>> - XS7 (Dundee) beta3 (kernel-3.10.96-479.383024.x86_64.rpm) work OK >>>> - XS7 release (kernel-3.10.96-484.383030.x86_64.rpm) crash >>>> - patch does not work, arch/x86/xen/mmu.c is very old in 3.10 >>>> - Can someone verify error ? >>>> >>>> Thanks, Martin Cerveny >>>> >>>> Crash (kernel-3.10.96-479.383024.x86_64.rpm): >>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>> correction: kernel-3.10.96-484.383030.x86_64.rpm >> If you can provide vmlinux (better) or System.map we can probably see >> whether it's the same signature. > > http://xenserver.org/open-source-virtualization-download.html > -> > XenServer-7.0.0-main.iso or XenServer-7.0.0-binpkg.iso > -> > kernel-3.10.96-484.383030.x86_64.rpm > -> > System.map-3.10.0+10 vmlinuz-3.10.0+10 > -> > http://s000.tinyupload.com/index.php?file_id=30528714656973136220 > > Thanks for analyzing, Martin This looks like a different problem, the stack is ... start_kernel cleanup_highmap xen_set_pmd_hyper arbitrary_virt_to_machine Can you reproduce this with a newer kernel? -boris
On Wed, 1 Jun 2016, Boris Ostrovsky wrote: > On 06/01/2016 05:01 PM, Martin Cerveny wrote: >> Hello. >> >> On Wed, 1 Jun 2016, Boris Ostrovsky wrote: >>> On 06/01/2016 12:23 PM, Martin Cerveny wrote: >>>> :-( >>>> >>>> On Wed, 1 Jun 2016, Martin Cerveny wrote: >>>>> I hit probably the same error with released "XenServer 7.0". >>>>> - I have Xen4.6.1 (commit d77bac5c064ffb9dbb5b89b55b89853f1b784ebf - >>>>> update Xen version to 4.6.1) >>>>> - XS7 (Dundee) beta3 (kernel-3.10.96-479.383024.x86_64.rpm) work OK >>>>> - XS7 release (kernel-3.10.96-484.383030.x86_64.rpm) crash >>>>> - patch does not work, arch/x86/xen/mmu.c is very old in 3.10 >>>>> - Can someone verify error ? >>>>> >>>>> Thanks, Martin Cerveny >>>>> >>>>> Crash (kernel-3.10.96-479.383024.x86_64.rpm): >>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>>> correction: kernel-3.10.96-484.383030.x86_64.rpm >>> If you can provide vmlinux (better) or System.map we can probably see >>> whether it's the same signature. >> >> http://xenserver.org/open-source-virtualization-download.html >> -> >> XenServer-7.0.0-main.iso or XenServer-7.0.0-binpkg.iso >> -> >> kernel-3.10.96-484.383030.x86_64.rpm >> -> >> System.map-3.10.0+10 vmlinuz-3.10.0+10 >> -> >> http://s000.tinyupload.com/index.php?file_id=30528714656973136220 >> >> Thanks for analyzing, Martin > > > This looks like a different problem, the stack is > ... > start_kernel > cleanup_highmap > xen_set_pmd_hyper > arbitrary_virt_to_machine > > Can you reproduce this with a newer kernel? Thanks for analysing. But there is no new kernel. XenServer7 has specially crafted Centos7 kernel ( https://github.com/xenserver/linux-3.x + https://github.com/xenserver/linux-3.x.pg ) and will not move to newer kernel. I must stay on this kernel because NVidia vgpu binary blob does not support newer and nvidia refuses to share sources to kernel bridge for vgpu ( https://gridforums.nvidia.com/default/topic/231/?comment=1920 ) I will stay on working XS7 beta3 kernel. Thanks, Martin Cerveny
On 01/06/16 17:12, Martin Cerveny wrote: > Hello. > > I hit probably the same error with released "XenServer 7.0". > - I have Xen4.6.1 (commit d77bac5c064ffb9dbb5b89b55b89853f1b784ebf - > update Xen version to 4.6.1) > - XS7 (Dundee) beta3 (kernel-3.10.96-479.383024.x86_64.rpm) work OK > - XS7 release (kernel-3.10.96-484.383030.x86_64.rpm) crash > - patch does not work, arch/x86/xen/mmu.c is very old in 3.10 > - Can someone verify error ? This list it not the correct place for XenServer support. See http://xenserver.org/discuss-virtualization.html for available options. David
On Thu, 2 Jun 2016, Martin Cerveny wrote: > > > On Wed, 1 Jun 2016, Boris Ostrovsky wrote: > >> On 06/01/2016 05:01 PM, Martin Cerveny wrote: >>> Hello. >>> >>> On Wed, 1 Jun 2016, Boris Ostrovsky wrote: >>>> On 06/01/2016 12:23 PM, Martin Cerveny wrote: >>>>> :-( >>>>> >>>>> On Wed, 1 Jun 2016, Martin Cerveny wrote: >>>>>> I hit probably the same error with released "XenServer 7.0". >>>>>> - I have Xen4.6.1 (commit d77bac5c064ffb9dbb5b89b55b89853f1b784ebf - >>>>>> update Xen version to 4.6.1) >>>>>> - XS7 (Dundee) beta3 (kernel-3.10.96-479.383024.x86_64.rpm) work OK >>>>>> - XS7 release (kernel-3.10.96-484.383030.x86_64.rpm) crash >>>>>> - patch does not work, arch/x86/xen/mmu.c is very old in 3.10 >>>>>> - Can someone verify error ? >>>>>> >>>>>> Thanks, Martin Cerveny >>>>>> >>>>>> Crash (kernel-3.10.96-479.383024.x86_64.rpm): >>>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>>>> correction: kernel-3.10.96-484.383030.x86_64.rpm >>>> If you can provide vmlinux (better) or System.map we can probably see >>>> whether it's the same signature. >>> >>> http://xenserver.org/open-source-virtualization-download.html >>> -> >>> XenServer-7.0.0-main.iso or XenServer-7.0.0-binpkg.iso >>> -> >>> kernel-3.10.96-484.383030.x86_64.rpm >>> -> >>> System.map-3.10.0+10 vmlinuz-3.10.0+10 >>> -> >>> http://s000.tinyupload.com/index.php?file_id=30528714656973136220 >>> >>> Thanks for analyzing, Martin >> >> >> This looks like a different problem, the stack is >> ... >> start_kernel >> cleanup_highmap >> xen_set_pmd_hyper >> arbitrary_virt_to_machine >> >> Can you reproduce this with a newer kernel? > > Thanks for analysing. > > But there is no new kernel. > > XenServer7 has specially crafted Centos7 kernel > ( https://github.com/xenserver/linux-3.x + > https://github.com/xenserver/linux-3.x.pg ) and will not move > to newer kernel. I must stay on this kernel because NVidia vgpu > binary blob does not support newer and nvidia refuses > to share sources to kernel bridge for vgpu ( > https://gridforums.nvidia.com/default/topic/231/?comment=1920 ) > I will stay on working XS7 beta3 kernel. > > Thanks, Martin Cerveny > Now I found the problem of my XS7 crash - surprisingly "crashkernel" xen parameter :-) I log error to XS https://bugs.xenserver.org/browse/XSO-554 Thanks for help, Martin Cerveny
diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c index 478a2de..897fad4 100644 --- a/arch/x86/xen/mmu.c +++ b/arch/x86/xen/mmu.c @@ -1562,7 +1562,7 @@ static pte_t __init mask_rw_pte(pte_t *ptep, pte_t pte) return pte; } #else /* CONFIG_X86_64 */ -static pte_t __init mask_rw_pte(pte_t *ptep, pte_t pte) +static pteval_t __init mask_rw_pte(pteval_t pte) { unsigned long pfn;