| Message ID | 20240306185032.103216-1-jason.andryuk@amd.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | x86/pvh: Support relocating dom0 kernel | expand |
On Wed, Mar 06, 2024 at 01:50:29PM -0500, Jason Andryuk wrote: > Xen tries to load a PVH dom0 kernel at the fixed guest physical address > from the elf headers. For Linux, this defaults to 0x1000000 (16MB), but > it can be configured. > > Unfortunately there exist firmwares that have reserved regions at this > address, so Xen fails to load the dom0 kernel since it's not RAM. > > The other issue is that the Linux PVH entry point is not > position-independent. It expects to run at the compiled > CONFIG_PHYSICAL_ADDRESS. > > This patch set expands the PVH dom0 builder to try to relocate the > kernel if needed and possible. XENFEAT_pvh_relocatable is added for > kernels to indicate they are relocatable. However, we may want to > switch to an additional ELF note with the kernel alignment. Linux > specifies a kernel alignment in the bzImage boot_params.setup_header, > but that is not present the ELF vmlinux file. I wonder whether we need a pair of notes, to signal the min/max addresses the kernel supports being relocated to. Thanks, Roger.
On 07.03.2024 11:00, Roger Pau Monné wrote: > On Wed, Mar 06, 2024 at 01:50:29PM -0500, Jason Andryuk wrote: >> Xen tries to load a PVH dom0 kernel at the fixed guest physical address >> from the elf headers. For Linux, this defaults to 0x1000000 (16MB), but >> it can be configured. >> >> Unfortunately there exist firmwares that have reserved regions at this >> address, so Xen fails to load the dom0 kernel since it's not RAM. >> >> The other issue is that the Linux PVH entry point is not >> position-independent. It expects to run at the compiled >> CONFIG_PHYSICAL_ADDRESS. >> >> This patch set expands the PVH dom0 builder to try to relocate the >> kernel if needed and possible. XENFEAT_pvh_relocatable is added for >> kernels to indicate they are relocatable. However, we may want to >> switch to an additional ELF note with the kernel alignment. Linux >> specifies a kernel alignment in the bzImage boot_params.setup_header, >> but that is not present the ELF vmlinux file. > > I wonder whether we need a pair of notes, to signal the min/max > addresses the kernel supports being relocated to. Plus, as per your other reply, a 3rd one to specify alignment needs. Then again - a single note can have multiple values. So perhaps it doesn't need to be more than one new notes (except if dealing with multi-value ones is deemed too complicated). Jan
On Thu, Mar 07, 2024 at 11:08:37AM +0100, Jan Beulich wrote: > On 07.03.2024 11:00, Roger Pau Monné wrote: > > On Wed, Mar 06, 2024 at 01:50:29PM -0500, Jason Andryuk wrote: > >> Xen tries to load a PVH dom0 kernel at the fixed guest physical address > >> from the elf headers. For Linux, this defaults to 0x1000000 (16MB), but > >> it can be configured. > >> > >> Unfortunately there exist firmwares that have reserved regions at this > >> address, so Xen fails to load the dom0 kernel since it's not RAM. > >> > >> The other issue is that the Linux PVH entry point is not > >> position-independent. It expects to run at the compiled > >> CONFIG_PHYSICAL_ADDRESS. > >> > >> This patch set expands the PVH dom0 builder to try to relocate the > >> kernel if needed and possible. XENFEAT_pvh_relocatable is added for > >> kernels to indicate they are relocatable. However, we may want to > >> switch to an additional ELF note with the kernel alignment. Linux > >> specifies a kernel alignment in the bzImage boot_params.setup_header, > >> but that is not present the ELF vmlinux file. > > > > I wonder whether we need a pair of notes, to signal the min/max > > addresses the kernel supports being relocated to. > > Plus, as per your other reply, a 3rd one to specify alignment needs. Alignment we could in theory get from the ELF program header, if OSes fill those reliably. FreeBSD seems to do so, haven't checked Linux. > Then again - a single note can have multiple values. So perhaps it > doesn't need to be more than one new notes (except if dealing with > multi-value ones is deemed too complicated). I've never dealt with a multi-value note, if that's not overly complicated I would be fine with it, otherwise multiple notes are OK IMO. Thanks, Roger.
On 2024-03-07 05:20, Roger Pau Monné wrote: > On Thu, Mar 07, 2024 at 11:08:37AM +0100, Jan Beulich wrote: >> On 07.03.2024 11:00, Roger Pau Monné wrote: >>> On Wed, Mar 06, 2024 at 01:50:29PM -0500, Jason Andryuk wrote: >>>> Xen tries to load a PVH dom0 kernel at the fixed guest physical address >>>> from the elf headers. For Linux, this defaults to 0x1000000 (16MB), but >>>> it can be configured. >>>> >>>> Unfortunately there exist firmwares that have reserved regions at this >>>> address, so Xen fails to load the dom0 kernel since it's not RAM. >>>> >>>> The other issue is that the Linux PVH entry point is not >>>> position-independent. It expects to run at the compiled >>>> CONFIG_PHYSICAL_ADDRESS. >>>> >>>> This patch set expands the PVH dom0 builder to try to relocate the >>>> kernel if needed and possible. XENFEAT_pvh_relocatable is added for >>>> kernels to indicate they are relocatable. However, we may want to >>>> switch to an additional ELF note with the kernel alignment. Linux >>>> specifies a kernel alignment in the bzImage boot_params.setup_header, >>>> but that is not present the ELF vmlinux file. >>> >>> I wonder whether we need a pair of notes, to signal the min/max >>> addresses the kernel supports being relocated to. >> >> Plus, as per your other reply, a 3rd one to specify alignment needs. > > Alignment we could in theory get from the ELF program header, if OSes > fill those reliably. FreeBSD seems to do so, haven't checked Linux. I will look into this more, but at first glance, I don't see a connection between Linux's CONFIG_PHYSICAL_ALIGN and the values in the elf headers. As a quick test, I set CONFIG_PHYSICAL_ALIGN=0x800000, but the elf align values are still 0x200000. The elf header values may be a suitable fallback though? >> Then again - a single note can have multiple values. So perhaps it >> doesn't need to be more than one new notes (except if dealing with >> multi-value ones is deemed too complicated). > > I've never dealt with a multi-value note, if that's not overly > complicated I would be fine with it, otherwise multiple notes are OK > IMO. It looks like a single note can be followed by arbitrary data, so putting three values in should be fine. PVH is defined with a 32bit entry point, so are 32bit min, max & align values acceptable, or should 64bit be specified? Linux uses _ASM_PTR for PHYS32_ENTRY so its size matches the bitness of the target (32 on 32/ 64 on 64). I guess I'll follow that unless I hear a preference otherwise. Regards, Jason
On 07.03.2024 18:33, Jason Andryuk wrote: > On 2024-03-07 05:20, Roger Pau Monné wrote: >> On Thu, Mar 07, 2024 at 11:08:37AM +0100, Jan Beulich wrote: >>> On 07.03.2024 11:00, Roger Pau Monné wrote: >>>> On Wed, Mar 06, 2024 at 01:50:29PM -0500, Jason Andryuk wrote: >>>>> Xen tries to load a PVH dom0 kernel at the fixed guest physical address >>>>> from the elf headers. For Linux, this defaults to 0x1000000 (16MB), but >>>>> it can be configured. >>>>> >>>>> Unfortunately there exist firmwares that have reserved regions at this >>>>> address, so Xen fails to load the dom0 kernel since it's not RAM. >>>>> >>>>> The other issue is that the Linux PVH entry point is not >>>>> position-independent. It expects to run at the compiled >>>>> CONFIG_PHYSICAL_ADDRESS. >>>>> >>>>> This patch set expands the PVH dom0 builder to try to relocate the >>>>> kernel if needed and possible. XENFEAT_pvh_relocatable is added for >>>>> kernels to indicate they are relocatable. However, we may want to >>>>> switch to an additional ELF note with the kernel alignment. Linux >>>>> specifies a kernel alignment in the bzImage boot_params.setup_header, >>>>> but that is not present the ELF vmlinux file. >>>> >>>> I wonder whether we need a pair of notes, to signal the min/max >>>> addresses the kernel supports being relocated to. >>> >>> Plus, as per your other reply, a 3rd one to specify alignment needs. >> >> Alignment we could in theory get from the ELF program header, if OSes >> fill those reliably. FreeBSD seems to do so, haven't checked Linux. > > I will look into this more, but at first glance, I don't see a > connection between Linux's CONFIG_PHYSICAL_ALIGN and the values in the > elf headers. As a quick test, I set CONFIG_PHYSICAL_ALIGN=0x800000, but > the elf align values are still 0x200000. > > The elf header values may be a suitable fallback though? Imo, given the above, explicit values should be required. Better not load a kernel than doing so and then getting hard to debug crashes. Jan