mbox series

[v1,0/4] mm: Implement ECC handling for pfn with no struct page

Message ID 20230920140210.12663-1-ankita@nvidia.com (mailing list archive)
Headers show
Series mm: Implement ECC handling for pfn with no struct page | expand

Message

Ankit Agrawal Sept. 20, 2023, 2:02 p.m. UTC 
From: Ankit Agrawal <ankita@nvidia.com>

The kernel MM currently handles ECC errors / poison only on memory page
backed by struct page. As part of [1], the nvgrace-gpu-vfio-pci module
maps the device memory to user VA (Qemu) using remap_pfn_range without
being added to the kernel. These pages are not backed by struct page.

Implement a new ECC handling for memory without struct pages. Kernel MM
expose registration APIs to allow modules that are managing the device
to register its memory region and a callback function. MM then tracks
such regions using interval tree.

The mechanism is largely similar to that of ECC on pfn with struct pages.
If there is an ECC error on a pfn, MM uses the registered memory failure
callback function to notify the module of the faulty PFN, so that the
module may take any required action. The pfn is then unmapped in Stage-2.
When the VM tries to access the page, it gets trapped in KVM, which calls
the vm ops fault function. If the module fault function returns
VM_FAULT_HWPOISON, KVM sends a BUS_MCEERR_AR to the usermode (Qemu) mapped
to the poisoned page.

Lastly, nvgrace-gpu-vfio-pci module make use of the new mechanism to get
poison handling support on the device memory.

Patch generated over v6.6-rc2 and with [1] applied. [1] is currently under
review.

[1] https://lore.kernel.org/all/20230915025415.6762-1-ankita@nvidia.com/

Ankit Agrawal (4):
  mm: handle poisoning of pfn without struct pages
  mm: Add poison error check in fixup_user_fault() for mapped pfn
  mm: Change ghes code to allow poison of non-struct pfn
  vfio/nvgpu: register device memory for poison handling

 drivers/acpi/apei/ghes.c            |  12 +--
 drivers/vfio/pci/nvgrace-gpu/main.c | 107 +++++++++++++++++++++-
 drivers/vfio/vfio.h                 |  11 ---
 drivers/vfio/vfio_main.c            |   3 +-
 include/linux/memory-failure.h      |  22 +++++
 include/linux/mm.h                  |   1 +
 include/linux/vfio.h                |  15 ++++
 include/ras/ras_event.h             |   1 +
 mm/Kconfig                          |   1 +
 mm/gup.c                            |   2 +-
 mm/memory-failure.c                 | 135 +++++++++++++++++++++++-----
 virt/kvm/kvm_main.c                 |   6 ++
 12 files changed, 270 insertions(+), 46 deletions(-)
 create mode 100644 include/linux/memory-failure.h

Comments

Andrew Morton Sept. 20, 2023, 4:02 p.m. UTC | #1 
On Wed, 20 Sep 2023 19:32:06 +0530 <ankita@nvidia.com> wrote:

> The kernel MM currently handles ECC errors / poison only on memory page
> backed by struct page. As part of [1], the nvgrace-gpu-vfio-pci module
> maps the device memory to user VA (Qemu) using remap_pfn_range without
> being added to the kernel. These pages are not backed by struct page.

Are you able to identify any other drivers which can (or will) use
this?  Or is it likely that this feature will only ever be for
nvgrace-gpu-vfio-pci?
Jason Gunthorpe Sept. 20, 2023, 4:04 p.m. UTC | #2 
On Wed, Sep 20, 2023 at 09:02:22AM -0700, Andrew Morton wrote:
> On Wed, 20 Sep 2023 19:32:06 +0530 <ankita@nvidia.com> wrote:
> 
> > The kernel MM currently handles ECC errors / poison only on memory page
> > backed by struct page. As part of [1], the nvgrace-gpu-vfio-pci module
> > maps the device memory to user VA (Qemu) using remap_pfn_range without
> > being added to the kernel. These pages are not backed by struct page.
> 
> Are you able to identify any other drivers which can (or will) use
> this?  Or is it likely that this feature will only ever be for
> nvgrace-gpu-vfio-pci?

I think a future vfio-cxl will have a similar desire at least.

Jason