| Message ID | 20200821022615.28596-3-song.bao.hua@hisilicon.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | make dma_alloc_coherent NUMA-aware by per-NUMA CMA | expand |
On Fri, Aug 21, 2020 at 02:26:15PM +1200, Barry Song wrote: > Right now, smmu is using dma_alloc_coherent() to get memory to save queues > and tables. Typically, on ARM64 server, there is a default CMA located at > node0, which could be far away from node2, node3 etc. > with this patch, smmu will get memory from local numa node to save command > queues and page tables. that means dma_unmap latency will be shrunk much. > Meanwhile, when iommu.passthrough is on, device drivers which call dma_ > alloc_coherent() will also get local memory and avoid the travel between > numa nodes. > > Cc: Christoph Hellwig <hch@lst.de> > Cc: Marek Szyprowski <m.szyprowski@samsung.com> > Cc: Will Deacon <will@kernel.org> > Cc: Robin Murphy <robin.murphy@arm.com> > Cc: Ganapatrao Kulkarni <ganapatrao.kulkarni@cavium.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Nicolas Saenz Julienne <nsaenzjulienne@suse.de> > Cc: Steve Capper <steve.capper@arm.com> > Cc: Andrew Morton <akpm@linux-foundation.org> > Cc: Mike Rapoport <rppt@linux.ibm.com> > Signed-off-by: Barry Song <song.bao.hua@hisilicon.com> > --- > -v6: rebase on top of 5.9-rc1 > > arch/arm64/mm/init.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c > index 481d22c32a2e..f1c75957ff3c 100644 > --- a/arch/arm64/mm/init.c > +++ b/arch/arm64/mm/init.c > @@ -429,6 +429,8 @@ void __init bootmem_init(void) > arm64_hugetlb_cma_reserve(); > #endif > > + dma_pernuma_cma_reserve(); I think will have to do for now, but I still wish that more of this was driven from the core code so that we don't have to worry about initialisation order and whether things are early/late enough on a per-arch basis. Acked-by: Will Deacon <will@kernel.org> Will
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 481d22c32a2e..f1c75957ff3c 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -429,6 +429,8 @@ void __init bootmem_init(void) arm64_hugetlb_cma_reserve(); #endif + dma_pernuma_cma_reserve(); + /* * sparse_init() tries to allocate memory from memblock, so must be * done after the fixed reservations
Right now, smmu is using dma_alloc_coherent() to get memory to save queues and tables. Typically, on ARM64 server, there is a default CMA located at node0, which could be far away from node2, node3 etc. with this patch, smmu will get memory from local numa node to save command queues and page tables. that means dma_unmap latency will be shrunk much. Meanwhile, when iommu.passthrough is on, device drivers which call dma_ alloc_coherent() will also get local memory and avoid the travel between numa nodes. Cc: Christoph Hellwig <hch@lst.de> Cc: Marek Szyprowski <m.szyprowski@samsung.com> Cc: Will Deacon <will@kernel.org> Cc: Robin Murphy <robin.murphy@arm.com> Cc: Ganapatrao Kulkarni <ganapatrao.kulkarni@cavium.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Nicolas Saenz Julienne <nsaenzjulienne@suse.de> Cc: Steve Capper <steve.capper@arm.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Mike Rapoport <rppt@linux.ibm.com> Signed-off-by: Barry Song <song.bao.hua@hisilicon.com> --- -v6: rebase on top of 5.9-rc1 arch/arm64/mm/init.c | 2 ++ 1 file changed, 2 insertions(+)