| Message ID | 230ea13ef8e9f576df849e1b03406184ca890ba8.1712642324.git.baruch@tkos.co.il (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | arm64: support DMA zone starting above 4GB | expand |
On Tue, Apr 09, 2024 at 09:17:55AM +0300, Baruch Siach wrote: > of_dma_get_max_cpu_address() returns the highest CPU address that > devices can use for DMA. The implicit assumption is that all CPU > addresses below that limit are suitable for DMA. However the > 'dma-ranges' property this code uses also encodes a lower limit for DMA > that is potentially non zero. > > Rename to of_dma_get_cpu_limits(), and extend to retrieve also the lower > limit for the same 'dma-ranges' property describing the high limit. I don't understand the reason for the lower limit. The way the Linux zones work is that ZONE_DMA always starts from the start of the RAM. It doesn't matter whether it's 0 or not, you'd not allocate below the start of RAM anyway. If you have a device that cannot use the bottom of the RAM, it is pretty broken and not supported by Linux. I think you added this limit before we tried to move away from zone_dma_bits to a non-power-of-two limit (zone_dma_limit). With the latter, we no longer need tricks with the lower limit, of_dma_get_max_cpu_address() should capture the smallest upper CPU address limit supported by all devices (and that's where ZONE_DMA should end).
Hi Catalin, On Tue, Jun 18 2024, Catalin Marinas wrote: > On Tue, Apr 09, 2024 at 09:17:55AM +0300, Baruch Siach wrote: >> of_dma_get_max_cpu_address() returns the highest CPU address that >> devices can use for DMA. The implicit assumption is that all CPU >> addresses below that limit are suitable for DMA. However the >> 'dma-ranges' property this code uses also encodes a lower limit for DMA >> that is potentially non zero. >> >> Rename to of_dma_get_cpu_limits(), and extend to retrieve also the lower >> limit for the same 'dma-ranges' property describing the high limit. > > I don't understand the reason for the lower limit. The way the Linux > zones work is that ZONE_DMA always starts from the start of the RAM. It > doesn't matter whether it's 0 or not, you'd not allocate below the start > of RAM anyway. If you have a device that cannot use the bottom of the > RAM, it is pretty broken and not supported by Linux. I won't argue with that assertion. My target system RAM happens to start at that the lower end of devices DMA zone, so I'm fine with skipping this patch. Just curious. What is the inherent limitation that prevents Linux from supporting DMA zone with lower limit above RAM start? Thanks, baruch
On Thu, Jul 25, 2024 at 02:49:01PM +0300, Baruch Siach wrote: > Hi Catalin, > > On Tue, Jun 18 2024, Catalin Marinas wrote: > > On Tue, Apr 09, 2024 at 09:17:55AM +0300, Baruch Siach wrote: > >> of_dma_get_max_cpu_address() returns the highest CPU address that > >> devices can use for DMA. The implicit assumption is that all CPU > >> addresses below that limit are suitable for DMA. However the > >> 'dma-ranges' property this code uses also encodes a lower limit for DMA > >> that is potentially non zero. > >> > >> Rename to of_dma_get_cpu_limits(), and extend to retrieve also the lower > >> limit for the same 'dma-ranges' property describing the high limit. > > > > I don't understand the reason for the lower limit. The way the Linux > > zones work is that ZONE_DMA always starts from the start of the RAM. It > > doesn't matter whether it's 0 or not, you'd not allocate below the start > > of RAM anyway. If you have a device that cannot use the bottom of the > > RAM, it is pretty broken and not supported by Linux. > > I won't argue with that assertion. My target system RAM happens to start > at that the lower end of devices DMA zone, so I'm fine with skipping > this patch. > > Just curious. What is the inherent limitation that prevents Linux from > supporting DMA zone with lower limit above RAM start? It's the way the zone allocation fallback mechanism works. Let's say a ZONE_DMA32 allocation fails, it falls back to ZONE_DMA and it's supposed to be compatible with the GFP_DMA32 request. If you have some other zone below ZONE_DMA, it should also be compatible with GFP_DMA allocations.
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 00508c69ca9e..77e942ca578b 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -128,7 +128,7 @@ static void __init zone_sizes_init(void) #ifdef CONFIG_ZONE_DMA acpi_zone_dma_limit = acpi_iort_dma_get_max_cpu_address(); - dt_zone_dma_limit = of_dma_get_max_cpu_address(NULL); + of_dma_get_cpu_limits(NULL, &dt_zone_dma_limit, NULL); zone_dma_limit = min(dt_zone_dma_limit, acpi_zone_dma_limit); arm64_dma_phys_limit = max_zone_phys(zone_dma_limit); max_zone_pfns[ZONE_DMA] = PFN_DOWN(arm64_dma_phys_limit); diff --git a/drivers/of/address.c b/drivers/of/address.c index ae46a3605904..ac009b3bb63b 100644 --- a/drivers/of/address.c +++ b/drivers/of/address.c @@ -964,21 +964,25 @@ int of_dma_get_range(struct device_node *np, const struct bus_dma_region **map) #endif /* CONFIG_HAS_DMA */ /** - * of_dma_get_max_cpu_address - Gets highest CPU address suitable for DMA + * of_dma_get_cpu_limits - Gets highest CPU address suitable for DMA * @np: The node to start searching from or NULL to start from the root + * @max: Pointer to high address limit or NULL if not needed + * @min: Pointer to low address limit or NULL if not needed * * Gets the highest CPU physical address that is addressable by all DMA masters - * in the sub-tree pointed by np, or the whole tree if NULL is passed. If no - * DMA constrained device is found, it returns PHYS_ADDR_MAX. + * in the sub-tree pointed by np, or the whole tree if @np in NULL. If no + * DMA constrained device is found, @*max is PHYS_ADDR_MAX, and @*low is 0. */ -phys_addr_t __init of_dma_get_max_cpu_address(struct device_node *np) +void __init of_dma_get_cpu_limits(struct device_node *np, + phys_addr_t *max, phys_addr_t *min) { phys_addr_t max_cpu_addr = PHYS_ADDR_MAX; struct of_range_parser parser; - phys_addr_t subtree_max_addr; + phys_addr_t min_cpu_addr = 0; struct device_node *child; struct of_range range; const __be32 *ranges; + u64 cpu_start = 0; u64 cpu_end = 0; int len; @@ -988,21 +992,33 @@ phys_addr_t __init of_dma_get_max_cpu_address(struct device_node *np) ranges = of_get_property(np, "dma-ranges", &len); if (ranges && len) { of_dma_range_parser_init(&parser, np); - for_each_of_range(&parser, &range) - if (range.cpu_addr + range.size > cpu_end) + for_each_of_range(&parser, &range) { + if (range.cpu_addr + range.size > cpu_end) { cpu_end = range.cpu_addr + range.size - 1; + cpu_start = range.cpu_addr; + } + } - if (max_cpu_addr > cpu_end) + if (max_cpu_addr > cpu_end) { max_cpu_addr = cpu_end; + min_cpu_addr = cpu_start; + } } for_each_available_child_of_node(np, child) { - subtree_max_addr = of_dma_get_max_cpu_address(child); - if (max_cpu_addr > subtree_max_addr) + phys_addr_t subtree_max_addr, subtree_min_addr; + + of_dma_get_cpu_limits(child, &subtree_max_addr, &subtree_min_addr); + if (max_cpu_addr > subtree_max_addr) { max_cpu_addr = subtree_max_addr; + min_cpu_addr = subtree_min_addr; + } } - return max_cpu_addr; + if (max) + *max = max_cpu_addr; + if (min) + *min = min_cpu_addr; } /** diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c index 6b5c36b6a758..2d632d4ec5b1 100644 --- a/drivers/of/unittest.c +++ b/drivers/of/unittest.c @@ -921,7 +921,7 @@ static void __init of_unittest_changeset(void) #endif } -static void __init of_unittest_dma_get_max_cpu_address(void) +static void __init of_unittest_dma_get_cpu_limits(void) { struct device_node *np; phys_addr_t cpu_addr; @@ -935,9 +935,9 @@ static void __init of_unittest_dma_get_max_cpu_address(void) return; } - cpu_addr = of_dma_get_max_cpu_address(np); + of_dma_get_cpu_limits(np, &cpu_addr, NULL); unittest(cpu_addr == 0x4fffffff, - "of_dma_get_max_cpu_address: wrong CPU addr %pad (expecting %x)\n", + "of_dma_get_cpu_limits: wrong CPU addr %pad (expecting %x)\n", &cpu_addr, 0x4fffffff); } @@ -4109,7 +4109,7 @@ static int __init of_unittest(void) of_unittest_changeset(); of_unittest_parse_interrupts(); of_unittest_parse_interrupts_extended(); - of_unittest_dma_get_max_cpu_address(); + of_unittest_dma_get_cpu_limits(); of_unittest_parse_dma_ranges(); of_unittest_pci_dma_ranges(); of_unittest_bus_ranges(); diff --git a/include/linux/of.h b/include/linux/of.h index a0bedd038a05..7756441d3ce0 100644 --- a/include/linux/of.h +++ b/include/linux/of.h @@ -454,7 +454,8 @@ int of_map_id(struct device_node *np, u32 id, const char *map_name, const char *map_mask_name, struct device_node **target, u32 *id_out); -phys_addr_t of_dma_get_max_cpu_address(struct device_node *np); +void of_dma_get_cpu_limits(struct device_node *np, phys_addr_t *max, + phys_addr_t *min); struct kimage; void *of_kexec_alloc_and_setup_fdt(const struct kimage *image, @@ -880,9 +881,13 @@ static inline int of_map_id(struct device_node *np, u32 id, return -EINVAL; } -static inline phys_addr_t of_dma_get_max_cpu_address(struct device_node *np) +static inline void of_dma_get_cpu_limits(struct device_node *np, + phys_addr_t *max, phys_addr_t *min) { - return PHYS_ADDR_MAX; + if (max) + *max = PHYS_ADDR_MAX; + if (min) + *min = 0; } static inline const void *of_device_get_match_data(const struct device *dev)
of_dma_get_max_cpu_address() returns the highest CPU address that devices can use for DMA. The implicit assumption is that all CPU addresses below that limit are suitable for DMA. However the 'dma-ranges' property this code uses also encodes a lower limit for DMA that is potentially non zero. Rename to of_dma_get_cpu_limits(), and extend to retrieve also the lower limit for the same 'dma-ranges' property describing the high limit. Update callers of of_dma_get_max_cpu_address(). No functional change intended. Signed-off-by: Baruch Siach <baruch@tkos.co.il> --- arch/arm64/mm/init.c | 2 +- drivers/of/address.c | 38 +++++++++++++++++++++++++++----------- drivers/of/unittest.c | 8 ++++---- include/linux/of.h | 11 ++++++++--- 4 files changed, 40 insertions(+), 19 deletions(-)