| Message ID | 20221021181016.14740-1-ansuelsmth@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | ARM: mach-qcom: fix support for ipq806x | expand |
On Fri, Oct 21, 2022 at 8:10 PM Christian Marangi <ansuelsmth@gmail.com> wrote: > Add a specific config flag for Qcom IPQ806x as this SoC can't use > AUTO_ZRELADDR and require the PHYS_OFFSET set to 0x42000000. > > This is needed as some legacy board (or some wrongly configured > bootloader) pass the wrong memory map and doesn't exclude the first > ~20MB of RAM reserved for the hardware network accellerators. > > With this change we can correctly support each board and prevent any > kind of misconfiguration done by the OEM. > > Signed-off-by: Christian Marangi <ansuelsmth@gmail.com> I had exactly this problem with the APQ8060 as well, then my problem went away. I was under the impression that this was solved. Is it not possible to use Geert's linux,usable-memory-range in the chosen node to make the kernel stay off the memory? (See examples by grep usable-memory in the kernel.) Yours, Linus Walleij
On Fri, Oct 21, 2022 at 11:44:56PM +0200, Linus Walleij wrote: > On Fri, Oct 21, 2022 at 8:10 PM Christian Marangi <ansuelsmth@gmail.com> wrote: > > > Add a specific config flag for Qcom IPQ806x as this SoC can't use > > AUTO_ZRELADDR and require the PHYS_OFFSET set to 0x42000000. > > > > This is needed as some legacy board (or some wrongly configured > > bootloader) pass the wrong memory map and doesn't exclude the first > > ~20MB of RAM reserved for the hardware network accellerators. > > > > With this change we can correctly support each board and prevent any > > kind of misconfiguration done by the OEM. > > > > Signed-off-by: Christian Marangi <ansuelsmth@gmail.com> > > I had exactly this problem with the APQ8060 as well, then my problem > went away. I was under the impression that this was solved. > > Is it not possible to use Geert's linux,usable-memory-range in > the chosen node to make the kernel stay off the memory? > (See examples by grep usable-memory in the kernel.) > Hi, just to confirm this is one of the example you are suggesting? chosen { bootargs = "console=ttyS0,115200 earlycon"; stdout-path = "serial0:115200n8"; linux,usable-memory-range = <0x80200000 0x1fe00000>; }; Main problem here is that uboot in some case doesn't support dt and pass wrong ATAGS (with the memory not reserved) and AUTO_ZRELADDR calculate the wrong addr I assume? I will test the usable-memory-range but isn't the same of declaring reserved space in the dts? Or the zimage decompressor checks linux,usable-memory-range bypassing atags?
On Fri, Oct 21, 2022 at 11:55 PM Christian Marangi <ansuelsmth@gmail.com> wrote: > On Fri, Oct 21, 2022 at 11:44:56PM +0200, Linus Walleij wrote: > > Is it not possible to use Geert's linux,usable-memory-range in > > the chosen node to make the kernel stay off the memory? > > (See examples by grep usable-memory in the kernel.) > > > > Hi, > just to confirm this is one of the example you are suggesting? > > chosen { > bootargs = "console=ttyS0,115200 earlycon"; > stdout-path = "serial0:115200n8"; > linux,usable-memory-range = <0x80200000 0x1fe00000>; > }; Yep that thing! > Main problem here is that uboot in some case doesn't support dt and pass > wrong ATAGS (with the memory not reserved) and AUTO_ZRELADDR calculate > the wrong addr I assume? You do have a DTB right, just that it is attached, and then the kernel uses the ATAGs to augment the memory? In that case what about disabling ARM_ATAG_DTB_COMPAT and adding the actual valid memory to the top-level DTS file? Just like that: memory { device_type = "memory"; reg = <0x42000000 0xnnnnnnnn>; }; > I will test the usable-memory-range but isn't the same of declaring > reserved space in the dts? Or the zimage decompressor checks > linux,usable-memory-range bypassing atags? As long as it just pass "too much" memory it should do the job, I *think*. Since I wrote this article: https://people.kernel.org/linusw/how-the-arm32-linux-kernel-decompresses Geert introduced some very elaborate low-level OF code and I do think it kicks in and makes sure to reserve this memory even before the decompressor goes to work (in difference from e.g. "reserved memory nodes" that are not inspected until later). See: commit 48342ae751c797ac73ac9c894b3f312df18ffd21 "ARM: 9124/1: uncompress: Parse "linux,usable-memory-range" DT property" Then if the memory node is in the DTB originally or patched in by U-Boot shouldn't really matter, usable-memory-range should kick in in either case. It is described as used for kexec (which I never use) but I think it can solve your problem too. The DT property is (by agreement) an undocumented Linux extension, so Geert knows the intended usecases better :) Yours, Linus Walleij
On Sat, Oct 22, 2022 at 04:21:28PM +0200, Linus Walleij wrote: > On Fri, Oct 21, 2022 at 11:55 PM Christian Marangi <ansuelsmth@gmail.com> wrote: > > On Fri, Oct 21, 2022 at 11:44:56PM +0200, Linus Walleij wrote: > > > > Is it not possible to use Geert's linux,usable-memory-range in > > > the chosen node to make the kernel stay off the memory? > > > (See examples by grep usable-memory in the kernel.) > > > > > just to confirm this is one of the example you are suggesting? > > > > chosen { > > bootargs = "console=ttyS0,115200 earlycon"; > > stdout-path = "serial0:115200n8"; > > linux,usable-memory-range = <0x80200000 0x1fe00000>; > > }; > > Yep that thing! > > > Main problem here is that uboot in some case doesn't support dt and pass > > wrong ATAGS (with the memory not reserved) and AUTO_ZRELADDR calculate > > the wrong addr I assume? > > You do have a DTB right, just that it is attached, and then the kernel > uses the ATAGs to augment the memory? > > In that case what about disabling ARM_ATAG_DTB_COMPAT > and adding the actual valid memory to the top-level DTS > file? Just like that: > > memory { > device_type = "memory"; > reg = <0x42000000 0xnnnnnnnn>; > }; The RB3011 (arch/arm/boot/dts/qcom-ipq8064-rb3011.dts) does this and has been working fine with AUTO_ZRELADDR (and no ATAGS support enabled) - I have a recollection it didn't used to, but it's certainly worked since the 5.15 timeframe. J.
On Sat, Oct 22, 2022 at 04:21:28PM +0200, Linus Walleij wrote: > On Fri, Oct 21, 2022 at 11:55 PM Christian Marangi <ansuelsmth@gmail.com> wrote: > > On Fri, Oct 21, 2022 at 11:44:56PM +0200, Linus Walleij wrote: > > > > Is it not possible to use Geert's linux,usable-memory-range in > > > the chosen node to make the kernel stay off the memory? > > > (See examples by grep usable-memory in the kernel.) > > > > > > > Hi, > > just to confirm this is one of the example you are suggesting? > > > > chosen { > > bootargs = "console=ttyS0,115200 earlycon"; > > stdout-path = "serial0:115200n8"; > > linux,usable-memory-range = <0x80200000 0x1fe00000>; > > }; > > Yep that thing! > > > Main problem here is that uboot in some case doesn't support dt and pass > > wrong ATAGS (with the memory not reserved) and AUTO_ZRELADDR calculate > > the wrong addr I assume? > > You do have a DTB right, just that it is attached, and then the kernel > uses the ATAGs to augment the memory? > > In that case what about disabling ARM_ATAG_DTB_COMPAT > and adding the actual valid memory to the top-level DTS > file? Just like that: > > memory { > device_type = "memory"; > reg = <0x42000000 0xnnnnnnnn>; > }; > > > > I will test the usable-memory-range but isn't the same of declaring > > reserved space in the dts? Or the zimage decompressor checks > > linux,usable-memory-range bypassing atags? > > As long as it just pass "too much" memory it should do the job, > I *think*. > > Since I wrote this article: > https://people.kernel.org/linusw/how-the-arm32-linux-kernel-decompresses > Geert introduced some very elaborate low-level OF code and I > do think it kicks in and makes sure to reserve this memory even > before the decompressor goes to work (in difference from e.g. > "reserved memory nodes" that are not inspected until later). > > See: > commit 48342ae751c797ac73ac9c894b3f312df18ffd21 > "ARM: 9124/1: uncompress: Parse "linux,usable-memory-range" DT property" > > Then if the memory node is in the DTB originally or patched in > by U-Boot shouldn't really matter, usable-memory-range should > kick in in either case. > > It is described as used for kexec (which I never use) but I think it can > solve your problem too. > > The DT property is (by agreement) an undocumented Linux extension, > so Geert knows the intended usecases better :) > Hi, bad news... yesterday I tested this binding and it's problematic. It does work and the router correctly boot... problem is that SMEM is broken with such configuration... I assume with this binding, by the system view ram starts from 0x42000000 instead of 0x40000000 and this cause SMEM to fail probe with the error "SBL didn't init SMEM". This is the location of SMEM entry in ram smem: smem@41000000 { compatible = "qcom,smem"; reg = <0x41000000 0x200000>; no-map; hwlocks = <&sfpb_mutex 3>; }; On openwrt (kernel 5.10 and 5.15) we currently use a mix of the old Makefile.boot infra and a patch to ignore atags. With the current configuration we can correctly bootup the system by passing the load addr to the decompressor to 0x42000000 (+TEXT_OFFEST) and also use SMEM as it gets correctly init in the not mapped ram addr. We are now working on adding 6.1 kernel support and since Makefile.boot infra got dropped, I'm searching a better solution that can also be upstreamed, for now PHY_OFFSET seems the only solution. Wonder if you have other ideas about this.
On Tue, Oct 25, 2022 at 1:47 AM Christian Marangi <ansuelsmth@gmail.com> wrote: > bad news... yesterday I tested this binding and it's problematic. It > does work and the router correctly boot... That's actually partly good news :D > problem is that SMEM is > broken with such configuration... I assume with this binding, by the > system view ram starts from 0x42000000 instead of 0x40000000 and this > cause SMEM to fail probe with the error "SBL didn't init SMEM". We need to fix this. > This is the location of SMEM entry in ram > > smem: smem@41000000 { > compatible = "qcom,smem"; > reg = <0x41000000 0x200000>; > no-map; > > hwlocks = <&sfpb_mutex 3>; > }; (...) > Wonder if you have other ideas about this. So the problem is that the resource is outside of the system RAM? I don't understand why that triggers it since this is per definition not system RAM, it is SMEM after all. And it is no different in esssence from any memory mapped IO or other things that are outside of the system RAM. The SMEM node is special since it is created without children thanks to the hack in drivers/of/platform.c. Then the driver in drivers/soc/qcom/smem.c contains things like this: rmem = of_reserved_mem_lookup(pdev->dev.of_node); if (rmem) { smem->regions[0].aux_base = rmem->base; smem->regions[0].size = rmem->size; } else { /* * Fall back to the memory-region reference, if we're not a * reserved-memory node. */ ret = qcom_smem_resolve_mem(smem, "memory-region", &smem->regions[0]); if (ret) return ret; } However it is treated as memory-mapped IO later: for (i = 1; i < num_regions; i++) { smem->regions[i].virt_base = devm_ioremap_wc(&pdev->dev, smem->regions[i].aux_base, smem->regions[i].size); if (!smem->regions[i].virt_base) { dev_err(&pdev->dev, "failed to remap %pa\n", &smem->regions[i].aux_base); return -ENOMEM; } } As a first hack I would check: 1. Is it the of_reserved_mem_lookup() or qcom_smem_resolve_smem() stuff in drivers/soc/qcom/smem.c that is failing? If yes then: 2. Add a fallback path just using of_iomap(node) for aux_base and size with some comment like /* smem is outside of the main memory map */ and see if that works. Yours, Linus Walleij
On Wed, Oct 26, 2022 at 10:19:21AM +0200, Linus Walleij wrote: > On Tue, Oct 25, 2022 at 1:47 AM Christian Marangi <ansuelsmth@gmail.com> wrote: > > > bad news... yesterday I tested this binding and it's problematic. It > > does work and the router correctly boot... > > That's actually partly good news :D Hi, sorry for the necroposting but I got some time and wanted to fix and bisect this for good since IPQ806x is finally in a better shape and is actually modern enough. > > > problem is that SMEM is > > broken with such configuration... I assume with this binding, by the > > system view ram starts from 0x42000000 instead of 0x40000000 and this > > cause SMEM to fail probe with the error "SBL didn't init SMEM". > > We need to fix this. > Totally but I think the problem is more deep... > > This is the location of SMEM entry in ram > > > > smem: smem@41000000 { > > compatible = "qcom,smem"; > > reg = <0x41000000 0x200000>; > > no-map; > > > > hwlocks = <&sfpb_mutex 3>; > > }; > (...) > > Wonder if you have other ideas about this. > > So the problem is that the resource is outside of the system RAM? > > I don't understand why that triggers it since this is per definition not > system RAM, it is SMEM after all. And it is no different in esssence > from any memory mapped IO or other things that are outside of > the system RAM. > > The SMEM node is special since it is created without children thanks > to the hack in drivers/of/platform.c. > > Then the driver in drivers/soc/qcom/smem.c > contains things like this: > > rmem = of_reserved_mem_lookup(pdev->dev.of_node); > if (rmem) { > smem->regions[0].aux_base = rmem->base; > smem->regions[0].size = rmem->size; > } else { > /* > * Fall back to the memory-region reference, if we're not a > * reserved-memory node. > */ > ret = qcom_smem_resolve_mem(smem, "memory-region", > &smem->regions[0]); > if (ret) > return ret; > } > > However it is treated as memory-mapped IO later: > > for (i = 1; i < num_regions; i++) { > smem->regions[i].virt_base = devm_ioremap_wc(&pdev->dev, > > smem->regions[i].aux_base, > > smem->regions[i].size); > if (!smem->regions[i].virt_base) { > dev_err(&pdev->dev, "failed to remap %pa\n", > &smem->regions[i].aux_base); > return -ENOMEM; > } > } > > As a first hack I would check: > > 1. Is it the of_reserved_mem_lookup() or qcom_smem_resolve_smem() stuff > in drivers/soc/qcom/smem.c that is failing? > > If yes then: > > 2. Add a fallback path just using of_iomap(node) for aux_base and size > with some comment like /* smem is outside of the main memory map */ > and see if that works. > I think we got confused and we didn't read the code correctly. The error is "SMEM is not initialized by SBL" that is triggered by... header = smem->regions[0].virt_base; if (le32_to_cpu(header->initialized) != 1 || le32_to_cpu(header->reserved)) { dev_err(&pdev->dev, "SMEM is not initialized by SBL\n",); return -EINVAL; } I verified correctly that aux_base and size are the correct values 0x41000000 and 0x200000. And from what I can see they get correctly iomapped. Problem is that initialized and reserved have garbage in it. (not random data tho but everytime the same data) My theory is that somehow the loader is still writing data there but I'm a bit lost on how to verify that. (the fact that the data in those values is always the same with the same compiled image makes me think it's actually just loaded data) I also tested with disabling the CONFIG_ARM_ATAG_DTB_COMPAT flag but I have the same result. What I'm using is this memory node memory@0 { reg = <0x42000000 0x1e000000>; device_type = "memory"; }; And in chosed I have chosen { bootargs = "earlycon"; linux,usable-memory-range = <0x42000000 0x10000000>; }; (the size is different just for the sake of it but it should not cause problem right?) Maybe there is a way to make the SMEM reclaim those RAM space and reinit it? (it's a workaround tho) Also with the current situation the kernel panics with... But I assume this is caused by SMEM malfunctioning (the panic happen right after rpm init when the RPM regulators are getting init. Looking at the affected codes maybe it's failing at the "Free unused pages" stage? [ 1.912392] 8<--- cut here --- [ 1.912431] Unable to handle kernel NULL pointer dereference at virtual address 00000000 [ 1.914356] [00000000] *pgd=00000000 [ 1.922676] Internal error: Oops: 80000007 [#1] SMP ARM [ 1.926158] Modules linked in: [ 1.931103] CPU: 1 PID: 84 Comm: modprobe Not tainted 6.1.65 #0 [ 1.934229] Hardware name: Generic DT based system [ 1.940045] PC is at 0x0 [ 1.944902] LR is at release_pages+0x114/0x36c [ 1.947595] pc : [<00000000>] lr : [<c04298dc>] psr: 40000013 [ 1.951851] sp : c27abe18 ip : c13cd5c1 fp : c27abe38 [ 1.958012] r10: 0000009c r9 : c4018268 r8 : 00000005 [ 1.963220] r7 : c243f400 r6 : c243f400 r5 : 00000098 r4 : df992b54 [ 1.968431] r3 : 00000000 r2 : 00000000 r1 : 60000013 r0 : df992b54 [ 1.975029] Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none [ 1.981543] Control: 10c5787d Table: 4367806a DAC: 00000051 [ 1.988744] Register r0 information: non-slab/vmalloc memory [ 1.994472] Register r1 information: non-paged memory [ 2.000200] Register r2 information: NULL pointer [ 2.005148] Register r3 information: NULL pointer [ 2.009834] Register r4 information: non-slab/vmalloc memory [ 2.014525] Register r5 information: non-paged memory [ 2.020252] Register r6 information: slab kmalloc-1k start c243f400 pointer offset 0 size 1024 [ 2.025206] Register r7 information: slab kmalloc-1k start c243f400 pointer offset 0 size 1024 [ 2.033714] Register r8 information: non-paged memory [ 2.042301] Register r9 information: non-slab/vmalloc memory [ 2.047424] Register r10 information: non-paged memory [ 2.053152] Register r11 information: non-slab/vmalloc memory [ 2.058100] Register r12 information: non-paged memory [ 2.063915] Process modprobe (pid: 84, stack limit = 0x(ptrval)) [ 2.068953] Stack: (0xc27abe18 to 0xc27ac000) [ 2.075115] be00: 00000000 00000000 [ 2.079378] be20: c147514c ffefffcf 00000000 00000000 0000009c 60000013 dfa12928 dfa12b44 [ 2.087537] be40: c27abf24 0000009c c4018000 c401800c c27abf0c c27abf24 00000000 000000f8 [ 2.095697] be60: 00000000 c045b248 ffffffff c27abf0c c35d1400 00000000 c35d1438 c045b4f8 [ 2.103858] be80: c27abf0c 00002000 00000000 c044fb14 00000000 c0b6c2bc c35d1400 ffffffff [ 2.112016] bea0: ffffffff c35a4c0c 00000000 ffffffff 00000000 00001c01 00000000 c3591510 [ 2.120176] bec0: 00000000 c35d1400 ffffffff c3591510 00000000 c35d1400 00000000 c0458f30 [ 2.128336] bee0: 00000000 c08f35c8 c36ebf00 c35d1400 00010000 00013fff c35a4c0c 00000000 [ 2.136496] bf00: ffffffff 00000000 00000101 c35d1400 ffffffff ffffffff c2420501 00000001 [ 2.144656] bf20: c4018000 c4018000 00000000 00000008 dfde733c dfde7360 dfde7384 dfde73a8 [ 2.152815] bf40: dfa12a44 dfa12948 dfa129d8 dfa12ad4 c35d1400 00000000 c35d1438 00000698 [ 2.160976] bf60: c27abf78 c0318a34 c35d1400 c2731000 c35d1438 c0320604 0000ff00 c258ea00 [ 2.169136] bf80: c2731000 c2456f40 c03002c4 c2456f40 00000000 c0320e0c 000000f8 c0320e6c [ 2.177294] bfa0: ffffffff c0300060 ffffffff bed38eb4 ffffffff bed38dcc 00000000 ffffffff [ 2.185455] bfc0: ffffffff bed38eb4 00010f60 000000f8 6474e552 00000020 00000000 00000000 [ 2.193614] bfe0: 6ffffff9 bed38e78 b6f91f1c b6fa4a44 60000010 ffffffff 00000000 00000000 [ 2.201777] release_pages from tlb_batch_pages_flush+0x3c/0x70 [ 2.209927] tlb_batch_pages_flush from tlb_finish_mmu+0x4c/0x130 [ 2.215656] tlb_finish_mmu from exit_mmap+0xec/0x1e0 [ 2.221903] exit_mmap from mmput+0x40/0x120 [ 2.226939] mmput from do_exit+0x238/0x890 [ 2.231279] do_exit from do_group_exit+0x34/0x84 [ 2.235184] do_group_exit from __wake_up_parent+0x0/0x18 [ 2.240053] Code: bad PC value [ 2.245556] ---[ end trace 0000000000000000 ]--- [ 2.248448] Kernel panic - not syncing: Fatal exception [ 2.253158] CPU0: stopping [ 2.253169] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G D 6.1.65 #0 [ 2.253180] Hardware name: Generic DT based system [ 2.253189] unwind_backtrace from show_stack+0x10/0x14 [ 2.253216] show_stack from dump_stack_lvl+0x40/0x4c [ 2.253249] dump_stack_lvl from do_handle_IPI+0xf0/0x124 [ 2.253276] do_handle_IPI from ipi_handler+0x18/0x20 [ 2.253293] ipi_handler from handle_percpu_devid_irq+0x78/0x134 [ 2.253313] handle_percpu_devid_irq from generic_handle_domain_irq+0x28/0x38 [ 2.253338] generic_handle_domain_irq from gic_handle_irq+0x74/0x88 [ 2.253361] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 [ 2.253391] generic_handle_arch_irq from call_with_stack+0x18/0x20 [ 2.253419] call_with_stack from __irq_svc+0x80/0x98 [ 2.253438] Exception stack(0xc1401f00 to 0xc1401f48) [ 2.253451] 1f00: 00000005 00000000 00000a61 c03128a0 c1408640 00000000 c1404f68 c1404fa4 [ 2.253461] 1f20: 00000000 c13c9c38 00000000 00000000 c14c1f00 c1401f50 c0307148 c030714c [ 2.253467] 1f40: 60000013 ffffffff [ 2.253474] __irq_svc from arch_cpu_idle+0x38/0x3c [ 2.253500] arch_cpu_idle from default_idle_call+0x24/0x34 [ 2.253526] default_idle_call from do_idle+0x1ec/0x240 [ 2.253545] do_idle from cpu_startup_entry+0x28/0x2c [ 2.253559] cpu_startup_entry from kernel_init+0x0/0x12c [ 2.376160] Rebooting in 1 seconds..
On Wed, Jan 17, 2024 at 02:17:03PM +0100, Christian Marangi wrote: > On Wed, Oct 26, 2022 at 10:19:21AM +0200, Linus Walleij wrote: > > On Tue, Oct 25, 2022 at 1:47 AM Christian Marangi <ansuelsmth@gmail.com> wrote: > > > > > bad news... yesterday I tested this binding and it's problematic. It > > > does work and the router correctly boot... > > > > That's actually partly good news :D > > Hi, > sorry for the necroposting but I got some time and wanted to fix and > bisect this for good since IPQ806x is finally in a better shape and is > actually modern enough. > > > > > > problem is that SMEM is > > > broken with such configuration... I assume with this binding, by the > > > system view ram starts from 0x42000000 instead of 0x40000000 and this > > > cause SMEM to fail probe with the error "SBL didn't init SMEM". > > > > We need to fix this. > > > > Totally but I think the problem is more deep... > > > > This is the location of SMEM entry in ram > > > > > > smem: smem@41000000 { > > > compatible = "qcom,smem"; > > > reg = <0x41000000 0x200000>; > > > no-map; > > > > > > hwlocks = <&sfpb_mutex 3>; > > > }; > > (...) > > > Wonder if you have other ideas about this. > > > > So the problem is that the resource is outside of the system RAM? > > > > I don't understand why that triggers it since this is per definition not > > system RAM, it is SMEM after all. And it is no different in esssence > > from any memory mapped IO or other things that are outside of > > the system RAM. > > > > The SMEM node is special since it is created without children thanks > > to the hack in drivers/of/platform.c. > > > > Then the driver in drivers/soc/qcom/smem.c > > contains things like this: > > > > rmem = of_reserved_mem_lookup(pdev->dev.of_node); > > if (rmem) { > > smem->regions[0].aux_base = rmem->base; > > smem->regions[0].size = rmem->size; > > } else { > > /* > > * Fall back to the memory-region reference, if we're not a > > * reserved-memory node. > > */ > > ret = qcom_smem_resolve_mem(smem, "memory-region", > > &smem->regions[0]); > > if (ret) > > return ret; > > } > > > > However it is treated as memory-mapped IO later: > > > > for (i = 1; i < num_regions; i++) { > > smem->regions[i].virt_base = devm_ioremap_wc(&pdev->dev, > > > > smem->regions[i].aux_base, > > > > smem->regions[i].size); > > if (!smem->regions[i].virt_base) { > > dev_err(&pdev->dev, "failed to remap %pa\n", > > &smem->regions[i].aux_base); > > return -ENOMEM; > > } > > } > > > > As a first hack I would check: > > > > 1. Is it the of_reserved_mem_lookup() or qcom_smem_resolve_smem() stuff > > in drivers/soc/qcom/smem.c that is failing? > > > > If yes then: > > > > 2. Add a fallback path just using of_iomap(node) for aux_base and size > > with some comment like /* smem is outside of the main memory map */ > > and see if that works. > > > > I think we got confused and we didn't read the code correctly. The > error is "SMEM is not initialized by SBL" that is triggered by... > > header = smem->regions[0].virt_base; > if (le32_to_cpu(header->initialized) != 1 || > le32_to_cpu(header->reserved)) { > dev_err(&pdev->dev, "SMEM is not initialized by SBL\n",); > return -EINVAL; > } > > I verified correctly that aux_base and size are the correct values > 0x41000000 and 0x200000. And from what I can see they get correctly > iomapped. > > Problem is that initialized and reserved have garbage in it. (not random > data tho but everytime the same data) > > My theory is that somehow the loader is still writing data there but I'm > a bit lost on how to verify that. (the fact that the data in those > values is always the same with the same compiled image makes me think > it's actually just loaded data) > > I also tested with disabling the CONFIG_ARM_ATAG_DTB_COMPAT flag but I > have the same result. > > What I'm using is this memory node > > memory@0 { > reg = <0x42000000 0x1e000000>; > device_type = "memory"; > }; > > And in chosed I have > > chosen { > bootargs = "earlycon"; > linux,usable-memory-range = <0x42000000 0x10000000>; > }; > > (the size is different just for the sake of it but it should not cause > problem right?) > > Maybe there is a way to make the SMEM reclaim those RAM space and reinit > it? (it's a workaround tho) > > Also with the current situation the kernel panics with... But I assume > this is caused by SMEM malfunctioning (the panic happen right after rpm > init when the RPM regulators are getting init. Looking at the affected > codes maybe it's failing at the "Free unused pages" stage? > > [ 1.912392] 8<--- cut here --- > [ 1.912431] Unable to handle kernel NULL pointer dereference at virtual address 00000000 > [ 1.914356] [00000000] *pgd=00000000 > [ 1.922676] Internal error: Oops: 80000007 [#1] SMP ARM > [ 1.926158] Modules linked in: > [ 1.931103] CPU: 1 PID: 84 Comm: modprobe Not tainted 6.1.65 #0 > [ 1.934229] Hardware name: Generic DT based system > [ 1.940045] PC is at 0x0 > [ 1.944902] LR is at release_pages+0x114/0x36c > [ 1.947595] pc : [<00000000>] lr : [<c04298dc>] psr: 40000013 > [ 1.951851] sp : c27abe18 ip : c13cd5c1 fp : c27abe38 > [ 1.958012] r10: 0000009c r9 : c4018268 r8 : 00000005 > [ 1.963220] r7 : c243f400 r6 : c243f400 r5 : 00000098 r4 : df992b54 > [ 1.968431] r3 : 00000000 r2 : 00000000 r1 : 60000013 r0 : df992b54 > [ 1.975029] Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none > [ 1.981543] Control: 10c5787d Table: 4367806a DAC: 00000051 > [ 1.988744] Register r0 information: non-slab/vmalloc memory > [ 1.994472] Register r1 information: non-paged memory > [ 2.000200] Register r2 information: NULL pointer > [ 2.005148] Register r3 information: NULL pointer > [ 2.009834] Register r4 information: non-slab/vmalloc memory > [ 2.014525] Register r5 information: non-paged memory > [ 2.020252] Register r6 information: slab kmalloc-1k start c243f400 pointer offset 0 size 1024 > [ 2.025206] Register r7 information: slab kmalloc-1k start c243f400 pointer offset 0 size 1024 > [ 2.033714] Register r8 information: non-paged memory > [ 2.042301] Register r9 information: non-slab/vmalloc memory > [ 2.047424] Register r10 information: non-paged memory > [ 2.053152] Register r11 information: non-slab/vmalloc memory > [ 2.058100] Register r12 information: non-paged memory > [ 2.063915] Process modprobe (pid: 84, stack limit = 0x(ptrval)) > [ 2.068953] Stack: (0xc27abe18 to 0xc27ac000) > [ 2.075115] be00: 00000000 00000000 > [ 2.079378] be20: c147514c ffefffcf 00000000 00000000 0000009c 60000013 dfa12928 dfa12b44 > [ 2.087537] be40: c27abf24 0000009c c4018000 c401800c c27abf0c c27abf24 00000000 000000f8 > [ 2.095697] be60: 00000000 c045b248 ffffffff c27abf0c c35d1400 00000000 c35d1438 c045b4f8 > [ 2.103858] be80: c27abf0c 00002000 00000000 c044fb14 00000000 c0b6c2bc c35d1400 ffffffff > [ 2.112016] bea0: ffffffff c35a4c0c 00000000 ffffffff 00000000 00001c01 00000000 c3591510 > [ 2.120176] bec0: 00000000 c35d1400 ffffffff c3591510 00000000 c35d1400 00000000 c0458f30 > [ 2.128336] bee0: 00000000 c08f35c8 c36ebf00 c35d1400 00010000 00013fff c35a4c0c 00000000 > [ 2.136496] bf00: ffffffff 00000000 00000101 c35d1400 ffffffff ffffffff c2420501 00000001 > [ 2.144656] bf20: c4018000 c4018000 00000000 00000008 dfde733c dfde7360 dfde7384 dfde73a8 > [ 2.152815] bf40: dfa12a44 dfa12948 dfa129d8 dfa12ad4 c35d1400 00000000 c35d1438 00000698 > [ 2.160976] bf60: c27abf78 c0318a34 c35d1400 c2731000 c35d1438 c0320604 0000ff00 c258ea00 > [ 2.169136] bf80: c2731000 c2456f40 c03002c4 c2456f40 00000000 c0320e0c 000000f8 c0320e6c > [ 2.177294] bfa0: ffffffff c0300060 ffffffff bed38eb4 ffffffff bed38dcc 00000000 ffffffff > [ 2.185455] bfc0: ffffffff bed38eb4 00010f60 000000f8 6474e552 00000020 00000000 00000000 > [ 2.193614] bfe0: 6ffffff9 bed38e78 b6f91f1c b6fa4a44 60000010 ffffffff 00000000 00000000 > [ 2.201777] release_pages from tlb_batch_pages_flush+0x3c/0x70 > [ 2.209927] tlb_batch_pages_flush from tlb_finish_mmu+0x4c/0x130 > [ 2.215656] tlb_finish_mmu from exit_mmap+0xec/0x1e0 > [ 2.221903] exit_mmap from mmput+0x40/0x120 > [ 2.226939] mmput from do_exit+0x238/0x890 > [ 2.231279] do_exit from do_group_exit+0x34/0x84 > [ 2.235184] do_group_exit from __wake_up_parent+0x0/0x18 > [ 2.240053] Code: bad PC value > [ 2.245556] ---[ end trace 0000000000000000 ]--- > [ 2.248448] Kernel panic - not syncing: Fatal exception > [ 2.253158] CPU0: stopping > [ 2.253169] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G D 6.1.65 #0 > [ 2.253180] Hardware name: Generic DT based system > [ 2.253189] unwind_backtrace from show_stack+0x10/0x14 > [ 2.253216] show_stack from dump_stack_lvl+0x40/0x4c > [ 2.253249] dump_stack_lvl from do_handle_IPI+0xf0/0x124 > [ 2.253276] do_handle_IPI from ipi_handler+0x18/0x20 > [ 2.253293] ipi_handler from handle_percpu_devid_irq+0x78/0x134 > [ 2.253313] handle_percpu_devid_irq from generic_handle_domain_irq+0x28/0x38 > [ 2.253338] generic_handle_domain_irq from gic_handle_irq+0x74/0x88 > [ 2.253361] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 > [ 2.253391] generic_handle_arch_irq from call_with_stack+0x18/0x20 > [ 2.253419] call_with_stack from __irq_svc+0x80/0x98 > [ 2.253438] Exception stack(0xc1401f00 to 0xc1401f48) > [ 2.253451] 1f00: 00000005 00000000 00000a61 c03128a0 c1408640 00000000 c1404f68 c1404fa4 > [ 2.253461] 1f20: 00000000 c13c9c38 00000000 00000000 c14c1f00 c1401f50 c0307148 c030714c > [ 2.253467] 1f40: 60000013 ffffffff > [ 2.253474] __irq_svc from arch_cpu_idle+0x38/0x3c > [ 2.253500] arch_cpu_idle from default_idle_call+0x24/0x34 > [ 2.253526] default_idle_call from do_idle+0x1ec/0x240 > [ 2.253545] do_idle from cpu_startup_entry+0x28/0x2c > [ 2.253559] cpu_startup_entry from kernel_init+0x0/0x12c > [ 2.376160] Rebooting in 1 seconds.. > Some followup on this... I manage to enable DEBUG_LL and can have debug output from the decompressor... From what I can see fdt_check_mem_start is not called at all... What I'm using with kernel config are: CONFIG_ARM_APPENDED_DTB=y CONFIG_ARM_ATAG_DTB_COMPAT=y And a downstream patch that mangle all the atags and takes only the cmdline one. The load and entry point is: 0x42208000 With the current setup I have this (I also added some debug log that print what is actually passed to do decompress DTB:0x42AED270 (0x00008BA7) Uncompressing Linux... 40208000 4220F10C done, booting the kernel. Where 40208000 is the value of output_start and 4220F10C is input_data. And I think this confirm that it's getting loaded in the wrong position actually in reserved memory... But how this is possible??? Hope can someone help me in this since I wasted the entire day with this and didn't manage to make any progress... aside from having fun with the head.S assembly code.
On Thu, Jan 18, 2024 at 12:04 AM Christian Marangi <ansuelsmth@gmail.com> wrote: > Some followup on this... I manage to enable DEBUG_LL and can have debug > output from the decompressor... Yeah that is helpful! > From what I can see fdt_check_mem_start is not called at all... > > What I'm using with kernel config are: > CONFIG_ARM_APPENDED_DTB=y > CONFIG_ARM_ATAG_DTB_COMPAT=y > And a downstream patch that mangle all the atags and takes only the > cmdline one. > > The load and entry point is: > 0x42208000 > > With the current setup I have this (I also added some debug log that > print what is actually passed to do decompress > > DTB:0x42AED270 (0x00008BA7) > Uncompressing Linux... > 40208000 > 4220F10C done, booting the kernel. > > Where 40208000 is the value of output_start and 4220F10C is input_data. > > And I think this confirm that it's getting loaded in the wrong position > actually in reserved memory... But how this is possible??? Hope can > someone help me in this since I wasted the entire day with this and > didn't manage to make any progress... aside from having fun with the > head.S assembly code. I have no idea how this happens, but when I boot images I do it using fastboot like this: fastboot --base 40200000 --cmdline "console=ttyMSM0,115200,n8" boot zImage So I definitely hammer it to boot from 0x40200000 (+0x8000). Yours, Linus Walleij
On Thu, Jan 18, 2024 at 10:02:37AM +0100, Linus Walleij wrote: > On Thu, Jan 18, 2024 at 12:04 AM Christian Marangi <ansuelsmth@gmail.com> wrote: > > > Some followup on this... I manage to enable DEBUG_LL and can have debug > > output from the decompressor... > > Yeah that is helpful! > > > From what I can see fdt_check_mem_start is not called at all... > > > > What I'm using with kernel config are: > > CONFIG_ARM_APPENDED_DTB=y > > CONFIG_ARM_ATAG_DTB_COMPAT=y > > And a downstream patch that mangle all the atags and takes only the > > cmdline one. > > > > The load and entry point is: > > 0x42208000 > > > > With the current setup I have this (I also added some debug log that > > print what is actually passed to do decompress > > > > DTB:0x42AED270 (0x00008BA7) > > Uncompressing Linux... > > 40208000 > > 4220F10C done, booting the kernel. > > > > Where 40208000 is the value of output_start and 4220F10C is input_data. > > > > And I think this confirm that it's getting loaded in the wrong position > > actually in reserved memory... But how this is possible??? Hope can > > someone help me in this since I wasted the entire day with this and > > didn't manage to make any progress... aside from having fun with the > > head.S assembly code. > > I have no idea how this happens, but when I boot images I do > it using fastboot like this: > > fastboot --base 40200000 --cmdline "console=ttyMSM0,115200,n8" boot zImage > > So I definitely hammer it to boot from 0x40200000 (+0x8000). > Consider that this is uboot so nothing about fastboot. Without AUTO_ZRELADDR enabled this is the output from the decompressor. Starting kernel ... DTB:0x42B214A0 (0x00008B79) C:0x422080C0-0x42B2A040->0x4349C600-0x43DBE580 DTB:0x43DB59E0 (0x00008B85) Uncompressing Linux... 42208000 434A362C done, booting the kernel. 42208000 input 434A362C output The DTB location match but I can see the ADDR to the right place and getting moved to a new location (I assume as it would get overwritten by itself...) guess the main problem is mov r0, pc (line 279) With pc veing 0x40200000 instead of 0x42200000
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 1af63e17b4ad..0818d35973ad 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -282,6 +282,7 @@ config PHYS_OFFSET default 0x30000000 if ARCH_S3C24XX default 0xa0000000 if ARCH_IOP32X || ARCH_PXA default 0xc0000000 if ARCH_EP93XX || ARCH_SA1100 + default 0x42000000 if ARCH_IPQ806X default 0 help Please provide the physical address corresponding to the @@ -1701,7 +1702,7 @@ config CRASH_DUMP config AUTO_ZRELADDR bool "Auto calculation of the decompressed kernel image address" if !ARCH_MULTIPLATFORM - default !(ARCH_FOOTBRIDGE || ARCH_RPC || ARCH_SA1100) + default !(ARCH_FOOTBRIDGE || ARCH_RPC || ARCH_SA1100 || ARCH_IPQ806X) help ZRELADDR is the physical address where the decompressed kernel image will be placed. If AUTO_ZRELADDR is selected, the address diff --git a/arch/arm/mach-qcom/Kconfig b/arch/arm/mach-qcom/Kconfig index 12a812e61c16..b11b6e391ff0 100644 --- a/arch/arm/mach-qcom/Kconfig +++ b/arch/arm/mach-qcom/Kconfig @@ -46,4 +46,17 @@ config ARCH_MDM9615 bool "Enable support for MDM9615" select CLKSRC_QCOM +config ARCH_IPQ806X + bool "Enable support for IPQ806x" + help + Enable support for the Qualcomm IPQ806x. + + IPQ806x require special PHYS_OFFSET and can't use AUTO_ZRELADDR. + The first ~20MB of RAM is reserved for the hardware network accelerators, + and the bootloader removes this section from the layout passed from the + ATAGS (when used by some bootloader doesn't even do that). + + To support every system and handle legacy systems, hardcode PHYS_OFFSET and + disable AUTO_ZRELADDR. + endif
Add a specific config flag for Qcom IPQ806x as this SoC can't use AUTO_ZRELADDR and require the PHYS_OFFSET set to 0x42000000. This is needed as some legacy board (or some wrongly configured bootloader) pass the wrong memory map and doesn't exclude the first ~20MB of RAM reserved for the hardware network accellerators. With this change we can correctly support each board and prevent any kind of misconfiguration done by the OEM. Signed-off-by: Christian Marangi <ansuelsmth@gmail.com> --- arch/arm/Kconfig | 3 ++- arch/arm/mach-qcom/Kconfig | 13 +++++++++++++ 2 files changed, 15 insertions(+), 1 deletion(-)