| Message ID | 20181024193256.23734-1-f.fainelli@gmail.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | arm64: Cut rebuild time when changing CONFIG_BLK_DEV_INITRD | expand |
On Wed, Oct 24, 2018 at 2:33 PM Florian Fainelli <f.fainelli@gmail.com> wrote: > > Hi all, > > While investigating why ARM64 required a ton of objects to be rebuilt > when toggling CONFIG_DEV_BLK_INITRD, it became clear that this was > because we define __early_init_dt_declare_initrd() differently and we do > that in arch/arm64/include/asm/memory.h which gets included by a fair > amount of other header files, and translation units as well. I scratch my head sometimes as to why some config options rebuild so much stuff. One down, ? to go. :) > Changing the value of CONFIG_DEV_BLK_INITRD is a common thing with build > systems that generate two kernels: one with the initramfs and one > without. buildroot is one of these build systems, OpenWrt is also > another one that does this. > > This patch series proposes adding an empty initrd.h to satisfy the need > for drivers/of/fdt.c to unconditionally include that file, and moves the > custom __early_init_dt_declare_initrd() definition away from > asm/memory.h > > This cuts the number of objects rebuilds from 1920 down to 26, so a > factor 73 approximately. > > Apologies for the long CC list, please let me know how you would go > about merging that and if another approach would be preferable, e.g: > introducing a CONFIG_ARCH_INITRD_BELOW_START_OK Kconfig option or > something like that. There may be a better way as of 4.20 because bootmem is now gone and only memblock is used. This should unify what each arch needs to do with initrd early. We need the physical address early for memblock reserving. Then later on we need the virtual address to access the initrd. Perhaps we should just change initrd_start and initrd_end to physical addresses (or add 2 new variables would be less invasive and allow for different translation than __va()). The sanity checks and memblock reserve could also perhaps be moved to a common location. Alternatively, given arm64 is the only oddball, I'd be fine with an "if (IS_ENABLED(CONFIG_ARM64))" condition in the default __early_init_dt_declare_initrd as long as we have a path to removing it like the above option. Rob
On 10/24/18 12:55 PM, Rob Herring wrote: > On Wed, Oct 24, 2018 at 2:33 PM Florian Fainelli <f.fainelli@gmail.com> wrote: >> >> Hi all, >> >> While investigating why ARM64 required a ton of objects to be rebuilt >> when toggling CONFIG_DEV_BLK_INITRD, it became clear that this was >> because we define __early_init_dt_declare_initrd() differently and we do >> that in arch/arm64/include/asm/memory.h which gets included by a fair >> amount of other header files, and translation units as well. > > I scratch my head sometimes as to why some config options rebuild so > much stuff. One down, ? to go. :) > This one was by far the most invasive one due to its include chain, but yes, there would be many more that could be optimized. >> Changing the value of CONFIG_DEV_BLK_INITRD is a common thing with build >> systems that generate two kernels: one with the initramfs and one >> without. buildroot is one of these build systems, OpenWrt is also >> another one that does this. >> >> This patch series proposes adding an empty initrd.h to satisfy the need >> for drivers/of/fdt.c to unconditionally include that file, and moves the >> custom __early_init_dt_declare_initrd() definition away from >> asm/memory.h >> >> This cuts the number of objects rebuilds from 1920 down to 26, so a >> factor 73 approximately. >> >> Apologies for the long CC list, please let me know how you would go >> about merging that and if another approach would be preferable, e.g: >> introducing a CONFIG_ARCH_INITRD_BELOW_START_OK Kconfig option or >> something like that. > > There may be a better way as of 4.20 because bootmem is now gone and > only memblock is used. This should unify what each arch needs to do > with initrd early. We need the physical address early for memblock > reserving. Then later on we need the virtual address to access the > initrd. Perhaps we should just change initrd_start and initrd_end to > physical addresses (or add 2 new variables would be less invasive and > allow for different translation than __va()). The sanity checks and > memblock reserve could also perhaps be moved to a common location. > > Alternatively, given arm64 is the only oddball, I'd be fine with an > "if (IS_ENABLED(CONFIG_ARM64))" condition in the default > __early_init_dt_declare_initrd as long as we have a path to removing > it like the above option. OK, let me cook a patch doing that and meanwhile I will look at how much work is involved to implement the above option you outlined, which also sounds entirely reasonable. Thanks!
On Wed, Oct 24, 2018 at 3:01 PM Florian Fainelli <f.fainelli@gmail.com> wrote: > > On 10/24/18 12:55 PM, Rob Herring wrote: > > On Wed, Oct 24, 2018 at 2:33 PM Florian Fainelli <f.fainelli@gmail.com> wrote: > >> > >> Hi all, > >> > >> While investigating why ARM64 required a ton of objects to be rebuilt > >> when toggling CONFIG_DEV_BLK_INITRD, it became clear that this was > >> because we define __early_init_dt_declare_initrd() differently and we do > >> that in arch/arm64/include/asm/memory.h which gets included by a fair > >> amount of other header files, and translation units as well. > > > > I scratch my head sometimes as to why some config options rebuild so > > much stuff. One down, ? to go. :) > > > > This one was by far the most invasive one due to its include chain, but > yes, there would be many more that could be optimized. > > >> Changing the value of CONFIG_DEV_BLK_INITRD is a common thing with build > >> systems that generate two kernels: one with the initramfs and one > >> without. buildroot is one of these build systems, OpenWrt is also > >> another one that does this. > >> > >> This patch series proposes adding an empty initrd.h to satisfy the need > >> for drivers/of/fdt.c to unconditionally include that file, and moves the > >> custom __early_init_dt_declare_initrd() definition away from > >> asm/memory.h > >> > >> This cuts the number of objects rebuilds from 1920 down to 26, so a > >> factor 73 approximately. > >> > >> Apologies for the long CC list, please let me know how you would go > >> about merging that and if another approach would be preferable, e.g: > >> introducing a CONFIG_ARCH_INITRD_BELOW_START_OK Kconfig option or > >> something like that. > > > > There may be a better way as of 4.20 because bootmem is now gone and > > only memblock is used. This should unify what each arch needs to do > > with initrd early. We need the physical address early for memblock > > reserving. Then later on we need the virtual address to access the > > initrd. Perhaps we should just change initrd_start and initrd_end to > > physical addresses (or add 2 new variables would be less invasive and > > allow for different translation than __va()). The sanity checks and > > memblock reserve could also perhaps be moved to a common location. > > > > Alternatively, given arm64 is the only oddball, I'd be fine with an > > "if (IS_ENABLED(CONFIG_ARM64))" condition in the default > > __early_init_dt_declare_initrd as long as we have a path to removing > > it like the above option. > > OK, let me cook a patch doing that and meanwhile I will look at how much > work is involved to implement the above option you outlined, which also > sounds entirely reasonable. BTW, I would suspect that initrd_below_start_ok being 1 is not okay for most arches. I'm not sure how that would work. min_low_pfn is typically based on the start of memory. arm64 is not even setting it. Rob > -- > Florian
On Wed, Oct 24, 2018 at 02:55:17PM -0500, Rob Herring wrote: > On Wed, Oct 24, 2018 at 2:33 PM Florian Fainelli <f.fainelli@gmail.com> wrote: > > > > Hi all, > > > > While investigating why ARM64 required a ton of objects to be rebuilt > > when toggling CONFIG_DEV_BLK_INITRD, it became clear that this was > > because we define __early_init_dt_declare_initrd() differently and we do > > that in arch/arm64/include/asm/memory.h which gets included by a fair > > amount of other header files, and translation units as well. > > I scratch my head sometimes as to why some config options rebuild so > much stuff. One down, ? to go. :) > > > Changing the value of CONFIG_DEV_BLK_INITRD is a common thing with build > > systems that generate two kernels: one with the initramfs and one > > without. buildroot is one of these build systems, OpenWrt is also > > another one that does this. > > > > This patch series proposes adding an empty initrd.h to satisfy the need > > for drivers/of/fdt.c to unconditionally include that file, and moves the > > custom __early_init_dt_declare_initrd() definition away from > > asm/memory.h > > > > This cuts the number of objects rebuilds from 1920 down to 26, so a > > factor 73 approximately. > > > > Apologies for the long CC list, please let me know how you would go > > about merging that and if another approach would be preferable, e.g: > > introducing a CONFIG_ARCH_INITRD_BELOW_START_OK Kconfig option or > > something like that. > > There may be a better way as of 4.20 because bootmem is now gone and > only memblock is used. This should unify what each arch needs to do > with initrd early. We need the physical address early for memblock > reserving. Then later on we need the virtual address to access the > initrd. Perhaps we should just change initrd_start and initrd_end to > physical addresses (or add 2 new variables would be less invasive and > allow for different translation than __va()). The sanity checks and > memblock reserve could also perhaps be moved to a common location. > > Alternatively, given arm64 is the only oddball, I'd be fine with an > "if (IS_ENABLED(CONFIG_ARM64))" condition in the default > __early_init_dt_declare_initrd as long as we have a path to removing > it like the above option. I think arm64 does not have to redefine __early_init_dt_declare_initrd(). Something like this might be just all we need (completely untested, probably it won't even compile): diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 9d9582c..e9ca238 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -62,6 +62,9 @@ s64 memstart_addr __ro_after_init = -1; phys_addr_t arm64_dma_phys_limit __ro_after_init; #ifdef CONFIG_BLK_DEV_INITRD + +static phys_addr_t initrd_start_phys, initrd_end_phys; + static int __init early_initrd(char *p) { unsigned long start, size; @@ -71,8 +74,8 @@ static int __init early_initrd(char *p) if (*endp == ',') { size = memparse(endp + 1, NULL); - initrd_start = start; - initrd_end = start + size; + initrd_start_phys = start; + initrd_end_phys = end; } return 0; } @@ -407,14 +410,27 @@ void __init arm64_memblock_init(void) memblock_add(__pa_symbol(_text), (u64)(_end - _text)); } - if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && initrd_start) { + if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && + (initrd_start || initrd_start_phys)) { + /* + * FIXME: ensure proper precendence between + * early_initrd and DT when both are present + */ + if (initrd_start) { + initrd_start_phys = __phys_to_virt(initrd_start); + initrd_end_phys = __phys_to_virt(initrd_end); + } else if (initrd_start_phys) { + initrd_start = __va(initrd_start_phys); + initrd_end = __va(initrd_start_phys); + } + /* * Add back the memory we just removed if it results in the * initrd to become inaccessible via the linear mapping. * Otherwise, this is a no-op */ - u64 base = initrd_start & PAGE_MASK; - u64 size = PAGE_ALIGN(initrd_end) - base; + u64 base = initrd_start_phys & PAGE_MASK; + u64 size = PAGE_ALIGN(initrd_end_phys) - base; /* * We can only add back the initrd memory if we don't end up @@ -458,7 +474,7 @@ void __init arm64_memblock_init(void) * pagetables with memblock. */ memblock_reserve(__pa_symbol(_text), _end - _text); -#ifdef CONFIG_BLK_DEV_INITRD +#if 0 if (initrd_start) { memblock_reserve(initrd_start, initrd_end - initrd_start); > Rob >
On Thu, Oct 25, 2018 at 10:38:34AM +0100, Mike Rapoport wrote: > On Wed, Oct 24, 2018 at 02:55:17PM -0500, Rob Herring wrote: > > On Wed, Oct 24, 2018 at 2:33 PM Florian Fainelli <f.fainelli@gmail.com> wrote: > > > > > > Hi all, > > > > > > While investigating why ARM64 required a ton of objects to be rebuilt > > > when toggling CONFIG_DEV_BLK_INITRD, it became clear that this was > > > because we define __early_init_dt_declare_initrd() differently and we do > > > that in arch/arm64/include/asm/memory.h which gets included by a fair > > > amount of other header files, and translation units as well. > > > > I scratch my head sometimes as to why some config options rebuild so > > much stuff. One down, ? to go. :) > > > > > Changing the value of CONFIG_DEV_BLK_INITRD is a common thing with build > > > systems that generate two kernels: one with the initramfs and one > > > without. buildroot is one of these build systems, OpenWrt is also > > > another one that does this. > > > > > > This patch series proposes adding an empty initrd.h to satisfy the need > > > for drivers/of/fdt.c to unconditionally include that file, and moves the > > > custom __early_init_dt_declare_initrd() definition away from > > > asm/memory.h > > > > > > This cuts the number of objects rebuilds from 1920 down to 26, so a > > > factor 73 approximately. > > > > > > Apologies for the long CC list, please let me know how you would go > > > about merging that and if another approach would be preferable, e.g: > > > introducing a CONFIG_ARCH_INITRD_BELOW_START_OK Kconfig option or > > > something like that. > > > > There may be a better way as of 4.20 because bootmem is now gone and > > only memblock is used. This should unify what each arch needs to do > > with initrd early. We need the physical address early for memblock > > reserving. Then later on we need the virtual address to access the > > initrd. Perhaps we should just change initrd_start and initrd_end to > > physical addresses (or add 2 new variables would be less invasive and > > allow for different translation than __va()). The sanity checks and > > memblock reserve could also perhaps be moved to a common location. > > > > Alternatively, given arm64 is the only oddball, I'd be fine with an > > "if (IS_ENABLED(CONFIG_ARM64))" condition in the default > > __early_init_dt_declare_initrd as long as we have a path to removing > > it like the above option. > > I think arm64 does not have to redefine __early_init_dt_declare_initrd(). > Something like this might be just all we need (completely untested, > probably it won't even compile): The alternative solution would be to replace initrd_start/initrd_end with physical address versions of these everywhere - that's what we're passed from DT, it's what 32-bit ARM would prefer, and seemingly what 64-bit ARM would also like as well. Grepping for initrd_start in arch/*/mm shows that there's lots of architectures that have virtual/physical conversions on these, and a number that have obviously been derived from 32-bit ARM's approach (with maintaining a phys_initrd_start variable to simplify things).
+Ard On Thu, Oct 25, 2018 at 4:38 AM Mike Rapoport <rppt@linux.ibm.com> wrote: > > On Wed, Oct 24, 2018 at 02:55:17PM -0500, Rob Herring wrote: > > On Wed, Oct 24, 2018 at 2:33 PM Florian Fainelli <f.fainelli@gmail.com> wrote: > > > > > > Hi all, > > > > > > While investigating why ARM64 required a ton of objects to be rebuilt > > > when toggling CONFIG_DEV_BLK_INITRD, it became clear that this was > > > because we define __early_init_dt_declare_initrd() differently and we do > > > that in arch/arm64/include/asm/memory.h which gets included by a fair > > > amount of other header files, and translation units as well. > > > > I scratch my head sometimes as to why some config options rebuild so > > much stuff. One down, ? to go. :) > > > > > Changing the value of CONFIG_DEV_BLK_INITRD is a common thing with build > > > systems that generate two kernels: one with the initramfs and one > > > without. buildroot is one of these build systems, OpenWrt is also > > > another one that does this. > > > > > > This patch series proposes adding an empty initrd.h to satisfy the need > > > for drivers/of/fdt.c to unconditionally include that file, and moves the > > > custom __early_init_dt_declare_initrd() definition away from > > > asm/memory.h > > > > > > This cuts the number of objects rebuilds from 1920 down to 26, so a > > > factor 73 approximately. > > > > > > Apologies for the long CC list, please let me know how you would go > > > about merging that and if another approach would be preferable, e.g: > > > introducing a CONFIG_ARCH_INITRD_BELOW_START_OK Kconfig option or > > > something like that. > > > > There may be a better way as of 4.20 because bootmem is now gone and > > only memblock is used. This should unify what each arch needs to do > > with initrd early. We need the physical address early for memblock > > reserving. Then later on we need the virtual address to access the > > initrd. Perhaps we should just change initrd_start and initrd_end to > > physical addresses (or add 2 new variables would be less invasive and > > allow for different translation than __va()). The sanity checks and > > memblock reserve could also perhaps be moved to a common location. > > > > Alternatively, given arm64 is the only oddball, I'd be fine with an > > "if (IS_ENABLED(CONFIG_ARM64))" condition in the default > > __early_init_dt_declare_initrd as long as we have a path to removing > > it like the above option. > > I think arm64 does not have to redefine __early_init_dt_declare_initrd(). > Something like this might be just all we need (completely untested, > probably it won't even compile): > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c > index 9d9582c..e9ca238 100644 > --- a/arch/arm64/mm/init.c > +++ b/arch/arm64/mm/init.c > @@ -62,6 +62,9 @@ s64 memstart_addr __ro_after_init = -1; > phys_addr_t arm64_dma_phys_limit __ro_after_init; > > #ifdef CONFIG_BLK_DEV_INITRD > + > +static phys_addr_t initrd_start_phys, initrd_end_phys; > + > static int __init early_initrd(char *p) > { > unsigned long start, size; > @@ -71,8 +74,8 @@ static int __init early_initrd(char *p) > if (*endp == ',') { > size = memparse(endp + 1, NULL); > > - initrd_start = start; > - initrd_end = start + size; > + initrd_start_phys = start; > + initrd_end_phys = end; > } > return 0; > } > @@ -407,14 +410,27 @@ void __init arm64_memblock_init(void) > memblock_add(__pa_symbol(_text), (u64)(_end - _text)); > } > > - if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && initrd_start) { > + if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && > + (initrd_start || initrd_start_phys)) { > + /* > + * FIXME: ensure proper precendence between > + * early_initrd and DT when both are present Command line takes precedence, so just reverse the order. > + */ > + if (initrd_start) { > + initrd_start_phys = __phys_to_virt(initrd_start); > + initrd_end_phys = __phys_to_virt(initrd_end); AIUI, the original issue was doing the P2V translation was happening too early and the VA could be wrong if the linear range is adjusted. So I don't think this would work. I suppose you could convert the VA back to a PA before any adjustments and then back to a VA again after. But that's kind of hacky. 2 wrongs making a right. > + } else if (initrd_start_phys) { > + initrd_start = __va(initrd_start_phys); > + initrd_end = __va(initrd_start_phys); > + } > + > /* > * Add back the memory we just removed if it results in the > * initrd to become inaccessible via the linear mapping. > * Otherwise, this is a no-op > */ > - u64 base = initrd_start & PAGE_MASK; > - u64 size = PAGE_ALIGN(initrd_end) - base; > + u64 base = initrd_start_phys & PAGE_MASK; > + u64 size = PAGE_ALIGN(initrd_end_phys) - base; > > /* > * We can only add back the initrd memory if we don't end up > @@ -458,7 +474,7 @@ void __init arm64_memblock_init(void) > * pagetables with memblock. > */ > memblock_reserve(__pa_symbol(_text), _end - _text); > -#ifdef CONFIG_BLK_DEV_INITRD > +#if 0 > if (initrd_start) { > memblock_reserve(initrd_start, initrd_end - initrd_start); > > > > Rob > > > > -- > Sincerely yours, > Mike. >
On Thu, Oct 25, 2018 at 08:15:15AM -0500, Rob Herring wrote: > +Ard > > On Thu, Oct 25, 2018 at 4:38 AM Mike Rapoport <rppt@linux.ibm.com> wrote: > > > > On Wed, Oct 24, 2018 at 02:55:17PM -0500, Rob Herring wrote: > > > On Wed, Oct 24, 2018 at 2:33 PM Florian Fainelli <f.fainelli@gmail.com> wrote: > > > > > > > > Hi all, > > > > > > > > While investigating why ARM64 required a ton of objects to be rebuilt > > > > when toggling CONFIG_DEV_BLK_INITRD, it became clear that this was > > > > because we define __early_init_dt_declare_initrd() differently and we do > > > > that in arch/arm64/include/asm/memory.h which gets included by a fair > > > > amount of other header files, and translation units as well. > > > > > > I scratch my head sometimes as to why some config options rebuild so > > > much stuff. One down, ? to go. :) > > > > > > > Changing the value of CONFIG_DEV_BLK_INITRD is a common thing with build > > > > systems that generate two kernels: one with the initramfs and one > > > > without. buildroot is one of these build systems, OpenWrt is also > > > > another one that does this. > > > > > > > > This patch series proposes adding an empty initrd.h to satisfy the need > > > > for drivers/of/fdt.c to unconditionally include that file, and moves the > > > > custom __early_init_dt_declare_initrd() definition away from > > > > asm/memory.h > > > > > > > > This cuts the number of objects rebuilds from 1920 down to 26, so a > > > > factor 73 approximately. > > > > > > > > Apologies for the long CC list, please let me know how you would go > > > > about merging that and if another approach would be preferable, e.g: > > > > introducing a CONFIG_ARCH_INITRD_BELOW_START_OK Kconfig option or > > > > something like that. > > > > > > There may be a better way as of 4.20 because bootmem is now gone and > > > only memblock is used. This should unify what each arch needs to do > > > with initrd early. We need the physical address early for memblock > > > reserving. Then later on we need the virtual address to access the > > > initrd. Perhaps we should just change initrd_start and initrd_end to > > > physical addresses (or add 2 new variables would be less invasive and > > > allow for different translation than __va()). The sanity checks and > > > memblock reserve could also perhaps be moved to a common location. > > > > > > Alternatively, given arm64 is the only oddball, I'd be fine with an > > > "if (IS_ENABLED(CONFIG_ARM64))" condition in the default > > > __early_init_dt_declare_initrd as long as we have a path to removing > > > it like the above option. > > > > I think arm64 does not have to redefine __early_init_dt_declare_initrd(). > > Something like this might be just all we need (completely untested, > > probably it won't even compile): > > > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c > > index 9d9582c..e9ca238 100644 > > --- a/arch/arm64/mm/init.c > > +++ b/arch/arm64/mm/init.c > > @@ -62,6 +62,9 @@ s64 memstart_addr __ro_after_init = -1; > > phys_addr_t arm64_dma_phys_limit __ro_after_init; > > > > #ifdef CONFIG_BLK_DEV_INITRD > > + > > +static phys_addr_t initrd_start_phys, initrd_end_phys; > > + > > static int __init early_initrd(char *p) > > { > > unsigned long start, size; > > @@ -71,8 +74,8 @@ static int __init early_initrd(char *p) > > if (*endp == ',') { > > size = memparse(endp + 1, NULL); > > > > - initrd_start = start; > > - initrd_end = start + size; > > + initrd_start_phys = start; > > + initrd_end_phys = end; > > } > > return 0; > > } > > @@ -407,14 +410,27 @@ void __init arm64_memblock_init(void) > > memblock_add(__pa_symbol(_text), (u64)(_end - _text)); > > } > > > > - if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && initrd_start) { > > + if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && > > + (initrd_start || initrd_start_phys)) { > > + /* > > + * FIXME: ensure proper precendence between > > + * early_initrd and DT when both are present > > Command line takes precedence, so just reverse the order. > > > + */ > > + if (initrd_start) { > > + initrd_start_phys = __phys_to_virt(initrd_start); > > + initrd_end_phys = __phys_to_virt(initrd_end); > > AIUI, the original issue was doing the P2V translation was happening > too early and the VA could be wrong if the linear range is adjusted. > So I don't think this would work. Probably things have changed since then, but in the current code there is initrd_start = __phys_to_virt(initrd_start); and in between only the code related to CONFIG_RANDOMIZE_BASE, so I believe it's safe to use __phys_to_virt() here as well. > I suppose you could convert the VA back to a PA before any adjustments > and then back to a VA again after. But that's kind of hacky. 2 wrongs > making a right. > > > + } else if (initrd_start_phys) { > > + initrd_start = __va(initrd_start_phys); > > + initrd_end = __va(initrd_start_phys); > > + } > > + > > /* > > * Add back the memory we just removed if it results in the > > * initrd to become inaccessible via the linear mapping. > > * Otherwise, this is a no-op > > */ > > - u64 base = initrd_start & PAGE_MASK; > > - u64 size = PAGE_ALIGN(initrd_end) - base; > > + u64 base = initrd_start_phys & PAGE_MASK; > > + u64 size = PAGE_ALIGN(initrd_end_phys) - base; > > > > /* > > * We can only add back the initrd memory if we don't end up > > @@ -458,7 +474,7 @@ void __init arm64_memblock_init(void) > > * pagetables with memblock. > > */ > > memblock_reserve(__pa_symbol(_text), _end - _text); > > -#ifdef CONFIG_BLK_DEV_INITRD > > +#if 0 > > if (initrd_start) { > > memblock_reserve(initrd_start, initrd_end - initrd_start); > > > > > > > Rob > > > > > > > -- > > Sincerely yours, > > Mike. > > >
On Thu, Oct 25, 2018 at 12:30 PM Mike Rapoport <rppt@linux.ibm.com> wrote: > > On Thu, Oct 25, 2018 at 08:15:15AM -0500, Rob Herring wrote: > > +Ard > > > > On Thu, Oct 25, 2018 at 4:38 AM Mike Rapoport <rppt@linux.ibm.com> wrote: > > > > > > On Wed, Oct 24, 2018 at 02:55:17PM -0500, Rob Herring wrote: > > > > On Wed, Oct 24, 2018 at 2:33 PM Florian Fainelli <f.fainelli@gmail.com> wrote: > > > > > > > > > > Hi all, > > > > > > > > > > While investigating why ARM64 required a ton of objects to be rebuilt > > > > > when toggling CONFIG_DEV_BLK_INITRD, it became clear that this was > > > > > because we define __early_init_dt_declare_initrd() differently and we do > > > > > that in arch/arm64/include/asm/memory.h which gets included by a fair > > > > > amount of other header files, and translation units as well. > > > > > > > > I scratch my head sometimes as to why some config options rebuild so > > > > much stuff. One down, ? to go. :) > > > > > > > > > Changing the value of CONFIG_DEV_BLK_INITRD is a common thing with build > > > > > systems that generate two kernels: one with the initramfs and one > > > > > without. buildroot is one of these build systems, OpenWrt is also > > > > > another one that does this. > > > > > > > > > > This patch series proposes adding an empty initrd.h to satisfy the need > > > > > for drivers/of/fdt.c to unconditionally include that file, and moves the > > > > > custom __early_init_dt_declare_initrd() definition away from > > > > > asm/memory.h > > > > > > > > > > This cuts the number of objects rebuilds from 1920 down to 26, so a > > > > > factor 73 approximately. > > > > > > > > > > Apologies for the long CC list, please let me know how you would go > > > > > about merging that and if another approach would be preferable, e.g: > > > > > introducing a CONFIG_ARCH_INITRD_BELOW_START_OK Kconfig option or > > > > > something like that. > > > > > > > > There may be a better way as of 4.20 because bootmem is now gone and > > > > only memblock is used. This should unify what each arch needs to do > > > > with initrd early. We need the physical address early for memblock > > > > reserving. Then later on we need the virtual address to access the > > > > initrd. Perhaps we should just change initrd_start and initrd_end to > > > > physical addresses (or add 2 new variables would be less invasive and > > > > allow for different translation than __va()). The sanity checks and > > > > memblock reserve could also perhaps be moved to a common location. > > > > > > > > Alternatively, given arm64 is the only oddball, I'd be fine with an > > > > "if (IS_ENABLED(CONFIG_ARM64))" condition in the default > > > > __early_init_dt_declare_initrd as long as we have a path to removing > > > > it like the above option. > > > > > > I think arm64 does not have to redefine __early_init_dt_declare_initrd(). > > > Something like this might be just all we need (completely untested, > > > probably it won't even compile): > > > > > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c > > > index 9d9582c..e9ca238 100644 > > > --- a/arch/arm64/mm/init.c > > > +++ b/arch/arm64/mm/init.c > > > @@ -62,6 +62,9 @@ s64 memstart_addr __ro_after_init = -1; > > > phys_addr_t arm64_dma_phys_limit __ro_after_init; > > > > > > #ifdef CONFIG_BLK_DEV_INITRD > > > + > > > +static phys_addr_t initrd_start_phys, initrd_end_phys; > > > + > > > static int __init early_initrd(char *p) > > > { > > > unsigned long start, size; > > > @@ -71,8 +74,8 @@ static int __init early_initrd(char *p) > > > if (*endp == ',') { > > > size = memparse(endp + 1, NULL); > > > > > > - initrd_start = start; > > > - initrd_end = start + size; > > > + initrd_start_phys = start; > > > + initrd_end_phys = end; > > > } > > > return 0; > > > } > > > @@ -407,14 +410,27 @@ void __init arm64_memblock_init(void) > > > memblock_add(__pa_symbol(_text), (u64)(_end - _text)); > > > } > > > > > > - if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && initrd_start) { > > > + if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && > > > + (initrd_start || initrd_start_phys)) { > > > + /* > > > + * FIXME: ensure proper precendence between > > > + * early_initrd and DT when both are present > > > > Command line takes precedence, so just reverse the order. > > > > > + */ > > > + if (initrd_start) { > > > + initrd_start_phys = __phys_to_virt(initrd_start); > > > + initrd_end_phys = __phys_to_virt(initrd_end); BTW, I think you meant virt_to_phys() here? > > > > AIUI, the original issue was doing the P2V translation was happening > > too early and the VA could be wrong if the linear range is adjusted. > > So I don't think this would work. > > Probably things have changed since then, but in the current code there is > > initrd_start = __phys_to_virt(initrd_start); > > and in between only the code related to CONFIG_RANDOMIZE_BASE, so I believe > it's safe to use __phys_to_virt() here as well. Here is fine yes, but I believe it was the the phys to virt in the DT code before adjusting the linear range that was the problem. Rob
On 10/25/18 2:13 PM, Rob Herring wrote: > On Thu, Oct 25, 2018 at 12:30 PM Mike Rapoport <rppt@linux.ibm.com> wrote: >> >> On Thu, Oct 25, 2018 at 08:15:15AM -0500, Rob Herring wrote: >>> +Ard >>> >>> On Thu, Oct 25, 2018 at 4:38 AM Mike Rapoport <rppt@linux.ibm.com> wrote: >>>> >>>> On Wed, Oct 24, 2018 at 02:55:17PM -0500, Rob Herring wrote: >>>>> On Wed, Oct 24, 2018 at 2:33 PM Florian Fainelli <f.fainelli@gmail.com> wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> While investigating why ARM64 required a ton of objects to be rebuilt >>>>>> when toggling CONFIG_DEV_BLK_INITRD, it became clear that this was >>>>>> because we define __early_init_dt_declare_initrd() differently and we do >>>>>> that in arch/arm64/include/asm/memory.h which gets included by a fair >>>>>> amount of other header files, and translation units as well. >>>>> >>>>> I scratch my head sometimes as to why some config options rebuild so >>>>> much stuff. One down, ? to go. :) >>>>> >>>>>> Changing the value of CONFIG_DEV_BLK_INITRD is a common thing with build >>>>>> systems that generate two kernels: one with the initramfs and one >>>>>> without. buildroot is one of these build systems, OpenWrt is also >>>>>> another one that does this. >>>>>> >>>>>> This patch series proposes adding an empty initrd.h to satisfy the need >>>>>> for drivers/of/fdt.c to unconditionally include that file, and moves the >>>>>> custom __early_init_dt_declare_initrd() definition away from >>>>>> asm/memory.h >>>>>> >>>>>> This cuts the number of objects rebuilds from 1920 down to 26, so a >>>>>> factor 73 approximately. >>>>>> >>>>>> Apologies for the long CC list, please let me know how you would go >>>>>> about merging that and if another approach would be preferable, e.g: >>>>>> introducing a CONFIG_ARCH_INITRD_BELOW_START_OK Kconfig option or >>>>>> something like that. >>>>> >>>>> There may be a better way as of 4.20 because bootmem is now gone and >>>>> only memblock is used. This should unify what each arch needs to do >>>>> with initrd early. We need the physical address early for memblock >>>>> reserving. Then later on we need the virtual address to access the >>>>> initrd. Perhaps we should just change initrd_start and initrd_end to >>>>> physical addresses (or add 2 new variables would be less invasive and >>>>> allow for different translation than __va()). The sanity checks and >>>>> memblock reserve could also perhaps be moved to a common location. >>>>> >>>>> Alternatively, given arm64 is the only oddball, I'd be fine with an >>>>> "if (IS_ENABLED(CONFIG_ARM64))" condition in the default >>>>> __early_init_dt_declare_initrd as long as we have a path to removing >>>>> it like the above option. >>>> >>>> I think arm64 does not have to redefine __early_init_dt_declare_initrd(). >>>> Something like this might be just all we need (completely untested, >>>> probably it won't even compile): >>>> >>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c >>>> index 9d9582c..e9ca238 100644 >>>> --- a/arch/arm64/mm/init.c >>>> +++ b/arch/arm64/mm/init.c >>>> @@ -62,6 +62,9 @@ s64 memstart_addr __ro_after_init = -1; >>>> phys_addr_t arm64_dma_phys_limit __ro_after_init; >>>> >>>> #ifdef CONFIG_BLK_DEV_INITRD >>>> + >>>> +static phys_addr_t initrd_start_phys, initrd_end_phys; >>>> + >>>> static int __init early_initrd(char *p) >>>> { >>>> unsigned long start, size; >>>> @@ -71,8 +74,8 @@ static int __init early_initrd(char *p) >>>> if (*endp == ',') { >>>> size = memparse(endp + 1, NULL); >>>> >>>> - initrd_start = start; >>>> - initrd_end = start + size; >>>> + initrd_start_phys = start; >>>> + initrd_end_phys = end; >>>> } >>>> return 0; >>>> } >>>> @@ -407,14 +410,27 @@ void __init arm64_memblock_init(void) >>>> memblock_add(__pa_symbol(_text), (u64)(_end - _text)); >>>> } >>>> >>>> - if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && initrd_start) { >>>> + if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && >>>> + (initrd_start || initrd_start_phys)) { >>>> + /* >>>> + * FIXME: ensure proper precendence between >>>> + * early_initrd and DT when both are present >>> >>> Command line takes precedence, so just reverse the order. >>> >>>> + */ >>>> + if (initrd_start) { >>>> + initrd_start_phys = __phys_to_virt(initrd_start); >>>> + initrd_end_phys = __phys_to_virt(initrd_end); > > BTW, I think you meant virt_to_phys() here? > >>> >>> AIUI, the original issue was doing the P2V translation was happening >>> too early and the VA could be wrong if the linear range is adjusted. >>> So I don't think this would work. >> >> Probably things have changed since then, but in the current code there is >> >> initrd_start = __phys_to_virt(initrd_start); >> >> and in between only the code related to CONFIG_RANDOMIZE_BASE, so I believe >> it's safe to use __phys_to_virt() here as well. > > Here is fine yes, but I believe it was the the phys to virt in the DT > code before adjusting the linear range that was the problem. FWIW, I am extracting the ARM implementation that parses the initrd early command line parameter and the "setup" code doing the page boundary alignment and memblock checking into a helper into lib/ that other architectures can re-use. So far, this removes the need for unicore32, arc and arm to duplicate essentially the same logic.
On Thu, Oct 25, 2018 at 04:13:10PM -0500, Rob Herring wrote: > On Thu, Oct 25, 2018 at 12:30 PM Mike Rapoport <rppt@linux.ibm.com> wrote: > > > > On Thu, Oct 25, 2018 at 08:15:15AM -0500, Rob Herring wrote: > > > +Ard > > > > > > On Thu, Oct 25, 2018 at 4:38 AM Mike Rapoport <rppt@linux.ibm.com> wrote: > > > > > > > > On Wed, Oct 24, 2018 at 02:55:17PM -0500, Rob Herring wrote: > > > > > On Wed, Oct 24, 2018 at 2:33 PM Florian Fainelli <f.fainelli@gmail.com> wrote: > > > > > > > > > > > > Hi all, > > > > > > > > > > > > While investigating why ARM64 required a ton of objects to be rebuilt > > > > > > when toggling CONFIG_DEV_BLK_INITRD, it became clear that this was > > > > > > because we define __early_init_dt_declare_initrd() differently and we do > > > > > > that in arch/arm64/include/asm/memory.h which gets included by a fair > > > > > > amount of other header files, and translation units as well. > > > > > > > > > > I scratch my head sometimes as to why some config options rebuild so > > > > > much stuff. One down, ? to go. :) > > > > > > > > > > > Changing the value of CONFIG_DEV_BLK_INITRD is a common thing with build > > > > > > systems that generate two kernels: one with the initramfs and one > > > > > > without. buildroot is one of these build systems, OpenWrt is also > > > > > > another one that does this. > > > > > > > > > > > > This patch series proposes adding an empty initrd.h to satisfy the need > > > > > > for drivers/of/fdt.c to unconditionally include that file, and moves the > > > > > > custom __early_init_dt_declare_initrd() definition away from > > > > > > asm/memory.h > > > > > > > > > > > > This cuts the number of objects rebuilds from 1920 down to 26, so a > > > > > > factor 73 approximately. > > > > > > > > > > > > Apologies for the long CC list, please let me know how you would go > > > > > > about merging that and if another approach would be preferable, e.g: > > > > > > introducing a CONFIG_ARCH_INITRD_BELOW_START_OK Kconfig option or > > > > > > something like that. > > > > > > > > > > There may be a better way as of 4.20 because bootmem is now gone and > > > > > only memblock is used. This should unify what each arch needs to do > > > > > with initrd early. We need the physical address early for memblock > > > > > reserving. Then later on we need the virtual address to access the > > > > > initrd. Perhaps we should just change initrd_start and initrd_end to > > > > > physical addresses (or add 2 new variables would be less invasive and > > > > > allow for different translation than __va()). The sanity checks and > > > > > memblock reserve could also perhaps be moved to a common location. > > > > > > > > > > Alternatively, given arm64 is the only oddball, I'd be fine with an > > > > > "if (IS_ENABLED(CONFIG_ARM64))" condition in the default > > > > > __early_init_dt_declare_initrd as long as we have a path to removing > > > > > it like the above option. > > > > > > > > I think arm64 does not have to redefine __early_init_dt_declare_initrd(). > > > > Something like this might be just all we need (completely untested, > > > > probably it won't even compile): > > > > > > > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c > > > > index 9d9582c..e9ca238 100644 > > > > --- a/arch/arm64/mm/init.c > > > > +++ b/arch/arm64/mm/init.c > > > > @@ -62,6 +62,9 @@ s64 memstart_addr __ro_after_init = -1; > > > > phys_addr_t arm64_dma_phys_limit __ro_after_init; > > > > > > > > #ifdef CONFIG_BLK_DEV_INITRD > > > > + > > > > +static phys_addr_t initrd_start_phys, initrd_end_phys; > > > > + > > > > static int __init early_initrd(char *p) > > > > { > > > > unsigned long start, size; > > > > @@ -71,8 +74,8 @@ static int __init early_initrd(char *p) > > > > if (*endp == ',') { > > > > size = memparse(endp + 1, NULL); > > > > > > > > - initrd_start = start; > > > > - initrd_end = start + size; > > > > + initrd_start_phys = start; > > > > + initrd_end_phys = end; > > > > } > > > > return 0; > > > > } > > > > @@ -407,14 +410,27 @@ void __init arm64_memblock_init(void) > > > > memblock_add(__pa_symbol(_text), (u64)(_end - _text)); > > > > } > > > > > > > > - if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && initrd_start) { > > > > + if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && > > > > + (initrd_start || initrd_start_phys)) { > > > > + /* > > > > + * FIXME: ensure proper precendence between > > > > + * early_initrd and DT when both are present > > > > > > Command line takes precedence, so just reverse the order. > > > > > > > + */ > > > > + if (initrd_start) { > > > > + initrd_start_phys = __phys_to_virt(initrd_start); > > > > + initrd_end_phys = __phys_to_virt(initrd_end); > > BTW, I think you meant virt_to_phys() here? Right, and then there is no problem at all do the conversion here :) > > > > > > AIUI, the original issue was doing the P2V translation was happening > > > too early and the VA could be wrong if the linear range is adjusted. > > > So I don't think this would work. > > > > Probably things have changed since then, but in the current code there is > > > > initrd_start = __phys_to_virt(initrd_start); > > > > and in between only the code related to CONFIG_RANDOMIZE_BASE, so I believe > > it's safe to use __phys_to_virt() here as well. > > Here is fine yes, but I believe it was the the phys to virt in the DT > code before adjusting the linear range that was the problem. > > Rob >
On Thu, Oct 25, 2018 at 04:07:13PM -0700, Florian Fainelli wrote: > On 10/25/18 2:13 PM, Rob Herring wrote: > > On Thu, Oct 25, 2018 at 12:30 PM Mike Rapoport <rppt@linux.ibm.com> wrote: > >> > >> On Thu, Oct 25, 2018 at 08:15:15AM -0500, Rob Herring wrote: > >>> +Ard > >>> > >>> On Thu, Oct 25, 2018 at 4:38 AM Mike Rapoport <rppt@linux.ibm.com> wrote: > >>>> > >>>> On Wed, Oct 24, 2018 at 02:55:17PM -0500, Rob Herring wrote: > >>>>> On Wed, Oct 24, 2018 at 2:33 PM Florian Fainelli <f.fainelli@gmail.com> wrote: > >>>>>> > >>>>>> Hi all, > >>>>>> > >>>>>> While investigating why ARM64 required a ton of objects to be rebuilt > >>>>>> when toggling CONFIG_DEV_BLK_INITRD, it became clear that this was > >>>>>> because we define __early_init_dt_declare_initrd() differently and we do > >>>>>> that in arch/arm64/include/asm/memory.h which gets included by a fair > >>>>>> amount of other header files, and translation units as well. > >>>>> > >>>> I think arm64 does not have to redefine __early_init_dt_declare_initrd(). > >>>> Something like this might be just all we need (completely untested, > >>>> probably it won't even compile): [ ... ] > FWIW, I am extracting the ARM implementation that parses the initrd > early command line parameter and the "setup" code doing the page > boundary alignment and memblock checking into a helper into lib/ that > other architectures can re-use. So far, this removes the need for > unicore32, arc and arm to duplicate essentially the same logic. Presuming you are going to need asm-generic/initrd.h for that as well, using override for __early_init_dt_declare_initrd in arm64 version of initrd.h might be the simplest option. > -- > Florian >
On 10/26/18 4:07 AM, Mike Rapoport wrote: > On Thu, Oct 25, 2018 at 04:07:13PM -0700, Florian Fainelli wrote: >> On 10/25/18 2:13 PM, Rob Herring wrote: >>> On Thu, Oct 25, 2018 at 12:30 PM Mike Rapoport <rppt@linux.ibm.com> wrote: >>>> >>>> On Thu, Oct 25, 2018 at 08:15:15AM -0500, Rob Herring wrote: >>>>> +Ard >>>>> >>>>> On Thu, Oct 25, 2018 at 4:38 AM Mike Rapoport <rppt@linux.ibm.com> wrote: >>>>>> >>>>>> On Wed, Oct 24, 2018 at 02:55:17PM -0500, Rob Herring wrote: >>>>>>> On Wed, Oct 24, 2018 at 2:33 PM Florian Fainelli <f.fainelli@gmail.com> wrote: >>>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> While investigating why ARM64 required a ton of objects to be rebuilt >>>>>>>> when toggling CONFIG_DEV_BLK_INITRD, it became clear that this was >>>>>>>> because we define __early_init_dt_declare_initrd() differently and we do >>>>>>>> that in arch/arm64/include/asm/memory.h which gets included by a fair >>>>>>>> amount of other header files, and translation units as well. >>>>>>> >>>>>> I think arm64 does not have to redefine __early_init_dt_declare_initrd(). >>>>>> Something like this might be just all we need (completely untested, >>>>>> probably it won't even compile): > > [ ... ] > >> FWIW, I am extracting the ARM implementation that parses the initrd >> early command line parameter and the "setup" code doing the page >> boundary alignment and memblock checking into a helper into lib/ that >> other architectures can re-use. So far, this removes the need for >> unicore32, arc and arm to duplicate essentially the same logic. > > Presuming you are going to need asm-generic/initrd.h for that as well, > using override for __early_init_dt_declare_initrd in arm64 version of > initrd.h might be the simplest option. What I am contemplating doing is promote phys_initrd_start/phys_initrd_size to be global variables (similar to initrd_start, initrd_end) and have a generic helper function for parsing the initrd= command line parameter and finally removing __early_init_dt_declare_initrd() because we could have early_init_dt_check_for_initrd() just populate phys_initrd_start/phys_initrd_size directly as well as initrd_start/initrd_end using __va() to preserve compatibility with architectures that rely on this. Then I would convert ARM64 to check for phys_initrd_start which is really what it is is trying to do in arch/arm64/mm/init.c::arm64_memblock_init(). Does that sound like a reasonable approach?
Hi all, While investigating why ARM64 required a ton of objects to be rebuilt when toggling CONFIG_DEV_BLK_INITRD, it became clear that this was because we define __early_init_dt_declare_initrd() differently and we do that in arch/arm64/include/asm/memory.h which gets included by a fair amount of other header files, and translation units as well. Changing the value of CONFIG_DEV_BLK_INITRD is a common thing with build systems that generate two kernels: one with the initramfs and one without. buildroot is one of these build systems, OpenWrt is also another one that does this. This patch series proposes adding an empty initrd.h to satisfy the need for drivers/of/fdt.c to unconditionally include that file, and moves the custom __early_init_dt_declare_initrd() definition away from asm/memory.h This cuts the number of objects rebuilds from 1920 down to 26, so a factor 73 approximately. Apologies for the long CC list, please let me know how you would go about merging that and if another approach would be preferable, e.g: introducing a CONFIG_ARCH_INITRD_BELOW_START_OK Kconfig option or something like that. Changes in v2: - put an /* empty */ comment in the asm-generic/initrd.h file - trim down the CC list to maximize the chances of people receiving this Florian Fainelli (2): arch: Add asm-generic/initrd.h and make use of it for most architectures arm64: Create asm/initrd.h arch/alpha/include/asm/Kbuild | 1 + arch/arc/include/asm/Kbuild | 1 + arch/arm/include/asm/Kbuild | 1 + arch/arm64/include/asm/initrd.h | 13 +++++++++++++ arch/arm64/include/asm/memory.h | 8 -------- arch/c6x/include/asm/Kbuild | 1 + arch/h8300/include/asm/Kbuild | 1 + arch/hexagon/include/asm/Kbuild | 1 + arch/ia64/include/asm/Kbuild | 1 + arch/m68k/include/asm/Kbuild | 1 + arch/microblaze/include/asm/Kbuild | 1 + arch/mips/include/asm/Kbuild | 1 + arch/nds32/include/asm/Kbuild | 1 + arch/nios2/include/asm/Kbuild | 1 + arch/openrisc/include/asm/Kbuild | 1 + arch/parisc/include/asm/Kbuild | 1 + arch/powerpc/include/asm/Kbuild | 1 + arch/riscv/include/asm/Kbuild | 1 + arch/s390/include/asm/Kbuild | 1 + arch/sh/include/asm/Kbuild | 1 + arch/sparc/include/asm/Kbuild | 1 + arch/um/include/asm/Kbuild | 1 + arch/unicore32/include/asm/Kbuild | 1 + arch/x86/include/asm/Kbuild | 1 + arch/xtensa/include/asm/Kbuild | 1 + drivers/of/fdt.c | 1 + include/asm-generic/initrd.h | 1 + 27 files changed, 38 insertions(+), 8 deletions(-) create mode 100644 arch/arm64/include/asm/initrd.h create mode 100644 include/asm-generic/initrd.h