[RFC/RFT,1/5] s390: make crashk_res resource a child of "System RAM"

Commit Message

Mike Rapoport May 31, 2021, 12:29 p.m. UTC

From: Mike Rapoport <[e-mail]>

Commit 4e042af463f8 ("s390/kexec: fix crash on resize of reserved memory")
added a comment that says "crash kernel resource should not be part of the
System RAM resource" but never explained why. As it looks from the code in
the kernel and in kexec there is no actual reason for that.

Keeping crashk_res inline with other resources makes code simpler and
cleaner, and allows future consolidation of the resources setup across
several architectures.

Signed-off-by: Mike Rapoport <[e-mail]>
---
 arch/s390/kernel/setup.c | 21 +++++----------------
 1 file changed, 5 insertions(+), 16 deletions(-)

Comments

David Hildenbrand June 1, 2021, 8:45 a.m. UTC | #1

On 31.05.21 14:29, Mike Rapoport wrote:
> From: Mike Rapoport <[e-mail]>
> 
> Commit 4e042af463f8 ("s390/kexec: fix crash on resize of reserved memory")
> added a comment that says "crash kernel resource should not be part of the
> System RAM resource" but never explained why. As it looks from the code in
> the kernel and in kexec there is no actual reason for that.

Are you sure?

Looking at kexec-tools: kexec/arch/s390/kexec-s390.c

get_memory_ranges_s390() wants "System RAM" and Crash kernel only with 
"with_crashk=1". Your patch would change that. "Crash kernel" would 
always be included if you make it a child of "System RAM".

Further, get_memory_ranges() and is_crashkernel_mem_reserved() look out 
for "Crash kernel\n" via parse_iomem_single().

However, parse_iomem_single() does not care about ranges that start with 
spaces IIRC via
   sscanf(line, "%llx-%llx : %n" ...

So once you make "Crash kernel" a child of "System RAM", kexec-tools 
would break if I'm not completely wrong.

David Hildenbrand June 1, 2021, 9:02 a.m. UTC | #2

On 01.06.21 10:45, David Hildenbrand wrote:
> On 31.05.21 14:29, Mike Rapoport wrote:
>> From: Mike Rapoport <[e-mail]>
>>
>> Commit 4e042af463f8 ("s390/kexec: fix crash on resize of reserved memory")
>> added a comment that says "crash kernel resource should not be part of the
>> System RAM resource" but never explained why. As it looks from the code in
>> the kernel and in kexec there is no actual reason for that.
> 
> Are you sure?
> 
> Looking at kexec-tools: kexec/arch/s390/kexec-s390.c
> 
> get_memory_ranges_s390() wants "System RAM" and Crash kernel only with
> "with_crashk=1". Your patch would change that. "Crash kernel" would
> always be included if you make it a child of "System RAM".
> 
> Further, get_memory_ranges() and is_crashkernel_mem_reserved() look out
> for "Crash kernel\n" via parse_iomem_single().
> 
> However, parse_iomem_single() does not care about ranges that start with
> spaces IIRC via
>     sscanf(line, "%llx-%llx : %n" ...

I think I'm wrong about that one because I read

"Input white-space characters (as specified by the isspace function) are 
skipped, unless the specification includes a [ , c , or n specifier"

So having it as a child won't affect parse_iomem_single().

Gerald Schaefer June 1, 2021, 1:18 p.m. UTC | #3

On Mon, 31 May 2021 15:29:55 +0300
Mike Rapoport <[e-mail]> wrote:

> From: Mike Rapoport <[e-mail]>
> 
> Commit 4e042af463f8 ("s390/kexec: fix crash on resize of reserved memory")
> added a comment that says "crash kernel resource should not be part of the
> System RAM resource" but never explained why. As it looks from the code in
> the kernel and in kexec there is no actual reason for that.

Still testing, but so far everything works fine.

> 
> Keeping crashk_res inline with other resources makes code simpler and
> cleaner, and allows future consolidation of the resources setup across
> several architectures.
> 
> Signed-off-by: Mike Rapoport <[e-mail]>
> ---
>  arch/s390/kernel/setup.c | 21 +++++----------------
>  1 file changed, 5 insertions(+), 16 deletions(-)
> 
> diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
> index 5aab59ad5688..30430e7c1b03 100644
> --- a/arch/s390/kernel/setup.c
> +++ b/arch/s390/kernel/setup.c
> @@ -500,6 +500,9 @@ static struct resource __initdata *standard_resources[] = {
>  	&code_resource,
>  	&data_resource,
>  	&bss_resource,
> +#ifdef CONFIG_CRASH_DUMP
> +	&crashk_res,
> +#endif
>  };
>  
>  static void __init setup_resources(void)
> @@ -535,7 +538,7 @@ static void __init setup_resources(void)
>  
>  		for (j = 0; j < ARRAY_SIZE(standard_resources); j++) {
>  			std_res = standard_resources[j];
> -			if (std_res->start < res->start ||
> +			if (!std_res->end || std_res->start < res->start ||
>  			    std_res->start > res->end)
>  				continue;
>  			if (std_res->end > res->end) {

Why is this extra check for !std_res->end added here? I assume it
might be needed later, after you moved this to common code, but I
cannot see how any of the other patches in this series would require
that.

> @@ -552,20 +555,6 @@ static void __init setup_resources(void)
>  			}
>  		}
>  	}
> -#ifdef CONFIG_CRASH_DUMP
> -	/*
> -	 * Re-add removed crash kernel memory as reserved memory. This makes
> -	 * sure it will be mapped with the identity mapping and struct pages
> -	 * will be created, so it can be resized later on.
> -	 * However add it later since the crash kernel resource should not be
> -	 * part of the System RAM resource.
> -	 */
> -	if (crashk_res.end) {
> -		memblock_add_node(crashk_res.start, resource_size(&crashk_res), 0);
> -		memblock_reserve(crashk_res.start, resource_size(&crashk_res));
> -		insert_resource(&iomem_resource, &crashk_res);
> -	}
> -#endif
>  }
>  
>  static void __init setup_ident_map_size(void)
> @@ -733,7 +722,7 @@ static void __init reserve_crashkernel(void)
>  		diag10_range(PFN_DOWN(crash_base), PFN_DOWN(crash_size));
>  	crashk_res.start = crash_base;
>  	crashk_res.end = crash_base + crash_size - 1;
> -	memblock_remove(crash_base, crash_size);
> +	memblock_reserve(crash_base, crash_size);
>  	pr_info("Reserving %lluMB of memory at %lluMB "
>  		"for crashkernel (System RAM: %luMB)\n",
>  		crash_size >> 20, crash_base >> 20,

Other architectures check the return value of memblock_reserve() at
this point, and exit crashkernel reservation if it fails. IIUC, the
only reason why memblock_reserve() could fail would be the same reason
why also memblock_remove() could fail, i.e. that memblock_double_array()
would fail. And since we also do not check that at the moment, your
patch would probably not (additionally) break anything.

Still, this might be something for an add-on patch (for us). Do you
happen to know how likely it would be that memblock_remove/reserve()
could fail at this point?

Mike Rapoport June 2, 2021, 6:25 a.m. UTC | #4

On Tue, Jun 01, 2021 at 11:02:17AM +0200, David Hildenbrand wrote:
> On 01.06.21 10:45, David Hildenbrand wrote:
> > On 31.05.21 14:29, Mike Rapoport wrote:
> > > From: Mike Rapoport <[e-mail]>
> > > 
> > > Commit 4e042af463f8 ("s390/kexec: fix crash on resize of reserved memory")
> > > added a comment that says "crash kernel resource should not be part of the
> > > System RAM resource" but never explained why. As it looks from the code in
> > > the kernel and in kexec there is no actual reason for that.
> > 
> > Are you sure?
> > 
> > Looking at kexec-tools: kexec/arch/s390/kexec-s390.c
> > 
> > get_memory_ranges_s390() wants "System RAM" and Crash kernel only with
> > "with_crashk=1". Your patch would change that. "Crash kernel" would
> > always be included if you make it a child of "System RAM".
> > 
> > Further, get_memory_ranges() and is_crashkernel_mem_reserved() look out
> > for "Crash kernel\n" via parse_iomem_single().
> > 
> > However, parse_iomem_single() does not care about ranges that start with
> > spaces IIRC via
> >     sscanf(line, "%llx-%llx : %n" ...
> 
> I think I'm wrong about that one because I read
> 
> "Input white-space characters (as specified by the isspace function) are
> skipped, unless the specification includes a [ , c , or n specifier"
> 
> So having it as a child won't affect parse_iomem_single().

Yes, this was my understanding as well.

Mike Rapoport June 2, 2021, 6:54 a.m. UTC | #5

On Tue, Jun 01, 2021 at 03:18:36PM +0200, Gerald Schaefer wrote:
> On Mon, 31 May 2021 15:29:55 +0300
> Mike Rapoport <[e-mail]> wrote:
> 
> > From: Mike Rapoport <[e-mail]>
> > 
> > Commit 4e042af463f8 ("s390/kexec: fix crash on resize of reserved memory")
> > added a comment that says "crash kernel resource should not be part of the
> > System RAM resource" but never explained why. As it looks from the code in
> > the kernel and in kexec there is no actual reason for that.
> 
> Still testing, but so far everything works fine.
> 
> > 
> > Keeping crashk_res inline with other resources makes code simpler and
> > cleaner, and allows future consolidation of the resources setup across
> > several architectures.
> > 
> > Signed-off-by: Mike Rapoport <[e-mail]>
> > ---
> >  arch/s390/kernel/setup.c | 21 +++++----------------
> >  1 file changed, 5 insertions(+), 16 deletions(-)
> > 
> > diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
> > index 5aab59ad5688..30430e7c1b03 100644
> > --- a/arch/s390/kernel/setup.c
> > +++ b/arch/s390/kernel/setup.c
> > @@ -500,6 +500,9 @@ static struct resource __initdata *standard_resources[] = {
> >  	&code_resource,
> >  	&data_resource,
> >  	&bss_resource,
> > +#ifdef CONFIG_CRASH_DUMP
> > +	&crashk_res,
> > +#endif
> >  };
> >  
> >  static void __init setup_resources(void)
> > @@ -535,7 +538,7 @@ static void __init setup_resources(void)
> >  
> >  		for (j = 0; j < ARRAY_SIZE(standard_resources); j++) {
> >  			std_res = standard_resources[j];
> > -			if (std_res->start < res->start ||
> > +			if (!std_res->end || std_res->start < res->start ||
> >  			    std_res->start > res->end)
> >  				continue;
> >  			if (std_res->end > res->end) {
> 
> Why is this extra check for !std_res->end added here? I assume it
> might be needed later, after you moved this to common code, but I
> cannot see how any of the other patches in this series would require
> that.

This is needed to avoid requesting empty crash_res.
 
> > @@ -552,20 +555,6 @@ static void __init setup_resources(void)
> >  			}
> >  		}
> >  	}
> > -#ifdef CONFIG_CRASH_DUMP
> > -	/*
> > -	 * Re-add removed crash kernel memory as reserved memory. This makes
> > -	 * sure it will be mapped with the identity mapping and struct pages
> > -	 * will be created, so it can be resized later on.
> > -	 * However add it later since the crash kernel resource should not be
> > -	 * part of the System RAM resource.
> > -	 */
> > -	if (crashk_res.end) {
> > -		memblock_add_node(crashk_res.start, resource_size(&crashk_res), 0);
> > -		memblock_reserve(crashk_res.start, resource_size(&crashk_res));
> > -		insert_resource(&iomem_resource, &crashk_res);
> > -	}
> > -#endif
> >  }
> >  
> >  static void __init setup_ident_map_size(void)
> > @@ -733,7 +722,7 @@ static void __init reserve_crashkernel(void)
> >  		diag10_range(PFN_DOWN(crash_base), PFN_DOWN(crash_size));
> >  	crashk_res.start = crash_base;
> >  	crashk_res.end = crash_base + crash_size - 1;
> > -	memblock_remove(crash_base, crash_size);
> > +	memblock_reserve(crash_base, crash_size);
> >  	pr_info("Reserving %lluMB of memory at %lluMB "
> >  		"for crashkernel (System RAM: %luMB)\n",
> >  		crash_size >> 20, crash_base >> 20,
> 
> Other architectures check the return value of memblock_reserve() at
> this point, and exit crashkernel reservation if it fails. IIUC, the
> only reason why memblock_reserve() could fail would be the same reason
> why also memblock_remove() could fail, i.e. that memblock_double_array()
> would fail. And since we also do not check that at the moment, your
> patch would probably not (additionally) break anything.

We don't check the return value of memblock_reserve() (or memblock_remove()
for that matter) in vast majority of cases all over the place, but it is
very much unlikely it will fail because there is not enough memory.
 
> Still, this might be something for an add-on patch (for us). Do you
> happen to know how likely it would be that memblock_remove/reserve()
> could fail at this point?

Generally, the worst case scenario is that memblock_remove/reserve() will
need to double one of the memblock arrays. This will require to allocate
several kilobytes of memory, but since the memory this early is mostly free
it should not be a problem.

Looking particularly at s390 case, there are handful of reservations before
setup_resouces(), so we can accommodate more than 100 calls to
memblock_reserve() before we'll need to increase memblock.reserved array.

Patch

diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
index 5aab59ad5688..30430e7c1b03 100644
--- a/arch/s390/kernel/setup.c
+++ b/arch/s390/kernel/setup.c
@@ -500,6 +500,9 @@  static struct resource __initdata *standard_resources[] = {
 	&code_resource,
 	&data_resource,
 	&bss_resource,
+#ifdef CONFIG_CRASH_DUMP
+	&crashk_res,
+#endif
 };
 
 static void __init setup_resources(void)
@@ -535,7 +538,7 @@  static void __init setup_resources(void)
 
 		for (j = 0; j < ARRAY_SIZE(standard_resources); j++) {
 			std_res = standard_resources[j];
-			if (std_res->start < res->start ||
+			if (!std_res->end || std_res->start < res->start ||
 			    std_res->start > res->end)
 				continue;
 			if (std_res->end > res->end) {
@@ -552,20 +555,6 @@  static void __init setup_resources(void)
 			}
 		}
 	}
-#ifdef CONFIG_CRASH_DUMP
-	/*
-	 * Re-add removed crash kernel memory as reserved memory. This makes
-	 * sure it will be mapped with the identity mapping and struct pages
-	 * will be created, so it can be resized later on.
-	 * However add it later since the crash kernel resource should not be
-	 * part of the System RAM resource.
-	 */
-	if (crashk_res.end) {
-		memblock_add_node(crashk_res.start, resource_size(&crashk_res), 0);
-		memblock_reserve(crashk_res.start, resource_size(&crashk_res));
-		insert_resource(&iomem_resource, &crashk_res);
-	}
-#endif
 }
 
 static void __init setup_ident_map_size(void)
@@ -733,7 +722,7 @@  static void __init reserve_crashkernel(void)
 		diag10_range(PFN_DOWN(crash_base), PFN_DOWN(crash_size));
 	crashk_res.start = crash_base;
 	crashk_res.end = crash_base + crash_size - 1;
-	memblock_remove(crash_base, crash_size);
+	memblock_reserve(crash_base, crash_size);
 	pr_info("Reserving %lluMB of memory at %lluMB "
 		"for crashkernel (System RAM: %luMB)\n",
 		crash_size >> 20, crash_base >> 20,