| Message ID | 20230625140931.1266216-2-songshuaishuai@tinylab.org (mailing list archive) |
|---|---|
| State | Rejected |
| Headers | show |
| Series | Revert huge-paged linear mapping and its related fixups | expand |
| Context | Check | Description |
|---|---|---|
| conchuod/tree_selection | fail | Failed to apply to next/pending-fixes, riscv/for-next or riscv/master |
On Sun, Jun 25, 2023 at 10:09:29PM +0800, Song Shuai wrote: > This reverts commit ed309ce522185583b163bd0c74f0d9f299fe1826. > > With the commit 3335068f8721 ("riscv: Use PUD/P4D/PGD pages for the > linear mapping") reverted, the MIN_MEMBLOCK_ADDR points the kernel > load address which was placed at a PMD boundary. > And firmware always > correctly mark resident memory, or memory protected with PMP as > per the devicetree specification and/or the UEFI specification. But this is not true? The versions of OpenSBI that you mention in your cover letter do not do this. Please explain. > So those regions will not be mapped in the linear mapping and they > can be safely saved/restored by hibernation. > > Signed-off-by: Song Shuai <songshuaishuai@tinylab.org> > --- > arch/riscv/Kconfig | 5 +---- > 1 file changed, 1 insertion(+), 4 deletions(-) > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig > index 5966ad97c30c..17b5fc7f54d4 100644 > --- a/arch/riscv/Kconfig > +++ b/arch/riscv/Kconfig > @@ -800,11 +800,8 @@ menu "Power management options" > > source "kernel/power/Kconfig" > > -# Hibernation is only possible on systems where the SBI implementation has > -# marked its reserved memory as not accessible from, or does not run > -# from the same memory as, Linux > config ARCH_HIBERNATION_POSSIBLE > - def_bool NONPORTABLE > + def_bool y > > config ARCH_HIBERNATION_HEADER > def_bool HIBERNATION > -- > 2.20.1 >
Sorry for the delayed reply, My tinylab email went something wrong, I'll use gmail in this thread. 在 2023/6/25 22:18, Conor Dooley 写道: > On Sun, Jun 25, 2023 at 10:09:29PM +0800, Song Shuai wrote: >> This reverts commit ed309ce522185583b163bd0c74f0d9f299fe1826. >> >> With the commit 3335068f8721 ("riscv: Use PUD/P4D/PGD pages for the >> linear mapping") reverted, the MIN_MEMBLOCK_ADDR points the kernel >> load address which was placed at a PMD boundary. > >> And firmware always >> correctly mark resident memory, or memory protected with PMP as >> per the devicetree specification and/or the UEFI specification. > > But this is not true? The versions of OpenSBI that you mention in your > cover letter do not do this. > Please explain. > At this time, OpenSbi [v0.8,v1.3) and edk2(RiscVVirt) indeed don't obey the DT/UEFI spec. This statement is excerpted from "Reserved memory for resident firmware" part from the upcoming riscv/boot.rst. It isn't accurate for now. How about deleting this one? Actually with 3335068f8721 reverted, the change of MIN_MEMBLOCK_ADDR can avoid the mapping of firmware memory, I will make it clear in the next version. >> So those regions will not be mapped in the linear mapping and they >> can be safely saved/restored by hibernation. >> >> Signed-off-by: Song Shuai <songshuaishuai@tinylab.org> >> --- >> arch/riscv/Kconfig | 5 +---- >> 1 file changed, 1 insertion(+), 4 deletions(-) >> >> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig >> index 5966ad97c30c..17b5fc7f54d4 100644 >> --- a/arch/riscv/Kconfig >> +++ b/arch/riscv/Kconfig >> @@ -800,11 +800,8 @@ menu "Power management options" >> >> source "kernel/power/Kconfig" >> >> -# Hibernation is only possible on systems where the SBI implementation has >> -# marked its reserved memory as not accessible from, or does not run >> -# from the same memory as, Linux >> config ARCH_HIBERNATION_POSSIBLE >> - def_bool NONPORTABLE >> + def_bool y >> >> config ARCH_HIBERNATION_HEADER >> def_bool HIBERNATION >> -- >> 2.20.1 >>
Hey, On Sun, Jun 25, 2023 at 11:09:21PM +0800, Song Shuai wrote: > Sorry for the delayed reply, It wasn't really delayed at all actually, you replied within an hour or so, AFAICT. > My tinylab email went something wrong, I'll use gmail in this thread. > > 在 2023/6/25 22:18, Conor Dooley 写道: > > On Sun, Jun 25, 2023 at 10:09:29PM +0800, Song Shuai wrote: > > > This reverts commit ed309ce522185583b163bd0c74f0d9f299fe1826. > > > > > > With the commit 3335068f8721 ("riscv: Use PUD/P4D/PGD pages for the > > > linear mapping") reverted, the MIN_MEMBLOCK_ADDR points the kernel > > > load address which was placed at a PMD boundary. > > > > > And firmware always > > > correctly mark resident memory, or memory protected with PMP as > > > per the devicetree specification and/or the UEFI specification. > > > > But this is not true? The versions of OpenSBI that you mention in your > > cover letter do not do this. > > Please explain. > > > > At this time, OpenSbi [v0.8,v1.3) and edk2(RiscVVirt) indeed don't obey the > DT/UEFI spec. This statement is excerpted from "Reserved memory for resident > firmware" part from the upcoming riscv/boot.rst. It isn't accurate for now. > How about deleting this one? It is incorrect, so it will need to be removed, yes. Unfortunately writing a doc does not fix the existing implementations :( > Actually with 3335068f8721 reverted, the change of MIN_MEMBLOCK_ADDR can > avoid the mapping of firmware memory, I will make it clear in the next > version. To be honest, I'd like to see this revert as the final commit in a series that deals with the problem by actually reserving the regions, rather than a set of reverts that go back to how we were. I was hoping that someone who cares about hibernation support would be interested in working on that - *cough* starfive *cough*, although maybe they just fixed their OpenSBI and moved on. If there were no volunteers, my intention was to add a firmware erratum that would probe the SBI implementation & version IDs, and add a firmware erratum that'd parse the DT for the offending regions and reserve them. Cheers, Conor. > > > So those regions will not be mapped in the linear mapping and they > > > can be safely saved/restored by hibernation. > > > > > > Signed-off-by: Song Shuai <songshuaishuai@tinylab.org> > > > --- > > > arch/riscv/Kconfig | 5 +---- > > > 1 file changed, 1 insertion(+), 4 deletions(-) > > > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig > > > index 5966ad97c30c..17b5fc7f54d4 100644 > > > --- a/arch/riscv/Kconfig > > > +++ b/arch/riscv/Kconfig > > > @@ -800,11 +800,8 @@ menu "Power management options" > > > source "kernel/power/Kconfig" > > > -# Hibernation is only possible on systems where the SBI implementation has > > > -# marked its reserved memory as not accessible from, or does not run > > > -# from the same memory as, Linux > > > config ARCH_HIBERNATION_POSSIBLE > > > - def_bool NONPORTABLE > > > + def_bool y > > > config ARCH_HIBERNATION_HEADER > > > def_bool HIBERNATION > > > -- > > > 2.20.1 >
On Sun, 25 Jun 2023 15:15:14 PDT (-0700), Conor Dooley wrote: > Hey, > > On Sun, Jun 25, 2023 at 11:09:21PM +0800, Song Shuai wrote: > >> Sorry for the delayed reply, > > It wasn't really delayed at all actually, you replied within an hour or > so, AFAICT. > >> My tinylab email went something wrong, I'll use gmail in this thread. >> >> 在 2023/6/25 22:18, Conor Dooley 写道: >> > On Sun, Jun 25, 2023 at 10:09:29PM +0800, Song Shuai wrote: >> > > This reverts commit ed309ce522185583b163bd0c74f0d9f299fe1826. >> > > >> > > With the commit 3335068f8721 ("riscv: Use PUD/P4D/PGD pages for the >> > > linear mapping") reverted, the MIN_MEMBLOCK_ADDR points the kernel >> > > load address which was placed at a PMD boundary. >> > >> > > And firmware always >> > > correctly mark resident memory, or memory protected with PMP as >> > > per the devicetree specification and/or the UEFI specification. >> > >> > But this is not true? The versions of OpenSBI that you mention in your >> > cover letter do not do this. >> > Please explain. >> > >> >> At this time, OpenSbi [v0.8,v1.3) and edk2(RiscVVirt) indeed don't obey the >> DT/UEFI spec. This statement is excerpted from "Reserved memory for resident >> firmware" part from the upcoming riscv/boot.rst. It isn't accurate for now. >> How about deleting this one? > > It is incorrect, so it will need to be removed, yes. > Unfortunately writing a doc does not fix the existing implementations :( > >> Actually with 3335068f8721 reverted, the change of MIN_MEMBLOCK_ADDR can >> avoid the mapping of firmware memory, I will make it clear in the next >> version. > > To be honest, I'd like to see this revert as the final commit in a > series that deals with the problem by actually reserving the regions, > rather than a set of reverts that go back to how we were. > I was hoping that someone who cares about hibernation support would be > interested in working on that - *cough* starfive *cough*, although maybe > they just fixed their OpenSBI and moved on. > If there were no volunteers, my intention was to add a firmware erratum > that would probe the SBI implementation & version IDs, and add a firmware > erratum that'd parse the DT for the offending regions and reserve them. Is there any actual use case for hibernation on these boards? Maybe it's simpler to just add a "reserved regions actually work" sort of property and then have new firmware set it -- that way we can avoid sorting through all the old stuff nobody cares about and just get on with fixing the stuff people use. > > Cheers, > Conor. > >> > > So those regions will not be mapped in the linear mapping and they >> > > can be safely saved/restored by hibernation. >> > > >> > > Signed-off-by: Song Shuai <songshuaishuai@tinylab.org> >> > > --- >> > > arch/riscv/Kconfig | 5 +---- >> > > 1 file changed, 1 insertion(+), 4 deletions(-) >> > > >> > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig >> > > index 5966ad97c30c..17b5fc7f54d4 100644 >> > > --- a/arch/riscv/Kconfig >> > > +++ b/arch/riscv/Kconfig >> > > @@ -800,11 +800,8 @@ menu "Power management options" >> > > source "kernel/power/Kconfig" >> > > -# Hibernation is only possible on systems where the SBI implementation has >> > > -# marked its reserved memory as not accessible from, or does not run >> > > -# from the same memory as, Linux >> > > config ARCH_HIBERNATION_POSSIBLE >> > > - def_bool NONPORTABLE >> > > + def_bool y >> > > config ARCH_HIBERNATION_HEADER >> > > def_bool HIBERNATION >> > > -- >> > > 2.20.1 >>
On Sun, Jun 25, 2023 at 03:36:06PM -0700, Palmer Dabbelt wrote: > On Sun, 25 Jun 2023 15:15:14 PDT (-0700), Conor Dooley wrote: > > On Sun, Jun 25, 2023 at 11:09:21PM +0800, Song Shuai wrote: > > > 在 2023/6/25 22:18, Conor Dooley 写道: > > > > On Sun, Jun 25, 2023 at 10:09:29PM +0800, Song Shuai wrote: > > > > > This reverts commit ed309ce522185583b163bd0c74f0d9f299fe1826. > > > > > > > With the commit 3335068f8721 ("riscv: Use PUD/P4D/PGD pages > > > for the > > > > > linear mapping") reverted, the MIN_MEMBLOCK_ADDR points the kernel > > > > > load address which was placed at a PMD boundary. > > > > > > And firmware always > > > > > correctly mark resident memory, or memory protected with PMP as > > > > > per the devicetree specification and/or the UEFI specification. > > > > > But this is not true? The versions of OpenSBI that you mention > > > in your > > > > cover letter do not do this. > > > > Please explain. > > > > > > > > > > At this time, OpenSbi [v0.8,v1.3) and edk2(RiscVVirt) indeed don't obey the > > > DT/UEFI spec. This statement is excerpted from "Reserved memory for resident > > > firmware" part from the upcoming riscv/boot.rst. It isn't accurate for now. > > > How about deleting this one? > > > > It is incorrect, so it will need to be removed, yes. > > Unfortunately writing a doc does not fix the existing implementations :( > > > > > Actually with 3335068f8721 reverted, the change of MIN_MEMBLOCK_ADDR can > > > avoid the mapping of firmware memory, I will make it clear in the next > > > version. > > > > To be honest, I'd like to see this revert as the final commit in a > > series that deals with the problem by actually reserving the regions, > > rather than a set of reverts that go back to how we were. > > I was hoping that someone who cares about hibernation support would be > > interested in working on that - *cough* starfive *cough*, although maybe > > they just fixed their OpenSBI and moved on. > > If there were no volunteers, my intention was to add a firmware erratum > > that would probe the SBI implementation & version IDs, and add a firmware > > erratum that'd parse the DT for the offending regions and reserve them. > > Is there any actual use case for hibernation on these boards? Maybe it's > simpler to just add a "reserved regions actually work" sort of property and > then have new firmware set it -- that way we can avoid sorting through all > the old stuff nobody cares about and just get on with fixing the stuff > people use. What is "old stuff nobody cares about"? The first version of OpenSBI with the fix shipped only the other day, so effectively all current stuff has this problem. Certainly everything shipping from vendors at the moment has the problem, and probably whatever downstream, custom versions of OpenSBI also have it. Also, the problem isn't just limited to hibernation apparently. I think it was mentioned in the cover letter that according to Rob, without being marked as no-map we could also see speculative access & potentially some of the memory debugging stuff walking these regions. I'm not sure how you'd intend communicating "reserved regions actually work", I figure you mean via DT? I don't really see the benefit of adding a property for those who are behaving, if we can detect the versions of the one relevant SBI implementation that are broken at runtime. DT hat on, even less so. Perhaps I am missing your point, and there's another angle (like trying to per firmware code)?
On Mon, 26 Jun 2023 06:34:43 PDT (-0700), Conor Dooley wrote: > On Sun, Jun 25, 2023 at 03:36:06PM -0700, Palmer Dabbelt wrote: >> On Sun, 25 Jun 2023 15:15:14 PDT (-0700), Conor Dooley wrote: >> > On Sun, Jun 25, 2023 at 11:09:21PM +0800, Song Shuai wrote: >> > > 在 2023/6/25 22:18, Conor Dooley 写道: >> > > > On Sun, Jun 25, 2023 at 10:09:29PM +0800, Song Shuai wrote: >> > > > > This reverts commit ed309ce522185583b163bd0c74f0d9f299fe1826. >> > > > > > > With the commit 3335068f8721 ("riscv: Use PUD/P4D/PGD pages >> > > for the >> > > > > linear mapping") reverted, the MIN_MEMBLOCK_ADDR points the kernel >> > > > > load address which was placed at a PMD boundary. >> > > > > > And firmware always >> > > > > correctly mark resident memory, or memory protected with PMP as >> > > > > per the devicetree specification and/or the UEFI specification. >> > > > > But this is not true? The versions of OpenSBI that you mention >> > > in your >> > > > cover letter do not do this. >> > > > Please explain. >> > > > >> > > >> > > At this time, OpenSbi [v0.8,v1.3) and edk2(RiscVVirt) indeed don't obey the >> > > DT/UEFI spec. This statement is excerpted from "Reserved memory for resident >> > > firmware" part from the upcoming riscv/boot.rst. It isn't accurate for now. >> > > How about deleting this one? >> > >> > It is incorrect, so it will need to be removed, yes. >> > Unfortunately writing a doc does not fix the existing implementations :( >> > >> > > Actually with 3335068f8721 reverted, the change of MIN_MEMBLOCK_ADDR can >> > > avoid the mapping of firmware memory, I will make it clear in the next >> > > version. >> > >> > To be honest, I'd like to see this revert as the final commit in a >> > series that deals with the problem by actually reserving the regions, >> > rather than a set of reverts that go back to how we were. >> > I was hoping that someone who cares about hibernation support would be >> > interested in working on that - *cough* starfive *cough*, although maybe >> > they just fixed their OpenSBI and moved on. >> > If there were no volunteers, my intention was to add a firmware erratum >> > that would probe the SBI implementation & version IDs, and add a firmware >> > erratum that'd parse the DT for the offending regions and reserve them. >> >> Is there any actual use case for hibernation on these boards? Maybe it's >> simpler to just add a "reserved regions actually work" sort of property and >> then have new firmware set it -- that way we can avoid sorting through all >> the old stuff nobody cares about and just get on with fixing the stuff >> people use. > > What is "old stuff nobody cares about"? The first version of OpenSBI with > the fix shipped only the other day, so effectively all current stuff has > this problem. Certainly everything shipping from vendors at the moment > has the problem, and probably whatever downstream, custom versions of > OpenSBI also have it. Ya, so "old stuff" is everything -- but that's all already broken, so nothing we can do about it. IIUC there's nothing shipping that functions correctly here, so it's just a matter of detecting everything before the bug. > Also, the problem isn't just limited to hibernation apparently. I > think it was mentioned in the cover letter that according to Rob, > without being marked as no-map we could also see speculative access & > potentially some of the memory debugging stuff walking these regions. We've got a bunch of other problems around speculative accesses to these regions in M-mode, so we'll have to deal with it at some point anyway. > I'm not sure how you'd intend communicating "reserved regions actually > work", I figure you mean via DT? Somewhere in DT. I hadn't thought about it a ton, just adding some property that says "this doesn't have the bug" was roughly the idea. > I don't really see the benefit of adding a property for those who are > behaving, if we can detect the versions of the one relevant SBI > implementation that are broken at runtime. DT hat on, even less so. > Perhaps I am missing your point, and there's another angle (like trying > to per firmware code)? If it's easy to figure out which versions are broken that seems fine to me. My worry was just that's hard to do (folks forking OpenSBI) and it might be easier to just
On Mon, Jun 26, 2023 at 07:43:42AM -0700, Palmer Dabbelt wrote: > On Mon, 26 Jun 2023 06:34:43 PDT (-0700), Conor Dooley wrote: > > On Sun, Jun 25, 2023 at 03:36:06PM -0700, Palmer Dabbelt wrote: > > > On Sun, 25 Jun 2023 15:15:14 PDT (-0700), Conor Dooley wrote: > > > > On Sun, Jun 25, 2023 at 11:09:21PM +0800, Song Shuai wrote: > > > > > 在 2023/6/25 22:18, Conor Dooley 写道: > > > > > > On Sun, Jun 25, 2023 at 10:09:29PM +0800, Song Shuai wrote: > > > > > > > This reverts commit ed309ce522185583b163bd0c74f0d9f299fe1826. > > > > > > > > > With the commit 3335068f8721 ("riscv: Use PUD/P4D/PGD pages > > > > > for the > > > > > > > linear mapping") reverted, the MIN_MEMBLOCK_ADDR points the kernel > > > > > > > load address which was placed at a PMD boundary. > > > > > > > > And firmware always > > > > > > > correctly mark resident memory, or memory protected with PMP as > > > > > > > per the devicetree specification and/or the UEFI specification. > > > > > > > But this is not true? The versions of OpenSBI that you mention > > > > > in your > > > > > > cover letter do not do this. > > > > > > Please explain. > > > > > > > > > > > > > At this time, OpenSbi [v0.8,v1.3) and edk2(RiscVVirt) indeed > > > don't obey the > > > > > DT/UEFI spec. This statement is excerpted from "Reserved memory for resident > > > > > firmware" part from the upcoming riscv/boot.rst. It isn't accurate for now. > > > > > How about deleting this one? > > > > > It is incorrect, so it will need to be removed, yes. > > > > Unfortunately writing a doc does not fix the existing implementations :( > > > > > > Actually with 3335068f8721 reverted, the change of > > > MIN_MEMBLOCK_ADDR can > > > > > avoid the mapping of firmware memory, I will make it clear in the next > > > > > version. > > > > > To be honest, I'd like to see this revert as the final commit in > > > a > > > > series that deals with the problem by actually reserving the regions, > > > > rather than a set of reverts that go back to how we were. > > > > I was hoping that someone who cares about hibernation support would be > > > > interested in working on that - *cough* starfive *cough*, although maybe > > > > they just fixed their OpenSBI and moved on. > > > > If there were no volunteers, my intention was to add a firmware erratum > > > > that would probe the SBI implementation & version IDs, and add a firmware > > > > erratum that'd parse the DT for the offending regions and reserve them. > > > > > > Is there any actual use case for hibernation on these boards? Maybe it's > > > simpler to just add a "reserved regions actually work" sort of property and > > > then have new firmware set it -- that way we can avoid sorting through all > > > the old stuff nobody cares about and just get on with fixing the stuff > > > people use. > > > > What is "old stuff nobody cares about"? The first version of OpenSBI with > > the fix shipped only the other day, so effectively all current stuff has > > this problem. Certainly everything shipping from vendors at the moment > > has the problem, and probably whatever downstream, custom versions of > > OpenSBI also have it. > > Ya, so "old stuff" is everything -- but that's all already broken, so > nothing we can do about it. IIUC there's nothing shipping that functions > correctly here, so it's just a matter of detecting everything before the > bug. g5soc functions correctly, because our HSS (which a small portion of OpenSBI is built into) doesn't run out of system memory. > > Also, the problem isn't just limited to hibernation apparently. I > > think it was mentioned in the cover letter that according to Rob, > > without being marked as no-map we could also see speculative access & > > potentially some of the memory debugging stuff walking these regions. > > We've got a bunch of other problems around speculative accesses to these > regions in M-mode, so we'll have to deal with it at some point anyway. > > > I'm not sure how you'd intend communicating "reserved regions actually > > work", I figure you mean via DT? > > Somewhere in DT. I hadn't thought about it a ton, just adding some property > that says "this doesn't have the bug" was roughly the idea. > > > I don't really see the benefit of adding a property for those who are > > behaving, if we can detect the versions of the one relevant SBI > > implementation that are broken at runtime. DT hat on, even less so. > > Perhaps I am missing your point, and there's another angle (like trying > > to per firmware code)? > > If it's easy to figure out which versions are broken that seems fine to me. > My worry was just that's hard to do (folks forking OpenSBI) and it might be > easier to just You cut yourself off here :/ If people are forking OpenSBI, but reusing the version/implementation info, and changing behaviours, I am not really sure what we are supposed to do in general. I'd vote for apply the erratum to those forks, based on the version info, until they make changes That's pretty much the same thing that'd happen under your proposal anyway? I went and sent a PR last week to get a custom ID for the HSS on g5soc so that we can be differentiated, everyone else that doesn't want to be tarred with the same brush probably should too, if they are sufficiently different.
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 5966ad97c30c..17b5fc7f54d4 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -800,11 +800,8 @@ menu "Power management options" source "kernel/power/Kconfig" -# Hibernation is only possible on systems where the SBI implementation has -# marked its reserved memory as not accessible from, or does not run -# from the same memory as, Linux config ARCH_HIBERNATION_POSSIBLE - def_bool NONPORTABLE + def_bool y config ARCH_HIBERNATION_HEADER def_bool HIBERNATION
This reverts commit ed309ce522185583b163bd0c74f0d9f299fe1826. With the commit 3335068f8721 ("riscv: Use PUD/P4D/PGD pages for the linear mapping") reverted, the MIN_MEMBLOCK_ADDR points the kernel load address which was placed at a PMD boundary. And firmware always correctly mark resident memory, or memory protected with PMP as per the devicetree specification and/or the UEFI specification. So those regions will not be mapped in the linear mapping and they can be safely saved/restored by hibernation. Signed-off-by: Song Shuai <songshuaishuai@tinylab.org> --- arch/riscv/Kconfig | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)