| Message ID | 20240814092333.7727-2-yangyicong@huawei.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Support Armv8.9/v9.4 FEAT_HAFT | expand |
On Wed, Aug 14, 2024 at 05:23:32PM +0800, Yicong Yang wrote: > --- a/arch/arm64/tools/sysreg > +++ b/arch/arm64/tools/sysreg > @@ -1688,6 +1688,7 @@ UnsignedEnum 3:0 HAFDBS > 0b0000 NI > 0b0001 AF > 0b0010 DBM > + 0b0011 HAFT > EndEnum > EndSysreg Please if you're doing this update the entire sysreg to whatever the current version of the XML is, ideally as a separate commit to avoid duplication of work. It makes it easier to track any issues in the sysreg conversions.
On 2024/8/14 20:07, Mark Brown wrote: > On Wed, Aug 14, 2024 at 05:23:32PM +0800, Yicong Yang wrote: > >> --- a/arch/arm64/tools/sysreg >> +++ b/arch/arm64/tools/sysreg >> @@ -1688,6 +1688,7 @@ UnsignedEnum 3:0 HAFDBS >> 0b0000 NI >> 0b0001 AF >> 0b0010 DBM >> + 0b0011 HAFT >> EndEnum >> EndSysreg > > Please if you're doing this update the entire sysreg to whatever the > current version of the XML is, ideally as a separate commit to avoid > duplication of work. It makes it easier to track any issues in the > sysreg conversions. > Got it. Currently I only update the it according to the feature to support. Will make it into a separate patch and update the definition of both HAFT and HDBSS according to the lastest register XML. Thanks.
On Wed, Aug 14, 2024 at 05:23:32PM +0800, Yicong Yang wrote: > From: Yicong Yang <yangyicong@hisilicon.com> > > Armv8.9/v9.4 introduces the feature Hardware managed Access Flag > for Table descriptors (FEAT_HAFT). The feature is indicated by > ID_AA64MMFR1_EL1.HAFDBS == 0b0011 and can be enabled by > TCR2_EL1.HAFT so it has a dependency on FEAT_TCR2. > > This patch adds the Kconfig for FEAT_HAFT and support detecting > and enabling the feature. > > Signed-off-by: Yicong Yang <yangyicong@hisilicon.com> > --- > arch/arm64/Kconfig | 19 +++++++++++++++++++ > arch/arm64/kernel/cpufeature.c | 26 ++++++++++++++++++++++++++ > arch/arm64/tools/cpucaps | 1 + > arch/arm64/tools/sysreg | 1 + > 4 files changed, 47 insertions(+) > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig > index a2f8ff354ca6..869792458a23 100644 > --- a/arch/arm64/Kconfig > +++ b/arch/arm64/Kconfig > @@ -2137,6 +2137,25 @@ config ARM64_EPAN > if the cpu does not implement the feature. > endmenu # "ARMv8.7 architectural features" > > +menu "ARMv8.9 architectural features" > + > +config ARM64_HAFT > + bool "Support for Hardware managed Access Flag for Table Descriptor" > + depends on ARM64_HW_AFDBM > + default y > + help > + The ARMv8.9/ARMv9.5 introduces the feature Hardware managed Access > + Flag for Table descriptors. When enabled an architectural executed > + memory access will update the Access Flag in each Table descriptor > + which is accessed during the translation table walk and for which > + the Access Flag is 0. The Access Flag of the Table descriptor use > + the same bit of PTE_AF. > + > + The feature will only be enabled if all the CPUs in the system > + support this feature. If unsure, say Y. > + > +endmenu # "ARMv8.9 architectural features" > + > config ARM64_SVE > bool "ARM Scalable Vector Extension support" > default y > diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c > index 646ecd3069fd..ed4c968be935 100644 > --- a/arch/arm64/kernel/cpufeature.c > +++ b/arch/arm64/kernel/cpufeature.c > @@ -2044,6 +2044,17 @@ static bool has_hw_dbm(const struct arm64_cpu_capabilities *cap, > > #endif > > +#if CONFIG_ARM64_HAFT > + > +static void cpu_enable_haft(struct arm64_cpu_capabilities const *cap) > +{ > + sysreg_clear_set_s(SYS_TCR2_EL1, 0, TCR2_EL1x_HAFT); > + isb(); > + local_flush_tlb_all(); > +} As this isn't a per-TTBR enable, should we be initialising the kernel table entries in TTBR1 as YOUNG to avoid potential races with the hardware update? It looks like the bit is ignored on CPUs without HAFT, so we can just do this unconditionally. At the very least, we should be able to enable HAFT in __cpu_setup(), like we do for HA. > +#endif > + > #ifdef CONFIG_ARM64_AMU_EXTN > > /* > @@ -2580,6 +2591,21 @@ static const struct arm64_cpu_capabilities arm64_features[] = { > .cpus = &dbm_cpus, > ARM64_CPUID_FIELDS(ID_AA64MMFR1_EL1, HAFDBS, DBM) > }, > +#endif > +#ifdef CONFIG_ARM64_HAFT > + { > + .desc = "Hardware managed Access Flag for Table Descriptor", > + /* > + * Contrary to the page/block access flag, the table access flag > + * cannot be emulated in software (no access fault will occur). > + * Therefore mandate that all CPUs have FEAT_HAFT. > + */ It's a bit of a pity that we can't handle this mismatch. After all, access flag data is imprecise (unlike the dirty bit) and so you could envisage a mechanism for falling back to leaf-level AF at runtime rather than refusing to online a CPU. Of course, it's hard to tell whether this really matters until we see what people try to glue together. Will
On 2024/8/21 0:18, Will Deacon wrote: > On Wed, Aug 14, 2024 at 05:23:32PM +0800, Yicong Yang wrote: >> From: Yicong Yang <yangyicong@hisilicon.com> >> >> Armv8.9/v9.4 introduces the feature Hardware managed Access Flag >> for Table descriptors (FEAT_HAFT). The feature is indicated by >> ID_AA64MMFR1_EL1.HAFDBS == 0b0011 and can be enabled by >> TCR2_EL1.HAFT so it has a dependency on FEAT_TCR2. >> >> This patch adds the Kconfig for FEAT_HAFT and support detecting >> and enabling the feature. >> >> Signed-off-by: Yicong Yang <yangyicong@hisilicon.com> >> --- >> arch/arm64/Kconfig | 19 +++++++++++++++++++ >> arch/arm64/kernel/cpufeature.c | 26 ++++++++++++++++++++++++++ >> arch/arm64/tools/cpucaps | 1 + >> arch/arm64/tools/sysreg | 1 + >> 4 files changed, 47 insertions(+) >> >> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig >> index a2f8ff354ca6..869792458a23 100644 >> --- a/arch/arm64/Kconfig >> +++ b/arch/arm64/Kconfig >> @@ -2137,6 +2137,25 @@ config ARM64_EPAN >> if the cpu does not implement the feature. >> endmenu # "ARMv8.7 architectural features" >> >> +menu "ARMv8.9 architectural features" >> + >> +config ARM64_HAFT >> + bool "Support for Hardware managed Access Flag for Table Descriptor" >> + depends on ARM64_HW_AFDBM >> + default y >> + help >> + The ARMv8.9/ARMv9.5 introduces the feature Hardware managed Access >> + Flag for Table descriptors. When enabled an architectural executed >> + memory access will update the Access Flag in each Table descriptor >> + which is accessed during the translation table walk and for which >> + the Access Flag is 0. The Access Flag of the Table descriptor use >> + the same bit of PTE_AF. >> + >> + The feature will only be enabled if all the CPUs in the system >> + support this feature. If unsure, say Y. >> + >> +endmenu # "ARMv8.9 architectural features" >> + >> config ARM64_SVE >> bool "ARM Scalable Vector Extension support" >> default y >> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c >> index 646ecd3069fd..ed4c968be935 100644 >> --- a/arch/arm64/kernel/cpufeature.c >> +++ b/arch/arm64/kernel/cpufeature.c >> @@ -2044,6 +2044,17 @@ static bool has_hw_dbm(const struct arm64_cpu_capabilities *cap, >> >> #endif >> >> +#if CONFIG_ARM64_HAFT >> + >> +static void cpu_enable_haft(struct arm64_cpu_capabilities const *cap) >> +{ >> + sysreg_clear_set_s(SYS_TCR2_EL1, 0, TCR2_EL1x_HAFT); >> + isb(); >> + local_flush_tlb_all(); >> +} > > As this isn't a per-TTBR enable, should we be initialising the kernel > table entries in TTBR1 as YOUNG to avoid potential races with the > hardware update? It looks like the bit is ignored on CPUs without HAFT, > so we can just do this unconditionally. > I'm a bit uncertain about the race here, is it because of the access fault or the inconsistent observation of table entries' AF status when enable the HAFT? and... > At the very least, we should be able to enable HAFT in __cpu_setup(), > like we do for HA. > ...if we enable this in __cpu_setup() as suggested we should have no race problem, since this is enabled before MMU on? >> +#endif >> + >> #ifdef CONFIG_ARM64_AMU_EXTN >> >> /* >> @@ -2580,6 +2591,21 @@ static const struct arm64_cpu_capabilities arm64_features[] = { >> .cpus = &dbm_cpus, >> ARM64_CPUID_FIELDS(ID_AA64MMFR1_EL1, HAFDBS, DBM) >> }, >> +#endif >> +#ifdef CONFIG_ARM64_HAFT >> + { >> + .desc = "Hardware managed Access Flag for Table Descriptor", >> + /* >> + * Contrary to the page/block access flag, the table access flag >> + * cannot be emulated in software (no access fault will occur). >> + * Therefore mandate that all CPUs have FEAT_HAFT. >> + */ > > It's a bit of a pity that we can't handle this mismatch. After all, > access flag data is imprecise (unlike the dirty bit) and so you could > envisage a mechanism for falling back to leaf-level AF at runtime rather > than refusing to online a CPU. > ok. I suppose enable this should be ok if no users. So it's possible to just try to enable this at each CPU's startup, but don't advertise this feature to the upper users if not all the CPUs in the system support this. This won't affect leaf-level AF since leaf-level AF has its own detection and enabling and doesn't depend on HAFT. > Of course, it's hard to tell whether this really matters until we see > what people try to glue together. > > Will > . >
On Thu, Aug 29, 2024 at 02:29:41PM +0800, Yicong Yang wrote: > On 2024/8/21 0:18, Will Deacon wrote: > > On Wed, Aug 14, 2024 at 05:23:32PM +0800, Yicong Yang wrote: > >> +static void cpu_enable_haft(struct arm64_cpu_capabilities const *cap) > >> +{ > >> + sysreg_clear_set_s(SYS_TCR2_EL1, 0, TCR2_EL1x_HAFT); > >> + isb(); > >> + local_flush_tlb_all(); > >> +} > > > > As this isn't a per-TTBR enable, should we be initialising the kernel > > table entries in TTBR1 as YOUNG to avoid potential races with the > > hardware update? It looks like the bit is ignored on CPUs without HAFT, > > so we can just do this unconditionally. > > I'm a bit uncertain about the race here, is it because of the access fault > or the inconsistent observation of table entries' AF status when enable the > HAFT? and... Linux doesn't use this mechanism on kernel page tables. The hardware update of the access bit is atomic and, while it could race with a non-atomic entry update, it shouldn't matter since no-one checks this bit. However, I'd still set the AF bit when creating the kernel page table, it saves the hardware from having to update them at run-time. > > At the very least, we should be able to enable HAFT in __cpu_setup(), > > like we do for HA. > > > > ...if we enable this in __cpu_setup() as suggested we should have no race problem, > since this is enabled before MMU on? I suspect Will was referring to the actual page tables. But it makes sense to enable this in __cpu_setup(), it saves us from having to do the TLBI. > > It's a bit of a pity that we can't handle this mismatch. After all, > > access flag data is imprecise (unlike the dirty bit) and so you could > > envisage a mechanism for falling back to leaf-level AF at runtime rather > > than refusing to online a CPU. > > ok. I suppose enable this should be ok if no users. So it's possible to just > try to enable this at each CPU's startup, but don't advertise this feature to > the upper users if not all the CPUs in the system support this. This won't > affect leaf-level AF since leaf-level AF has its own detection and enabling > and doesn't depend on HAFT. Yes, I think this should work as long as arch_has_hw_pte_young() returns false if one of the CPUs doesn't support it. I had a quick grep through the kernel and all (most?) cases where the non-leaf pmd is checked or the AF bit cleared are conditional on should_clear_pmd_young(). Personally I'd have introduced pmd_nonleaf_young() etc. functions in the core code to make it more explicit. I guess, if we really want, we could add a VM_WARN_ON if pmd_young() or pmdp_test_and_clear_young() is called on a non-leaf pmd and FEAT_HAFT is disabled.
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a2f8ff354ca6..869792458a23 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2137,6 +2137,25 @@ config ARM64_EPAN if the cpu does not implement the feature. endmenu # "ARMv8.7 architectural features" +menu "ARMv8.9 architectural features" + +config ARM64_HAFT + bool "Support for Hardware managed Access Flag for Table Descriptor" + depends on ARM64_HW_AFDBM + default y + help + The ARMv8.9/ARMv9.5 introduces the feature Hardware managed Access + Flag for Table descriptors. When enabled an architectural executed + memory access will update the Access Flag in each Table descriptor + which is accessed during the translation table walk and for which + the Access Flag is 0. The Access Flag of the Table descriptor use + the same bit of PTE_AF. + + The feature will only be enabled if all the CPUs in the system + support this feature. If unsure, say Y. + +endmenu # "ARMv8.9 architectural features" + config ARM64_SVE bool "ARM Scalable Vector Extension support" default y diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 646ecd3069fd..ed4c968be935 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -2044,6 +2044,17 @@ static bool has_hw_dbm(const struct arm64_cpu_capabilities *cap, #endif +#if CONFIG_ARM64_HAFT + +static void cpu_enable_haft(struct arm64_cpu_capabilities const *cap) +{ + sysreg_clear_set_s(SYS_TCR2_EL1, 0, TCR2_EL1x_HAFT); + isb(); + local_flush_tlb_all(); +} + +#endif + #ifdef CONFIG_ARM64_AMU_EXTN /* @@ -2580,6 +2591,21 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .cpus = &dbm_cpus, ARM64_CPUID_FIELDS(ID_AA64MMFR1_EL1, HAFDBS, DBM) }, +#endif +#ifdef CONFIG_ARM64_HAFT + { + .desc = "Hardware managed Access Flag for Table Descriptor", + /* + * Contrary to the page/block access flag, the table access flag + * cannot be emulated in software (no access fault will occur). + * Therefore mandate that all CPUs have FEAT_HAFT. + */ + .type = ARM64_CPUCAP_BOOT_CPU_FEATURE, + .capability = ARM64_HAFT, + .matches = has_cpuid_feature, + .cpu_enable = cpu_enable_haft, + ARM64_CPUID_FIELDS(ID_AA64MMFR1_EL1, HAFDBS, HAFT) + }, #endif { .desc = "CRC32 instructions", diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps index ac3429d892b9..0b7a3a237e5d 100644 --- a/arch/arm64/tools/cpucaps +++ b/arch/arm64/tools/cpucaps @@ -55,6 +55,7 @@ HAS_TLB_RANGE HAS_VA52 HAS_VIRT_HOST_EXTN HAS_WFXT +HAFT HW_DBM KVM_HVHE KVM_PROTECTED_MODE diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg index 7ceaa1e0b4bc..9b3d15ea8a63 100644 --- a/arch/arm64/tools/sysreg +++ b/arch/arm64/tools/sysreg @@ -1688,6 +1688,7 @@ UnsignedEnum 3:0 HAFDBS 0b0000 NI 0b0001 AF 0b0010 DBM + 0b0011 HAFT EndEnum EndSysreg