| Message ID | 20211108071149.823570-3-reijiw@google.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | arm64: DC {ZVA,GVA,GZVA} shouldn't be used when DCZID_EL0.DZP == 1 | expand |
On Sun, Nov 07, 2021 at 11:11:49PM -0800, Reiji Watanabe wrote: > diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S > index e83643b3995f..e62c048af337 100644 > --- a/arch/arm64/lib/mte.S > +++ b/arch/arm64/lib/mte.S > @@ -43,17 +43,23 @@ SYM_FUNC_END(mte_clear_page_tags) > * x0 - address to the beginning of the page > */ > SYM_FUNC_START(mte_zero_clear_page_tags) > + and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag > mrs x1, dczid_el0 > + tbnz x1, #4, 2f // Branch if DC GZVA is prohibited > and w1, w1, #0xf > mov x2, #4 > lsl x1, x2, x1 > - and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag > > 1: dc gzva, x0 > add x0, x0, x1 > tst x0, #(PAGE_SIZE - 1) > b.ne 1b > ret > + > +2: stzg x0, [x0], #16 Nitpick: MTE_GRANULE_SIZE instead of 16. > + tst x0, #(PAGE_SIZE - 1) > + b.ne 2b > + ret > SYM_FUNC_END(mte_zero_clear_page_tags) We can use stz2g here since we know it's always a PAGE_SIZE and an even number of tag granules.
On Fri, Dec 03, 2021 at 06:29:48PM +0000, Catalin Marinas wrote: > On Sun, Nov 07, 2021 at 11:11:49PM -0800, Reiji Watanabe wrote: > > diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S > > index e83643b3995f..e62c048af337 100644 > > --- a/arch/arm64/lib/mte.S > > +++ b/arch/arm64/lib/mte.S > > @@ -43,17 +43,23 @@ SYM_FUNC_END(mte_clear_page_tags) > > * x0 - address to the beginning of the page > > */ > > SYM_FUNC_START(mte_zero_clear_page_tags) > > + and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag > > mrs x1, dczid_el0 > > + tbnz x1, #4, 2f // Branch if DC GZVA is prohibited > > and w1, w1, #0xf > > mov x2, #4 > > lsl x1, x2, x1 > > - and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag > > > > 1: dc gzva, x0 > > add x0, x0, x1 > > tst x0, #(PAGE_SIZE - 1) > > b.ne 1b > > ret > > + > > +2: stzg x0, [x0], #16 > > Nitpick: MTE_GRANULE_SIZE instead of 16. > > > + tst x0, #(PAGE_SIZE - 1) > > + b.ne 2b > > + ret > > SYM_FUNC_END(mte_zero_clear_page_tags) > > We can use stz2g here since we know it's always a PAGE_SIZE and an even > number of tag granules. I should have replied on v3. The comment is the same.
On Fri, Dec 3, 2021 at 10:52 AM Catalin Marinas <catalin.marinas@arm.com> wrote: > > On Fri, Dec 03, 2021 at 06:29:48PM +0000, Catalin Marinas wrote: > > On Sun, Nov 07, 2021 at 11:11:49PM -0800, Reiji Watanabe wrote: > > > diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S > > > index e83643b3995f..e62c048af337 100644 > > > --- a/arch/arm64/lib/mte.S > > > +++ b/arch/arm64/lib/mte.S > > > @@ -43,17 +43,23 @@ SYM_FUNC_END(mte_clear_page_tags) > > > * x0 - address to the beginning of the page > > > */ > > > SYM_FUNC_START(mte_zero_clear_page_tags) > > > + and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag > > > mrs x1, dczid_el0 > > > + tbnz x1, #4, 2f // Branch if DC GZVA is prohibited > > > and w1, w1, #0xf > > > mov x2, #4 > > > lsl x1, x2, x1 > > > - and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag > > > > > > 1: dc gzva, x0 > > > add x0, x0, x1 > > > tst x0, #(PAGE_SIZE - 1) > > > b.ne 1b > > > ret > > > + > > > +2: stzg x0, [x0], #16 > > > > Nitpick: MTE_GRANULE_SIZE instead of 16. I will fix it. > > > > > + tst x0, #(PAGE_SIZE - 1) > > > + b.ne 2b > > > + ret > > > SYM_FUNC_END(mte_zero_clear_page_tags) > > > > We can use stz2g here since we know it's always a PAGE_SIZE and an even > > number of tag granules. Ah, yes, I will fix it using stz2g. > I should have replied on v3. The comment is the same. Thank you for the review and I will be working for v4 ! Regards, Reiji
diff --git a/arch/arm64/include/asm/mte-kasan.h b/arch/arm64/include/asm/mte-kasan.h index 22420e1f8c03..26e013e540ae 100644 --- a/arch/arm64/include/asm/mte-kasan.h +++ b/arch/arm64/include/asm/mte-kasan.h @@ -84,10 +84,12 @@ static inline void __dc_gzva(u64 p) static inline void mte_set_mem_tag_range(void *addr, size_t size, u8 tag, bool init) { - u64 curr, mask, dczid_bs, end1, end2, end3; + u64 curr, mask, dczid, dczid_bs, dczid_dzp, end1, end2, end3; /* Read DC G(Z)VA block size from the system register. */ - dczid_bs = 4ul << (read_cpuid(DCZID_EL0) & 0xf); + dczid = read_cpuid(DCZID_EL0); + dczid_bs = 4ul << (dczid & 0xf); + dczid_dzp = (dczid >> 4) & 1; curr = (u64)__tag_set(addr, tag); mask = dczid_bs - 1; @@ -106,7 +108,7 @@ static inline void mte_set_mem_tag_range(void *addr, size_t size, u8 tag, */ #define SET_MEMTAG_RANGE(stg_post, dc_gva) \ do { \ - if (size >= 2 * dczid_bs) { \ + if (!dczid_dzp && size >= 2 * dczid_bs) {\ do { \ curr = stg_post(curr); \ } while (curr < end1); \ diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S index e83643b3995f..e62c048af337 100644 --- a/arch/arm64/lib/mte.S +++ b/arch/arm64/lib/mte.S @@ -43,17 +43,23 @@ SYM_FUNC_END(mte_clear_page_tags) * x0 - address to the beginning of the page */ SYM_FUNC_START(mte_zero_clear_page_tags) + and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag mrs x1, dczid_el0 + tbnz x1, #4, 2f // Branch if DC GZVA is prohibited and w1, w1, #0xf mov x2, #4 lsl x1, x2, x1 - and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag 1: dc gzva, x0 add x0, x0, x1 tst x0, #(PAGE_SIZE - 1) b.ne 1b ret + +2: stzg x0, [x0], #16 + tst x0, #(PAGE_SIZE - 1) + b.ne 2b + ret SYM_FUNC_END(mte_zero_clear_page_tags) /*
Currently, mte_set_mem_tag_range() and mte_zero_clear_page_tags() use DC {GVA,GZVA} unconditionally. But, they should make sure that DCZID_EL0.DZP, which indicates whether or not use of those instructions is prohibited, is zero when using those instructions. Use ST{G,ZG} instead when DCZID_EL0.DZP == 1. Fixes: 013bb59dbb7c ("arm64: mte: handle tags zeroing at page allocation time") Fixes: 3d0cca0b02ac ("kasan: speed up mte_set_mem_tag_range") Signed-off-by: Reiji Watanabe <reijiw@google.com> --- arch/arm64/include/asm/mte-kasan.h | 8 +++++--- arch/arm64/lib/mte.S | 8 +++++++- 2 files changed, 12 insertions(+), 4 deletions(-)