[RFC,1/2] riscv: vendors: andes: Add support to configure the PMA regions

Commit Message

Prabhakar Sept. 6, 2022, 10:21 a.m. UTC

The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA)
block that allows dynamic adjustment of memory attributes in the runtime.
It contains a configurable amount of PMA entries implemented as CSR
registers to control the attributes of memory locations in interest.

Below are the memory attributes supported:
* Device, Non-bufferable
* Device, bufferable
* Memory, Non-cacheable, Non-bufferable
* Memory, Non-cacheable, Bufferable
* Memory, Write-back, No-allocate
* Memory, Write-back, Read-allocate
* Memory, Write-back, Write-allocate
* Memory, Write-back, Read and Write-allocate

This patch adds support to configure the memory attributes of the
memory regions as passed from the core node. Currently the OpenSBI code
implements support for "Memory, Non-cacheable, Non-bufferable" option
with SBI_EXT_ANDES_SET_PMA.

More info about PMA (section 10.3):
http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf

Signed-off-by: Lad Prabhakar <[e-mail]>
---
Note: the current implementation only supports "Memory, Non-cacheable,
Bufferable" option.
---
 arch/riscv/Kbuild                      |  2 +
 arch/riscv/include/asm/sbi.h           |  1 +
 arch/riscv/vendors/Makefile            |  3 +
 arch/riscv/vendors/andes/Makefile      |  3 +
 arch/riscv/vendors/andes/ax45mp.c      | 93 ++++++++++++++++++++++++++
 arch/riscv/vendors/andes/include/sbi.h | 27 ++++++++
 6 files changed, 129 insertions(+)
 create mode 100644 arch/riscv/vendors/Makefile
 create mode 100644 arch/riscv/vendors/andes/Makefile
 create mode 100644 arch/riscv/vendors/andes/ax45mp.c
 create mode 100644 arch/riscv/vendors/andes/include/sbi.h

Comments

Conor Dooley Sept. 6, 2022, 10:39 a.m. UTC | #1

On 06/09/2022 11:21, Lad Prabhakar wrote:

> diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> index 2a0ef738695e..10a7c855d125 100644
> --- a/arch/riscv/include/asm/sbi.h
> +++ b/arch/riscv/include/asm/sbi.h
> @@ -37,6 +37,7 @@ enum sbi_ext_id {
> 
>          /* Vendor extensions must lie within this range */
>          SBI_EXT_VENDOR_START = 0x09000000,
> +       SBI_EXT_ANDES = 0x0900031E,
>          SBI_EXT_VENDOR_END = 0x09FFFFFF,
>   };

Everything else aside, I am very interested in what's happening
here. I'll take a proper look through things later, but for now:

For PolarFire SoC we have an InterHart Communication SBI EXT that
would would like to upstream support for. We are not aiming to put
the driver itself in arch/riscv - it's just a mailbox driver, but
I would like to use sbi.h for defining the vendor id etc.

I am not sure how this all aligns with:
> We’ll only accept patches for new modules or extensions if the
> specifications for those modules or extensions are listed as being
> “Frozen” or “Ratified” by the RISC-V Foundation. (Developers may, of
> course, maintain their own Linux kernel trees that contain code for
> any draft extensions that they wish.)
> 
> Additionally, the RISC-V specification allows implementors to create
> their own custom extensions. These custom extensions aren’t required
> to go through any review or ratification process by the RISC-V
> Foundation. To avoid the maintenance complexity and potential
> performance impact of adding kernel code for implementor-specific
> RISC-V extensions, we’ll only to accept patches for extensions that
> have been officially frozen or ratified by the RISC-V Foundation.
> (Implementors, may, of course, maintain their own Linux kernel trees
> containing code for any custom extensions that they wish.)

Which is in: https://docs.kernel.org/riscv/patch-acceptance.html

It is unclear to me whether that is just for ISA extensions or if that
covers SBI extensions too. At least for us, since we don't touch arch
code there is relatively little friction & there's no concerns about
reducing the portability of a kernel since it is just a regular old
driver.

I was planning on cornering some people *cough* Palmer *cough* at
LPC and asking him what his thoughts were there.

FWIW this is what we have been doing:
https://github.com/linux4microchip/linux/blob/linux-5.15-mchp/drivers/mailbox/mailbox-miv-ihc.c#L27

The IP itself has not stabilised yet, so we have not sent any patches
yet, but we do intend doing so...

But yea, I'll take a properly look at what you're doing here soonTM,
although at this point it may be the other side of LPC.

btw, where can I get my hands on your hardware?

Thanks,
Conor.

Lad, Prabhakar Sept. 6, 2022, 11:23 a.m. UTC | #2

Hi Conor,

Thanks for the quick glance!

On Tue, Sep 6, 2022 at 11:39 AM <[e-mail]> wrote:
>
> On 06/09/2022 11:21, Lad Prabhakar wrote:
>
> > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> > index 2a0ef738695e..10a7c855d125 100644
> > --- a/arch/riscv/include/asm/sbi.h
> > +++ b/arch/riscv/include/asm/sbi.h
> > @@ -37,6 +37,7 @@ enum sbi_ext_id {
> >
> >          /* Vendor extensions must lie within this range */
> >          SBI_EXT_VENDOR_START = 0x09000000,
> > +       SBI_EXT_ANDES = 0x0900031E,
> >          SBI_EXT_VENDOR_END = 0x09FFFFFF,
> >   };
>
> Everything else aside, I am very interested in what's happening
> here. I'll take a proper look through things later, but for now:
>
> For PolarFire SoC we have an InterHart Communication SBI EXT that
> would would like to upstream support for. We are not aiming to put
> the driver itself in arch/riscv - it's just a mailbox driver, but
> I would like to use sbi.h for defining the vendor id etc.
>
sbi.h seems appropriate for now, unless the maintainers have other ideas.

> I am not sure how this all aligns with:
> > We’ll only accept patches for new modules or extensions if the
> > specifications for those modules or extensions are listed as being
> > “Frozen” or “Ratified” by the RISC-V Foundation. (Developers may, of
> > course, maintain their own Linux kernel trees that contain code for
> > any draft extensions that they wish.)
> >
> > Additionally, the RISC-V specification allows implementors to create
> > their own custom extensions. These custom extensions aren’t required
> > to go through any review or ratification process by the RISC-V
> > Foundation. To avoid the maintenance complexity and potential
> > performance impact of adding kernel code for implementor-specific
> > RISC-V extensions, we’ll only to accept patches for extensions that
> > have been officially frozen or ratified by the RISC-V Foundation.
> > (Implementors, may, of course, maintain their own Linux kernel trees
> > containing code for any custom extensions that they wish.)
>
> Which is in: https://docs.kernel.org/riscv/patch-acceptance.html
>
I had completely missed this, thanks for pointing it out.

> It is unclear to me whether that is just for ISA extensions or if that
> covers SBI extensions too. At least for us, since we don't touch arch
> code there is relatively little friction & there's no concerns about
> reducing the portability of a kernel since it is just a regular old
> driver.
>
> I was planning on cornering some people *cough* Palmer *cough* at
> LPC and asking him what his thoughts were there.
>
I too will be attending the LPC (virtually though) and would like to
attend/chat on this topic. Please keep me posted.

> FWIW this is what we have been doing:
> https://github.com/linux4microchip/linux/blob/linux-5.15-mchp/drivers/mailbox/mailbox-miv-ihc.c#L27
>
From the looks of it it's on similar lines ;)

> The IP itself has not stabilised yet, so we have not sent any patches
> yet, but we do intend doing so...
>
I see..

> But yea, I'll take a properly look at what you're doing here soonTM,
> although at this point it may be the other side of LPC.
>
Thanks.

> btw, where can I get my hands on your hardware?
>
I shall share the link as soon as it's available.

Cheers,
Prabhakar

Chris Paterson Sept. 7, 2022, 8:18 a.m. UTC | #3

Hello Conor,

Thank you for your interest.

> From: [e-mail] <[e-mail]>
> Sent: 06 September 2022 11:39
> 
> On 06/09/2022 11:21, Lad Prabhakar wrote:
> 
> > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> > index 2a0ef738695e..10a7c855d125 100644
> > --- a/arch/riscv/include/asm/sbi.h
> > +++ b/arch/riscv/include/asm/sbi.h
> > @@ -37,6 +37,7 @@ enum sbi_ext_id {
> >
> >          /* Vendor extensions must lie within this range */
> >          SBI_EXT_VENDOR_START = 0x09000000,
> > +       SBI_EXT_ANDES = 0x0900031E,
> >          SBI_EXT_VENDOR_END = 0x09FFFFFF,
> >   };
> 
> Everything else aside, I am very interested in what's happening
> here. I'll take a proper look through things later, but for now:
> 
> For PolarFire SoC we have an InterHart Communication SBI EXT that
> would would like to upstream support for. We are not aiming to put
> the driver itself in arch/riscv - it's just a mailbox driver, but
> I would like to use sbi.h for defining the vendor id etc.
> 
> I am not sure how this all aligns with:
> > We’ll only accept patches for new modules or extensions if the
> > specifications for those modules or extensions are listed as being
> > “Frozen” or “Ratified” by the RISC-V Foundation. (Developers may, of
> > course, maintain their own Linux kernel trees that contain code for
> > any draft extensions that they wish.)
> >
> > Additionally, the RISC-V specification allows implementors to create
> > their own custom extensions. These custom extensions aren’t required
> > to go through any review or ratification process by the RISC-V
> > Foundation. To avoid the maintenance complexity and potential
> > performance impact of adding kernel code for implementor-specific
> > RISC-V extensions, we’ll only to accept patches for extensions that
> > have been officially frozen or ratified by the RISC-V Foundation.
> > (Implementors, may, of course, maintain their own Linux kernel trees
> > containing code for any custom extensions that they wish.)
> 
> Which is in: https://docs.kernel.org/riscv/patch-acceptance.html
> 
> It is unclear to me whether that is just for ISA extensions or if that
> covers SBI extensions too. At least for us, since we don't touch arch
> code there is relatively little friction & there's no concerns about
> reducing the portability of a kernel since it is just a regular old
> driver.
> 
> I was planning on cornering some people *cough* Palmer *cough* at
> LPC and asking him what his thoughts were there.
> 
> FWIW this is what we have been doing:
> https://github.com/linux4microchip/linux/blob/linux-5.15-mchp/drivers/mailbox/mailbox-miv-ihc.c#L27
> 
> The IP itself has not stabilised yet, so we have not sent any patches
> yet, but we do intend doing so...
> 
> But yea, I'll take a properly look at what you're doing here soonTM,
> although at this point it may be the other side of LPC.
> 
> btw, where can I get my hands on your hardware?

The latest information I have is that it should be available via various distributors on the 21st Sept.

Kind regards, Chris

> 
> Thanks,
> Conor.
>

Conor Dooley Sept. 7, 2022, 11:37 p.m. UTC | #4

On 07/09/2022 22:52, Atish Patra wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you
> know the content is safe
> 
> 
> On Tue, Sep 6, 2022 at 3:40 AM <[e-mail]
> <mailto:[e-mail]>> wrote:
> 
> On 06/09/2022 11:21, Lad Prabhakar wrote:
> 
>> diff --git a/arch/riscv/include/asm/sbi.h
>> b/arch/riscv/include/asm/sbi.h index 2a0ef738695e..10a7c855d125
>> 100644 --- a/arch/riscv/include/asm/sbi.h +++
>> b/arch/riscv/include/asm/sbi.h @@ -37,6 +37,7 @@ enum sbi_ext_id {
>> 
>> /* Vendor extensions must lie within this range */ 
>> SBI_EXT_VENDOR_START = 0x09000000, +       SBI_EXT_ANDES =
>> 0x0900031E, SBI_EXT_VENDOR_END = 0x09FFFFFF, };
> 
> Everything else aside, I am very interested in what's happening here.
> I'll take a proper look through things later, but for now:
> 
> For PolarFire SoC we have an InterHart Communication SBI EXT that 
> would would like to upstream support for. We are not aiming to put 
> the driver itself in arch/riscv - it's just a mailbox driver, but I
> would like to use sbi.h for defining the vendor id etc.
> 
> I am not sure how this all aligns with:
>> We’ll only accept patches for new modules or extensions if the 
>> specifications for those modules or extensions are listed as being 
>> “Frozen” or “Ratified” by the RISC-V Foundation. (Developers may,
>> of course, maintain their own Linux kernel trees that contain code
>> for any draft extensions that they wish.)
>> 
>> Additionally, the RISC-V specification allows implementors to
>> create their own custom extensions. These custom extensions aren’t
>> required to go through any review or ratification process by the
>> RISC-V Foundation. To avoid the maintenance complexity and
>> potential performance impact of adding kernel code for
>> implementor-specific RISC-V extensions, we’ll only to accept
>> patches for extensions that have been officially frozen or ratified
>> by the RISC-V Foundation. (Implementors, may, of course, maintain
>> their own Linux kernel trees containing code for any custom
>> extensions that they wish.)
> 
> Which is in: https://docs.kernel.org/riscv/patch-acceptance.html
> <https://docs.kernel.org/riscv/patch-acceptance.html>
> 
> It is unclear to me whether that is just for ISA extensions or if
> that covers SBI extensions too. At least for us, since we don't touch
> arch code there is relatively little friction & there's no concerns
> about reducing the portability of a kernel since it is just a regular
> old driver.
> 
> 
> It covers the standard SBI extensions as well. However, I don't think
> this includes a vendor extension as there is no freeze or
> ratification associated with vendor extensions.
> 
> It would be good to discuss the policy around vendor SBI extensions
> during LPC as well. We also need to discuss the ACPI policy as well.
> We most likely need a BoF to discuss these adhoc topics. I will check
> if we can schedule a BoF in advance.

I did briefly mention this to Palmer on IRC last night, just was busy
today & didn't get a chance to reply here. The indication there was
that yes, this paragraph did cover SBI extensions - which would make
vendor extensions not permitted upstream. 

We (microchip) are "only" doing a few ecalls in a driver but this
seems a fair bit more intrusive since it is in arch code. Even if the
answer is a "no" - a no from the horses mouth rather than on IRC &
maybe some rewording of that doc to be clearer would be nice.

I'd be down for a BoF, even if just to get a "no" in person haha

Conor.

> 
> I was planning on cornering some people *cough* Palmer *cough* at LPC
> and asking him what his thoughts were there.
> 
> FWIW this is what we have been doing: 
> https://github.com/linux4microchip/linux/blob/linux-5.15-mchp/drivers/mailbox/mailbox-miv-ihc.c#L27
> <https://github.com/linux4microchip/linux/blob/linux-5.15-mchp/drivers/mailbox/mailbox-miv-ihc.c#L27>
>
>  The IP itself has not stabilised yet, so we have not sent any
> patches yet, but we do intend doing so...
> 
> But yea, I'll take a properly look at what you're doing here soonTM, 
> although at this point it may be the other side of LPC.
> 
> btw, where can I get my hands on your hardware?
> 
> Thanks, Conor.
> 
> 
> _______________________________________________ linux-riscv mailing
> list [e-mail]
> <mailto:[e-mail]> 
> http://lists.infradead.org/mailman/listinfo/linux-riscv
> <http://lists.infradead.org/mailman/listinfo/linux-riscv>
> 
> 
> 
> -- Regards, Atish

Biju Das Sept. 8, 2022, 8:39 a.m. UTC | #5

Hi Conor, Atish,

What RISC-V devices you have?

Ours is RISC-V uniprocessor without IO Coherence Port.

Currently USB, ethernet, SDHI/eMMC doesn't work with standard
DMA api's.

On RISC-V world, how do we handle DMA api for uniprocessor without
IO Coherence Port?

Cheers,
Biju



> -----Original Message-----
> From: [e-mail] <[e-mail]>
> Sent: 08 September 2022 00:38
> To: [e-mail]
> Cc: Prabhakar Mahadev Lad <[e-mail]>;
> [e-mail]; [e-mail]; [e-mail];
> [e-mail]; [e-mail]; [e-mail];
> [e-mail]; [e-mail];
> [e-mail]; [e-mail]; Biju Das
> <[e-mail]>
> Subject: Re: [RFC PATCH 1/2] riscv: vendors: andes: Add support to
> configure the PMA regions
> 
> On 07/09/2022 22:52, Atish Patra wrote:
> > EXTERNAL EMAIL: Do not click links or open attachments unless you know
> > the content is safe
> >
> >
> > On Tue, Sep 6, 2022 at 3:40 AM <[e-mail]
> > <mailto:[e-mail]>> wrote:
> >
> > On 06/09/2022 11:21, Lad Prabhakar wrote:
> >
> >> diff --git a/arch/riscv/include/asm/sbi.h
> >> b/arch/riscv/include/asm/sbi.h index 2a0ef738695e..10a7c855d125
> >> 100644 --- a/arch/riscv/include/asm/sbi.h +++
> >> b/arch/riscv/include/asm/sbi.h @@ -37,6 +37,7 @@ enum sbi_ext_id {
> >>
> >> /* Vendor extensions must lie within this range */
> >> SBI_EXT_VENDOR_START = 0x09000000, +       SBI_EXT_ANDES =
> >> 0x0900031E, SBI_EXT_VENDOR_END = 0x09FFFFFF, };
> >
> > Everything else aside, I am very interested in what's happening here.
> > I'll take a proper look through things later, but for now:
> >
> > For PolarFire SoC we have an InterHart Communication SBI EXT that
> > would would like to upstream support for. We are not aiming to put the
> > driver itself in arch/riscv - it's just a mailbox driver, but I would
> > like to use sbi.h for defining the vendor id etc.
> >
> > I am not sure how this all aligns with:
> >> We'll only accept patches for new modules or extensions if the
> >> specifications for those modules or extensions are listed as being
> >> "Frozen" or "Ratified" by the RISC-V Foundation. (Developers may, of
> >> course, maintain their own Linux kernel trees that contain code for
> >> any draft extensions that they wish.)
> >>
> >> Additionally, the RISC-V specification allows implementors to create
> >> their own custom extensions. These custom extensions aren't required
> >> to go through any review or ratification process by the RISC-V
> >> Foundation. To avoid the maintenance complexity and potential
> >> performance impact of adding kernel code for implementor-specific
> >> RISC-V extensions, we'll only to accept patches for extensions that
> >> have been officially frozen or ratified by the RISC-V Foundation.
> >> (Implementors, may, of course, maintain their own Linux kernel trees
> >> containing code for any custom extensions that they wish.)
> >
> > Which is in:
> > https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs
> > .kernel.org%2Friscv%2Fpatch-acceptance.html&amp;data=05%7C01%7Cbiju.da
> > s.jz%40bp.renesas.com%7C7fd3275accdb450e547a08da912a0042%7C53d82571da1
> > 947e49cb4625a166a4a2a%7C0%7C0%7C637981906834865446%7CUnknown%7CTWFpbGZ
> > sb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3
> > D%7C3000%7C%7C%7C&amp;sdata=YAV2Ahz7TFMJJ3wCj%2FAdDuDEcPq0kXXL%2B3o6FA
> > d0%2BUI%3D&amp;reserved=0
> > <https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc
> > s.kernel.org%2Friscv%2Fpatch-acceptance.html&amp;data=05%7C01%7Cbiju.d
> > as.jz%40bp.renesas.com%7C7fd3275accdb450e547a08da912a0042%7C53d82571da
> > 1947e49cb4625a166a4a2a%7C0%7C0%7C637981906834865446%7CUnknown%7CTWFpbG
> > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%
> > 3D%7C3000%7C%7C%7C&amp;sdata=YAV2Ahz7TFMJJ3wCj%2FAdDuDEcPq0kXXL%2B3o6F
> > Ad0%2BUI%3D&amp;reserved=0>
> >
> > It is unclear to me whether that is just for ISA extensions or if that
> > covers SBI extensions too. At least for us, since we don't touch arch
> > code there is relatively little friction & there's no concerns about
> > reducing the portability of a kernel since it is just a regular old
> > driver.
> >
> >
> > It covers the standard SBI extensions as well. However, I don't think
> > this includes a vendor extension as there is no freeze or ratification
> > associated with vendor extensions.
> >
> > It would be good to discuss the policy around vendor SBI extensions
> > during LPC as well. We also need to discuss the ACPI policy as well.
> > We most likely need a BoF to discuss these adhoc topics. I will check
> > if we can schedule a BoF in advance.
> 
> I did briefly mention this to Palmer on IRC last night, just was busy
> today & didn't get a chance to reply here. The indication there was that
> yes, this paragraph did cover SBI extensions - which would make vendor
> extensions not permitted upstream.
> 
> We (microchip) are "only" doing a few ecalls in a driver but this seems a
> fair bit more intrusive since it is in arch code. Even if the answer is a
> "no" - a no from the horses mouth rather than on IRC & maybe some
> rewording of that doc to be clearer would be nice.
> 
> I'd be down for a BoF, even if just to get a "no" in person haha
> 
> Conor.
> 
> >
> > I was planning on cornering some people *cough* Palmer *cough* at LPC
> > and asking him what his thoughts were there.
> >
> > FWIW this is what we have been doing:
> > https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> > ub.com%2Flinux4microchip%2Flinux%2Fblob%2Flinux-5.15-mchp%2Fdrivers%2F
> > mailbox%2Fmailbox-miv-ihc.c%23L27&amp;data=05%7C01%7Cbiju.das.jz%40bp.
> > renesas.com%7C7fd3275accdb450e547a08da912a0042%7C53d82571da1947e49cb46
> > 25a166a4a2a%7C0%7C0%7C637981906834865446%7CUnknown%7CTWFpbGZsb3d8eyJWI
> > joiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
> > C%7C%7C&amp;sdata=seNiuv6EsY1u0SIT33%2F0CWHJu401d5zSaNmVb%2BUHKPM%3D&a
> > mp;reserved=0
> > <https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit
> > hub.com%2Flinux4microchip%2Flinux%2Fblob%2Flinux-5.15-mchp%2Fdrivers%2
> > Fmailbox%2Fmailbox-miv-ihc.c%23L27&amp;data=05%7C01%7Cbiju.das.jz%40bp
> > .renesas.com%7C7fd3275accdb450e547a08da912a0042%7C53d82571da1947e49cb4
> > 625a166a4a2a%7C0%7C0%7C637981906834865446%7CUnknown%7CTWFpbGZsb3d8eyJW
> > IjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%
> > 7C%7C%7C&amp;sdata=seNiuv6EsY1u0SIT33%2F0CWHJu401d5zSaNmVb%2BUHKPM%3D&
> > amp;reserved=0>
> >
> >  The IP itself has not stabilised yet, so we have not sent any patches
> > yet, but we do intend doing so...
> >
> > But yea, I'll take a properly look at what you're doing here soonTM,
> > although at this point it may be the other side of LPC.
> >
> > btw, where can I get my hands on your hardware?
> >
> > Thanks, Conor.
> >
> >
> > _______________________________________________ linux-riscv mailing
> > list [e-mail]
> > <mailto:[e-mail]>
> > https://jpn01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists
> > .infradead.org%2Fmailman%2Flistinfo%2Flinux-riscv&amp;data=05%7C01%7Cb
> > iju.das.jz%40bp.renesas.com%7C7fd3275accdb450e547a08da912a0042%7C53d82
> > 571da1947e49cb4625a166a4a2a%7C0%7C0%7C637981906834865446%7CUnknown%7CT
> > WFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI
> > 6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=AaMGs%2BqRpuw8MOWci3qPvJ1W5U296Vp%2B
> > higFxDSoHmo%3D&amp;reserved=0
> > <https://jpn01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flist
> > s.infradead.org%2Fmailman%2Flistinfo%2Flinux-riscv&amp;data=05%7C01%7C
> > biju.das.jz%40bp.renesas.com%7C7fd3275accdb450e547a08da912a0042%7C53d8
> > 2571da1947e49cb4625a166a4a2a%7C0%7C0%7C637981906834865446%7CUnknown%7C
> > TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVC
> > I6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=AaMGs%2BqRpuw8MOWci3qPvJ1W5U296Vp%2
> > BhigFxDSoHmo%3D&amp;reserved=0>
> >
> >
> >
> > -- Regards, Atish

Lad, Prabhakar Sept. 8, 2022, 11:50 a.m. UTC | #6

Hi Guo,

> > -----Original Message-----
> > From: [e-mail] <[e-mail]>
> > Sent: 08 September 2022 00:38
> > To: [e-mail]
> > Cc: Prabhakar Mahadev Lad <[e-mail]>;
> > [e-mail]; [e-mail]; [e-mail];
> > [e-mail]; [e-mail]; [e-mail];
> > [e-mail]; [e-mail];
> > [e-mail]; [e-mail]; Biju Das
> > <[e-mail]>
> > Subject: Re: [RFC PATCH 1/2] riscv: vendors: andes: Add support to
> > configure the PMA regions
> >
> > On 07/09/2022 22:52, Atish Patra wrote:
> > > EXTERNAL EMAIL: Do not click links or open attachments unless you know
> > > the content is safe
> > >
> > >
> > > On Tue, Sep 6, 2022 at 3:40 AM <[e-mail]
> > > <mailto:[e-mail]>> wrote:
> > >
> > > On 06/09/2022 11:21, Lad Prabhakar wrote:
> > >
> > >> diff --git a/arch/riscv/include/asm/sbi.h
> > >> b/arch/riscv/include/asm/sbi.h index 2a0ef738695e..10a7c855d125
> > >> 100644 --- a/arch/riscv/include/asm/sbi.h +++
> > >> b/arch/riscv/include/asm/sbi.h @@ -37,6 +37,7 @@ enum sbi_ext_id {
> > >>
> > >> /* Vendor extensions must lie within this range */
> > >> SBI_EXT_VENDOR_START = 0x09000000, +       SBI_EXT_ANDES =
> > >> 0x0900031E, SBI_EXT_VENDOR_END = 0x09FFFFFF, };
> > >
I am interested to know what is the status of your patch series [0].

[0] https://lore.kernel.org/lkml/[e-mail]/T/#m7f4c4cdfb92d6c8672bbfabebda729ce4700e177

Cheers,
Prabhakar

Conor Dooley Sept. 8, 2022, 11:59 a.m. UTC | #7

On 08/09/2022 12:50, Lad, Prabhakar wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> 
> Hi Guo,
> 
>>> -----Original Message-----
>>> From: [e-mail] <[e-mail]>
>>> Sent: 08 September 2022 00:38
>>> To: [e-mail]
>>> Cc: Prabhakar Mahadev Lad <[e-mail]>;
>>> [e-mail]; [e-mail]; [e-mail];
>>> [e-mail]; [e-mail]; [e-mail];
>>> [e-mail]; [e-mail];
>>> [e-mail]; [e-mail]; Biju Das
>>> <[e-mail]>
>>> Subject: Re: [RFC PATCH 1/2] riscv: vendors: andes: Add support to
>>> configure the PMA regions
>>>
>>> On 07/09/2022 22:52, Atish Patra wrote:
>>>> EXTERNAL EMAIL: Do not click links or open attachments unless you know
>>>> the content is safe
>>>>
>>>>
>>>> On Tue, Sep 6, 2022 at 3:40 AM <[e-mail]
>>>> <mailto:[e-mail]>> wrote:
>>>>
>>>> On 06/09/2022 11:21, Lad Prabhakar wrote:
>>>>
>>>>> diff --git a/arch/riscv/include/asm/sbi.h
>>>>> b/arch/riscv/include/asm/sbi.h index 2a0ef738695e..10a7c855d125
>>>>> 100644 --- a/arch/riscv/include/asm/sbi.h +++
>>>>> b/arch/riscv/include/asm/sbi.h @@ -37,6 +37,7 @@ enum sbi_ext_id {
>>>>>
>>>>> /* Vendor extensions must lie within this range */
>>>>> SBI_EXT_VENDOR_START = 0x09000000, +       SBI_EXT_ANDES =
>>>>> 0x0900031E, SBI_EXT_VENDOR_END = 0x09FFFFFF, };
>>>>
> I am interested to know what is the status of your patch series [0].
> 
> [0] https://lore.kernel.org/lkml/[e-mail]/T/#m7f4c4cdfb92d6c8672bbfabebda729ce4700e177

Heh, take a look at:
git grep "ERRATA_THEAD*"

This is all can-of-worms territory that we are heading to here, as
the Zicbo* extensions is what is meant to be doing these sorts of
things..

Lad, Prabhakar Sept. 8, 2022, 12:13 p.m. UTC | #8

On Thu, Sep 8, 2022 at 1:00 PM <[e-mail]> wrote:
>
> On 08/09/2022 12:50, Lad, Prabhakar wrote:
> > EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> >
> > Hi Guo,
> >
> >>> -----Original Message-----
> >>> From: [e-mail] <[e-mail]>
> >>> Sent: 08 September 2022 00:38
> >>> To: [e-mail]
> >>> Cc: Prabhakar Mahadev Lad <[e-mail]>;
> >>> [e-mail]; [e-mail]; [e-mail];
> >>> [e-mail]; [e-mail]; [e-mail];
> >>> [e-mail]; [e-mail];
> >>> [e-mail]; [e-mail]; Biju Das
> >>> <[e-mail]>
> >>> Subject: Re: [RFC PATCH 1/2] riscv: vendors: andes: Add support to
> >>> configure the PMA regions
> >>>
> >>> On 07/09/2022 22:52, Atish Patra wrote:
> >>>> EXTERNAL EMAIL: Do not click links or open attachments unless you know
> >>>> the content is safe
> >>>>
> >>>>
> >>>> On Tue, Sep 6, 2022 at 3:40 AM <[e-mail]
> >>>> <mailto:[e-mail]>> wrote:
> >>>>
> >>>> On 06/09/2022 11:21, Lad Prabhakar wrote:
> >>>>
> >>>>> diff --git a/arch/riscv/include/asm/sbi.h
> >>>>> b/arch/riscv/include/asm/sbi.h index 2a0ef738695e..10a7c855d125
> >>>>> 100644 --- a/arch/riscv/include/asm/sbi.h +++
> >>>>> b/arch/riscv/include/asm/sbi.h @@ -37,6 +37,7 @@ enum sbi_ext_id {
> >>>>>
> >>>>> /* Vendor extensions must lie within this range */
> >>>>> SBI_EXT_VENDOR_START = 0x09000000, +       SBI_EXT_ANDES =
> >>>>> 0x0900031E, SBI_EXT_VENDOR_END = 0x09FFFFFF, };
> >>>>
> > I am interested to know what is the status of your patch series [0].
> >
> > [0] https://lore.kernel.org/lkml/[e-mail]/T/#m7f4c4cdfb92d6c8672bbfabebda729ce4700e177
>
> Heh, take a look at:
> git grep "ERRATA_THEAD*"
>
> This is all can-of-worms territory that we are heading to here, as
> the Zicbo* extensions is what is meant to be doing these sorts of
> things..
>
Aha thanks for the pointer.

Cheers,
Prabhakar

Conor Dooley Sept. 8, 2022, 12:43 p.m. UTC | #9

On 08/09/2022 09:39, Biju Das wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> 
> Hi Conor, Atish,
> 
> What RISC-V devices you have?

A bunch ;)

A __couple__ PolarFire SoC boards, HiFive Unleashed, D1 Nezha,
Canaan k210 MAIX something & the VisionFive.

> Ours is RISC-V uniprocessor without IO Coherence Port.

What does "IO Coherence Port" mean? Zicbo*?

> Currently USB, ethernet, SDHI/eMMC doesn't work with standard
> DMA api's.

Sounds pretty similar to the D1 so.

> On RISC-V world, how do we handle DMA api for uniprocessor without
> IO Coherence Port?

If you do mean Zicbo* you're into errata territory there & I don't
know if that'll be acceptable upstream - not for me to make that
call...

Thanks,
Conor.

> 
> Cheers,
> Biju
> 
> 
> 
>> -----Original Message-----
>> From: [e-mail] <[e-mail]>
>> Sent: 08 September 2022 00:38
>> To: [e-mail]
>> Cc: Prabhakar Mahadev Lad <[e-mail]>;
>> [e-mail]; [e-mail]; [e-mail];
>> [e-mail]; [e-mail]; [e-mail];
>> [e-mail]; [e-mail];
>> [e-mail]; [e-mail]; Biju Das
>> <[e-mail]>
>> Subject: Re: [RFC PATCH 1/2] riscv: vendors: andes: Add support to
>> configure the PMA regions
>>
>> On 07/09/2022 22:52, Atish Patra wrote:
>>> EXTERNAL EMAIL: Do not click links or open attachments unless you know
>>> the content is safe
>>>
>>>
>>> On Tue, Sep 6, 2022 at 3:40 AM <[e-mail]
>>> <mailto:[e-mail]>> wrote:
>>>
>>> On 06/09/2022 11:21, Lad Prabhakar wrote:
>>>
>>>> diff --git a/arch/riscv/include/asm/sbi.h
>>>> b/arch/riscv/include/asm/sbi.h index 2a0ef738695e..10a7c855d125
>>>> 100644 --- a/arch/riscv/include/asm/sbi.h +++
>>>> b/arch/riscv/include/asm/sbi.h @@ -37,6 +37,7 @@ enum sbi_ext_id {
>>>>
>>>> /* Vendor extensions must lie within this range */
>>>> SBI_EXT_VENDOR_START = 0x09000000, +       SBI_EXT_ANDES =
>>>> 0x0900031E, SBI_EXT_VENDOR_END = 0x09FFFFFF, };
>>>
>>> Everything else aside, I am very interested in what's happening here.
>>> I'll take a proper look through things later, but for now:
>>>
>>> For PolarFire SoC we have an InterHart Communication SBI EXT that
>>> would would like to upstream support for. We are not aiming to put the
>>> driver itself in arch/riscv - it's just a mailbox driver, but I would
>>> like to use sbi.h for defining the vendor id etc.
>>>
>>> I am not sure how this all aligns with:
>>>> We'll only accept patches for new modules or extensions if the
>>>> specifications for those modules or extensions are listed as being
>>>> "Frozen" or "Ratified" by the RISC-V Foundation. (Developers may, of
>>>> course, maintain their own Linux kernel trees that contain code for
>>>> any draft extensions that they wish.)
>>>>
>>>> Additionally, the RISC-V specification allows implementors to create
>>>> their own custom extensions. These custom extensions aren't required
>>>> to go through any review or ratification process by the RISC-V
>>>> Foundation. To avoid the maintenance complexity and potential
>>>> performance impact of adding kernel code for implementor-specific
>>>> RISC-V extensions, we'll only to accept patches for extensions that
>>>> have been officially frozen or ratified by the RISC-V Foundation.
>>>> (Implementors, may, of course, maintain their own Linux kernel trees
>>>> containing code for any custom extensions that they wish.)
>>>
>>> Which is in:
>>> https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs
>>> .kernel.org%2Friscv%2Fpatch-acceptance.html&amp;data=05%7C01%7Cbiju.da
>>> s.jz%40bp.renesas.com%7C7fd3275accdb450e547a08da912a0042%7C53d82571da1
>>> 947e49cb4625a166a4a2a%7C0%7C0%7C637981906834865446%7CUnknown%7CTWFpbGZ
>>> sb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3
>>> D%7C3000%7C%7C%7C&amp;sdata=YAV2Ahz7TFMJJ3wCj%2FAdDuDEcPq0kXXL%2B3o6FA
>>> d0%2BUI%3D&amp;reserved=0
>>> <https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc
>>> s.kernel.org%2Friscv%2Fpatch-acceptance.html&amp;data=05%7C01%7Cbiju.d
>>> as.jz%40bp.renesas.com%7C7fd3275accdb450e547a08da912a0042%7C53d82571da
>>> 1947e49cb4625a166a4a2a%7C0%7C0%7C637981906834865446%7CUnknown%7CTWFpbG
>>> Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%
>>> 3D%7C3000%7C%7C%7C&amp;sdata=YAV2Ahz7TFMJJ3wCj%2FAdDuDEcPq0kXXL%2B3o6F
>>> Ad0%2BUI%3D&amp;reserved=0>
>>>
>>> It is unclear to me whether that is just for ISA extensions or if that
>>> covers SBI extensions too. At least for us, since we don't touch arch
>>> code there is relatively little friction & there's no concerns about
>>> reducing the portability of a kernel since it is just a regular old
>>> driver.
>>>
>>>
>>> It covers the standard SBI extensions as well. However, I don't think
>>> this includes a vendor extension as there is no freeze or ratification
>>> associated with vendor extensions.
>>>
>>> It would be good to discuss the policy around vendor SBI extensions
>>> during LPC as well. We also need to discuss the ACPI policy as well.
>>> We most likely need a BoF to discuss these adhoc topics. I will check
>>> if we can schedule a BoF in advance.
>>
>> I did briefly mention this to Palmer on IRC last night, just was busy
>> today & didn't get a chance to reply here. The indication there was that
>> yes, this paragraph did cover SBI extensions - which would make vendor
>> extensions not permitted upstream.
>>
>> We (microchip) are "only" doing a few ecalls in a driver but this seems a
>> fair bit more intrusive since it is in arch code. Even if the answer is a
>> "no" - a no from the horses mouth rather than on IRC & maybe some
>> rewording of that doc to be clearer would be nice.
>>
>> I'd be down for a BoF, even if just to get a "no" in person haha
>>
>> Conor.
>>
>>>
>>> I was planning on cornering some people *cough* Palmer *cough* at LPC
>>> and asking him what his thoughts were there.
>>>
>>> FWIW this is what we have been doing:
>>> https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
>>> ub.com%2Flinux4microchip%2Flinux%2Fblob%2Flinux-5.15-mchp%2Fdrivers%2F
>>> mailbox%2Fmailbox-miv-ihc.c%23L27&amp;data=05%7C01%7Cbiju.das.jz%40bp.
>>> renesas.com%7C7fd3275accdb450e547a08da912a0042%7C53d82571da1947e49cb46
>>> 25a166a4a2a%7C0%7C0%7C637981906834865446%7CUnknown%7CTWFpbGZsb3d8eyJWI
>>> joiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
>>> C%7C%7C&amp;sdata=seNiuv6EsY1u0SIT33%2F0CWHJu401d5zSaNmVb%2BUHKPM%3D&a
>>> mp;reserved=0
>>> <https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit
>>> hub.com%2Flinux4microchip%2Flinux%2Fblob%2Flinux-5.15-mchp%2Fdrivers%2
>>> Fmailbox%2Fmailbox-miv-ihc.c%23L27&amp;data=05%7C01%7Cbiju.das.jz%40bp
>>> .renesas.com%7C7fd3275accdb450e547a08da912a0042%7C53d82571da1947e49cb4
>>> 625a166a4a2a%7C0%7C0%7C637981906834865446%7CUnknown%7CTWFpbGZsb3d8eyJW
>>> IjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%
>>> 7C%7C%7C&amp;sdata=seNiuv6EsY1u0SIT33%2F0CWHJu401d5zSaNmVb%2BUHKPM%3D&
>>> amp;reserved=0>
>>>
>>>   The IP itself has not stabilised yet, so we have not sent any patches
>>> yet, but we do intend doing so...
>>>
>>> But yea, I'll take a properly look at what you're doing here soonTM,
>>> although at this point it may be the other side of LPC.
>>>
>>> btw, where can I get my hands on your hardware?
>>>
>>> Thanks, Conor.
>>>
>>>
>>> _______________________________________________ linux-riscv mailing
>>> list [e-mail]
>>> <mailto:[e-mail]>
>>> https://jpn01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists
>>> .infradead.org%2Fmailman%2Flistinfo%2Flinux-riscv&amp;data=05%7C01%7Cb
>>> iju.das.jz%40bp.renesas.com%7C7fd3275accdb450e547a08da912a0042%7C53d82
>>> 571da1947e49cb4625a166a4a2a%7C0%7C0%7C637981906834865446%7CUnknown%7CT
>>> WFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI
>>> 6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=AaMGs%2BqRpuw8MOWci3qPvJ1W5U296Vp%2B
>>> higFxDSoHmo%3D&amp;reserved=0
>>> <https://jpn01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flist
>>> s.infradead.org%2Fmailman%2Flistinfo%2Flinux-riscv&amp;data=05%7C01%7C
>>> biju.das.jz%40bp.renesas.com%7C7fd3275accdb450e547a08da912a0042%7C53d8
>>> 2571da1947e49cb4625a166a4a2a%7C0%7C0%7C637981906834865446%7CUnknown%7C
>>> TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVC
>>> I6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=AaMGs%2BqRpuw8MOWci3qPvJ1W5U296Vp%2
>>> BhigFxDSoHmo%3D&amp;reserved=0>
>>>
>>>
>>>
>>> -- Regards, Atish
>

Biju Das Sept. 8, 2022, 1:01 p.m. UTC | #10

Hi Conor,

Thanks for the feedback.

> Subject: Re: [RFC PATCH 1/2] riscv: vendors: andes: Add support to
> configure the PMA regions
> 
> On 08/09/2022 09:39, Biju Das wrote:
> > EXTERNAL EMAIL: Do not click links or open attachments unless you know
> > the content is safe
> >
> > Hi Conor, Atish,
> >
> > What RISC-V devices you have?
> 
> A bunch ;)
> 
> A __couple__ PolarFire SoC boards, HiFive Unleashed, D1 Nezha, Canaan
> k210 MAIX something & the VisionFive.

If standard DMA api works without any issue means, on these platforms
IO Coherence port is enabled in the hardware. So all peripherals
involving DMA work as expected.

> > Ours is RISC-V uniprocessor without IO Coherence Port.
> 
> What does "IO Coherence Port" mean? Zicbo*?

The HW will provide coherency between CPU and peripheral.

If Zibco* is uniprocessor, then highly it may not have IO coherence
Port enabled in their design.

Guo, Please confirm.

> 
> > Currently USB, ethernet, SDHI/eMMC doesn't work with standard DMA
> > api's.
> 
> Sounds pretty similar to the D1 so.
> 
> > On RISC-V world, how do we handle DMA api for uniprocessor without IO
> > Coherence Port?
> 
> If you do mean Zicbo* you're into errata territory there & I don't know
> if that'll be acceptable upstream - not for me to make that call...

It is not errata for sure. It is a HW design where we don't have
IO cache coherency port enabled in the HW. So looks like it is not
an extension or errata but it is core stuff.

Something similar to incoherency mentioned in below [1] for uniprocessor
Systems.
[1] https://elinux.org/images/8/80/Initializing-riscv.pdf

Cheers,
Biju

> >
> >
> >
> >> -----Original Message-----
> >> From: [e-mail] <[e-mail]>
> >> Sent: 08 September 2022 00:38
> >> To: [e-mail]
> >> Cc: Prabhakar Mahadev Lad <[e-mail]>;
> >> [e-mail]; [e-mail]; [e-mail];
> >> [e-mail]; [e-mail];
> >> [e-mail]; [e-mail];
> >> [e-mail];
> >> [e-mail]; [e-mail]; Biju Das
> >> <[e-mail]>
> >> Subject: Re: [RFC PATCH 1/2] riscv: vendors: andes: Add support to
> >> configure the PMA regions
> >>
> >> On 07/09/2022 22:52, Atish Patra wrote:
> >>> EXTERNAL EMAIL: Do not click links or open attachments unless you
> >>> know the content is safe
> >>>
> >>>
> >>> On Tue, Sep 6, 2022 at 3:40 AM <[e-mail]
> >>> <mailto:[e-mail]>> wrote:
> >>>
> >>> On 06/09/2022 11:21, Lad Prabhakar wrote:
> >>>
> >>>> diff --git a/arch/riscv/include/asm/sbi.h
> >>>> b/arch/riscv/include/asm/sbi.h index 2a0ef738695e..10a7c855d125
> >>>> 100644 --- a/arch/riscv/include/asm/sbi.h +++
> >>>> b/arch/riscv/include/asm/sbi.h @@ -37,6 +37,7 @@ enum sbi_ext_id {
> >>>>
> >>>> /* Vendor extensions must lie within this range */
> >>>> SBI_EXT_VENDOR_START = 0x09000000, +       SBI_EXT_ANDES =
> >>>> 0x0900031E, SBI_EXT_VENDOR_END = 0x09FFFFFF, };
> >>>
> >>> Everything else aside, I am very interested in what's happening here.
> >>> I'll take a proper look through things later, but for now:
> >>>
> >>> For PolarFire SoC we have an InterHart Communication SBI EXT that
> >>> would would like to upstream support for. We are not aiming to put
> >>> the driver itself in arch/riscv - it's just a mailbox driver, but I
> >>> would like to use sbi.h for defining the vendor id etc.
> >>>
> >>> I am not sure how this all aligns with:
> >>>> We'll only accept patches for new modules or extensions if the
> >>>> specifications for those modules or extensions are listed as being
> >>>> "Frozen" or "Ratified" by the RISC-V Foundation. (Developers may,
> >>>> of course, maintain their own Linux kernel trees that contain code
> >>>> for any draft extensions that they wish.)
> >>>>
> >>>> Additionally, the RISC-V specification allows implementors to
> >>>> create their own custom extensions. These custom extensions aren't
> >>>> required to go through any review or ratification process by the
> >>>> RISC-V Foundation. To avoid the maintenance complexity and
> >>>> potential performance impact of adding kernel code for
> >>>> implementor-specific RISC-V extensions, we'll only to accept
> >>>> patches for extensions that have been officially frozen or ratified
> by the RISC-V Foundation.
> >>>> (Implementors, may, of course, maintain their own Linux kernel
> >>>> trees containing code for any custom extensions that they wish.)
> >>>
> >>> Which is in:
> >>>
> >>> It is unclear to me whether that is just for ISA extensions or if
> >>> that covers SBI extensions too. At least for us, since we don't
> >>> touch arch code there is relatively little friction & there's no
> >>> concerns about reducing the portability of a kernel since it is just
> >>> a regular old driver.
> >>>
> >>>
> >>> It covers the standard SBI extensions as well. However, I don't
> >>> think this includes a vendor extension as there is no freeze or
> >>> ratification associated with vendor extensions.
> >>>
> >>> It would be good to discuss the policy around vendor SBI extensions
> >>> during LPC as well. We also need to discuss the ACPI policy as well.
> >>> We most likely need a BoF to discuss these adhoc topics. I will
> >>> check if we can schedule a BoF in advance.
> >>
> >> I did briefly mention this to Palmer on IRC last night, just was busy
> >> today & didn't get a chance to reply here. The indication there was
> >> that yes, this paragraph did cover SBI extensions - which would make
> >> vendor extensions not permitted upstream.
> >>
> >> We (microchip) are "only" doing a few ecalls in a driver but this
> >> seems a fair bit more intrusive since it is in arch code. Even if the
> >> answer is a "no" - a no from the horses mouth rather than on IRC &
> >> maybe some rewording of that doc to be clearer would be nice.
> >>
> >> I'd be down for a BoF, even if just to get a "no" in person haha
> >>
> >> Conor.
> >>
> >>>
> >>> I was planning on cornering some people *cough* Palmer *cough* at
> >>> LPC and asking him what his thoughts were there.
> >>>
> >>> FWIW this is what we have been doing:
> >>>
> >>>   The IP itself has not stabilised yet, so we have not sent any
> >>> patches yet, but we do intend doing so...
> >>>
> >>> But yea, I'll take a properly look at what you're doing here soonTM,
> >>> although at this point it may be the other side of LPC.
> >>>
> >>> btw, where can I get my hands on your hardware?
> >>>
> >>> Thanks, Conor.
> >

Geert Uytterhoeven Sept. 8, 2022, 1:20 p.m. UTC | #11

Hi Biju,

On Thu, Sep 8, 2022 at 3:01 PM Biju Das <[e-mail]> wrote:
> > Subject: Re: [RFC PATCH 1/2] riscv: vendors: andes: Add support to
> > configure the PMA regions
> >
> > On 08/09/2022 09:39, Biju Das wrote:
> > > EXTERNAL EMAIL: Do not click links or open attachments unless you know
> > > the content is safe
> > >
> > > Hi Conor, Atish,
> > >
> > > What RISC-V devices you have?
> >
> > A bunch ;)
> >
> > A __couple__ PolarFire SoC boards, HiFive Unleashed, D1 Nezha, Canaan
> > k210 MAIX something & the VisionFive.
>
> If standard DMA api works without any issue means, on these platforms
> IO Coherence port is enabled in the hardware. So all peripherals
> involving DMA work as expected.
>
> > > Ours is RISC-V uniprocessor without IO Coherence Port.
> >
> > What does "IO Coherence Port" mean? Zicbo*?
>
> The HW will provide coherency between CPU and peripheral.
>
> If Zibco* is uniprocessor, then highly it may not have IO coherence
> Port enabled in their design.

Zicbo* is a set of extensions for the instructions.
These cannot be retrofitted to existing silicon, but perhaps they can
be trapped and emulated?

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [e-mail]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Biju Das Sept. 8, 2022, 1:54 p.m. UTC | #12

Hi Geert,


> Subject: Re: [RFC PATCH 1/2] riscv: vendors: andes: Add support to
> configure the PMA regions
> 
> Hi Biju,
> 
> On Thu, Sep 8, 2022 at 3:01 PM Biju Das <[e-mail]>
> wrote:
> > > Subject: Re: [RFC PATCH 1/2] riscv: vendors: andes: Add support to
> > > configure the PMA regions
> > >
> > > On 08/09/2022 09:39, Biju Das wrote:
> > > > EXTERNAL EMAIL: Do not click links or open attachments unless you
> > > > know the content is safe
> > > >
> > > > Hi Conor, Atish,
> > > >
> > > > What RISC-V devices you have?
> > >
> > > A bunch ;)
> > >
> > > A __couple__ PolarFire SoC boards, HiFive Unleashed, D1 Nezha,
> > > Canaan
> > > k210 MAIX something & the VisionFive.
> >
> > If standard DMA api works without any issue means, on these platforms
> > IO Coherence port is enabled in the hardware. So all peripherals
> > involving DMA work as expected.
> >
> > > > Ours is RISC-V uniprocessor without IO Coherence Port.
> > >
> > > What does "IO Coherence Port" mean? Zicbo*?
> >
> > The HW will provide coherency between CPU and peripheral.
> >
> > If Zibco* is uniprocessor, then highly it may not have IO coherence
> > Port enabled in their design.
> 
> Zicbo* is a set of extensions for the instructions.
> These cannot be retrofitted to existing silicon, but perhaps they can be
> trapped and emulated?

Thanks for clarifying this. I just gone through RISC-V CMOs.
https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.1.pdf

Cheers,
Biju

Conor Dooley Sept. 8, 2022, 2:04 p.m. UTC | #13

On 08/09/2022 14:01, Biju Das wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> 
> Hi Conor,
> 
> Thanks for the feedback.
> 
>> Subject: Re: [RFC PATCH 1/2] riscv: vendors: andes: Add support to
>> configure the PMA regions
>>
>> On 08/09/2022 09:39, Biju Das wrote:
>>> EXTERNAL EMAIL: Do not click links or open attachments unless you know
>>> the content is safe
>>>
>>> Hi Conor, Atish,
>>>
>>> What RISC-V devices you have?
>>
>> A bunch ;)
>>
>> A __couple__ PolarFire SoC boards, HiFive Unleashed, D1 Nezha, Canaan
>> k210 MAIX something & the VisionFive.
> 
> If standard DMA api works without any issue means, on these platforms
> IO Coherence port is enabled in the hardware. So all peripherals
> involving DMA work as expected.
> 
>>> Ours is RISC-V uniprocessor without IO Coherence Port.
>>
>> What does "IO Coherence Port" mean? Zicbo*?
> 
> The HW will provide coherency between CPU and peripheral.
> 
> If Zibco* is uniprocessor, then highly it may not have IO coherence
> Port enabled in their design.

Zicbo* are cache management extensions as Geert pointed out.

> 
> Guo, Please confirm.
> 
>>
>>> Currently USB, ethernet, SDHI/eMMC doesn't work with standard DMA
>>> api's.
>>
>> Sounds pretty similar to the D1 so.
>>
>>> On RISC-V world, how do we handle DMA api for uniprocessor without IO
>>> Coherence Port?
>>
>> If you do mean Zicbo* you're into errata territory there & I don't know
>> if that'll be acceptable upstream - not for me to make that call...
> 
> It is not errata for sure. It is a HW design where we don't have
> IO cache coherency port enabled in the HW. So looks like it is not
> an extension or errata but it is core stuff.

If you do non-coherent stuff that is not Zicbo*, the precedence set by
the D1 is errata. As I said to Prabhakar earlier, do a
`git grep "ERRATA_THEAD*`. I am not a maintainer so I don't know the
"rules" about doing cache management without the dedicated extensions
are.

Hope that at least helps,
Conor.

Lad, Prabhakar Sept. 8, 2022, 2:39 p.m. UTC | #14

On Thu, Sep 8, 2022 at 3:04 PM <[e-mail]> wrote:
>
> On 08/09/2022 14:01, Biju Das wrote:
> > EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> >
> > Hi Conor,
> >
> > Thanks for the feedback.
> >
> >> Subject: Re: [RFC PATCH 1/2] riscv: vendors: andes: Add support to
> >> configure the PMA regions
> >>
> >> On 08/09/2022 09:39, Biju Das wrote:
> >>> EXTERNAL EMAIL: Do not click links or open attachments unless you know
> >>> the content is safe
> >>>
> >>> Hi Conor, Atish,
> >>>
> >>> What RISC-V devices you have?
> >>
> >> A bunch ;)
> >>
> >> A __couple__ PolarFire SoC boards, HiFive Unleashed, D1 Nezha, Canaan
> >> k210 MAIX something & the VisionFive.
> >
> > If standard DMA api works without any issue means, on these platforms
> > IO Coherence port is enabled in the hardware. So all peripherals
> > involving DMA work as expected.
> >
> >>> Ours is RISC-V uniprocessor without IO Coherence Port.
> >>
> >> What does "IO Coherence Port" mean? Zicbo*?
> >
> > The HW will provide coherency between CPU and peripheral.
> >
> > If Zibco* is uniprocessor, then highly it may not have IO coherence
> > Port enabled in their design.
>
> Zicbo* are cache management extensions as Geert pointed out.
>
> >
> > Guo, Please confirm.
> >
> >>
> >>> Currently USB, ethernet, SDHI/eMMC doesn't work with standard DMA
> >>> api's.
> >>
> >> Sounds pretty similar to the D1 so.
> >>
> >>> On RISC-V world, how do we handle DMA api for uniprocessor without IO
> >>> Coherence Port?
> >>
> >> If you do mean Zicbo* you're into errata territory there & I don't know
> >> if that'll be acceptable upstream - not for me to make that call...
> >
> > It is not errata for sure. It is a HW design where we don't have
> > IO cache coherency port enabled in the HW. So looks like it is not
> > an extension or errata but it is core stuff.
>
> If you do non-coherent stuff that is not Zicbo*, the precedence set by
> the D1 is errata. As I said to Prabhakar earlier, do a
> `git grep "ERRATA_THEAD*`. I am not a maintainer so I don't know the
> "rules" about doing cache management without the dedicated extensions
> are.
>
Maybe we could have a discussion about this topic at LPC too ;)

Cheers,
Prabhakar

Chris Paterson Sept. 22, 2022, 3 p.m. UTC | #15

Hello Conor,

> >
> > btw, where can I get my hands on your hardware?
> 
> The latest information I have is that it should be available via various
> distributors on the 21st Sept.

I've now seen it at the following distributors, perhaps there is also availability elsewhere.
https://uk.farnell.com/renesas/rtk9743f01s01000be/evaluation-board-kit-64bit-risc/dp/4051136
https://www.futureelectronics.com/p/semiconductors--embedded-solutions--embedded-boards--embedded-software/rtk9743f01s01000be-renesas-7166252
https://www.avnet.com/shop/us/products/renesas-electronics/rtk9743f01s01000be-3074457345649323718/

For those interested, the part number to google for is RTK9743F01S01000BE.

Kind regards, Chris

Patch

diff --git a/arch/riscv/Kbuild b/arch/riscv/Kbuild
index afa83e307a2e..d6821f660fc3 100644
--- a/arch/riscv/Kbuild
+++ b/arch/riscv/Kbuild
@@ -5,6 +5,8 @@  obj-$(CONFIG_BUILTIN_DTB) += boot/dts/
 obj-y += errata/
 obj-$(CONFIG_KVM) += kvm/
 
+obj-y += vendors/
+
 obj-$(CONFIG_ARCH_HAS_KEXEC_PURGATORY) += purgatory/
 
 # for cleaning
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index 2a0ef738695e..10a7c855d125 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -37,6 +37,7 @@  enum sbi_ext_id {
 
 	/* Vendor extensions must lie within this range */
 	SBI_EXT_VENDOR_START = 0x09000000,
+	SBI_EXT_ANDES = 0x0900031E,
 	SBI_EXT_VENDOR_END = 0x09FFFFFF,
 };
 
diff --git a/arch/riscv/vendors/Makefile b/arch/riscv/vendors/Makefile
new file mode 100644
index 000000000000..0a5a5541d2a3
--- /dev/null
+++ b/arch/riscv/vendors/Makefile
@@ -0,0 +1,3 @@ 
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_ARCH_R9A07G043) += andes/
diff --git a/arch/riscv/vendors/andes/Makefile b/arch/riscv/vendors/andes/Makefile
new file mode 100644
index 000000000000..60fa8226c4a3
--- /dev/null
+++ b/arch/riscv/vendors/andes/Makefile
@@ -0,0 +1,3 @@ 
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_ARCH_R9A07G043) += ax45mp.o
diff --git a/arch/riscv/vendors/andes/ax45mp.c b/arch/riscv/vendors/andes/ax45mp.c
new file mode 100644
index 000000000000..931cba754f41
--- /dev/null
+++ b/arch/riscv/vendors/andes/ax45mp.c
@@ -0,0 +1,93 @@ 
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * AX45MP setup
+ * - PMA
+ *
+ * Copyright (C) 2022 Renesas Electronics Corp.
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/of.h>
+
+#include <asm/sbi.h>
+
+#include "include/sbi.h"
+
+#define ANDES_AX45MP_MAX_PMA_REGIONS	16
+
+struct pma_arg_t {
+	phys_addr_t offset;
+	unsigned long vaddr;
+	size_t size;
+	size_t entry_id;
+};
+
+static long sbi_set_pma(void *arg)
+{
+	phys_addr_t offset = ((struct pma_arg_t *)arg)->offset;
+	unsigned long vaddr = ((struct pma_arg_t *)arg)->vaddr;
+	size_t entry_id = ((struct pma_arg_t *)arg)->entry_id;
+	size_t size = ((struct pma_arg_t *)arg)->size;
+	struct sbiret ret;
+
+	ret = sbi_ecall(SBI_EXT_ANDES, SBI_EXT_ANDES_SET_PMA, offset, vaddr, size, entry_id, 0, 0);
+
+	return ret.value;
+}
+
+static unsigned long cpu_nocache_area_set(unsigned long start,
+					  unsigned long size,
+					  unsigned long entry_id)
+{
+	struct pma_arg_t pma_arg;
+	unsigned long ret = 0;
+
+	pma_arg.offset = start;
+	pma_arg.size = size;
+	pma_arg.vaddr = start + size;
+	pma_arg.entry_id = entry_id;
+	ret = sbi_set_pma(&pma_arg);
+
+	return ret;
+}
+
+static void ax45mp_configure_pma_regions(struct device_node *np, int count)
+{
+	u64 start, size;
+	unsigned int i;
+
+	for (i = 0 ; i < count ; i++) {
+		of_property_read_u64_index(np, "pma-regions", (i << 1), &start);
+		of_property_read_u64_index(np, "pma-regions", (i << 1) + 1, &size);
+		cpu_nocache_area_set(start, size, i);
+	}
+}
+
+static const struct of_device_id ax45mp_ids[] = {
+	{ .compatible = "andestech,ax45mp" },
+	{ /* sentinel */ }
+};
+
+static int __init ax45mp_init(void)
+{
+	struct device_node *np;
+	int count;
+
+	np = of_find_matching_node(NULL, ax45mp_ids);
+	if (!np)
+		return -ENODEV;
+
+	count = of_property_count_elems_of_size(np, "pma-regions",
+						sizeof(u32) * 4);
+	if (count < 0)
+		return 0;
+
+	if (count > ANDES_AX45MP_MAX_PMA_REGIONS)
+		return -EINVAL;
+
+	ax45mp_configure_pma_regions(np, count);
+
+	return 0;
+}
+arch_initcall(ax45mp_init);
diff --git a/arch/riscv/vendors/andes/include/sbi.h b/arch/riscv/vendors/andes/include/sbi.h
new file mode 100644
index 000000000000..6dcd215bb5f8
--- /dev/null
+++ b/arch/riscv/vendors/andes/include/sbi.h
@@ -0,0 +1,27 @@ 
+/* SPDX-License-Identifier: GPL-2.0+ */
+
+#ifndef __RISCV_ANDES_SBI_H
+#define __RISCV_ANDES_SBI_H
+
+enum sbi_ext_andes_fid {
+	SBI_EXT_ANDES_GET_MCACHE_CTL_STATUS = 0,
+	SBI_EXT_ANDES_GET_MMISC_CTL_STATUS,
+	SBI_EXT_ANDES_SET_MCACHE_CTL,
+	SBI_EXT_ANDES_SET_MMISC_CTL,
+	SBI_EXT_ANDES_ICACHE_OP,
+	SBI_EXT_ANDES_DCACHE_OP,
+	SBI_EXT_ANDES_L1CACHE_I_PREFETCH,
+	SBI_EXT_ANDES_L1CACHE_D_PREFETCH,
+	SBI_EXT_ANDES_NON_BLOCKING_LOAD_STORE,
+	SBI_EXT_ANDES_WRITE_AROUND,
+	SBI_EXT_ANDES_SET_PMA,
+	SBI_EXT_ANDES_FREE_PMA,
+	SBI_EXT_ANDES_PROBE_PMA,
+	SBI_EXT_ANDES_DCACHE_WBINVAL_ALL,
+	SBI_EXT_ANDES_GET_MICM_CTL_STATUS,
+	SBI_EXT_ANDES_GET_MDCM_CTL_STATUS,
+	SBI_EXT_ANDES_GET_MMSC_CTL_STATUS,
+	SBI_EXT_ANDES_GET_MISA_CTL_STATUS,
+};
+
+#endif