[RFC] drivers: irq-chip: Enable SGI's for Secure-to-NonSecure communication use-cases

Commit Message

Vaibhav Hiremath Jan. 6, 2014, 5:09 a.m. UTC

Hi,

Currently the Software Generated Interrupts (SGI) are restricted to use only
for SMP architecture for inter-processor communication as rightly documented 
in ARM GIC spec V1.

In the system with the uniprocessor (and/or multiprocessor variants) architecture with 
TRUSTZONE enabled device (like, AM43xx device), the SGI can be used for 
communication between secure-to-nonsecure world.
And in order to enable SGI event from secure world to non-secure world,
GIC driver __must__ support registration of interrupt service routines for SGI's;
which is currently restricted by GIC driver.

The usecase is something like,

On any asynchronous HW or SW events, certain secure functionality gets triggered 
and SGI will be used to notify to the public world on the completion and/or result of operation.

Non Secure           |           Secure (TrustZone)
 (Linux Booted      |         (Secure software init happed 
   To prompt)         |          and trusted code getting executed)
                                |
    (On any secure operation Where we would
          like public world communication)
                                |
                                |         - Use SGI to trigger event to public Linux code
                                |         - And share the public info/data with public world for further processing
                          <=====      
  Public code         |
  handles it            |

In order to prototype and to make sure that it works, 
I did change the GIC driver to allow registration of SGI interrupts (interrupts 0 - 16) and 
tried it on AM43xx EVM and pasted the diff of the changes below.
I have also validated using SGI for secure to non-secure communication.

The idea behind this RFC (or rather query) is, to get feedback or 
comments on the use-case of using SGI for secure-to-nonsecure communication 
on non-SMP architecture or SMP architecture with uniprocessor.
I understand that, lot of things I need to take care from SMP architecture perspective. 
Based on the feedback I can spend more time to make below changes more generic to handle 
both uniprocessor and multi-processor architectures including more validation.

Also, please note that, this requires change in all the DT files using GIC interrupt controller.

Any pointers and suggestions are welcome here.

Thanks,
Vaibhav

-               hwirq_base = 16;
+               hwirq_base = 0;
                if (irq_start != -1)
                        irq_start = (irq_start & ~31) + 16;
        } else {
-               hwirq_base = 32;
+               hwirq_base = 0;
        }

        /*
@@ -803,7 +798,7 @@ void __init gic_init_bases(unsigned int gic_nr, int irq_start,
        gic->gic_irqs = gic_irqs;

        gic_irqs -= hwirq_base; /* calculate # of irqs to allocate */
-       irq_base = irq_alloc_descs(irq_start, 16, gic_irqs, numa_node_id());
+       irq_base = irq_alloc_descs(irq_start, 0, gic_irqs, numa_node_id());
        if (IS_ERR_VALUE(irq_base)) {
                WARN(1, "Cannot allocate irq_descs @ IRQ%d, assuming pre-allocated\n",
                     irq_start);

--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comments

[e-mail] Jan. 6, 2014, 5:32 a.m. UTC | #1

> -----Original Message-----
> From: linux-arm-kernel [mailto:linux-arm-kernel-
> [e-mail]] On Behalf Of Hiremath, Vaibhav
> Sent: Monday, January 06, 2014 10:39 AM
> To: [e-mail]
> Cc: Tony Lindgren; [e-mail]; Russell King
> Subject: [RFC] drivers: irq-chip: Enable SGI's for Secure-to-NonSecure
> communication use-cases
> 
> Hi,
> 
> Currently the Software Generated Interrupts (SGI) are restricted to use
> only for SMP architecture for inter-processor communication as rightly
> documented in ARM GIC spec V1.
> 
> In the system with the uniprocessor (and/or multiprocessor variants)
> architecture with TRUSTZONE enabled device (like, AM43xx device), the SGI
> can be used for communication between secure-to-nonsecure world.
> And in order to enable SGI event from secure world to non-secure world,
> GIC driver __must__ support registration of interrupt service routines
> for SGI's; which is currently restricted by GIC driver.

I am not an expert on this, but as per my understanding the model recommended by ARM is the use of IRQ as
a Normal world interrupt source, and FIQ as the Secure world source.

If IRQ is received by the Secure world, it should cause a hardware trap to the monitor and the monitor
mode should cause a context switch and jumps to the normal world, where the interrupt handler should execute
(see reference [1]).

For making a transition from the secure world to the normal world and vice-versa, the core should transition
via the monitor mode. Assuming a Uniprocessor system running both Normal and Secure world - thus providing a 
view of two virtual processors running in a time-sliced fashion, the world in which the processor is executing
should be indicated by the NS-bit in the Secure Configuration Register (SCR) in CP15. The IRQ, FIQ, Abort exceptions
can all be configured to cause the processor to switch into monitor mode.

The software that executes within monitor mode is implementation defined, but it generally saves the state of the
current world and restores the state of the world being switched to.
It then performs a return-from-exception to restart processing in the restored world. (see reference [2]).

Does this RFC implementation take into account the monitor mode switch while switching/passing information b/w
the two worlds?

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.prd29-genc-009492c/CACCDCDH.html

[2] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.prd29-genc-009492c/ch03s03s01.html

Regards,
Bhupesh

> The usecase is something like,
> 
> On any asynchronous HW or SW events, certain secure functionality gets
> triggered and SGI will be used to notify to the public world on the
> completion and/or result of operation.
> 
> Non Secure           |           Secure (TrustZone)
>  (Linux Booted      |         (Secure software init happed
>    To prompt)         |          and trusted code getting executed)
>                                 |
>     (On any secure operation Where we would
>           like public world communication)
>                                 |
>                                 |         - Use SGI to trigger event to
> public Linux code
>                                 |         - And share the public
> info/data with public world for further processing
>                           <=====
>   Public code         |
>   handles it            |
> 
> In order to prototype and to make sure that it works, I did change the
> GIC driver to allow registration of SGI interrupts (interrupts 0 - 16)
> and tried it on AM43xx EVM and pasted the diff of the changes below.
> I have also validated using SGI for secure to non-secure communication.
> 
> The idea behind this RFC (or rather query) is, to get feedback or
> comments on the use-case of using SGI for secure-to-nonsecure
> communication on non-SMP architecture or SMP architecture with
> uniprocessor.
> I understand that, lot of things I need to take care from SMP
> architecture perspective.
> Based on the feedback I can spend more time to make below changes more
> generic to handle both uniprocessor and multi-processor architectures
> including more validation.
> 
> Also, please note that, this requires change in all the DT files using
> GIC interrupt controller.
> 
> Any pointers and suggestions are welcome here.
> 
> Thanks,
> Vaibhav
> 
> diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c index
> d0e9480..135385a 100644
> --- a/drivers/irqchip/irq-gic.c
> +++ b/drivers/irqchip/irq-gic.c
> @@ -290,7 +290,7 @@ static asmlinkage void __exception_irq_entry
> gic_handle_irq(struct pt_regs *regs
>                 irqstat = readl_relaxed(cpu_base + GIC_CPU_INTACK);
>                 irqnr = irqstat & ~0x1c00;
> 
> -               if (likely(irqnr > 15 && irqnr < 1021)) {
> +               if (likely(irqnr >= 0 && irqnr < 1021)) {
>                         irqnr = irq_find_mapping(gic->domain, irqnr);
>                         handle_IRQ(irqnr, regs);
>                         continue;
> @@ -324,7 +324,7 @@ static void gic_handle_cascade_irq(unsigned int irq,
> struct irq_desc *desc)
>                 goto out;
> 
>         cascade_irq = irq_find_mapping(chip_data->domain, gic_irq);
> -       if (unlikely(gic_irq < 32 || gic_irq > 1020))
> +       if (unlikely(gic_irq > 1020))
>                 handle_bad_irq(cascade_irq, desc);
>         else
>                 generic_handle_irq(cascade_irq); @@ -395,20 +395,20 @@
> static void __init gic_dist_init(struct gic_chip_data *gic)
>         cpumask = gic_get_cpumask(gic);
>         cpumask |= cpumask << 8;
>         cpumask |= cpumask << 16;
> -       for (i = 32; i < gic_irqs; i += 4)
> +       for (i = 0; i < gic_irqs; i += 4)
>                 writel_relaxed(cpumask, base + GIC_DIST_TARGET + i * 4 /
> 4);
> 
>         /*
>          * Set priority on all global interrupts.
>          */
> -       for (i = 32; i < gic_irqs; i += 4)
> +       for (i = 0; i < gic_irqs; i += 4)
>                 writel_relaxed(0xa0a0a0a0, base + GIC_DIST_PRI + i * 4 /
> 4);
> 
>         /*
>          * Disable all interrupts.  Leave the PPI and SGIs alone
>          * as these enables are banked registers.
>          */
> -       for (i = 32; i < gic_irqs; i += 32)
> +       for (i = 0; i < gic_irqs; i += 32)
>                 writel_relaxed(0xffffffff, base + GIC_DIST_ENABLE_CLEAR +
> i * 4 / 32);
> 
>         writel_relaxed(1, base + GIC_DIST_CTRL); @@ -672,7 +672,7 @@ void
> gic_raise_softirq(const struct cpumask *mask, unsigned int irq)  static
> int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
>                                 irq_hw_number_t hw)  {
> -       if (hw < 32) {
> +       if (hw < 0) {
>                 irq_set_percpu_devid(irq);
>                 irq_set_chip_and_handler(irq, &gic_chip,
>                                          handle_percpu_devid_irq); @@ -
> 696,8 +696,11 @@ static int gic_irq_domain_xlate(struct irq_domain *d,
> 			if (intsize < 3)
>                 return -EINVAL;
> 
> -        /* Get the interrupt number and add 16 to skip over SGIs */
> -       *out_hwirq = intspec[1] + 16;
> -
> -       /* For SPIs, we need to add 16 more to get the GIC irq ID number
> */
> -       if (!intspec[0])
> -               *out_hwirq += 16;
> -
> +       *out_hwirq = intspec[1];
>         *out_type = intspec[2] & IRQ_TYPE_SENSE_MASK;
>         return 0;
>  }
> @@ -785,11 +780,11 @@ void __init gic_init_bases(unsigned int gic_nr, int
> irq_start,
>          * For secondary GICs, skip over PPIs, too.
>          */
>         if (gic_nr == 0 && (irq_start & 31) > 0) {
> -               hwirq_base = 16;
> +               hwirq_base = 0;
>                 if (irq_start != -1)
>                         irq_start = (irq_start & ~31) + 16;
>         } else {
> -               hwirq_base = 32;
> +               hwirq_base = 0;
>         }
> 
>         /*
> @@ -803,7 +798,7 @@ void __init gic_init_bases(unsigned int gic_nr, int
> irq_start,
>         gic->gic_irqs = gic_irqs;
> 
>         gic_irqs -= hwirq_base; /* calculate # of irqs to allocate */
> -       irq_base = irq_alloc_descs(irq_start, 16, gic_irqs,
> numa_node_id());
> +       irq_base = irq_alloc_descs(irq_start, 0, gic_irqs,
> + numa_node_id());
>         if (IS_ERR_VALUE(irq_base)) {
>                 WARN(1, "Cannot allocate irq_descs @ IRQ%d, assuming pre-
> allocated\n",
>                      irq_start);
> 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> [e-mail]
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Vaibhav Hiremath Jan. 6, 2014, 6:31 a.m. UTC | #2

> -----Original Message-----
> From: [e-mail]
> [mailto:[e-mail]]
> Sent: Monday, January 06, 2014 11:02 AM
> To: Hiremath, Vaibhav; '[e-mail]'
> Cc: 'Tony Lindgren'; '[e-mail]'; 'Russell King'
> Subject: RE: [RFC] drivers: irq-chip: Enable SGI's for Secure-to-NonSecure
> communication use-cases
> 
> > -----Original Message-----
> > From: linux-arm-kernel [mailto:linux-arm-kernel-
> > [e-mail]] On Behalf Of Hiremath, Vaibhav
> > Sent: Monday, January 06, 2014 10:39 AM
> > To: [e-mail]
> > Cc: Tony Lindgren; [e-mail]; Russell King
> > Subject: [RFC] drivers: irq-chip: Enable SGI's for Secure-to-NonSecure
> > communication use-cases
> >
> > Hi,
> >
> > Currently the Software Generated Interrupts (SGI) are restricted to
> > use only for SMP architecture for inter-processor communication as
> > rightly documented in ARM GIC spec V1.
> >
> > In the system with the uniprocessor (and/or multiprocessor variants)
> > architecture with TRUSTZONE enabled device (like, AM43xx device), the
> > SGI can be used for communication between secure-to-nonsecure world.
> > And in order to enable SGI event from secure world to non-secure
> > world, GIC driver __must__ support registration of interrupt service
> > routines for SGI's; which is currently restricted by GIC driver.
> 
> I am not an expert on this, but as per my understanding the model
> recommended by ARM is the use of IRQ as a Normal world interrupt source, and
> FIQ as the Secure world source.
> 
[Hiremath, Vaibhav] Absolutely and I also follow same recommendation here.

> If IRQ is received by the Secure world, it should cause a hardware trap to the
> monitor and the monitor mode should cause a context switch and jumps to the
> normal world, where the interrupt handler should execute (see reference [1]).
> 

I think I missed to explicitly talk about FIQ in the RFC, sorry for the confusion.
Yes, IRQ received by secure world should cause HW trap to monitor mode and 
Further should cause context switch to public world to handle the interrupt.

And this RFC wants to leverage this, where, any operations on secure world,

        - Functional operation (entered through monitor mode) which want to pass on the 
          event to public world
        - Any FIQ (secure interrupts) wants to pass on certain events or information to
           Public world - It could be any secure peripheral interrupt causing FIQ

Secure world can use IRQ, which is generated by software to pass on the event information
to public world. Please note that, the SGI is non-secure, so if you raise and SGI from secure 
world it will follow the execution mentioned above.

So this RFC enables asynchronous communication channel between secure world to 
Non-secure world using SGI.

> For making a transition from the secure world to the normal world and vice-
> versa, the core should transition via the monitor mode. Assuming a
> Uniprocessor system running both Normal and Secure world - thus providing a
> view of two virtual processors running in a time-sliced fashion, the world in
> which the processor is executing should be indicated by the NS-bit in the Secure
> Configuration Register (SCR) in CP15. The IRQ, FIQ, Abort exceptions can all be
> configured to cause the processor to switch into monitor mode.
> 
> The software that executes within monitor mode is implementation defined, but
> it generally saves the state of the current world and restores the state of the
> world being switched to.
> It then performs a return-from-exception to restart processing in the restored
> world. (see reference [2]).
> 
> Does this RFC implementation take into account the monitor mode switch while
> switching/passing information b/w the two worlds?
> 

Yes, as recommended by ARM, this RFC is also based on the recommendation, where,
monitor mode is responsible for saving and restoring contexts.
RFC doesn't handle the monitor mode implementation which lacks context save/restore.

Please let me know if you still have any confusion on the usecase and on this RFC

Thanks,
Vaibhav

--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Russell King - ARM Linux Jan. 6, 2014, 1:25 p.m. UTC | #3

On Mon, Jan 06, 2014 at 05:09:04AM +0000, Hiremath, Vaibhav wrote:
> The idea behind this RFC (or rather query) is, to get feedback or 
> comments on the use-case of using SGI for secure-to-nonsecure communication 
> on non-SMP architecture or SMP architecture with uniprocessor.
> I understand that, lot of things I need to take care from SMP architecture perspective. 
> Based on the feedback I can spend more time to make below changes more generic to handle 
> both uniprocessor and multi-processor architectures including more validation.

So it seems that your intention is to use the existing infrastructure
for this by directing SGIs through the normal IRQ processing.

To that idea, I say no way.

I also think you need to think more about the changes you're making in
your patch - several of them seem to have just been a case of s/32/0/
without any further thought whether that change is actually appropriate.
Really, I'd suggest reading the GIC documentation, especially the bits
about the first register being banked between each CPU.

So changing the setup in the distributor initialisation is only going to
hit the register on the CPU it's running on...

Vaibhav Hiremath Jan. 6, 2014, 4:12 p.m. UTC | #4

> -----Original Message-----
> From: Russell King - ARM Linux [mailto:[e-mail]]
> Sent: Monday, January 06, 2014 6:56 PM
> To: Hiremath, Vaibhav
> Cc: [e-mail]; [e-mail]; Tony
> Lindgren
> Subject: Re: [RFC] drivers: irq-chip: Enable SGI's for Secure-to-NonSecure
> communication use-cases
> 
> On Mon, Jan 06, 2014 at 05:09:04AM +0000, Hiremath, Vaibhav wrote:
> > The idea behind this RFC (or rather query) is, to get feedback or
> > comments on the use-case of using SGI for secure-to-nonsecure
> > communication on non-SMP architecture or SMP architecture with
> uniprocessor.
> > I understand that, lot of things I need to take care from SMP architecture
> perspective.
> > Based on the feedback I can spend more time to make below changes more
> > generic to handle both uniprocessor and multi-processor architectures
> including more validation.
> 
> So it seems that your intention is to use the existing infrastructure for this by
> directing SGIs through the normal IRQ processing.
> 
> To that idea, I say no way.
> 

You have any other alternative?

> I also think you need to think more about the changes you're making in your
> patch - several of them seem to have just been a case of s/32/0/ without any
> further thought whether that change is actually appropriate.
> Really, I'd suggest reading the GIC documentation, especially the bits about the
> first register being banked between each CPU.
> 
> So changing the setup in the distributor initialisation is only going to hit the
> register on the CPU it's running on...

I completely agree with you and I am willing to spend time to have more generic implementation,
which will also handle banked registers. The changes were just to prove the secure-to-nonsecure 
 communication concept which I wanted to introduce here as part of this RFC.

I will quote my statement again 

"I understand that, lot of things I need to take care from SMP architecture
perspective.
Based on the feedback I can spend more time to make below changes more
generic to handle both uniprocessor and multi-processor architectures
including more validation."

Thanks,
Vaibhav
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index d0e9480..135385a 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -290,7 +290,7 @@  static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs
                irqstat = readl_relaxed(cpu_base + GIC_CPU_INTACK);
                irqnr = irqstat & ~0x1c00;

-               if (likely(irqnr > 15 && irqnr < 1021)) {
+               if (likely(irqnr >= 0 && irqnr < 1021)) {
                        irqnr = irq_find_mapping(gic->domain, irqnr);
                        handle_IRQ(irqnr, regs);
                        continue;
@@ -324,7 +324,7 @@  static void gic_handle_cascade_irq(unsigned int irq, struct irq_desc *desc)
                goto out;

        cascade_irq = irq_find_mapping(chip_data->domain, gic_irq);
-       if (unlikely(gic_irq < 32 || gic_irq > 1020))
+       if (unlikely(gic_irq > 1020))
                handle_bad_irq(cascade_irq, desc);
        else
                generic_handle_irq(cascade_irq);
@@ -395,20 +395,20 @@  static void __init gic_dist_init(struct gic_chip_data *gic)
        cpumask = gic_get_cpumask(gic);
        cpumask |= cpumask << 8;
        cpumask |= cpumask << 16;
-       for (i = 32; i < gic_irqs; i += 4)
+       for (i = 0; i < gic_irqs; i += 4)
                writel_relaxed(cpumask, base + GIC_DIST_TARGET + i * 4 / 4);

        /*
         * Set priority on all global interrupts.
         */
-       for (i = 32; i < gic_irqs; i += 4)
+       for (i = 0; i < gic_irqs; i += 4)
                writel_relaxed(0xa0a0a0a0, base + GIC_DIST_PRI + i * 4 / 4);

        /*
         * Disable all interrupts.  Leave the PPI and SGIs alone
         * as these enables are banked registers.
         */
-       for (i = 32; i < gic_irqs; i += 32)
+       for (i = 0; i < gic_irqs; i += 32)
                writel_relaxed(0xffffffff, base + GIC_DIST_ENABLE_CLEAR + i * 4 / 32);

        writel_relaxed(1, base + GIC_DIST_CTRL);
@@ -672,7 +672,7 @@  void gic_raise_softirq(const struct cpumask *mask, unsigned int irq)
 static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
                                irq_hw_number_t hw)
 {
-       if (hw < 32) {
+       if (hw < 0) {
                irq_set_percpu_devid(irq);
                irq_set_chip_and_handler(irq, &gic_chip,
                                         handle_percpu_devid_irq);
@@ -696,8 +696,11 @@  static int gic_irq_domain_xlate(struct irq_domain *d,
			if (intsize < 3)
                return -EINVAL;

-        /* Get the interrupt number and add 16 to skip over SGIs */
-       *out_hwirq = intspec[1] + 16;
-
-       /* For SPIs, we need to add 16 more to get the GIC irq ID number */
-       if (!intspec[0])
-               *out_hwirq += 16;
-
+       *out_hwirq = intspec[1];
        *out_type = intspec[2] & IRQ_TYPE_SENSE_MASK;
        return 0;
 }
@@ -785,11 +780,11 @@  void __init gic_init_bases(unsigned int gic_nr, int irq_start,
         * For secondary GICs, skip over PPIs, too.
         */
        if (gic_nr == 0 && (irq_start & 31) > 0) {