| Message ID | 1503347275-13039-11-git-send-email-volodymyr_babchuk@epam.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 21/08/17 21:27, Volodymyr Babchuk wrote: > This feature indicates that hypervisor is compatible with ARM > SMC calling convention. Hypervisor will not inject an undefined > instruction exception if an invalid SMC function were called and > will not crash a domain if an invlalid HVC functions were called. s/invlalid/invalid/ The last sentence is misleading. Xen will still inject and undefined instruction for some SMC/HVC. You may want to rework it to make it clear. > > Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com> > --- > xen/include/public/features.h | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/xen/include/public/features.h b/xen/include/public/features.h > index 2110b04..1a989b8 100644 > --- a/xen/include/public/features.h > +++ b/xen/include/public/features.h > @@ -102,6 +102,9 @@ > /* Guest can use XENMEMF_vnode to specify virtual node for memory op. */ > #define XENFEAT_memory_op_vnode_supported 13 > > +/* arm: Hypervisor supports ARM SMC calling convention. */ > +#define XENFEAT_ARM_SMCCC_supported 14 > + > #define XENFEAT_NR_SUBMAPS 1 > > #endif /* __XEN_PUBLIC_FEATURES_H__ */ > Cheers,
Hi Volodymyr, hi Julien, On 08/24/2017 07:25 PM, Julien Grall wrote: > > > On 21/08/17 21:27, Volodymyr Babchuk wrote: >> This feature indicates that hypervisor is compatible with ARM >> SMC calling convention. Hypervisor will not inject an undefined >> instruction exception if an invalid SMC function were called and >> will not crash a domain if an invlalid HVC functions were called. > > s/invlalid/invalid/ > > The last sentence is misleading. Xen will still inject and undefined > instruction for some SMC/HVC. You may want to rework it to make it clear. > Now that you say that Xen will still inject an undefined instruction exception for some SMCs, I have a to ask for which exactly? I might be off topic here, so please tell me if you believe this is not the right place for this question. In this case I will open an new thread. Right now, I am working with the previous implementation of do_trap_smc that was extended in this patch. Yet, as far as I understand, the behavior should not change, which is why I am asking this quesiton in this thread. Currently, I am working on SMC guest injections and trying to understand the resulting behavior. Every time, right after the execution of an injected SMC instruction, the guest traps into the undefined instruction exception handler in EL1 and I simply don't understand why. As far as I understand, as soon an injected SMC instruction gets executed, it should _transparently_ trap into the hypervisor (assuming MDCR_EL2.TDE is set). As soon as the hypervisor returns (e.g. to PC+4 or to the trapping PC that now contains the original instruction instead of the injected SMC), the guest should simply continue its execution. Now, according to ARM DDI0487B.a D1-1873, the following holds: "If HCR_EL2.TSC or HCR.TSC traps attempted EL1 execution of SMC instructions to EL2, that trap has priority over this disable". So this means that if SMCs are disabled for NS EL1, the guest will trap into the hypervisor on SMC execution. Yet, since SMCs are disabled from NS EL1, the guest will execute an undefined instrcution exception. Which is what I was thinking about is currently happening on my ARMv8 dev board (Lemaker Hikey). On the other hand I believe that it is highly unlikely that the EFI loader explicitly disables SMC's for NS EL1. However, since I don't have access to SCR_EL3.SMD from EL2, I can't tell whether this is the reason for the behavior I am experiencing on my board or not. It would be of great help if you would provide me with some more clarity on my case. I am sure that I have missed something that simply needs clarification. Thank you very much in advance. Thanks, ~Sergej
Hello Sergej, On 31.08.17 15:20, Sergej Proskurin wrote: > Hi Volodymyr, hi Julien, > > > On 08/24/2017 07:25 PM, Julien Grall wrote: >> >> >> On 21/08/17 21:27, Volodymyr Babchuk wrote: >>> This feature indicates that hypervisor is compatible with ARM >>> SMC calling convention. Hypervisor will not inject an undefined >>> instruction exception if an invalid SMC function were called and >>> will not crash a domain if an invlalid HVC functions were called. >> >> s/invlalid/invalid/ >> >> The last sentence is misleading. Xen will still inject and undefined >> instruction for some SMC/HVC. You may want to rework it to make it clear. >> > > Now that you say that Xen will still inject an undefined instruction > exception for some SMCs, I have a to ask for which exactly? For ones that are compatible with ARM SMCCC [1]. E.g if you are running SMCCC-compatible system and you are calling SMC/HVC with immediate value 0, then you are safe. > I might be off topic here, so please tell me if you believe this is not > the right place for this question. In this case I will open an new > thread. Right now, I am working with the previous implementation of > do_trap_smc that was extended in this patch. Yet, as far as I > understand, the behavior should not change, which is why I am asking > this quesiton in this thread. If you are talking about forwarding SMC exception to VM monitor, then yes, that should not change. > Currently, I am working on SMC guest injections and trying to understand > the resulting behavior. Every time, right after the execution of an > injected SMC instruction, the guest traps into the undefined instruction > exception handler in EL1 and I simply don't understand why. As far as I > understand, as soon an injected SMC instruction gets executed, it should > _transparently_ trap into the hypervisor (assuming MDCR_EL2.TDE is set). > As soon as the hypervisor returns (e.g. to PC+4 or to the trapping PC > that now contains the original instruction instead of the injected SMC), > the guest should simply continue its execution. Hm. What do you mean under "SMC instruction injection?". Current code in hypervisor will always inject undefined instruction exception when you call SMC (unless you installed VM monitor for the guest). Also, it will not increase PC. So, if you'll try to remove inject_undef_exception() call, you'll get into an infinite loop. > Now, according to ARM DDI0487B.a D1-1873, the following holds: "If > HCR_EL2.TSC or HCR.TSC traps attempted EL1 execution of SMC instructions > to EL2, that trap has priority over this disable". So this means that if > SMCs are disabled for NS EL1, the guest will trap into the hypervisor on > SMC execution. Yet, since SMCs are disabled from NS EL1, the guest will > execute an undefined instrcution exception. Which is what I was thinking > about is currently happening on my ARMv8 dev board (Lemaker Hikey). On > the other hand I believe that it is highly unlikely that the EFI loader > explicitly disables SMC's for NS EL1. However, since I don't have access > to SCR_EL3.SMD from EL2, I can't tell whether this is the reason for the > behavior I am experiencing on my board or not. According to ARM ARM, hypervisor should trap SMC even if was disabled by EL3. I think, that in your case the problem is in current implementation of do_trap_smc() > It would be of great help if you would provide me with some more clarity > on my case. I am sure that I have missed something that simply needs > clarification. Thank you very much in advance. I don't quite understood, what you are trying to achieve. But I think that pair of printk()s in do_trap_smc() will reveal much. [1] http://infocenter.arm.com/help/topic/com.arm.doc.den0028b/ARM_DEN0028B_SMC_Calling_Convention.pdf
Hi Volodymyr, On 08/31/2017 02:44 PM, Volodymyr Babchuk wrote: > Hello Sergej, > > On 31.08.17 15:20, Sergej Proskurin wrote: >> Hi Volodymyr, hi Julien, >> >> >> On 08/24/2017 07:25 PM, Julien Grall wrote: >>> >>> >>> On 21/08/17 21:27, Volodymyr Babchuk wrote: >>>> This feature indicates that hypervisor is compatible with ARM >>>> SMC calling convention. Hypervisor will not inject an undefined >>>> instruction exception if an invalid SMC function were called and >>>> will not crash a domain if an invlalid HVC functions were called. >>> >>> s/invlalid/invalid/ >>> >>> The last sentence is misleading. Xen will still inject and undefined >>> instruction for some SMC/HVC. You may want to rework it to make it >>> clear. >>> >> >> Now that you say that Xen will still inject an undefined instruction >> exception for some SMCs, I have a to ask for which exactly? > For ones that are compatible with ARM SMCCC [1]. E.g if you are > running SMCCC-compatible system and you are calling SMC/HVC with > immediate value 0, then you are safe. > Alright, as far as I understand this is exactly what I do right now. I inject an SMC that is encoded as 0xD4000003. >> I might be off topic here, so please tell me if you believe this is not >> the right place for this question. In this case I will open an new >> thread. Right now, I am working with the previous implementation of >> do_trap_smc that was extended in this patch. Yet, as far as I >> understand, the behavior should not change, which is why I am asking >> this quesiton in this thread. > If you are talking about forwarding SMC exception to VM monitor, then > yes, that should not change. Yes, exactly. Sorry, I forgot to mention that I have a modified xen-access version running in dom0 that registers an SMC monitor and also increases the PC by 4 (or dependent on the case, simply leaves it as it is) on every SMC trap. > >> Currently, I am working on SMC guest injections and trying to understand >> the resulting behavior. Every time, right after the execution of an >> injected SMC instruction, the guest traps into the undefined instruction >> exception handler in EL1 and I simply don't understand why. As far as I >> understand, as soon an injected SMC instruction gets executed, it should >> _transparently_ trap into the hypervisor (assuming MDCR_EL2.TDE is set). >> As soon as the hypervisor returns (e.g. to PC+4 or to the trapping PC >> that now contains the original instruction instead of the injected SMC), >> the guest should simply continue its execution. > Hm. What do you mean under "SMC instruction injection?". My code runs in dom0 and "injects" an SMC instruction to predefined addresses inside the guest as to simulate software breakpoints. By this, I mean that the code replaces the original guest instruction at a certain address with an SMC. Think of a debugger that uses software breakpoints. The idea is to put back the original instruction right after the SMC gets called, so that the guest can continue with its execution. You can find more information about that in [0], yet please consider that I try to trap the SMC directly in Xen instead of TrustZone. > Current code in hypervisor will always inject undefined instruction > exception when you call SMC (unless you installed VM monitor for the > guest). Also, it will not increase PC. So, if you'll try to remove > inject_undef_exception() call, you'll get into an infinite loop. > I have a registered SMC monitor running in dom0 that does not reinject the undefined instruction exception in do_trap_smc(). So there is no indefinite loop at this point. What I see is that as soon as my code in xen-access (dom0) increments the trapped guest PC by 4 (and also if it doesn't) the next instruction inside the guest will be inside the undef instruction handler (I can see that because I have implemented a single stepping mechanism for AArch64 in Xen that gets activated right after the guest executes the injected SMC instruction). >> Now, according to ARM DDI0487B.a D1-1873, the following holds: "If >> HCR_EL2.TSC or HCR.TSC traps attempted EL1 execution of SMC instructions >> to EL2, that trap has priority over this disable". So this means that if >> SMCs are disabled for NS EL1, the guest will trap into the hypervisor on >> SMC execution. Yet, since SMCs are disabled from NS EL1, the guest will >> execute an undefined instrcution exception. Which is what I was thinking >> about is currently happening on my ARMv8 dev board (Lemaker Hikey). On >> the other hand I believe that it is highly unlikely that the EFI loader >> explicitly disables SMC's for NS EL1. However, since I don't have access >> to SCR_EL3.SMD from EL2, I can't tell whether this is the reason for the >> behavior I am experiencing on my board or not. > According to ARM ARM, hypervisor should trap SMC even if was disabled > by EL3. I think, that in your case the problem is in current > implementation of do_trap_smc() > Unfortunately, I don't think that this is the problem of do_trap_smc() (see above). But let me check one more time. >> It would be of great help if you would provide me with some more clarity >> on my case. I am sure that I have missed something that simply needs >> clarification. Thank you very much in advance. > I don't quite understood, what you are trying to achieve. But I think > that pair of printk()s in do_trap_smc() will reveal much. > Yea, the idea is to inject SMC instructions into the guest to simulate software breakpoints for guest analysis purposes. Please let me cleanup my current printk output to better present my issue. > > [1] > http://infocenter.arm.com/help/topic/com.arm.doc.den0028b/ARM_DEN0028B_SMC_Calling_Convention.pdf Thank you, ~Sergej [0] http://www.cse.psu.edu/~trj1/papers/most14.pdf
Hi Sergej On 31.08.17 16:51, Sergej Proskurin wrote: > Hi Volodymyr, > > > On 08/31/2017 02:44 PM, Volodymyr Babchuk wrote: >> Hello Sergej, >> >> On 31.08.17 15:20, Sergej Proskurin wrote: >>> Hi Volodymyr, hi Julien, >>> >>> >>> On 08/24/2017 07:25 PM, Julien Grall wrote: >>>> >>>> >>>> On 21/08/17 21:27, Volodymyr Babchuk wrote: >>>>> This feature indicates that hypervisor is compatible with ARM >>>>> SMC calling convention. Hypervisor will not inject an undefined >>>>> instruction exception if an invalid SMC function were called and >>>>> will not crash a domain if an invlalid HVC functions were called. >>>> >>>> s/invlalid/invalid/ >>>> >>>> The last sentence is misleading. Xen will still inject and undefined >>>> instruction for some SMC/HVC. You may want to rework it to make it >>>> clear. >>>> >>> >>> Now that you say that Xen will still inject an undefined instruction >>> exception for some SMCs, I have a to ask for which exactly? >> For ones that are compatible with ARM SMCCC [1]. E.g if you are >> running SMCCC-compatible system and you are calling SMC/HVC with >> immediate value 0, then you are safe. >> > > Alright, as far as I understand this is exactly what I do right now. I > inject an SMC that is encoded as 0xD4000003. Actually, this patch series are not merged yet, so no SMCCC support right. But this should not a problem in your case. >>> I might be off topic here, so please tell me if you believe this is not >>> the right place for this question. In this case I will open an new >>> thread. Right now, I am working with the previous implementation of >>> do_trap_smc that was extended in this patch. Yet, as far as I >>> understand, the behavior should not change, which is why I am asking >>> this quesiton in this thread. >> If you are talking about forwarding SMC exception to VM monitor, then >> yes, that should not change. > > Yes, exactly. Sorry, I forgot to mention that I have a modified > xen-access version running in dom0 that registers an SMC monitor and > also increases the PC by 4 (or dependent on the case, simply leaves it > as it is) on every SMC trap. Aha, I see. I never was able to test this feature fully. I played with my own VM monitor, when I tried to offload SMC handling to another domain. But I had to comment out most of the VM monitor code in XEN. >> >>> Currently, I am working on SMC guest injections and trying to understand >>> the resulting behavior. Every time, right after the execution of an >>> injected SMC instruction, the guest traps into the undefined instruction >>> exception handler in EL1 and I simply don't understand why. As far as I >>> understand, as soon an injected SMC instruction gets executed, it should >>> _transparently_ trap into the hypervisor (assuming MDCR_EL2.TDE is set). >>> As soon as the hypervisor returns (e.g. to PC+4 or to the trapping PC >>> that now contains the original instruction instead of the injected SMC), >>> the guest should simply continue its execution. >> Hm. What do you mean under "SMC instruction injection?". > > My code runs in dom0 and "injects" an SMC instruction to predefined > addresses inside the guest as to simulate software breakpoints. By this, > I mean that the code replaces the original guest instruction at a > certain address with an SMC. Think of a debugger that uses software > breakpoints. The idea is to put back the original instruction right > after the SMC gets called, so that the guest can continue with its > execution. You can find more information about that in [0], yet please > consider that I try to trap the SMC directly in Xen instead of TrustZone. Yep, I see. Immediate question: do you flush icache after you put original instruction back? Then I can't see, why this should not work. If only VM monitor core in XEN is not broken. I don't know any users of this. I'm just curious, why are you using SMC, not BRK instruction? >> Current code in hypervisor will always inject undefined instruction >> exception when you call SMC (unless you installed VM monitor for the >> guest). Also, it will not increase PC. So, if you'll try to remove >> inject_undef_exception() call, you'll get into an infinite loop. >> > > I have a registered SMC monitor running in dom0 that does not reinject > the undefined instruction exception in do_trap_smc(). So there is no > indefinite loop at this point. What I see is that as soon as my code in > xen-access (dom0) increments the trapped guest PC by 4 (and also if it > doesn't) the next instruction inside the guest will be inside the undef > instruction handler (I can see that because I have implemented a single > stepping mechanism for AArch64 in Xen that gets activated right after > the guest executes the injected SMC instruction). That's strange. Can you print whole vCPU state to determine that PC points to the right place? Also you can check DFAR. Probably you can even dump memory pointed by DFAR to make sure that you written back correct instruction. >>> Now, according to ARM DDI0487B.a D1-1873, the following holds: "If >>> HCR_EL2.TSC or HCR.TSC traps attempted EL1 execution of SMC instructions >>> to EL2, that trap has priority over this disable". So this means that if >>> SMCs are disabled for NS EL1, the guest will trap into the hypervisor on >>> SMC execution. Yet, since SMCs are disabled from NS EL1, the guest will >>> execute an undefined instrcution exception. Which is what I was thinking >>> about is currently happening on my ARMv8 dev board (Lemaker Hikey). On >>> the other hand I believe that it is highly unlikely that the EFI loader >>> explicitly disables SMC's for NS EL1. However, since I don't have access >>> to SCR_EL3.SMD from EL2, I can't tell whether this is the reason for the >>> behavior I am experiencing on my board or not. >> According to ARM ARM, hypervisor should trap SMC even if was disabled >> by EL3. I think, that in your case the problem is in current >> implementation of do_trap_smc() >> > > Unfortunately, I don't think that this is the problem of do_trap_smc() > (see above). But let me check one more time. As I said, I know no users for SMC monitor, and I'm not exactly sure that it works properly. > >>> It would be of great help if you would provide me with some more clarity >>> on my case. I am sure that I have missed something that simply needs >>> clarification. Thank you very much in advance. >> I don't quite understood, what you are trying to achieve. But I think >> that pair of printk()s in do_trap_smc() will reveal much. >> > > Yea, the idea is to inject SMC instructions into the guest to simulate > software breakpoints for guest analysis purposes. Please let me cleanup > my current printk output to better present my issue. > >> >> [1] >> http://infocenter.arm.com/help/topic/com.arm.doc.den0028b/ARM_DEN0028B_SMC_Calling_Convention.pdf > > Thank you, > ~Sergej > > [0] http://www.cse.psu.edu/~trj1/papers/most14.pdf >
Hi Volodymyr, On 08/31/2017 04:58 PM, Volodymyr Babchuk wrote: > Hi Sergej > > On 31.08.17 16:51, Sergej Proskurin wrote: >> Hi Volodymyr, >> >> >> On 08/31/2017 02:44 PM, Volodymyr Babchuk wrote: >>> Hello Sergej, >>> >>> On 31.08.17 15:20, Sergej Proskurin wrote: >>>> Hi Volodymyr, hi Julien, >>>> >>>> >>>> On 08/24/2017 07:25 PM, Julien Grall wrote: >>>>> >>>>> >>>>> On 21/08/17 21:27, Volodymyr Babchuk wrote: >>>>>> This feature indicates that hypervisor is compatible with ARM >>>>>> SMC calling convention. Hypervisor will not inject an undefined >>>>>> instruction exception if an invalid SMC function were called and >>>>>> will not crash a domain if an invlalid HVC functions were called. >>>>> >>>>> s/invlalid/invalid/ >>>>> >>>>> The last sentence is misleading. Xen will still inject and undefined >>>>> instruction for some SMC/HVC. You may want to rework it to make it >>>>> clear. >>>>> >>>> >>>> Now that you say that Xen will still inject an undefined instruction >>>> exception for some SMCs, I have a to ask for which exactly? >>> For ones that are compatible with ARM SMCCC [1]. E.g if you are >>> running SMCCC-compatible system and you are calling SMC/HVC with >>> immediate value 0, then you are safe. >>> >> >> Alright, as far as I understand this is exactly what I do right now. I >> inject an SMC that is encoded as 0xD4000003. > Actually, this patch series are not merged yet, so no SMCCC support > right. But this should not a problem in your case. > >>>> I might be off topic here, so please tell me if you believe this is >>>> not >>>> the right place for this question. In this case I will open an new >>>> thread. Right now, I am working with the previous implementation of >>>> do_trap_smc that was extended in this patch. Yet, as far as I >>>> understand, the behavior should not change, which is why I am asking >>>> this quesiton in this thread. >>> If you are talking about forwarding SMC exception to VM monitor, then >>> yes, that should not change. >> >> Yes, exactly. Sorry, I forgot to mention that I have a modified >> xen-access version running in dom0 that registers an SMC monitor and >> also increases the PC by 4 (or dependent on the case, simply leaves it >> as it is) on every SMC trap. > Aha, I see. I never was able to test this feature fully. I played with > my own VM monitor, when I tried to offload SMC handling to another > domain. But I had to comment out most of the VM monitor code in XEN. > >>> >>>> Currently, I am working on SMC guest injections and trying to >>>> understand >>>> the resulting behavior. Every time, right after the execution of an >>>> injected SMC instruction, the guest traps into the undefined >>>> instruction >>>> exception handler in EL1 and I simply don't understand why. As far >>>> as I >>>> understand, as soon an injected SMC instruction gets executed, it >>>> should >>>> _transparently_ trap into the hypervisor (assuming MDCR_EL2.TDE is >>>> set). >>>> As soon as the hypervisor returns (e.g. to PC+4 or to the trapping PC >>>> that now contains the original instruction instead of the injected >>>> SMC), >>>> the guest should simply continue its execution. >>> Hm. What do you mean under "SMC instruction injection?". >> >> My code runs in dom0 and "injects" an SMC instruction to predefined >> addresses inside the guest as to simulate software breakpoints. By this, >> I mean that the code replaces the original guest instruction at a >> certain address with an SMC. Think of a debugger that uses software >> breakpoints. The idea is to put back the original instruction right >> after the SMC gets called, so that the guest can continue with its >> execution. You can find more information about that in [0], yet please >> consider that I try to trap the SMC directly in Xen instead of >> TrustZone. > Yep, I see. Immediate question: do you flush icache after you put > original instruction back? Yeap. But the current behavior does not let me to go this far, as I the system jumps into the interrupt handler and single-steps the handler instead of the instruction of interest. > Then I can't see, why this should not work. If only VM monitor core in > XEN is not broken. I don't know any users of this. > I'm just curious, why are you using SMC, not BRK instruction? > I use SMC instructions as the guest can register for BRK events. The guest cannot register for SMC events. So, in order stay stealthy towards the guest and also not to cope with BRK re-injections, SMC's seemed to be the right choice :) >>> Current code in hypervisor will always inject undefined instruction >>> exception when you call SMC (unless you installed VM monitor for the >>> guest). Also, it will not increase PC. So, if you'll try to remove >>> inject_undef_exception() call, you'll get into an infinite loop. >>> >> >> I have a registered SMC monitor running in dom0 that does not reinject >> the undefined instruction exception in do_trap_smc(). So there is no >> indefinite loop at this point. What I see is that as soon as my code in >> xen-access (dom0) increments the trapped guest PC by 4 (and also if it >> doesn't) the next instruction inside the guest will be inside the undef >> instruction handler (I can see that because I have implemented a single >> stepping mechanism for AArch64 in Xen that gets activated right after >> the guest executes the injected SMC instruction). > That's strange. Can you print whole vCPU state to determine that PC > points to the right place? Also you can check DFAR. Probably you can > even dump memory pointed by DFAR to make sure that you written back > correct instruction. Yea, I do that. And both the SMC injection, as well as further vCPU state seems to be correct at this point. Today, I saw an interesting behavior in my single-stepping implementation, which is the reason for my late reply. I can't explain what is going wrong, yet. So I will need to further investigate this behavior and post and RFC for the single-stepping mechanism as to put more eyes on the issue. Maybe, this will help solve it. But anyway, thank you very much for your help! I really appreciate it :) > >>>> Now, according to ARM DDI0487B.a D1-1873, the following holds: "If >>>> HCR_EL2.TSC or HCR.TSC traps attempted EL1 execution of SMC >>>> instructions >>>> to EL2, that trap has priority over this disable". So this means >>>> that if >>>> SMCs are disabled for NS EL1, the guest will trap into the >>>> hypervisor on >>>> SMC execution. Yet, since SMCs are disabled from NS EL1, the guest >>>> will >>>> execute an undefined instrcution exception. Which is what I was >>>> thinking >>>> about is currently happening on my ARMv8 dev board (Lemaker Hikey). On >>>> the other hand I believe that it is highly unlikely that the EFI >>>> loader >>>> explicitly disables SMC's for NS EL1. However, since I don't have >>>> access >>>> to SCR_EL3.SMD from EL2, I can't tell whether this is the reason >>>> for the >>>> behavior I am experiencing on my board or not. >>> According to ARM ARM, hypervisor should trap SMC even if was disabled >>> by EL3. I think, that in your case the problem is in current >>> implementation of do_trap_smc() >>> >> >> Unfortunately, I don't think that this is the problem of do_trap_smc() >> (see above). But let me check one more time. > As I said, I know no users for SMC monitor, and I'm not exactly sure > that it works properly. > >> >>>> It would be of great help if you would provide me with some more >>>> clarity >>>> on my case. I am sure that I have missed something that simply needs >>>> clarification. Thank you very much in advance. >>> I don't quite understood, what you are trying to achieve. But I think >>> that pair of printk()s in do_trap_smc() will reveal much. >>> >> >> Yea, the idea is to inject SMC instructions into the guest to simulate >> software breakpoints for guest analysis purposes. Please let me cleanup >> my current printk output to better present my issue. >> >>> >>> [1] >>> http://infocenter.arm.com/help/topic/com.arm.doc.den0028b/ARM_DEN0028B_SMC_Calling_Convention.pdf >>> >> >> Thank you, >> ~Sergej >> >> [0] http://www.cse.psu.edu/~trj1/papers/most14.pdf >> Thanks, ~Sergej
Hello, Sorry for the formatting, writing from my phone. Ki On Thu, 31 Aug 2017, 22:18 Sergej Proskurin <proskurin@sec.in.tum.de> wrote: > Hi Volodymyr, > > > On 08/31/2017 04:58 PM, Volodymyr Babchuk wrote: > > Hi Sergej > > > > On 31.08.17 16:51, Sergej Proskurin wrote: > >> Hi Volodymyr, > >> > >> > >> On 08/31/2017 02:44 PM, Volodymyr Babchuk wrote: > >>> Hello Sergej, > >>> > >>> On 31.08.17 15:20, Sergej Proskurin wrote: > >>>> Hi Volodymyr, hi Julien, > >>>> > >>>> > >>>> On 08/24/2017 07:25 PM, Julien Grall wrote: > >>>>> > >>>>> > >>>>> On 21/08/17 21:27, Volodymyr Babchuk wrote: > >>>>>> This feature indicates that hypervisor is compatible with ARM > >>>>>> SMC calling convention. Hypervisor will not inject an undefined > >>>>>> instruction exception if an invalid SMC function were called and > >>>>>> will not crash a domain if an invlalid HVC functions were called. > >>>>> > >>>>> s/invlalid/invalid/ > >>>>> > >>>>> The last sentence is misleading. Xen will still inject and undefined > >>>>> instruction for some SMC/HVC. You may want to rework it to make it > >>>>> clear. > >>>>> > >>>> > >>>> Now that you say that Xen will still inject an undefined instruction > >>>> exception for some SMCs, I have a to ask for which exactly? > >>> For ones that are compatible with ARM SMCCC [1]. E.g if you are > >>> running SMCCC-compatible system and you are calling SMC/HVC with > >>> immediate value 0, then you are safe. > >>> > >> > >> Alright, as far as I understand this is exactly what I do right now. I > >> inject an SMC that is encoded as 0xD4000003. > > Actually, this patch series are not merged yet, so no SMCCC support > > right. But this should not a problem in your case. > > > >>>> I might be off topic here, so please tell me if you believe this is > >>>> not > >>>> the right place for this question. In this case I will open an new > >>>> thread. Right now, I am working with the previous implementation of > >>>> do_trap_smc that was extended in this patch. Yet, as far as I > >>>> understand, the behavior should not change, which is why I am asking > >>>> this quesiton in this thread. > >>> If you are talking about forwarding SMC exception to VM monitor, then > >>> yes, that should not change. > >> > >> Yes, exactly. Sorry, I forgot to mention that I have a modified > >> xen-access version running in dom0 that registers an SMC monitor and > >> also increases the PC by 4 (or dependent on the case, simply leaves it > >> as it is) on every SMC trap. > > Aha, I see. I never was able to test this feature fully. I played with > > my own VM monitor, when I tried to offload SMC handling to another > > domain. But I had to comment out most of the VM monitor code in XEN. > > > >>> > >>>> Currently, I am working on SMC guest injections and trying to > >>>> understand > >>>> the resulting behavior. Every time, right after the execution of an > >>>> injected SMC instruction, the guest traps into the undefined > >>>> instruction > >>>> exception handler in EL1 and I simply don't understand why. As far > >>>> as I > >>>> understand, as soon an injected SMC instruction gets executed, it > >>>> should > >>>> _transparently_ trap into the hypervisor (assuming MDCR_EL2.TDE is > >>>> set). > >>>> As soon as the hypervisor returns (e.g. to PC+4 or to the trapping PC > >>>> that now contains the original instruction instead of the injected > >>>> SMC), > >>>> the guest should simply continue its execution. > >>> Hm. What do you mean under "SMC instruction injection?". > >> > >> My code runs in dom0 and "injects" an SMC instruction to predefined > >> addresses inside the guest as to simulate software breakpoints. By this, > >> I mean that the code replaces the original guest instruction at a > >> certain address with an SMC. Think of a debugger that uses software > >> breakpoints. The idea is to put back the original instruction right > >> after the SMC gets called, so that the guest can continue with its > >> execution. You can find more information about that in [0], yet please > >> consider that I try to trap the SMC directly in Xen instead of > >> TrustZone. > > Yep, I see. Immediate question: do you flush icache after you put > > original instruction back? > > Yeap. But the current behavior does not let me to go this far, as I the > system jumps into the interrupt handler and single-steps the handler > instead of the instruction of interest. On your first mail, you started with "smc injection doesn't work", then "I replace instruction" and now you mention about single-stepping. This doesn't help at all to understand what you are doing and really not related to this thread. So can you please details exactly what you are doing rather than giving bits by bits? > > Then I can't see, why this should not work. If only VM monitor core in > > XEN is not broken. I don't know any users of this. > > I'm just curious, why are you using SMC, not BRK instruction? > > > > I use SMC instructions as the guest can register for BRK events. The > guest cannot register for SMC events. So, in order stay stealthy towards > the guest and also not to cope with BRK re-injections, SMC's seemed to > be the right choice : I have already said that using SMC is a pretty bad idea when Tamas added the trapping and you guys still seem to think it is a good idea... > >>> Current code in hypervisor will always inject undefined instruction > >>> exception when you call SMC (unless you installed VM monitor for the > >>> guest). Also, it will not increase PC. So, if you'll try to remove > >>> inject_undef_exception() call, you'll get into an infinite loop. > >>> > >> > >> I have a registered SMC monitor running in dom0 that does not reinject > >> the undefined instruction exception in do_trap_smc(). So there is no > >> indefinite loop at this point. What I see is that as soon as my code in > >> xen-access (dom0) increments the trapped guest PC by 4 (and also if it > >> doesn't) the next instruction inside the guest will be inside the undef > >> instruction handler (I can see that because I have implemented a single > >> stepping mechanism for AArch64 in Xen that gets activated right after > >> the guest executes the injected SMC instruction). > > That's strange. Can you print whole vCPU state to determine that PC > > points to the right place? Also you can check DFAR. Probably you can > > even dump memory pointed by DFAR to make sure that you written back > > correct instruction. > > Yea, I do that. And both the SMC injection, as well as further vCPU > state seems to be correct at this point. > > Today, I saw an interesting behavior in my single-stepping > implementation, which is the reason for my late reply. I can't explain > what is going wrong, yet. So I will need to further investigate this > behavior and post and RFC for the single-stepping mechanism as to put > more eyes on the issue. Maybe, this will help solve it. > > But anyway, thank you very much for your help! I really appreciate it :) > You probably want to look at https://lists.xen.org/archives/html/xen-devel/2017-08/msg00661.html and maybe sync-up with this person if you are not working with him. Cheers, > > > >>>> Now, according to ARM DDI0487B.a D1-1873, the following holds: "If > >>>> HCR_EL2.TSC or HCR.TSC traps attempted EL1 execution of SMC > >>>> instructions > >>>> to EL2, that trap has priority over this disable". So this means > >>>> that if > >>>> SMCs are disabled for NS EL1, the guest will trap into the > >>>> hypervisor on > >>>> SMC execution. Yet, since SMCs are disabled from NS EL1, the guest > >>>> will > >>>> execute an undefined instrcution exception. Which is what I was > >>>> thinking > >>>> about is currently happening on my ARMv8 dev board (Lemaker Hikey). On > >>>> the other hand I believe that it is highly unlikely that the EFI > >>>> loader > >>>> explicitly disables SMC's for NS EL1. However, since I don't have > >>>> access > >>>> to SCR_EL3.SMD from EL2, I can't tell whether this is the reason > >>>> for the > >>>> behavior I am experiencing on my board or not. > >>> According to ARM ARM, hypervisor should trap SMC even if was disabled > >>> by EL3. I think, that in your case the problem is in current > >>> implementation of do_trap_smc() > >>> > >> > >> Unfortunately, I don't think that this is the problem of do_trap_smc() > >> (see above). But let me check one more time. > > As I said, I know no users for SMC monitor, and I'm not exactly sure > > that it works properly. > > > >> > >>>> It would be of great help if you would provide me with some more > >>>> clarity > >>>> on my case. I am sure that I have missed something that simply needs > >>>> clarification. Thank you very much in advance. > >>> I don't quite understood, what you are trying to achieve. But I think > >>> that pair of printk()s in do_trap_smc() will reveal much. > >>> > >> > >> Yea, the idea is to inject SMC instructions into the guest to simulate > >> software breakpoints for guest analysis purposes. Please let me cleanup > >> my current printk output to better present my issue. > >> > >>> > >>> [1] > >>> > http://infocenter.arm.com/help/topic/com.arm.doc.den0028b/ARM_DEN0028B_SMC_Calling_Convention.pdf > >>> > >> > >> Thank you, > >> ~Sergej > >> > >> [0] http://www.cse.psu.edu/~trj1/papers/most14.pdf > >> > > Thanks, > ~Sergej > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > https://lists.xen.org/xen-devel >
Hi Julien, On 09/04/2017 08:07 AM, Julien Grall wrote: > Hello, > > Sorry for the formatting, writing from my phone. Ki > > On Thu, 31 Aug 2017, 22:18 Sergej Proskurin <proskurin@sec.in.tum.de> wrote: > [...] > > On your first mail, you started with "smc injection doesn't work", then "I > replace instruction" and now you mention about single-stepping. > > This doesn't help at all to understand what you are doing and really not > related to this thread. > > So can you please details exactly what you are doing rather than giving > bits by bits? > I will provide more information in a separate thread soon so that the actual issue, hopefully, will become clearer. Thank you. >> I use SMC instructions as the guest can register for BRK events. The >> guest cannot register for SMC events. So, in order stay stealthy towards >> the guest and also not to cope with BRK re-injections, SMC's seemed to >> be the right choice : > > I have already said that using SMC is a pretty bad idea when Tamas added > the trapping and you guys still seem to think it is a good idea... I did not know about this conversation with Tamas. Why do you believe that using SMC instructions is not a good idea? Could you please refer me to the particular thread? Thank you. >>>>> Current code in hypervisor will always inject undefined instruction >>>>> exception when you call SMC (unless you installed VM monitor for the >>>>> guest). Also, it will not increase PC. So, if you'll try to remove >>>>> inject_undef_exception() call, you'll get into an infinite loop. >>>>> >>>> I have a registered SMC monitor running in dom0 that does not reinject >>>> the undefined instruction exception in do_trap_smc(). So there is no >>>> indefinite loop at this point. What I see is that as soon as my code in >>>> xen-access (dom0) increments the trapped guest PC by 4 (and also if it >>>> doesn't) the next instruction inside the guest will be inside the undef >>>> instruction handler (I can see that because I have implemented a single >>>> stepping mechanism for AArch64 in Xen that gets activated right after >>>> the guest executes the injected SMC instruction). >>> That's strange. Can you print whole vCPU state to determine that PC >>> points to the right place? Also you can check DFAR. Probably you can >>> even dump memory pointed by DFAR to make sure that you written back >>> correct instruction. >> Yea, I do that. And both the SMC injection, as well as further vCPU >> state seems to be correct at this point. >> >> Today, I saw an interesting behavior in my single-stepping >> implementation, which is the reason for my late reply. I can't explain >> what is going wrong, yet. So I will need to further investigate this >> behavior and post and RFC for the single-stepping mechanism as to put >> more eyes on the issue. Maybe, this will help solve it. >> >> But anyway, thank you very much for your help! I really appreciate it :) >> > You probably want to look at > https://lists.xen.org/archives/html/xen-devel/2017-08/msg00661.html and > maybe sync-up with this person if you are not working with him. Thanks, for mentioning that. Florian is a student of mine who has also looked at single-stepping on ARMv8. We have collaborated on this topic together. I will take over on that, as his work goes slightly into a different direction. Thanks, ~Sergej
On 04/09/17 10:57, Sergej Proskurin wrote: > Hi Julien, > > > On 09/04/2017 08:07 AM, Julien Grall wrote: >> Hello, >> >> Sorry for the formatting, writing from my phone. Ki >> >> On Thu, 31 Aug 2017, 22:18 Sergej Proskurin <proskurin@sec.in.tum.de> wrote: >> > > [...] > >> >> On your first mail, you started with "smc injection doesn't work", then "I >> replace instruction" and now you mention about single-stepping. >> >> This doesn't help at all to understand what you are doing and really not >> related to this thread. >> >> So can you please details exactly what you are doing rather than giving >> bits by bits? >> > > I will provide more information in a separate thread soon so that the > actual issue, hopefully, will become clearer. Thank you. > >>> I use SMC instructions as the guest can register for BRK events. The >>> guest cannot register for SMC events. So, in order stay stealthy towards >>> the guest and also not to cope with BRK re-injections, SMC's seemed to >>> be the right choice : >> >> I have already said that using SMC is a pretty bad idea when Tamas added >> the trapping and you guys still seem to think it is a good idea... > > I did not know about this conversation with Tamas. Why do you believe > that using SMC instructions is not a good idea? Could you please refer > me to the particular thread? Thank you. I am not sure on which thread it was discussed. https://lists.gt.net/xen/devel/427459 may contain some details. By definition SMC is for Secure Monitor Call. It might be possible to have a guest access the secure firmware (e.g imagine an Android guest doing video decoding). Given that you don't identify the immediate of the SMC (which is BTW not easily available on ARMv7), you cannot have both interacting together. And even with that you don't know if the SMC #imm might be used by the firmware... Furthermore, SMC cannot be executed at EL0 so you can't monitor application. They also won't be available if the platform that does not have EL3. So yes SMC is a pretty bad choice here if you want to get a generic solution. Cheers,
diff --git a/xen/include/public/features.h b/xen/include/public/features.h index 2110b04..1a989b8 100644 --- a/xen/include/public/features.h +++ b/xen/include/public/features.h @@ -102,6 +102,9 @@ /* Guest can use XENMEMF_vnode to specify virtual node for memory op. */ #define XENFEAT_memory_op_vnode_supported 13 +/* arm: Hypervisor supports ARM SMC calling convention. */ +#define XENFEAT_ARM_SMCCC_supported 14 + #define XENFEAT_NR_SUBMAPS 1 #endif /* __XEN_PUBLIC_FEATURES_H__ */
This feature indicates that hypervisor is compatible with ARM SMC calling convention. Hypervisor will not inject an undefined instruction exception if an invalid SMC function were called and will not crash a domain if an invlalid HVC functions were called. Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com> --- xen/include/public/features.h | 3 +++ 1 file changed, 3 insertions(+)