| Message ID | 1491817485-16896-1-git-send-email-rcojocaru@bitdefender.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Mon, Apr 10, 2017 at 12:44:45PM +0300, Razvan Cojocaru wrote: > This patch adds support for testing instruction emulation when > required by the vm_event reply sent for MEM_ACCESS events. To this > end, it adds the "emulate_write" and "emulate_exec" parameters > that behave like the old "write" and "exec" parameters, except > instead of allowing writes / executes for a hit page, they emulate > the trigger instruction. The new parameters don't mark all of the > guest's pages, instead they stop at the arbitrary low limit of > the first 1000 pages - otherwise the guest would slow to a crawl. > Since the emulator is still incomplete and has trouble with > emulating competing writes in SMP scenarios, the new tests are > only meant for debugging issues. > > Signed-off-by: Razvan Cojocaru <rcojocaru@bitdefender.com> FAOD this should be acked by Tamas. I also suggest you repost this once the development window opens, regardless whether it is acked or not. Wei.
On 04/12/2017 07:50 PM, Wei Liu wrote: > On Mon, Apr 10, 2017 at 12:44:45PM +0300, Razvan Cojocaru wrote: >> This patch adds support for testing instruction emulation when >> required by the vm_event reply sent for MEM_ACCESS events. To this >> end, it adds the "emulate_write" and "emulate_exec" parameters >> that behave like the old "write" and "exec" parameters, except >> instead of allowing writes / executes for a hit page, they emulate >> the trigger instruction. The new parameters don't mark all of the >> guest's pages, instead they stop at the arbitrary low limit of >> the first 1000 pages - otherwise the guest would slow to a crawl. >> Since the emulator is still incomplete and has trouble with >> emulating competing writes in SMP scenarios, the new tests are >> only meant for debugging issues. >> >> Signed-off-by: Razvan Cojocaru <rcojocaru@bitdefender.com> > > FAOD this should be acked by Tamas. > > I also suggest you repost this once the development window opens, > regardless whether it is acked or not. Of course, he's probably just not near a computer or has not seen it. Thanks, Razvan
On Mon, Apr 10, 2017 at 3:44 AM, Razvan Cojocaru <rcojocaru@bitdefender.com> wrote: > This patch adds support for testing instruction emulation when > required by the vm_event reply sent for MEM_ACCESS events. To this > end, it adds the "emulate_write" and "emulate_exec" parameters > that behave like the old "write" and "exec" parameters, except > instead of allowing writes / executes for a hit page, they emulate > the trigger instruction. The new parameters don't mark all of the > guest's pages, instead they stop at the arbitrary low limit of > the first 1000 pages - otherwise the guest would slow to a crawl. > Since the emulator is still incomplete and has trouble with > emulating competing writes in SMP scenarios, the new tests are > only meant for debugging issues. > > Signed-off-by: Razvan Cojocaru <rcojocaru@bitdefender.com> > --- > tools/tests/xen-access/xen-access.c | 38 ++++++++++++++++++++++++++++-- > ------- > 1 file changed, 29 insertions(+), 9 deletions(-) > > diff --git a/tools/tests/xen-access/xen-access.c > b/tools/tests/xen-access/xen-access.c > index ff4d289..0ba2e45 100644 > --- a/tools/tests/xen-access/xen-access.c > +++ b/tools/tests/xen-access/xen-access.c > @@ -335,7 +335,7 @@ static void put_response(vm_event_t *vm_event, > vm_event_response_t *rsp) > > void usage(char* progname) > { > - fprintf(stderr, "Usage: %s [-m] <domain_id> write|exec", progname); > + fprintf(stderr, "Usage: %s [-m] <domain_id> write|exec|emulate_write|emulate_exec", > progname); > These options are only for x86, so they need to be moved into the #if block below. > #if defined(__i386__) || defined(__x86_64__) > fprintf(stderr, "|breakpoint|altp2m_write| > altp2m_exec|debug|cpuid|desc_access"); > #elif defined(__arm__) || defined(__aarch64__) > @@ -369,6 +369,7 @@ int main(int argc, char *argv[]) > int debug = 0; > int cpuid = 0; > int desc_access = 0; > + int emulate = 0; > uint16_t altp2m_view_id = 0; > > char* progname = argv[0]; > @@ -404,12 +405,26 @@ int main(int argc, char *argv[]) > after_first_access = XENMEM_access_rwx; > memaccess = 1; > } > + else if ( !strcmp(argv[0], "emulate_write") ) > + { > + default_access = XENMEM_access_rx; > + after_first_access = XENMEM_access_rwx; > Setting after_first_access not needed. + emulate = 1; > + memaccess = 1; > + } > else if ( !strcmp(argv[0], "exec") ) > { > default_access = XENMEM_access_rw; > after_first_access = XENMEM_access_rwx; > memaccess = 1; > } > + else if ( !strcmp(argv[0], "emulate_exec") ) > + { > + default_access = XENMEM_access_rw; > + after_first_access = XENMEM_access_rwx; > Setting after_first_access not needed. > > + emulate = 1; > + memaccess = 1; > + } > #if defined(__i386__) || defined(__x86_64__) > else if ( !strcmp(argv[0], "breakpoint") ) > { > @@ -536,7 +551,7 @@ int main(int argc, char *argv[]) > } > > rc = xc_set_mem_access(xch, domain_id, default_access, START_PFN, > - (xenaccess->max_gpfn - START_PFN) ); > + emulate ? 1000 : (xenaccess->max_gpfn - > START_PFN)); > Why only 1000? What if the domain has less then 1000? > > if ( rc < 0 ) > { > @@ -702,15 +717,20 @@ int main(int argc, char *argv[]) > } > else if ( default_access != after_first_access ) > { > - rc = xc_set_mem_access(xch, domain_id, > after_first_access, > - req.u.mem_access.gfn, 1); > - if (rc < 0) > + if ( !emulate ) > { > - ERROR("Error %d setting gfn to access_type %d\n", > rc, > - after_first_access); > - interrupted = -1; > - continue; > + rc = xc_set_mem_access(xch, domain_id, > after_first_access, > + req.u.mem_access.gfn, 1); > + if (rc < 0) > + { > + ERROR("Error %d setting gfn to access_type > %d\n", rc, > + after_first_access); > + interrupted = -1; > + continue; > + } > } > + else > + rsp.flags |= VM_EVENT_FLAG_EMULATE; > } > > rsp.u.mem_access = req.u.mem_access; > -- > 1.9.1 > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > https://lists.xen.org/xen-devel >
On 04/12/2017 08:11 PM, Tamas K Lengyel wrote: > > > On Mon, Apr 10, 2017 at 3:44 AM, Razvan Cojocaru > <rcojocaru@bitdefender.com <mailto:rcojocaru@bitdefender.com>> wrote: > > This patch adds support for testing instruction emulation when > required by the vm_event reply sent for MEM_ACCESS events. To this > end, it adds the "emulate_write" and "emulate_exec" parameters > that behave like the old "write" and "exec" parameters, except > instead of allowing writes / executes for a hit page, they emulate > the trigger instruction. The new parameters don't mark all of the > guest's pages, instead they stop at the arbitrary low limit of > the first 1000 pages - otherwise the guest would slow to a crawl. > Since the emulator is still incomplete and has trouble with > emulating competing writes in SMP scenarios, the new tests are > only meant for debugging issues. > > Signed-off-by: Razvan Cojocaru <rcojocaru@bitdefender.com > <mailto:rcojocaru@bitdefender.com>> > --- > tools/tests/xen-access/xen-access.c | 38 > ++++++++++++++++++++++++++++--------- > 1 file changed, 29 insertions(+), 9 deletions(-) > > diff --git a/tools/tests/xen-access/xen-access.c > b/tools/tests/xen-access/xen-access.c > index ff4d289..0ba2e45 100644 > --- a/tools/tests/xen-access/xen-access.c > +++ b/tools/tests/xen-access/xen-access.c > @@ -335,7 +335,7 @@ static void put_response(vm_event_t *vm_event, > vm_event_response_t *rsp) > > void usage(char* progname) > { > - fprintf(stderr, "Usage: %s [-m] <domain_id> write|exec", progname); > + fprintf(stderr, "Usage: %s [-m] <domain_id> > write|exec|emulate_write|emulate_exec", progname); > > > These options are only for x86, so they need to be moved into the #if > block below. Sure. > #if defined(__i386__) || defined(__x86_64__) > fprintf(stderr, > "|breakpoint|altp2m_write|altp2m_exec|debug|cpuid|desc_access"); > #elif defined(__arm__) || defined(__aarch64__) > @@ -369,6 +369,7 @@ int main(int argc, char *argv[]) > int debug = 0; > int cpuid = 0; > int desc_access = 0; > + int emulate = 0; > uint16_t altp2m_view_id = 0; > > char* progname = argv[0]; > @@ -404,12 +405,26 @@ int main(int argc, char *argv[]) > after_first_access = XENMEM_access_rwx; > memaccess = 1; > } > + else if ( !strcmp(argv[0], "emulate_write") ) > + { > + default_access = XENMEM_access_rx; > + after_first_access = XENMEM_access_rwx; > > > Setting after_first_access not needed. True. I got carried away with the copy-paste. > + emulate = 1; > + memaccess = 1; > + } > else if ( !strcmp(argv[0], "exec") ) > { > default_access = XENMEM_access_rw; > after_first_access = XENMEM_access_rwx; > memaccess = 1; > } > + else if ( !strcmp(argv[0], "emulate_exec") ) > + { > + default_access = XENMEM_access_rw; > + after_first_access = XENMEM_access_rwx; > > > Setting after_first_access not needed. Also true. > + emulate = 1; > + memaccess = 1; > + } > #if defined(__i386__) || defined(__x86_64__) > else if ( !strcmp(argv[0], "breakpoint") ) > { > @@ -536,7 +551,7 @@ int main(int argc, char *argv[]) > } > > rc = xc_set_mem_access(xch, domain_id, default_access, > START_PFN, > - (xenaccess->max_gpfn - START_PFN) ); > + emulate ? 1000 : > (xenaccess->max_gpfn - START_PFN)); > > > Why only 1000? What if the domain has less then 1000? Because it will kill the guest to emulate everything, and the emulator still can't handle all instructions (this is easy to see by using all the guest's pages and looking at the output of xl dmesg with loglvl=all guest_loglvl=all on the Xen command line). As for less than 1000, I'm not sure what domain that would be :), but I'm happy to change the code to min(1000, max_gpfn). Thanks, Razvan
On Thu, Apr 13, 2017 at 4:20 AM, Razvan Cojocaru <rcojocaru@bitdefender.com> wrote: > On 04/12/2017 08:11 PM, Tamas K Lengyel wrote: > > > > > > On Mon, Apr 10, 2017 at 3:44 AM, Razvan Cojocaru > > <rcojocaru@bitdefender.com <mailto:rcojocaru@bitdefender.com>> wrote: > > > > This patch adds support for testing instruction emulation when > > required by the vm_event reply sent for MEM_ACCESS events. To this > > end, it adds the "emulate_write" and "emulate_exec" parameters > > that behave like the old "write" and "exec" parameters, except > > instead of allowing writes / executes for a hit page, they emulate > > the trigger instruction. The new parameters don't mark all of the > > guest's pages, instead they stop at the arbitrary low limit of > > the first 1000 pages - otherwise the guest would slow to a crawl. > > Since the emulator is still incomplete and has trouble with > > emulating competing writes in SMP scenarios, the new tests are > > only meant for debugging issues. > > > > Signed-off-by: Razvan Cojocaru <rcojocaru@bitdefender.com > > <mailto:rcojocaru@bitdefender.com>> > > --- > > tools/tests/xen-access/xen-access.c | 38 > > ++++++++++++++++++++++++++++--------- > > 1 file changed, 29 insertions(+), 9 deletions(-) > > > > diff --git a/tools/tests/xen-access/xen-access.c > > b/tools/tests/xen-access/xen-access.c > > index ff4d289..0ba2e45 100644 > > --- a/tools/tests/xen-access/xen-access.c > > +++ b/tools/tests/xen-access/xen-access.c > > @@ -335,7 +335,7 @@ static void put_response(vm_event_t *vm_event, > > vm_event_response_t *rsp) > > > > void usage(char* progname) > > { > > - fprintf(stderr, "Usage: %s [-m] <domain_id> write|exec", > progname); > > + fprintf(stderr, "Usage: %s [-m] <domain_id> > > write|exec|emulate_write|emulate_exec", progname); > > > > > > These options are only for x86, so they need to be moved into the #if > > block below. > > Sure. > > > #if defined(__i386__) || defined(__x86_64__) > > fprintf(stderr, > > "|breakpoint|altp2m_write|altp2m_exec|debug|cpuid|desc_access"); > > #elif defined(__arm__) || defined(__aarch64__) > > @@ -369,6 +369,7 @@ int main(int argc, char *argv[]) > > int debug = 0; > > int cpuid = 0; > > int desc_access = 0; > > + int emulate = 0; > > uint16_t altp2m_view_id = 0; > > > > char* progname = argv[0]; > > @@ -404,12 +405,26 @@ int main(int argc, char *argv[]) > > after_first_access = XENMEM_access_rwx; > > memaccess = 1; > > } > > + else if ( !strcmp(argv[0], "emulate_write") ) > > + { > > + default_access = XENMEM_access_rx; > > + after_first_access = XENMEM_access_rwx; > > > > > > Setting after_first_access not needed. > > True. I got carried away with the copy-paste. > > > + emulate = 1; > > + memaccess = 1; > > + } > > else if ( !strcmp(argv[0], "exec") ) > > { > > default_access = XENMEM_access_rw; > > after_first_access = XENMEM_access_rwx; > > memaccess = 1; > > } > > + else if ( !strcmp(argv[0], "emulate_exec") ) > > + { > > + default_access = XENMEM_access_rw; > > + after_first_access = XENMEM_access_rwx; > > > > > > Setting after_first_access not needed. > > Also true. > > > + emulate = 1; > > + memaccess = 1; > > + } > > #if defined(__i386__) || defined(__x86_64__) > > else if ( !strcmp(argv[0], "breakpoint") ) > > { > > @@ -536,7 +551,7 @@ int main(int argc, char *argv[]) > > } > > > > rc = xc_set_mem_access(xch, domain_id, default_access, > > START_PFN, > > - (xenaccess->max_gpfn - START_PFN) ); > > + emulate ? 1000 : > > (xenaccess->max_gpfn - START_PFN)); > > > > > > Why only 1000? What if the domain has less then 1000? > > Because it will kill the guest to emulate everything, and the emulator > still can't handle all instructions (this is easy to see by using all > the guest's pages and looking at the output of xl dmesg with loglvl=all > guest_loglvl=all on the Xen command line). > So what's the guarantee that the emulator will work if you only do it only up to the first 1000 pages? Seems totally arbitrary to me. If the emulator can't handle all instructions then you would have to check that the instruction for which you are returning the emulate flag is in the list of instruction that can be handled.. Can such a list be derived right now? Tamas
On 04/14/2017 09:08 PM, Tamas K Lengyel wrote: > > > On Thu, Apr 13, 2017 at 4:20 AM, Razvan Cojocaru > <rcojocaru@bitdefender.com <mailto:rcojocaru@bitdefender.com>> wrote: > > On 04/12/2017 08:11 PM, Tamas K Lengyel wrote: > > > > > > On Mon, Apr 10, 2017 at 3:44 AM, Razvan Cojocaru > > + emulate = 1; > > + memaccess = 1; > > + } > > #if defined(__i386__) || defined(__x86_64__) > > else if ( !strcmp(argv[0], "breakpoint") ) > > { > > @@ -536,7 +551,7 @@ int main(int argc, char *argv[]) > > } > > > > rc = xc_set_mem_access(xch, domain_id, default_access, > > START_PFN, > > - (xenaccess->max_gpfn - START_PFN) ); > > + emulate ? 1000 : > > (xenaccess->max_gpfn - START_PFN)); > > > > > > Why only 1000? What if the domain has less then 1000? > > Because it will kill the guest to emulate everything, and the emulator > still can't handle all instructions (this is easy to see by using all > the guest's pages and looking at the output of xl dmesg with loglvl=all > guest_loglvl=all on the Xen command line). > > > So what's the guarantee that the emulator will work if you only do it > only up to the first 1000 pages? Seems totally arbitrary to me. If the > emulator can't handle all instructions then you would have to check that > the instruction for which you are returning the emulate flag is in the > list of instruction that can be handled.. Can such a list be derived > right now? If an instruction can't be emulated it will be shown as such in the ring buffer used by xl dmesg. Speaking of that, I'd like to, at some point, send a patch that sends a vm_event saying that emulation failed to userspace when that is the case, to give it a chance to do something else (for example use altp2m, or lift the page restrictions). We can also probably go through the emulator code and build an exact list of all the officially supported instructions, but I believe that that would have to be manual work - I am not aware of a tool to extract them or a header file that lists them in some structure. I'd love to be wrong about this. As for your question, there's no guarantee that the emulator will work,obut that's not why I chose 1000. I chose that number because the application will get less EPT events, and the guest will not be bogged down by handling them. But in my experiments it's also less likely to hit unhandleable instructions in the first 1000 pages since those are usually used by the guest kernel, drivers, and so on, and are less likely to cause problems. In any case, I don't mind dropping the 1000 pages limit - I can always build a custom xen-access when I need it. Thanks, Razvan
On Fri, Apr 14, 2017 at 1:03 PM, Razvan Cojocaru <rcojocaru@bitdefender.com> wrote: > On 04/14/2017 09:08 PM, Tamas K Lengyel wrote: > > > > > > On Thu, Apr 13, 2017 at 4:20 AM, Razvan Cojocaru > > <rcojocaru@bitdefender.com <mailto:rcojocaru@bitdefender.com>> wrote: > > > > On 04/12/2017 08:11 PM, Tamas K Lengyel wrote: > > > > > > > > > On Mon, Apr 10, 2017 at 3:44 AM, Razvan Cojocaru > > > + emulate = 1; > > > + memaccess = 1; > > > + } > > > #if defined(__i386__) || defined(__x86_64__) > > > else if ( !strcmp(argv[0], "breakpoint") ) > > > { > > > @@ -536,7 +551,7 @@ int main(int argc, char *argv[]) > > > } > > > > > > rc = xc_set_mem_access(xch, domain_id, default_access, > > > START_PFN, > > > - (xenaccess->max_gpfn - > START_PFN) ); > > > + emulate ? 1000 : > > > (xenaccess->max_gpfn - START_PFN)); > > > > > > > > > Why only 1000? What if the domain has less then 1000? > > > > Because it will kill the guest to emulate everything, and the > emulator > > still can't handle all instructions (this is easy to see by using all > > the guest's pages and looking at the output of xl dmesg with > loglvl=all > > guest_loglvl=all on the Xen command line). > > > > > > So what's the guarantee that the emulator will work if you only do it > > only up to the first 1000 pages? Seems totally arbitrary to me. If the > > emulator can't handle all instructions then you would have to check that > > the instruction for which you are returning the emulate flag is in the > > list of instruction that can be handled.. Can such a list be derived > > right now? > > If an instruction can't be emulated it will be shown as such in the ring > buffer used by xl dmesg. Speaking of that, I'd like to, at some point, > send a patch that sends a vm_event saying that emulation failed to > userspace when that is the case, to give it a chance to do something > else (for example use altp2m, or lift the page restrictions). > I think that would be a much needed addition to make this system more robust. > > We can also probably go through the emulator code and build an exact > list of all the officially supported instructions, but I believe that > that would have to be manual work - I am not aware of a tool to extract > them or a header file that lists them in some structure. I'd love to be > wrong about this. > > As for your question, there's no guarantee that the emulator will > work,obut that's not why I chose 1000. I chose that number because the > application will get less EPT events, and the guest will not be bogged > down by handling them. But in my experiments it's also less likely to > hit unhandleable instructions in the first 1000 pages since those are > usually used by the guest kernel, drivers, and so on, and are less > likely to cause problems. > > In any case, I don't mind dropping the 1000 pages limit - I can always > build a custom xen-access when I need it. > I don't mind setting it only for a 1000 in the test program, just wanted to understand rationale behind it. I think a comment in the program explaining what has been discussed here would also be helpful. Tamas
diff --git a/tools/tests/xen-access/xen-access.c b/tools/tests/xen-access/xen-access.c index ff4d289..0ba2e45 100644 --- a/tools/tests/xen-access/xen-access.c +++ b/tools/tests/xen-access/xen-access.c @@ -335,7 +335,7 @@ static void put_response(vm_event_t *vm_event, vm_event_response_t *rsp) void usage(char* progname) { - fprintf(stderr, "Usage: %s [-m] <domain_id> write|exec", progname); + fprintf(stderr, "Usage: %s [-m] <domain_id> write|exec|emulate_write|emulate_exec", progname); #if defined(__i386__) || defined(__x86_64__) fprintf(stderr, "|breakpoint|altp2m_write|altp2m_exec|debug|cpuid|desc_access"); #elif defined(__arm__) || defined(__aarch64__) @@ -369,6 +369,7 @@ int main(int argc, char *argv[]) int debug = 0; int cpuid = 0; int desc_access = 0; + int emulate = 0; uint16_t altp2m_view_id = 0; char* progname = argv[0]; @@ -404,12 +405,26 @@ int main(int argc, char *argv[]) after_first_access = XENMEM_access_rwx; memaccess = 1; } + else if ( !strcmp(argv[0], "emulate_write") ) + { + default_access = XENMEM_access_rx; + after_first_access = XENMEM_access_rwx; + emulate = 1; + memaccess = 1; + } else if ( !strcmp(argv[0], "exec") ) { default_access = XENMEM_access_rw; after_first_access = XENMEM_access_rwx; memaccess = 1; } + else if ( !strcmp(argv[0], "emulate_exec") ) + { + default_access = XENMEM_access_rw; + after_first_access = XENMEM_access_rwx; + emulate = 1; + memaccess = 1; + } #if defined(__i386__) || defined(__x86_64__) else if ( !strcmp(argv[0], "breakpoint") ) { @@ -536,7 +551,7 @@ int main(int argc, char *argv[]) } rc = xc_set_mem_access(xch, domain_id, default_access, START_PFN, - (xenaccess->max_gpfn - START_PFN) ); + emulate ? 1000 : (xenaccess->max_gpfn - START_PFN)); if ( rc < 0 ) { @@ -702,15 +717,20 @@ int main(int argc, char *argv[]) } else if ( default_access != after_first_access ) { - rc = xc_set_mem_access(xch, domain_id, after_first_access, - req.u.mem_access.gfn, 1); - if (rc < 0) + if ( !emulate ) { - ERROR("Error %d setting gfn to access_type %d\n", rc, - after_first_access); - interrupted = -1; - continue; + rc = xc_set_mem_access(xch, domain_id, after_first_access, + req.u.mem_access.gfn, 1); + if (rc < 0) + { + ERROR("Error %d setting gfn to access_type %d\n", rc, + after_first_access); + interrupted = -1; + continue; + } } + else + rsp.flags |= VM_EVENT_FLAG_EMULATE; } rsp.u.mem_access = req.u.mem_access;
This patch adds support for testing instruction emulation when required by the vm_event reply sent for MEM_ACCESS events. To this end, it adds the "emulate_write" and "emulate_exec" parameters that behave like the old "write" and "exec" parameters, except instead of allowing writes / executes for a hit page, they emulate the trigger instruction. The new parameters don't mark all of the guest's pages, instead they stop at the arbitrary low limit of the first 1000 pages - otherwise the guest would slow to a crawl. Since the emulator is still incomplete and has trouble with emulating competing writes in SMP scenarios, the new tests are only meant for debugging issues. Signed-off-by: Razvan Cojocaru <rcojocaru@bitdefender.com> --- tools/tests/xen-access/xen-access.c | 38 ++++++++++++++++++++++++++++--------- 1 file changed, 29 insertions(+), 9 deletions(-)