xen-mapcache: avoid a race on memory map while using MAP_FIXED

Commit Message

Igor Druzhinin April 20, 2021, 3:35 a.m. UTC

When we're replacing the existing mapping there is possibility of a race
on memory map with other threads doing mmap operations - the address being
unmapped/re-mapped could be occupied by another thread in between.

Linux mmap man page recommends keeping the existing mappings in place to
reserve the place and instead utilize the fact that the next mmap operation
with MAP_FIXED flag passed will implicitly destroy the existing mappings
behind the chosen address. This behavior is guaranteed by POSIX / BSD and
therefore is portable.

Note that it wouldn't make the replacement atomic for parallel accesses to
the replaced region - those might still fail with SIGBUS due to
xenforeignmemory_map not being atomic. So we're still not expecting those.

Tested-by: Anthony PERARD <[e-mail]>
Signed-off-by: Igor Druzhinin <[e-mail]>
---
 hw/i386/xen/xen-mapcache.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

Comments

[e-mail] April 20, 2021, 3:39 a.m. UTC | #1

Patchew URL: https://patchew.org/QEMU/[e-mail]/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: [e-mail]
Subject: [PATCH] xen-mapcache: avoid a race on memory map while using MAP_FIXED

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]         patchew/[e-mail] -> patchew/[e-mail]
Switched to a new branch 'test'
3102519 xen-mapcache: avoid a race on memory map while using MAP_FIXED

=== OUTPUT BEGIN ===
ERROR: Author email address is mangled by the mailing list
#2: 
Author: Igor Druzhinin via <[e-mail]>

total: 1 errors, 0 warnings, 21 lines checked

Commit 31025199a1b4 (xen-mapcache: avoid a race on memory map while using MAP_FIXED) has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/[e-mail]/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [e-mail]

Paul Durrant April 20, 2021, 7:03 a.m. UTC | #2

On 20/04/2021 04:35, Igor Druzhinin wrote:
> When we're replacing the existing mapping there is possibility of a race
> on memory map with other threads doing mmap operations - the address being
> unmapped/re-mapped could be occupied by another thread in between.
> 
> Linux mmap man page recommends keeping the existing mappings in place to
> reserve the place and instead utilize the fact that the next mmap operation
> with MAP_FIXED flag passed will implicitly destroy the existing mappings
> behind the chosen address. This behavior is guaranteed by POSIX / BSD and
> therefore is portable.
> 
> Note that it wouldn't make the replacement atomic for parallel accesses to
> the replaced region - those might still fail with SIGBUS due to
> xenforeignmemory_map not being atomic. So we're still not expecting those.
> 
> Tested-by: Anthony PERARD <[e-mail]>
> Signed-off-by: Igor Druzhinin <[e-mail]>

Reviewed-by: Paul Durrant <[e-mail]>

> ---
>   hw/i386/xen/xen-mapcache.c | 15 ++++++++++++++-
>   1 file changed, 14 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/i386/xen/xen-mapcache.c b/hw/i386/xen/xen-mapcache.c
> index 5b120ed..e82b7dc 100644
> --- a/hw/i386/xen/xen-mapcache.c
> +++ b/hw/i386/xen/xen-mapcache.c
> @@ -171,7 +171,20 @@ static void xen_remap_bucket(MapCacheEntry *entry,
>           if (!(entry->flags & XEN_MAPCACHE_ENTRY_DUMMY)) {
>               ram_block_notify_remove(entry->vaddr_base, entry->size);
>           }
> -        if (munmap(entry->vaddr_base, entry->size) != 0) {
> +
> +        /*
> +         * If an entry is being replaced by another mapping and we're using
> +         * MAP_FIXED flag for it - there is possibility of a race for vaddr
> +         * address with another thread doing an mmap call itself
> +         * (see man 2 mmap). To avoid that we skip explicit unmapping here
> +         * and allow the kernel to destroy the previous mappings by replacing
> +         * them in mmap call later.
> +         *
> +         * Non-identical replacements are not allowed therefore.
> +         */
> +        assert(!vaddr || (entry->vaddr_base == vaddr && entry->size == size));
> +
> +        if (!vaddr && munmap(entry->vaddr_base, entry->size) != 0) {
>               perror("unmap fails");
>               exit(-1);
>           }
>

Roger Pau Monné April 20, 2021, 8:53 a.m. UTC | #3

On Tue, Apr 20, 2021 at 04:35:02AM +0100, Igor Druzhinin wrote:
> When we're replacing the existing mapping there is possibility of a race
> on memory map with other threads doing mmap operations - the address being
> unmapped/re-mapped could be occupied by another thread in between.
> 
> Linux mmap man page recommends keeping the existing mappings in place to
> reserve the place and instead utilize the fact that the next mmap operation
> with MAP_FIXED flag passed will implicitly destroy the existing mappings
> behind the chosen address. This behavior is guaranteed by POSIX / BSD and
> therefore is portable.
> 
> Note that it wouldn't make the replacement atomic for parallel accesses to
> the replaced region - those might still fail with SIGBUS due to
> xenforeignmemory_map not being atomic. So we're still not expecting those.
> 
> Tested-by: Anthony PERARD <[e-mail]>
> Signed-off-by: Igor Druzhinin <[e-mail]>

Should we add a 'Fixes: 759235653de ...' or similar tag here?

> ---
>  hw/i386/xen/xen-mapcache.c | 15 ++++++++++++++-
>  1 file changed, 14 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/i386/xen/xen-mapcache.c b/hw/i386/xen/xen-mapcache.c
> index 5b120ed..e82b7dc 100644
> --- a/hw/i386/xen/xen-mapcache.c
> +++ b/hw/i386/xen/xen-mapcache.c
> @@ -171,7 +171,20 @@ static void xen_remap_bucket(MapCacheEntry *entry,
>          if (!(entry->flags & XEN_MAPCACHE_ENTRY_DUMMY)) {
>              ram_block_notify_remove(entry->vaddr_base, entry->size);
>          }
> -        if (munmap(entry->vaddr_base, entry->size) != 0) {
> +
> +        /*
> +         * If an entry is being replaced by another mapping and we're using
> +         * MAP_FIXED flag for it - there is possibility of a race for vaddr
> +         * address with another thread doing an mmap call itself
> +         * (see man 2 mmap). To avoid that we skip explicit unmapping here
> +         * and allow the kernel to destroy the previous mappings by replacing
> +         * them in mmap call later.
> +         *
> +         * Non-identical replacements are not allowed therefore.
> +         */
> +        assert(!vaddr || (entry->vaddr_base == vaddr && entry->size == size));
> +
> +        if (!vaddr && munmap(entry->vaddr_base, entry->size) != 0) {

Oh, so remappings of foreign addresses must always use the same
virtual address space, I somehow assumed that you could remap an
existing foreign map (or dummy mapping) into a different virtual
address if you so wanted.

Thanks, Roger.

Igor Druzhinin April 20, 2021, 9:45 a.m. UTC | #4

On 20/04/2021 09:53, Roger Pau Monné wrote:
> On Tue, Apr 20, 2021 at 04:35:02AM +0100, Igor Druzhinin wrote:
>> When we're replacing the existing mapping there is possibility of a race
>> on memory map with other threads doing mmap operations - the address being
>> unmapped/re-mapped could be occupied by another thread in between.
>>
>> Linux mmap man page recommends keeping the existing mappings in place to
>> reserve the place and instead utilize the fact that the next mmap operation
>> with MAP_FIXED flag passed will implicitly destroy the existing mappings
>> behind the chosen address. This behavior is guaranteed by POSIX / BSD and
>> therefore is portable.
>>
>> Note that it wouldn't make the replacement atomic for parallel accesses to
>> the replaced region - those might still fail with SIGBUS due to
>> xenforeignmemory_map not being atomic. So we're still not expecting those.
>>
>> Tested-by: Anthony PERARD <[e-mail]>
>> Signed-off-by: Igor Druzhinin <[e-mail]>
> 
> Should we add a 'Fixes: 759235653de ...' or similar tag here?

I was thinking about it and decided - no. There wasn't a bug here until 
QEMU introduced usage of iothreads at the state loading phase. 
Originally this process was totally single-threaded. And it's hard to 
pinpoint the exact moment when it happened which is also not directly 
related to the change here.

Igor

Igor Druzhinin April 20, 2021, 9:51 a.m. UTC | #5

On 20/04/2021 04:39, [e-mail] wrote:
> Patchew URL: https://patchew.org/QEMU/[e-mail]/
> 
> 
> 
> Hi,
> 
> This series seems to have some coding style problems. See output below for
> more information:
> 
> Type: series
> Message-id: [e-mail]
> Subject: [PATCH] xen-mapcache: avoid a race on memory map while using MAP_FIXED
> 
> === TEST SCRIPT BEGIN ===
> #!/bin/bash
> git rev-parse base > /dev/null || exit 0
> git config --local diff.renamelimit 0
> git config --local diff.renames True
> git config --local diff.algorithm histogram
> ./scripts/checkpatch.pl --mailback base..
> === TEST SCRIPT END ===
> 
> Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
>  From https://github.com/patchew-project/qemu
>   * [new tag]         patchew/[e-mail] -> patchew/[e-mail]
> Switched to a new branch 'test'
> 3102519 xen-mapcache: avoid a race on memory map while using MAP_FIXED
> 
> === OUTPUT BEGIN ===
> ERROR: Author email address is mangled by the mailing list
> #2:
> Author: Igor Druzhinin via <[e-mail]>
> 
> total: 1 errors, 0 warnings, 21 lines checked
>

Anthony,

Is there any action required from me here? I don't completely understand 
how that happened. But I found this:

"In some cases the Author: email address in patches submitted to the
list gets mangled such that it says

     John Doe via Qemu-devel <[e-mail]>

This change is a result of workarounds for DMARC policies.

Subsystem maintainers accepting patches need to catch these and fix
them before sending pull requests, so a checkpatch.pl test is highly
desirable."

Igor

Roger Pau Monné April 20, 2021, 10:26 a.m. UTC | #6

On Tue, Apr 20, 2021 at 10:45:03AM +0100, Igor Druzhinin wrote:
> On 20/04/2021 09:53, Roger Pau Monné wrote:
> > On Tue, Apr 20, 2021 at 04:35:02AM +0100, Igor Druzhinin wrote:
> > > When we're replacing the existing mapping there is possibility of a race
> > > on memory map with other threads doing mmap operations - the address being
> > > unmapped/re-mapped could be occupied by another thread in between.
> > > 
> > > Linux mmap man page recommends keeping the existing mappings in place to
> > > reserve the place and instead utilize the fact that the next mmap operation
> > > with MAP_FIXED flag passed will implicitly destroy the existing mappings
> > > behind the chosen address. This behavior is guaranteed by POSIX / BSD and
> > > therefore is portable.
> > > 
> > > Note that it wouldn't make the replacement atomic for parallel accesses to
> > > the replaced region - those might still fail with SIGBUS due to
> > > xenforeignmemory_map not being atomic. So we're still not expecting those.
> > > 
> > > Tested-by: Anthony PERARD <[e-mail]>
> > > Signed-off-by: Igor Druzhinin <[e-mail]>
> > 
> > Should we add a 'Fixes: 759235653de ...' or similar tag here?
> 
> I was thinking about it and decided - no. There wasn't a bug here until QEMU
> introduced usage of iothreads at the state loading phase. Originally this
> process was totally single-threaded. And it's hard to pinpoint the exact
> moment when it happened which is also not directly related to the change
> here.

Right, might be worth mentioning in the commit message then, that the
code was designed to be used without any parallelism, and that the
addition of iothreads is what actually triggered the bug here.

Thanks, Roger.

Anthony PERARD April 20, 2021, 10:51 a.m. UTC | #7

On Tue, Apr 20, 2021 at 10:51:47AM +0100, Igor Druzhinin wrote:
> On 20/04/2021 04:39, [e-mail] wrote:
> > === OUTPUT BEGIN ===
> > ERROR: Author email address is mangled by the mailing list
> > #2:
> > Author: Igor Druzhinin via <[e-mail]>
> > 
> > total: 1 errors, 0 warnings, 21 lines checked
> > 
> 
> Anthony,
> 
> Is there any action required from me here? I don't completely understand how
> that happened. But I found this:

No action, I just need to remember to fix the patch's author before
submitting a pull request.

Citrix's policy is just too strict and don't even allow space changes in
some headers.

Patch

diff --git a/hw/i386/xen/xen-mapcache.c b/hw/i386/xen/xen-mapcache.c
index 5b120ed..e82b7dc 100644
--- a/hw/i386/xen/xen-mapcache.c
+++ b/hw/i386/xen/xen-mapcache.c
@@ -171,7 +171,20 @@  static void xen_remap_bucket(MapCacheEntry *entry,
         if (!(entry->flags & XEN_MAPCACHE_ENTRY_DUMMY)) {
             ram_block_notify_remove(entry->vaddr_base, entry->size);
         }
-        if (munmap(entry->vaddr_base, entry->size) != 0) {
+
+        /*
+         * If an entry is being replaced by another mapping and we're using
+         * MAP_FIXED flag for it - there is possibility of a race for vaddr
+         * address with another thread doing an mmap call itself
+         * (see man 2 mmap). To avoid that we skip explicit unmapping here
+         * and allow the kernel to destroy the previous mappings by replacing
+         * them in mmap call later.
+         *
+         * Non-identical replacements are not allowed therefore.
+         */
+        assert(!vaddr || (entry->vaddr_base == vaddr && entry->size == size));
+
+        if (!vaddr && munmap(entry->vaddr_base, entry->size) != 0) {
             perror("unmap fails");
             exit(-1);
         }