[RFC,0/2] KVM: s390: avoid having to enable vm.alloc_pgste

From: Martin Schwidefsky <schwidefsky@de.ibm.com>

On Fri, 2 Jun 2017 15:20:33 +0200
Christian Borntraeger <borntraeger@de.ibm.com> wrote:

> On 06/02/2017 12:53 PM, Martin Schwidefsky wrote:
> > On Fri, 2 Jun 2017 12:19:19 +0200
> > Christian Borntraeger <borntraeger@de.ibm.com> wrote:
> >   
> >> On 06/02/2017 11:46 AM, Martin Schwidefsky wrote:  
> >>> On Fri, 2 Jun 2017 09:02:10 +0200
> >>> Heiko Carstens <heiko.carstens@de.ibm.com> wrote:
> >>>     
> >>>> On Thu, Jun 01, 2017 at 12:46:51PM +0200, Martin Schwidefsky wrote:    
> >>>>>> Unfortunately, converting all page tables to 4k pgste page tables is
> >>>>>> not possible without provoking various race conditions.      
> >>>>>
> >>>>> That is one approach we tried and was found to be buggy. The point is that
> >>>>> you are not allowed to reallocate a page table while a VMA exists that is
> >>>>> in the address range of that page table.
> >>>>>
> >>>>> Another approach we tried is to use an ELF flag on the qemu executable.
> >>>>> That does not work either because fs/exec.c allocates and populates the
> >>>>> new mm struct for the argument pages before fs/binfmt_elf.c comes into
> >>>>> play.      
> >>>>
> >>>> How about if you would fail the system call within arch_check_elf() if you
> >>>> detect that the binary requires pgstes (as indicated by elf flags) and then
> >>>> restart the system call?
> >>>>
> >>>> That is: arch_check_elf() e.g. would set a thread flag that future mm's
> >>>> should be allocated with pgstes. Then do_execve() would cleanup everything
> >>>> and return to entry.S. Upon return to userspace we detect this condition
> >>>> and simply restart the system call, similar to signals vs -ERESTARTSYS.
> >>>>
> >>>> That would make do_execve() cleanup everything and upon reentering it would
> >>>> allocate an mm with the pgste flag set.
> >>>>
> >>>> Maybe this is a bit over-simplified, but might work.    
> >>>
> >>> This is not over-simplified at all, that does work:    
> >>
> >>
> >> Nice. Next question is how to integrate that elf flag into the qemu
> >> build environment.  
> > 
> > So far I use a small C program to set the flag:
> > 
> > #include <elf.h>
> > #include <fcntl.h>
> > #include <stdio.h>
> > #include <stdlib.h>
> > 
> > #define ERREXIT(...)                    \
> > do {                                    \
> >         fprintf(stderr, __VA_ARGS__);   \
> >         exit(-1);                       \
> > } while (0)
> > 
> > int main(int argc, char *argv[])
> > {
> >         Elf64_Ehdr ehdr;
> >         int fd, rc;
> > 
> >         if (argc != 2)
> >                 ERREXIT("Usage: %s <elf-file>\n", argv[0]);
> > 
> >         fd = open(argv[1], O_RDWR);
> >         if (fd < 0)
> >                 ERREXIT("Unable to open file %s\n", argv[1]);
> > 
> >         if (pread(fd, &ehdr, sizeof(ehdr), 0) != sizeof(ehdr) ||
> >             memcmp(&ehdr.e_ident, ELFMAG, SELFMAG) != 0 ||
> >                 ehdr.e_ident[EI_CLASS] != ELFCLASS64 ||
> >             ehdr.e_machine != EM_S390)
> >                 ERREXIT("Invalid ELF file %s\n", argv[1]);
> > 
> >         ehdr.e_flags |= 0x00000002;
> > 
> >         if (pwrite(fd, &ehdr, sizeof(ehdr), 0) != sizeof(ehdr))
> >                 ERREXIT("Write to of file %s failed\n", argv[1]);
> > 
> >         close(fd);
> >         return 0;
> > }
> >   
> 
> Thanks for that. I assume this is mostly for testing and we want to have 
> toolchain support for that. Otherwise things like  build-id (the sha variant) 
> might be wrong, no?

Andreas Krebbel suggested to use a ELF segment type to mark the qemu binary,
similar to GNU_STACK. For the kernel part of this solution see below, now
we just need toolchain support to generate a qemu executable with the
PT_S390_REQUEST_PGSTE segment. The -z linker option comes to mind, e.g.
"-z request-pgste"

--
From 90c15b34c07644395d0bd96a632826740c539561 Mon Sep 17 00:00:00 2001
From: Martin Schwidefsky <schwidefsky@de.ibm.com>
Date: Wed, 7 Jun 2017 14:10:24 +0200
Subject: [PATCH] s390/kvm: avoid global config of vm.alloc_pgste=1

The system control vm.alloc_pgste is used to control the size of the
page tables, either 2K or 4K. The idea is that a KVM host sets the
vm.alloc_pgste control to 1 which causes *all* new processes to run
with 4K page tables. For a non-kvm system the control should stay off
to save on memory used for page tables.

Trouble is that distributions choose to set the control globally to
be able to run KVM guests. This wastes memory on non-KVM systems.

Introduce the PT_S390_REQUEST_PGSTE ELF segment type to "mark" the
qemu executable with it. All executables with this (empty) segment in
its ELF phdr array will be started with 4K page tables. Any executable
without PT_S390_REQUEST_PGSTE will run with the default 2K page tables.

This removes the need to set vm.alloc_pgste=1 for a KVM host and
minimizes the waste of memory for page tables.

Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
---
 arch/s390/Kconfig                   |  1 +
 arch/s390/include/asm/elf.h         | 26 ++++++++++++++++++++++++++
 arch/s390/include/asm/mmu_context.h |  3 ++-
 arch/s390/include/asm/ptrace.h      |  2 ++
 arch/s390/include/asm/thread_info.h |  1 +
 arch/s390/kernel/entry.S            | 13 ++++++++++++-
 6 files changed, 44 insertions(+), 2 deletions(-)

[RFC,0/2] KVM: s390: avoid having to enable vm.alloc_pgste

Commit Message

Comments

Patch