Message ID | 20200603144914.41645-1-david@redhat.com (mailing list archive) |
---|---|
Headers | show |
Series | virtio-mem: Paravirtualized memory hot(un)plug | expand |
On 6/3/20 9:48 AM, David Hildenbrand wrote: > This is the very basic, initial version of virtio-mem. More info on > virtio-mem in general can be found in the Linux kernel driver v2 posting > [1] and in patch #10. The latest Linux driver v4 can be found at [2]. > > This series is based on [3]: > "[PATCH v1] pc: Support coldplugging of virtio-pmem-pci devices on all > buses" > Let's spell that in a form patchew can recognize: Based-on: <20200525084511.51379-1-david@redhat.com>
On Wed, Jun 03, 2020 at 04:48:54PM +0200, David Hildenbrand wrote: > This is the very basic, initial version of virtio-mem. More info on > virtio-mem in general can be found in the Linux kernel driver v2 posting > [1] and in patch #10. The latest Linux driver v4 can be found at [2]. > > This series is based on [3]: > "[PATCH v1] pc: Support coldplugging of virtio-pmem-pci devices on all > buses" > > The patches can be found at: > https://github.com/davidhildenbrand/qemu.git virtio-mem-v3 So given we tweaked the config space a bit, this needs a respin. > "The basic idea of virtio-mem is to provide a flexible, > cross-architecture memory hot(un)plug solution that avoids many limitations > imposed by existing technologies, architectures, and interfaces." > > There are a lot of addons in the works (esp. protection of unplugged > memory, better hugepage support (esp. when reading unplugged memory), > resizeable memory backends, support for more architectures, ...), this is > the very basic version to get the ball rolling. > > The first 8 patches make sure we don't have any sudden surprises e.g., if > somebody tries to pin all memory in RAM blocks, resulting in a higher > memory consumption than desired. The remaining patches add basic virtio-mem > along with support for x86-64. > > > Note: Since commit 7d2ef6dcc1cf ("hmp: Simplify qom-set"), the behavior of > qom-set changed and we can no longer pass proper sizes (e.g., 300M). The > description in patch #10 is outdated - but I hope that we'll bring back the > old behaviour, so I kept it for now :) > > [1] https://lkml.kernel.org/r/20200311171422.10484-1-david@redhat.com > [2] https://lkml.kernel.org/r/20200507140139.17083-1-david@redhat.com > [3] https://lkml.kernel.org/r/20200525084511.51379-1-david@redhat.com > > Cc: teawater <teawaterz@linux.alibaba.com> > Cc: Pankaj Gupta <pankaj.gupta.linux@gmail.com> > > v2 -> v3: > - Rebased on upstream/[3] > - "virtio-mem: Exclude unplugged memory during migration" > -- Added > - "virtio-mem: Paravirtualized memory hot(un)plug" > -- Simplify bitmap operations, find consecutive areas > -- Tweak error messages > -- Reshuffle some checks > -- Minor cleanups > - "accel/kvm: Convert to ram_block_discard_disable()" > - "target/i386: sev: Use ram_block_discard_disable()" > -- Keep asserts clean of functional things > > v1 -> v2: > - Rebased to object_property_*() changes > - "exec: Introduce ram_block_discard_(disable|require)()" > -- Change the function names and rephrase/add comments > - "virtio-balloon: Rip out qemu_balloon_inhibit()" > -- Add and use "migration_in_incoming_postcopy()" > - "migration/rdma: Use ram_block_discard_disable()" > -- Add a comment regarding pin_all vs. !pin_all > - "virtio-mem: Paravirtualized memory hot(un)plug" > -- Replace virtio_mem_discard_inhibited() by > migration_in_incoming_postcopy() > -- Drop some asserts > -- Drop virtio_mem_bad_request(), use virtio_error() directly, printing > more information > -- Replace "Note: Discarding should never fail ..." comments by > error_report() > -- Replace virtio_stw_p() by cpu_to_le16() > -- Drop migration_addr and migration_block_size > -- Minor cleanups > - "linux-headers: update to contain virtio-mem" > -- Updated to latest v4 in Linux > - General changes > -- Fixup the users of the renamed ram_block_discard_(disable|require) > -- Use "X: cannot disable RAM discard"-styled error messages > - Added > -- "virtio-mem: Migration sanity checks" > -- "virtio-mem: Add trace events" > > David Hildenbrand (20): > exec: Introduce ram_block_discard_(disable|require)() > vfio: Convert to ram_block_discard_disable() > accel/kvm: Convert to ram_block_discard_disable() > s390x/pv: Convert to ram_block_discard_disable() > virtio-balloon: Rip out qemu_balloon_inhibit() > target/i386: sev: Use ram_block_discard_disable() > migration/rdma: Use ram_block_discard_disable() > migration/colo: Use ram_block_discard_disable() > linux-headers: update to contain virtio-mem > virtio-mem: Paravirtualized memory hot(un)plug > virtio-pci: Proxy for virtio-mem > MAINTAINERS: Add myself as virtio-mem maintainer > hmp: Handle virtio-mem when printing memory device info > numa: Handle virtio-mem in NUMA stats > pc: Support for virtio-mem-pci > virtio-mem: Allow notifiers for size changes > virtio-pci: Send qapi events when the virtio-mem size changes > virtio-mem: Migration sanity checks > virtio-mem: Add trace events > virtio-mem: Exclude unplugged memory during migration > > MAINTAINERS | 8 + > accel/kvm/kvm-all.c | 4 +- > balloon.c | 17 - > exec.c | 52 ++ > hw/core/numa.c | 6 + > hw/i386/Kconfig | 1 + > hw/i386/pc.c | 49 +- > hw/s390x/s390-virtio-ccw.c | 22 +- > hw/vfio/ap.c | 10 +- > hw/vfio/ccw.c | 11 +- > hw/vfio/common.c | 53 +- > hw/vfio/pci.c | 6 +- > hw/virtio/Kconfig | 11 + > hw/virtio/Makefile.objs | 2 + > hw/virtio/trace-events | 10 + > hw/virtio/virtio-balloon.c | 8 +- > hw/virtio/virtio-mem-pci.c | 157 ++++ > hw/virtio/virtio-mem-pci.h | 34 + > hw/virtio/virtio-mem.c | 876 ++++++++++++++++++++ > include/exec/memory.h | 41 + > include/hw/pci/pci.h | 1 + > include/hw/vfio/vfio-common.h | 4 +- > include/hw/virtio/virtio-mem.h | 86 ++ > include/migration/colo.h | 2 +- > include/migration/misc.h | 2 + > include/standard-headers/linux/virtio_ids.h | 1 + > include/standard-headers/linux/virtio_mem.h | 211 +++++ > include/sysemu/balloon.h | 2 - > migration/migration.c | 15 +- > migration/postcopy-ram.c | 23 - > migration/rdma.c | 18 +- > migration/savevm.c | 11 +- > monitor/hmp-cmds.c | 16 + > monitor/monitor.c | 1 + > qapi/misc.json | 64 +- > target/i386/sev.c | 7 + > 36 files changed, 1721 insertions(+), 121 deletions(-) > create mode 100644 hw/virtio/virtio-mem-pci.c > create mode 100644 hw/virtio/virtio-mem-pci.h > create mode 100644 hw/virtio/virtio-mem.c > create mode 100644 include/hw/virtio/virtio-mem.h > create mode 100644 include/standard-headers/linux/virtio_mem.h > > -- > 2.25.4
On 09.06.20 15:11, Michael S. Tsirkin wrote: > On Wed, Jun 03, 2020 at 04:48:54PM +0200, David Hildenbrand wrote: >> This is the very basic, initial version of virtio-mem. More info on >> virtio-mem in general can be found in the Linux kernel driver v2 posting >> [1] and in patch #10. The latest Linux driver v4 can be found at [2]. >> >> This series is based on [3]: >> "[PATCH v1] pc: Support coldplugging of virtio-pmem-pci devices on all >> buses" >> >> The patches can be found at: >> https://github.com/davidhildenbrand/qemu.git virtio-mem-v3 > > So given we tweaked the config space a bit, this needs a respin. Yeah, the virtio-mem-v4 branch already contains a fixed-up version. Will send during the next days.
On Tue, Jun 09, 2020 at 03:26:08PM +0200, David Hildenbrand wrote: > On 09.06.20 15:11, Michael S. Tsirkin wrote: > > On Wed, Jun 03, 2020 at 04:48:54PM +0200, David Hildenbrand wrote: > >> This is the very basic, initial version of virtio-mem. More info on > >> virtio-mem in general can be found in the Linux kernel driver v2 posting > >> [1] and in patch #10. The latest Linux driver v4 can be found at [2]. > >> > >> This series is based on [3]: > >> "[PATCH v1] pc: Support coldplugging of virtio-pmem-pci devices on all > >> buses" > >> > >> The patches can be found at: > >> https://github.com/davidhildenbrand/qemu.git virtio-mem-v3 > > > > So given we tweaked the config space a bit, this needs a respin. > > Yeah, the virtio-mem-v4 branch already contains a fixed-up version. Will > send during the next days. BTW. People don't normally capitalize the letter after ":". So a better subject is virtio-mem: paravirtualized memory hot(un)plug and so on for all patches. > -- > Thanks, > > David / dhildenb
On Tue, Jun 09, 2020 at 11:59:04AM -0400, Michael S. Tsirkin wrote: > On Tue, Jun 09, 2020 at 03:26:08PM +0200, David Hildenbrand wrote: > > On 09.06.20 15:11, Michael S. Tsirkin wrote: > > > On Wed, Jun 03, 2020 at 04:48:54PM +0200, David Hildenbrand wrote: > > >> This is the very basic, initial version of virtio-mem. More info on > > >> virtio-mem in general can be found in the Linux kernel driver v2 posting > > >> [1] and in patch #10. The latest Linux driver v4 can be found at [2]. > > >> > > >> This series is based on [3]: > > >> "[PATCH v1] pc: Support coldplugging of virtio-pmem-pci devices on all > > >> buses" > > >> > > >> The patches can be found at: > > >> https://github.com/davidhildenbrand/qemu.git virtio-mem-v3 > > > > > > So given we tweaked the config space a bit, this needs a respin. > > > > Yeah, the virtio-mem-v4 branch already contains a fixed-up version. Will > > send during the next days. > > BTW. People don't normally capitalize the letter after ":". > So a better subject is > virtio-mem: paravirtualized memory hot(un)plug I'm not sure that's still the rule: [qemu/(49ee115552...)]$ git log --oneline v4.0.0.. | egrep ': [A-Z]' | wc -l 5261 [qemu/(49ee115552...)]$ git log --oneline v4.0.0.. | egrep ': [a-z]' | wc -l 2921
On 09.06.20 18:18, Eduardo Habkost wrote: > On Tue, Jun 09, 2020 at 11:59:04AM -0400, Michael S. Tsirkin wrote: >> On Tue, Jun 09, 2020 at 03:26:08PM +0200, David Hildenbrand wrote: >>> On 09.06.20 15:11, Michael S. Tsirkin wrote: >>>> On Wed, Jun 03, 2020 at 04:48:54PM +0200, David Hildenbrand wrote: >>>>> This is the very basic, initial version of virtio-mem. More info on >>>>> virtio-mem in general can be found in the Linux kernel driver v2 posting >>>>> [1] and in patch #10. The latest Linux driver v4 can be found at [2]. >>>>> >>>>> This series is based on [3]: >>>>> "[PATCH v1] pc: Support coldplugging of virtio-pmem-pci devices on all >>>>> buses" >>>>> >>>>> The patches can be found at: >>>>> https://github.com/davidhildenbrand/qemu.git virtio-mem-v3 >>>> >>>> So given we tweaked the config space a bit, this needs a respin. >>> >>> Yeah, the virtio-mem-v4 branch already contains a fixed-up version. Will >>> send during the next days. >> >> BTW. People don't normally capitalize the letter after ":". >> So a better subject is >> virtio-mem: paravirtualized memory hot(un)plug > > I'm not sure that's still the rule: > > [qemu/(49ee115552...)]$ git log --oneline v4.0.0.. | egrep ': [A-Z]' | wc -l > 5261 > [qemu/(49ee115552...)]$ git log --oneline v4.0.0.. | egrep ': [a-z]' | wc -l > 2921 > Yeah, I switched to this scheme some years ago (I even remember that some QEMU maintainer recommended it). I decided to just always capitalize. Not that it should really matter ... :)
On Tue, Jun 09, 2020 at 08:38:15PM +0200, David Hildenbrand wrote: > On 09.06.20 18:18, Eduardo Habkost wrote: > > On Tue, Jun 09, 2020 at 11:59:04AM -0400, Michael S. Tsirkin wrote: > >> On Tue, Jun 09, 2020 at 03:26:08PM +0200, David Hildenbrand wrote: > >>> On 09.06.20 15:11, Michael S. Tsirkin wrote: > >>>> On Wed, Jun 03, 2020 at 04:48:54PM +0200, David Hildenbrand wrote: > >>>>> This is the very basic, initial version of virtio-mem. More info on > >>>>> virtio-mem in general can be found in the Linux kernel driver v2 posting > >>>>> [1] and in patch #10. The latest Linux driver v4 can be found at [2]. > >>>>> > >>>>> This series is based on [3]: > >>>>> "[PATCH v1] pc: Support coldplugging of virtio-pmem-pci devices on all > >>>>> buses" > >>>>> > >>>>> The patches can be found at: > >>>>> https://github.com/davidhildenbrand/qemu.git virtio-mem-v3 > >>>> > >>>> So given we tweaked the config space a bit, this needs a respin. > >>> > >>> Yeah, the virtio-mem-v4 branch already contains a fixed-up version. Will > >>> send during the next days. > >> > >> BTW. People don't normally capitalize the letter after ":". > >> So a better subject is > >> virtio-mem: paravirtualized memory hot(un)plug > > > > I'm not sure that's still the rule: > > > > [qemu/(49ee115552...)]$ git log --oneline v4.0.0.. | egrep ': [A-Z]' | wc -l > > 5261 > > [qemu/(49ee115552...)]$ git log --oneline v4.0.0.. | egrep ': [a-z]' | wc -l > > 2921 > > > > Yeah, I switched to this scheme some years ago (I even remember that > some QEMU maintainer recommended it). I decided to just always > capitalize. Not that it should really matter ... :) Don't mind about qemu but you don't want to do that for Linux. > -- > Thanks, > > David / dhildenb
On Tue, Jun 09, 2020 at 02:42:59PM -0400, Michael S. Tsirkin wrote: > On Tue, Jun 09, 2020 at 08:38:15PM +0200, David Hildenbrand wrote: > > On 09.06.20 18:18, Eduardo Habkost wrote: > > > On Tue, Jun 09, 2020 at 11:59:04AM -0400, Michael S. Tsirkin wrote: > > >> On Tue, Jun 09, 2020 at 03:26:08PM +0200, David Hildenbrand wrote: > > >>> On 09.06.20 15:11, Michael S. Tsirkin wrote: > > >>>> On Wed, Jun 03, 2020 at 04:48:54PM +0200, David Hildenbrand wrote: > > >>>>> This is the very basic, initial version of virtio-mem. More info on > > >>>>> virtio-mem in general can be found in the Linux kernel driver v2 posting > > >>>>> [1] and in patch #10. The latest Linux driver v4 can be found at [2]. > > >>>>> > > >>>>> This series is based on [3]: > > >>>>> "[PATCH v1] pc: Support coldplugging of virtio-pmem-pci devices on all > > >>>>> buses" > > >>>>> > > >>>>> The patches can be found at: > > >>>>> https://github.com/davidhildenbrand/qemu.git virtio-mem-v3 > > >>>> > > >>>> So given we tweaked the config space a bit, this needs a respin. > > >>> > > >>> Yeah, the virtio-mem-v4 branch already contains a fixed-up version. Will > > >>> send during the next days. > > >> > > >> BTW. People don't normally capitalize the letter after ":". > > >> So a better subject is > > >> virtio-mem: paravirtualized memory hot(un)plug > > > > > > I'm not sure that's still the rule: > > > > > > [qemu/(49ee115552...)]$ git log --oneline v4.0.0.. | egrep ': [A-Z]' | wc -l > > > 5261 > > > [qemu/(49ee115552...)]$ git log --oneline v4.0.0.. | egrep ': [a-z]' | wc -l > > > 2921 > > > > > > > Yeah, I switched to this scheme some years ago (I even remember that > > some QEMU maintainer recommended it). I decided to just always > > capitalize. Not that it should really matter ... :) > > Don't mind about qemu but you don't want to do that for Linux. Heh, depends on who you ask. The tip tree maintainers (strongly) prefer capitalizing the first word after the colon[*], and that naturally percolates into a lot of other subsystems, e.g. I follow that pattern for KVM so that I don't have to remember to switch when submitting patches against a tip branch. +Patch subject +^^^^^^^^^^^^^ + +The tip tree preferred format for patch subject prefixes is +'subsys/component:', e.g. 'x86/apic:', 'x86/mm/fault:', 'sched/fair:', +'genirq/core:'. Please do not use file names or complete file paths as +prefix. 'git log path/to/file' should give you a reasonable hint in most +cases. + +The condensed patch description in the subject line should start with a +uppercase letter and should be written in imperative tone. [*] https://lkml.kernel.org/r/20181107171149.165693799@linutronix.de
> Am 09.06.2020 um 21:42 schrieb Sean Christopherson <sean.j.christopherson@intel.com>: > > On Tue, Jun 09, 2020 at 02:42:59PM -0400, Michael S. Tsirkin wrote: >>> On Tue, Jun 09, 2020 at 08:38:15PM +0200, David Hildenbrand wrote: >>> On 09.06.20 18:18, Eduardo Habkost wrote: >>>> On Tue, Jun 09, 2020 at 11:59:04AM -0400, Michael S. Tsirkin wrote: >>>>> On Tue, Jun 09, 2020 at 03:26:08PM +0200, David Hildenbrand wrote: >>>>>> On 09.06.20 15:11, Michael S. Tsirkin wrote: >>>>>>> On Wed, Jun 03, 2020 at 04:48:54PM +0200, David Hildenbrand wrote: >>>>>>>> This is the very basic, initial version of virtio-mem. More info on >>>>>>>> virtio-mem in general can be found in the Linux kernel driver v2 posting >>>>>>>> [1] and in patch #10. The latest Linux driver v4 can be found at [2]. >>>>>>>> >>>>>>>> This series is based on [3]: >>>>>>>> "[PATCH v1] pc: Support coldplugging of virtio-pmem-pci devices on all >>>>>>>> buses" >>>>>>>> >>>>>>>> The patches can be found at: >>>>>>>> https://github.com/davidhildenbrand/qemu.git virtio-mem-v3 >>>>>>> >>>>>>> So given we tweaked the config space a bit, this needs a respin. >>>>>> >>>>>> Yeah, the virtio-mem-v4 branch already contains a fixed-up version. Will >>>>>> send during the next days. >>>>> >>>>> BTW. People don't normally capitalize the letter after ":". >>>>> So a better subject is >>>>> virtio-mem: paravirtualized memory hot(un)plug >>>> >>>> I'm not sure that's still the rule: >>>> >>>> [qemu/(49ee115552...)]$ git log --oneline v4.0.0.. | egrep ': [A-Z]' | wc -l >>>> 5261 >>>> [qemu/(49ee115552...)]$ git log --oneline v4.0.0.. | egrep ': [a-z]' | wc -l >>>> 2921 >>>> >>> >>> Yeah, I switched to this scheme some years ago (I even remember that >>> some QEMU maintainer recommended it). I decided to just always >>> capitalize. Not that it should really matter ... :) >> >> Don't mind about qemu but you don't want to do that for Linux. > > Heh, depends on who you ask. The tip tree maintainers (strongly) prefer > capitalizing the first word after the colon[*], and that naturally > percolates into a lot of other subsystems, e.g. I follow that pattern for > KVM so that I don't have to remember to switch when submitting patches > against a tip branch. > Dito, most developers have other things to worry about. E.g., Andrew (-mm tree) automatically converts everything to lower-case when applying patches.
On 09.06.20 18:18, Eduardo Habkost wrote: > On Tue, Jun 09, 2020 at 11:59:04AM -0400, Michael S. Tsirkin wrote: >> On Tue, Jun 09, 2020 at 03:26:08PM +0200, David Hildenbrand wrote: >>> On 09.06.20 15:11, Michael S. Tsirkin wrote: >>>> On Wed, Jun 03, 2020 at 04:48:54PM +0200, David Hildenbrand wrote: >>>>> This is the very basic, initial version of virtio-mem. More info on >>>>> virtio-mem in general can be found in the Linux kernel driver v2 posting >>>>> [1] and in patch #10. The latest Linux driver v4 can be found at [2]. >>>>> >>>>> This series is based on [3]: >>>>> "[PATCH v1] pc: Support coldplugging of virtio-pmem-pci devices on all >>>>> buses" >>>>> >>>>> The patches can be found at: >>>>> https://github.com/davidhildenbrand/qemu.git virtio-mem-v3 >>>> >>>> So given we tweaked the config space a bit, this needs a respin. >>> >>> Yeah, the virtio-mem-v4 branch already contains a fixed-up version. Will >>> send during the next days. >> >> BTW. People don't normally capitalize the letter after ":". >> So a better subject is >> virtio-mem: paravirtualized memory hot(un)plug > > I'm not sure that's still the rule: > > [qemu/(49ee115552...)]$ git log --oneline v4.0.0.. | egrep ': [A-Z]' | wc -l > 5261 > [qemu/(49ee115552...)]$ git log --oneline v4.0.0.. | egrep ': [a-z]' | wc -l > 2921 > The kernel is slightly different, but it does not look like there is a real rule nowadays t480s: ~/git/linux virtio-mem-v4 $ git log --oneline v5.6..v5.7 | egrep ': [a-z]' | wc -l 9530 t480s: ~/git/linux virtio-mem-v4 $ git log --oneline v5.6..v5.7 | egrep ': [A-Z]' | wc -l 7689