From patchwork Fri Dec 3 23:34:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Shan X-Patchwork-Id: 12656065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 219B5C433FE for ; Fri, 3 Dec 2021 23:38:35 +0000 (UTC) Received: from localhost ([::1]:58504 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mtI8Q-0004hb-4c for qemu-devel@archiver.kernel.org; Fri, 03 Dec 2021 18:38:34 -0500 Received: from eggs.gnu.org ([209.51.188.92]:40910) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mtI4t-0006pC-NU for qemu-devel@nongnu.org; Fri, 03 Dec 2021 18:34:56 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:26178) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mtI4r-0001ql-KV for qemu-devel@nongnu.org; Fri, 03 Dec 2021 18:34:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1638574493; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VtNz+daFdRVqMzXQEdDO0e9qGyb3lRfv8GHRF45nWMQ=; b=PmcEMQjxdNL25GQOS3y3h643JqBkmItSHBjH0b3sTkcxOdYcpG6102ZKibKMgiihwcYidc KyjHoqVeO43hbOtHrYQMOS8NdE2XAicxlDv8cHsGb9oRxAyshjtq2CCmcMPZCBPGLgJl+A zMAU2MoZexV4UVvrFcd8zyt4H2+IVFI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-593-3nRp0qFaPeSLBo2uqC7Afw-1; Fri, 03 Dec 2021 18:34:50 -0500 X-MC-Unique: 3nRp0qFaPeSLBo2uqC7Afw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D3A528042E1; Fri, 3 Dec 2021 23:34:48 +0000 (UTC) Received: from gshan.redhat.com (vpn2-54-43.bne.redhat.com [10.64.54.43]) by smtp.corp.redhat.com (Postfix) with ESMTP id C199B60843; Fri, 3 Dec 2021 23:34:33 +0000 (UTC) From: Gavin Shan To: qemu-arm@nongnu.org Subject: [PATCH v3 2/2] hw/arm/virt: Support for virtio-mem-pci Date: Sat, 4 Dec 2021 07:34:04 +0800 Message-Id: <20211203233404.37313-3-gshan@redhat.com> In-Reply-To: <20211203233404.37313-1-gshan@redhat.com> References: <20211203233404.37313-1-gshan@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=gshan@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.129.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.717, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, drjones@redhat.com, david@redhat.com, richard.henderson@linaro.org, qemu-devel@nongnu.org, eric.auger@redhat.com, shan.gavin@gmail.com, Jonathan.Cameron@huawei.com, imammedo@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This supports virtio-mem-pci device on "virt" platform, by simply following the implementation on x86. * This implements the hotplug handlers to support virtio-mem-pci device hot-add, while the hot-remove isn't supported as we have on x86. * The block size is 512MB on ARM64 instead of 128MB on x86. * It has been passing the tests with various combinations like 64KB and 4KB page sizes on host and guest, different memory device backends like normal, transparent huge page and HugeTLB, plus migration. Co-developed-by: David Hildenbrand Co-developed-by: Jonathan Cameron Signed-off-by: Gavin Shan Reviewed-by: Jonathan Cameron Reviewed-by: David Hildenbrand --- hw/arm/Kconfig | 1 + hw/arm/virt.c | 68 +++++++++++++++++++++++++++++++++++++++++- hw/virtio/virtio-mem.c | 4 ++- 3 files changed, 71 insertions(+), 2 deletions(-) diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig index 2d37d29f02..15aff8efb8 100644 --- a/hw/arm/Kconfig +++ b/hw/arm/Kconfig @@ -27,6 +27,7 @@ config ARM_VIRT select DIMM select ACPI_HW_REDUCED select ACPI_APEI + select VIRTIO_MEM_SUPPORTED config CHEETAH bool diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 30da05dfe0..db1544760d 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -72,9 +72,11 @@ #include "hw/arm/smmuv3.h" #include "hw/acpi/acpi.h" #include "target/arm/internals.h" +#include "hw/mem/memory-device.h" #include "hw/mem/pc-dimm.h" #include "hw/mem/nvdimm.h" #include "hw/acpi/generic_event_device.h" +#include "hw/virtio/virtio-mem-pci.h" #include "hw/virtio/virtio-iommu.h" #include "hw/char/pl011.h" #include "qemu/guest-random.h" @@ -2483,6 +2485,63 @@ static void virt_memory_plug(HotplugHandler *hotplug_dev, dev, &error_abort); } +static void virt_virtio_md_pci_pre_plug(HotplugHandler *hotplug_dev, + DeviceState *dev, Error **errp) +{ + HotplugHandler *hotplug_dev2 = qdev_get_bus_hotplug_handler(dev); + Error *local_err = NULL; + + if (!hotplug_dev2 && dev->hotplugged) { + /* + * Without a bus hotplug handler, we cannot control the plug/unplug + * order. We should never reach this point when hotplugging on x86, + * however, better add a safety net. + */ + error_setg(errp, "hotplug of virtio based memory devices not supported" + " on this bus."); + return; + } + /* + * First, see if we can plug this memory device at all. If that + * succeeds, branch of to the actual hotplug handler. + */ + memory_device_pre_plug(MEMORY_DEVICE(dev), MACHINE(hotplug_dev), NULL, + &local_err); + if (!local_err && hotplug_dev2) { + hotplug_handler_pre_plug(hotplug_dev2, dev, &local_err); + } + error_propagate(errp, local_err); +} + +static void virt_virtio_md_pci_plug(HotplugHandler *hotplug_dev, + DeviceState *dev, Error **errp) +{ + HotplugHandler *hotplug_dev2 = qdev_get_bus_hotplug_handler(dev); + Error *local_err = NULL; + + /* + * Plug the memory device first and then branch off to the actual + * hotplug handler. If that one fails, we can easily undo the memory + * device bits. + */ + memory_device_plug(MEMORY_DEVICE(dev), MACHINE(hotplug_dev)); + if (hotplug_dev2) { + hotplug_handler_plug(hotplug_dev2, dev, &local_err); + if (local_err) { + memory_device_unplug(MEMORY_DEVICE(dev), MACHINE(hotplug_dev)); + } + } + error_propagate(errp, local_err); +} + +static void virt_virtio_md_pci_unplug_request(HotplugHandler *hotplug_dev, + DeviceState *dev, Error **errp) +{ + /* We don't support hot unplug of virtio based memory devices */ + error_setg(errp, "virtio based memory devices cannot be unplugged."); +} + + static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp) { @@ -2516,6 +2575,8 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, qdev_prop_set_uint32(dev, "len-reserved-regions", 1); qdev_prop_set_string(dev, "reserved-regions[0]", resv_prop_str); g_free(resv_prop_str); + } else if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) { + virt_virtio_md_pci_pre_plug(hotplug_dev, dev, errp); } } @@ -2541,6 +2602,8 @@ static void virt_machine_device_plug_cb(HotplugHandler *hotplug_dev, vms->iommu = VIRT_IOMMU_VIRTIO; vms->virtio_iommu_bdf = pci_get_bdf(pdev); create_virtio_iommu_dt_bindings(vms); + } else if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) { + virt_virtio_md_pci_plug(hotplug_dev, dev, errp); } } @@ -2591,6 +2654,8 @@ static void virt_machine_device_unplug_request_cb(HotplugHandler *hotplug_dev, { if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) { virt_dimm_unplug_request(hotplug_dev, dev, errp); + } else if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) { + virt_virtio_md_pci_unplug_request(hotplug_dev, dev, errp); } else { error_setg(errp, "device unplug request for unsupported device" " type: %s", object_get_typename(OBJECT(dev))); @@ -2614,7 +2679,8 @@ static HotplugHandler *virt_machine_get_hotplug_handler(MachineState *machine, MachineClass *mc = MACHINE_GET_CLASS(machine); if (device_is_dynamic_sysbus(mc, dev) || - (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM))) { + object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) || + object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) { return HOTPLUG_HANDLER(machine); } if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_IOMMU_PCI)) { diff --git a/hw/virtio/virtio-mem.c b/hw/virtio/virtio-mem.c index b20595a496..21e4d572ab 100644 --- a/hw/virtio/virtio-mem.c +++ b/hw/virtio/virtio-mem.c @@ -125,7 +125,7 @@ static uint64_t virtio_mem_default_block_size(RAMBlock *rb) * The memory block size corresponds mostly to the section size. * * This allows e.g., to add 20MB with a section size of 128MB on x86_64, and - * a section size of 1GB on arm64 (as long as the start address is properly + * a section size of 512MB on arm64 (as long as the start address is properly * aligned, similar to ordinary DIMMs). * * We can change this at any time and maybe even make it configurable if @@ -134,6 +134,8 @@ static uint64_t virtio_mem_default_block_size(RAMBlock *rb) */ #if defined(TARGET_X86_64) || defined(TARGET_I386) #define VIRTIO_MEM_USABLE_EXTENT (2 * (128 * MiB)) +#elif defined(TARGET_ARM) +#define VIRTIO_MEM_USABLE_EXTENT (2 * (512 * MiB)) #else #error VIRTIO_MEM_USABLE_EXTENT not defined #endif