From patchwork Thu Dec 21 17:43:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13502454 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3613BC35274 for ; Thu, 21 Dec 2023 17:43:44 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rGN56-0001zC-6Q; Thu, 21 Dec 2023 12:43:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGN51-0001yS-CQ for qemu-devel@nongnu.org; Thu, 21 Dec 2023 12:43:31 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGN4z-0004Ll-JB for qemu-devel@nongnu.org; Thu, 21 Dec 2023 12:43:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703180608; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=wq93ywKzzt9qHkacvkFoeQJ+KNnlOUGXehbS0CEIyLo=; b=EchdVcx0BYPGtXonY7D2W6P1cAnQ4IR6mPRHh48klGNdwL22C5ahBAA407MzWhOGcSK7Yj vkanJDW4p+UTuQdsKykmPP368jCRoXHSNLBz9qSHJvJa3RpDiyUqUc8GnBaWegDuQ0sYAd AKoRIKl+9Bcq/gOjnYz85Cgrix4K7Sk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-446-iREC8SG9PCyVZNETmfWKyg-1; Thu, 21 Dec 2023 12:43:26 -0500 X-MC-Unique: iREC8SG9PCyVZNETmfWKyg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4A56F85A58A; Thu, 21 Dec 2023 17:43:26 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.3]) by smtp.corp.redhat.com (Postfix) with ESMTP id 89C8C492BC6; Thu, 21 Dec 2023 17:43:24 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Dragos Tatulea , Zhu Lingshan , Parav Pandit , Stefano Garzarella , "Michael S. Tsirkin" , Jason Wang , si-wei.liu@oracle.com, Laurent Vivier , Lei Yang Subject: [PATCH v4 00/13] Consolidate common vdpa members in VhostVDPAShared Date: Thu, 21 Dec 2023 18:43:09 +0100 Message-Id: <20231221174322.3130442-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.061, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Current memory operations like pinning may take a lot of time at the destination. Currently they are done after the source of the migration is stopped, and before the workload is resumed at the destination. This is a period where neigher traffic can flow, nor the VM workload can continue (downtime). We can do better as we know the memory layout of the guest RAM at the destination from the moment the migration starts. Moving that operation allows QEMU to communicate the kernel the maps while the workload is still running in the source, so Linux can start mapping them. Ideally, all IOMMU is configured, but if the vDPA parent driver uses on-chip IOMMU and .set_map we're still saving all the pinning time. This is a first required step to consolidate all the members in a common struct. This is needed because the destination does not know what vhost_vdpa struct will have the registered listener member, so it is easier to place them in a shared struct rather to keep them in vhost_vdpa struct. v4: * Actually fix the SIGSEGV, unregistering shared->listener in devices with vq_index > 1 while shared has been freed in net/vhost-vdpa.c freeing the device with vq_index == 0. Keeping the free in 0 as fd is closed at index 0. v3: * Cherry-pick Si-Wei's fixes. * Only memory_listener_unregister at vhost_vdpa_cleanup in the last dev. SIGSEGV detected by both Lei Yang [1] and Si-Wei [2]. v2: * Avoid repeated setting shared->shadow_data by squashing Si-Wei's patch [2]. v1 from RFC: * Fix vhost_vdpa_net_cvq_start checking for always_svq instead of shadow_data. This could cause CVQ not being shadowed if vhost_vdpa_net_cvq_start was called in the middle of a migration. [1] https://patchwork.kernel.org/project/qemu-devel/cover/20231124171430.2964464-1-eperezma@redhat.com/ [2] https://patchwork.kernel.org/project/qemu-devel/patch/1701970793-6865-10-git-send-email-si-wei.liu@oracle.com/ Eugenio Pérez (13): vdpa: add VhostVDPAShared vdpa: move iova tree to the shared struct vdpa: move iova_range to vhost_vdpa_shared vdpa: move shadow_data to vhost_vdpa_shared vdpa: use vdpa shared for tracing vdpa: move file descriptor to vhost_vdpa_shared vdpa: move iotlb_batch_begin_sent to vhost_vdpa_shared vdpa: move backend_cap to vhost_vdpa_shared vdpa: remove msg type of vhost_vdpa vdpa: move iommu_list to vhost_vdpa_shared vdpa: use VhostVDPAShared in vdpa_dma_map and unmap vdpa: use dev_shared in vdpa_iommu vdpa: move memory listener to vhost_vdpa_shared include/hw/virtio/vhost-vdpa.h | 36 +++++--- hw/virtio/vdpa-dev.c | 7 +- hw/virtio/vhost-vdpa.c | 160 +++++++++++++++++---------------- net/vhost-vdpa.c | 116 ++++++++++++------------ hw/virtio/trace-events | 14 +-- 5 files changed, 173 insertions(+), 160 deletions(-)