From patchwork Wed Feb 19 08:22:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EB9DCC3DA4A for ; Wed, 19 Feb 2025 08:27:11 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfPU-0005Ra-0a; Wed, 19 Feb 2025 03:26:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPR-0005R0-R9 for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:22 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPP-0004Za-Ic for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953580; x=1771489580; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BBos9TMEnyoSZ/PC59m1etHG0x5EWREqcsDCxiYnHsk=; b=jkayOLn/ILNYCQ+cIG9oLtKnSucCFnmXPBMbRjpLUodt1IIcypdgZxUf /tVdZtzh2cXOgQBagj7+XiSm3TkEKy9/gcqEH0SsUHJG1l5eHqvdAjshv ffn3a9LFetGTSnR7LHhx4NxCGBpq0iofn8xpByMzyUy8cIBZu0l+SNVaG xSyPSovKYGWu8I2JuxzlWecj1LMifUg7nG93pKACtec9H1fH9azmvyDVN hCaID0BulwlWJNco/Yy3Mh6S0eo8NtXt+uqA2C5L5bsMejS0WeudSyLuw +9IOiS5mOSokyMMuNyBiAGj2/Tpw66o1PdmXDApvKuwJgq7W45vnRlzYL A==; X-CSE-ConnectionGUID: QAMNEVJTQD6xP3WyVUofxw== X-CSE-MsgGUID: q/mo3sKyQIuyUMN7Xal9kw== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544042" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544042" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:09 -0800 X-CSE-ConnectionGUID: uGqC7U+UTVq/HngNGwYrhw== X-CSE-MsgGUID: rTyTWlLASpCovG4yNCQEtQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119850808" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:05 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan Subject: [PATCH rfcv2 01/20] backends/iommufd: Add helpers for invalidating user-managed HWPT Date: Wed, 19 Feb 2025 16:22:09 +0800 Message-Id: <20250219082228.3303163-2-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Nicolin Chen Signed-off-by: Zhenzhong Duan --- include/system/iommufd.h | 3 +++ backends/iommufd.c | 30 ++++++++++++++++++++++++++++++ backends/trace-events | 1 + 3 files changed, 34 insertions(+) diff --git a/include/system/iommufd.h b/include/system/iommufd.h index cbab75bfbf..5d02e9d148 100644 --- a/include/system/iommufd.h +++ b/include/system/iommufd.h @@ -61,6 +61,9 @@ bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id, uint64_t iova, ram_addr_t size, uint64_t page_size, uint64_t *data, Error **errp); +int iommufd_backend_invalidate_cache(IOMMUFDBackend *be, uint32_t hwpt_id, + uint32_t data_type, uint32_t entry_len, + uint32_t *entry_num, void *data_ptr); #define TYPE_HOST_IOMMU_DEVICE_IOMMUFD TYPE_HOST_IOMMU_DEVICE "-iommufd" #endif diff --git a/backends/iommufd.c b/backends/iommufd.c index d57da44755..fc32aad5cb 100644 --- a/backends/iommufd.c +++ b/backends/iommufd.c @@ -311,6 +311,36 @@ bool iommufd_backend_get_device_info(IOMMUFDBackend *be, uint32_t devid, return true; } +int iommufd_backend_invalidate_cache(IOMMUFDBackend *be, uint32_t hwpt_id, + uint32_t data_type, uint32_t entry_len, + uint32_t *entry_num, void *data_ptr) +{ + int ret, fd = be->fd; + struct iommu_hwpt_invalidate cache = { + .size = sizeof(cache), + .hwpt_id = hwpt_id, + .data_type = data_type, + .entry_len = entry_len, + .entry_num = *entry_num, + .data_uptr = (uintptr_t)data_ptr, + }; + + ret = ioctl(fd, IOMMU_HWPT_INVALIDATE, &cache); + + trace_iommufd_backend_invalidate_cache(fd, hwpt_id, data_type, entry_len, + *entry_num, cache.entry_num, + (uintptr_t)data_ptr, ret); + if (ret) { + *entry_num = cache.entry_num; + error_report("IOMMU_HWPT_INVALIDATE failed: %s", strerror(errno)); + ret = -errno; + } else { + g_assert(*entry_num == cache.entry_num); + } + + return ret; +} + static int hiod_iommufd_get_cap(HostIOMMUDevice *hiod, int cap, Error **errp) { HostIOMMUDeviceCaps *caps = &hiod->caps; diff --git a/backends/trace-events b/backends/trace-events index 40811a3162..5a23db6c8a 100644 --- a/backends/trace-events +++ b/backends/trace-events @@ -18,3 +18,4 @@ iommufd_backend_alloc_hwpt(int iommufd, uint32_t dev_id, uint32_t pt_id, uint32_ iommufd_backend_free_id(int iommufd, uint32_t id, int ret) " iommufd=%d id=%d (%d)" iommufd_backend_set_dirty(int iommufd, uint32_t hwpt_id, bool start, int ret) " iommufd=%d hwpt=%u enable=%d (%d)" iommufd_backend_get_dirty_bitmap(int iommufd, uint32_t hwpt_id, uint64_t iova, uint64_t size, uint64_t page_size, int ret) " iommufd=%d hwpt=%u iova=0x%"PRIx64" size=0x%"PRIx64" page_size=0x%"PRIx64" (%d)" +iommufd_backend_invalidate_cache(int iommufd, uint32_t hwpt_id, uint32_t data_type, uint32_t entry_len, uint32_t entry_num, uint32_t done_num, uint64_t data_ptr, int ret) " iommufd=%d hwpt_id=%u data_type=%u entry_len=%u entry_num=%u done_num=%u data_ptr=0x%"PRIx64" (%d)" From patchwork Wed Feb 19 08:22:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EDD3BC3ABA0 for ; Wed, 19 Feb 2025 08:27:11 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfPc-0005Sm-A1; Wed, 19 Feb 2025 03:26:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPY-0005SK-KZ for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:28 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPS-0004Za-9F for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953582; x=1771489582; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=XVdv5gWUOU2aVOqYrjh81efF2mZVpI8Jj4EkhPdEab8=; b=DaIcN+ZjxsaL0MEyS2RwGQW8L8jWsYnlS4FIuizK5a6v8+YSvkDFIYRO Lt/kCeJtj4Pim/LNwtSH/YaS5XihFj+HHlVdDDXny3TqydGn/jBhv2ya1 4uYG0AdAPG5BjFkHCIbNM0oa2xDdyuBeGELGYXdLWnSpMw/JYoASKPMcl PvWUL4WFl3D+jRjWy7GPiUlk2a2LDrYjC/LAlaSt/iyDcO6IPNuQW3U9o AEioAKyUicF4jtRxpCJJhpVyR+Ijej6JAzHn0vB+rsETZ3q6Fv7U0tNWk 3qwU7FjDHW1L2sx/nZAku9Hxq5UDAi1HzL2w3zdHQi2892viX0jsFCmQq w==; X-CSE-ConnectionGUID: KzBuC2lBTbmCz/KWWDlfjw== X-CSE-MsgGUID: Wq1qZlXOSDqGLAGjtjsw9Q== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544064" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544064" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:14 -0800 X-CSE-ConnectionGUID: pipGzqtCR76Z9rAxOAnUvw== X-CSE-MsgGUID: L0b805NbSMmHlrDb1/kuIg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119850861" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:10 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan Subject: [PATCH rfcv2 02/20] vfio/iommufd: Add properties and handlers to TYPE_HOST_IOMMU_DEVICE_IOMMUFD Date: Wed, 19 Feb 2025 16:22:10 +0800 Message-Id: <20250219082228.3303163-3-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org New added properties include IOMMUFD handle, devid and hwpt_id. IOMMUFD handle and devid are used to allocate/free ioas and hwpt. hwpt_id is used to re-attach IOMMUFD backed device to its default VFIO sub-system created hwpt, i.e., when vIOMMU is disabled by guest. These properties are initialized in .realize_late() handler. New added handlers include [at|de]tach_hwpt. They are used to attach/detach hwpt. VFIO and VDPA have different way to attach and detach, so implementation will be in sub-class instead of HostIOMMUDeviceIOMMUFD. Add two wrappers host_iommu_device_iommufd_[at|de]tach_hwpt to wrap the two handlers. This is a prerequisite patch for following ones. Signed-off-by: Zhenzhong Duan --- include/system/iommufd.h | 50 ++++++++++++++++++++++++++++++++++++++++ backends/iommufd.c | 22 ++++++++++++++++++ 2 files changed, 72 insertions(+) diff --git a/include/system/iommufd.h b/include/system/iommufd.h index 5d02e9d148..a871601df5 100644 --- a/include/system/iommufd.h +++ b/include/system/iommufd.h @@ -66,4 +66,54 @@ int iommufd_backend_invalidate_cache(IOMMUFDBackend *be, uint32_t hwpt_id, uint32_t *entry_num, void *data_ptr); #define TYPE_HOST_IOMMU_DEVICE_IOMMUFD TYPE_HOST_IOMMU_DEVICE "-iommufd" +OBJECT_DECLARE_TYPE(HostIOMMUDeviceIOMMUFD, HostIOMMUDeviceIOMMUFDClass, + HOST_IOMMU_DEVICE_IOMMUFD) + +/* Abstract of host IOMMU device with iommufd backend */ +struct HostIOMMUDeviceIOMMUFD { + HostIOMMUDevice parent_obj; + + IOMMUFDBackend *iommufd; + uint32_t devid; + uint32_t hwpt_id; +}; + +struct HostIOMMUDeviceIOMMUFDClass { + HostIOMMUDeviceClass parent_class; + + /** + * @attach_hwpt: attach host IOMMU device to IOMMUFD hardware page table. + * VFIO and VDPA device can have different implementation. + * + * Mandatory callback. + * + * @idev: host IOMMU device backed by IOMMUFD backend. + * + * @hwpt_id: ID of IOMMUFD hardware page table. + * + * @errp: pass an Error out when attachment fails. + * + * Returns: true on success, false on failure. + */ + bool (*attach_hwpt)(HostIOMMUDeviceIOMMUFD *idev, uint32_t hwpt_id, + Error **errp); + /** + * @detach_hwpt: detach host IOMMU device from IOMMUFD hardware page table. + * VFIO and VDPA device can have different implementation. + * + * Mandatory callback. + * + * @idev: host IOMMU device backed by IOMMUFD backend. + * + * @errp: pass an Error out when attachment fails. + * + * Returns: true on success, false on failure. + */ + bool (*detach_hwpt)(HostIOMMUDeviceIOMMUFD *idev, Error **errp); +}; + +bool host_iommu_device_iommufd_attach_hwpt(HostIOMMUDeviceIOMMUFD *idev, + uint32_t hwpt_id, Error **errp); +bool host_iommu_device_iommufd_detach_hwpt(HostIOMMUDeviceIOMMUFD *idev, + Error **errp); #endif diff --git a/backends/iommufd.c b/backends/iommufd.c index fc32aad5cb..574f330c27 100644 --- a/backends/iommufd.c +++ b/backends/iommufd.c @@ -341,6 +341,26 @@ int iommufd_backend_invalidate_cache(IOMMUFDBackend *be, uint32_t hwpt_id, return ret; } +bool host_iommu_device_iommufd_attach_hwpt(HostIOMMUDeviceIOMMUFD *idev, + uint32_t hwpt_id, Error **errp) +{ + HostIOMMUDeviceIOMMUFDClass *idevc = + HOST_IOMMU_DEVICE_IOMMUFD_GET_CLASS(idev); + + g_assert(idevc->attach_hwpt); + return idevc->attach_hwpt(idev, hwpt_id, errp); +} + +bool host_iommu_device_iommufd_detach_hwpt(HostIOMMUDeviceIOMMUFD *idev, + Error **errp) +{ + HostIOMMUDeviceIOMMUFDClass *idevc = + HOST_IOMMU_DEVICE_IOMMUFD_GET_CLASS(idev); + + g_assert(idevc->detach_hwpt); + return idevc->detach_hwpt(idev, errp); +} + static int hiod_iommufd_get_cap(HostIOMMUDevice *hiod, int cap, Error **errp) { HostIOMMUDeviceCaps *caps = &hiod->caps; @@ -379,6 +399,8 @@ static const TypeInfo types[] = { }, { .name = TYPE_HOST_IOMMU_DEVICE_IOMMUFD, .parent = TYPE_HOST_IOMMU_DEVICE, + .instance_size = sizeof(HostIOMMUDeviceIOMMUFD), + .class_size = sizeof(HostIOMMUDeviceIOMMUFDClass), .class_init = hiod_iommufd_class_init, .abstract = true, } From patchwork Wed Feb 19 08:22:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B0336C021AA for ; Wed, 19 Feb 2025 08:27:26 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfPa-0005SX-9b; Wed, 19 Feb 2025 03:26:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPW-0005S1-UG for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:27 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPS-0004ZM-R7 for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953583; x=1771489583; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=H45ctgYIUrvH9KPj2VkTio9PfTeWbUSQHoamcNdQlrs=; b=EW/u+OCvHEa390I4IRsI/8aiLLXLJQ78A/uV8be4pgtZW6D569vqpKne H+JZUTb4JCgQlH9jHWRTDLAZaqpb6VXeb6W6jecZJSmcnHr5+GM0Pfl/6 IkJV4XCtuNAY6iT7SIWhqJTTmcR0VaWcYXGfy/kPPxPP/x3AkgvHixShH /Qug9kPn3Gl6+fyJ5hMKf7KC+Wz9f5UQcIF+xYdTFDk/nhmJhTovDQEZ0 HjcDWlgse18iGsXsvvwaSRUbj7b8dXf7ldsP05SWaqyqhmAI1UdQ+MZhS ye/+ST1ZdON5ozwkn4jCUCGa5LbvKTd6c7UBtnrLJCFvP1RuJZlnoOasv Q==; X-CSE-ConnectionGUID: QVlOIY0KTgiAyX3SF6i9ag== X-CSE-MsgGUID: XBIl4ZI3R7W+wE92cTffOA== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544076" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544076" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:18 -0800 X-CSE-ConnectionGUID: ryudde52TiSc6eS/a4d2lw== X-CSE-MsgGUID: 8NtVIyRkQN+n9KvOZB0gCg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119850916" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:14 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan Subject: [PATCH rfcv2 03/20] HostIOMMUDevice: Introduce realize_late callback Date: Wed, 19 Feb 2025 16:22:11 +0800 Message-Id: <20250219082228.3303163-4-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Currently we have realize() callback which is called before attachment. But there are still some elements e.g., hwpt_id is not ready before attachment. So we need a realize_late() callback to further initialize them. Currently, this callback is only useful for iommufd backend. For legacy backend nothing needs to be initialized after attachment. Signed-off-by: Zhenzhong Duan --- include/system/host_iommu_device.h | 17 +++++++++++++++++ hw/vfio/common.c | 17 ++++++++++++++--- 2 files changed, 31 insertions(+), 3 deletions(-) diff --git a/include/system/host_iommu_device.h b/include/system/host_iommu_device.h index 809cced4ba..df782598f2 100644 --- a/include/system/host_iommu_device.h +++ b/include/system/host_iommu_device.h @@ -66,6 +66,23 @@ struct HostIOMMUDeviceClass { * Returns: true on success, false on failure. */ bool (*realize)(HostIOMMUDevice *hiod, void *opaque, Error **errp); + /** + * @realize_late: initialize host IOMMU device instance after attachment, + * some elements e.g., ioas are ready only after attachment. + * This callback initialize them. + * + * Optional callback. + * + * @hiod: pointer to a host IOMMU device instance. + * + * @opaque: pointer to agent device of this host IOMMU device, + * e.g., VFIO base device or VDPA device. + * + * @errp: pass an Error out when realize fails. + * + * Returns: true on success, false on failure. + */ + bool (*realize_late)(HostIOMMUDevice *hiod, void *opaque, Error **errp); /** * @get_cap: check if a host IOMMU device capability is supported. * diff --git a/hw/vfio/common.c b/hw/vfio/common.c index abbdc56b6d..e198b1e5a2 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1550,6 +1550,7 @@ bool vfio_attach_device(char *name, VFIODevice *vbasedev, const VFIOIOMMUClass *ops = VFIO_IOMMU_CLASS(object_class_by_name(TYPE_VFIO_IOMMU_LEGACY)); HostIOMMUDevice *hiod = NULL; + HostIOMMUDeviceClass *hiod_ops = NULL; if (vbasedev->iommufd) { ops = VFIO_IOMMU_CLASS(object_class_by_name(TYPE_VFIO_IOMMU_IOMMUFD)); @@ -1560,16 +1561,26 @@ bool vfio_attach_device(char *name, VFIODevice *vbasedev, if (!vbasedev->mdev) { hiod = HOST_IOMMU_DEVICE(object_new(ops->hiod_typename)); + hiod_ops = HOST_IOMMU_DEVICE_GET_CLASS(hiod); vbasedev->hiod = hiod; } if (!ops->attach_device(name, vbasedev, as, errp)) { - object_unref(hiod); - vbasedev->hiod = NULL; - return false; + goto err_attach; + } + + if (hiod_ops && hiod_ops->realize_late && + !hiod_ops->realize_late(hiod, vbasedev, errp)) { + ops->detach_device(vbasedev); + goto err_attach; } return true; + +err_attach: + object_unref(hiod); + vbasedev->hiod = NULL; + return false; } void vfio_detach_device(VFIODevice *vbasedev) From patchwork Wed Feb 19 08:22:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981688 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D58B5C021AA for ; Wed, 19 Feb 2025 08:27:11 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfPZ-0005SL-Cd; Wed, 19 Feb 2025 03:26:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPW-0005S2-UN for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:27 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPU-0004Zq-2F for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953584; x=1771489584; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Y3if1GlQNkp7W4HQVB5jdhNRUVirRmci7xxwosuRS/Y=; b=ZF/mcAERzRkU1WmNjOM8mCC+bCgrlkDp42UqTvfHgX/9k8m9HuxvePZ3 X22SyvXFaHIL4Vwfkex7CSM5zkbsu7/EWsbkvKRcmHL1Xk9X99jT70L3M T7jCJ53BxHvL6yRUCx1Tn+Bc9c/lMP1N16FeF3rcebxDylke/fFgDQecv VbnKTmYJOqJA1FWqCtl64DURBBSChZthP7SJ6hdqmPs/vJhi65Wa3N7Ps MEKFwmGZ6uZm4uWQj962A3R+rxrbrRRKbfRa/5WiyIhoGcGUB1fUUti0D 8Nkv2U7hLh7NGIxZtUYP3gjv28cmB0s1PDMfI45ySKese9bZpjGzn7sns A==; X-CSE-ConnectionGUID: POQMg3NwS5yeU00AkcL27g== X-CSE-MsgGUID: J5y/9BuiTkuKitiERF+DAg== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544086" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544086" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:23 -0800 X-CSE-ConnectionGUID: 4lcLPtHAQSiWyV/wGuxT8Q== X-CSE-MsgGUID: tB+6zdv7QI2UEqyiJCSk9w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119850973" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:18 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan Subject: [PATCH rfcv2 04/20] vfio/iommufd: Implement HostIOMMUDeviceClass::realize_late() handler Date: Wed, 19 Feb 2025 16:22:12 +0800 Message-Id: <20250219082228.3303163-5-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org There are three iommufd related elements iommufd handle, devid and hwpt_id. hwpt_id is ready only after VFIO device attachment. Device id and iommufd handle are ready before attachment, but they are all iommufd related stuff, initialize them together with hwpt_id. Signed-off-by: Zhenzhong Duan --- hw/vfio/iommufd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/hw/vfio/iommufd.c b/hw/vfio/iommufd.c index df61edffc0..53639bf88b 100644 --- a/hw/vfio/iommufd.c +++ b/hw/vfio/iommufd.c @@ -828,6 +828,19 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque, return true; } +static bool hiod_iommufd_vfio_realize_late(HostIOMMUDevice *hiod, void *opaque, + Error **errp) +{ + VFIODevice *vdev = opaque; + HostIOMMUDeviceIOMMUFD *idev = HOST_IOMMU_DEVICE_IOMMUFD(hiod); + + idev->iommufd = vdev->iommufd; + idev->devid = vdev->devid; + idev->hwpt_id = vdev->hwpt->hwpt_id; + + return true; +} + static GList * hiod_iommufd_vfio_get_iova_ranges(HostIOMMUDevice *hiod) { @@ -852,6 +865,7 @@ static void hiod_iommufd_vfio_class_init(ObjectClass *oc, void *data) HostIOMMUDeviceClass *hiodc = HOST_IOMMU_DEVICE_CLASS(oc); hiodc->realize = hiod_iommufd_vfio_realize; + hiodc->realize_late = hiod_iommufd_vfio_realize_late; hiodc->get_iova_ranges = hiod_iommufd_vfio_get_iova_ranges; hiodc->get_page_size_mask = hiod_iommufd_vfio_get_page_size_mask; }; From patchwork Wed Feb 19 08:22:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981698 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 28FA0C021AA for ; Wed, 19 Feb 2025 08:28:22 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfPf-0005Tp-E2; Wed, 19 Feb 2025 03:26:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPb-0005Sw-VM for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:32 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPY-0004aU-Jt for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953589; x=1771489589; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DwB7fnwqRjP31Cbn8rb1uxbObLpHT+AdggbslqZmPsU=; b=nBsJXM1L0IVCzgBXFISNKdtwpuV3zVdenXeKBg0xbq/IzKSE9mNuO16x c1t+VUR1gcueakh+pUu/jVJMJv4Mt83Ry5oxz+NBXgJ/Tw/MFnJU5mtMp hB8TBWZd03NpXpQIVKk+LSTMlJKVSmMjbqij/aquRtKZ26Nyiu8xiIMqj xn5rtDwrIPgdjCUUOeNbVGYlLdIaYjvziQaUe2ao7B9F6wJL3CFB/K6kM 4AJCSOwZ4/FHRp+wi+SbhSKzC0Rh3/gn5cw7dVqR+uEYOpQbsTW9bbBQu ejw65CERcUoB+LiLwPuQdTdY1tCDiUoqa1WQOpjPw44a8hkWIyy99TRvd g==; X-CSE-ConnectionGUID: qrZancl4TtKo7JIiI6KoOg== X-CSE-MsgGUID: luorRDEZRRmF4NbjF/mKiw== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544098" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544098" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:27 -0800 X-CSE-ConnectionGUID: /PW0ZOciRkmss4EAucLceA== X-CSE-MsgGUID: iXWbtxWUTN6qeovslgMyog== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851023" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:23 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan Subject: [PATCH rfcv2 05/20] vfio/iommufd: Implement [at|de]tach_hwpt handlers Date: Wed, 19 Feb 2025 16:22:13 +0800 Message-Id: <20250219082228.3303163-6-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Implement [at|de]tach_hwpt handlers in VFIO subsystem. vIOMMU utilizes them to attach to or detach from hwpt on host side. Signed-off-by: Yi Liu Signed-off-by: Zhenzhong Duan --- hw/vfio/iommufd.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/hw/vfio/iommufd.c b/hw/vfio/iommufd.c index 53639bf88b..175c4fe1f4 100644 --- a/hw/vfio/iommufd.c +++ b/hw/vfio/iommufd.c @@ -802,6 +802,24 @@ static void vfio_iommu_iommufd_class_init(ObjectClass *klass, void *data) vioc->query_dirty_bitmap = iommufd_query_dirty_bitmap; }; +static bool +host_iommu_device_iommufd_vfio_attach_hwpt(HostIOMMUDeviceIOMMUFD *idev, + uint32_t hwpt_id, Error **errp) +{ + VFIODevice *vbasedev = HOST_IOMMU_DEVICE(idev)->agent; + + return !iommufd_cdev_attach_ioas_hwpt(vbasedev, hwpt_id, errp); +} + +static bool +host_iommu_device_iommufd_vfio_detach_hwpt(HostIOMMUDeviceIOMMUFD *idev, + Error **errp) +{ + VFIODevice *vbasedev = HOST_IOMMU_DEVICE(idev)->agent; + + return iommufd_cdev_detach_ioas_hwpt(vbasedev, errp); +} + static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque, Error **errp) { @@ -863,11 +881,15 @@ hiod_iommufd_vfio_get_page_size_mask(HostIOMMUDevice *hiod) static void hiod_iommufd_vfio_class_init(ObjectClass *oc, void *data) { HostIOMMUDeviceClass *hiodc = HOST_IOMMU_DEVICE_CLASS(oc); + HostIOMMUDeviceIOMMUFDClass *idevc = HOST_IOMMU_DEVICE_IOMMUFD_CLASS(oc); hiodc->realize = hiod_iommufd_vfio_realize; hiodc->realize_late = hiod_iommufd_vfio_realize_late; hiodc->get_iova_ranges = hiod_iommufd_vfio_get_iova_ranges; hiodc->get_page_size_mask = hiod_iommufd_vfio_get_page_size_mask; + + idevc->attach_hwpt = host_iommu_device_iommufd_vfio_attach_hwpt; + idevc->detach_hwpt = host_iommu_device_iommufd_vfio_detach_hwpt; }; static const TypeInfo types[] = { From patchwork Wed Feb 19 08:22:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CA4C4C021AB for ; Wed, 19 Feb 2025 08:28:10 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfPi-0005UU-7V; Wed, 19 Feb 2025 03:26:38 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPe-0005TY-5K for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:34 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPc-0004Za-43 for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953592; x=1771489592; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yY0Gz0Q42ysq6vk2ZY5+LCBjvrX+hXo/OD6lHeKpxl0=; b=VZJ9YCfes6WCyzx2CIQ9AwzZhTULfIqEod3wzorwVfz05B5kVyDgt3BK /ioJZju35XK3MiJA/+ll7ZB14IL/Ebt9ftF+PV/WuP0JCBswPTsnJI/im 2pnrGBdnHFZuCE96JYZyv4y41caTbrtPuITwFFRu6xND4oUN+uCKRxB3s oDhu99/q8PVrxyhkrLkxQR56eIKkD1O9jUfv6E/S8vEjtfGbos7QEiCs5 BJNJVONl5l9kwJTFe7Bpd1qr3Ze2hW7qywxy2xYeVE0u6Ket4qjBvXQKH u7T3Ye1XHrIEgvJVHA1DHZwrvSLPZ/Bu4azA3tcV8738wx3AeTNRRU+Gl g==; X-CSE-ConnectionGUID: hOMrjjKoSPKnJqht6Q969w== X-CSE-MsgGUID: mLepFD5XS+26L6AZGEq/4Q== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544110" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544110" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:32 -0800 X-CSE-ConnectionGUID: CwNuqpgnRyms8A2SBZhgyg== X-CSE-MsgGUID: eKpwg1SZSeuGEnW4klmXKg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851049" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:27 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan Subject: [PATCH rfcv2 06/20] host_iommu_device: Define two new capabilities HOST_IOMMU_DEVICE_CAP_[NESTING|FS1GP] Date: Wed, 19 Feb 2025 16:22:14 +0800 Message-Id: <20250219082228.3303163-7-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Zhenzhong Duan --- include/system/host_iommu_device.h | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/include/system/host_iommu_device.h b/include/system/host_iommu_device.h index df782598f2..18f8b5e5cf 100644 --- a/include/system/host_iommu_device.h +++ b/include/system/host_iommu_device.h @@ -22,10 +22,16 @@ * * @hw_caps: host platform IOMMU capabilities (e.g. on IOMMUFD this represents * the @out_capabilities value returned from IOMMU_GET_HW_INFO ioctl) + * + * @nesting: nesting page table support. + * + * @fs1gp: first stage(a.k.a, Stage-1) 1GB huge page support. */ typedef struct HostIOMMUDeviceCaps { uint32_t type; uint64_t hw_caps; + bool nesting; + bool fs1gp; } HostIOMMUDeviceCaps; #define TYPE_HOST_IOMMU_DEVICE "host-iommu-device" @@ -122,6 +128,8 @@ struct HostIOMMUDeviceClass { */ #define HOST_IOMMU_DEVICE_CAP_IOMMU_TYPE 0 #define HOST_IOMMU_DEVICE_CAP_AW_BITS 1 +#define HOST_IOMMU_DEVICE_CAP_NESTING 2 +#define HOST_IOMMU_DEVICE_CAP_FS1GP 3 #define HOST_IOMMU_DEVICE_CAP_AW_BITS_MAX 64 #endif From patchwork Wed Feb 19 08:22:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E2544C021AA for ; Wed, 19 Feb 2025 08:30:31 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfPo-0005VS-5X; Wed, 19 Feb 2025 03:26:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPl-0005V0-93 for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:41 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPi-0004bS-Pq for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:40 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953599; x=1771489599; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5YWhRrVeEhH3CktmhBAgaRZUYzH0H93Jc1vO/aXiBcc=; b=I6gw4OHgofd0gjSp2uR9ZGbFWI70eX9IDs2aJUG0m5CXgu50L3b//Tdl nypcytG9z7WtQvMLLAHA7e3JtLnPhoGwDc6xUqXW+n23CpeB27Y6ppSnb 3JVSVqoGBKWUryKGr17Ujjw4GucLxV8JEqEx8hCk0uRWhOc/CWXxPTJH+ ZH6Xtm2gD290v22g4k6NaWXoUZu6a/eHRDrnuseMFwTWqUGO+0KGbypz6 t628VdVGa6LI7oKt/aE3JzPV7TwrdZ4pnO0Q19dOBoaaIpnD6Kife7R+a Qzo6AYb3W/lCcnYw0CUnM57g8QpHXv4i0IasBInupHAyjaQ/ZaQts5dQE Q==; X-CSE-ConnectionGUID: 5w7ZRfY6QjKPTZUihaIPBQ== X-CSE-MsgGUID: f8uXy9M3Rf2olP/acYULMQ== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544128" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544128" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:37 -0800 X-CSE-ConnectionGUID: wwxKKnOtRKqw2quFCqXQpQ== X-CSE-MsgGUID: ZLg8sdTPSjSYAzVPi7K6sA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851069" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:32 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Marcel Apfelbaum , Paolo Bonzini , Richard Henderson , Eduardo Habkost Subject: [PATCH rfcv2 07/20] iommufd: Implement query of HOST_IOMMU_DEVICE_CAP_[NESTING|FS1GP] Date: Wed, 19 Feb 2025 16:22:15 +0800 Message-Id: <20250219082228.3303163-8-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Implement query of HOST_IOMMU_DEVICE_CAP_[NESTING|FS1GP] for IOMMUFD backed host IOMMU device. Query on these two capabilities is not supported for legacy backend because there is no plan to support nesting with leacy backend backed host device. Signed-off-by: Zhenzhong Duan --- hw/i386/intel_iommu_internal.h | 1 + backends/iommufd.c | 4 ++++ hw/vfio/iommufd.c | 11 +++++++++++ 3 files changed, 16 insertions(+) diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h index e8b211e8b0..2cda744786 100644 --- a/hw/i386/intel_iommu_internal.h +++ b/hw/i386/intel_iommu_internal.h @@ -191,6 +191,7 @@ #define VTD_ECAP_PT (1ULL << 6) #define VTD_ECAP_SC (1ULL << 7) #define VTD_ECAP_MHMV (15ULL << 20) +#define VTD_ECAP_NEST (1ULL << 26) #define VTD_ECAP_SRS (1ULL << 31) #define VTD_ECAP_PASID (1ULL << 40) #define VTD_ECAP_SMTS (1ULL << 43) diff --git a/backends/iommufd.c b/backends/iommufd.c index 574f330c27..0a1a40cbba 100644 --- a/backends/iommufd.c +++ b/backends/iommufd.c @@ -370,6 +370,10 @@ static int hiod_iommufd_get_cap(HostIOMMUDevice *hiod, int cap, Error **errp) return caps->type; case HOST_IOMMU_DEVICE_CAP_AW_BITS: return vfio_device_get_aw_bits(hiod->agent); + case HOST_IOMMU_DEVICE_CAP_NESTING: + return caps->nesting; + case HOST_IOMMU_DEVICE_CAP_FS1GP: + return caps->fs1gp; default: error_setg(errp, "%s: unsupported capability %x", hiod->name, cap); return -EINVAL; diff --git a/hw/vfio/iommufd.c b/hw/vfio/iommufd.c index 175c4fe1f4..df6a12d200 100644 --- a/hw/vfio/iommufd.c +++ b/hw/vfio/iommufd.c @@ -26,6 +26,7 @@ #include "qemu/chardev_open.h" #include "pci.h" #include "exec/ram_addr.h" +#include "hw/i386/intel_iommu_internal.h" static int iommufd_cdev_map(const VFIOContainerBase *bcontainer, hwaddr iova, ram_addr_t size, void *vaddr, bool readonly) @@ -843,6 +844,16 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque, caps->type = type; caps->hw_caps = hw_caps; + switch (type) { + case IOMMU_HW_INFO_TYPE_INTEL_VTD: + caps->nesting = !!(data.vtd.ecap_reg & VTD_ECAP_NEST); + caps->fs1gp = !!(data.vtd.cap_reg & VTD_CAP_FS1GP); + break; + case IOMMU_HW_INFO_TYPE_ARM_SMMUV3: + case IOMMU_HW_INFO_TYPE_NONE: + break; + } + return true; } From patchwork Wed Feb 19 08:22:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981694 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D7BC0C021AA for ; Wed, 19 Feb 2025 08:27:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfPq-0005W6-SX; Wed, 19 Feb 2025 03:26:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPo-0005VT-5w for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:44 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPl-0004bS-U0 for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:43 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953602; x=1771489602; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fAmvZPigYvBGTxKieocJue8kt2n1eYxTq2NEydSanWg=; b=EDVvb0fN8DmWph4H72G4C0rW6F6nMI0VpuphJBDWMmuBpAJcDOJ1CGtf 9Q/7+NQTT+tG+DKZiAhXkiB7+8vSZY73nc8ck8M7HaORs4LyJarPlUyvi Ukf72rPIZdWd7xDO+LCV3kyBa8clAHibJNwK+qvYU8WzcDU3p3DvAK6wP QWvgJYN5wAWK6woDhplpObK5WKvR6DwhMAnajig196LkE1mMbYV2lcwbh TBUBzCqlQlWBg0oI6VBPuxY4uLl3X22tEFQ07QpiTZ7Hc8TQWDN/vsGpE FrYQRuj+nNpeyUBvomcUw7UXXwQoVLBL89OSDI9ZatL4Nko/auKPpKssI w==; X-CSE-ConnectionGUID: EH7fJzaqRMG/Odhe2PkEfA== X-CSE-MsgGUID: LaYrINppQJSqHisw4jfQsw== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544139" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544139" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:41 -0800 X-CSE-ConnectionGUID: mDGEis9XTrexhxxEbRd71w== X-CSE-MsgGUID: /aVnDjqzSRuFAEj7y7MIFg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851075" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:37 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan Subject: [PATCH rfcv2 08/20] iommufd: Implement query of HOST_IOMMU_DEVICE_CAP_ERRATA Date: Wed, 19 Feb 2025 16:22:16 +0800 Message-Id: <20250219082228.3303163-9-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Implement query of HOST_IOMMU_DEVICE_CAP_ERRATA for IOMMUFD backed host IOMMU device. Query on this capability is not supported for legacy backend because there is no plan to support nesting with leacy backend backed host device. Signed-off-by: Zhenzhong Duan --- include/system/host_iommu_device.h | 2 ++ backends/iommufd.c | 2 ++ hw/vfio/iommufd.c | 1 + 3 files changed, 5 insertions(+) diff --git a/include/system/host_iommu_device.h b/include/system/host_iommu_device.h index 18f8b5e5cf..250600fc1d 100644 --- a/include/system/host_iommu_device.h +++ b/include/system/host_iommu_device.h @@ -32,6 +32,7 @@ typedef struct HostIOMMUDeviceCaps { uint64_t hw_caps; bool nesting; bool fs1gp; + uint32_t errata; } HostIOMMUDeviceCaps; #define TYPE_HOST_IOMMU_DEVICE "host-iommu-device" @@ -130,6 +131,7 @@ struct HostIOMMUDeviceClass { #define HOST_IOMMU_DEVICE_CAP_AW_BITS 1 #define HOST_IOMMU_DEVICE_CAP_NESTING 2 #define HOST_IOMMU_DEVICE_CAP_FS1GP 3 +#define HOST_IOMMU_DEVICE_CAP_ERRATA 4 #define HOST_IOMMU_DEVICE_CAP_AW_BITS_MAX 64 #endif diff --git a/backends/iommufd.c b/backends/iommufd.c index 0a1a40cbba..3c23caef96 100644 --- a/backends/iommufd.c +++ b/backends/iommufd.c @@ -374,6 +374,8 @@ static int hiod_iommufd_get_cap(HostIOMMUDevice *hiod, int cap, Error **errp) return caps->nesting; case HOST_IOMMU_DEVICE_CAP_FS1GP: return caps->fs1gp; + case HOST_IOMMU_DEVICE_CAP_ERRATA: + return caps->errata; default: error_setg(errp, "%s: unsupported capability %x", hiod->name, cap); return -EINVAL; diff --git a/hw/vfio/iommufd.c b/hw/vfio/iommufd.c index df6a12d200..58bff030e1 100644 --- a/hw/vfio/iommufd.c +++ b/hw/vfio/iommufd.c @@ -848,6 +848,7 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque, case IOMMU_HW_INFO_TYPE_INTEL_VTD: caps->nesting = !!(data.vtd.ecap_reg & VTD_ECAP_NEST); caps->fs1gp = !!(data.vtd.cap_reg & VTD_CAP_FS1GP); + caps->errata = data.vtd.flags & IOMMU_HW_INFO_VTD_ERRATA_772415_SPR17; break; case IOMMU_HW_INFO_TYPE_ARM_SMMUV3: case IOMMU_HW_INFO_TYPE_NONE: From patchwork Wed Feb 19 08:22:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981708 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2B6F5C021AB for ; Wed, 19 Feb 2025 08:30:35 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfPx-0005Wj-UD; Wed, 19 Feb 2025 03:26:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPu-0005WP-6y for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:50 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPr-0004bS-7a for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:49 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953607; x=1771489607; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nYVvYeFiriU3MW0297bFJQJAXZzg8hZWGvEREGWkhUQ=; b=W5BMvw0MMO8eOnwtFkj0+M1GSy3K+6yAp+wGSLfpfB/771Q0IynL4sy7 FkR5Ene9Qe/z7vb/OQr9znKW8BdT6lhQw/4OKYB5kwfqJhdqVlekDFIaD y5HfbWLqZb07x27g+IpSOb85tK/Af+pr7mJyc5JJBuyRxx3lK8bt/zsOw zZT/OrsMMlkDz3p11Yp8Ke3wEGiVgG2KVjSYeQjsKytnPeqnLH5QckF5v kq+cdB/V6+KUtc4ApNWS9KoNY8ixgC19qmKPkQD9gSu5xXVh0U3LuP5Ao nKqFcYmnmC5V0dyhS6ZPba922TJT0sJ+lpd8Mt75w68ltdNFHq2dtGOVk Q==; X-CSE-ConnectionGUID: WVL3ErPhTN6owjbKbWELmw== X-CSE-MsgGUID: gO9cqvWBRDK83PdHPxDwsQ== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544145" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544145" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:47 -0800 X-CSE-ConnectionGUID: PO/Fj+PiRi+vdPCYuQWL9Q== X-CSE-MsgGUID: ALOClG/ISx2JRUft0ZxT8A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851079" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:41 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Marcel Apfelbaum , Paolo Bonzini , Richard Henderson , Eduardo Habkost Subject: [PATCH rfcv2 09/20] intel_iommu: Rename vtd_ce_get_rid2pasid_entry to vtd_ce_get_pasid_entry Date: Wed, 19 Feb 2025 16:22:17 +0800 Message-Id: <20250219082228.3303163-10-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In early days vtd_ce_get_rid2pasid_entry() is used to get pasid entry of rid2pasid, then extend to any pasid. So a new name vtd_ce_get_pasid_entry is better to match its functions. No functional change intended. Signed-off-by: Zhenzhong Duan Reviewed-by: Clément Mathieu--Drif --- hw/i386/intel_iommu.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 7fde0603bf..df5fb30bc8 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -944,7 +944,7 @@ static int vtd_get_pe_from_pasid_table(IntelIOMMUState *s, return 0; } -static int vtd_ce_get_rid2pasid_entry(IntelIOMMUState *s, +static int vtd_ce_get_pasid_entry(IntelIOMMUState *s, VTDContextEntry *ce, VTDPASIDEntry *pe, uint32_t pasid) @@ -1025,7 +1025,7 @@ static uint32_t vtd_get_iova_level(IntelIOMMUState *s, VTDPASIDEntry pe; if (s->root_scalable) { - vtd_ce_get_rid2pasid_entry(s, ce, &pe, pasid); + vtd_ce_get_pasid_entry(s, ce, &pe, pasid); if (s->flts) { return VTD_PE_GET_FL_LEVEL(&pe); } else { @@ -1048,7 +1048,7 @@ static uint32_t vtd_get_iova_agaw(IntelIOMMUState *s, VTDPASIDEntry pe; if (s->root_scalable) { - vtd_ce_get_rid2pasid_entry(s, ce, &pe, pasid); + vtd_ce_get_pasid_entry(s, ce, &pe, pasid); return 30 + ((pe.val[0] >> 2) & VTD_SM_PASID_ENTRY_AW) * 9; } @@ -1116,7 +1116,7 @@ static dma_addr_t vtd_get_iova_pgtbl_base(IntelIOMMUState *s, VTDPASIDEntry pe; if (s->root_scalable) { - vtd_ce_get_rid2pasid_entry(s, ce, &pe, pasid); + vtd_ce_get_pasid_entry(s, ce, &pe, pasid); if (s->flts) { return pe.val[2] & VTD_SM_PASID_ENTRY_FLPTPTR; } else { @@ -1522,7 +1522,7 @@ static int vtd_ce_rid2pasid_check(IntelIOMMUState *s, * has valid rid2pasid setting, which includes valid * rid2pasid field and corresponding pasid entry setting */ - return vtd_ce_get_rid2pasid_entry(s, ce, &pe, PCI_NO_PASID); + return vtd_ce_get_pasid_entry(s, ce, &pe, PCI_NO_PASID); } /* Map a device to its corresponding domain (context-entry) */ @@ -1611,7 +1611,7 @@ static uint16_t vtd_get_domain_id(IntelIOMMUState *s, VTDPASIDEntry pe; if (s->root_scalable) { - vtd_ce_get_rid2pasid_entry(s, ce, &pe, pasid); + vtd_ce_get_pasid_entry(s, ce, &pe, pasid); return VTD_SM_PASID_ENTRY_DID(pe.val[1]); } @@ -1687,7 +1687,7 @@ static bool vtd_dev_pt_enabled(IntelIOMMUState *s, VTDContextEntry *ce, int ret; if (s->root_scalable) { - ret = vtd_ce_get_rid2pasid_entry(s, ce, &pe, pasid); + ret = vtd_ce_get_pasid_entry(s, ce, &pe, pasid); if (ret) { /* * This error is guest triggerable. We should assumt PT From patchwork Wed Feb 19 08:22:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981690 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E800AC021AB for ; Wed, 19 Feb 2025 08:27:11 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfPz-0005XT-Nu; Wed, 19 Feb 2025 03:26:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPy-0005Wz-Ep for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:54 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfPw-0004bS-DG for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:26:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953613; x=1771489613; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZSRjZEL5xQevlQwiqc5TcPp+0Tek1XsaolTTkz/enMw=; b=JODBjT+AEcSYZRxtiVnPo9r5j32QBDdTeN0vWtzHdJJSh1DWPY4+gn7k Q1j5Y2XwmnYlgymt80cIkzinO6sCpv8QnOVN13bpWtYq2LCvTwkd59yjx be2hnKCiAFN/v9RbFrwGJr+LfsfnC+mp/lB3CTPPP5bp48FVKXE6TFQ91 rkIFJnm7xA+7wf3gaTleQgCkd3LN1rrMCIc23wblzkTEJFn1nigGzwLPJ 8eTl8xK8m94b+GgNJshdi0WrWbsyH0YCuvlMJqBnSTWptD9sR2QNxWI6z tun+68htykVMvSSaToQ7yA/ZkIp/xLSay4EEflThjbOYk+sW1fT4kEcrS A==; X-CSE-ConnectionGUID: v7ejwVxdRoOpJTxcUtJnOw== X-CSE-MsgGUID: n3CuJ7WrSZK51OoWCzCevQ== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544166" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544166" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:52 -0800 X-CSE-ConnectionGUID: n1WCmcaPSQq1TYUhTDn36Q== X-CSE-MsgGUID: wES+8Q3zTKWRmaBuTJ/rYQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851111" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:46 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Paolo Bonzini , Richard Henderson , Eduardo Habkost , Marcel Apfelbaum Subject: [PATCH rfcv2 10/20] intel_iommu: Optimize context entry cache utilization Date: Wed, 19 Feb 2025 16:22:18 +0800 Message-Id: <20250219082228.3303163-11-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org There are many call sites referencing context entry by calling vtd_as_to_context_entry() which will traverse the DMAR table. In most cases we can use cached context entry in vtd_as->context_cache_entry except it's stale. Currently only global and domain context invalidation stales it. So introduce a helper function vtd_as_to_context_entry() to fetch from cache before trying with vtd_dev_to_context_entry(). Signed-off-by: Zhenzhong Duan --- hw/i386/intel_iommu.c | 36 +++++++++++++++++++++++------------- 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index df5fb30bc8..7709f55be5 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -1597,6 +1597,22 @@ static int vtd_dev_to_context_entry(IntelIOMMUState *s, uint8_t bus_num, return 0; } +static int vtd_as_to_context_entry(VTDAddressSpace *vtd_as, VTDContextEntry *ce) +{ + IntelIOMMUState *s = vtd_as->iommu_state; + uint8_t bus_num = pci_bus_num(vtd_as->bus); + uint8_t devfn = vtd_as->devfn; + VTDContextCacheEntry *cc_entry = &vtd_as->context_cache_entry; + + /* Try to fetch context-entry from cache first */ + if (cc_entry->context_cache_gen == s->context_cache_gen) { + *ce = cc_entry->context_entry; + return 0; + } else { + return vtd_dev_to_context_entry(s, bus_num, devfn, ce); + } +} + static int vtd_sync_shadow_page_hook(const IOMMUTLBEvent *event, void *private) { @@ -1649,9 +1665,7 @@ static int vtd_address_space_sync(VTDAddressSpace *vtd_as) return 0; } - ret = vtd_dev_to_context_entry(vtd_as->iommu_state, - pci_bus_num(vtd_as->bus), - vtd_as->devfn, &ce); + ret = vtd_as_to_context_entry(vtd_as, &ce); if (ret) { if (ret == -VTD_FR_CONTEXT_ENTRY_P) { /* @@ -1710,8 +1724,7 @@ static bool vtd_as_pt_enabled(VTDAddressSpace *as) assert(as); s = as->iommu_state; - if (vtd_dev_to_context_entry(s, pci_bus_num(as->bus), as->devfn, - &ce)) { + if (vtd_as_to_context_entry(as, &ce)) { /* * Possibly failed to parse the context entry for some reason * (e.g., during init, or any guest configuration errors on @@ -2443,8 +2456,7 @@ static void vtd_iotlb_domain_invalidate(IntelIOMMUState *s, uint16_t domain_id) vtd_iommu_unlock(s); QLIST_FOREACH(vtd_as, &s->vtd_as_with_notifiers, next) { - if (!vtd_dev_to_context_entry(s, pci_bus_num(vtd_as->bus), - vtd_as->devfn, &ce) && + if (!vtd_as_to_context_entry(vtd_as, &ce) && domain_id == vtd_get_domain_id(s, &ce, vtd_as->pasid)) { vtd_address_space_sync(vtd_as); } @@ -2466,8 +2478,7 @@ static void vtd_iotlb_page_invalidate_notify(IntelIOMMUState *s, hwaddr size = (1 << am) * VTD_PAGE_SIZE; QLIST_FOREACH(vtd_as, &(s->vtd_as_with_notifiers), next) { - ret = vtd_dev_to_context_entry(s, pci_bus_num(vtd_as->bus), - vtd_as->devfn, &ce); + ret = vtd_as_to_context_entry(vtd_as, &ce); if (!ret && domain_id == vtd_get_domain_id(s, &ce, vtd_as->pasid)) { uint32_t rid2pasid = PCI_NO_PASID; @@ -2974,8 +2985,7 @@ static void vtd_piotlb_pasid_invalidate(IntelIOMMUState *s, vtd_iommu_unlock(s); QLIST_FOREACH(vtd_as, &s->vtd_as_with_notifiers, next) { - if (!vtd_dev_to_context_entry(s, pci_bus_num(vtd_as->bus), - vtd_as->devfn, &ce) && + if (!vtd_as_to_context_entry(vtd_as, &ce) && domain_id == vtd_get_domain_id(s, &ce, vtd_as->pasid)) { uint32_t rid2pasid = VTD_CE_GET_RID2PASID(&ce); @@ -4154,7 +4164,7 @@ static void vtd_report_ir_illegal_access(VTDAddressSpace *vtd_as, assert(vtd_as->pasid != PCI_NO_PASID); /* Try out best to fetch FPD, we can't do anything more */ - if (vtd_dev_to_context_entry(s, bus_n, vtd_as->devfn, &ce) == 0) { + if (vtd_as_to_context_entry(vtd_as, &ce) == 0) { is_fpd_set = ce.lo & VTD_CONTEXT_ENTRY_FPD; if (!is_fpd_set && s->root_scalable) { vtd_ce_get_pasid_fpd(s, &ce, &is_fpd_set, vtd_as->pasid); @@ -4491,7 +4501,7 @@ static void vtd_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) /* replay is protected by BQL, page walk will re-setup it safely */ iova_tree_remove(vtd_as->iova_tree, map); - if (vtd_dev_to_context_entry(s, bus_n, vtd_as->devfn, &ce) == 0) { + if (vtd_as_to_context_entry(vtd_as, &ce) == 0) { trace_vtd_replay_ce_valid(s->root_scalable ? "scalable mode" : "legacy mode", bus_n, PCI_SLOT(vtd_as->devfn), From patchwork Wed Feb 19 08:22:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981692 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 37693C021B0 for ; Wed, 19 Feb 2025 08:27:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfQB-0005YT-UL; Wed, 19 Feb 2025 03:27:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQ4-0005Xg-KC for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:00 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQ2-0004bS-0K for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953618; x=1771489618; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YkKG3U4URwqi/DdDGTpyXo93ZK+iuSB6xF85FkczKCU=; b=RVdSvWdHBtbKnpDz0ExFOrBGCi8srG4tmN6jRUtgbFUnJPm6rkr4mcWR x3oN4IOMw2ImHodOZxZFdHvjOPNH0BtCVCHEwpFIIzNhP35lEglQ+CA9u 1Usmm/+Nf17br/L/L0RmFIk2Sda0uCNK/pmp1EpgpL5r/EuN9BHGsdszu mGLs8l9esLceTkj15dGrE2ZscvIcHCMRgLJeMAUs/93sMJyj523GwdqhQ Mmqc1Rw4n01+LsGtzZCB2hJNi/dRO+nLhYvhJnaRgrl8uzuVaFhG/0nk6 Kkkjg+kQUeolxYk+IzFnWIyckJn5ZI9MS6tpbyRWZ0e2k0RBeeygw/FLM w==; X-CSE-ConnectionGUID: 8aSCvewAR76oUfh7CYoVnA== X-CSE-MsgGUID: Mf0PcxSjRtG0rbOKut6MnQ== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544177" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544177" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:57 -0800 X-CSE-ConnectionGUID: 6pA7Z+QOTtyNUZOvEhlqZg== X-CSE-MsgGUID: 1zxlyqKPSpi0XiQUr3ShiA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851135" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:51 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Paolo Bonzini , Richard Henderson , Eduardo Habkost , Marcel Apfelbaum Subject: [PATCH rfcv2 11/20] intel_iommu: Check for compatibility with IOMMUFD backed device when x-flts=on Date: Wed, 19 Feb 2025 16:22:19 +0800 Message-Id: <20250219082228.3303163-12-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org When vIOMMU is configured x-flts=on in scalable mode, stage-1 page table is passed to host to construct nested page table. We need to check compatibility of some critical IOMMU capabilities between vIOMMU and host IOMMU to ensure guest stage-1 page table could be used by host. For instance, vIOMMU supports stage-1 1GB huge page mapping, but host does not, then this IOMMUFD backed device should be failed. Declare an enum type host_iommu_device_iommu_hw_info_type aliased to iommu_hw_info_type which come from iommufd header file. This can avoid build failure on windows which doesn't support iommufd. Signed-off-by: Yi Liu Signed-off-by: Zhenzhong Duan --- include/system/host_iommu_device.h | 13 ++++++++++++ hw/i386/intel_iommu.c | 34 ++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+) diff --git a/include/system/host_iommu_device.h b/include/system/host_iommu_device.h index 250600fc1d..aa3885d7ee 100644 --- a/include/system/host_iommu_device.h +++ b/include/system/host_iommu_device.h @@ -133,5 +133,18 @@ struct HostIOMMUDeviceClass { #define HOST_IOMMU_DEVICE_CAP_FS1GP 3 #define HOST_IOMMU_DEVICE_CAP_ERRATA 4 +/** + * enum host_iommu_device_iommu_hw_info_type - IOMMU Hardware Info Types + * @HOST_IOMMU_DEVICE_IOMMU_HW_INFO_TYPE_NONE: Used by the drivers that do not + * report hardware info + * @HOST_IOMMU_DEVICE_IOMMU_HW_INFO_TYPE_INTEL_VTD: Intel VT-d iommu info type + * + * This is alias to enum iommu_hw_info_type but for general purpose. + */ +enum host_iommu_device_iommu_hw_info_type { + HOST_IOMMU_DEVICE_IOMMU_HW_INFO_TYPE_NONE, + HOST_IOMMU_DEVICE_IOMMU_HW_INFO_TYPE_INTEL_VTD, +}; + #define HOST_IOMMU_DEVICE_CAP_AW_BITS_MAX 64 #endif diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 7709f55be5..9de60e607d 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -39,6 +39,7 @@ #include "kvm/kvm_i386.h" #include "migration/vmstate.h" #include "trace.h" +#include "system/iommufd.h" /* context entry operations */ #define VTD_CE_GET_RID2PASID(ce) \ @@ -4346,6 +4347,39 @@ static bool vtd_check_hiod(IntelIOMMUState *s, HostIOMMUDevice *hiod, return true; } + /* Remaining checks are all stage-1 translation specific */ + if (!object_dynamic_cast(OBJECT(hiod), TYPE_HOST_IOMMU_DEVICE_IOMMUFD)) { + error_setg(errp, "Need IOMMUFD backend when x-flts=on"); + return false; + } + + ret = hiodc->get_cap(hiod, HOST_IOMMU_DEVICE_CAP_IOMMU_TYPE, errp); + if (ret < 0) { + return false; + } + if (ret != HOST_IOMMU_DEVICE_IOMMU_HW_INFO_TYPE_INTEL_VTD) { + error_setg(errp, "Incompatible host platform IOMMU type %d", ret); + return false; + } + + ret = hiodc->get_cap(hiod, HOST_IOMMU_DEVICE_CAP_NESTING, errp); + if (ret < 0) { + return false; + } + if (ret != 1) { + error_setg(errp, "Host IOMMU doesn't support nested translation"); + return false; + } + + ret = hiodc->get_cap(hiod, HOST_IOMMU_DEVICE_CAP_FS1GP, errp); + if (ret < 0) { + return false; + } + if (s->fs1gp && ret != 1) { + error_setg(errp, "Stage-1 1GB huge page is unsupported by host IOMMU"); + return false; + } + error_setg(errp, "host device is uncompatible with stage-1 translation"); return false; } From patchwork Wed Feb 19 08:22:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981701 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 591C4C021AA for ; Wed, 19 Feb 2025 08:29:21 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfQK-0005p5-8k; Wed, 19 Feb 2025 03:27:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQ9-0005YI-4D for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:07 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQ7-0004bS-4r for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953623; x=1771489623; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/zNkhlxkZ6uSup+2lkPM6XBkJH2cf5Y2mFnuhAqoLwo=; b=eE3hwck3rzNuafdqGDMlcMeUeUq6MZZvplDRDtT2l3frZ2m/PXqXnq/d HpI8f18OBXv7JfR1pLpUhfPaVVhkDNEfgPYJ8UmbdfZEvKgQe5XVMXqUj JZozWz1K2MUKzjGJJFmQYyJXOFGq1jXLU+zhFENaT359Kz94fE3SV9mcf BQ+BwvidBweLdboKeHW/U31lOTKLVgwb2a4W2ZdVzaqOhFIldw5QH7gpP IQKi1AYKdJmwfUiyrZH+mpT7DIWpctQAmSDXQHi28BDTy85a5FweiE/4w CUHwo3MjyueYyo2JIAZqjGLJnSK5yyDd/bSVQ87/xhK4cN+ckBFSfm0tY w==; X-CSE-ConnectionGUID: VjVOKjHbTM+paTXydiInPg== X-CSE-MsgGUID: 605OAERhRgWArYDo/gqQIQ== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544193" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544193" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:02 -0800 X-CSE-ConnectionGUID: Ks7ePFmqQsCp7tne/7dLgA== X-CSE-MsgGUID: 1cHVgyvfTvOyJ4Z5dxNU7w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851224" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:26:57 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Marcel Apfelbaum , Paolo Bonzini , Richard Henderson , Eduardo Habkost Subject: [PATCH rfcv2 12/20] intel_iommu: Introduce a new structure VTDHostIOMMUDevice Date: Wed, 19 Feb 2025 16:22:20 +0800 Message-Id: <20250219082228.3303163-13-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Introduce a new structure VTDHostIOMMUDevice which replaces HostIOMMUDevice to be stored in hash table. It includes a reference to HostIOMMUDevice and IntelIOMMUState, also includes BDF information which will be used in future patches. Signed-off-by: Zhenzhong Duan --- hw/i386/intel_iommu_internal.h | 7 +++++++ include/hw/i386/intel_iommu.h | 2 +- hw/i386/intel_iommu.c | 14 ++++++++++++-- 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h index 2cda744786..18bc22fc72 100644 --- a/hw/i386/intel_iommu_internal.h +++ b/hw/i386/intel_iommu_internal.h @@ -28,6 +28,7 @@ #ifndef HW_I386_INTEL_IOMMU_INTERNAL_H #define HW_I386_INTEL_IOMMU_INTERNAL_H #include "hw/i386/intel_iommu.h" +#include "system/host_iommu_device.h" /* * Intel IOMMU register specification @@ -608,4 +609,10 @@ typedef struct VTDRootEntry VTDRootEntry; /* Bits to decide the offset for each level */ #define VTD_LEVEL_BITS 9 +typedef struct VTDHostIOMMUDevice { + IntelIOMMUState *iommu_state; + PCIBus *bus; + uint8_t devfn; + HostIOMMUDevice *hiod; +} VTDHostIOMMUDevice; #endif diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h index e95477e855..50f9b27a45 100644 --- a/include/hw/i386/intel_iommu.h +++ b/include/hw/i386/intel_iommu.h @@ -295,7 +295,7 @@ struct IntelIOMMUState { /* list of registered notifiers */ QLIST_HEAD(, VTDAddressSpace) vtd_as_with_notifiers; - GHashTable *vtd_host_iommu_dev; /* HostIOMMUDevice */ + GHashTable *vtd_host_iommu_dev; /* VTDHostIOMMUDevice */ /* interrupt remapping */ bool intr_enabled; /* Whether guest enabled IR */ diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 9de60e607d..fafa199f52 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -281,7 +281,10 @@ static gboolean vtd_hiod_equal(gconstpointer v1, gconstpointer v2) static void vtd_hiod_destroy(gpointer v) { - object_unref(v); + VTDHostIOMMUDevice *vtd_hiod = v; + + object_unref(vtd_hiod->hiod); + g_free(vtd_hiod); } static gboolean vtd_hash_remove_by_domain(gpointer key, gpointer value, @@ -4388,6 +4391,7 @@ static bool vtd_dev_set_iommu_device(PCIBus *bus, void *opaque, int devfn, HostIOMMUDevice *hiod, Error **errp) { IntelIOMMUState *s = opaque; + VTDHostIOMMUDevice *vtd_hiod; struct vtd_as_key key = { .bus = bus, .devfn = devfn, @@ -4404,6 +4408,12 @@ static bool vtd_dev_set_iommu_device(PCIBus *bus, void *opaque, int devfn, return false; } + vtd_hiod = g_malloc0(sizeof(VTDHostIOMMUDevice)); + vtd_hiod->bus = bus; + vtd_hiod->devfn = (uint8_t)devfn; + vtd_hiod->iommu_state = s; + vtd_hiod->hiod = hiod; + if (!vtd_check_hiod(s, hiod, errp)) { vtd_iommu_unlock(s); return false; @@ -4414,7 +4424,7 @@ static bool vtd_dev_set_iommu_device(PCIBus *bus, void *opaque, int devfn, new_key->devfn = devfn; object_ref(hiod); - g_hash_table_insert(s->vtd_host_iommu_dev, new_key, hiod); + g_hash_table_insert(s->vtd_host_iommu_dev, new_key, vtd_hiod); vtd_iommu_unlock(s); From patchwork Wed Feb 19 08:22:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981696 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D55DAC021AB for ; Wed, 19 Feb 2025 08:28:14 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfQn-00069w-D1; Wed, 19 Feb 2025 03:27:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQL-0005rV-DJ for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:18 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQH-0004eO-Sn for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953634; x=1771489634; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2sVWgDs4RI7dmwZT9u6bFAhLp8LgsfK8oLChJpWHE7M=; b=bU1Q/InRK6Bo7NDYkB8vY/QaKhfs2BVCvm2DN61DrrUlJ119K7kB4p2I UM6rcy3xZ0kyu3ylYsKtmZytE3cde1X5E+G1MZetJZIiOwK14NV7o4Bnb xyOqUCgBQSo5rrayFFo2z0Gd1673DXadjdvS8crzYQfmZFU6vM0/0NqWl QuO2mLupHYS4w6FBqcpRSDGcG0fP8lLdEGyNfmHJ9XpNAVitcq+bdVWxR 1lJXCDOBwPbX08iPlVgC+u45aYk7QBxQnaOSlv3sqo6e6jYLEDGKX5mh6 X57HtGO2ZMywtlpgfaeFSyMjJxF+lKyIPDnTGWvvTHvWkqB0F3vRmwli+ Q==; X-CSE-ConnectionGUID: w5vPf67HQ+eSy7D8BF9GJQ== X-CSE-MsgGUID: ueEag07OQ+iJaISGGQHfKw== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544239" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544239" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:09 -0800 X-CSE-ConnectionGUID: ObXa9MF/T/2YyiY9TIqGKQ== X-CSE-MsgGUID: kpdNl1mFSP2VAD0WPT3TwQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851265" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:02 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Yi Sun , Paolo Bonzini , Richard Henderson , Eduardo Habkost , Marcel Apfelbaum Subject: [PATCH rfcv2 13/20] intel_iommu: Add PASID cache management infrastructure Date: Wed, 19 Feb 2025 16:22:21 +0800 Message-Id: <20250219082228.3303163-14-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This adds an new entry VTDPASIDCacheEntry in VTDAddressSpace to cache the pasid entry and track PASID usage and future PASID tagged DMA address translation support in vIOMMU. VTDAddressSpace of PCI_NO_PASID is allocated when device is plugged and never freed. For other pasid, VTDAddressSpace instance is created/destroyed per the guest pasid entry set up/destroy for passthrough devices. While for emulated devices, VTDAddressSpace instance is created in the PASID tagged DMA translation and be destroyed per guest PASID cache invalidation. This focuses on the PASID cache management for passthrough devices as there is no PASID capable emulated devices yet. When guest modifies a PASID entry, QEMU will capture the guest pasid selective pasid cache invalidation, allocate or remove a VTDAddressSpace instance per the invalidation reasons: *) a present pasid entry moved to non-present *) a present pasid entry to be a present entry *) a non-present pasid entry moved to present vIOMMU emulator could figure out the reason by fetching latest guest pasid entry and compare it with the PASID cache. Signed-off-by: Yi Liu Signed-off-by: Yi Sun Signed-off-by: Zhenzhong Duan --- hw/i386/intel_iommu_internal.h | 29 ++ include/hw/i386/intel_iommu.h | 6 + hw/i386/intel_iommu.c | 484 ++++++++++++++++++++++++++++++++- hw/i386/trace-events | 4 + 4 files changed, 513 insertions(+), 10 deletions(-) diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h index 18bc22fc72..632fda2853 100644 --- a/hw/i386/intel_iommu_internal.h +++ b/hw/i386/intel_iommu_internal.h @@ -315,6 +315,7 @@ typedef enum VTDFaultReason { * request while disabled */ VTD_FR_IR_SID_ERR = 0x26, /* Invalid Source-ID */ + VTD_FR_RTADDR_INV_TTM = 0x31, /* Invalid TTM in RTADDR */ /* PASID directory entry access failure */ VTD_FR_PASID_DIR_ACCESS_ERR = 0x50, /* The Present(P) field of pasid directory entry is 0 */ @@ -492,6 +493,15 @@ typedef union VTDInvDesc VTDInvDesc; #define VTD_INV_DESC_PIOTLB_RSVD_VAL0 0xfff000000000f1c0ULL #define VTD_INV_DESC_PIOTLB_RSVD_VAL1 0xf80ULL +#define VTD_INV_DESC_PASIDC_G (3ULL << 4) +#define VTD_INV_DESC_PASIDC_PASID(val) (((val) >> 32) & 0xfffffULL) +#define VTD_INV_DESC_PASIDC_DID(val) (((val) >> 16) & VTD_DOMAIN_ID_MASK) +#define VTD_INV_DESC_PASIDC_RSVD_VAL0 0xfff000000000f1c0ULL + +#define VTD_INV_DESC_PASIDC_DSI (0ULL << 4) +#define VTD_INV_DESC_PASIDC_PASID_SI (1ULL << 4) +#define VTD_INV_DESC_PASIDC_GLOBAL (3ULL << 4) + /* Information about page-selective IOTLB invalidate */ struct VTDIOTLBPageInvInfo { uint16_t domain_id; @@ -548,10 +558,28 @@ typedef struct VTDRootEntry VTDRootEntry; #define VTD_CTX_ENTRY_LEGACY_SIZE 16 #define VTD_CTX_ENTRY_SCALABLE_SIZE 32 +#define VTD_SM_CONTEXT_ENTRY_PDTS(val) (((val) >> 9) & 0x7) #define VTD_SM_CONTEXT_ENTRY_RID2PASID_MASK 0xfffff #define VTD_SM_CONTEXT_ENTRY_RSVD_VAL0(aw) (0x1e0ULL | ~VTD_HAW_MASK(aw)) #define VTD_SM_CONTEXT_ENTRY_RSVD_VAL1 0xffffffffffe00000ULL +typedef enum VTDPCInvType { + /* force reset all */ + VTD_PASID_CACHE_FORCE_RESET = 0, + /* pasid cache invalidation rely on guest PASID entry */ + VTD_PASID_CACHE_GLOBAL_INV, + VTD_PASID_CACHE_DOMSI, + VTD_PASID_CACHE_PASIDSI, +} VTDPCInvType; + +typedef struct VTDPASIDCacheInfo { + VTDPCInvType type; + uint16_t domain_id; + uint32_t pasid; + PCIBus *bus; + uint16_t devfn; +} VTDPASIDCacheInfo; + /* PASID Table Related Definitions */ #define VTD_PASID_DIR_BASE_ADDR_MASK (~0xfffULL) #define VTD_PASID_TABLE_BASE_ADDR_MASK (~0xfffULL) @@ -563,6 +591,7 @@ typedef struct VTDRootEntry VTDRootEntry; #define VTD_PASID_TABLE_BITS_MASK (0x3fULL) #define VTD_PASID_TABLE_INDEX(pasid) ((pasid) & VTD_PASID_TABLE_BITS_MASK) #define VTD_PASID_ENTRY_FPD (1ULL << 1) /* Fault Processing Disable */ +#define VTD_PASID_TBL_ENTRY_NUM (1ULL << 6) /* PASID Granular Translation Type Mask */ #define VTD_PASID_ENTRY_P 1ULL diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h index 50f9b27a45..fbc9da903a 100644 --- a/include/hw/i386/intel_iommu.h +++ b/include/hw/i386/intel_iommu.h @@ -95,6 +95,11 @@ struct VTDPASIDEntry { uint64_t val[8]; }; +typedef struct VTDPASIDCacheEntry { + struct VTDPASIDEntry pasid_entry; + bool cache_filled; +} VTDPASIDCacheEntry; + struct VTDAddressSpace { PCIBus *bus; uint8_t devfn; @@ -107,6 +112,7 @@ struct VTDAddressSpace { MemoryRegion iommu_ir_fault; /* Interrupt region for catching fault */ IntelIOMMUState *iommu_state; VTDContextCacheEntry context_cache_entry; + VTDPASIDCacheEntry pasid_cache_entry; QLIST_ENTRY(VTDAddressSpace) next; /* Superset of notifier flags that this address space has */ IOMMUNotifierFlag notifier_flags; diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index fafa199f52..b8f3b85803 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -86,6 +86,8 @@ struct vtd_iotlb_key { static void vtd_address_space_refresh_all(IntelIOMMUState *s); static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n); +static void vtd_pasid_cache_reset(IntelIOMMUState *s); + static void vtd_panic_require_caching_mode(void) { error_report("We need to set caching-mode=on for intel-iommu to enable " @@ -390,6 +392,7 @@ static void vtd_reset_caches(IntelIOMMUState *s) vtd_iommu_lock(s); vtd_reset_iotlb_locked(s); vtd_reset_context_cache_locked(s); + vtd_pasid_cache_reset(s); vtd_iommu_unlock(s); } @@ -825,6 +828,16 @@ static inline bool vtd_pe_type_check(IntelIOMMUState *s, VTDPASIDEntry *pe) } } +static inline uint32_t vtd_sm_ce_get_pdt_entry_num(VTDContextEntry *ce) +{ + return 1U << (VTD_SM_CONTEXT_ENTRY_PDTS(ce->val[0]) + 7); +} + +static inline uint16_t vtd_pe_get_domain_id(VTDPASIDEntry *pe) +{ + return VTD_SM_PASID_ENTRY_DID((pe)->val[1]); +} + static inline bool vtd_pdire_present(VTDPASIDDirEntry *pdire) { return pdire->val & 1; @@ -1617,6 +1630,54 @@ static int vtd_as_to_context_entry(VTDAddressSpace *vtd_as, VTDContextEntry *ce) } } +/* Translate to iommu pasid if PCI_NO_PASID */ +static int vtd_as_to_iommu_pasid(VTDAddressSpace *vtd_as, uint32_t *pasid) +{ + VTDContextEntry ce; + int ret; + + ret = vtd_as_to_context_entry(vtd_as, &ce); + if (ret) { + return ret; + } + + if (vtd_as->pasid == PCI_NO_PASID) { + *pasid = VTD_CE_GET_RID2PASID(&ce); + } else { + *pasid = vtd_as->pasid; + } + + return 0; +} + +static gboolean vtd_find_as_by_sid_and_iommu_pasid(gpointer key, gpointer value, + gpointer user_data) +{ + VTDAddressSpace *vtd_as = (VTDAddressSpace *)value; + struct vtd_as_raw_key *target = (struct vtd_as_raw_key *)user_data; + uint16_t sid = PCI_BUILD_BDF(pci_bus_num(vtd_as->bus), vtd_as->devfn); + uint32_t pasid; + + if (vtd_as_to_iommu_pasid(vtd_as, &pasid)) { + return false; + } + + return (pasid == target->pasid) && (sid == target->sid); +} + +/* Translate iommu pasid to vtd_as */ +static VTDAddressSpace *vtd_as_from_iommu_pasid(IntelIOMMUState *s, + uint16_t sid, uint32_t pasid) +{ + struct vtd_as_raw_key key = { + .sid = sid, + .pasid = pasid + }; + + return g_hash_table_find(s->vtd_address_spaces, + vtd_find_as_by_sid_and_iommu_pasid, &key); +} + static int vtd_sync_shadow_page_hook(const IOMMUTLBEvent *event, void *private) { @@ -3062,6 +3123,412 @@ static bool vtd_process_piotlb_desc(IntelIOMMUState *s, return true; } +static inline int vtd_dev_get_pe_from_pasid(VTDAddressSpace *vtd_as, + uint32_t pasid, VTDPASIDEntry *pe) +{ + IntelIOMMUState *s = vtd_as->iommu_state; + VTDContextEntry ce; + int ret; + + if (!s->root_scalable) { + return -VTD_FR_RTADDR_INV_TTM; + } + + ret = vtd_as_to_context_entry(vtd_as, &ce); + if (ret) { + return ret; + } + + return vtd_ce_get_pasid_entry(s, &ce, pe, pasid); +} + +static bool vtd_pasid_entry_compare(VTDPASIDEntry *p1, VTDPASIDEntry *p2) +{ + return !memcmp(p1, p2, sizeof(*p1)); +} + +/* + * This function fills in the pasid entry in &vtd_as. Caller + * of this function should hold iommu_lock. + */ +static void vtd_fill_pe_in_cache(IntelIOMMUState *s, VTDAddressSpace *vtd_as, + VTDPASIDEntry *pe) +{ + VTDPASIDCacheEntry *pc_entry = &vtd_as->pasid_cache_entry; + + if (vtd_pasid_entry_compare(pe, &pc_entry->pasid_entry)) { + /* No need to go further as cached pasid entry is latest */ + return; + } + + pc_entry->pasid_entry = *pe; + pc_entry->cache_filled = true; + /* + * TODO: send pasid bind to host for passthru devices + */ +} + +/* + * This function is used to clear cached pasid entry in vtd_as + * instances. Caller of this function should hold iommu_lock. + */ +static gboolean vtd_flush_pasid(gpointer key, gpointer value, + gpointer user_data) +{ + VTDPASIDCacheInfo *pc_info = user_data; + VTDAddressSpace *vtd_as = value; + IntelIOMMUState *s = vtd_as->iommu_state; + VTDPASIDCacheEntry *pc_entry = &vtd_as->pasid_cache_entry; + VTDPASIDEntry pe; + uint16_t did; + uint32_t pasid; + int ret; + + /* Replay only fill pasid entry cache for passthrough device */ + if (!pc_entry->cache_filled) { + return false; + } + did = vtd_pe_get_domain_id(&pc_entry->pasid_entry); + + if (vtd_as_to_iommu_pasid(vtd_as, &pasid)) { + goto remove; + } + + switch (pc_info->type) { + case VTD_PASID_CACHE_FORCE_RESET: + goto remove; + case VTD_PASID_CACHE_PASIDSI: + if (pc_info->pasid != pasid) { + return false; + } + /* Fall through */ + case VTD_PASID_CACHE_DOMSI: + if (pc_info->domain_id != did) { + return false; + } + /* Fall through */ + case VTD_PASID_CACHE_GLOBAL_INV: + break; + default: + error_report("invalid pc_info->type"); + abort(); + } + + /* + * pasid cache invalidation may indicate a present pasid + * entry to present pasid entry modification. To cover such + * case, vIOMMU emulator needs to fetch latest guest pasid + * entry and check cached pasid entry, then update pasid + * cache and send pasid bind/unbind to host properly. + */ + ret = vtd_dev_get_pe_from_pasid(vtd_as, pasid, &pe); + if (ret) { + /* + * No valid pasid entry in guest memory. e.g. pasid entry + * was modified to be either all-zero or non-present. Either + * case means existing pasid cache should be removed. + */ + goto remove; + } + + vtd_fill_pe_in_cache(s, vtd_as, &pe); + return false; + +remove: + /* + * TODO: send pasid unbind to host for passthru devices + */ + pc_entry->cache_filled = false; + + /* + * Don't remove address space of PCI_NO_PASID which is created by PCI + * sub-system. + */ + if (vtd_as->pasid == PCI_NO_PASID) { + return false; + } + return true; +} + +/* Caller of this function should hold iommu_lock */ +static void vtd_pasid_cache_reset(IntelIOMMUState *s) +{ + VTDPASIDCacheInfo pc_info; + + trace_vtd_pasid_cache_reset(); + + pc_info.type = VTD_PASID_CACHE_FORCE_RESET; + + /* + * Reset pasid cache is a big hammer, so use + * g_hash_table_foreach_remove which will free + * the vtd_as instances. Also, as a big + * hammer, use VTD_PASID_CACHE_FORCE_RESET to + * ensure all the vtd_as instances are + * dropped, meanwhile the change will be passed + * to host if HostIOMMUDeviceIOMMUFD is available. + */ + g_hash_table_foreach_remove(s->vtd_address_spaces, + vtd_flush_pasid, &pc_info); +} + +/* Caller of this function should hold iommu_lock. */ +static void vtd_sm_pasid_table_walk_one(IntelIOMMUState *s, + dma_addr_t pt_base, + int start, + int end, + VTDPASIDCacheInfo *info) +{ + VTDPASIDEntry pe; + int pasid = start; + int pasid_next; + + while (pasid < end) { + pasid_next = pasid + 1; + + if (!vtd_get_pe_in_pasid_leaf_table(s, pasid, pt_base, &pe) + && vtd_pe_present(&pe)) { + int bus_n = pci_bus_num(info->bus), devfn = info->devfn; + uint16_t sid = PCI_BUILD_BDF(bus_n, devfn); + VTDAddressSpace *vtd_as; + + vtd_as = vtd_as_from_iommu_pasid(s, sid, pasid); + if (!vtd_as) { + vtd_as = vtd_find_add_as(s, info->bus, devfn, pasid); + } + + if ((info->type == VTD_PASID_CACHE_DOMSI || + info->type == VTD_PASID_CACHE_PASIDSI) && + !(info->domain_id == vtd_pe_get_domain_id(&pe))) { + /* + * VTD_PASID_CACHE_DOMSI and VTD_PASID_CACHE_PASIDSI + * requires domain ID check. If domain Id check fail, + * go to next pasid. + */ + pasid = pasid_next; + continue; + } + vtd_fill_pe_in_cache(s, vtd_as, &pe); + } + pasid = pasid_next; + } +} + +/* + * Currently, VT-d scalable mode pasid table is a two level table, + * this function aims to loop a range of PASIDs in a given pasid + * table to identify the pasid config in guest. + * Caller of this function should hold iommu_lock. + */ +static void vtd_sm_pasid_table_walk(IntelIOMMUState *s, + dma_addr_t pdt_base, + int start, + int end, + VTDPASIDCacheInfo *info) +{ + VTDPASIDDirEntry pdire; + int pasid = start; + int pasid_next; + dma_addr_t pt_base; + + while (pasid < end) { + pasid_next = ((end - pasid) > VTD_PASID_TBL_ENTRY_NUM) ? + (pasid + VTD_PASID_TBL_ENTRY_NUM) : end; + if (!vtd_get_pdire_from_pdir_table(pdt_base, pasid, &pdire) + && vtd_pdire_present(&pdire)) { + pt_base = pdire.val & VTD_PASID_TABLE_BASE_ADDR_MASK; + vtd_sm_pasid_table_walk_one(s, pt_base, pasid, pasid_next, info); + } + pasid = pasid_next; + } +} + +static void vtd_replay_pasid_bind_for_dev(IntelIOMMUState *s, + int start, int end, + VTDPASIDCacheInfo *info) +{ + VTDContextEntry ce; + VTDAddressSpace *vtd_as; + + vtd_as = vtd_find_add_as(s, info->bus, info->devfn, PCI_NO_PASID); + + if (!vtd_as_to_context_entry(vtd_as, &ce)) { + uint32_t max_pasid; + + max_pasid = vtd_sm_ce_get_pdt_entry_num(&ce) * VTD_PASID_TBL_ENTRY_NUM; + if (end > max_pasid) { + end = max_pasid; + } + vtd_sm_pasid_table_walk(s, + VTD_CE_GET_PASID_DIR_TABLE(&ce), + start, + end, + info); + } +} + +/* + * This function replay the guest pasid bindings to hosts by + * walking the guest PASID table. This ensures host will have + * latest guest pasid bindings. Caller should hold iommu_lock. + */ +static void vtd_replay_guest_pasid_bindings(IntelIOMMUState *s, + VTDPASIDCacheInfo *pc_info) +{ + VTDHostIOMMUDevice *vtd_hiod; + int start = 0, end = 1; /* only rid2pasid is supported */ + VTDPASIDCacheInfo walk_info; + GHashTableIter as_it; + + switch (pc_info->type) { + case VTD_PASID_CACHE_PASIDSI: + start = pc_info->pasid; + end = pc_info->pasid + 1; + /* + * PASID selective invalidation is within domain, + * thus fall through. + */ + case VTD_PASID_CACHE_DOMSI: + case VTD_PASID_CACHE_GLOBAL_INV: + /* loop all assigned devices */ + break; + case VTD_PASID_CACHE_FORCE_RESET: + /* For force reset, no need to go further replay */ + return; + default: + error_report("invalid pc_info->type for replay"); + abort(); + } + + /* + * In this replay, only needs to care about the devices which + * are backed by host IOMMU. For such devices, their vtd_hiod + * instances are in the s->vtd_host_iommu_dev. For devices which + * are not backed by host IOMMU, it is not necessary to replay + * the bindings since their cache could be re-created in the future + * DMA address translation. + */ + walk_info = *pc_info; + g_hash_table_iter_init(&as_it, s->vtd_host_iommu_dev); + while (g_hash_table_iter_next(&as_it, NULL, (void **)&vtd_hiod)) { + /* bus|devfn fields are not identical with pc_info */ + walk_info.bus = vtd_hiod->bus; + walk_info.devfn = vtd_hiod->devfn; + vtd_replay_pasid_bind_for_dev(s, start, end, &walk_info); + } +} + +/* + * This function syncs the pasid bindings between guest and host. + * It includes updating the pasid cache in vIOMMU and updating the + * pasid bindings per guest's latest pasid entry presence. + */ +static void vtd_pasid_cache_sync(IntelIOMMUState *s, + VTDPASIDCacheInfo *pc_info) +{ + if (!s->flts || !s->root_scalable || !s->dmar_enabled) { + return; + } + + /* + * Regards to a pasid cache invalidation, e.g. a PSI. + * it could be either cases of below: + * a) a present pasid entry moved to non-present + * b) a present pasid entry to be a present entry + * c) a non-present pasid entry moved to present + * + * Different invalidation granularity may affect different device + * scope and pasid scope. But for each invalidation granularity, + * it needs to do two steps to sync host and guest pasid binding. + * + * Here is the handling of a PSI: + * 1) loop all the existing vtd_as instances to update them + * according to the latest guest pasid entry in pasid table. + * this will make sure affected existing vtd_as instances + * cached the latest pasid entries. Also, during the loop, the + * host should be notified if needed. e.g. pasid unbind or pasid + * update. Should be able to cover case a) and case b). + * + * 2) loop all devices to cover case c) + * - For devices which are backed by HostIOMMUDeviceIOMMUFD instances, + * we loop them and check if guest pasid entry exists. If yes, + * it is case c), we update the pasid cache and also notify + * host. + * - For devices which are not backed by HostIOMMUDeviceIOMMUFD, + * it is not necessary to create pasid cache at this phase since + * it could be created when vIOMMU does DMA address translation. + * This is not yet implemented since there is no emulated + * pasid-capable devices today. If we have such devices in + * future, the pasid cache shall be created there. + * Other granularity follow the same steps, just with different scope + * + */ + + vtd_iommu_lock(s); + /* Step 1: loop all the existing vtd_as instances */ + g_hash_table_foreach_remove(s->vtd_address_spaces, + vtd_flush_pasid, pc_info); + + /* + * Step 2: loop all the existing vtd_hiod instances. + * Ideally, needs to loop all devices to find if there is any new + * PASID binding regards to the PASID cache invalidation request. + * But it is enough to loop the devices which are backed by host + * IOMMU. For devices backed by vIOMMU (a.k.a emulated devices), + * if new PASID happened on them, their vtd_as instance could + * be created during future vIOMMU DMA translation. + */ + vtd_replay_guest_pasid_bindings(s, pc_info); + vtd_iommu_unlock(s); +} + +static bool vtd_process_pasid_desc(IntelIOMMUState *s, + VTDInvDesc *inv_desc) +{ + uint16_t domain_id; + uint32_t pasid; + VTDPASIDCacheInfo pc_info; + uint64_t mask[4] = {VTD_INV_DESC_PASIDC_RSVD_VAL0, VTD_INV_DESC_ALL_ONE, + VTD_INV_DESC_ALL_ONE, VTD_INV_DESC_ALL_ONE}; + + if (!vtd_inv_desc_reserved_check(s, inv_desc, mask, true, + __func__, "pasid cache inv")) { + return false; + } + + domain_id = VTD_INV_DESC_PASIDC_DID(inv_desc->val[0]); + pasid = VTD_INV_DESC_PASIDC_PASID(inv_desc->val[0]); + + switch (inv_desc->val[0] & VTD_INV_DESC_PASIDC_G) { + case VTD_INV_DESC_PASIDC_DSI: + trace_vtd_pasid_cache_dsi(domain_id); + pc_info.type = VTD_PASID_CACHE_DOMSI; + pc_info.domain_id = domain_id; + break; + + case VTD_INV_DESC_PASIDC_PASID_SI: + /* PASID selective implies a DID selective */ + trace_vtd_pasid_cache_psi(domain_id, pasid); + pc_info.type = VTD_PASID_CACHE_PASIDSI; + pc_info.domain_id = domain_id; + pc_info.pasid = pasid; + break; + + case VTD_INV_DESC_PASIDC_GLOBAL: + trace_vtd_pasid_cache_gsi(); + pc_info.type = VTD_PASID_CACHE_GLOBAL_INV; + break; + + default: + error_report_once("invalid-inv-granu-in-pc_inv_desc hi: 0x%" PRIx64 + " lo: 0x%" PRIx64, inv_desc->val[1], inv_desc->val[0]); + return false; + } + + vtd_pasid_cache_sync(s, &pc_info); + return true; +} + static bool vtd_process_inv_iec_desc(IntelIOMMUState *s, VTDInvDesc *inv_desc) { @@ -3223,6 +3690,13 @@ static bool vtd_process_inv_desc(IntelIOMMUState *s) } break; + case VTD_INV_DESC_PC: + trace_vtd_inv_desc("pasid-cache", inv_desc.val[1], inv_desc.val[0]); + if (!vtd_process_pasid_desc(s, &inv_desc)) { + return false; + } + break; + case VTD_INV_DESC_PIOTLB: trace_vtd_inv_desc("p-iotlb", inv_desc.val[1], inv_desc.val[0]); if (!vtd_process_piotlb_desc(s, &inv_desc)) { @@ -3258,16 +3732,6 @@ static bool vtd_process_inv_desc(IntelIOMMUState *s) } break; - /* - * TODO: the entity of below two cases will be implemented in future series. - * To make guest (which integrates scalable mode support patch set in - * iommu driver) work, just return true is enough so far. - */ - case VTD_INV_DESC_PC: - if (s->scalable_mode) { - break; - } - /* fallthrough */ default: error_report_once("%s: invalid inv desc: hi=%"PRIx64", lo=%"PRIx64 " (unknown type)", __func__, inv_desc.hi, diff --git a/hw/i386/trace-events b/hw/i386/trace-events index 53c02d7ac8..a26b38b52c 100644 --- a/hw/i386/trace-events +++ b/hw/i386/trace-events @@ -24,6 +24,10 @@ vtd_inv_qi_head(uint16_t head) "read head %d" vtd_inv_qi_tail(uint16_t head) "write tail %d" vtd_inv_qi_fetch(void) "" vtd_context_cache_reset(void) "" +vtd_pasid_cache_gsi(void) "" +vtd_pasid_cache_reset(void) "" +vtd_pasid_cache_dsi(uint16_t domain) "Domain slective PC invalidation domain 0x%"PRIx16 +vtd_pasid_cache_psi(uint16_t domain, uint32_t pasid) "PASID slective PC invalidation domain 0x%"PRIx16" pasid 0x%"PRIx32 vtd_re_not_present(uint8_t bus) "Root entry bus %"PRIu8" not present" vtd_ce_not_present(uint8_t bus, uint8_t devfn) "Context entry bus %"PRIu8" devfn %"PRIu8" not present" vtd_iotlb_page_hit(uint16_t sid, uint64_t addr, uint64_t slpte, uint16_t domain) "IOTLB page hit sid 0x%"PRIx16" iova 0x%"PRIx64" slpte 0x%"PRIx64" domain 0x%"PRIx16 From patchwork Wed Feb 19 08:22:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 515E2C021AA for ; Wed, 19 Feb 2025 08:28:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfQt-0006mX-V4; Wed, 19 Feb 2025 03:27:52 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQQ-0005sh-76 for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:22 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQK-0004ec-4k for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953636; x=1771489636; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sOBnLHCFTP5L2Gm+1vdPakaLuqtDi38bAAMzuIlq5Bk=; b=RGOP6m2QVfBUc09y2fcPd6ZQPbSKhFSF34hPLhpfrowGJRS+KgkkfZLd irdF+1d14HQ23GA1Bmq88t2zwk0SZaJDWwvBNIyuFjfPrxHqcZVsKtX7+ J85zmoRJ3HsqbLqA9Vbmyl6iLj/IzbxA1XfHGdoc6j3o99coP+fRAF/mp YaGicGLnC+cXGSxdQdv1y89VcseXuhXT6st/YTnJu5oxWzUduubzy1S+l EoMhcy5OIQEra8vmHHjwObUWlZv88Bu723RPFYVS6exEB6eZSUMsdxI0c nE6fO6/AVu+aaMK1nFBiv44tPpvSkarun+lKbHyqeixdCtFaCbUq7PNNY w==; X-CSE-ConnectionGUID: 1dH22jbcTCuKV4sdR3R2Bg== X-CSE-MsgGUID: rPBF+nK7TRq2Tpn1vm/KqQ== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544255" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544255" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:14 -0800 X-CSE-ConnectionGUID: M7CLyQkvSJ+XPAwJj2Gj5w== X-CSE-MsgGUID: /XdgePEQT4ef7gROyQWwrw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851314" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:08 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Yi Sun , Marcel Apfelbaum , Paolo Bonzini , Richard Henderson , Eduardo Habkost Subject: [PATCH rfcv2 14/20] intel_iommu: Bind/unbind guest page table to host Date: Wed, 19 Feb 2025 16:22:22 +0800 Message-Id: <20250219082228.3303163-15-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This captures the guest PASID table entry modifications and propagates the changes to host to attach a hwpt with type determined per guest PGTT configuration. When PGTT is Pass-through(100b), the hwpt on host side is a stage-2 page table(GPA->HPA). When PGTT is First-stage Translation only(001b), the hwpt on host side is a nested page table. The guest page table is configured as stage-1 page table (gIOVA->GPA) whose translation result would further go through host VT-d stage-2 page table(GPA->HPA) under nested translation mode. This is the key to support gIOVA over stage-1 page table for Intel VT-d in virtualization environment. Stage-2 page table could be shared by different devices if there is no conflict and devices link to same iommufd object, i.e. devices under same host IOMMU can share same stage-2 page table. If there is conflict, i.e. there is one device under non cache coherency mode which is different from others, it requires a separate stage-2 page table in non-CC mode. See below example diagram: IntelIOMMUState | V .------------------. .------------------. | VTDIOASContainer |--->| VTDIOASContainer |--->... | (iommufd0) | | (iommufd1) | .------------------. .------------------. | | | .-->... V .-------------------. .-------------------. | VTDS2Hwpt(CC) |--->| VTDS2Hwpt(non-CC) |-->... .-------------------. .-------------------. | | | | | | .-----------. .-----------. .------------. | IOMMUFD | | IOMMUFD | | IOMMUFD | | Device(CC)| | Device(CC)| | Device | | (iommufd0)| | (iommufd0)| | (non-CC) | | | | | | (iommufd0) | .-----------. .-----------. .------------. Co-Authored-by: Yi Liu Signed-off-by: Yi Liu Signed-off-by: Yi Sun Signed-off-by: Zhenzhong Duan --- hw/i386/intel_iommu_internal.h | 11 + include/hw/i386/intel_iommu.h | 24 ++ hw/i386/intel_iommu.c | 581 +++++++++++++++++++++++++++++++-- hw/i386/trace-events | 8 + 4 files changed, 604 insertions(+), 20 deletions(-) diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h index 632fda2853..23b7e236b0 100644 --- a/hw/i386/intel_iommu_internal.h +++ b/hw/i386/intel_iommu_internal.h @@ -563,6 +563,13 @@ typedef struct VTDRootEntry VTDRootEntry; #define VTD_SM_CONTEXT_ENTRY_RSVD_VAL0(aw) (0x1e0ULL | ~VTD_HAW_MASK(aw)) #define VTD_SM_CONTEXT_ENTRY_RSVD_VAL1 0xffffffffffe00000ULL +typedef enum VTDPASIDOp { + VTD_PASID_BIND, + VTD_PASID_UPDATE, + VTD_PASID_UNBIND, + VTD_OP_NUM +} VTDPASIDOp; + typedef enum VTDPCInvType { /* force reset all */ VTD_PASID_CACHE_FORCE_RESET = 0, @@ -578,6 +585,7 @@ typedef struct VTDPASIDCacheInfo { uint32_t pasid; PCIBus *bus; uint16_t devfn; + bool error_happened; } VTDPASIDCacheInfo; /* PASID Table Related Definitions */ @@ -606,6 +614,9 @@ typedef struct VTDPASIDCacheInfo { #define VTD_SM_PASID_ENTRY_FLPM 3ULL #define VTD_SM_PASID_ENTRY_FLPTPTR (~0xfffULL) +#define VTD_SM_PASID_ENTRY_SRE_BIT(val) (!!((val) & 1ULL)) +#define VTD_SM_PASID_ENTRY_WPE_BIT(val) (!!(((val) >> 4) & 1ULL)) +#define VTD_SM_PASID_ENTRY_EAFE_BIT(val) (!!(((val) >> 7) & 1ULL)) /* First Level Paging Structure */ /* Masks for First Level Paging Entry */ diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h index fbc9da903a..594281c1d3 100644 --- a/include/hw/i386/intel_iommu.h +++ b/include/hw/i386/intel_iommu.h @@ -100,10 +100,32 @@ typedef struct VTDPASIDCacheEntry { bool cache_filled; } VTDPASIDCacheEntry; +typedef struct VTDIOASContainer { + struct IOMMUFDBackend *iommufd; + uint32_t ioas_id; + MemoryListener listener; + QLIST_HEAD(, VTDS2Hwpt) s2_hwpt_list; + QLIST_ENTRY(VTDIOASContainer) next; + Error *error; +} VTDIOASContainer; + +typedef struct VTDS2Hwpt { + uint32_t users; + uint32_t hwpt_id; + VTDIOASContainer *container; + QLIST_ENTRY(VTDS2Hwpt) next; +} VTDS2Hwpt; + +typedef struct VTDHwpt { + uint32_t hwpt_id; + VTDS2Hwpt *s2_hwpt; +} VTDHwpt; + struct VTDAddressSpace { PCIBus *bus; uint8_t devfn; uint32_t pasid; + VTDHwpt hwpt; AddressSpace as; IOMMUMemoryRegion iommu; MemoryRegion root; /* The root container of the device */ @@ -303,6 +325,8 @@ struct IntelIOMMUState { GHashTable *vtd_host_iommu_dev; /* VTDHostIOMMUDevice */ + QLIST_HEAD(, VTDIOASContainer) containers; + /* interrupt remapping */ bool intr_enabled; /* Whether guest enabled IR */ dma_addr_t intr_root; /* Interrupt remapping table pointer */ diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index b8f3b85803..e36ac44110 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -20,6 +20,7 @@ */ #include "qemu/osdep.h" +#include CONFIG_DEVICES /* CONFIG_IOMMUFD */ #include "qemu/error-report.h" #include "qemu/main-loop.h" #include "qapi/error.h" @@ -40,6 +41,9 @@ #include "migration/vmstate.h" #include "trace.h" #include "system/iommufd.h" +#ifdef CONFIG_IOMMUFD +#include +#endif /* context entry operations */ #define VTD_CE_GET_RID2PASID(ce) \ @@ -838,11 +842,40 @@ static inline uint16_t vtd_pe_get_domain_id(VTDPASIDEntry *pe) return VTD_SM_PASID_ENTRY_DID((pe)->val[1]); } +static inline dma_addr_t vtd_pe_get_flpt_base(VTDPASIDEntry *pe) +{ + return pe->val[2] & VTD_SM_PASID_ENTRY_FLPTPTR; +} + +static inline uint32_t vtd_pe_get_fl_aw(VTDPASIDEntry *pe) +{ + return 48 + ((pe->val[2] >> 2) & VTD_SM_PASID_ENTRY_FLPM) * 9; +} + +static inline bool vtd_pe_pgtt_is_pt(VTDPASIDEntry *pe) +{ + return (VTD_PE_GET_TYPE(pe) == VTD_SM_PASID_ENTRY_PT); +} + +/* check if pgtt is first stage translation */ +static inline bool vtd_pe_pgtt_is_flt(VTDPASIDEntry *pe) +{ + return (VTD_PE_GET_TYPE(pe) == VTD_SM_PASID_ENTRY_FLT); +} + static inline bool vtd_pdire_present(VTDPASIDDirEntry *pdire) { return pdire->val & 1; } +static inline void pasid_cache_info_set_error(VTDPASIDCacheInfo *pc_info) +{ + if (pc_info->error_happened) { + return; + } + pc_info->error_happened = true; +} + /** * Caller of this function should check present bit if wants * to use pdir entry for further usage except for fpd bit check. @@ -1774,7 +1807,7 @@ static bool vtd_dev_pt_enabled(IntelIOMMUState *s, VTDContextEntry *ce, */ return false; } - return (VTD_PE_GET_TYPE(&pe) == VTD_SM_PASID_ENTRY_PT); + return vtd_pe_pgtt_is_pt(&pe); } return (vtd_ce_get_type(ce) == VTD_CONTEXT_TT_PASS_THROUGH); @@ -2409,6 +2442,497 @@ static void vtd_context_global_invalidate(IntelIOMMUState *s) vtd_iommu_replay_all(s); } +#ifdef CONFIG_IOMMUFD +static bool iommufd_listener_skipped_section(MemoryRegionSection *section) +{ + return !memory_region_is_ram(section->mr) || + memory_region_is_protected(section->mr) || + /* + * Sizing an enabled 64-bit BAR can cause spurious mappings to + * addresses in the upper part of the 64-bit address space. These + * are never accessed by the CPU and beyond the address width of + * some IOMMU hardware. TODO: VFIO should tell us the IOMMU width. + */ + section->offset_within_address_space & (1ULL << 63); +} + +static void iommufd_listener_region_add_s2domain(MemoryListener *listener, + MemoryRegionSection *section) +{ + VTDIOASContainer *container = container_of(listener, + VTDIOASContainer, listener); + IOMMUFDBackend *iommufd = container->iommufd; + uint32_t ioas_id = container->ioas_id; + hwaddr iova; + Int128 llend, llsize; + void *vaddr; + Error *err = NULL; + int ret; + + if (iommufd_listener_skipped_section(section)) { + return; + } + iova = REAL_HOST_PAGE_ALIGN(section->offset_within_address_space); + llend = int128_make64(section->offset_within_address_space); + llend = int128_add(llend, section->size); + llend = int128_and(llend, int128_exts64(qemu_real_host_page_mask())); + llsize = int128_sub(llend, int128_make64(iova)); + vaddr = memory_region_get_ram_ptr(section->mr) + + section->offset_within_region + + (iova - section->offset_within_address_space); + + memory_region_ref(section->mr); + + ret = iommufd_backend_map_dma(iommufd, ioas_id, iova, int128_get64(llsize), + vaddr, section->readonly); + if (!ret) { + return; + } + + error_setg(&err, + "iommufd_listener_region_add_s2domain(%p, 0x%"HWADDR_PRIx", " + "0x%"HWADDR_PRIx", %p) = %d (%s)", + container, iova, int128_get64(llsize), vaddr, ret, + strerror(-ret)); + + if (memory_region_is_ram_device(section->mr)) { + /* Allow unexpected mappings not to be fatal for RAM devices */ + error_report_err(err); + return; + } + + if (!container->error) { + error_propagate_prepend(&container->error, err, "Region %s: ", + memory_region_name(section->mr)); + } else { + error_free(err); + } +} + +static void iommufd_listener_region_del_s2domain(MemoryListener *listener, + MemoryRegionSection *section) +{ + VTDIOASContainer *container = container_of(listener, + VTDIOASContainer, listener); + IOMMUFDBackend *iommufd = container->iommufd; + uint32_t ioas_id = container->ioas_id; + hwaddr iova; + Int128 llend, llsize; + int ret; + + if (iommufd_listener_skipped_section(section)) { + return; + } + iova = REAL_HOST_PAGE_ALIGN(section->offset_within_address_space); + llend = int128_make64(section->offset_within_address_space); + llend = int128_add(llend, section->size); + llend = int128_and(llend, int128_exts64(qemu_real_host_page_mask())); + llsize = int128_sub(llend, int128_make64(iova)); + + ret = iommufd_backend_unmap_dma(iommufd, ioas_id, + iova, int128_get64(llsize)); + if (ret) { + error_report("iommufd_listener_region_del_s2domain(%p, " + "0x%"HWADDR_PRIx", 0x%"HWADDR_PRIx") = %d (%s)", + container, iova, int128_get64(llsize), ret, + strerror(-ret)); + } + + memory_region_unref(section->mr); +} + +static const MemoryListener iommufd_s2domain_memory_listener = { + .name = "iommufd_s2domain", + .priority = 1000, + .region_add = iommufd_listener_region_add_s2domain, + .region_del = iommufd_listener_region_del_s2domain, +}; + +static void vtd_init_s1_hwpt_data(struct iommu_hwpt_vtd_s1 *vtd, + VTDPASIDEntry *pe) +{ + memset(vtd, 0, sizeof(*vtd)); + + vtd->flags = (VTD_SM_PASID_ENTRY_SRE_BIT(pe->val[2]) ? + IOMMU_VTD_S1_SRE : 0) | + (VTD_SM_PASID_ENTRY_WPE_BIT(pe->val[2]) ? + IOMMU_VTD_S1_WPE : 0) | + (VTD_SM_PASID_ENTRY_EAFE_BIT(pe->val[2]) ? + IOMMU_VTD_S1_EAFE : 0); + vtd->addr_width = vtd_pe_get_fl_aw(pe); + vtd->pgtbl_addr = (uint64_t)vtd_pe_get_flpt_base(pe); +} + +static int vtd_create_s1_hwpt(HostIOMMUDeviceIOMMUFD *idev, + VTDS2Hwpt *s2_hwpt, VTDHwpt *hwpt, + VTDPASIDEntry *pe, Error **errp) +{ + struct iommu_hwpt_vtd_s1 vtd; + uint32_t hwpt_id, s2_hwpt_id = s2_hwpt->hwpt_id; + + vtd_init_s1_hwpt_data(&vtd, pe); + + if (!iommufd_backend_alloc_hwpt(idev->iommufd, idev->devid, + s2_hwpt_id, 0, IOMMU_HWPT_DATA_VTD_S1, + sizeof(vtd), &vtd, &hwpt_id, errp)) { + return -EINVAL; + } + + hwpt->hwpt_id = hwpt_id; + + return 0; +} + +static void vtd_destroy_s1_hwpt(HostIOMMUDeviceIOMMUFD *idev, VTDHwpt *hwpt) +{ + iommufd_backend_free_id(idev->iommufd, hwpt->hwpt_id); +} + +static VTDS2Hwpt *vtd_ioas_container_get_s2_hwpt(VTDIOASContainer *container, + uint32_t hwpt_id) +{ + VTDS2Hwpt *s2_hwpt; + + QLIST_FOREACH(s2_hwpt, &container->s2_hwpt_list, next) { + if (s2_hwpt->hwpt_id == hwpt_id) { + return s2_hwpt; + } + } + + s2_hwpt = g_malloc0(sizeof(*s2_hwpt)); + + s2_hwpt->hwpt_id = hwpt_id; + s2_hwpt->container = container; + QLIST_INSERT_HEAD(&container->s2_hwpt_list, s2_hwpt, next); + + return s2_hwpt; +} + +static void vtd_ioas_container_put_s2_hwpt(VTDS2Hwpt *s2_hwpt) +{ + VTDIOASContainer *container = s2_hwpt->container; + + if (s2_hwpt->users) { + return; + } + + QLIST_REMOVE(s2_hwpt, next); + iommufd_backend_free_id(container->iommufd, s2_hwpt->hwpt_id); + g_free(s2_hwpt); +} + +static void vtd_ioas_container_destroy(VTDIOASContainer *container) +{ + if (!QLIST_EMPTY(&container->s2_hwpt_list)) { + return; + } + + QLIST_REMOVE(container, next); + memory_listener_unregister(&container->listener); + iommufd_backend_free_id(container->iommufd, container->ioas_id); + g_free(container); +} + +static int vtd_device_attach_hwpt(VTDHostIOMMUDevice *vtd_hiod, + uint32_t pasid, VTDPASIDEntry *pe, + VTDS2Hwpt *s2_hwpt, VTDHwpt *hwpt, + Error **errp) +{ + HostIOMMUDeviceIOMMUFD *idev = HOST_IOMMU_DEVICE_IOMMUFD(vtd_hiod->hiod); + int ret; + + if (vtd_pe_pgtt_is_flt(pe)) { + ret = vtd_create_s1_hwpt(idev, s2_hwpt, hwpt, pe, errp); + if (ret) { + return ret; + } + } else { + hwpt->hwpt_id = s2_hwpt->hwpt_id; + } + + ret = !host_iommu_device_iommufd_attach_hwpt(idev, hwpt->hwpt_id, errp); + trace_vtd_device_attach_hwpt(idev->devid, pasid, hwpt->hwpt_id, ret); + if (ret) { + if (vtd_pe_pgtt_is_flt(pe)) { + vtd_destroy_s1_hwpt(idev, hwpt); + } + hwpt->hwpt_id = 0; + error_report("devid %d pasid %d failed to attach hwpt %d", + idev->devid, pasid, hwpt->hwpt_id); + return ret; + } + + s2_hwpt->users++; + hwpt->s2_hwpt = s2_hwpt; + + return 0; +} + +static void vtd_device_detach_hwpt(VTDHostIOMMUDevice *vtd_hiod, + uint32_t pasid, VTDPASIDEntry *pe, + VTDHwpt *hwpt, Error **errp) +{ + HostIOMMUDeviceIOMMUFD *idev = HOST_IOMMU_DEVICE_IOMMUFD(vtd_hiod->hiod); + int ret; + + if (vtd_hiod->iommu_state->dmar_enabled) { + ret = !host_iommu_device_iommufd_detach_hwpt(idev, errp); + trace_vtd_device_detach_hwpt(idev->devid, pasid, ret); + } else { + ret = !host_iommu_device_iommufd_attach_hwpt(idev, idev->hwpt_id, errp); + trace_vtd_device_reattach_def_hwpt(idev->devid, pasid, idev->hwpt_id, + ret); + } + + if (ret) { + error_report("devid %d pasid %d failed to attach hwpt %d", + idev->devid, pasid, hwpt->hwpt_id); + } + + if (vtd_pe_pgtt_is_flt(pe)) { + vtd_destroy_s1_hwpt(idev, hwpt); + } + + hwpt->s2_hwpt->users--; + hwpt->s2_hwpt = NULL; + hwpt->hwpt_id = 0; +} + +static int vtd_device_attach_container(VTDHostIOMMUDevice *vtd_hiod, + VTDIOASContainer *container, + uint32_t pasid, VTDPASIDEntry *pe, + VTDHwpt *hwpt, Error **errp) +{ + HostIOMMUDeviceIOMMUFD *idev = HOST_IOMMU_DEVICE_IOMMUFD(vtd_hiod->hiod); + IOMMUFDBackend *iommufd = idev->iommufd; + VTDS2Hwpt *s2_hwpt; + uint32_t s2_hwpt_id; + Error *err = NULL; + int ret; + + /* try to attach to an existing hwpt in this container */ + QLIST_FOREACH(s2_hwpt, &container->s2_hwpt_list, next) { + ret = vtd_device_attach_hwpt(vtd_hiod, pasid, pe, s2_hwpt, hwpt, &err); + if (ret) { + const char *msg = error_get_pretty(err); + + trace_vtd_device_fail_attach_existing_hwpt(msg); + error_free(err); + err = NULL; + } else { + goto found_hwpt; + } + } + + if (!iommufd_backend_alloc_hwpt(iommufd, idev->devid, container->ioas_id, + IOMMU_HWPT_ALLOC_NEST_PARENT, + IOMMU_HWPT_DATA_NONE, 0, NULL, + &s2_hwpt_id, errp)) { + return -EINVAL; + } + + s2_hwpt = vtd_ioas_container_get_s2_hwpt(container, s2_hwpt_id); + + /* Attach vtd device to a new allocated hwpt within iommufd */ + ret = vtd_device_attach_hwpt(vtd_hiod, pasid, pe, s2_hwpt, hwpt, errp); + if (ret) { + goto err_attach_hwpt; + } + +found_hwpt: + trace_vtd_device_attach_container(iommufd->fd, idev->devid, pasid, + container->ioas_id, hwpt->hwpt_id); + return 0; + +err_attach_hwpt: + vtd_ioas_container_put_s2_hwpt(s2_hwpt); + return ret; +} + +static void vtd_device_detach_container(VTDHostIOMMUDevice *vtd_hiod, + uint32_t pasid, VTDPASIDEntry *pe, + VTDHwpt *hwpt, Error **errp) +{ + HostIOMMUDeviceIOMMUFD *idev = HOST_IOMMU_DEVICE_IOMMUFD(vtd_hiod->hiod); + IOMMUFDBackend *iommufd = idev->iommufd; + VTDS2Hwpt *s2_hwpt = hwpt->s2_hwpt; + + trace_vtd_device_detach_container(iommufd->fd, idev->devid, pasid); + vtd_device_detach_hwpt(vtd_hiod, pasid, pe, hwpt, errp); + vtd_ioas_container_put_s2_hwpt(s2_hwpt); +} + +static int vtd_device_attach_iommufd(VTDHostIOMMUDevice *vtd_hiod, + uint32_t pasid, VTDPASIDEntry *pe, + VTDHwpt *hwpt, Error **errp) +{ + HostIOMMUDeviceIOMMUFD *idev = HOST_IOMMU_DEVICE_IOMMUFD(vtd_hiod->hiod); + IOMMUFDBackend *iommufd = idev->iommufd; + IntelIOMMUState *s = vtd_hiod->iommu_state; + VTDIOASContainer *container; + Error *err = NULL; + uint32_t ioas_id; + int ret; + + /* try to attach to an existing container in this space */ + QLIST_FOREACH(container, &s->containers, next) { + if (container->iommufd != iommufd) { + continue; + } + + if (vtd_device_attach_container(vtd_hiod, container, pasid, pe, hwpt, + &err)) { + const char *msg = error_get_pretty(err); + + trace_vtd_device_fail_attach_existing_container(msg); + error_free(err); + err = NULL; + } else { + return 0; + } + } + + /* Need to allocate a new dedicated container */ + ret = iommufd_backend_alloc_ioas(iommufd, &ioas_id, errp); + if (ret < 0) { + return ret; + } + + trace_vtd_device_alloc_ioas(iommufd->fd, ioas_id); + + container = g_malloc0(sizeof(*container)); + container->iommufd = iommufd; + container->ioas_id = ioas_id; + QLIST_INIT(&container->s2_hwpt_list); + + if (vtd_device_attach_container(vtd_hiod, container, pasid, pe, hwpt, + errp)) { + goto err_attach_container; + } + + container->listener = iommufd_s2domain_memory_listener; + memory_listener_register(&container->listener, &address_space_memory); + + if (container->error) { + ret = -1; + error_propagate_prepend(errp, container->error, + "memory listener initialization failed: "); + goto err_listener_register; + } + + QLIST_INSERT_HEAD(&s->containers, container, next); + + return 0; + +err_listener_register: + vtd_device_detach_container(vtd_hiod, pasid, pe, hwpt, errp); +err_attach_container: + iommufd_backend_free_id(iommufd, container->ioas_id); + g_free(container); + return ret; +} + +static void vtd_device_detach_iommufd(VTDHostIOMMUDevice *vtd_hiod, + uint32_t pasid, VTDPASIDEntry *pe, + VTDHwpt *hwpt, Error **errp) +{ + VTDIOASContainer *container = hwpt->s2_hwpt->container; + + vtd_device_detach_container(vtd_hiod, pasid, pe, hwpt, errp); + vtd_ioas_container_destroy(container); +} + +static int vtd_device_attach_pgtbl(VTDHostIOMMUDevice *vtd_hiod, + VTDAddressSpace *vtd_as, VTDPASIDEntry *pe) +{ + /* + * If pe->gptt != FLT, should be go ahead to do bind as host only + * accepts guest FLT under nesting. If pe->pgtt==PT, should setup + * the pasid with GPA page table. Otherwise should return failure. + */ + if (!vtd_pe_pgtt_is_flt(pe) && !vtd_pe_pgtt_is_pt(pe)) { + return -EINVAL; + } + + /* Should fail if the FLPT base is 0 */ + if (vtd_pe_pgtt_is_flt(pe) && !vtd_pe_get_flpt_base(pe)) { + return -EINVAL; + } + + return vtd_device_attach_iommufd(vtd_hiod, vtd_as->pasid, pe, + &vtd_as->hwpt, &error_abort); +} + +static int vtd_device_detach_pgtbl(VTDHostIOMMUDevice *vtd_hiod, + VTDAddressSpace *vtd_as) +{ + VTDPASIDEntry *cached_pe = vtd_as->pasid_cache_entry.cache_filled ? + &vtd_as->pasid_cache_entry.pasid_entry : NULL; + + if (!cached_pe || + (!vtd_pe_pgtt_is_flt(cached_pe) && !vtd_pe_pgtt_is_pt(cached_pe))) { + return 0; + } + + vtd_device_detach_iommufd(vtd_hiod, vtd_as->pasid, cached_pe, + &vtd_as->hwpt, &error_abort); + + return 0; +} + +/** + * Caller should hold iommu_lock. + */ +static int vtd_bind_guest_pasid(VTDAddressSpace *vtd_as, + VTDPASIDEntry *pe, VTDPASIDOp op) +{ + IntelIOMMUState *s = vtd_as->iommu_state; + VTDHostIOMMUDevice *vtd_hiod; + int devfn = vtd_as->devfn; + int ret = -EINVAL; + struct vtd_as_key key = { + .bus = vtd_as->bus, + .devfn = devfn, + }; + + vtd_hiod = g_hash_table_lookup(s->vtd_host_iommu_dev, &key); + if (!vtd_hiod || !vtd_hiod->hiod) { + /* means no need to go further, e.g. for emulated devices */ + return 0; + } + + if (vtd_as->pasid != PCI_NO_PASID) { + error_report("Non-rid_pasid %d not supported yet", vtd_as->pasid); + return ret; + } + + switch (op) { + case VTD_PASID_UPDATE: + case VTD_PASID_BIND: + { + ret = vtd_device_attach_pgtbl(vtd_hiod, vtd_as, pe); + break; + } + case VTD_PASID_UNBIND: + { + ret = vtd_device_detach_pgtbl(vtd_hiod, vtd_as); + break; + } + default: + error_report_once("Unknown VTDPASIDOp!!!\n"); + break; + } + + return ret; +} +#else +static int vtd_bind_guest_pasid(VTDAddressSpace *vtd_as, + VTDPASIDEntry *pe, VTDPASIDOp op) +{ + return 0; +} +#endif + /* Do a context-cache device-selective invalidation. * @func_mask: FM field after shifting */ @@ -3151,21 +3675,27 @@ static bool vtd_pasid_entry_compare(VTDPASIDEntry *p1, VTDPASIDEntry *p2) * This function fills in the pasid entry in &vtd_as. Caller * of this function should hold iommu_lock. */ -static void vtd_fill_pe_in_cache(IntelIOMMUState *s, VTDAddressSpace *vtd_as, - VTDPASIDEntry *pe) +static int vtd_fill_pe_in_cache(IntelIOMMUState *s, VTDAddressSpace *vtd_as, + VTDPASIDEntry *pe) { VTDPASIDCacheEntry *pc_entry = &vtd_as->pasid_cache_entry; + int ret; - if (vtd_pasid_entry_compare(pe, &pc_entry->pasid_entry)) { - /* No need to go further as cached pasid entry is latest */ - return; + if (pc_entry->cache_filled) { + if (vtd_pasid_entry_compare(pe, &pc_entry->pasid_entry)) { + /* No need to go further as cached pasid entry is latest */ + return 0; + } + ret = vtd_bind_guest_pasid(vtd_as, pe, VTD_PASID_UPDATE); + } else { + ret = vtd_bind_guest_pasid(vtd_as, pe, VTD_PASID_BIND); } - pc_entry->pasid_entry = *pe; - pc_entry->cache_filled = true; - /* - * TODO: send pasid bind to host for passthru devices - */ + if (!ret) { + pc_entry->pasid_entry = *pe; + pc_entry->cache_filled = true; + } + return ret; } /* @@ -3231,14 +3761,20 @@ static gboolean vtd_flush_pasid(gpointer key, gpointer value, goto remove; } - vtd_fill_pe_in_cache(s, vtd_as, &pe); + if (vtd_fill_pe_in_cache(s, vtd_as, &pe)) { + pasid_cache_info_set_error(pc_info); + } return false; remove: - /* - * TODO: send pasid unbind to host for passthru devices - */ - pc_entry->cache_filled = false; + if (pc_entry->cache_filled) { + if (vtd_bind_guest_pasid(vtd_as, NULL, VTD_PASID_UNBIND)) { + pasid_cache_info_set_error(pc_info); + return false; + } else { + pc_entry->cache_filled = false; + } + } /* * Don't remove address space of PCI_NO_PASID which is created by PCI @@ -3253,7 +3789,7 @@ remove: /* Caller of this function should hold iommu_lock */ static void vtd_pasid_cache_reset(IntelIOMMUState *s) { - VTDPASIDCacheInfo pc_info; + VTDPASIDCacheInfo pc_info = { .error_happened = false, }; trace_vtd_pasid_cache_reset(); @@ -3308,7 +3844,9 @@ static void vtd_sm_pasid_table_walk_one(IntelIOMMUState *s, pasid = pasid_next; continue; } - vtd_fill_pe_in_cache(s, vtd_as, &pe); + if (vtd_fill_pe_in_cache(s, vtd_as, &pe)) { + pasid_cache_info_set_error(info); + } } pasid = pasid_next; } @@ -3416,6 +3954,9 @@ static void vtd_replay_guest_pasid_bindings(IntelIOMMUState *s, walk_info.devfn = vtd_hiod->devfn; vtd_replay_pasid_bind_for_dev(s, start, end, &walk_info); } + if (walk_info.error_happened) { + pasid_cache_info_set_error(pc_info); + } } /* @@ -3485,9 +4026,9 @@ static void vtd_pasid_cache_sync(IntelIOMMUState *s, static bool vtd_process_pasid_desc(IntelIOMMUState *s, VTDInvDesc *inv_desc) { + VTDPASIDCacheInfo pc_info = { .error_happened = false, }; uint16_t domain_id; uint32_t pasid; - VTDPASIDCacheInfo pc_info; uint64_t mask[4] = {VTD_INV_DESC_PASIDC_RSVD_VAL0, VTD_INV_DESC_ALL_ONE, VTD_INV_DESC_ALL_ONE, VTD_INV_DESC_ALL_ONE}; @@ -3526,7 +4067,7 @@ static bool vtd_process_pasid_desc(IntelIOMMUState *s, } vtd_pasid_cache_sync(s, &pc_info); - return true; + return !pc_info.error_happened ? true : false; } static bool vtd_process_inv_iec_desc(IntelIOMMUState *s, diff --git a/hw/i386/trace-events b/hw/i386/trace-events index a26b38b52c..22559eb787 100644 --- a/hw/i386/trace-events +++ b/hw/i386/trace-events @@ -72,6 +72,14 @@ vtd_frr_new(int index, uint64_t hi, uint64_t lo) "index %d high 0x%"PRIx64" low vtd_warn_invalid_qi_tail(uint16_t tail) "tail 0x%"PRIx16 vtd_warn_ir_vector(uint16_t sid, int index, int vec, int target) "sid 0x%"PRIx16" index %d vec %d (should be: %d)" vtd_warn_ir_trigger(uint16_t sid, int index, int trig, int target) "sid 0x%"PRIx16" index %d trigger %d (should be: %d)" +vtd_device_attach_hwpt(uint32_t dev_id, uint32_t pasid, uint32_t hwpt_id, int ret) "dev_id %d pasid %d hwpt_id %d, ret: %d" +vtd_device_detach_hwpt(uint32_t dev_id, uint32_t pasid, int ret) "dev_id %d pasid %d ret: %d" +vtd_device_reattach_def_hwpt(uint32_t dev_id, uint32_t pasid, uint32_t hwpt_id, int ret) "dev_id %d pasid %d hwpt_id %d, ret: %d" +vtd_device_fail_attach_existing_hwpt(const char *msg) " %s" +vtd_device_attach_container(int fd, uint32_t dev_id, uint32_t pasid, uint32_t ioas_id, uint32_t hwpt_id) "iommufd %d dev_id %d pasid %d ioas_id %d hwpt_id %d" +vtd_device_detach_container(int fd, uint32_t dev_id, uint32_t pasid) "iommufd %d dev_id %d pasid %d" +vtd_device_fail_attach_existing_container(const char *msg) " %s" +vtd_device_alloc_ioas(int fd, uint32_t ioas_id) "iommufd %d ioas_id %d" # amd_iommu.c amdvi_evntlog_fail(uint64_t addr, uint32_t head) "error: fail to write at addr 0x%"PRIx64" + offset 0x%"PRIx32 From patchwork Wed Feb 19 08:22:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981706 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 369E1C021AA for ; Wed, 19 Feb 2025 08:30:02 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfQo-0006Fr-Dx; Wed, 19 Feb 2025 03:27:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQR-0005tS-Ca for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:23 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQN-0004eO-Pm for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953640; x=1771489640; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=e1ZoscbXzm2c5PiXufM2JI3ZULck95SCPA/nxqF9N3o=; b=F+E8ZtqXTjZB9F8nepYHLtMiupki9lFl3anZ0Fn0IxvYR2y1YSgFGLeJ 77ANMUiyA5RBE0B9k2pIT5vaz3YTLq7rISKQjGGCvx95p6h7CaqLZsbQG BjU5PdaCjGK9BCwspsGaaL4khyg8gJ5lr43aO6yE+XZH7MHL1SkeILGsm WRsf44LOQU2vixpQdGPNHJyK6piRiBlMcaLxEl96FdXAXsc/eWrcyat7d WWzNUoVAgwkL3mzNwBXGxI7kHDcmiJArLSQAp6CqAciqf+HOevZgLSYHO tlkWEX+VooB2vZxx7cTr0gHPh0WqvoDXN8gbvVfr3BQRgBa+LIaUN0Hve Q==; X-CSE-ConnectionGUID: TIAzrPTOT8eKzDmg+h/sQA== X-CSE-MsgGUID: +xZfryd/TqeR8CKydeOoCA== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544269" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544269" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:19 -0800 X-CSE-ConnectionGUID: ShIrIk7VSU+LIMYgGF3efQ== X-CSE-MsgGUID: jcMoocxfTTm8iXKV3FEq8A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851359" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:13 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Paolo Bonzini , Richard Henderson , Eduardo Habkost , Marcel Apfelbaum Subject: [PATCH rfcv2 15/20] intel_iommu: ERRATA_772415 workaround Date: Wed, 19 Feb 2025 16:22:23 +0800 Message-Id: <20250219082228.3303163-16-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org On a system influenced by ERRATA_772415, IOMMU_HW_INFO_VTD_ERRATA_772415_SPR17 is repored by IOMMU_DEVICE_GET_HW_INFO. Due to this errata, even the readonly range mapped on stage-2 page table could still be written. Reference from 4th Gen Intel Xeon Processor Scalable Family Specification Update, Errata Details, SPR17. [0] https://edc.intel.com/content/www/us/en/design/products-and-solutions/processors-and-chipsets/eagle-stream/sapphire-rapids-specification-update We utilize the new added IOMMUFD container/ioas/hwpt management framework in VTD. Add a check to create new VTDIOASContainer to hold RW-only mappings, then this VTDIOASContainer can be used as backend for device with ERRATA_772415. See below diagram for details: IntelIOMMUState | V .------------------. .------------------. .-------------------. | VTDIOASContainer |--->| VTDIOASContainer |--->| VTDIOASContainer |-->... | (iommufd0,RW&RO) | | (iommufd1,RW&RO) | | (iommufd0,RW only)| .------------------. .------------------. .-------------------. | | | | .-->... | V V .-------------------. .-------------------. .---------------. | VTDS2Hwpt(CC) |--->| VTDS2Hwpt(non-CC) |-->... | VTDS2Hwpt(CC) |-->... .-------------------. .-------------------. .---------------. | | | | | | | | .-----------. .-----------. .------------. .------------. | IOMMUFD | | IOMMUFD | | IOMMUFD | | IOMMUFD | | Device(CC)| | Device(CC)| | Device | | Device(CC) | | (iommufd0)| | (iommufd0)| | (non-CC) | | (errata) | | | | | | (iommufd0) | | (iommufd0) | .-----------. .-----------. .------------. .------------. Changed to pass VTDHostIOMMUDevice pointer to vtd_check_hdev() so errata could be saved. Suggested-by: Yi Liu Signed-off-by: Zhenzhong Duan --- hw/i386/intel_iommu_internal.h | 1 + include/hw/i386/intel_iommu.h | 1 + hw/i386/intel_iommu.c | 26 +++++++++++++++++++------- 3 files changed, 21 insertions(+), 7 deletions(-) diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h index 23b7e236b0..8558781af8 100644 --- a/hw/i386/intel_iommu_internal.h +++ b/hw/i386/intel_iommu_internal.h @@ -654,5 +654,6 @@ typedef struct VTDHostIOMMUDevice { PCIBus *bus; uint8_t devfn; HostIOMMUDevice *hiod; + uint32_t errata; } VTDHostIOMMUDevice; #endif diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h index 594281c1d3..9b156dc32e 100644 --- a/include/hw/i386/intel_iommu.h +++ b/include/hw/i386/intel_iommu.h @@ -103,6 +103,7 @@ typedef struct VTDPASIDCacheEntry { typedef struct VTDIOASContainer { struct IOMMUFDBackend *iommufd; uint32_t ioas_id; + uint32_t errata; MemoryListener listener; QLIST_HEAD(, VTDS2Hwpt) s2_hwpt_list; QLIST_ENTRY(VTDIOASContainer) next; diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index e36ac44110..dae1716629 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -2443,7 +2443,8 @@ static void vtd_context_global_invalidate(IntelIOMMUState *s) } #ifdef CONFIG_IOMMUFD -static bool iommufd_listener_skipped_section(MemoryRegionSection *section) +static bool iommufd_listener_skipped_section(VTDIOASContainer *container, + MemoryRegionSection *section) { return !memory_region_is_ram(section->mr) || memory_region_is_protected(section->mr) || @@ -2453,7 +2454,8 @@ static bool iommufd_listener_skipped_section(MemoryRegionSection *section) * are never accessed by the CPU and beyond the address width of * some IOMMU hardware. TODO: VFIO should tell us the IOMMU width. */ - section->offset_within_address_space & (1ULL << 63); + section->offset_within_address_space & (1ULL << 63) || + (container->errata && section->readonly); } static void iommufd_listener_region_add_s2domain(MemoryListener *listener, @@ -2469,7 +2471,7 @@ static void iommufd_listener_region_add_s2domain(MemoryListener *listener, Error *err = NULL; int ret; - if (iommufd_listener_skipped_section(section)) { + if (iommufd_listener_skipped_section(container, section)) { return; } iova = REAL_HOST_PAGE_ALIGN(section->offset_within_address_space); @@ -2520,7 +2522,7 @@ static void iommufd_listener_region_del_s2domain(MemoryListener *listener, Int128 llend, llsize; int ret; - if (iommufd_listener_skipped_section(section)) { + if (iommufd_listener_skipped_section(container, section)) { return; } iova = REAL_HOST_PAGE_ALIGN(section->offset_within_address_space); @@ -2776,7 +2778,8 @@ static int vtd_device_attach_iommufd(VTDHostIOMMUDevice *vtd_hiod, /* try to attach to an existing container in this space */ QLIST_FOREACH(container, &s->containers, next) { - if (container->iommufd != iommufd) { + if (container->iommufd != iommufd || + container->errata != vtd_hiod->errata) { continue; } @@ -2803,6 +2806,7 @@ static int vtd_device_attach_iommufd(VTDHostIOMMUDevice *vtd_hiod, container = g_malloc0(sizeof(*container)); container->iommufd = iommufd; container->ioas_id = ioas_id; + container->errata = vtd_hiod->errata; QLIST_INIT(&container->s2_hwpt_list); if (vtd_device_attach_container(vtd_hiod, container, pasid, pe, hwpt, @@ -5329,9 +5333,10 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, return vtd_dev_as; } -static bool vtd_check_hiod(IntelIOMMUState *s, HostIOMMUDevice *hiod, +static bool vtd_check_hiod(IntelIOMMUState *s, VTDHostIOMMUDevice *vtd_hiod, Error **errp) { + HostIOMMUDevice *hiod = vtd_hiod->hiod; HostIOMMUDeviceClass *hiodc = HOST_IOMMU_DEVICE_GET_CLASS(hiod); int ret; @@ -5388,6 +5393,12 @@ static bool vtd_check_hiod(IntelIOMMUState *s, HostIOMMUDevice *hiod, return false; } + ret = hiodc->get_cap(hiod, HOST_IOMMU_DEVICE_CAP_ERRATA, errp); + if (ret < 0) { + return false; + } + vtd_hiod->errata = ret; + error_setg(errp, "host device is uncompatible with stage-1 translation"); return false; } @@ -5419,7 +5430,8 @@ static bool vtd_dev_set_iommu_device(PCIBus *bus, void *opaque, int devfn, vtd_hiod->iommu_state = s; vtd_hiod->hiod = hiod; - if (!vtd_check_hiod(s, hiod, errp)) { + if (!vtd_check_hiod(s, vtd_hiod, errp)) { + g_free(vtd_hiod); vtd_iommu_unlock(s); return false; } From patchwork Wed Feb 19 08:22:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 25610C021B0 for ; Wed, 19 Feb 2025 08:29:45 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfQq-0006OR-1E; Wed, 19 Feb 2025 03:27:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQa-00063Y-Hd for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:37 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQW-0004fc-9h for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953648; x=1771489648; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yady3f19P/LfBVml+NtIfOp8QxskBtxjiZEwijgIpnY=; b=RsAvefW+Y0hCBBbBPNOfep2byQoT4I7sg/ncl1iarnpO+/dDP8Bsay4U /BWqfKttxz9Ic1aPJ6noaj5xcb+34/shyAy8F8pGXbgCT5HtiZ3GZ+cbP 7EfV2UjSqfzTzhOBJxzLWHmm6H5co6ll6hlrlF00xN6gYWfBHGno2HtMc ETeRpbWv7Vm6gvEqWSv10nHcCmw117ylc6znvizRRDVf6g1mFwuK4693s HeuXo4cCcLwOrf2zIabk3Ivfx2ZhRqbyPwW5D4KrDdbKRbldC4cNbg20U n2yl/5amr/kwhfhFTGPIoXRfwXttIm5yuHSr1486h1UwwmCvv4Mtk4lhq w==; X-CSE-ConnectionGUID: qWfmITiRQ4eQxR9IvGpnTw== X-CSE-MsgGUID: eoMt8YE9TyyMFJhn4ST9Ww== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544284" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544284" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:25 -0800 X-CSE-ConnectionGUID: UOj10ZOaRqCTJ8gN/2vmRA== X-CSE-MsgGUID: 9pyUlC9iSe6xhnGsU1Dk+w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851394" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:19 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Yi Sun , Zhenzhong Duan , Marcel Apfelbaum , Paolo Bonzini , Richard Henderson , Eduardo Habkost Subject: [PATCH rfcv2 16/20] intel_iommu: Replay pasid binds after context cache invalidation Date: Wed, 19 Feb 2025 16:22:24 +0800 Message-Id: <20250219082228.3303163-17-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Yi Liu This replays guest pasid attachments after context cache invalidation. This is a behavior to ensure safety. Actually, programmer should issue pasid cache invalidation with proper granularity after issuing a context cache invalidation. Signed-off-by: Yi Liu Signed-off-by: Yi Sun Signed-off-by: Zhenzhong Duan --- hw/i386/intel_iommu_internal.h | 1 + hw/i386/intel_iommu.c | 51 ++++++++++++++++++++++++++++++++-- hw/i386/trace-events | 1 + 3 files changed, 51 insertions(+), 2 deletions(-) diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h index 8558781af8..8f7be7f123 100644 --- a/hw/i386/intel_iommu_internal.h +++ b/hw/i386/intel_iommu_internal.h @@ -575,6 +575,7 @@ typedef enum VTDPCInvType { VTD_PASID_CACHE_FORCE_RESET = 0, /* pasid cache invalidation rely on guest PASID entry */ VTD_PASID_CACHE_GLOBAL_INV, + VTD_PASID_CACHE_DEVSI, VTD_PASID_CACHE_DOMSI, VTD_PASID_CACHE_PASIDSI, } VTDPCInvType; diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index dae1716629..e7376ba6a7 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -91,6 +91,10 @@ static void vtd_address_space_refresh_all(IntelIOMMUState *s); static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n); static void vtd_pasid_cache_reset(IntelIOMMUState *s); +static void vtd_pasid_cache_sync(IntelIOMMUState *s, + VTDPASIDCacheInfo *pc_info); +static void vtd_pasid_cache_devsi(IntelIOMMUState *s, + PCIBus *bus, uint16_t devfn); static void vtd_panic_require_caching_mode(void) { @@ -2423,6 +2427,8 @@ static void vtd_iommu_replay_all(IntelIOMMUState *s) static void vtd_context_global_invalidate(IntelIOMMUState *s) { + VTDPASIDCacheInfo pc_info = { .error_happened = false, }; + trace_vtd_inv_desc_cc_global(); /* Protects context cache */ vtd_iommu_lock(s); @@ -2440,6 +2446,9 @@ static void vtd_context_global_invalidate(IntelIOMMUState *s) * VT-d emulation codes. */ vtd_iommu_replay_all(s); + + pc_info.type = VTD_PASID_CACHE_GLOBAL_INV; + vtd_pasid_cache_sync(s, &pc_info); } #ifdef CONFIG_IOMMUFD @@ -2995,6 +3004,21 @@ static void vtd_context_device_invalidate(IntelIOMMUState *s, * happened. */ vtd_address_space_sync(vtd_as); + /* + * Per spec, context flush should also followed with PASID + * cache and iotlb flush. Regards to a device selective + * context cache invalidation: + * if (emaulted_device) + * invalidate pasid cache and pasid-based iotlb + * else if (assigned_device) + * check if the device has been bound to any pasid + * invoke pasid_unbind regards to each bound pasid + * Here, we have vtd_pasid_cache_devsi() to invalidate pasid + * caches, while for piotlb in QEMU, we don't have it yet, so + * no handling. For assigned device, host iommu driver would + * flush piotlb when a pasid unbind is pass down to it. + */ + vtd_pasid_cache_devsi(s, vtd_as->bus, devfn); } } } @@ -3743,6 +3767,11 @@ static gboolean vtd_flush_pasid(gpointer key, gpointer value, /* Fall through */ case VTD_PASID_CACHE_GLOBAL_INV: break; + case VTD_PASID_CACHE_DEVSI: + if (pc_info->bus != vtd_as->bus || pc_info->devfn != vtd_as->devfn) { + return false; + } + break; default: error_report("invalid pc_info->type"); abort(); @@ -3934,6 +3963,11 @@ static void vtd_replay_guest_pasid_bindings(IntelIOMMUState *s, case VTD_PASID_CACHE_GLOBAL_INV: /* loop all assigned devices */ break; + case VTD_PASID_CACHE_DEVSI: + walk_info.bus = pc_info->bus; + walk_info.devfn = pc_info->devfn; + vtd_replay_pasid_bind_for_dev(s, start, end, &walk_info); + return; case VTD_PASID_CACHE_FORCE_RESET: /* For force reset, no need to go further replay */ return; @@ -3968,8 +4002,7 @@ static void vtd_replay_guest_pasid_bindings(IntelIOMMUState *s, * It includes updating the pasid cache in vIOMMU and updating the * pasid bindings per guest's latest pasid entry presence. */ -static void vtd_pasid_cache_sync(IntelIOMMUState *s, - VTDPASIDCacheInfo *pc_info) +static void vtd_pasid_cache_sync(IntelIOMMUState *s, VTDPASIDCacheInfo *pc_info) { if (!s->flts || !s->root_scalable || !s->dmar_enabled) { return; @@ -4027,6 +4060,20 @@ static void vtd_pasid_cache_sync(IntelIOMMUState *s, vtd_iommu_unlock(s); } +static void vtd_pasid_cache_devsi(IntelIOMMUState *s, + PCIBus *bus, uint16_t devfn) +{ + VTDPASIDCacheInfo pc_info = { .error_happened = false, }; + + trace_vtd_pasid_cache_devsi(devfn); + + pc_info.type = VTD_PASID_CACHE_DEVSI; + pc_info.bus = bus; + pc_info.devfn = devfn; + + vtd_pasid_cache_sync(s, &pc_info); +} + static bool vtd_process_pasid_desc(IntelIOMMUState *s, VTDInvDesc *inv_desc) { diff --git a/hw/i386/trace-events b/hw/i386/trace-events index 22559eb787..e520133f18 100644 --- a/hw/i386/trace-events +++ b/hw/i386/trace-events @@ -28,6 +28,7 @@ vtd_pasid_cache_gsi(void) "" vtd_pasid_cache_reset(void) "" vtd_pasid_cache_dsi(uint16_t domain) "Domain slective PC invalidation domain 0x%"PRIx16 vtd_pasid_cache_psi(uint16_t domain, uint32_t pasid) "PASID slective PC invalidation domain 0x%"PRIx16" pasid 0x%"PRIx32 +vtd_pasid_cache_devsi(uint16_t devfn) "Dev selective PC invalidation dev: 0x%"PRIx16 vtd_re_not_present(uint8_t bus) "Root entry bus %"PRIu8" not present" vtd_ce_not_present(uint8_t bus, uint8_t devfn) "Context entry bus %"PRIu8" devfn %"PRIu8" not present" vtd_iotlb_page_hit(uint16_t sid, uint64_t addr, uint64_t slpte, uint16_t domain) "IOTLB page hit sid 0x%"PRIx16" iova 0x%"PRIx64" slpte 0x%"PRIx64" domain 0x%"PRIx16 From patchwork Wed Feb 19 08:22:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CB5F5C021B1 for ; Wed, 19 Feb 2025 08:29:22 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfQt-0006ks-Cp; Wed, 19 Feb 2025 03:27:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQc-00063c-Cw for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:37 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQZ-0004g2-N4 for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953652; x=1771489652; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tBh6tA0va6cynvyYuShl6iuhaQZExg1BzepN1L8cTG4=; b=F/hlCVrEjRhgkDbcxgFeaI8P4yiyEo6D32ioqpCO0p6WzRWu3HN9nOGp 9mmcu/KvZnZfqYWeN2Y1K9VvXcK2Hvr1jYgqbYHY48iHcQR/pP0g7uLOg M7Rg0TmxFIaaGeMhAZaUNCR1nUrqBWfkMEejkovLAqtsOdGwkfPOhAiOd CA+CdBy6IRkUBCaR3rH/rV2Vy70rfwOm/6genUmPBoB+UPTh890liEbqE rdAZXhKbgaug7/+h36gXpvrh95rbBJc5H1ZIvQ9a8yD4Rj4LS/Cdckg2+ 946eBhUkWXDmUfIw1I3hwcZr4+pq+QUIvi3etMr9u6tjae03Frc9Sc9G6 A==; X-CSE-ConnectionGUID: dhMSwklvQHioGrZWgtCYIg== X-CSE-MsgGUID: OGkpPujSSkS79jXeapN2bQ== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544300" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544300" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:30 -0800 X-CSE-ConnectionGUID: p6mYlznwThua97D5YVrlgw== X-CSE-MsgGUID: htWFNE/ZSqWZS2MwoRJ6CQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851408" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:24 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Yi Sun , Zhenzhong Duan , Paolo Bonzini , Richard Henderson , Eduardo Habkost , Marcel Apfelbaum Subject: [PATCH rfcv2 17/20] intel_iommu: Propagate PASID-based iotlb invalidation to host Date: Wed, 19 Feb 2025 16:22:25 +0800 Message-Id: <20250219082228.3303163-18-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Yi Liu This traps the guest PASID-based iotlb invalidation request and propagate it to host. Intel VT-d 3.0 supports nested translation in PASID granular. Guest SVA support could be implemented by configuring nested translation on specific PASID. This is also known as dual stage DMA translation. Under such configuration, guest owns the GVA->GPA translation which is configured as stage-1 page table in host side for a specific pasid, and host owns GPA->HPA translation. As guest owns stage-1 translation table, piotlb invalidation should be propagated to host since host IOMMU will cache first level page table related mappings during DMA address translation. Signed-off-by: Yi Liu Signed-off-by: Yi Sun Signed-off-by: Zhenzhong Duan --- hw/i386/intel_iommu_internal.h | 6 ++ hw/i386/intel_iommu.c | 116 ++++++++++++++++++++++++++++++++- 2 files changed, 120 insertions(+), 2 deletions(-) diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h index 8f7be7f123..630394a8c3 100644 --- a/hw/i386/intel_iommu_internal.h +++ b/hw/i386/intel_iommu_internal.h @@ -589,6 +589,12 @@ typedef struct VTDPASIDCacheInfo { bool error_happened; } VTDPASIDCacheInfo; +typedef struct VTDPIOTLBInvInfo { + uint16_t domain_id; + uint32_t pasid; + struct iommu_hwpt_vtd_s1_invalidate *inv_data; +} VTDPIOTLBInvInfo; + /* PASID Table Related Definitions */ #define VTD_PASID_DIR_BASE_ADDR_MASK (~0xfffULL) #define VTD_PASID_TABLE_BASE_ADDR_MASK (~0xfffULL) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index e7376ba6a7..8f7fb473f5 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -2938,12 +2938,108 @@ static int vtd_bind_guest_pasid(VTDAddressSpace *vtd_as, return ret; } + +/* + * Caller of this function should hold iommu_lock. + */ +static void vtd_invalidate_piotlb(VTDAddressSpace *vtd_as, + struct iommu_hwpt_vtd_s1_invalidate *cache) +{ + VTDHostIOMMUDevice *vtd_hiod; + HostIOMMUDeviceIOMMUFD *idev; + VTDHwpt *hwpt = &vtd_as->hwpt; + int devfn = vtd_as->devfn; + struct vtd_as_key key = { + .bus = vtd_as->bus, + .devfn = devfn, + }; + IntelIOMMUState *s = vtd_as->iommu_state; + uint32_t entry_num = 1; /* Only implement one request for simplicity */ + + if (!hwpt) { + return; + } + + vtd_hiod = g_hash_table_lookup(s->vtd_host_iommu_dev, &key); + if (!vtd_hiod || !vtd_hiod->hiod) { + return; + } + idev = HOST_IOMMU_DEVICE_IOMMUFD(vtd_hiod->hiod); + + if (iommufd_backend_invalidate_cache(idev->iommufd, hwpt->hwpt_id, + IOMMU_HWPT_INVALIDATE_DATA_VTD_S1, + sizeof(*cache), &entry_num, cache)) { + error_report("Cache flush failed, entry_num %d", entry_num); + } +} + +/* + * This function is a loop function for the s->vtd_address_spaces + * list with VTDPIOTLBInvInfo as execution filter. It propagates + * the piotlb invalidation to host. Caller of this function + * should hold iommu_lock. + */ +static void vtd_flush_pasid_iotlb(gpointer key, gpointer value, + gpointer user_data) +{ + VTDPIOTLBInvInfo *piotlb_info = user_data; + VTDAddressSpace *vtd_as = value; + VTDPASIDCacheEntry *pc_entry = &vtd_as->pasid_cache_entry; + uint32_t pasid; + uint16_t did; + + /* Replay only fill pasid entry cache for passthrough device */ + if (!pc_entry->cache_filled || + !vtd_pe_pgtt_is_flt(&pc_entry->pasid_entry)) { + return; + } + + if (vtd_as_to_iommu_pasid(vtd_as, &pasid)) { + return; + } + + did = vtd_pe_get_domain_id(&pc_entry->pasid_entry); + + if (piotlb_info->domain_id == did && piotlb_info->pasid == pasid) { + vtd_invalidate_piotlb(vtd_as, piotlb_info->inv_data); + } +} + +static void vtd_flush_pasid_iotlb_all(IntelIOMMUState *s, + uint16_t domain_id, uint32_t pasid, + hwaddr addr, uint64_t npages, bool ih) +{ + struct iommu_hwpt_vtd_s1_invalidate cache_info = { 0 }; + VTDPIOTLBInvInfo piotlb_info; + + cache_info.addr = addr; + cache_info.npages = npages; + cache_info.flags = ih ? IOMMU_VTD_INV_FLAGS_LEAF : 0; + + piotlb_info.domain_id = domain_id; + piotlb_info.pasid = pasid; + piotlb_info.inv_data = &cache_info; + + /* + * Here loops all the vtd_as instances in s->vtd_address_spaces + * to find out the affected devices since piotlb invalidation + * should check pasid cache per architecture point of view. + */ + g_hash_table_foreach(s->vtd_address_spaces, + vtd_flush_pasid_iotlb, &piotlb_info); +} #else static int vtd_bind_guest_pasid(VTDAddressSpace *vtd_as, VTDPASIDEntry *pe, VTDPASIDOp op) { return 0; } + +static void vtd_flush_pasid_iotlb_all(IntelIOMMUState *s, + uint16_t domain_id, uint32_t pasid, + hwaddr addr, uint64_t npages, bool ih) +{ +} #endif /* Do a context-cache device-selective invalidation. @@ -3597,6 +3693,13 @@ static void vtd_piotlb_pasid_invalidate(IntelIOMMUState *s, info.pasid = pasid; vtd_iommu_lock(s); + /* + * Here loops all the vtd_as instances in s->vtd_as + * to find out the affected devices since piotlb invalidation + * should check pasid cache per architecture point of view. + */ + vtd_flush_pasid_iotlb_all(s, domain_id, pasid, 0, (uint64_t)-1, 0); + g_hash_table_foreach_remove(s->iotlb, vtd_hash_remove_by_pasid, &info); vtd_iommu_unlock(s); @@ -3619,7 +3722,8 @@ static void vtd_piotlb_pasid_invalidate(IntelIOMMUState *s, } static void vtd_piotlb_page_invalidate(IntelIOMMUState *s, uint16_t domain_id, - uint32_t pasid, hwaddr addr, uint8_t am) + uint32_t pasid, hwaddr addr, uint8_t am, + bool ih) { VTDIOTLBPageInvInfo info; @@ -3629,6 +3733,13 @@ static void vtd_piotlb_page_invalidate(IntelIOMMUState *s, uint16_t domain_id, info.mask = ~((1 << am) - 1); vtd_iommu_lock(s); + /* + * Here loops all the vtd_as instances in s->vtd_as + * to find out the affected devices since piotlb invalidation + * should check pasid cache per architecture point of view. + */ + vtd_flush_pasid_iotlb_all(s, domain_id, pasid, addr, 1 << am, ih); + g_hash_table_foreach_remove(s->iotlb, vtd_hash_remove_by_page_piotlb, &info); vtd_iommu_unlock(s); @@ -3662,7 +3773,8 @@ static bool vtd_process_piotlb_desc(IntelIOMMUState *s, case VTD_INV_DESC_PIOTLB_PSI_IN_PASID: am = VTD_INV_DESC_PIOTLB_AM(inv_desc->val[1]); addr = (hwaddr) VTD_INV_DESC_PIOTLB_ADDR(inv_desc->val[1]); - vtd_piotlb_page_invalidate(s, domain_id, pasid, addr, am); + vtd_piotlb_page_invalidate(s, domain_id, pasid, addr, am, + VTD_INV_DESC_PIOTLB_IH(inv_desc->val[1])); break; default: From patchwork Wed Feb 19 08:22:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981704 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 79738C021AA for ; Wed, 19 Feb 2025 08:29:40 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfQr-0006br-ME; Wed, 19 Feb 2025 03:27:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQg-00066W-QD for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:41 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQe-0004ge-VL for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:38 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953657; x=1771489657; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Dh29gxAI0UuaBsVzjJ/FGrKdWjQRK2XyNd4DnP/LCxE=; b=Ibk4kxka8y8BbrXv3yWlWulcrgk5TkIQ6IvLTv8vI6sZ02wbxaWvhQt6 wAhgqVNpKrdWF7xQC9UB65yjfBkN+Rc2TXiJq7R0sGhpr1LtMM6lEZi99 7+GVdcmVpXldM/EM6kAOZTu89gJkFhn1T0QnSOZW/V0SqN5ENgW/jjutj OQdOaHDkdluRFLFzxRi1LBm+FwlR0LPRAviD+AZgavOE68bBIA2fd7W3q OQ8AOpeqXC2FsgT6PFAnQcf/2ohCTXVF8Fs2NDGlmzf2doXjArA/efoJq DkVZL9FjQdGDny3t6Vr1jDFB1jV7BvQoE4IRvn281IFmytUtbHdNjFU3/ w==; X-CSE-ConnectionGUID: Q5tAOMqXQh2Gz99MCtnFcg== X-CSE-MsgGUID: My3WnW/2SUO3iwnLclp3xg== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544313" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544313" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:35 -0800 X-CSE-ConnectionGUID: fTxj8ptnTo+W/tUYy5wWkg== X-CSE-MsgGUID: 1OcJ0nzZSoaYoFplt1NM1g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851437" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:30 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Paolo Bonzini , Richard Henderson , Eduardo Habkost , Marcel Apfelbaum Subject: [PATCH rfcv2 18/20] intel_iommu: Refresh pasid bind when either SRTP or TE bit is changed Date: Wed, 19 Feb 2025 16:22:26 +0800 Message-Id: <20250219082228.3303163-19-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Yi Liu When either 'Set Root Table Pointer' or 'Translation Enable' bit is changed, the pasid bindings on host side become stale and need to be updated. Introduce a helper function vtd_refresh_pasid_bind() for that purpose. Signed-off-by: Yi Liu Signed-off-by: Zhenzhong Duan --- hw/i386/intel_iommu.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 8f7fb473f5..225e332132 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -89,6 +89,7 @@ struct vtd_iotlb_key { static void vtd_address_space_refresh_all(IntelIOMMUState *s); static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n); +static void vtd_refresh_pasid_bind(IntelIOMMUState *s); static void vtd_pasid_cache_reset(IntelIOMMUState *s); static void vtd_pasid_cache_sync(IntelIOMMUState *s, @@ -3366,6 +3367,7 @@ static void vtd_handle_gcmd_srtp(IntelIOMMUState *s) vtd_set_clear_mask_long(s, DMAR_GSTS_REG, 0, VTD_GSTS_RTPS); vtd_reset_caches(s); vtd_address_space_refresh_all(s); + vtd_refresh_pasid_bind(s); } /* Set Interrupt Remap Table Pointer */ @@ -3400,6 +3402,7 @@ static void vtd_handle_gcmd_te(IntelIOMMUState *s, bool en) vtd_reset_caches(s); vtd_address_space_refresh_all(s); + vtd_refresh_pasid_bind(s); } /* Handle Interrupt Remap Enable/Disable */ @@ -4109,6 +4112,28 @@ static void vtd_replay_guest_pasid_bindings(IntelIOMMUState *s, } } +static void vtd_refresh_pasid_bind(IntelIOMMUState *s) +{ + VTDPASIDCacheInfo pc_info = { .error_happened = false, + .type = VTD_PASID_CACHE_GLOBAL_INV }; + + /* + * Only when dmar is enabled, should pasid bindings replayed, + * otherwise no need to replay. + */ + if (!s->dmar_enabled) { + return; + } + + if (!s->flts || !s->root_scalable) { + return; + } + + vtd_iommu_lock(s); + vtd_replay_guest_pasid_bindings(s, &pc_info); + vtd_iommu_unlock(s); +} + /* * This function syncs the pasid bindings between guest and host. * It includes updating the pasid cache in vIOMMU and updating the From patchwork Wed Feb 19 08:22:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981700 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A613CC021AB for ; Wed, 19 Feb 2025 08:29:01 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfQo-0006K2-T6; Wed, 19 Feb 2025 03:27:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQk-000691-Sp for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:43 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQj-0004ge-2b for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953661; x=1771489661; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=OA1DA9tb9D+FuuGPsY1Kf6tlYXcvdCgV9qaFOVXTlBY=; b=djohVP2bOpOdnLWah2pkd5PGT9rNadZ5HJnI1hKi6vC2DdZeU3H7JcKe oV2/uAQMPKa4iYAWGY1bfAhn4uV2lpEbNjEegKUrAX4PtGABMkQXtQg9Y 3jy96Q/i4F1i0vd0EQHkB47AOK5bhn180Uilw6lSyiGg62Gn+GjGVKTPg xUfBSF/ZFSGYA1K72vgmCWtCiAb/XsPoK6lh8rYugt3l0RnyubCHrUErD 0xDv2kt0VfaKixRXugNm1WBmpkAV9CZ4YaFNZpqxVRkqaU/Q+kgpyHHss w2mHiK8bylKGTV1Bu0k/McMVHZUbEYMmqVGFrmCoc3ksyJtk01mZiGV5c Q==; X-CSE-ConnectionGUID: xIcxOOMuQCS9MOOuunDNLw== X-CSE-MsgGUID: MSmNk24ETmm9V8WUv5BwDA== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544329" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544329" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:40 -0800 X-CSE-ConnectionGUID: DYzE6qIGTFKbry88xqHU1w== X-CSE-MsgGUID: t6XifuAgTAOhfzlcU27WIQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851461" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:35 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Marcel Apfelbaum , Paolo Bonzini , Richard Henderson , Eduardo Habkost Subject: [PATCH rfcv2 19/20] intel_iommu: Bypass replay in stage-1 page table mode Date: Wed, 19 Feb 2025 16:22:27 +0800 Message-Id: <20250219082228.3303163-20-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -47 X-Spam_score: -4.8 X-Spam_bar: ---- X-Spam_report: (-4.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.423, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org VFIO utilizes replay to setup initial shadow iommu mappings. But when stage-1 page table is configured, it is passed to host to construct nested page table, there is no replay needed. Signed-off-by: Zhenzhong Duan --- hw/i386/intel_iommu.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 225e332132..e4b83cbe50 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -5743,6 +5743,14 @@ static void vtd_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) VTDContextEntry ce; DMAMap map = { .iova = 0, .size = HWADDR_MAX }; + /* + * Replay on stage-1 page table is meaningless as stage-1 page table + * is passthroughed to host to construct nested page table + */ + if (s->flts && s->root_scalable) { + return; + } + /* replay is protected by BQL, page walk will re-setup it safely */ iova_tree_remove(vtd_as->iova_tree, map); From patchwork Wed Feb 19 08:22:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13981697 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5FADCC021B0 for ; Wed, 19 Feb 2025 08:28:17 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tkfQw-00071J-7Z; Wed, 19 Feb 2025 03:27:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQt-0006oL-WA for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:52 -0500 Received: from mgamail.intel.com ([192.198.163.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tkfQo-0004ge-Li for qemu-devel@nongnu.org; Wed, 19 Feb 2025 03:27:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739953667; x=1771489667; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0qmvEU6XCMXu31ihN2FZGKqfkGmXtaFOTK6nWdoHSus=; b=PzAFbHgAQcncvfg33S90Y4h7IAQ7o9nXXdGLCIvHaxe6ztA9xhTFTR6z OQPDDwio6IOvqVdLddroR+RhRg1qw/1zEZxZXrjSm36l9GCatP+P58w5t C4zEtqBqXv5LLC4OhzQjv6FN+a30brsEsWC7/TX30zuXRIn/fLGhU42h7 ZyDVuh0guUBJOB5AbPIm6pqeRiO4NertLN/I56Aq0jXbvLgmHze+Md1KJ pQDeWkE6z6iimKuu50UybMGz+5XjjEh5EVuvVNFFtT1zWwjO7Ce35kRpU e2tsD+HrJqtJBbiwyvUkM3RTm2GJIuRlXG1g0p4Hk4nqzt93frTCqp9Zz Q==; X-CSE-ConnectionGUID: cNbFuY/aQsC47BX+UbUENg== X-CSE-MsgGUID: l7qHe+9lSbKBz5esp45yIg== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="40544345" X-IronPort-AV: E=Sophos;i="6.13,298,1732608000"; d="scan'208";a="40544345" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:46 -0800 X-CSE-ConnectionGUID: HP5HhGKbQbG4ocpl8CByxQ== X-CSE-MsgGUID: PTdAgokAQv65nHUNicTE3Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="119851479" Received: from spr-s2600bt.bj.intel.com ([10.240.192.127]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2025 00:27:40 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, mst@redhat.com, jasowang@redhat.com, peterx@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, shameerali.kolothum.thodi@huawei.com, joao.m.martins@oracle.com, clement.mathieu--drif@eviden.com, kevin.tian@intel.com, yi.l.liu@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Paolo Bonzini , Richard Henderson , Eduardo Habkost , Marcel Apfelbaum Subject: [PATCH rfcv2 20/20] intel_iommu: Enable host device when x-flts=on in scalable mode Date: Wed, 19 Feb 2025 16:22:28 +0800 Message-Id: <20250219082228.3303163-21-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219082228.3303163-1-zhenzhong.duan@intel.com> References: <20250219082228.3303163-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.198.163.17; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -39 X-Spam_score: -4.0 X-Spam_bar: ---- X-Spam_report: (-4.0 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Now that all infrastructures of supporting passthrough device running with stage-1 translation are there, enable it now. Signed-off-by: Zhenzhong Duan --- hw/i386/intel_iommu.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index e4b83cbe50..908c28f9be 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -5583,8 +5583,7 @@ static bool vtd_check_hiod(IntelIOMMUState *s, VTDHostIOMMUDevice *vtd_hiod, } vtd_hiod->errata = ret; - error_setg(errp, "host device is uncompatible with stage-1 translation"); - return false; + return true; } static bool vtd_dev_set_iommu_device(PCIBus *bus, void *opaque, int devfn,