From patchwork Mon Jun 3 05:59:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: CLEMENT MATHIEU--DRIF X-Patchwork-Id: 13683280 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 24B37C25B75 for ; Mon, 3 Jun 2024 06:03:44 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sE0jd-0006fc-8y; Mon, 03 Jun 2024 01:59:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sE0jb-0006eo-8e for qemu-devel@nongnu.org; Mon, 03 Jun 2024 01:59:55 -0400 Received: from smarthost4.eviden.com ([80.78.11.85]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sE0jV-00071T-VL for qemu-devel@nongnu.org; Mon, 03 Jun 2024 01:59:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=eviden.com; i=@eviden.com; q=dns/txt; s=mail; t=1717394390; x=1748930390; h=from:to:cc:subject:date:message-id:content-id: content-transfer-encoding:mime-version; bh=SIeu1ywuoHGYO6QCaomSD+w6mCyRsjhBCta6xVoRJDk=; b=PVhGLbxE139HRDl7Gi5VsMURoOp89jMVly3joxwiZU1nzq1Gi8wbaT8V jCfA/0RshWYcW42omVBUoy3nPL6S3lYXbtP5iaCM713FluKDFJ4ZN/SmX PBBTJfF3WKAuwTUUk0lmyYCFVV5Dp9RMdeY8EDJU2kZSLy3yFzW4JEXmg cN6yavIhPc84+j09FkODW4JQOuCRQ73bJkh6/D+fvOn+CsSBptRdPDi4Y Zxd56OE6icT5/2OnB8jLl34IPeL8ByuHWaC8JH7dVHxz/0zvBXnviSejQ LQG7iqftz7dcyHyu75VlEsZFQLLmgD5/WgmTYTKne3MsqBCmk+ENgYAzC w==; X-IronPort-AV: E=Sophos;i="6.08,210,1712613600"; d="scan'208";a="13815872" X-MGA-submission: MDHFqjOjsc91RhfCD0/klwUmYDoYGk8/qStO5DAr0SE3WnpUt7l3ADOYgYucoaof+GKxAa/MpCtdpEwGwu5WPXn/yt27aLS6yiXyFPoG7ci9RSPKEThnyETd1r7EXhatDGuSbpPCmwNyPjyVzrl8mPmYq5PFift25ktpWxnfpYKtCw== Received: from mail-he1eur01lp2050.outbound.protection.outlook.com (HELO EUR01-HE1-obe.outbound.protection.outlook.com) ([104.47.0.50]) by smarthost4.eviden.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jun 2024 07:59:45 +0200 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ffovNp6ucwDyl1+r40lLUXzUgEJljMQSiMOaCohwvcoYN4/KOD37dnlxI+DEA3d3N8gZ3/3sEcl03xFYg0TRX7JXXTNLNveMeTpRbrPHN35Ax1Rdra9KTWo1jJi/xQuzkbtUZ/QfMVDN9QqM/KPeDmwVrpcsWDPge4Fiq79PX63sudOxajngiHRUAcnddpCC6mRCDYKR8FvoneT3xgSRH+XtVwZg5KynJwe4fw+vyiEeZxvf2dLAu/Ed+TOjvhl9ZZLoVKlXZMG3ReO0tv5AoKDmen5jhh1ksRAU+lZIv6U5AAGJFJazxc/nf2M1gAcCUGVrmbFlhKKTHe2T6ODnDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SIeu1ywuoHGYO6QCaomSD+w6mCyRsjhBCta6xVoRJDk=; b=I0EZD/em49GW5DkMyT0sxktO5ytLuApph28epCPuySBPOJfPdP5/ApSKp2+W7WfTcG8As64+3lwqMSyaZd6Ma+gGidgToVVcLn32uYyudKGZKjiuoRLGIfQbu7IR39gBMUG/MU96kdnd5B65AviHNK50dkP1Gt2PMgj7YK2BkwSI04P2bYGrkVdYwnhJwzTW7P/4W1dGkGl++UUMibOHJqi62D9Gbt05cMicHJwDgoLqxuNuNUoEoqXtnqrUhAED+Ym9RyEDVBsTIqeqFfqdDS1xxhd4rAYN+VRwbuScDj/hgT6ofJXJpOTG7u2GnZJLVN3GhdfNYurnU3NAxhFzBg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=eviden.com; dmarc=pass action=none header.from=eviden.com; dkim=pass header.d=eviden.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Eviden.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SIeu1ywuoHGYO6QCaomSD+w6mCyRsjhBCta6xVoRJDk=; b=qEVqHazRcfn5VZxs16Cggc4pWkC1ShxUc44P8aj9LAFUkU0t5TVW+8s5OGXNf0iJqe+5iih3QCWoJN3RlkSQ+ix9FAzdxRisau38X2MZ0T6r4ZS7YfRrEtuUW8nrrTc/lpNA+gn9MXYUOjTHhyBKmtToKnuD5/EbGDOikFMx74lvx5ABOH5Y3nwEWzOcJS3M2ULe0WCgbZsY3w1yk40J7HASmVpH24TcpM7qS+EuaWO3hXxB2o6dv7M4G/rUEU+IjbROveGbiJWsiZ2EOYC5BVoI6nCvgja32IR4caaOqzmoU/BdfS/fX7rLTxvFjITaH7LXv9GNg51R8Ln7lfROXg== Received: from AM8PR07MB7602.eurprd07.prod.outlook.com (2603:10a6:20b:24b::7) by VI1PR07MB6238.eurprd07.prod.outlook.com (2603:10a6:800:136::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21; Mon, 3 Jun 2024 05:59:39 +0000 Received: from AM8PR07MB7602.eurprd07.prod.outlook.com ([fe80::fbd7:ca71:b636:6f9d]) by AM8PR07MB7602.eurprd07.prod.outlook.com ([fe80::fbd7:ca71:b636:6f9d%5]) with mapi id 15.20.7633.021; Mon, 3 Jun 2024 05:59:38 +0000 From: CLEMENT MATHIEU--DRIF To: "qemu-devel@nongnu.org" CC: "jasowang@redhat.com" , "zhenzhong.duan@intel.com" , "kevin.tian@intel.com" , "yi.l.liu@intel.com" , "joao.m.martins@oracle.com" , "peterx@redhat.com" , "mst@redhat.com" , CLEMENT MATHIEU--DRIF Subject: [PATCH ats_vtd v5 00/22] ATS support for VT-d Thread-Topic: [PATCH ats_vtd v5 00/22] ATS support for VT-d Thread-Index: AQHatXs42Vo/2A63BES9fBhuPzCosQ== Date: Mon, 3 Jun 2024 05:59:38 +0000 Message-ID: <20240603055917.18735-1-clement.mathieu--drif@eviden.com> Accept-Language: en-GB, fr-FR, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=eviden.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: AM8PR07MB7602:EE_|VI1PR07MB6238:EE_ x-ms-office365-filtering-correlation-id: e06d7e04-1fdd-4b26-eaa0-08dc83925aaf x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230031|376005|366007|1800799015|38070700009; x-microsoft-antispam-message-info: =?utf-8?q?NGGUCdRxoaRWQWrZgvGnozyypiXxMPd?= =?utf-8?q?+medNYaBm4C7sffaAFlUkn+3d3e+M4v53xNTmCbGUo/dW2aP0J7vaUuAKE2xDu5CS?= =?utf-8?q?14cnA8G9nMTZeRSGeFzvgFFmW0XGG6gxykbLPFFV35TKO0fM+jf9EVU1xGYMVM0EV?= =?utf-8?q?VclFdh5hzskQDkqomshFO/yp9g1b3KdU4CmIwyOrnYojEr/wzaHRHg4csbGvcBfk1?= =?utf-8?q?2vqdQG1gNpUHU7mZxckD3z8hcenwlUORNCHaYd+q4CfThAszen+7f+F40aH4EanmX?= =?utf-8?q?dlwrkTajPXi+e6f6+WDBs10Eq4iUmsIVWqXo8aoya7b7zDv6bl7oumqyoETYmxthC?= =?utf-8?q?B6NQwOB14f9B8S29alkOjS37Wat8mXnH8OuwlcjrqDDuERBa6W6feophWavHvb7XE?= =?utf-8?q?SOvGhPDEWL/iWXLFc25X16iAKazX7VKMHQwGKgccuBm9YxHpmacbwLSf40PfRkYjs?= =?utf-8?q?yu1orA+OTfWiKB1M1jWAZl2KJPh1oMC1cTt/aDZSN+trYxTOZ8C6DvvxiQHeiLf0r?= =?utf-8?q?bY+AbWhM3bGvJJRl+BrIa+wDBeB6sUdjAtJDfg6IBDW6OZ6JOwrkDsIDUD2ZXuofs?= =?utf-8?q?0TzbYtlgQVUNA7u5/+Ah1g/fvh947BdYt5AmIXPfhd6TL9RzoOSavcwOU1PLuvH5M?= =?utf-8?q?dJcT0EhmdTpcY17Nkz/hnQyxdNH4gPfNKSwjGlWprOIWZ9nn+9aqlqbd9NQ6zS+5H?= =?utf-8?q?PsGWr7iqNN6mn96wb9HrLZqEF+E0qRCTTPcIN8OFcstlPVwppFc/qDeZs78OHx1t4?= =?utf-8?q?9rxHj4X2X0K94ulYuQVAlLvUL+c7zHYIYWYUGd0oW/EeDug0TTSz7Up5/SDkveWY/?= =?utf-8?q?8CjClnPOjCK9jfVHaX6468n6e9PuRZAHUJKGvwN56UVNu3hwGZ3aukaEXz9InFRCC?= =?utf-8?q?EhuLhx6Q2shcCkO5RW0FI0kx1x3h2+6IFwT9QeT8S8VXTqxEB/HIORKdBGKlarHfM?= =?utf-8?q?6E9SmsbkdBuTmklmcm55yWBMnOhhuSIEFoPw9ksbTmDOiIj1EEdOoKbr+JeTYOzMs?= =?utf-8?q?MLgSG6Hpz2kdRpWrTlN7CgAPcRU1Vfm6Hx5Y7J7ekKGjy8dci+QA0aIdeynHpTxU2?= =?utf-8?q?QdsXLVeM4gdw1eVvigOQvOptWEgh840kmwysaQkMtymbWouQXJrCNLmsTxabv5p7R?= =?utf-8?q?sEfqbN16n8D1vegcyZAQPsCscziAXNUP6oep6arP0NIjlAJDgrMwkeFAjYWQ11Ja3?= =?utf-8?q?H4dQ7v4g9jsAdOhNWhIKnB14gzoXEtEc9wU1WrVIIQdRPatZfTBeAl1y7lFDTsiRW?= =?utf-8?q?X2mKwy3X5wpN5EGAZjSTu4dBwpiXyD9+g7w=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM8PR07MB7602.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(376005)(366007)(1800799015)(38070700009); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?4cHjkue8YrGpKq96slg+ga5LaZbz?= =?utf-8?q?ZbH3gIkN0Jl/Iq45tl+lZmwNIAG+xkZvOJFxA2KTWBAacrL0Y2mQFfO5c999kjf95?= =?utf-8?q?YWL6Vjus7HkQTBsMRkFvHhO6qExoEKe75lHmWc15IlLeKc7iO7IvpGrDsLkahnXXt?= =?utf-8?q?y+oqrASixZ8Aif8RxwHT8FlHg3n04RN+oMxOkaM7jmAFj3Hg7RD0Fc+b99gTVwgtn?= =?utf-8?q?uim2xGBLj6/dhidiARHMGlT+XzEtZaZweD/FdDxQEuf5CZuLzoQDm1ygtcSOOiHva?= =?utf-8?q?nrym3dNqbXM0PLtVLdhlxUUfBA3XrOJsDzvt3YaGxXitx5wKRmpOi69zgCq7tjbHu?= =?utf-8?q?m7eRozD6fdghJqhJGG9cd2IW1YB/8gENNFNmDb45q4sSQr0U3/TQcgRKqA0BPPZnr?= =?utf-8?q?dX2xwhLUJFZBS4Wu6o1H9jeXtmV30IDv9m8zaIQPgbD7XAPfPGYhbIKOihw/i6Orq?= =?utf-8?q?6rniJeK93sLAfiKvJeLHgyWbWD/WNGEppf9i09lHJPYY0g6hzEIGJxfL10EnnitvX?= =?utf-8?q?tUMMnwvHo+o4FQ7aM2o8BiT9YnVO7VVQmaIHXowFeik4KXPXPXeqUf3wBnkTY2olq?= =?utf-8?q?hyICgKbzuk9+miCJdfO5EdW1XEATmQL9YyYE69xDdpHU4TATeTZGuo2dR7p5sWaSc?= =?utf-8?q?SpfNc3ypm2t3KDzSTm35WvpwtSswQQK4BZXs/Ci76/8I1ftqh/zmQP1F9QRMXc1Zw?= =?utf-8?q?o9c5hgujTptd5aA5pkZqJPl0rWUQm6kE9ICXEgbybTXfI4+rbRnoDd66l57jMyziy?= =?utf-8?q?LakXMpuHXCCwD0Mn+e3MRr0tddrQLi9kLpXwWfIAvRVJRhotzRTrE7tQSZ77DaUEd?= =?utf-8?q?z+LLxV9y/ifmA6CXt0Slwo4mel201FFJyiOQo3eaOGNWnoaPPigRJ2ARZ8joAE6gK?= =?utf-8?q?TSTCb0ThMT/yO1MTJWjp2lpWv8VaUbqCDFSeQjDwY0BNt4J4QkcECS9UDc8/pdkJ2?= =?utf-8?q?mhpgKbTIpfA53zmnZFRMLn5/hdK0E+RCW2gFNa0yygPMUtozuegpgX/15uvdEaBjl?= =?utf-8?q?Mr0OEEipQAfIxq0muDLeqNb7z/GBUQ1Obw59DAbpQ16UWmEnaKsZtVobFQRrFkMus?= =?utf-8?q?goM7p9/t3+Ymyga1Gn3y0LuZyrF8YZ6mfaXNTol7KSVomxfWAwU5CkLG19MZFev2X?= =?utf-8?q?RE9nYFITMKpr1z4E9DY/3+/azwZqwMtOG+zRBwtnMbm77wDJ5CZa69H32/5JsFUJ/?= =?utf-8?q?Z0BU0T82toiZM3tLaQ97LGARDTi4QfWD2hOJ7PVfH6AhuyJoVVbYBbNShdD2CjI1b?= =?utf-8?q?0cqiNlcR9wDHCyn8XXRoKxkNVi9Qu6lDYpfJIxGcgYsIsMYaYZ2CDI1wwKQpuTOVh?= =?utf-8?q?GwHAgZYxFUENDpjRzYhk3lzZlr2/oP3Nt3ukep2b4UWP/ZIxH59n+vjeNP6XxIvk2?= =?utf-8?q?XlOVB2cZGOJmnDcDegBGCPb4SGKH4rp0GHOLlCxjvoixR84bSx0Dy0sIE+p3jyzmS?= =?utf-8?q?95w2cEyR65yWsLaQTPkwgJTt9GD1fai7NxSaExTVr1SBFb+a1h66p2L8rbKEqDW23?= =?utf-8?q?h5er1fJEFW5Hyofs8SLqaQHQwlTPB50HtTOI2JQj31dbYTRSgKqsZWI=3D?= Content-ID: MIME-Version: 1.0 X-OriginatorOrg: eviden.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM8PR07MB7602.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: e06d7e04-1fdd-4b26-eaa0-08dc83925aaf X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Jun 2024 05:59:38.9442 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7d1c7785-2d8a-437d-b842-1ed5d8fbe00a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ih9MbTU4qJE65odQ34mPriZjYMS9g7koDaf+vKHyQUF29FRyhg2lwGjC3UKkdtigQZ4kUjup/Muu13xqMwJlMETnKODHsjy9FQIoSnb2H6/f7JHU6R8H4hcicCJ16Ctn X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR07MB6238 Received-SPF: pass client-ip=80.78.11.85; envelope-from=clement.mathieu--drif@eviden.com; helo=smarthost4.eviden.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, T_SPF_HELO_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Clément Mathieu--Drif This series belongs to a list of series that add SVM support for VT-d. As a starting point, we use the series called 'intel_iommu: Enable stage-1 translation' (rfc2) by Zhenzhong Duan and Yi Liu. Here we focus on the implementation of ATS support in the IOMMU and on a PCI-level API for ATS to be used by virtual devices. This work is based on the VT-d specification version 4.1 (March 2023). Here is a link to a GitHub repository where you can find the following elements : - Qemu with all the patches for SVM - ATS - PRI - Device IOTLB invalidations - Requests with already translated addresses - A demo device - A simple driver for the demo device - A userspace program (for testing and demonstration purposes) https://github.com/BullSequana/Qemu-in-guest-SVM-demo v2 - handle huge pages better by detecting the page table level at which the translation errors occur - Changes after review by ZhenZhong Duan : - Set the access bit after checking permissions - helper for PASID and ATS : make the commit message more accurate ('present' replaced with 'enabled') - pcie_pasid_init: add PCI_PASID_CAP_WIDTH_SHIFT and use it instead of PCI_EXT_CAP_PASID_SIZEOF for shifting the pasid width when preparing the capability register - pci: do not check pci_bus_bypass_iommu after calling pci_device_get_iommu_bus_devfn - do not alter formatting of IOMMUTLBEntry declaration - vtd_iova_fl_check_canonical : directly use s->aw_bits instead of aw for the sake of clarity v3 - rebase on new version of Zhenzhong's flts implementation - fix the atc lookup operation (check the mask before returning an entry) - add a unit test for the ATC - store a user pointer in the iommu notifiers to simplify the implementation of svm devices Changes after review by Zhenzhong : - store the input pasid instead of rid2pasid when returning an entry after a translation - split the ATC implementation and its unit tests v4 Changes after internal review - Fix the nowrite optimization, an ATS translation without the nowrite flag should not fail when the write permission is not set v5 Changes after review by Philippe : - change the type of 'level' to unsigned in vtd_lookup_iotlb Clément Mathieu--Drif (22): intel_iommu: fix FRCD construction macro. intel_iommu: make types match intel_iommu: return page walk level even when the translation fails intel_iommu: do not consider wait_desc as an invalid descriptor memory: add permissions in IOMMUAccessFlags pcie: add helper to declare PASID capability for a pcie device pcie: helper functions to check if PASID and ATS are enabled intel_iommu: declare supported PASID size pci: cache the bus mastering status in the device pci: add IOMMU operations to get address spaces and memory regions with PASID memory: store user data pointer in the IOMMU notifiers pci: add a pci-level initialization function for iommu notifiers intel_iommu: implement the get_address_space_pasid iommu operation intel_iommu: implement the get_memory_region_pasid iommu operation memory: Allow to store the PASID in IOMMUTLBEntry intel_iommu: fill the PASID field when creating an instance of IOMMUTLBEntry atc: generic ATC that can be used by PCIe devices that support SVM atc: add unit tests memory: add an API for ATS support pci: add a pci-level API for ATS intel_iommu: set the address mask even when a translation fails intel_iommu: add support for ATS hw/i386/intel_iommu.c | 142 +++++- hw/i386/intel_iommu_internal.h | 6 +- hw/pci/pci.c | 127 +++++- hw/pci/pcie.c | 42 ++ include/exec/memory.h | 51 ++- include/hw/i386/intel_iommu.h | 2 +- include/hw/pci/pci.h | 101 +++++ include/hw/pci/pci_device.h | 1 + include/hw/pci/pcie.h | 9 +- include/hw/pci/pcie_regs.h | 3 + include/standard-headers/linux/pci_regs.h | 1 + system/memory.c | 20 + tests/unit/meson.build | 1 + tests/unit/test-atc.c | 527 ++++++++++++++++++++++ util/atc.c | 211 +++++++++ util/atc.h | 117 +++++ util/meson.build | 1 + 17 files changed, 1330 insertions(+), 32 deletions(-) create mode 100644 tests/unit/test-atc.c create mode 100644 util/atc.c create mode 100644 util/atc.h