From patchwork Tue Feb 20 07:29:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankit Agrawal X-Patchwork-Id: 13563530 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02A80C48BC4 for ; Tue, 20 Feb 2024 07:31:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D61B8D0003; Tue, 20 Feb 2024 02:31:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8860F8D0001; Tue, 20 Feb 2024 02:31:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 727638D0003; Tue, 20 Feb 2024 02:31:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 63B278D0001 for ; Tue, 20 Feb 2024 02:31:12 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 3530D8058B for ; Tue, 20 Feb 2024 07:31:12 +0000 (UTC) X-FDA: 81811361184.17.3C928B7 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2059.outbound.protection.outlook.com [40.107.94.59]) by imf26.hostedemail.com (Postfix) with ESMTP id 6B996140003 for ; Tue, 20 Feb 2024 07:31:09 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="iQ1Ht/1l"; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf26.hostedemail.com: domain of ankita@nvidia.com designates 40.107.94.59 as permitted sender) smtp.mailfrom=ankita@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708414269; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xnAqZkcNokEttrDLhF6HxAeD/JKRA/iQvz7b26LJdsc=; b=0iIovpsuZ6EdzDKzDI5Or7hJ3/VxW1hRXygud0XaZTy05sM3jOdzfGm1JLXkPCXEB5kVOA pqmB5bEzIHaMBscV4adiaam08UplRcbR1c27ZQ0NWqM+AXBcc0J41oCqw5Ds/+m8fo2yIx 2pyC0tE3zVX8xZDTEU66VocGpek/+Jc= ARC-Authentication-Results: i=2; imf26.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="iQ1Ht/1l"; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf26.hostedemail.com: domain of ankita@nvidia.com designates 40.107.94.59 as permitted sender) smtp.mailfrom=ankita@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1708414269; a=rsa-sha256; cv=pass; b=ezClkiH9UUUideanQxydUEYfRiFT+SvbuTsHbzg75SXLjPIUahaLFwPN424e2UOW3qvCJu S1DB9ghGM4BPxp3iBSiYNEYMDahpYjjWKiCcx3btEVeHTuHXk5xQnNgP4dbjLJ36+h2iuQ izSr4FaqTU8/N1vvk5+uoWmK6r0Onq4= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Hl3lYAmmFABsVsUk1SDzYUkVtZLoj1GgpwcxQOjAPyzsHV0uurKn4hYPQvhSuY7tV4HvE7yIrdx2qerPizygWuTuvj1lXZOkyv3qaThSUE0aNuEsfTz07SEfjLKZ3rTmXkuawA3+aneKswTEdDZKmZ3XqrGRZyoq2PSoQMiQSrbYxctBd2IvlGIbkzqQ2HzxT6iDIpTFLYRqa/zcPbVZc+ba67tgvuIhadXALYMim+QgTxv65+DGUR/JG7sBcy6ulPeBVeaAWJcyJz7LfnKRGGaUGJe05lMk/2qghuDKSdFcmDDY9dzmR4V8BSAuJ5oLBDqAVFLNuB0TOXFhpOFs9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xnAqZkcNokEttrDLhF6HxAeD/JKRA/iQvz7b26LJdsc=; b=M39CV30MF/HSpEwVRamVqWQFdwS20GHXoJSaAM2p0Zdcr/uPgZDQ/E8+9RHFZnM2TBz7Xrs/lJXKzwR79AZYNeHS/dqnwvN+ivOjtnetM0Kgf9D0IoxxB08b7+BMCwGKeCwnibu3RyIBgb5RJgLbZ3hRDwAMUaycfeBLJEUXQveMm9tzePqRVc6Uy24/OO0AFL7Qz6+c78nZaKgspSXG05nOLl8ljWy9+vvnC+1j1/fIfHORzD8tzHXRU/y4csk9yBTwN20A1P3pkSpGMgUsY7wBXdFmLfpi+lFHbjJfLr0a1Exg11/D0pnQo70lgSUYZxOgrf1W0AaH3s/RM9KbOA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xnAqZkcNokEttrDLhF6HxAeD/JKRA/iQvz7b26LJdsc=; b=iQ1Ht/1lVZO8OZto6mOfBzOWGopZqaI5KTESNGP3TecPcSBOFNfBokJW6Xkd3SI0XMOOhAd3FBWMazX72IRXe4edWZOT1bdVk2Pa17LhNB3b/VZyA0RLtFNIRANprPwdfvXYETSskBEqfkFoeitg2AkE+lR48cwDldLA9uGE65wwYDuT9ALIDEjkCIcOG38Y2D4EVnfijMrecyxd09yS58Ck+JSaYIAza9GruN1+sWXZloMTFMuFT6HLRCXhlMOGqtKj2MeyVQGvgG1OhxcrXJfncTDVrmMvyqUG3hjntrNile3EGnjM+l0zj5WamvrDwKEINHYePAW54si3+OYHHQ== Received: from CYXPR02CA0078.namprd02.prod.outlook.com (2603:10b6:930:ce::16) by CO6PR12MB5490.namprd12.prod.outlook.com (2603:10b6:303:13d::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.20; Tue, 20 Feb 2024 07:31:06 +0000 Received: from CY4PEPF0000FCC2.namprd03.prod.outlook.com (2603:10b6:930:ce:cafe::c4) by CYXPR02CA0078.outlook.office365.com (2603:10b6:930:ce::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.39 via Frontend Transport; Tue, 20 Feb 2024 07:31:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by CY4PEPF0000FCC2.mail.protection.outlook.com (10.167.242.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25 via Frontend Transport; Tue, 20 Feb 2024 07:31:05 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Mon, 19 Feb 2024 23:30:37 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.12; Mon, 19 Feb 2024 23:30:37 -0800 Received: from sgarnayak-dt.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.12 via Frontend Transport; Mon, 19 Feb 2024 23:30:24 -0800 From: To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , CC: , , , , , , , , , , , , , , Subject: [PATCH v8 4/4] vfio: convey kvm that the vfio-pci device is wc safe Date: Tue, 20 Feb 2024 12:59:26 +0530 Message-ID: <20240220072926.6466-5-ankita@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240220072926.6466-1-ankita@nvidia.com> References: <20240220072926.6466-1-ankita@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCC2:EE_|CO6PR12MB5490:EE_ X-MS-Office365-Filtering-Correlation-Id: ca6f1e37-2f67-42b9-2cc3-08dc31e5e61d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZPxQ8GbEkSvLOBerMzuGU69zQzy7PyYzNBC7d6g2WxpR9tao6hI77HbWfD2eEuu2k3TWrJ4HWzvHb9S4OLYHTJBUq2N/+FhdP7+bAnGZ5V0prD65XoFnqDMO1fCa5DmV2qZWBhDhU1aEY9z++Cu5+8J0hbE4J9+tjzzN6ghurQP/NMqCDU/ed9QR+vFckQAhnAfH5VAM+tTy464XkWX+UCKLlotcT+dZpMpeRh8oLe29A83EpMjj82LEPPiQ6dtJIWonyofQ48pcls5BR7Pomap/yf+ZsLSS3h2BPknECLU0iCG17AVE5rCNRUQmVwoDL9b71TN23kVdXE2Jq/YhCylFK7ztV8M+t3dadu7+unyYP2c5ORQ/r9jHoKA7rEM59LzCTSQAFytXelNOrFvpFOR6piyBmYyasXCWL74Z17o/7AIWXj0Pv/L7Ul1Hj7ZySgup/eulIMJ2tiNqVngTu0jS1bRTe7SVDTPCB8I7rHIP36i3hpUTZEUmsExgVfwaENjx+QHqLgBVL9Zz7Jn4GSi6RyMNKadKoFZNDHtKCp69oRvsrJEBrS8DWlVXoc87uux9Pb0Uu/SaACnnhS1yjF48g5m2rMjFQjYWONvmoQ327VGzZwqxMsyoozH6qKMdY1kbA4qrKSOYKag6blLUvGIYNoTLjn4k1TpCFXfy7tg= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230031)(36860700004)(46966006)(40470700004)(921011);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Feb 2024 07:31:05.5367 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ca6f1e37-2f67-42b9-2cc3-08dc31e5e61d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000FCC2.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5490 X-Rspamd-Queue-Id: 6B996140003 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 67mfcjoa9pgx4f7t6ej1gtakxmmgwk1m X-HE-Tag: 1708414269-356529 X-HE-Meta: U2FsdGVkX1/rMP07HWV07AMwwhSoFgoUCZpQyyeKBJVxBMD1UjxG+sh2qZgGDE4lE+zy4yiqt86I9UShpABRuFvB5Mj5L6sBUsec1k3d4Yp0qNTtn2XuxrZjX7gsSDjSOE4QY+WeDQZw2MGWc4HKs03HPiRSs2mnEtao12cgn7ZcTgR5iNuUJWLoNw/rK6YHOOmUCv7R/11Y/f98SADwQmL3hFAngZYflX5Ll17wX6CRHnlj/Or5BELKVPPy+4MvavK10Qw8SaUseJXCShBWaYnQ89EKfEMQ0n6CZuOjV2ovnprEqxfT6bTd5t8TXdJJ+xBFcRhrNtY5Ti3JVyzFWsu9IPwRYN/9lItaYVZJyHP0lqm2k58oLo3wdDGdTvbK+xIxC8RG9jG10/9oZMp2FoxaObo1J1yev4leoZ1p4HNpRSI7pOaezmYm+EdXE6HP2raMZYtj4xGVnkGaq2On1QEeH/mbvOC5X4su1CXW4oC1OO0imVAX6a87+DF40Ia+XYqc3AaxHGZJa1MAJIIIaILNh0wfVWW1mQ0n/pkE2G+bk3uvy4ix3kg2j5orR5ln5mg7hdiwqCKyoHmy7mp1bo5/pNxhiM1sW0P2P0OfeT5Xb6g2EbTdWEmK9GM01EokMh4A5B5NZyEO2I4JotODgRriw3xMreGOgInYrkTztLrAAkZ4nI/vMVnORISklm3yLhbCvpo9rFHex4map8a9HpBHEcmJXoIQkdCrKg8DsDTu8tWazo1SsFGq1sDJmnkHiuHF6fBbMEe2KIlG/kx5v7BjIKN3p/fGzE7n/8+mmjozu34WdgXFi0R6W9a0zVeBvcIZD8nTvdyBdHY0fRKwj+CYgaaGrOPRxk+gmmmcePB5M6dyctmzmkLp8edsV9LVZomwZ/LKGDQTHP5GeMtULw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Ankit Agrawal The VM_ALLOW_ANY_UNCACHED flag is implemented for ARM64, allowing KVM stage 2 device mapping attributes to use Normal-NC rather than DEVICE_nGnRE, which allows guest mappings supporting combining attributes (WC). ARM does not architecturally guarantee this is safe, and indeed some MMIO regions like the GICv2 VCPU interface can trigger uncontained faults if Normal-NC is used. To safely use VFIO in KVM the platform must guarantee full safety in the guest where no action taken against a MMIO mapping can trigger an uncontained failure. We belive that most VFIO PCI platforms support this for both mapping types, at least in common flows, based on some expectations of how PCI IP is integrated. So make vfio-pci set the VM_ALLOW_ANY_UNCACHED flag. Suggested-by: Catalin Marinas Acked-by: Jason Gunthorpe Acked-by: Catalin Marinas Reviewed-by: David Hildenbrand Signed-off-by: Ankit Agrawal Acked-by: Alex Williamson --- drivers/vfio/pci/vfio_pci_core.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index 1cbc990d42e0..c93bea18fc4b 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1862,8 +1862,24 @@ int vfio_pci_core_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma /* * See remap_pfn_range(), called from vfio_pci_fault() but we can't * change vm_flags within the fault handler. Set them now. + * + * VM_ALLOW_ANY_UNCACHED: The VMA flag is implemented for ARM64, + * allowing KVM stage 2 device mapping attributes to use Normal-NC + * rather than DEVICE_nGnRE, which allows guest mappings + * supporting combining attributes (WC). ARM does not + * architecturally guarantee this is safe, and indeed some MMIO + * regions like the GICv2 VCPU interface can trigger uncontained + * faults if Normal-NC is used. + * + * To safely use VFIO in KVM the platform must guarantee full + * safety in the guest where no action taken against a MMIO + * mapping can trigger an uncontained failure. We belive that + * most VFIO PCI platforms support this for both mapping types, + * at least in common flows, based on some expectations of how + * PCI IP is integrated. So set VM_ALLOW_ANY_UNCACHED in VMA flags. */ - vm_flags_set(vma, VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP); + vm_flags_set(vma, VM_ALLOW_ANY_UNCACHED | VM_IO | VM_PFNMAP | + VM_DONTEXPAND | VM_DONTDUMP); vma->vm_ops = &vfio_pci_mmap_ops; return 0;