From patchwork Thu Sep 7 18:14:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankit Agrawal X-Patchwork-Id: 13376777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 91CCDEC8726 for ; Thu, 7 Sep 2023 18:15:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:Subject:CC :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=pdGfiBxWGBKXwU/DKHJsG6W6jOatmTQWoKNo00QP6Kg=; b=cZn4JHD0QUn4n7 eghM5105IhcKLWDfsnNaPbAN1VaM3uNKXnLrQRYS0UkbbtjRzhEwZ4wJ73j9K/q8oTGtJjK8MM+Pw PP6d6ObFZQRC9kQzEfmBWU2nfJb6roEP3ZEY8NVr/tw27pjJgtP7m5mu3jUD4iuR7mf8YDZvtZjCX lmjLR1qC1/7/dZ0OKdk+PuPS94dOekcXqOHHULDPPOlMsqmpAZ0Zj2A3Sj/WvNCxVns1LfEiInJmW rYnoXrM7zWojZpkuJkZV/mcQWGWacZ5EfkkQCkC6eGUoQs2Ib/PtY8or2z2JGWZkkJG5QGJ43Nu0J UA1/lourm5H+L8acwmIA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qeJXC-00CXGE-0k; Thu, 07 Sep 2023 18:15:18 +0000 Received: from mail-dm6nam12on20608.outbound.protection.outlook.com ([2a01:111:f400:fe59::608] helo=NAM12-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qeJX8-00CXEd-2f for linux-arm-kernel@lists.infradead.org; Thu, 07 Sep 2023 18:15:16 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F+LON2yt7J5Kuu5tt/97XDV94NGOjSwagFThwBHAYUYk+7kx6773bXL+gJ4Xss9Gt7Wq/+am6vQip4/RcBzyPXqGSOj1aKamF/JER0+E7lf0J8ppcj/ntxEM2zITCaL+M3hdTFtiPzJIcQwHpM6fc/2o3CrLzPXSoHI6xa2NR+NCxZTE+nlpV2T9ePeFw8YGhWcwRTRv1fJZ9gAcwFOwYf0auTWX1GIFPZw33qAFlhMuBTSXWS1NOpTw6Jd/ocWEUjiadqU+WUZPzsUyOUpPY0qfud34UjDVGb69hBKOj19JVodT6c7Q598f0/Y/l39s2EeX7CVNBhHjItynTT3Dqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=e7guXSnVgzXmBLZaK/Htm+O00j14TMVn7RHlHieV1do=; b=L6KjqXgsqkLm/JEqaz3KSo7TlLbuoAnW97NOm1YZCAcTAbgOD3fcTJMSXEfMEcEycA0//BOuwSrnsh37NaIlTfb34YMQfP1lp7Sm0xCKDholIJP5I2VH+eFjpa7ymKFr7rc6P3X/k55j8HX3Wb6zTozbA4cu75hzlKHjMvoEQJNGVC0P3gOeIkxJdn8HHIb8GjOYmMsVpBB1+9nTO+0FPGY0gx0eNqCzp1k2fddvPaz+YcsM+SfaLq3Cle89Ptia2AjqARBGjUPJL3D7xA8MCES3U2E3GWkvlXJ3bgko6gnQg8E2N1NLRwVRzRPlw3ZJ5tFlGKWW7xh+LFjEfhynSQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=lists.infradead.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=e7guXSnVgzXmBLZaK/Htm+O00j14TMVn7RHlHieV1do=; b=q9YCgBFCWpA+gMsSjwANoGiSaR81zc1YRg5BrmKQoaUGpEbsrDW6f9t0PNUTAYYz0sElIIEBp89Pkv7XIu5WgmKRDvkSsD7X9LqqNlFs6G/9Zr8k2f1st41KfE8yhwN8C/HLK8IwD+55ZB3hmiwnm5b3aK+e2lB4wnpkUdaspZKyGXyV5b1Rzw+VUVCCjsvciyMU+zYjZNFDneCh1e8yhGEc1kLpB/9Ouy3J/DVTCXheSbqEk6YsMcakyeVMGK3RIVtMq6niDoZO6ENUW74V0eQyPKju6o7Tl7Vg+1GJJkNJ+7e3HJ+J0bne/V9xvHD7rM/7H4OcdSWV3muShb+NeQ== Received: from DM6PR08CA0040.namprd08.prod.outlook.com (2603:10b6:5:1e0::14) by CY5PR12MB6275.namprd12.prod.outlook.com (2603:10b6:930:20::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.36; Thu, 7 Sep 2023 18:15:09 +0000 Received: from CY4PEPF0000FCC5.namprd03.prod.outlook.com (2603:10b6:5:1e0:cafe::c3) by DM6PR08CA0040.outlook.office365.com (2603:10b6:5:1e0::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6768.30 via Frontend Transport; Thu, 7 Sep 2023 18:15:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CY4PEPF0000FCC5.mail.protection.outlook.com (10.167.242.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6768.25 via Frontend Transport; Thu, 7 Sep 2023 18:15:08 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Thu, 7 Sep 2023 11:15:00 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Thu, 7 Sep 2023 11:14:59 -0700 Received: from ankita-dt2.nvidia.com (10.127.8.14) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37 via Frontend Transport; Thu, 7 Sep 2023 11:14:59 -0700 From: To: , , , , , CC: , , , , , , , , , , , Subject: [PATCH v1 0/2] KVM: arm64: support write combining and cachable IO memory in VMs Date: Thu, 7 Sep 2023 11:14:57 -0700 Message-ID: <20230907181459.18145-1-ankita@nvidia.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCC5:EE_|CY5PR12MB6275:EE_ X-MS-Office365-Filtering-Correlation-Id: 1888e66f-57b2-4968-36ad-08dbafce5e94 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YDhCUTRNHKGkajrk+hQ4gbi+WI0L5/QDIhVbxM/I6omfpvF1HQcSF2Qb86PAb1VBV0qKwBZqiGjMCGA9HM4iMef3ifBg+ds7w7AsSvP2X3zBuhMCBBW2reA+Ufuhlb0jNOJB07o56gAWVt1KuNQaOIrXQXqp8NsKbIEcgGJSbmh6E5GBhimhEDsKGHUnEn/1dbl7uWsLMo3bS/NmVqwac0Tox+pPfctF0XbxnQIJdo0Y5i26y5IUxrBRQP9+HjxI11NLGFvtSMVC4/tQp20/FE/ETRHTIw2HeynNHz9ImehvufX6NNW/wSPlDOq4Gs+dNDbV5lrPM76GLaP82j4QPcK2ZUzdjWNF1aA7sa0zsaHLvXuo7HNHVyP5QqyppRtp93EMMVLksx+6cSm2tqK99qsEJIEnHL6Y4APMkbRPvB11daQQ/mph/v1HKCywC6bu7tbikkOWN/Hm1bquFib0vkdaEce9fTCQBqTTvK2gc+Iv0wh1nrByK3ErCi127NPI6o/86IV/asY7iC2+63367hBhtfzUlwYTQWLA0y8/+03lQxCHgwfS9GfTKk+JGkJWcmUfgNHRLIhcmr0U4opKzbfisLX0HeCUg74Yf/biweydwVzx998WibEe+4LXw/VO0fBT/x7mjcWKDQCvT4ubSS1orv611lteWh6WLxkXtn2SeLutmwCUSq0WcFu+OxuwMQ6Xu90OZpsVl3FkLaC4SoBh7sT9RqQhUBCHOBKlm1ilwEZvovgybFeKm8BdRQj+ X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(396003)(376002)(346002)(136003)(82310400011)(451199024)(186009)(1800799009)(46966006)(36840700001)(40470700004)(7696005)(478600001)(83380400001)(2906002)(1076003)(2616005)(26005)(336012)(426003)(110136005)(2876002)(41300700001)(316002)(54906003)(70586007)(4326008)(70206006)(5660300002)(8676002)(8936002)(40460700003)(82740400003)(40480700001)(47076005)(36860700001)(36756003)(86362001)(356005)(7636003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2023 18:15:08.6546 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1888e66f-57b2-4968-36ad-08dbafce5e94 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000FCC5.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6275 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230907_111514_927948_C8B32A12 X-CRM114-Status: GOOD ( 15.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ankit Agrawal Today KVM forces all of the memory to either NORMAL or DEVICE_nGnRE largely based on pfn_is_map_memory() and ignores the per-VMA flags that indicate the memory attributes. This means there is no way for a VM to get NORMAL_NC IO memory (what Linux calls WC memory, which is used for performance in certain NIC and GPU devices). There is also no way for a VM to get cachable IO memory (like from a CXL or pre-CXL device). In both cases the memory will be forced to be DEVICE_nGnRE and the VM's memory attributes will be ignored. After some discussions with ARM and CPU architects we reached the conclusion there was no need for KVM to prevent the VM from selecting between DEVICE_* and NORMAL_NC for IO memory in VMs. There was a fear that NORMAL_NC could result in uncontained failures, but upon deeper analysis it turns out there are already possible cases for uncontained failures with DEVICE types too. Ultimately the platform must be implemented in a way that ensures that all DEVICE_* and NORMAL_NC accesses have no uncontained failures. Fortunately real platforms do tend to implement this. This is similar to x86 where various BIOS choices can result in uncontained failures (machine checks in x86 speak) on IO memory. On either architecture there is no way for KVM to know if the underlying platform is fully safe or not. This series expands the assumption that any uncachable IO will not have any uncontained failures for DEVICE_* and NORMAL_NC and that cachable IO will not have uncontained failures for NORMAL. We hope ARM will publish information helping platform designers follow these guidelines. Additionally have KVM drive the decision on cachable or not entirely on the VMA. If the VMA is cachable then the KVM memory is made NORMAL regardless of pfn_is_map_memory(). This closes a hole where cachable VMA mappings in a process could be accessed via an uncached alias through KVM. Applied over next-20230906 Ankit Agrawal (2): KVM: arm64: determine memory type from VMA KVM: arm64: allow the VM to select DEVICE_* and NORMAL_NC for IO memory arch/arm64/include/asm/kvm_pgtable.h | 8 ++++++ arch/arm64/include/asm/memory.h | 2 ++ arch/arm64/kvm/hyp/pgtable.c | 4 +-- arch/arm64/kvm/mmu.c | 40 +++++++++++++++++++++++++--- 4 files changed, 48 insertions(+), 6 deletions(-)