From patchwork Wed Jun 22 00:49:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kechen Lu X-Patchwork-Id: 12890000 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C49ACC433EF for ; Wed, 22 Jun 2022 00:55:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1355446AbiFVAzA (ORCPT ); Tue, 21 Jun 2022 20:55:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52772 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355474AbiFVAy4 (ORCPT ); Tue, 21 Jun 2022 20:54:56 -0400 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2057.outbound.protection.outlook.com [40.107.223.57]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF01031350; Tue, 21 Jun 2022 17:54:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dw8fyVlon9qJopNjKAgtc4NrqgRF5tdu/WwEWHs57gdYzBzBllJYXSrDlr92KJG0DXNDOP8EEXq+8Ozlvq82VwGffHE1pwpeTBxS48C79rraYArsh7qaaDsG5vqZZz1Lr5d+OjozpdC59ynAD3qiatPF+/JNQ2TZOkx1Pwadt3QFkWkxcYYAro2wmMv70IQR8scHnoGieNqjQRF2VzBIhiHUhzPjVZKGeGA70xGciGz6UApmTJs15eEV/baOhoHs3WYrksFNTvLyf76D5a5bop8lYyrOKNztxiXnwb5OLvtrdTQT4JCapghUai3GoHkw3nT20417ecaERpxHPohoiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/4qcnsEzsReGXlF0v/p/wJHUhIkauYOL3i4UPcyj+9s=; b=bB8GiqCrg3wd3UP1Kj9kIgXJj052MhHG8iMKSQRAepaoFp4mYO7KyTN/rP+weZyfqWzxPkq/Tsy0Bv0QYfaY1KB32koJZx/1aLNbcBPWJCZEhfZq4JfjDC48w3l3hWv7QMyOX1rKW8QwnLqO8BaK/6X8TxCqirs6eFmcuJ7HfAV7S8cvzqDnfz9EzC6fyD5kRW9sZxONNCrgKAc3agZXTgiMLwF80Us6NZpkkZyV6sWCgMCNDuR99w1T+OT0+T5A0RHQD6bb8RVkBHyEy5HKxuy2I3YJ3eitvkcbUfBbKplpUndwGfVrnUqB7MhyteaQOhOoYPNy76SYbtkiw34w2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/4qcnsEzsReGXlF0v/p/wJHUhIkauYOL3i4UPcyj+9s=; b=awyiPLF7OymjBX/YmrLk3KZhix279bCBP04WrdPqZS9w6Ip0Q0K0zR/icAi9TyGHVHEMFkYrdHh+8BPDwLxSsWqqKjm/IM/kF/1kPz20Qw+fAYJ0zjB17nsfXbCEtxUGp4lGRvj9pNDRKuZKbKP8EKHkGEoiRPcfBoDDKZuT9aRqChcVyqA/g5nNTVGdBGwtMZq6a9NUvGNdXUTRBYzu4hgsaJIFSnb1EheKwbPi6zKsswSkjGjeeGPm7eIWtId8IKpANJr0tbSb7a+djAP5sr0bHO/+RCub9aSxS3ftpCoe60JjJN819PBdBDZMh6mF80LxDnKw4scrC90i6/n9NQ== Received: from MWHPR08CA0037.namprd08.prod.outlook.com (2603:10b6:300:c0::11) by BL1PR12MB5237.namprd12.prod.outlook.com (2603:10b6:208:30b::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.14; Wed, 22 Jun 2022 00:54:52 +0000 Received: from CO1NAM11FT006.eop-nam11.prod.protection.outlook.com (2603:10b6:300:c0:cafe::50) by MWHPR08CA0037.outlook.office365.com (2603:10b6:300:c0::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.14 via Frontend Transport; Wed, 22 Jun 2022 00:54:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.238) by CO1NAM11FT006.mail.protection.outlook.com (10.13.174.246) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5353.14 via Frontend Transport; Wed, 22 Jun 2022 00:54:51 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Wed, 22 Jun 2022 00:54:51 +0000 Received: from foundations-user-AS-2114GT-DNR-C1-NC24B.nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Tue, 21 Jun 2022 17:54:50 -0700 From: Kechen Lu To: , CC: , , , , , , Subject: [RFC PATCH v4 1/7] KVM: x86: only allow exits disable before vCPUs created Date: Tue, 21 Jun 2022 17:49:18 -0700 Message-ID: <20220622004924.155191-2-kechenl@nvidia.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220622004924.155191-1-kechenl@nvidia.com> References: <20220622004924.155191-1-kechenl@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9f9d780f-baed-4a95-8fde-08da53e9d0a1 X-MS-TrafficTypeDiagnostic: BL1PR12MB5237:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cBOily90FaONk07azMdktj8MTPT5Ermf2pUAx/gLRkJd8/bccl4C2AXCMS3R52bpGd+jKEygzWcmOMylkEaCsm5HdSV2UNO1uKe2GbOP4XBbemHuTC09nyKDlC0jVy/Nnf7XTSE7dWGGhZlDeu+fODPl5IIe4CR4bbjiVRBuhZeQekE46Zna7zCkeAshFrnAt2OeyWwLukE4RBhifGmpRHI/cIH5CxyX4hp+5bj4r8tnnc7GUNnI6oFbd3orDCAns8wxfGIEYa+/2TvssZVUmD8BKDprZTS26qVDtDWE1WkMgmtVOm+A1JS+lMuq1Wyq7WzrHajxYI2ymORU+Non9I71MQiA7g2IN42jZyew5YWppvJO3V+96lJo0sY8HuB/63yV76D3MxwKlNaJfUOiKvOEbLDorPMr7fei0JNm4nu2acEcfiIDZGmG+UwZPm+PJ14zhCc/qH0e0kc2S5vCn2aMTuxY9Ne/lpJogMvROdZS6/FfP+LAxKMi+nN5DKzVa2i9iuGzSXR9ZQb9Z8VnutqXIRUWIeWPH8sboH/Cr0P10wYq9OeXJesSz9WVC7wSuaKNqfHArLv2LmPpS4AiyFifED/Mq57D2qv7xDFO/CPn80OPeDS8+3SngdDLeDg/W5nai5eLYJuDwLUXMYZuo04J1X0xyTaIJO3JcDmfqGqFHyU7UBr97c0LFRSG7IZvkwSe0GawBY9LZOoipiYWF7J33Kw9OS0cQWpWqalHEdlY2WIPcemZ4au8YXNprDtF X-Forefront-Antispam-Report: CIP:12.22.5.238;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230016)(4636009)(136003)(346002)(39860400002)(396003)(376002)(46966006)(40470700004)(36840700001)(4326008)(8936002)(41300700001)(26005)(6666004)(110136005)(82740400003)(54906003)(2906002)(40480700001)(82310400005)(86362001)(36860700001)(356005)(36756003)(47076005)(70586007)(5660300002)(1076003)(336012)(426003)(40460700003)(2616005)(478600001)(186003)(70206006)(81166007)(83380400001)(316002)(7696005)(8676002)(16526019)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jun 2022 00:54:51.7831 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9f9d780f-baed-4a95-8fde-08da53e9d0a1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.238];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT006.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5237 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Sean Christopherson Since VMX and SVM both would never update the control bits if exits are disable after vCPUs are created, only allow setting exits disable flag before vCPU creation. Fixes: 4d5422cea3b6 ("KVM: X86: Provide a capability to disable MWAIT intercepts") Signed-off-by: Sean Christopherson Signed-off-by: Kechen Lu Cc: stable@vger.kernel.org --- Documentation/virt/kvm/api.rst | 1 + arch/x86/kvm/x86.c | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 11e00a46c610..d0d8749591a8 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6933,6 +6933,7 @@ branch to guests' 0x200 interrupt vector. :Architectures: x86 :Parameters: args[0] defines which exits are disabled :Returns: 0 on success, -EINVAL when args[0] contains invalid exits + or if any vCPU has already been created Valid bits in args[0] are:: diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 158b2e135efc..3ac6329e6d43 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -6006,6 +6006,10 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, if (cap->args[0] & ~KVM_X86_DISABLE_VALID_EXITS) break; + mutex_lock(&kvm->lock); + if (kvm->created_vcpus) + goto disable_exits_unlock; + if ((cap->args[0] & KVM_X86_DISABLE_EXITS_MWAIT) && kvm_can_mwait_in_guest()) kvm->arch.mwait_in_guest = true; @@ -6016,6 +6020,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, if (cap->args[0] & KVM_X86_DISABLE_EXITS_CSTATE) kvm->arch.cstate_in_guest = true; r = 0; +disable_exits_unlock: + mutex_unlock(&kvm->lock); break; case KVM_CAP_MSR_PLATFORM_INFO: kvm->arch.guest_can_read_msr_platform_info = cap->args[0]; From patchwork Wed Jun 22 00:49:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kechen Lu X-Patchwork-Id: 12890001 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78300CCA473 for ; Wed, 22 Jun 2022 00:55:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354781AbiFVAzC (ORCPT ); Tue, 21 Jun 2022 20:55:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355266AbiFVAy6 (ORCPT ); Tue, 21 Jun 2022 20:54:58 -0400 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2088.outbound.protection.outlook.com [40.107.94.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3EE2630F6C; Tue, 21 Jun 2022 17:54:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WwSqR1aY60VUeI5fnYNQBEmaJx6OaZSKM3FQNDIteYqMd3p2aepOC7pU+mpe0KIs3wqblkuM47SpcpMFx8UmdZMwy0nSSaLmvbJVip40ohuFP+v6Pjy3e3sjtdORojnVMprfZLcW5QnC+r/CFSly4Ogi9vGh5x/khhI8MTHpbIQed0lu2qD0W9H/Rop0Pgk/aI7ftHFYqu44+I8mvbl3muVjc8xGOzrLxkIRl76YZPPVJktxZ4vWxNqnegcrobc6fqofT371V+l/RPRP89WTBEPqAiB+Z/6n9NAF2JkyH0XU9FmHh/ob5/ONZwUZzrFoe8J30jVhQF4uXVUG2p6LIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MQFMSfQAbsEUSI3Jph/BToZ98JCiMeJ93p5rV04Dm5U=; b=QD9IQGtUaWUEDx6/7T+uFhWWcWs9HD8kDHD0YtZhZb3z3jfAv4c453Iwa6cob/sSu+rZSqb8sC0Ibm3RWc6+EwUJfx9OyCp5Uqr2ZzqjZG/yaBlmOcH85hR0hA1QKPRSDgVt0zw4IhQC4dr7QQrH6lvuA+8LrG7xAkD51q38Zre/HW76Z0ajTN3i+FquLOdDrJbww9cPkMsP4Zn+EIz5ZEH0LQVdedCaLnOxv1EvRuH2WEK4Wdd4uQDjDgdzk1BOtq1ckeVpURsJvGYbpiIh1kYR/aFOEsmqFgdgXMtL+YcoOFVODLSMtbbWOBNbOzunnvHUVxXCeXDbemUeGD5JiA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MQFMSfQAbsEUSI3Jph/BToZ98JCiMeJ93p5rV04Dm5U=; b=rZC3miiTw4Bl0wAKvRxdVXVzzAyVIqDxhECns68H7jRTG0bKGHainPhovn0bifMlRAh1OqSmJlf/KDzXARSjMupPViDc4Ia+cOHyTn8rboUUoRPa/dkmK/EUjIh8ZIN8bIRak5gmEbX6gABlpuLc79D4PSjaikt7zK5c5HTySmXmkh2mPpGUkkikWEUqAv9UQMhyF80HKGM36x9iQ7aKhtZyZdEsHloAvTUWy4YAA+oNOYfMDpNqGKdxJvNN7qehRBBemCrPmhtt10e75qK4aFZYelufd+nnilcYlOtXO8LhRqH4zEROiVS7FxiMzHsXy0k4O1AnQg6XqwaOlad6eQ== Received: from BN9P223CA0008.NAMP223.PROD.OUTLOOK.COM (2603:10b6:408:10b::13) by MWHPR12MB1360.namprd12.prod.outlook.com (2603:10b6:300:12::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.18; Wed, 22 Jun 2022 00:54:55 +0000 Received: from BN8NAM11FT061.eop-nam11.prod.protection.outlook.com (2603:10b6:408:10b:cafe::c7) by BN9P223CA0008.outlook.office365.com (2603:10b6:408:10b::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5373.15 via Frontend Transport; Wed, 22 Jun 2022 00:54:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.234) by BN8NAM11FT061.mail.protection.outlook.com (10.13.177.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5373.15 via Frontend Transport; Wed, 22 Jun 2022 00:54:54 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Wed, 22 Jun 2022 00:54:53 +0000 Received: from foundations-user-AS-2114GT-DNR-C1-NC24B.nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Tue, 21 Jun 2022 17:54:53 -0700 From: Kechen Lu To: , CC: , , , , , Subject: [RFC PATCH v4 2/7] KVM: x86: Move *_in_guest power management flags to vCPU scope Date: Tue, 21 Jun 2022 17:49:19 -0700 Message-ID: <20220622004924.155191-3-kechenl@nvidia.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220622004924.155191-1-kechenl@nvidia.com> References: <20220622004924.155191-1-kechenl@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1ca513bc-fd3a-4a2a-5820-08da53e9d256 X-MS-TrafficTypeDiagnostic: MWHPR12MB1360:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jPiRK8CZVS7Qh2GtDRE5dn3kczNKzFMktXQpUsJkIb3WOzKT3tllt6u43reqgYmpqEr1nr4AybR9lMfR91IjWAetSxzbMxh72rDoGBqudx6kmHr4vT8FO+O5WTqYmkNM+c8g83as6LkRw8rhif1Q1Ma3L6/JUWltHyh0M4J93ZYlcHRK4N0MkaDu0APoFNkLW/qhJKYA36a17lCsJ/o+BPCDFrs16dePEcRD23TG2Y9cNyMGcFUCVJdUzOkDO84aKIVr4Q+dU6OthPifoDDexwIhMYmBWwpQ9xZvBE3sza01Zb55/Jfbqn9o8tc9Blz+dL0ZHpY9DGRnM5Fh7m4R+77rqDwcu44ulnCbrmcQPUKKU2XZT55S9qzkjFNn7Ui54hUW7qVwmXhZvpWUNfxCz5OPpRHRP/V4Xo0rJRIvPJCHibDH5ro6f6V97j+aauvGzGNuZERKxQWKCDUutOC5regnd6/INkGiIclXIYYcjs7nvq8W/acs0aoa6zeQn/UIizbSNGjB6Cpfji6Vzi0kqLv4xumT9YPDOD5j9Yl976ToXcwh9K08RgrrpnqQy4jg/rxtweiO/eMM38elmVKUvtu5CKgP1daMEhSdWwgWdajpeX5lu6H8vv7rJ/6a3BeMl3WKerm/Y3awd30kMw1jfonIM3auDd4J14CB5irDs9omxWQSo7qUoQvv58jpXkLUqh3czPIsCFMqB/HrOBM9ExYEGtCp07bQI0M0UjpyX+Vas7IoFWGjOLuldYeKgyyH X-Forefront-Antispam-Report: CIP:12.22.5.234;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230016)(4636009)(376002)(396003)(346002)(136003)(39860400002)(40470700004)(46966006)(36840700001)(336012)(2616005)(186003)(16526019)(86362001)(81166007)(83380400001)(478600001)(47076005)(40480700001)(1076003)(41300700001)(426003)(82310400005)(36860700001)(40460700003)(26005)(70586007)(7696005)(82740400003)(4326008)(30864003)(8676002)(70206006)(8936002)(2906002)(316002)(110136005)(54906003)(36756003)(356005)(6666004)(5660300002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jun 2022 00:54:54.5362 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1ca513bc-fd3a-4a2a-5820-08da53e9d256 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.234];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT061.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1360 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Make the runtime disabled mwait/hlt/pause/cstate exits flags vCPU scope to allow finer-grained, per-vCPU control. The VM-scoped control is only allowed before vCPUs are created, thus preserving the existing behavior is a simple matter of snapshotting the flags at vCPU creation. Signed-off-by: Kechen Lu Suggested-by: Sean Christopherson Reviewed-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 5 +++++ arch/x86/kvm/cpuid.c | 4 ++-- arch/x86/kvm/lapic.c | 7 +++---- arch/x86/kvm/svm/nested.c | 4 ++-- arch/x86/kvm/svm/svm.c | 12 ++++++------ arch/x86/kvm/vmx/vmx.c | 16 ++++++++-------- arch/x86/kvm/x86.c | 6 +++++- arch/x86/kvm/x86.h | 16 ++++++++-------- 8 files changed, 39 insertions(+), 31 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 9217bd6cf0d1..573a39bf7a84 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -924,6 +924,11 @@ struct kvm_vcpu_arch { #if IS_ENABLED(CONFIG_HYPERV) hpa_t hv_root_tdp; #endif + + bool mwait_in_guest; + bool hlt_in_guest; + bool pause_in_guest; + bool cstate_in_guest; }; struct kvm_lpage_info { diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index de6d44e07e34..f013ff4f49c5 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -245,8 +245,8 @@ static void __kvm_update_cpuid_runtime(struct kvm_vcpu *vcpu, struct kvm_cpuid_e best->ebx = xstate_required_size(vcpu->arch.xcr0, true); best = __kvm_find_kvm_cpuid_features(vcpu, entries, nent); - if (kvm_hlt_in_guest(vcpu->kvm) && best && - (best->eax & (1 << KVM_FEATURE_PV_UNHALT))) + if (kvm_hlt_in_guest(vcpu) && + best && (best->eax & (1 << KVM_FEATURE_PV_UNHALT))) best->eax &= ~(1 << KVM_FEATURE_PV_UNHALT); if (!kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT)) { diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 0e68b4c937fc..9e29d658a8c2 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -147,14 +147,13 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) static bool kvm_can_post_timer_interrupt(struct kvm_vcpu *vcpu) { return pi_inject_timer && kvm_vcpu_apicv_active(vcpu) && - (kvm_mwait_in_guest(vcpu->kvm) || kvm_hlt_in_guest(vcpu->kvm)); + (kvm_mwait_in_guest(vcpu) || kvm_hlt_in_guest(vcpu)); } bool kvm_can_use_hv_timer(struct kvm_vcpu *vcpu) { - return kvm_x86_ops.set_hv_timer - && !(kvm_mwait_in_guest(vcpu->kvm) || - kvm_can_post_timer_interrupt(vcpu)); + return kvm_x86_ops.set_hv_timer && + !(kvm_mwait_in_guest(vcpu) || kvm_can_post_timer_interrupt(vcpu)); } EXPORT_SYMBOL_GPL(kvm_can_use_hv_timer); diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index ba7cd26f438f..f143ec757467 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -675,7 +675,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm) pause_count12 = svm->pause_filter_enabled ? svm->nested.ctl.pause_filter_count : 0; pause_thresh12 = svm->pause_threshold_enabled ? svm->nested.ctl.pause_filter_thresh : 0; - if (kvm_pause_in_guest(svm->vcpu.kvm)) { + if (kvm_pause_in_guest(&svm->vcpu)) { /* use guest values since host doesn't intercept PAUSE */ vmcb02->control.pause_filter_count = pause_count12; vmcb02->control.pause_filter_thresh = pause_thresh12; @@ -951,7 +951,7 @@ int nested_svm_vmexit(struct vcpu_svm *svm) vmcb12->control.event_inj = svm->nested.ctl.event_inj; vmcb12->control.event_inj_err = svm->nested.ctl.event_inj_err; - if (!kvm_pause_in_guest(vcpu->kvm)) { + if (!kvm_pause_in_guest(vcpu)) { vmcb01->control.pause_filter_count = vmcb02->control.pause_filter_count; vmcb_mark_dirty(vmcb01, VMCB_INTERCEPTS); diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 87da90360bc7..b32987f54ace 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -921,7 +921,7 @@ static void grow_ple_window(struct kvm_vcpu *vcpu) struct vmcb_control_area *control = &svm->vmcb->control; int old = control->pause_filter_count; - if (kvm_pause_in_guest(vcpu->kvm)) + if (kvm_pause_in_guest(vcpu)) return; control->pause_filter_count = __grow_ple_window(old, @@ -942,7 +942,7 @@ static void shrink_ple_window(struct kvm_vcpu *vcpu) struct vmcb_control_area *control = &svm->vmcb->control; int old = control->pause_filter_count; - if (kvm_pause_in_guest(vcpu->kvm)) + if (kvm_pause_in_guest(vcpu)) return; control->pause_filter_count = @@ -1136,12 +1136,12 @@ static void init_vmcb(struct kvm_vcpu *vcpu) svm_set_intercept(svm, INTERCEPT_RDPRU); svm_set_intercept(svm, INTERCEPT_RSM); - if (!kvm_mwait_in_guest(vcpu->kvm)) { + if (!kvm_mwait_in_guest(vcpu)) { svm_set_intercept(svm, INTERCEPT_MONITOR); svm_set_intercept(svm, INTERCEPT_MWAIT); } - if (!kvm_hlt_in_guest(vcpu->kvm)) + if (!kvm_hlt_in_guest(vcpu)) svm_set_intercept(svm, INTERCEPT_HLT); control->iopm_base_pa = __sme_set(iopm_base); @@ -1185,7 +1185,7 @@ static void init_vmcb(struct kvm_vcpu *vcpu) svm->nested.vmcb12_gpa = INVALID_GPA; svm->nested.last_vmcb12_gpa = INVALID_GPA; - if (!kvm_pause_in_guest(vcpu->kvm)) { + if (!kvm_pause_in_guest(vcpu)) { control->pause_filter_count = pause_filter_count; if (pause_filter_thresh) control->pause_filter_thresh = pause_filter_thresh; @@ -4269,7 +4269,7 @@ static void svm_handle_exit_irqoff(struct kvm_vcpu *vcpu) static void svm_sched_in(struct kvm_vcpu *vcpu, int cpu) { - if (!kvm_pause_in_guest(vcpu->kvm)) + if (!kvm_pause_in_guest(vcpu)) shrink_ple_window(vcpu); } diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 553dd2317b9c..f24c9a357f70 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -1597,7 +1597,7 @@ static void vmx_clear_hlt(struct kvm_vcpu *vcpu) * then the instruction is already executing and RIP has already been * advanced. */ - if (kvm_hlt_in_guest(vcpu->kvm) && + if (kvm_hlt_in_guest(vcpu) && vmcs_read32(GUEST_ACTIVITY_STATE) == GUEST_ACTIVITY_HLT) vmcs_write32(GUEST_ACTIVITY_STATE, GUEST_ACTIVITY_ACTIVE); } @@ -4212,10 +4212,10 @@ static u32 vmx_exec_control(struct vcpu_vmx *vmx) exec_control |= CPU_BASED_CR3_STORE_EXITING | CPU_BASED_CR3_LOAD_EXITING | CPU_BASED_INVLPG_EXITING; - if (kvm_mwait_in_guest(vmx->vcpu.kvm)) + if (kvm_mwait_in_guest(&vmx->vcpu)) exec_control &= ~(CPU_BASED_MWAIT_EXITING | CPU_BASED_MONITOR_EXITING); - if (kvm_hlt_in_guest(vmx->vcpu.kvm)) + if (kvm_hlt_in_guest(&vmx->vcpu)) exec_control &= ~CPU_BASED_HLT_EXITING; return exec_control; } @@ -4294,7 +4294,7 @@ static u32 vmx_secondary_exec_control(struct vcpu_vmx *vmx) } if (!enable_unrestricted_guest) exec_control &= ~SECONDARY_EXEC_UNRESTRICTED_GUEST; - if (kvm_pause_in_guest(vmx->vcpu.kvm)) + if (kvm_pause_in_guest(&vmx->vcpu)) exec_control &= ~SECONDARY_EXEC_PAUSE_LOOP_EXITING; if (!kvm_vcpu_apicv_active(vcpu)) exec_control &= ~(SECONDARY_EXEC_APIC_REGISTER_VIRT | @@ -4397,7 +4397,7 @@ static void init_vmcs(struct vcpu_vmx *vmx) vmcs_write64(POSTED_INTR_DESC_ADDR, __pa((&vmx->pi_desc))); } - if (!kvm_pause_in_guest(vmx->vcpu.kvm)) { + if (!kvm_pause_in_guest(&vmx->vcpu)) { vmcs_write32(PLE_GAP, ple_gap); vmx->ple_window = ple_window; vmx->ple_window_dirty = true; @@ -5562,7 +5562,7 @@ static void shrink_ple_window(struct kvm_vcpu *vcpu) */ static int handle_pause(struct kvm_vcpu *vcpu) { - if (!kvm_pause_in_guest(vcpu->kvm)) + if (!kvm_pause_in_guest(vcpu)) grow_ple_window(vcpu); /* @@ -7059,7 +7059,7 @@ static int vmx_vcpu_create(struct kvm_vcpu *vcpu) vmx_disable_intercept_for_msr(vcpu, MSR_IA32_SYSENTER_CS, MSR_TYPE_RW); vmx_disable_intercept_for_msr(vcpu, MSR_IA32_SYSENTER_ESP, MSR_TYPE_RW); vmx_disable_intercept_for_msr(vcpu, MSR_IA32_SYSENTER_EIP, MSR_TYPE_RW); - if (kvm_cstate_in_guest(vcpu->kvm)) { + if (kvm_cstate_in_guest(vcpu)) { vmx_disable_intercept_for_msr(vcpu, MSR_CORE_C1_RES, MSR_TYPE_R); vmx_disable_intercept_for_msr(vcpu, MSR_CORE_C3_RESIDENCY, MSR_TYPE_R); vmx_disable_intercept_for_msr(vcpu, MSR_CORE_C6_RESIDENCY, MSR_TYPE_R); @@ -7597,7 +7597,7 @@ static void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu) static void vmx_sched_in(struct kvm_vcpu *vcpu, int cpu) { - if (!kvm_pause_in_guest(vcpu->kvm)) + if (!kvm_pause_in_guest(vcpu)) shrink_ple_window(vcpu); } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 3ac6329e6d43..b419b258ed90 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -11355,6 +11355,10 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) #if IS_ENABLED(CONFIG_HYPERV) vcpu->arch.hv_root_tdp = INVALID_PAGE; #endif + vcpu->arch.mwait_in_guest = vcpu->kvm->arch.mwait_in_guest; + vcpu->arch.hlt_in_guest = vcpu->kvm->arch.hlt_in_guest; + vcpu->arch.pause_in_guest = vcpu->kvm->arch.pause_in_guest; + vcpu->arch.cstate_in_guest = vcpu->kvm->arch.cstate_in_guest; r = static_call(kvm_x86_vcpu_create)(vcpu); if (r) @@ -12539,7 +12543,7 @@ bool kvm_can_do_async_pf(struct kvm_vcpu *vcpu) vcpu->arch.exception.pending)) return false; - if (kvm_hlt_in_guest(vcpu->kvm) && !kvm_can_deliver_async_pf(vcpu)) + if (kvm_hlt_in_guest(vcpu) && !kvm_can_deliver_async_pf(vcpu)) return false; /* diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 588792f00334..a59b73e11726 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -324,24 +324,24 @@ static inline u64 nsec_to_cycles(struct kvm_vcpu *vcpu, u64 nsec) __rem; \ }) -static inline bool kvm_mwait_in_guest(struct kvm *kvm) +static inline bool kvm_mwait_in_guest(struct kvm_vcpu *vcpu) { - return kvm->arch.mwait_in_guest; + return vcpu->arch.mwait_in_guest; } -static inline bool kvm_hlt_in_guest(struct kvm *kvm) +static inline bool kvm_hlt_in_guest(struct kvm_vcpu *vcpu) { - return kvm->arch.hlt_in_guest; + return vcpu->arch.hlt_in_guest; } -static inline bool kvm_pause_in_guest(struct kvm *kvm) +static inline bool kvm_pause_in_guest(struct kvm_vcpu *vcpu) { - return kvm->arch.pause_in_guest; + return vcpu->arch.pause_in_guest; } -static inline bool kvm_cstate_in_guest(struct kvm *kvm) +static inline bool kvm_cstate_in_guest(struct kvm_vcpu *vcpu) { - return kvm->arch.cstate_in_guest; + return vcpu->arch.cstate_in_guest; } enum kvm_intr_type { From patchwork Wed Jun 22 00:49:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kechen Lu X-Patchwork-Id: 12890002 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 514DCC433EF for ; Wed, 22 Jun 2022 00:55:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1355474AbiFVAzD (ORCPT ); Tue, 21 Jun 2022 20:55:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52572 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355415AbiFVAzA (ORCPT ); Tue, 21 Jun 2022 20:55:00 -0400 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2086.outbound.protection.outlook.com [40.107.236.86]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E40A30F76; Tue, 21 Jun 2022 17:54:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PyjkMT56npsTofwLCjkARDqkTEp45TeEe5hPYqWmIiz8a1NpUBskxIU8ROReDESlGw1f379hArV+ZyLZfbqG/dLcgSxt7KKKCl902DIt+8UC6ENRYYh6Yxta09lwuEc1gLZpZXRiHpbqkFifpvWaGHyd34kD5dC3B2TUres2xCLn30jssal49NDk9uu/LZmhAmMxPE7lYLrZ3pCVqyw4b4oXLrKqTclIZyu7PLP9TqTSdlx32oOPG9Tuv7GqqJ2dR3np+OVM7DZv/hf8DK3IZK1Y4ze67n01HUN21MGEu6DqZ8ZS9soztMIhQEuubJV1Y5mldawVI/Qu0Ko8JFxTHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uTS+1CW2lTYu3a6LqmawL+VUdNwdn77A1oLP1tdn0OQ=; b=Rau/rAreVrQmef24JtIt+Ou8jW30GaSksNAhXlmrjmTucy6oTB9Hwz2bjeDeqDtaOUqVyEgO9J7IjCr9Rf4xCT14gpwy0+qSwwznAqAjHybrDjticWhNXh7qPtlwCmQIjf8CTQSj7YTwO6PzOrHTFOFFNknMyFPb1L+YHSNZUHk9H7EnJsBiaaZTerbiGwk+zcbS2uvv4ZlqkxWzz/n8I6e0U0EAPLAJeAqbYUidURvdB1YX6A8c6uBVRu2RWZLc7DIvGqhu2CEReYaz2m1Qo+xAHTGGIy5JpgLopP+mFbCAiXNl5LxfvtvXy4R3lGmLP85m7346XksQ1ksvCSme+w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uTS+1CW2lTYu3a6LqmawL+VUdNwdn77A1oLP1tdn0OQ=; b=CNc3wPyGyjgVLhEx/GpxljYcnatJvq1BRv68o4jgBNJaJPA0Z9AO95fHedzkcIlq6IrPKjir5fp26MDPxTDvIRkT/Yz/ik5lLrrg7i6SFBlE9pGHZRMNOySojpuRVnjp7G3NpicFOn2RZhgh03NuGGLW3mzvgwF9MAaAXZW2Cv0l+awhJf8BFygYx3ZPZ3QahZU5OkyGFTdiyXd00dapRcUa5+xdeMVgnqtlPHJIpj8aT8l8eAvBJv3tsGunguUfsQVgl64ZDyEGFKEY1suwpaiv7BlJVxdbup4SfOAmiBKBmePNYIgdLUgEFi5pTkCfesEo7z4vlG/HTa8o1iI30g== Received: from BN0PR04CA0073.namprd04.prod.outlook.com (2603:10b6:408:ea::18) by MN2PR12MB3439.namprd12.prod.outlook.com (2603:10b6:208:cf::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.19; Wed, 22 Jun 2022 00:54:56 +0000 Received: from BN8NAM11FT003.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ea:cafe::84) by BN0PR04CA0073.outlook.office365.com (2603:10b6:408:ea::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5373.15 via Frontend Transport; Wed, 22 Jun 2022 00:54:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.236) by BN8NAM11FT003.mail.protection.outlook.com (10.13.177.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5373.15 via Frontend Transport; Wed, 22 Jun 2022 00:54:56 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Wed, 22 Jun 2022 00:54:55 +0000 Received: from foundations-user-AS-2114GT-DNR-C1-NC24B.nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Tue, 21 Jun 2022 17:54:54 -0700 From: Kechen Lu To: , CC: , , , , , Subject: [RFC PATCH v4 3/7] KVM: x86: Reject disabling of MWAIT interception when not allowed Date: Tue, 21 Jun 2022 17:49:20 -0700 Message-ID: <20220622004924.155191-4-kechenl@nvidia.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220622004924.155191-1-kechenl@nvidia.com> References: <20220622004924.155191-1-kechenl@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2243881d-fa03-4f8f-98f3-08da53e9d356 X-MS-TrafficTypeDiagnostic: MN2PR12MB3439:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: eClEytRk8hWxlsJcHP8KOIFzk07kjdnMfNE2Ti0fvSN5AXArrpwzrKQDTZ1DJN2CxFSuHUnfNI0up137PBHkRkez8IklWAjOHD/QlYncfMdDRNZpLxXhdg8nvqOyqkMCF/C8jpr/DTxthLShtdg2br+aa6yr1mB1enkq2G77gO7l9vR/Su1VJeXVo3msdTYYL8ePFz1uoMoQfppHcAVELY/ME/DJzzWffOAzFkUx7FTdo+8nwn1Hn7z6Keib0vg7jeJP8BakrLcAapXMm7ZodNoEAmDmNDVCNvezS8hHFytvXXQLdJcm6HwMINZYiHjnvacfc1Ac6t2HKBkmLpEW8jaJ5bCWLsS2reS8te4tFLE4JIq5QcF/73OsXx56elh+fywslnY5p7ZnNY+Fo1ixCYSUuCOvj1QaSpQfQQoPKu3QMHp07hJRIWSgWsmd/P97x+/eeIID70Kx1Vq1uGys+zXFAo1b6jnQoIY76OxR09iw0WqtCMatpDlki440dQj0maslqbo9tgwHLRZ/0G17nB0/flJiT6O8vxuwOaN79dUYJaQEozrS3pN4h/ArpFxNAMehdSncBVIUDnvomxOyOStJ7TyegvfKmnvGfNILrIvlAl9bcOyzasRCXldUPdt19/tpbc5PbyqWFAuYqpn7xqI01qWE1VpmC3/oXRpL3F2KrfPOIY/7WhovQfAwwfpyO0CLS7wO1R+Rr5yraQs4y7EAwDqv/YYOO6B4/hNAXWXnqt92nOP4Qdo4Rsr2JVuV X-Forefront-Antispam-Report: CIP:12.22.5.236;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230016)(4636009)(136003)(39860400002)(346002)(376002)(396003)(40470700004)(46966006)(36840700001)(2906002)(478600001)(26005)(40480700001)(426003)(41300700001)(47076005)(8936002)(81166007)(7696005)(5660300002)(110136005)(16526019)(2616005)(186003)(82310400005)(82740400003)(70586007)(54906003)(1076003)(36860700001)(86362001)(70206006)(36756003)(83380400001)(356005)(6666004)(4326008)(8676002)(40460700003)(316002)(336012)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jun 2022 00:54:56.2781 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2243881d-fa03-4f8f-98f3-08da53e9d356 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.236];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT003.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3439 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Sean Christopherson Reject KVM_CAP_X86_DISABLE_EXITS if userspace attempts to disable MWAIT exits and KVM previously reported (via KVM_CHECK_EXTENSION) that MWAIT is not allowed in guest, e.g. because it's not supported or the CPU doesn't have an aways-running APIC timer. Fixes: 4d5422cea3b6 ("KVM: X86: Provide a capability to disable MWAIT intercepts") Signed-off-by: Sean Christopherson Co-developed-by: Kechen Lu Suggested-by: Chao Gao --- arch/x86/kvm/x86.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b419b258ed90..6ec01362a7d8 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4199,6 +4199,16 @@ static inline bool kvm_can_mwait_in_guest(void) boot_cpu_has(X86_FEATURE_ARAT); } +static u64 kvm_get_allowed_disable_exits(void) +{ + u64 r = KVM_X86_DISABLE_VALID_EXITS; + + if(!kvm_can_mwait_in_guest()) + r &= ~KVM_X86_DISABLE_EXITS_MWAIT; + + return r; +} + static int kvm_ioctl_get_supported_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 __user *cpuid_arg) { @@ -4318,10 +4328,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) r = KVM_CLOCK_VALID_FLAGS; break; case KVM_CAP_X86_DISABLE_EXITS: - r |= KVM_X86_DISABLE_EXITS_HLT | KVM_X86_DISABLE_EXITS_PAUSE | - KVM_X86_DISABLE_EXITS_CSTATE; - if(kvm_can_mwait_in_guest()) - r |= KVM_X86_DISABLE_EXITS_MWAIT; + r |= kvm_get_allowed_disable_exits(); break; case KVM_CAP_X86_SMM: /* SMBASE is usually relocated above 1M on modern chipsets, @@ -6003,15 +6010,14 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, break; case KVM_CAP_X86_DISABLE_EXITS: r = -EINVAL; - if (cap->args[0] & ~KVM_X86_DISABLE_VALID_EXITS) + if (cap->args[0] & ~kvm_get_allowed_disable_exits()) break; mutex_lock(&kvm->lock); if (kvm->created_vcpus) goto disable_exits_unlock; - if ((cap->args[0] & KVM_X86_DISABLE_EXITS_MWAIT) && - kvm_can_mwait_in_guest()) + if (cap->args[0] & KVM_X86_DISABLE_EXITS_MWAIT) kvm->arch.mwait_in_guest = true; if (cap->args[0] & KVM_X86_DISABLE_EXITS_HLT) kvm->arch.hlt_in_guest = true; From patchwork Wed Jun 22 00:49:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kechen Lu X-Patchwork-Id: 12890003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14580C433EF for ; Wed, 22 Jun 2022 00:55:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1355498AbiFVAzH (ORCPT ); Tue, 21 Jun 2022 20:55:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52520 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355459AbiFVAzB (ORCPT ); Tue, 21 Jun 2022 20:55:01 -0400 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2075.outbound.protection.outlook.com [40.107.94.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6172930F50; Tue, 21 Jun 2022 17:55:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nHQhPdXNx0p8ocsFbtSlyyWEmSdzuAA1K1RPKxIUkCF0uUK9A2gsPFg9h37A6lm7dqt2/rqVAmZjHwAbfCwvXy1YQntzlB+88xm+KcLSSXYQVU6wvZUbEMaEhJbtG0KEd96PWKHj52Vz++5cm4HnND8aFQRxjJUCmelMjFOZdXu0BORuLAE10lrnkt/ecc1eM3C8sPq1D9y3blkBGLlmwfmxeG5ZH0KN1y53qGY2+zwb3ya/JzBy+XrW6jm+tsrEQ9oS6m5H+tAzuZBLyPe6Bi6fLOd8xOfBaALDnVIENwV37Ibhsu2wy3uzlHNiN82RsVBkGSKE3GJ1Ch6Q8C2tgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ccIl/E+UF+vZ5VwPRBoaMcmya4evfUFZtd4XoHWCXwE=; b=Z1J8f8hwlOaRQ068BDUkrMWP8o7VzUIpfgcRJbOp3a13hlEvO3bF2ypbTSU/oOO7kduD9Unb/lGyNfYe76KLjjS22LCuz6BmN6PqqkVsLcA1kgrHsrlJu0hoyd9H0xRiaxYjRGcqwvAtqGhtkwxD4HZvPPTRYP6bPeGAGnfRGrkqTjb7CoP7pI0izFjcld4VKHNfCuciHy638KpBDnyn/rYKL1g1+EP6pD1HhxpNq5d+WPtD6Gs2bJnLwUB3b3e7gFNG1/eqRkFy/lC+XZRAcXrXu27hPgydwGdVYBzrEk3DzPb12j9y0hut+mUtw6OTTH0/yorZuxqP2KgOx2PEfA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ccIl/E+UF+vZ5VwPRBoaMcmya4evfUFZtd4XoHWCXwE=; b=HXYbpEZxdEixmax1P3aOEhcWS1SAQh6rJMSkSNMEeC7hxHOb6n9rHeUy/FrCT3XDLjNXqfZfihRWPn17uCAs/NNhL2l4UhgH3nIWv2WE6ETfPzoIDOnmq54B1yM7yg3kXrTdDFqpHV34F9wdVr9qu8U1mMMJj3uTsMMQ2ksGP4UWs6L47SrRw0SDKgRkbeHhaZCiw26kp1kbsmxgl+U4pTZ/t+bFWFxIHBSm7G2c9bf9f1/+zkcm9FBkKPFMXfV5vWeZE25Mch17SApw2u9HGPFxmGKbTcSevlfr88yqoLnwxxaYvCBqJwXEWcRiSJRrWxhEoZEoX+cYDBzxZ24hOQ== Received: from DS7PR06CA0007.namprd06.prod.outlook.com (2603:10b6:8:2a::25) by DM5PR1201MB2554.namprd12.prod.outlook.com (2603:10b6:3:ec::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.16; Wed, 22 Jun 2022 00:54:58 +0000 Received: from DM6NAM11FT006.eop-nam11.prod.protection.outlook.com (2603:10b6:8:2a:cafe::e7) by DS7PR06CA0007.outlook.office365.com (2603:10b6:8:2a::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.14 via Frontend Transport; Wed, 22 Jun 2022 00:54:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.235) by DM6NAM11FT006.mail.protection.outlook.com (10.13.173.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5353.14 via Frontend Transport; Wed, 22 Jun 2022 00:54:58 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Wed, 22 Jun 2022 00:54:57 +0000 Received: from foundations-user-AS-2114GT-DNR-C1-NC24B.nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Tue, 21 Jun 2022 17:54:56 -0700 From: Kechen Lu To: , CC: , , , , , Subject: [RFC PATCH v4 4/7] KVM: x86: Let userspace re-enable previously disabled exits Date: Tue, 21 Jun 2022 17:49:21 -0700 Message-ID: <20220622004924.155191-5-kechenl@nvidia.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220622004924.155191-1-kechenl@nvidia.com> References: <20220622004924.155191-1-kechenl@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: fda30173-7991-4d5b-34a9-08da53e9d477 X-MS-TrafficTypeDiagnostic: DM5PR1201MB2554:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Kn/OA3VH6nNu+Z0gbRL7HpU9YLCHkDCCHNNjKtKt44hhIAk8lNB60okmNVFRzWs61HsED1PAz0bbOfjcL6lcypL6Cx5DSZKf99gR0jTd2Jk8+cZPtyRJjULpQmQWDBvmNUCePLNWKF8RvErJBJH/pyp/MsJo59S/jCHPB5Si10LRogZtyk5gubwYVAMA0otUNgwC48JtmuQvDvyS68WxoeiK79TQnNxIuBkqv2xRWg4q/2xptrMTvHW+mXv0tlD2DeptTJfcJqO+1jax0vdfAs0gis1MPfOD5ExKtUqQEkjxGBQ28afVeaoOqi79SMrKhn1qIeNs1KLVwGxk9m5PHyxueWqnObVPwQ5YjUpBdFFknuc5wGGuudfT62QNFuEXl3Fi8aBquYUwwxHGGaVsz4TAsbL2uCapNEKbp0bVICRHTGoX5RwKAIBhjyxjufV67vpsN2B4Qm31AJNuN57ZNeDpi6f+aZDfyEyd1B+LjtQ8fhF8VPOm22opuk3WMxlNR3tNZ4vYrd2q+PrUyCsBZjNhQ4/1FrHsiLLDuGRHXxembNrBh7G8gs+Jvjox9xFtYylSfJeYkTmgC1i1xfwBESiUrlCOSPzKM6OH6y47pnhFk7I6ZRKXb8+zo60lrzIG2pY+EhtVk3qa7DRPCOh0z342o3ibRr3X19PIm+7rZz3PJ7MQ0esN/SKSnOSDOnXoV0Oyp6D7M9t76/L90L8ES7od+IBAUQFtCK8OSARAuGg+y9nUSh1dLktVc36Chkk6 X-Forefront-Antispam-Report: CIP:12.22.5.235;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230016)(4636009)(346002)(376002)(39860400002)(396003)(136003)(36840700001)(40470700004)(46966006)(110136005)(54906003)(2906002)(478600001)(4326008)(36756003)(26005)(5660300002)(8936002)(70206006)(316002)(2616005)(70586007)(41300700001)(426003)(40460700003)(86362001)(8676002)(83380400001)(1076003)(47076005)(336012)(16526019)(81166007)(7696005)(40480700001)(82310400005)(82740400003)(6666004)(356005)(36860700001)(186003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jun 2022 00:54:58.2164 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fda30173-7991-4d5b-34a9-08da53e9d477 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.235];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT006.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1201MB2554 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Sean Christopherson Add an OVERRIDE flag to KVM_CAP_X86_DISABLE_EXITS allow userspace to re-enable exits and/or override previous settings. There's no real use case for the the per-VM ioctl, but a future per-vCPU variant wants to let userspace toggle interception while the vCPU is running; add the OVERRIDE functionality now to provide consistent between between the per-VM and per-vCPU variants. Signed-off-by: Sean Christopherson --- Documentation/virt/kvm/api.rst | 5 +++++ arch/x86/kvm/x86.c | 32 ++++++++++++++++++++++++-------- include/uapi/linux/kvm.h | 4 +++- 3 files changed, 32 insertions(+), 9 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index d0d8749591a8..89e13b6783b5 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6941,6 +6941,7 @@ Valid bits in args[0] are:: #define KVM_X86_DISABLE_EXITS_HLT (1 << 1) #define KVM_X86_DISABLE_EXITS_PAUSE (1 << 2) #define KVM_X86_DISABLE_EXITS_CSTATE (1 << 3) + #define KVM_X86_DISABLE_EXITS_OVERRIDE (1ull << 63) Enabling this capability on a VM provides userspace with a way to no longer intercept some instructions for improved latency in some @@ -6949,6 +6950,10 @@ physical CPUs. More bits can be added in the future; userspace can just pass the KVM_CHECK_EXTENSION result to KVM_ENABLE_CAP to disable all such vmexits. +By default, this capability only disables exits. To re-enable an exit, or to +override previous settings, userspace can set KVM_X86_DISABLE_EXITS_OVERRIDE, +in which case KVM will enable/disable according to the mask (a '1' == disable). + Do not enable KVM_FEATURE_PV_UNHALT if you disable HLT exits. 7.14 KVM_CAP_S390_HPAGE_1M diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 6ec01362a7d8..fe114e319a89 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -5263,6 +5263,28 @@ static int kvm_vcpu_ioctl_device_attr(struct kvm_vcpu *vcpu, return r; } + +#define kvm_ioctl_disable_exits(a, mask) \ +({ \ + if (!kvm_can_mwait_in_guest()) \ + (mask) &= KVM_X86_DISABLE_EXITS_MWAIT; \ + if ((mask) & KVM_X86_DISABLE_EXITS_OVERRIDE) { \ + (a).mwait_in_guest = (mask) & KVM_X86_DISABLE_EXITS_MWAIT; \ + (a).hlt_in_guest = (mask) & KVM_X86_DISABLE_EXITS_HLT; \ + (a).pause_in_guest = (mask) & KVM_X86_DISABLE_EXITS_PAUSE; \ + (a).cstate_in_guest = (mask) & KVM_X86_DISABLE_EXITS_CSTATE; \ + } else { \ + if ((mask) & KVM_X86_DISABLE_EXITS_MWAIT) \ + (a).mwait_in_guest = true; \ + if ((mask) & KVM_X86_DISABLE_EXITS_HLT) \ + (a).hlt_in_guest = true; \ + if ((mask) & KVM_X86_DISABLE_EXITS_PAUSE) \ + (a).pause_in_guest = true; \ + if ((mask) & KVM_X86_DISABLE_EXITS_CSTATE) \ + (a).cstate_in_guest = true; \ + } \ +}) + static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu, struct kvm_enable_cap *cap) { @@ -6017,14 +6039,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, if (kvm->created_vcpus) goto disable_exits_unlock; - if (cap->args[0] & KVM_X86_DISABLE_EXITS_MWAIT) - kvm->arch.mwait_in_guest = true; - if (cap->args[0] & KVM_X86_DISABLE_EXITS_HLT) - kvm->arch.hlt_in_guest = true; - if (cap->args[0] & KVM_X86_DISABLE_EXITS_PAUSE) - kvm->arch.pause_in_guest = true; - if (cap->args[0] & KVM_X86_DISABLE_EXITS_CSTATE) - kvm->arch.cstate_in_guest = true; + kvm_ioctl_disable_exits(kvm->arch, cap->args[0]); + r = 0; disable_exits_unlock: mutex_unlock(&kvm->lock); diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 5088bd9f1922..f2e76e436be5 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -814,10 +814,12 @@ struct kvm_ioeventfd { #define KVM_X86_DISABLE_EXITS_HLT (1 << 1) #define KVM_X86_DISABLE_EXITS_PAUSE (1 << 2) #define KVM_X86_DISABLE_EXITS_CSTATE (1 << 3) +#define KVM_X86_DISABLE_EXITS_OVERRIDE (1ull << 63) #define KVM_X86_DISABLE_VALID_EXITS (KVM_X86_DISABLE_EXITS_MWAIT | \ KVM_X86_DISABLE_EXITS_HLT | \ KVM_X86_DISABLE_EXITS_PAUSE | \ - KVM_X86_DISABLE_EXITS_CSTATE) + KVM_X86_DISABLE_EXITS_CSTATE | \ + KVM_X86_DISABLE_EXITS_OVERRIDE) /* for KVM_ENABLE_CAP */ struct kvm_enable_cap { From patchwork Wed Jun 22 00:49:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kechen Lu X-Patchwork-Id: 12890005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93A9AC43334 for ; Wed, 22 Jun 2022 00:55:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354770AbiFVAzQ (ORCPT ); Tue, 21 Jun 2022 20:55:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355542AbiFVAzL (ORCPT ); Tue, 21 Jun 2022 20:55:11 -0400 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2048.outbound.protection.outlook.com [40.107.243.48]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A69C3120B; Tue, 21 Jun 2022 17:55:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XbRuXpuOi5pVo8dfTCX9BPYauPETUHIDgJTO4ffWLsGsLnUcmntiyk12ZM5/4nBbETtzHJKU+Ki4MawRfLIq4xy4AGeKtKBIrx0nlfrBkKeyIQaxo+kO+TV7xmCQtntb9cXMVWXfazPoErN7iE83GqEjxhl0tWzam5UCs37O7gQjBxpHManr8cO8/rDLijWZ+ZWk72x4OH0f39Kgie2KN28FJLPxYREpuvLNKnwvadqZKhRM50J3J+B3Rqs1r+mVGsOfJwNvsnbMGF7wqDkr2Ik2Vj0h29m3uHQcR2lO59aXQW2nz4zTsp6CZlomlcI3J9giQKh28i1/1GylRazAcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AfD+SPzU3vtV5ahCycR/JCfR6nOVjscLOMf4x/bZHtI=; b=AW7YuQmjN/Ow7MamkBBMEcnDGCN7qe6GkBxKZlSrcWOjjkVw4Snt/f6plugWtjjQoXnfrFHK4h5jUNGRPFxIOkzLPKKoJQLz4zwRzyLaTObDI88fZ0zTrMfxLL9tc1GkoAL1f6xPNBqSUTqIcLCqfHFKz0NKtpnlJLDIg0TL/juDkO74d4K8moOFtnV5UILwycKWBOXcfWyKdxl21sG51tALJHz6lQFPq7IyKOFkgevCs0XHWeeXSZblkzA3Lo9kvcmF3XMTfzR1YUCSm1NvGWap9gJVftflXCBuu7Rl+xYsl+PKFH9rlmQpoR3lrE9Q9DSzgosG/6ifeMiwVdkPpA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AfD+SPzU3vtV5ahCycR/JCfR6nOVjscLOMf4x/bZHtI=; b=lMq/hDzcAln7JHtvRkuEbTGPMbcFePUIRKxa/CSQhg6WDzi5phgwYfFRbG5G+NaZreTAWO/qJxSpPM6IhX75d7Zc0Cz3qIawo/Ikx9TnHMaGFND4R9HtrY2Z/uPTg31vIOxaZRfNcCwqMlZU3A01zxEfunSZxWO5xX9Acpnf+VtXoQTL2NbXq0yz0GMAeicF56fGxqKZGv/fUCQ/p3f/+UtRfV/YRhqii1SI9VcJTVyqifkk6X8oSGHj+o7K3+ieU8dkRGW1BhVyHCmZX7owNm4jXJ2SiYiXB+Qc2JuCIPrYcLr75o8EBclz6m9PkxZrGDobcuFpGLNW1N9qK+tjlA== Received: from MWHPR22CA0060.namprd22.prod.outlook.com (2603:10b6:300:12a::22) by MWHPR12MB1757.namprd12.prod.outlook.com (2603:10b6:300:111::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.16; Wed, 22 Jun 2022 00:55:03 +0000 Received: from CO1NAM11FT053.eop-nam11.prod.protection.outlook.com (2603:10b6:300:12a:cafe::64) by MWHPR22CA0060.outlook.office365.com (2603:10b6:300:12a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5373.15 via Frontend Transport; Wed, 22 Jun 2022 00:55:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.238) by CO1NAM11FT053.mail.protection.outlook.com (10.13.175.63) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5373.15 via Frontend Transport; Wed, 22 Jun 2022 00:55:03 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Wed, 22 Jun 2022 00:54:59 +0000 Received: from foundations-user-AS-2114GT-DNR-C1-NC24B.nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Tue, 21 Jun 2022 17:54:58 -0700 From: Kechen Lu To: , CC: , , , , , Subject: [RFC PATCH v4 5/7] KVM: x86: add vCPU scoped toggling for disabled exits Date: Tue, 21 Jun 2022 17:49:22 -0700 Message-ID: <20220622004924.155191-6-kechenl@nvidia.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220622004924.155191-1-kechenl@nvidia.com> References: <20220622004924.155191-1-kechenl@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5fe14544-d8d2-4e66-2a34-08da53e9d788 X-MS-TrafficTypeDiagnostic: MWHPR12MB1757:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: divr3U3P56P8R42Jj/xjF6Zbv2a4u6tyaxuVWzLT141ldJtBqBjVJZrva+3zPZGhZZ0g/6pUyMUs8glGqaPkG5Mw/O5A/P8vayATZ+SIVkHmN8j49XH4AbqwnYScoYkUmx/FRp3Xn/Z6bg59kUCIOIHUuQIlYpaptX9qK0fTky74X/LXkZEdSCvF+J/UsHx/lVHc5+2nbiVDhvYnBjzdvTJ/JVpQ0OfNy9nYW3ALMKMnGEYnV22Hnw2fpSw42ZbUh8RCF+d9/3gPUejW5GjFCva+VVVuXKFkwvhkJnnYSPHGuhom+8dXlhc2ldoJC72c4jQHR5S21FmVgWHRZaC5Kr21e05JQV4oNT/coXvxdnMWkdioBSdd4jTJEslWOCecAQqQHRa6aVszVYGjQhlt7hkknTmQsUh9l0MMe7+hnV0CwlplTIPkwJLodvbGyrp+Nz/XHJUjGjE7IWq5d06ZyP7E4VWAjn4SfGV1KUDop6PsaY/Zr0adamlNKwbu4PT6NXBrWbsHq8CWpxYBeO49D9rk6ZAEKfSDil20ATsVOJ7isoS4XOtAVvlgKA4UGH8mGElbAvU8CrZDy87QBHwF67e4NfCSDefl5f+ESHFZaameHLWuzWYnadDzJbaIN1zQL9HM9AXNKli+dQNxq+8CSsB5vQ0pDOKsxpbKfyZ8ucV2j7pXP1WsJzr+QKTMAUCimPrklBBjYzHdmtN8ABr3IaRMQADVlM15i0kE3QLlPES7hulifkzMFrkgSbyluaRC X-Forefront-Antispam-Report: CIP:12.22.5.238;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230016)(4636009)(136003)(39860400002)(376002)(396003)(346002)(40470700004)(36840700001)(46966006)(6666004)(40460700003)(41300700001)(8936002)(82310400005)(36756003)(7696005)(8676002)(26005)(316002)(54906003)(110136005)(478600001)(70206006)(426003)(336012)(4326008)(83380400001)(86362001)(82740400003)(16526019)(356005)(186003)(47076005)(2616005)(40480700001)(5660300002)(81166007)(2906002)(36860700001)(1076003)(70586007)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jun 2022 00:55:03.3607 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5fe14544-d8d2-4e66-2a34-08da53e9d788 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.238];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT053.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1757 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Introduce support of vCPU-scoped ioctl with KVM_CAP_X86_DISABLE_EXITS cap for disabling exits to enable finer-grained VM exits disabling on per vCPU scales instead of whole guest. This patch enabled the vCPU-scoped exits control toggling, also align the VM-scoped exits control behaviors. Add a new kvm request KVM_REQ_DISABLE_EXITS to guarantee updating the vmcs before vCPU entry especially for toggling the VM-scoped exits. In use cases like Windows guest running heavy CPU-bound workloads, disabling HLT VM-exits could mitigate host sched ctx switch overhead. Simply HLT disabling on all vCPUs could bring performance benefits, but if no pCPUs reserved for host threads, could happened to the forced preemption as host does not know the time to do the schedule for other host threads want to run. With this patch, we could only disable part of vCPUs HLT exits for one guest, this still keeps performance benefits, and also shows resiliency to host stressing workload running at the same time. In the host stressing workload experiment with Windows guest heavy CPU-bound workloads, it shows good resiliency and having the ~3% performance improvement. E.g. Passmark running in a Windows guest with this patch disabling HLT exits on only half of vCPUs still showing 2.4% higher main score v/s baseline. Suggested-by: Sean Christopherson Suggested-by: Chao Gao Signed-off-by: Kechen Lu --- Documentation/virt/kvm/api.rst | 2 +- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/svm/svm.c | 30 ++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 37 ++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 23 +++++++++++++++---- 6 files changed, 91 insertions(+), 5 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 89e13b6783b5..7f614b7d5ad8 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6948,7 +6948,7 @@ longer intercept some instructions for improved latency in some workloads, and is suggested when vCPUs are associated to dedicated physical CPUs. More bits can be added in the future; userspace can just pass the KVM_CHECK_EXTENSION result to KVM_ENABLE_CAP to disable -all such vmexits. +all such vmexits. VM scoped and vCPU scoped capability are both supported. By default, this capability only disables exits. To re-enable an exit, or to override previous settings, userspace can set KVM_X86_DISABLE_EXITS_OVERRIDE, diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index da47f60a4650..c17d417cb3cf 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -128,6 +128,7 @@ KVM_X86_OP(msr_filter_changed) KVM_X86_OP(complete_emulated_msr) KVM_X86_OP(vcpu_deliver_sipi_vector) KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); +KVM_X86_OP(update_disabled_exits) #undef KVM_X86_OP #undef KVM_X86_OP_OPTIONAL diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 573a39bf7a84..86baae62af86 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -105,6 +105,7 @@ KVM_ARCH_REQ_FLAGS(30, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) #define KVM_REQ_MMU_FREE_OBSOLETE_ROOTS \ KVM_ARCH_REQ_FLAGS(31, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) +#define KVM_REQ_DISABLE_EXITS KVM_ARCH_REQ(32) #define CR0_RESERVED_BITS \ (~(unsigned long)(X86_CR0_PE | X86_CR0_MP | X86_CR0_EM | X86_CR0_TS \ @@ -1584,6 +1585,8 @@ struct kvm_x86_ops { * Returns vCPU specific APICv inhibit reasons */ unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu); + + void (*update_disabled_exits)(struct kvm_vcpu *vcpu); }; struct kvm_x86_nested_ops { diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index b32987f54ace..7b3d64b3b901 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4589,6 +4589,33 @@ static void svm_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector) sev_vcpu_deliver_sipi_vector(vcpu, vector); } +static void svm_update_disabled_exits(struct kvm_vcpu *vcpu) +{ + struct vcpu_svm *svm = to_svm(vcpu); + struct vmcb_control_area *control = &svm->vmcb->control; + + if (kvm_hlt_in_guest(vcpu)) + svm_clr_intercept(svm, INTERCEPT_HLT); + else + svm_set_intercept(svm, INTERCEPT_HLT); + + if (kvm_mwait_in_guest(vcpu)) { + svm_clr_intercept(svm, INTERCEPT_MONITOR); + svm_clr_intercept(svm, INTERCEPT_MWAIT); + } else { + svm_set_intercept(svm, INTERCEPT_MONITOR); + svm_set_intercept(svm, INTERCEPT_MWAIT); + } + + if (kvm_pause_in_guest(vcpu)) { + svm_clr_intercept(svm, INTERCEPT_PAUSE); + } else { + control->pause_filter_count = pause_filter_count; + if (pause_filter_thresh) + control->pause_filter_thresh = pause_filter_thresh; + } +} + static void svm_vm_destroy(struct kvm *kvm) { avic_vm_destroy(kvm); @@ -4732,7 +4759,10 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .complete_emulated_msr = svm_complete_emulated_msr, .vcpu_deliver_sipi_vector = svm_vcpu_deliver_sipi_vector, + .vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons, + + .update_disabled_exits = svm_update_disabled_exits, }; /* diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index f24c9a357f70..2d000638cc9b 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7716,6 +7716,41 @@ static bool vmx_check_apicv_inhibit_reasons(enum kvm_apicv_inhibit reason) return supported & BIT(reason); } +static void vmx_update_disabled_exits(struct kvm_vcpu *vcpu) +{ + struct vcpu_vmx *vmx = to_vmx(vcpu); + + if (kvm_hlt_in_guest(vcpu)) + exec_controls_clearbit(vmx, CPU_BASED_HLT_EXITING); + else + exec_controls_setbit(vmx, CPU_BASED_HLT_EXITING); + + if (kvm_mwait_in_guest(vcpu)) + exec_controls_clearbit(vmx, CPU_BASED_MWAIT_EXITING | + CPU_BASED_MONITOR_EXITING); + else + exec_controls_setbit(vmx, CPU_BASED_MWAIT_EXITING | + CPU_BASED_MONITOR_EXITING); + + if (!kvm_pause_in_guest(vcpu)) { + vmcs_write32(PLE_GAP, ple_gap); + vmx->ple_window = ple_window; + vmx->ple_window_dirty = true; + } + + if (kvm_cstate_in_guest(vcpu)) { + vmx_disable_intercept_for_msr(vcpu, MSR_CORE_C1_RES, MSR_TYPE_R); + vmx_disable_intercept_for_msr(vcpu, MSR_CORE_C3_RESIDENCY, MSR_TYPE_R); + vmx_disable_intercept_for_msr(vcpu, MSR_CORE_C6_RESIDENCY, MSR_TYPE_R); + vmx_disable_intercept_for_msr(vcpu, MSR_CORE_C7_RESIDENCY, MSR_TYPE_R); + } else { + vmx_enable_intercept_for_msr(vcpu, MSR_CORE_C1_RES, MSR_TYPE_R); + vmx_enable_intercept_for_msr(vcpu, MSR_CORE_C3_RESIDENCY, MSR_TYPE_R); + vmx_enable_intercept_for_msr(vcpu, MSR_CORE_C6_RESIDENCY, MSR_TYPE_R); + vmx_enable_intercept_for_msr(vcpu, MSR_CORE_C7_RESIDENCY, MSR_TYPE_R); + } +} + static struct kvm_x86_ops vmx_x86_ops __initdata = { .name = "kvm_intel", @@ -7849,6 +7884,8 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .complete_emulated_msr = kvm_complete_insn_gp, .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, + + .update_disabled_exits = vmx_update_disabled_exits, }; static unsigned int vmx_handle_intel_pt_intr(void) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index fe114e319a89..6165f0b046ed 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -5331,6 +5331,13 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu, if (vcpu->arch.pv_cpuid.enforce) kvm_update_pv_runtime(vcpu); + return 0; + case KVM_CAP_X86_DISABLE_EXITS: + if (cap->args[0] & ~kvm_get_allowed_disable_exits()) + return -EINVAL; + + kvm_ioctl_disable_exits(vcpu->arch, cap->args[0]); + kvm_make_request(KVM_REQ_DISABLE_EXITS, vcpu); return 0; default: return -EINVAL; @@ -5980,6 +5987,8 @@ int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irq_event, int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) { + struct kvm_vcpu *vcpu; + unsigned long i; int r; if (cap->flags) @@ -6036,14 +6045,17 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, break; mutex_lock(&kvm->lock); - if (kvm->created_vcpus) - goto disable_exits_unlock; + if (kvm->created_vcpus) { + kvm_for_each_vcpu(i, vcpu, kvm) { + kvm_ioctl_disable_exits(vcpu->arch, cap->args[0]); + kvm_make_request(KVM_REQ_DISABLE_EXITS, vcpu); + } + } + mutex_unlock(&kvm->lock); kvm_ioctl_disable_exits(kvm->arch, cap->args[0]); r = 0; -disable_exits_unlock: - mutex_unlock(&kvm->lock); break; case KVM_CAP_MSR_PLATFORM_INFO: kvm->arch.guest_can_read_msr_platform_info = cap->args[0]; @@ -10175,6 +10187,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) if (kvm_check_request(KVM_REQ_UPDATE_CPU_DIRTY_LOGGING, vcpu)) static_call(kvm_x86_update_cpu_dirty_logging)(vcpu); + + if (kvm_check_request(KVM_REQ_DISABLE_EXITS, vcpu)) + static_call(kvm_x86_update_disabled_exits)(vcpu); } if (kvm_check_request(KVM_REQ_EVENT, vcpu) || req_int_win || From patchwork Wed Jun 22 00:49:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kechen Lu X-Patchwork-Id: 12890004 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0350C433EF for ; Wed, 22 Jun 2022 00:55:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1355506AbiFVAzP (ORCPT ); Tue, 21 Jun 2022 20:55:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355419AbiFVAzF (ORCPT ); Tue, 21 Jun 2022 20:55:05 -0400 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2064.outbound.protection.outlook.com [40.107.93.64]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4FE8830F7E; Tue, 21 Jun 2022 17:55:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YbkqFebNAkBMnpT9OZ4zJfc+2jGMTAvVQJsZZCzp9rPUzY7mv9K5RxKT/2Uk0si3zJsVgD9riWPxlSS9PPiXL4Ef9arulIs9Ahwm7+1gp4smiAdLN33nFEdCVVJm7RRD+47LKX9HhKsPrOOejyN5WUbKxmtBEHSB2/p/tpaW8Vk7bd9jNlTOVHVYNbl+eAyeoHCXqlMwGCBb4djSSWP0cCHIO3rksBc4KsUYLxuOPmZurlycMtdZ2D3a2HkJLqQGlP8J3eUhic7CDTh/ZnzhXFFTzdGXCTOZTXoKQpNMlBXRNidx0rLH5Im29YhHeAbFbhq2xaq6YCoiDKwLhCkpmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=M+gidTR64ir8kbPrzSZvcklZUAFYlf+10rXGAH65cA4=; b=LXCTK3e7gfD0PvStpjhu1nQg4aheVTdDc7/1m98OOuoXCswtGsHiTrUrTSPr8i8Y0Lpa6ItxK6C+nOEVZoE10Pjg4BDLKylgYvJUNDicHPYhVCfi0V150AQyLV2lB+rTdvJN6Le4TWisbJxEqGzaH0zEHZNLpZShcg0DFsb38NtlY1BBM+qGSF8rMx3pcrWRnTX5BFkiUxl4zMWGt1fPNc8lcO8XQgycrr+bxZ82EYP9SOWt3bwc9rwgwGT7M0Y9+ZUCdUfWg+1REPpB1AKGmoOliMV0suKICAC3tdETrA5Opcxc+vKINnIVruKcd2yZ5LNi8piJUcYhjhmOEBRh3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=M+gidTR64ir8kbPrzSZvcklZUAFYlf+10rXGAH65cA4=; b=lHrM+Am6khk4WVfK8w6iqX+l8y2EoXOMKHLbzlYYpH0rhuu50ayGjZjCIrprsiSm9TT6VWheiisFsF9502At9e6x98IkJE1mDKmwpbuE2EM6pzY0PVIlL/XFr2EFBSjQZitZ8DK/Tv5A0FVnilGTeXAbM4OF6IBOYykV0QjC85fGVFWv4H5roSaRZzlF+DnB54bVtXwJyJ//Ufb6GdKlEYYV2rmEymkNg5w4m8DtkHEfz6MdMHH+RVZeHT92YpLyIQrOF5yRRZUp5fjKiVcIdXImWYqORNAX1/3sHxYOicR1cg/E/IwzOIVs9G4E9phLPA5QVjGBzY0ngeL9TpP4Mw== Received: from BN9PR03CA0602.namprd03.prod.outlook.com (2603:10b6:408:106::7) by CY4PR1201MB0165.namprd12.prod.outlook.com (2603:10b6:910:1c::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.18; Wed, 22 Jun 2022 00:55:02 +0000 Received: from BN8NAM11FT063.eop-nam11.prod.protection.outlook.com (2603:10b6:408:106:cafe::63) by BN9PR03CA0602.outlook.office365.com (2603:10b6:408:106::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.16 via Frontend Transport; Wed, 22 Jun 2022 00:55:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.234) by BN8NAM11FT063.mail.protection.outlook.com (10.13.177.110) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5373.15 via Frontend Transport; Wed, 22 Jun 2022 00:55:02 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Wed, 22 Jun 2022 00:55:01 +0000 Received: from foundations-user-AS-2114GT-DNR-C1-NC24B.nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Tue, 21 Jun 2022 17:55:00 -0700 From: Kechen Lu To: , CC: , , , , , Subject: [RFC PATCH v4 6/7] KVM: x86: Add a new guest_debug flag forcing exit to userspace Date: Tue, 21 Jun 2022 17:49:23 -0700 Message-ID: <20220622004924.155191-7-kechenl@nvidia.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220622004924.155191-1-kechenl@nvidia.com> References: <20220622004924.155191-1-kechenl@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 64b50cf6-682e-4b07-952a-08da53e9d6c1 X-MS-TrafficTypeDiagnostic: CY4PR1201MB0165:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hAZ2TcOUEaj2vrKZfr1x8oE/gQsfkxqNlIB6wLqWShK4Znam17zawGtieg8IHDYQHZiax0DYywVOh8sdhM/g6UzELA9JGz8XEtYuJppmepRHaxAsr7nYiPq8agr12R8nObGxy8mDfP04TGqgnzngt3HgJTGiWYTpiFBqfhuCH+P2O4wt5ykQXK4ACjdWE4XD68pgnFJbFCL1sdUR9aAenVc16FXWXfCtm0NYq0IOiy9zIO4+rDmu5kZnjtP4KkkC7MuwIEB+ZMuqiifIyBmhAY0f5vJm1F4HP1uEaSmDPNeQqu7zqSIuodphF6UXHtCmD+3Cj7+dcWAiA5y6pJtvISZuSfn3d25tIOUhSrvK5WHoNhSNPw3pjdFy2GTpI35HE5X6vR66xuQVyKdMUGPoxFpguFklAeOkRqqzr6308IGM5dgobsCdQ2u4nROuI2n3yu/kJkQ/tF1lnrlESA+/E2r1tcf0/C+5Uiua7GS9ukPk/8BwwL/jtQ6BERCBmMAsrTWvtacZdSsSkix7xBiKLQnNj5wMXQV3rOnPMK9pWcnKQJl/NM3JfzcdmJcOubsikAJojnv4eeQ4pgHMQicpZ6jvwhkGGfsGfI0uQ7Sj7F6MNGXnJ0D441IUOXpUqBPNlRxdJcCEIhP/uyAc7VzOVRnYDGItuJrNjYSxFKsfqnXOrHQ6fefAnf7+ZVb7023Nor+/+RDu6cR5NJ0cTP6LI2AkpcKSZHmk6Bx5C0km/jTD3n7BHb4X4t49P04ICUul X-Forefront-Antispam-Report: CIP:12.22.5.234;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230016)(4636009)(136003)(39860400002)(346002)(376002)(396003)(40470700004)(46966006)(36840700001)(40460700003)(86362001)(36860700001)(82310400005)(5660300002)(81166007)(478600001)(8936002)(2906002)(316002)(356005)(70586007)(426003)(47076005)(54906003)(16526019)(8676002)(40480700001)(4326008)(70206006)(336012)(186003)(6666004)(83380400001)(82740400003)(7696005)(41300700001)(110136005)(2616005)(26005)(1076003)(36756003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jun 2022 00:55:02.0124 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 64b50cf6-682e-4b07-952a-08da53e9d6c1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.234];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT063.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1201MB0165 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org For debug and test purposes, there are needs to explicitly make instruction triggered exits could be trapped to userspace. Simply add a new flag for guest_debug interface could achieve this. This patch also fills the userspace accessible field vcpu->run->hw.hardware_exit_reason for userspace to determine the original triggered VM-exits. Signed-off-by: Kechen Lu --- arch/x86/kvm/svm/svm.c | 2 ++ arch/x86/kvm/vmx/vmx.c | 1 + arch/x86/kvm/x86.c | 2 ++ include/uapi/linux/kvm.h | 1 + tools/include/uapi/linux/kvm.h | 1 + 5 files changed, 7 insertions(+) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 7b3d64b3b901..e7ced6c3fbea 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -3259,6 +3259,8 @@ int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code) if (!svm_check_exit_valid(exit_code)) return svm_handle_invalid_exit(vcpu, exit_code); + vcpu->run->hw.hardware_exit_reason = exit_code; + #ifdef CONFIG_RETPOLINE if (exit_code == SVM_EXIT_MSR) return msr_interception(vcpu); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 2d000638cc9b..c32c20c4aa4d 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6151,6 +6151,7 @@ static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath) if (exit_reason.basic >= kvm_vmx_max_exit_handlers) goto unexpected_vmexit; + vcpu->run->hw.hardware_exit_reason = exit_reason.basic; #ifdef CONFIG_RETPOLINE if (exit_reason.basic == EXIT_REASON_MSR_WRITE) return kvm_emulate_wrmsr(vcpu); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 6165f0b046ed..91384a56ae0a 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8349,6 +8349,8 @@ int kvm_skip_emulated_instruction(struct kvm_vcpu *vcpu) */ if (unlikely(rflags & X86_EFLAGS_TF)) r = kvm_vcpu_do_singlestep(vcpu); + r &= !(vcpu->guest_debug & KVM_GUESTDBG_EXIT_USERSPACE); + return r; } EXPORT_SYMBOL_GPL(kvm_skip_emulated_instruction); diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index f2e76e436be5..23c335a6a285 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -777,6 +777,7 @@ struct kvm_s390_irq_state { #define KVM_GUESTDBG_ENABLE 0x00000001 #define KVM_GUESTDBG_SINGLESTEP 0x00000002 +#define KVM_GUESTDBG_EXIT_USERSPACE 0x00000004 struct kvm_guest_debug { __u32 control; diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h index 6a184d260c7f..373b4a2b7fe9 100644 --- a/tools/include/uapi/linux/kvm.h +++ b/tools/include/uapi/linux/kvm.h @@ -773,6 +773,7 @@ struct kvm_s390_irq_state { #define KVM_GUESTDBG_ENABLE 0x00000001 #define KVM_GUESTDBG_SINGLESTEP 0x00000002 +#define KVM_GUESTDBG_EXIT_USERSPACE 0x00000004 struct kvm_guest_debug { __u32 control; From patchwork Wed Jun 22 00:49:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kechen Lu X-Patchwork-Id: 12890006 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C189CC433EF for ; Wed, 22 Jun 2022 00:55:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1355651AbiFVAzS (ORCPT ); Tue, 21 Jun 2022 20:55:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355521AbiFVAzJ (ORCPT ); Tue, 21 Jun 2022 20:55:09 -0400 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2081.outbound.protection.outlook.com [40.107.101.81]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6487F3121F; Tue, 21 Jun 2022 17:55:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DItf89OdARekr+axdNaqbJCYdsLKGDFfjCSKHdQdf8f+2e0422hAyv3xGkR8ax5RKEPX5bVrfvVnQFRyyW+JO1BKCtT7QaOnjmhoVbRXrMdKGrOdPlyEG1Nx/Bu7aSi627chlGz1yHZ2VW1m/oHzZXn28NIZ2CB/1aqneTXGUrM1oKRHqVN7Wq2wvtDLpeeJ23VIDP6J1gC15w7BzBr0ynfic+SSjLDTDcrQ4Fj9XVthf+4D7zuMWZYVjVG4xurgHQuUOtiHFjPmcjg46ay01wK6+nW4Gge8CHaQtmemg7OSzZoKwZsXMNm73yD0K5+MSUR1lst9DgEwzRNotJQxQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=s6sM/iRwF74JFgrOO0JWhCmP5hQ8DNSTpvZcZ8gt6u4=; b=mFtGCchzJRnidZsxP5tl6LPWMeVkk5/4i29bk4dV7X3MqVqCHl5gLzY9+SqWTXqOEE2/fNrreNpE37CYUDdlTr863jZ5HV5uZQYBUAEIYQ/wADXbr7Xt6hTXryEXWTKtsv8hd77ThPnT5k18I6m9UZxw5zNpEYeoLAx94RrLKEWUlM8x3Y00Is9fEU6Nvy0RC0b9WSOCasnNR+LtOI/TgXvGMLBiiD3YM2t6aRxXO1UDieQcNFIVcV2Aj61tjSXpy+nAaG5N9tCs2qlWYfYvEcOol8eMXmrjv1xQ4woz2pIWGn1XaOwqy50GHm94NEG/GamU+TICN0J5/+ExUx9bxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=s6sM/iRwF74JFgrOO0JWhCmP5hQ8DNSTpvZcZ8gt6u4=; b=Jybnq8PzNJ8OXoNCylngpIXHWcCuxZ5x/Aga0Fegy3qbTquB8rdlpm6JcIKD7li1+LMSb5fWNWEFx3yyNBy0RN+UWygpr6/gXSBSFCkTnAJl6tzow1xJ711y3y0n/kTzwgj0016zxC6sGC5X7dne1mszNtZKypS1+5qYJHUkiuAfpSQ8VTzefnohJR+sCCcS1MkZt6k068uJAKmqJgUxgDWiqRRExs1lwlprb4gxboH3/bGwB0BUFm8kQ2VRfXdhNy+z1/t13IqRsU6UkVqUteeyZ3bETtAbHOsb1ep7tCA1Z9YUprUoQ8K/2xFIzlAulMvV7tm/cjWzIUplXqKKCg== Received: from BN6PR16CA0008.namprd16.prod.outlook.com (2603:10b6:404:f5::18) by BY5PR12MB4903.namprd12.prod.outlook.com (2603:10b6:a03:1d6::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.16; Wed, 22 Jun 2022 00:55:04 +0000 Received: from BN8NAM11FT038.eop-nam11.prod.protection.outlook.com (2603:10b6:404:f5:cafe::7f) by BN6PR16CA0008.outlook.office365.com (2603:10b6:404:f5::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5373.15 via Frontend Transport; Wed, 22 Jun 2022 00:55:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.236) by BN8NAM11FT038.mail.protection.outlook.com (10.13.176.246) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5353.14 via Frontend Transport; Wed, 22 Jun 2022 00:55:04 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Wed, 22 Jun 2022 00:55:03 +0000 Received: from foundations-user-AS-2114GT-DNR-C1-NC24B.nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Tue, 21 Jun 2022 17:55:02 -0700 From: Kechen Lu To: , CC: , , , , , Subject: [RFC PATCH v4 7/7] KVM: selftests: Add tests for VM and vCPU cap KVM_CAP_X86_DISABLE_EXITS Date: Tue, 21 Jun 2022 17:49:24 -0700 Message-ID: <20220622004924.155191-8-kechenl@nvidia.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220622004924.155191-1-kechenl@nvidia.com> References: <20220622004924.155191-1-kechenl@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bbe318e8-ddcc-40ae-8a6d-08da53e9d804 X-MS-TrafficTypeDiagnostic: BY5PR12MB4903:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VWzlnUNc3fZnO3sbaRkjAklTHYOEJd38ODbX98gLquszMmBz5y027v0PdduUhrf506Pul6roFlFrCRhIFTGomc2dnullkR95fkUBVs6zAFIhHdxu447muZZVrMlHP2fhqsarxYgbVHNrwvJogAgR9HP703sDo4MEXNyYh6gF07kVR48Y+EtN3vVVdp+9AhoSCyazMtV7DhXkb6NGaVktWMu2unx0OxTWL9SuPBOsjFkr32++FphdPZlSbPF6kvfMV7dmT4/VDWizNIVmqsn+t0m3Pfg3VXZuqWJf4rEfRuF6/BkYSKcyS6fygissyqx4/hNzpul5dLgjesAHSClQVpqDznY3xCvF5WbrSmFopAOLt2aoyGZ/SwECSgmyg2fmgNg2ppHteyeG8P7EFXQDY4jUONfNJWW1sfnaC3memuz5TL/W1F8ggpBdmttfxS7wbuY28o4hAoUe1xAjTneyehAGvbMl3jQz0b0/2Ex/o+aSvAwex8i1KPJdPMuwAAUlbLfdaup18lzaRboS5zaz/gS8hGk6HCt84EULP3Wvx4GVr+m9rf2lKPGN+iswSFCvFdUo9CfFv+LfBb0ziw8aXpv1XS3npnxIlVqiM8hKrGLlD4KKDVEZWZsBHcP4Qu3U1B8pw/np3ZWWQHn1oTR2uY6NqvDjknwoh1flyxCphPFQAqIo8rhWP1rmilH/PluGqIrixFQjI4SGOYENL4IfuMmMInTtvFXLiJN66LdBFuI/ZUG+mxBFQfWtakNyGaWk X-Forefront-Antispam-Report: CIP:12.22.5.236;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230016)(4636009)(39860400002)(396003)(376002)(136003)(346002)(40470700004)(36840700001)(46966006)(82740400003)(41300700001)(336012)(1076003)(4326008)(36860700001)(110136005)(316002)(2616005)(70586007)(426003)(356005)(8676002)(40460700003)(82310400005)(83380400001)(16526019)(40480700001)(36756003)(186003)(47076005)(86362001)(70206006)(26005)(81166007)(6666004)(8936002)(478600001)(54906003)(5660300002)(7696005)(2906002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jun 2022 00:55:04.1133 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bbe318e8-ddcc-40ae-8a6d-08da53e9d804 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.236];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT038.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4903 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add tests for KVM cap KVM_CAP_X86_DISABLE_EXITS overriding flags in VM and vCPU scope both works as expected. Suggested-by: Chao Gao Signed-off-by: Kechen Lu --- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/include/x86_64/svm_util.h | 1 + .../selftests/kvm/x86_64/disable_exits_test.c | 145 ++++++++++++++++++ 4 files changed, 148 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86_64/disable_exits_test.c diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore index 4509a3a7eeae..2b50170db9b2 100644 --- a/tools/testing/selftests/kvm/.gitignore +++ b/tools/testing/selftests/kvm/.gitignore @@ -15,6 +15,7 @@ /x86_64/cpuid_test /x86_64/cr4_cpuid_sync_test /x86_64/debug_regs +/x86_64/disable_exits_test /x86_64/evmcs_test /x86_64/emulator_error_test /x86_64/fix_hypercall_test diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index 22423c871ed6..de11d1f95700 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -115,6 +115,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/xen_shinfo_test TEST_GEN_PROGS_x86_64 += x86_64/xen_vmcall_test TEST_GEN_PROGS_x86_64 += x86_64/sev_migrate_tests TEST_GEN_PROGS_x86_64 += x86_64/amx_test +TEST_GEN_PROGS_x86_64 += x86_64/disable_exits_test TEST_GEN_PROGS_x86_64 += access_tracking_perf_test TEST_GEN_PROGS_x86_64 += demand_paging_test TEST_GEN_PROGS_x86_64 += dirty_log_test diff --git a/tools/testing/selftests/kvm/include/x86_64/svm_util.h b/tools/testing/selftests/kvm/include/x86_64/svm_util.h index a25aabd8f5e7..d8cad1cff578 100644 --- a/tools/testing/selftests/kvm/include/x86_64/svm_util.h +++ b/tools/testing/selftests/kvm/include/x86_64/svm_util.h @@ -17,6 +17,7 @@ #define CPUID_SVM BIT_ULL(CPUID_SVM_BIT) #define SVM_EXIT_MSR 0x07c +#define SVM_EXIT_HLT 0x078 #define SVM_EXIT_VMMCALL 0x081 struct svm_test_data { diff --git a/tools/testing/selftests/kvm/x86_64/disable_exits_test.c b/tools/testing/selftests/kvm/x86_64/disable_exits_test.c new file mode 100644 index 000000000000..2811b07e8885 --- /dev/null +++ b/tools/testing/selftests/kvm/x86_64/disable_exits_test.c @@ -0,0 +1,145 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Test per-VM and per-vCPU disable exits cap + * + */ + +#define _GNU_SOURCE /* for program_invocation_short_name */ +#include + +#include "test_util.h" +#include "kvm_util.h" +#include "svm_util.h" +#include "vmx.h" +#include "processor.h" + +#define VCPU_ID_1 0 +#define VCPU_ID_2 1 + +static void guest_code_exits(void) { + asm volatile("sti; hlt; cli"); +} + +/* Set debug control for trapped instruction exiting to userspace */ +static void vcpu_set_debug_exit_userspace(struct kvm_vm *vm, int vcpu_id) { + struct kvm_guest_debug debug; + memset(&debug, 0, sizeof(debug)); + debug.control = KVM_GUESTDBG_ENABLE | KVM_GUESTDBG_EXIT_USERSPACE; + vcpu_set_guest_debug(vm, vcpu_id, &debug); +} + +static void test_vm_cap_disable_exits(void) { + struct kvm_enable_cap cap = { + .cap = KVM_CAP_X86_DISABLE_EXITS, + .args[0] = KVM_X86_DISABLE_EXITS_HLT|KVM_X86_DISABLE_EXITS_OVERRIDE, + }; + struct kvm_vm *vm; + struct kvm_run *run; + + /* Create VM */ + vm = vm_create_without_vcpus(VM_MODE_DEFAULT, DEFAULT_GUEST_PHY_PAGES); + + /* Test Case #1 + * Default without disabling HLT exits in VM scope + */ + vm_vcpu_add_default(vm, VCPU_ID_1, (void *)guest_code_exits); + vcpu_set_debug_exit_userspace(vm, VCPU_ID_1); + run = vcpu_state(vm, VCPU_ID_1); + vcpu_run(vm, VCPU_ID_1); + /* Exit reason should be HLT */ + if (is_amd_cpu()) + TEST_ASSERT(run->hw.hardware_exit_reason == SVM_EXIT_HLT, + "Got exit_reason other than HLT: 0x%llx\n", + run->hw.hardware_exit_reason); + else + TEST_ASSERT(run->hw.hardware_exit_reason == EXIT_REASON_HLT, + "Got exit_reason other than HLT: 0x%llx\n", + run->hw.hardware_exit_reason); + + /* Test Case #2 + * Disabling HLT exits in VM scope + */ + vm_vcpu_add_default(vm, VCPU_ID_2, (void *)guest_code_exits); + vcpu_set_debug_exit_userspace(vm, VCPU_ID_2); + run = vcpu_state(vm, VCPU_ID_2); + /* Set VM scoped cap arg + * KVM_X86_DISABLE_EXITS_HLT|KVM_X86_DISABLE_EXITS_OVERRIDE + * after vCPUs creation so requiring override flag + */ + TEST_ASSERT(!vm_enable_cap(vm, &cap), "Failed to set KVM_CAP_X86_DISABLE_EXITS"); + vcpu_run(vm, VCPU_ID_2); + /* Exit reason should not be HLT, would finish the guest + * running and exit (e.g. SVM_EXIT_SHUTDOWN) + */ + if (is_amd_cpu()) + TEST_ASSERT(run->hw.hardware_exit_reason != SVM_EXIT_HLT, + "Got exit_reason as HLT: 0x%llx\n", + run->hw.hardware_exit_reason); + else + TEST_ASSERT(run->hw.hardware_exit_reason != EXIT_REASON_HLT, + "Got exit_reason as HLT: 0x%llx\n", + run->hw.hardware_exit_reason); + + kvm_vm_free(vm); +} + +static void test_vcpu_cap_disable_exits(void) { + struct kvm_enable_cap cap = { + .cap = KVM_CAP_X86_DISABLE_EXITS, + .args[0] = KVM_X86_DISABLE_EXITS_HLT|KVM_X86_DISABLE_EXITS_OVERRIDE, + }; + struct kvm_vm *vm; + struct kvm_run *run; + + /* Create VM */ + vm = vm_create_without_vcpus(VM_MODE_DEFAULT, DEFAULT_GUEST_PHY_PAGES); + vm_vcpu_add_default(vm, VCPU_ID_1, (void *)guest_code_exits); + vcpu_set_debug_exit_userspace(vm, VCPU_ID_1); + vm_vcpu_add_default(vm, VCPU_ID_2, (void *)guest_code_exits); + vcpu_set_debug_exit_userspace(vm, VCPU_ID_2); + /* Set vCPU 2 scoped cap arg + * KVM_X86_DISABLE_EXITS_HLT|KVM_X86_DISABLE_EXITS_OVERRIDE + */ + TEST_ASSERT(!vcpu_enable_cap(vm, VCPU_ID_2, &cap), "Failed to set KVM_CAP_X86_DISABLE_EXITS"); + + /* Test Case #3 + * Default without disabling HLT exits in this vCPU 1 + */ + run = vcpu_state(vm, VCPU_ID_1); + vcpu_run(vm, VCPU_ID_1); + /* Exit reason should be HLT */ + if (is_amd_cpu()) + TEST_ASSERT(run->hw.hardware_exit_reason == SVM_EXIT_HLT, + "Got exit_reason other than HLT: 0x%llx\n", + run->hw.hardware_exit_reason); + else + TEST_ASSERT(run->hw.hardware_exit_reason == EXIT_REASON_HLT, + "Got exit_reason other than HLT: 0x%llx\n", + run->hw.hardware_exit_reason); + + /* Test Case #4 + * Disabling HLT exits in vCPU 2 + */ + run = vcpu_state(vm, VCPU_ID_2); + vcpu_run(vm, VCPU_ID_2); + /* Exit reason should not be HLT, would finish the guest + * running and exit (e.g. SVM_EXIT_SHUTDOWN) + */ + if (is_amd_cpu()) + TEST_ASSERT(run->hw.hardware_exit_reason != SVM_EXIT_HLT, + "Got exit_reason as HLT: 0x%llx\n", + run->hw.hardware_exit_reason); + else + TEST_ASSERT(run->hw.hardware_exit_reason != EXIT_REASON_HLT, + "Got exit_reason as HLT: 0x%llx\n", + run->hw.hardware_exit_reason); + + kvm_vm_free(vm); +} + +int main(int argc, char *argv[]) +{ + test_vm_cap_disable_exits(); + test_vcpu_cap_disable_exits(); + return 0; +}