From patchwork Wed Dec 14 19:39:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Roth X-Patchwork-Id: 13073503 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 688F8C4332F for ; Wed, 14 Dec 2022 20:08:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 08EC98E001F; Wed, 14 Dec 2022 15:08:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 03E708E0002; Wed, 14 Dec 2022 15:08:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF9FA8E001F; Wed, 14 Dec 2022 15:08:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id D0E368E0002 for ; Wed, 14 Dec 2022 15:08:33 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id AB7EEAB659 for ; Wed, 14 Dec 2022 20:08:33 +0000 (UTC) X-FDA: 80241999306.20.74101FC Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2062.outbound.protection.outlook.com [40.107.94.62]) by imf26.hostedemail.com (Postfix) with ESMTP id D90B1140006 for ; Wed, 14 Dec 2022 20:08:30 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=1sNyg8++; spf=pass (imf26.hostedemail.com: domain of Michael.Roth@amd.com designates 40.107.94.62 as permitted sender) smtp.mailfrom=Michael.Roth@amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=quarantine) header.from=amd.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671048511; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4PugxjeKUReufCsaUR5iD8KRKurR+tDaJmIsUTovgDI=; b=UrJK8eHq/mwFOEnnUX07V+qJ3RwaKjKM17bMZ4ASERZxvhWFeQdii+kI6zyxM8PZKAg3LG za3riTay+qcl14CX5l8dpuFb/ny3Opsg2ZFh4RjS5mo7BmQnCVIisLtY9VT559Mouh7I9B 4oppUCxk6qbmOp8C76HN6nMdHKDejsI= ARC-Authentication-Results: i=2; imf26.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=1sNyg8++; spf=pass (imf26.hostedemail.com: domain of Michael.Roth@amd.com designates 40.107.94.62 as permitted sender) smtp.mailfrom=Michael.Roth@amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=quarantine) header.from=amd.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1671048511; a=rsa-sha256; cv=pass; b=6prg2Rjk+iPIdetVIvCBKgkqxZYWZFS/WyW+X0lj+mtHVJZywyzmT6dxGaaqGhoPh7aruj cYARHXnfY2qFGbeVeEQxqQtyC8FaGrdZ3FUXbbYmxKj7goGBu4ZWiMBTBUfoROGWrFAF73 2+s+8bzkwHnkLzM9z4n2MZcjoiRwVy0= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IW1LqFlxYaAZcf0tG59JNxLFJwnTJu7ESOecM4DkS5fpX884bakIZsnv0Wzrd3Wpzq+VuCyI88ulo11qM3cNamcV1z3pV9NMbQlcN5t5SMRCnQxDofPGkeIq/ERtCVuId9SPcQsaEhD6aqJGCHie4lU0CodkrIj1gpfLlYqNkemQWwtGHZ9GErLLT5luRE0rk111MNPowPwZXhgnEXJCbxYNEFcki18vSHgbKHfevD7y9XcOa6hcCqD1IIbgFIL2qvDHR76RFjJlQj0WR0MLWbmSXZHcsSm7QFZKxDLfrJvbifc153r3cLsU57bccc0Z0rmhYMBWSkLtsQt2f9u1iQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4PugxjeKUReufCsaUR5iD8KRKurR+tDaJmIsUTovgDI=; b=lp/r6lEs3PHiKF/N+d33K6MLeiMdlsnndLQ+tOieo85pWBIXpXiG1R8JiFWnE4Czj7wCMH2K/B8Yd14PUtE/bSjmcvypdD0hjr6Jh8ocwhYM9LqyjBOf/tLbm/mXmsrVHLL9Hxsl4rJQYA4Yqr+uOAZkB14JxOAUWBhR9ztSye7D1Ak+Nt/U5c8HpxUP1y/AjeWlMyRZU9x2/yvuvQwF/63sMgzsTqBt+auB176V5AyMrR41qQOJZGw5KAkJYrQ6KvRIIfrp6M6u5Zrha7/GBgIPSXRBPLXuk/WwRlZGom1kIlBRLB7n0ahu4kV21n076Fe2dRBVmuutO4IUDR2eNg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4PugxjeKUReufCsaUR5iD8KRKurR+tDaJmIsUTovgDI=; b=1sNyg8++djEc+T5MQZld18lcz2OhVnwGmcOxKHz46yWWZ2Oo8FEoL3aoDHXZYGJiHCJ7v3XkCXtC6IX72f3q2p9RqEqewnowPErBeuT6TtqvIqNi4MYd6F3pSsSO2GPwcYpPidcZtLobK048Pd4iAZ7WjT+j3S5CEG8vL/rI1AM= Received: from MW4P220CA0020.NAMP220.PROD.OUTLOOK.COM (2603:10b6:303:115::25) by SJ0PR12MB7066.namprd12.prod.outlook.com (2603:10b6:a03:4ae::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5880.19; Wed, 14 Dec 2022 20:08:27 +0000 Received: from CO1NAM11FT044.eop-nam11.prod.protection.outlook.com (2603:10b6:303:115:cafe::82) by MW4P220CA0020.outlook.office365.com (2603:10b6:303:115::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5924.11 via Frontend Transport; Wed, 14 Dec 2022 20:08:27 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT044.mail.protection.outlook.com (10.13.175.188) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5924.11 via Frontend Transport; Wed, 14 Dec 2022 20:08:27 +0000 Received: from localhost (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Wed, 14 Dec 2022 14:08:26 -0600 From: Michael Roth To: CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH RFC v7 06/64] KVM: x86: Add platform hooks for private memory invalidations Date: Wed, 14 Dec 2022 13:39:58 -0600 Message-ID: <20221214194056.161492-7-michael.roth@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221214194056.161492-1-michael.roth@amd.com> References: <20221214194056.161492-1-michael.roth@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT044:EE_|SJ0PR12MB7066:EE_ X-MS-Office365-Filtering-Correlation-Id: c3f862d7-ab99-45e8-cd93-08dade0ef6ae X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: g+y53TCxwyYfogXia3NgoNVSRPB5gpXUDeGtMMpLuFuVX0+hghRKLh7LJYPVurhKQFbm1cbkNerVh9y5RgwlBZ7IW+p68ZzhIeIMIJc9hrfVvrw3Lg0GUYfPwayMZ1NYJIJRNAhI7Pyk377XDq5shB5Sj8AMN2OtuoGvypoIXI9rQb/jEN46vPIWuqptorgONyq1n+3tHZsf0JnxxLcF5etB3R/skpyyNAz0mT7uwPiB6VYkuU23z/LHfB0M5rn/9arKU97bHlFyCsdbPFH48Lv1oH8NgYylyVSdMxjDSzNBZGqujp3RyB82olOxhYXYGJ3RpYh6FgF9y32ISswZoBELdexj07YKOLEYbI6BE41u+bXblqpbukUJnOVA8DrQI7O0LOi6qjtL4UIuetRafpliFcUJ5GDoBa0G9OFWa00Ntio8X885ACX8zTx/qggWZEyjfkCzEcEdX6DEm5IqvRsasERHNxt8GGtf5brQceQ8P2I6zI3bUmJeK5lSvn80FwEeNYMHkkoBSceXn/AAyg4HL8r6VW2PSQF44b2kpONI2ddbE2kczlaJ8k6r3JsDwjKiKUln/WwOyJ4MKT4mCmp3Z0zwFQLwtmiC2W9S2xRt1NcaAbQ/eU9SQkZnYntCggybhSl4Cq/KH9Gt9sVCeRwKJ4eOySh8wNdF55yTFbp9krr5E+szR0NMKzTP2C0eHhqvh66QumAk7RzyDAMtOStxunEs63Vc2zw7tnzL7fk= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:CA;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(39860400002)(376002)(346002)(396003)(136003)(451199015)(36840700001)(46966006)(40470700004)(7406005)(81166007)(356005)(36860700001)(86362001)(5660300002)(40460700003)(478600001)(7416002)(40480700001)(41300700001)(44832011)(6666004)(6916009)(8936002)(70206006)(4326008)(83380400001)(54906003)(316002)(47076005)(82310400005)(186003)(26005)(336012)(1076003)(70586007)(2616005)(82740400003)(8676002)(16526019)(426003)(2906002)(36756003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Dec 2022 20:08:27.4499 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c3f862d7-ab99-45e8-cd93-08dade0ef6ae X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT044.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB7066 X-Rspamd-Queue-Id: D90B1140006 X-Stat-Signature: g6mx8c73f9phdxyd7k83w3k1hj9hbhk9 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1671048510-312765 X-HE-Meta: U2FsdGVkX1/N6+ok0Uhu67u7r6ngId2sh90eNRnZxBL1d4eA9N77vCown16srFknChjE/FvDDBsQQzG0Fm28h/NA31BTUlRvuKuxPBEtD4ztmABpSUtNlr7ba0atorCGV9xt+klK8AfRJA/oIaJ9g6AZlhD8R/jrDkpRL3Qjt52DN5lsFziX3d4QGRnceG060nvzpjgbQ2iGf0kD59q02Jj5hKQsH7m6qfdFAPzSVUA8Mzlbm+uuvC0RgAtXvqVxlJvu6YeSxpIEzYSlsx9xc4XOEIQoolHWTFxpo2IYoDGs0J+Za3UCky2vv5rWGvwCJvCFpfIkfESZFLZO62gSr8RBIQOtzWMlVOXkul8c0vQ1NTKA2XyU3OCLHlt8LhTqxiFvwX8Yf/FiZSkMbFfwC0onz4rLIwY8/A0u2YRPMVuh0CWXRdKDR9+51CXqEshWbIPnEhYBV9E2z4vTxsz0zuM4tCx2gq9rvqOcruZtkGQC1RflidQLKebpB2uxra/Bo3hbfaZacSngBZnD/0UyiZFqBWnv+z1uvzGvGv0voHalz7fXZl0xI1Vnb4JJpt45xbp+6dSak8m0os/J6xAwBvNQkkmeoEaoEhB2eB+0z0Ich17R6AcV/fwKVCDwZ8Ai0reBIk4q8Vpdh1xMfFfK8n5INfKljoKa9MoySpacuIwhIr2xbuYICWaVTUmT+Dk+ORMFg0ZvXuOJyZ13FNmP9SzSH+91ic06zwxls9MZEP9MjATPqGgO5wGoCPBkic3ADO6SSMnctF+GwXZeGGCxJt/UjTgBObokA/4HcNHrGTGuf7P1Ec0RWnWO8tDazTURdx7tAeRzRybLHSSfdVNs7RvQ653z84WzZFwaBhPmZi0x/EMI9R4+kP87yJVn9qWYY3hslsnHXFLt1apfcHdukUV4vKvsHMjXQe23TlhEzW2uq9wk5wUpxyQ7N2RTXEB1rqKUskpW3+c6evtZWbs SiBk1RU9 Lf2/A X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In some cases, like with SEV-SNP, guest memory needs to be updated in a platform-specific manner before it can be safely freed back to the host. Add hooks to wire up handling of this sort to the invalidation notifiers for restricted memory. Also issue invalidations of all allocated pages during notifier unregistration so that the pages are not left in an unusable state when they eventually get freed back to the host upon FD release. Signed-off-by: Michael Roth --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 5 +++++ include/linux/kvm_host.h | 2 ++ mm/restrictedmem.c | 16 ++++++++++++++++ virt/kvm/kvm_main.c | 5 +++++ 6 files changed, 30 insertions(+) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index 52f94a0ba5e9..c71df44b0f02 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -134,6 +134,7 @@ KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); KVM_X86_OP_OPTIONAL_RET0(private_mem_enabled); KVM_X86_OP_OPTIONAL_RET0(fault_is_private); KVM_X86_OP_OPTIONAL_RET0(update_mem_attr) +KVM_X86_OP_OPTIONAL(invalidate_restricted_mem) #undef KVM_X86_OP #undef KVM_X86_OP_OPTIONAL diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 13802389f0f9..9ef8d73455d9 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1639,6 +1639,7 @@ struct kvm_x86_ops { int (*fault_is_private)(struct kvm *kvm, gpa_t gpa, u64 error_code, bool *private_fault); int (*update_mem_attr)(struct kvm_memory_slot *slot, unsigned int attr, gfn_t start, gfn_t end); + void (*invalidate_restricted_mem)(struct kvm_memory_slot *slot, gfn_t start, gfn_t end); bool (*has_wbinvd_exit)(void); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a0c41d391547..2713632e5061 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -7183,3 +7183,8 @@ void kvm_arch_set_memory_attributes(struct kvm *kvm, kvm_update_lpage_private_shared_mixed(kvm, slot, attrs, start, end); } + +void kvm_arch_invalidate_restricted_mem(struct kvm_memory_slot *slot, gfn_t start, gfn_t end) +{ + static_call_cond(kvm_x86_invalidate_restricted_mem)(slot, start, end); +} diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index f032d878e034..f72a2e0b8699 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2327,6 +2327,7 @@ void kvm_arch_set_memory_attributes(struct kvm *kvm, struct kvm_memory_slot *slot, unsigned long attrs, gfn_t start, gfn_t end); + #else static inline void kvm_arch_set_memory_attributes(struct kvm *kvm, struct kvm_memory_slot *slot, @@ -2366,6 +2367,7 @@ static inline int kvm_restricted_mem_get_pfn(struct kvm_memory_slot *slot, } void kvm_arch_memory_mce(struct kvm *kvm); +void kvm_arch_invalidate_restricted_mem(struct kvm_memory_slot *slot, gfn_t start, gfn_t end); #endif /* CONFIG_HAVE_KVM_RESTRICTED_MEM */ #endif diff --git a/mm/restrictedmem.c b/mm/restrictedmem.c index 56953c204e5c..74fa2cfb8618 100644 --- a/mm/restrictedmem.c +++ b/mm/restrictedmem.c @@ -54,6 +54,11 @@ static int restrictedmem_release(struct inode *inode, struct file *file) { struct restrictedmem_data *data = inode->i_mapping->private_data; + pr_debug("%s: releasing memfd, invalidating page offsets 0x0-0x%llx\n", + __func__, inode->i_size >> PAGE_SHIFT); + restrictedmem_invalidate_start(data, 0, inode->i_size >> PAGE_SHIFT); + restrictedmem_invalidate_end(data, 0, inode->i_size >> PAGE_SHIFT); + fput(data->memfd); kfree(data); return 0; @@ -258,6 +263,17 @@ void restrictedmem_unregister_notifier(struct file *file, struct restrictedmem_notifier *notifier) { struct restrictedmem_data *data = file->f_mapping->private_data; + struct inode *inode = file_inode(data->memfd); + + /* TODO: this will issue notifications to all registered notifiers, + * but it's only the one being unregistered that needs to process + * invalidations for any ranges still allocated at this point in + * time. For now this relies on KVM currently being the only notifier. + */ + pr_debug("%s: unregistering notifier, invalidating page offsets 0x0-0x%llx\n", + __func__, inode->i_size >> PAGE_SHIFT); + restrictedmem_invalidate_start(data, 0, inode->i_size >> PAGE_SHIFT); + restrictedmem_invalidate_end(data, 0, inode->i_size >> PAGE_SHIFT); mutex_lock(&data->lock); list_del(¬ifier->list); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index d2d829d23442..d2daa049e94a 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -974,6 +974,9 @@ static void kvm_restrictedmem_invalidate_begin(struct restrictedmem_notifier *no &gfn_start, &gfn_end)) return; + pr_debug("%s: start: 0x%lx, end: 0x%lx, roffset: 0x%llx, gfn_start: 0x%llx, gfn_end: 0x%llx\n", + __func__, start, end, slot->restricted_offset, gfn_start, gfn_end); + gfn_range.start = gfn_start; gfn_range.end = gfn_end; gfn_range.slot = slot; @@ -988,6 +991,8 @@ static void kvm_restrictedmem_invalidate_begin(struct restrictedmem_notifier *no if (kvm_unmap_gfn_range(kvm, &gfn_range)) kvm_flush_remote_tlbs(kvm); + kvm_arch_invalidate_restricted_mem(slot, gfn_start, gfn_end); + KVM_MMU_UNLOCK(kvm); srcu_read_unlock(&kvm->srcu, idx); }