From patchwork Wed Sep 20 14:02:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankit Agrawal X-Patchwork-Id: 13392825 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51CD8CE79D1 for ; Wed, 20 Sep 2023 14:03:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A770E6B0173; Wed, 20 Sep 2023 10:03:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A27B66B0174; Wed, 20 Sep 2023 10:03:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C7776B0175; Wed, 20 Sep 2023 10:03:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 7B6A06B0173 for ; Wed, 20 Sep 2023 10:03:33 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 296A9120633 for ; Wed, 20 Sep 2023 14:03:33 +0000 (UTC) X-FDA: 81257143506.21.BE08F10 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2072.outbound.protection.outlook.com [40.107.244.72]) by imf01.hostedemail.com (Postfix) with ESMTP id 76FDF4032B for ; Wed, 20 Sep 2023 14:02:49 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=qc9yeCCT; spf=pass (imf01.hostedemail.com: domain of ankita@nvidia.com designates 40.107.244.72 as permitted sender) smtp.mailfrom=ankita@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695218570; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1/esQzqxfrhDN5pKRJQJwHN3r7WLnGtT5aODQpWge2I=; b=WAPRc613sxBuKkxA9cQCEVcpmqUDaHGKSpC7LfoocB7J0tXdYBPxUdRdy2JnKCKv74ruDL KT0z0qOLdPEd39qjE0fetctoFfoKntTrvwMMlxU7dMSAlacDi2U/L8KdrFejcavZtt6yhT fKaSmpDFwSPiOAE5m+Eg/X7d/zpj3GI= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1695218570; a=rsa-sha256; cv=pass; b=4FeD8Y3cuj1fs4XSG0AVBtoMy3sQ6ZmEmoY6Y9iL0reHxbOEhSs6gZbDi8q69hom0nuwXC 6WlvP+xkgZwesjmRPedBkm1t7gSfdOsqbpMTfAGuSBef/2YoYhpov07Cnr8IOQ6q7R/t6n ebKMfN5eW96I5/QqtI6IgtZWMU7Y0YI= ARC-Authentication-Results: i=2; imf01.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=qc9yeCCT; spf=pass (imf01.hostedemail.com: domain of ankita@nvidia.com designates 40.107.244.72 as permitted sender) smtp.mailfrom=ankita@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OVaTSCbfPXN4Oh4u3RPpFjsccPAKhHwXzDIIrf+hC21etT/xAC97c5mT+m0RdtjXCIsUjybMLfuJi40cKjs+zeseR37a1x9ylQ2M7J7/Rw4MwwSIKX4MKI2O7DYnQ8lxCD6xhRyRK+hsVA5TAeBIqrRBLtpDpUWWRn5jypLC8YMG7DkBZNYtwVgWGX1Txw7FcCQ2LJtwvzF1fb9EpcFKPhlt+ivXMXcbhvWfVyR9wRJwMoTqg6Ml3mz2aSxgm0DtLKanljt0k1NbqXunhYrxIFzBqiQhxyFEtSVjF0TC61Yi0OObMGygctNGjof3s4mlstHhnHaJUBa1MfNNXnwD2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1/esQzqxfrhDN5pKRJQJwHN3r7WLnGtT5aODQpWge2I=; b=ZPbDVi6gq8ENdomvUkHpsIMeVkqER+QiMV0vtqs3wtfo8jTASME9PTlbFr+SP7c5NgEnTLWXMwZ868sseU+VYqNTRR31kkGhe6WJZbM8TnKdwgFssu52XejCXP4Y1y7W5Gm7cdWJ3+EwCgC4Z9N38apY2RAOPK8WW2NOGF7eytWG9HkHF/9f2BOqOa1xJlelDkMNW7P1cYNJV6dDrGnWlyCgdnOTcXgMyWr72MG8t9pFupj0d9pZQZCY2bXmyT7EMK2KGSeJiiLmHvRtPFfnJLW8r1gw5Zewd7G/lhTkxbpZFA5jCCN5Qp9AK9QbDgE+PY7rhSr1AODh7GRH/Q0dlg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=kvack.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1/esQzqxfrhDN5pKRJQJwHN3r7WLnGtT5aODQpWge2I=; b=qc9yeCCTr1MtLoD9tT0Su3rNNOEfS44Rta1ZxI3/znmbc1mLNdWosGMoHk0+E3bXP52mJ5+HysIuYxn8bvw7hUHly8k23jZVMpScXCpcbC1UeWqAnjaeL3B1EBgKYqndsw2pOx5gAUiTzjFvhEqKY86g54AwhMmqg/z40UrSvFRFZwyM87e1UMBNcNAHz0iCTu8wmb4EKkWDnnw31NxiQTbGQzUjq5P6E0zNuHzOSxI7LykTObG1H4QJ+6cA2VtgLMg99yMUhwKwr9aVLe1Y3XfJovalUCsOPN1wojOX1sY7oE8n11ywD7B846iUuRhi2oLBWKUeJk3AIc0NghPidg== Received: from DM5PR07CA0070.namprd07.prod.outlook.com (2603:10b6:4:ad::35) by SN7PR12MB6931.namprd12.prod.outlook.com (2603:10b6:806:261::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.27; Wed, 20 Sep 2023 14:02:38 +0000 Received: from DS2PEPF00003444.namprd04.prod.outlook.com (2603:10b6:4:ad:cafe::a7) by DM5PR07CA0070.outlook.office365.com (2603:10b6:4:ad::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.28 via Frontend Transport; Wed, 20 Sep 2023 14:02:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003444.mail.protection.outlook.com (10.167.17.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.19 via Frontend Transport; Wed, 20 Sep 2023 14:02:37 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Wed, 20 Sep 2023 07:02:24 -0700 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Wed, 20 Sep 2023 07:02:24 -0700 Received: from sgarnayak-dt.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41 via Frontend Transport; Wed, 20 Sep 2023 07:02:18 -0700 From: To: , , , , , , , CC: , , , , , , , , , , Subject: [PATCH v1 1/4] mm: handle poisoning of pfn without struct pages Date: Wed, 20 Sep 2023 19:32:07 +0530 Message-ID: <20230920140210.12663-2-ankita@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230920140210.12663-1-ankita@nvidia.com> References: <20230920140210.12663-1-ankita@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003444:EE_|SN7PR12MB6931:EE_ X-MS-Office365-Filtering-Correlation-Id: 21ab9138-4687-41fc-ce88-08dbb9e23f32 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Qs+0LhfxU7AbH5bsjHByhHm+4xGwWndLWXGec1sGCvY8/wrLqw+nw1JpCJIdEPd2HFuxT3hwDqcgZG6MKOevvf9RhXWQEQzBIL52KdZH7dh6mLALIHWPW/Vpbj+MuYI+7ULxtQK9PNmau0An1+m+7weH+/6h21I+06fTtP6Y8t5pOALMAH9UAVDuGXIPzdya6Hf6PoQnf7+/lxNqmIkorvShm9KVXijmf2lUMfMPic/BZe7isVn7ytR1zA4OQkOMxy7/+79mk//ZJ3OkKyCFoNy637chJA44isLfN9qHo7Jwbm78NBM6Ipgq9mELPQ9nTb2BueJSqlgvxuP+N7Yrza0tO1Lz6ZnKrEVMUV4E1dUoLsIJ2UoAmwsA9/U4NX+dIZV7XXWN78y3j1d0gLmyFzmUwjnZiEdzqkx6oeUfEEAaViskRmiXWdDMNZCY16XmJqqIfp8YPH7NuTsN0u35uS9m3fRDeecWNE4WMBRtQx8pBuD9j4C92CtLo/PuphCFZ6WuccXidg0xBhhn72GOVCQTv8X2WSwfupg8aTguBPfLHi2IoYDMDyPFUEgK6iX+yPC/JxpG7VG5CU7aIZjWIkJqN3YgdejemboR5/6EjGxcf6FSL1C+78VbBGmOaKJZOIAmxo5L+eA0ek9tF8qWfl8s2JX8z1U0ZsLpbwX4dEAxEPEH+cRd61r09gsaUYyUxxtF2ZJKlQ7bJQ56qUfOgfr2n8oMyYgnNuli+r8jD74KzNWg0ng0GiTeLPLOYEnseYW/Dr2mAnlJlSJ5FM9duQ== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(136003)(346002)(396003)(376002)(39860400002)(186009)(82310400011)(1800799009)(451199024)(40470700004)(46966006)(36840700001)(7636003)(356005)(2616005)(82740400003)(8936002)(26005)(40460700003)(1076003)(4326008)(8676002)(7416002)(2906002)(2876002)(36860700001)(83380400001)(47076005)(36756003)(30864003)(426003)(336012)(5660300002)(86362001)(7696005)(110136005)(40480700001)(6666004)(54906003)(478600001)(70586007)(316002)(41300700001)(70206006)(2101003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2023 14:02:37.5536 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 21ab9138-4687-41fc-ce88-08dbb9e23f32 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003444.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB6931 X-Rspamd-Queue-Id: 76FDF4032B X-Rspam-User: X-Stat-Signature: ppgg3mtkjtrh3i43c9ou6bhwrqnpkpao X-Rspamd-Server: rspam03 X-HE-Tag: 1695218569-822908 X-HE-Meta: U2FsdGVkX18KEFUyTEFXuak0ygEvKPU5MC96BU8M+3uws7ZdafXvZTMsfauMNZV7nQgaJJp9GvpkXNH1lLUXcKDAwgCG3Oz70ZGrwOhG48//gedbu8u2u9NY99J8/Cu6E9FCbqtkcbauWeI856qOQwOnu2tNeu/pLrnW+fEm7FqnbR/eu/HPlPPFvx/t35FAqX8qJvAeuFfsTOk+qTn/+jvFCPMG1M+se9+57aYXkmiv7B2mVOPq4ldXbZwt3ZK2PgIw7pPUohAw6rC9qoLUtE8L+ZvLbj7CyAhEbSwRh22mUhdHqGlloAwq9MzX48pen9YJxhzIsmxEFDuaKCb54bRBSi6bx4Xc8mYS1PN7ZLX8wdSQ8Qr8SGZjf8M4CzzpOrjc1M5PJgq0uaJHbWtN1sQ1dG1tCJxWTcR1cAN8SOdj0SiVR9mL7gWaHrTFrbcOL9IoZLe53XlTFDWHzwz9lGsfoNQRgjXXX9voXg4dfp3gPArLnQurFBZ1qTB+ragO1WR1NM2DReW04dTLXfWmMJkhVWw/knQ75WcMJQuhMqzHfz2y+9Nsv2qkjgKZp1IGLcg1ZDqBqkmcKTDRghiyJKKpBYhRnEwdU48IG0ViVtpgpdMjksOTVPZvzsKrTwtb41sDROArmd5hwS1m15N2O/b8NGN0CpaOAC7Yel2QUHvXAskenWq5Ce7jRP1c5UpXNpDLffcZs+Oh9qEjEkZkE4NCD/pXdnKVTNVc38ZcZplKxX1SLyNSn9zYZudFnIBxNi9ijrs+//4stms73WuP7D34fTu7TKM49wtrkE92K9+XPBn8JGIA1boFFo65FLbt2p2UIeqWZ9ZTIxlJWCxeAPRZ6nCdnyeT3gCS5E3swa/f6jx2RG49PT2kCkKUFOHkfA6wKU/QCXa66a7Eld1LwxhzEDnScXSwCpFfjsZPn1O1kDJddAsmSxOY/WT48HWNBFAFTLnb/qj2h6ThfGu ynC8CZh+ ztBJI6WrNqLyfvB6/Zhk41DUvJwfq/bDV7Lxzu6zP00amfz+b5klrThIN9xaRxYZhQf/V1d4/ud9VYQeGzx+buS0/Ye4iLzpPqfpyNOe3TLuENSuxjtMHiUx4/9nPJ7ctTbbqdMlGAt8xmvN8WjnPOBkqAi2ChffEHm3Z0HU8353zjipM6XiWcFirdjX7KVlhbxGWtiHhpvyl2BrBKSFrjYqF12/l0dRJuD6f X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Ankit Agrawal The kernel MM currently does not handle ECC errors / poison on a memory region that is not backed by struct pages. If a memory region is mapped using remap_pfn_range(), but not added to the kernel, MM will not have associated struct pages. Add a new mechanism to handle memory failure on such memory. Make kernel MM expose a function to allow modules managing the device memory to register a failure function and the physical address space associated with the device memory. MM maintains this information as interval tree. The registered memory failure function is used by MM to notify the kernel module managing the PFN, so that the module may take any required action. The module for example may use the information to track the poisoned pages. In this implementation, kernel MM follows the following sequence similar (mostly) to the memory_failure() handler for struct page backed memory: 1. memory_failure() is triggered on reception of a poison error. An absence of struct page is detected and consequently memory_failure_pfn() is executed. 2. memory_failure_pfn() call the newly introduced failure handler exposed by the module managing the poisoned memory to notify it of the problematic PFN. 3. memory_failure_pfn() unmaps the stage-2 mapping to the PFN. 4. memory_failure_pfn() collects the processes mapped to the PFN. 5. memory_failure_pfn() sends SIGBUS (BUS_MCEERR_AO) to all the processes mapping the faulty PFN using kill_procs(). 6. An access to the faulty PFN by an operation in VM at a later point is trapped and user_mem_abort() is called. 7. The vma ops fault function gets called due to the absence of Stage-2 mapping. It is expected to return VM_FAULT_HWPOISON on the PFN. 8. __gfn_to_pfn_memslot() then returns KVM_PFN_ERR_HWPOISON, which cause the poison with SIGBUS (BUS_MCEERR_AR) to be sent to the QEMU process through kvm_send_hwpoison_signal(). Signed-off-by: Ankit Agrawal --- include/linux/memory-failure.h | 22 ++++++ include/linux/mm.h | 1 + include/ras/ras_event.h | 1 + mm/Kconfig | 1 + mm/memory-failure.c | 135 ++++++++++++++++++++++++++++----- 5 files changed, 139 insertions(+), 21 deletions(-) create mode 100644 include/linux/memory-failure.h diff --git a/include/linux/memory-failure.h b/include/linux/memory-failure.h new file mode 100644 index 000000000000..9a579960972a --- /dev/null +++ b/include/linux/memory-failure.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_MEMORY_FAILURE_H +#define _LINUX_MEMORY_FAILURE_H + +#include + +struct pfn_address_space; + +struct pfn_address_space_ops { + void (*failure)(struct pfn_address_space *pfn_space, unsigned long pfn); +}; + +struct pfn_address_space { + struct interval_tree_node node; + const struct pfn_address_space_ops *ops; + struct address_space *mapping; +}; + +int register_pfn_address_space(struct pfn_address_space *pfn_space); +void unregister_pfn_address_space(struct pfn_address_space *pfn_space); + +#endif /* _LINUX_MEMORY_FAILURE_H */ diff --git a/include/linux/mm.h b/include/linux/mm.h index bf5d0b1b16f4..d677688c016c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3934,6 +3934,7 @@ enum mf_action_page_type { MF_MSG_BUDDY, MF_MSG_DAX, MF_MSG_UNSPLIT_THP, + MF_MSG_PFN_MAP, MF_MSG_UNKNOWN, }; diff --git a/include/ras/ras_event.h b/include/ras/ras_event.h index cbd3ddd7c33d..05c3e6f6bd02 100644 --- a/include/ras/ras_event.h +++ b/include/ras/ras_event.h @@ -373,6 +373,7 @@ TRACE_EVENT(aer_event, EM ( MF_MSG_BUDDY, "free buddy page" ) \ EM ( MF_MSG_DAX, "dax page" ) \ EM ( MF_MSG_UNSPLIT_THP, "unsplit thp" ) \ + EM ( MF_MSG_PFN_MAP, "non struct page pfn" ) \ EMe ( MF_MSG_UNKNOWN, "unknown page" ) /* diff --git a/mm/Kconfig b/mm/Kconfig index 264a2df5ecf5..2ee42ff8b6ca 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -762,6 +762,7 @@ config MEMORY_FAILURE depends on ARCH_SUPPORTS_MEMORY_FAILURE bool "Enable recovery from hardware memory errors" select MEMORY_ISOLATION + select INTERVAL_TREE select RAS help Enables code to recover from some memory failures on systems diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 4d6e43c88489..e1e1d96fd6a2 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -38,6 +38,7 @@ #include #include +#include #include #include #include @@ -60,6 +61,7 @@ #include #include #include +#include #include "swap.h" #include "internal.h" #include "ras/ras_event.h" @@ -144,6 +146,10 @@ static struct ctl_table memory_failure_table[] = { { } }; +static struct rb_root_cached pfn_space_itree = RB_ROOT_CACHED; + +static DEFINE_MUTEX(pfn_space_lock); + /* * Return values: * 1: the page is dissolved (if needed) and taken off from buddy, @@ -422,15 +428,15 @@ static unsigned long dev_pagemap_mapping_shift(struct vm_area_struct *vma, * Schedule a process for later kill. * Uses GFP_ATOMIC allocations to avoid potential recursions in the VM. * - * Note: @fsdax_pgoff is used only when @p is a fsdax page and a - * filesystem with a memory failure handler has claimed the - * memory_failure event. In all other cases, page->index and - * page->mapping are sufficient for mapping the page back to its + * Notice: @pgoff is used either when @p is a fsdax page or a PFN is not + * backed by struct page and a filesystem with a memory failure handler + * has claimed the memory_failure event. In all other cases, page->index + * and page->mapping are sufficient for mapping the page back to its * corresponding user virtual address. */ static void __add_to_kill(struct task_struct *tsk, struct page *p, struct vm_area_struct *vma, struct list_head *to_kill, - unsigned long ksm_addr, pgoff_t fsdax_pgoff) + unsigned long ksm_addr, pgoff_t pgoff) { struct to_kill *tk; @@ -440,13 +446,18 @@ static void __add_to_kill(struct task_struct *tsk, struct page *p, return; } - tk->addr = ksm_addr ? ksm_addr : page_address_in_vma(p, vma); - if (is_zone_device_page(p)) { - if (fsdax_pgoff != FSDAX_INVALID_PGOFF) - tk->addr = vma_pgoff_address(fsdax_pgoff, 1, vma); - tk->size_shift = dev_pagemap_mapping_shift(vma, tk->addr); - } else - tk->size_shift = page_shift(compound_head(p)); + if (vma->vm_flags | PFN_MAP) { + tk->addr = vma_pgoff_address(pgoff, 1, vma); + tk->size_shift = PAGE_SHIFT; + } else { + tk->addr = ksm_addr ? ksm_addr : page_address_in_vma(p, vma); + if (is_zone_device_page(p)) { + if (pgoff != FSDAX_INVALID_PGOFF) + tk->addr = vma_pgoff_address(pgoff, 1, vma); + tk->size_shift = dev_pagemap_mapping_shift(vma, tk->addr); + } else + tk->size_shift = page_shift(compound_head(p)); + } /* * Send SIGKILL if "tk->addr == -EFAULT". Also, as @@ -666,8 +677,7 @@ static void collect_procs_file(struct page *page, struct list_head *to_kill, i_mmap_unlock_read(mapping); } -#ifdef CONFIG_FS_DAX -static void add_to_kill_fsdax(struct task_struct *tsk, struct page *p, +static void add_to_kill_pgoff(struct task_struct *tsk, struct page *p, struct vm_area_struct *vma, struct list_head *to_kill, pgoff_t pgoff) { @@ -677,9 +687,9 @@ static void add_to_kill_fsdax(struct task_struct *tsk, struct page *p, /* * Collect processes when the error hit a fsdax page. */ -static void collect_procs_fsdax(struct page *page, - struct address_space *mapping, pgoff_t pgoff, - struct list_head *to_kill) +static void collect_procs_pgoff(struct page *page, + struct address_space *mapping, pgoff_t pgoff, + struct list_head *to_kill) { struct vm_area_struct *vma; struct task_struct *tsk; @@ -693,13 +703,12 @@ static void collect_procs_fsdax(struct page *page, continue; vma_interval_tree_foreach(vma, &mapping->i_mmap, pgoff, pgoff) { if (vma->vm_mm == t->mm) - add_to_kill_fsdax(t, page, vma, to_kill, pgoff); + add_to_kill_pgoff(t, page, vma, to_kill, pgoff); } } rcu_read_unlock(); i_mmap_unlock_read(mapping); } -#endif /* CONFIG_FS_DAX */ /* * Collect the processes who have the corrupted page mapped to kill. @@ -893,6 +902,7 @@ static const char * const action_page_types[] = { [MF_MSG_BUDDY] = "free buddy page", [MF_MSG_DAX] = "dax page", [MF_MSG_UNSPLIT_THP] = "unsplit thp", + [MF_MSG_PFN_MAP] = "non struct page pfn", [MF_MSG_UNKNOWN] = "unknown page", }; @@ -1324,7 +1334,8 @@ static int action_result(unsigned long pfn, enum mf_action_page_type type, num_poisoned_pages_inc(pfn); - update_per_node_mf_stats(pfn, result); + if (type != MF_MSG_PFN_MAP) + update_per_node_mf_stats(pfn, result); pr_err("%#lx: recovery action for %s: %s\n", pfn, action_page_types[type], action_name[result]); @@ -1805,7 +1816,7 @@ int mf_dax_kill_procs(struct address_space *mapping, pgoff_t index, SetPageHWPoison(page); - collect_procs_fsdax(page, mapping, index, &to_kill); + collect_procs_pgoff(page, mapping, index, &to_kill); unmap_and_kill(&to_kill, page_to_pfn(page), mapping, index, mf_flags); unlock: @@ -2144,6 +2155,83 @@ static int memory_failure_dev_pagemap(unsigned long pfn, int flags, return rc; } +int register_pfn_address_space(struct pfn_address_space *pfn_space) +{ + if (!pfn_space) + return -EINVAL; + + if (!request_mem_region(pfn_space->node.start << PAGE_SHIFT, + (pfn_space->node.last - pfn_space->node.start + 1) << PAGE_SHIFT, "")) + return -EBUSY; + + mutex_lock(&pfn_space_lock); + interval_tree_insert(&pfn_space->node, &pfn_space_itree); + mutex_unlock(&pfn_space_lock); + + return 0; +} +EXPORT_SYMBOL_GPL(register_pfn_address_space); + +void unregister_pfn_address_space(struct pfn_address_space *pfn_space) +{ + if (!pfn_space) + return; + + mutex_lock(&pfn_space_lock); + interval_tree_remove(&pfn_space->node, &pfn_space_itree); + mutex_unlock(&pfn_space_lock); + release_mem_region(pfn_space->node.start << PAGE_SHIFT, + (pfn_space->node.last - pfn_space->node.start + 1) << PAGE_SHIFT); +} +EXPORT_SYMBOL_GPL(unregister_pfn_address_space); + +static int memory_failure_pfn(unsigned long pfn, int flags) +{ + struct interval_tree_node *node; + int res = MF_FAILED; + LIST_HEAD(tokill); + + mutex_lock(&pfn_space_lock); + /* + * Modules registers with MM the address space mapping to the device memory they + * manage. Iterate to identify exactly which address space has mapped to this + * failing PFN. + */ + for (node = interval_tree_iter_first(&pfn_space_itree, pfn, pfn); node; + node = interval_tree_iter_next(node, pfn, pfn)) { + struct pfn_address_space *pfn_space = + container_of(node, struct pfn_address_space, node); + /* + * Modules managing the device memory need to be conveyed about the + * memory failure so that the poisoned PFN can be tracked. + */ + if (pfn_space->ops) + pfn_space->ops->failure(pfn_space, pfn); + + collect_procs_pgoff(NULL, pfn_space->mapping, pfn, &tokill); + + unmap_mapping_range(pfn_space->mapping, pfn << PAGE_SHIFT, + PAGE_SIZE, 0); + + res = MF_RECOVERED; + } + mutex_unlock(&pfn_space_lock); + + if (res == MF_FAILED) + return action_result(pfn, MF_MSG_PFN_MAP, res); + + /* + * Unlike System-RAM there is no possibility to swap in a different + * physical page at a given virtual address, so all userspace + * consumption of direct PFN memory necessitates SIGBUS (i.e. + * MF_MUST_KILL) + */ + flags |= MF_ACTION_REQUIRED | MF_MUST_KILL; + kill_procs(&tokill, true, false, pfn, flags); + + return action_result(pfn, MF_MSG_PFN_MAP, MF_RECOVERED); +} + /** * memory_failure - Handle memory failure of a page. * @pfn: Page Number of the corrupted page @@ -2183,6 +2271,11 @@ int memory_failure(unsigned long pfn, int flags) if (!(flags & MF_SW_SIMULATED)) hw_memory_failure = true; + if (!pfn_valid(pfn) && !arch_is_platform_page(PFN_PHYS(pfn))) { + res = memory_failure_pfn(pfn, flags); + goto unlock_mutex; + } + p = pfn_to_online_page(pfn); if (!p) { res = arch_memory_failure(pfn, flags); From patchwork Wed Sep 20 14:02:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankit Agrawal X-Patchwork-Id: 13392826 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11851CE79D1 for ; Wed, 20 Sep 2023 14:03:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 93E986B0175; Wed, 20 Sep 2023 10:03:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8EE996B0176; Wed, 20 Sep 2023 10:03:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B6AE6B0177; Wed, 20 Sep 2023 10:03:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 686C46B0175 for ; Wed, 20 Sep 2023 10:03:49 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 4A5731405D8 for ; Wed, 20 Sep 2023 14:03:49 +0000 (UTC) X-FDA: 81257144178.05.931FB9B Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2044.outbound.protection.outlook.com [40.107.244.44]) by imf19.hostedemail.com (Postfix) with ESMTP id 1226D1A0186 for ; Wed, 20 Sep 2023 14:02:49 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=Y9GzWAjk; spf=pass (imf19.hostedemail.com: domain of ankita@nvidia.com designates 40.107.244.44 as permitted sender) smtp.mailfrom=ankita@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695218571; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yIG/Gq5YZPbCV92/HN+i3wEWEJiFnj6txDrJHePjssI=; b=M3hN+U4pGnpO1MuDPCMptm3alufzTaXO7USS0pdfBWFPGdrhuvK+V8BWxbx8i6Y406+8Nf qJfCMMVdgk4amDtk2SYNXONS7eq931JZH6vIykde9DZshfTI94XObFI7rLXrsg4AU/bDGm 8Zce7pX1H+pX6E61YKtefSZqfNXcYA0= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1695218571; a=rsa-sha256; cv=pass; b=cgrOzDYQHxt14LGp/Rr4E8GUZjLuKR1RWdkO8f0Emgw2pNyTkJiEvWzYxG28mxXC6o0dAN fxfkwGgQsqjkSR8v1lcuHJH5Le0dw9himIn9G/KEBwET7q1hOfwqxLtThV+zVUybdV0aCm AraYdxBSzOMB1uqy5UTiNQancFauqaw= ARC-Authentication-Results: i=2; imf19.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=Y9GzWAjk; spf=pass (imf19.hostedemail.com: domain of ankita@nvidia.com designates 40.107.244.44 as permitted sender) smtp.mailfrom=ankita@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fZLxeAbGMoe0MueMDiUeG7cAr3ofxm+KiMY0ulFs3FwJ8IHlNQ2dR615VoVjbFKloJxn0AVsfhWqp+dz5WTroCVwOsanITYQxg65cgGXqkNXdgg/NHpLHJ5He1pIz30HI1c7Xnf+XaZgjXSf47uULXZMpRwiQPgsw7+8xTqP12mhlrN+6ZyhLzWXjtRZO+n/yOAI8rcNCCVjX8Mv8fIGM6ipsckLNrGtu9LluTFSbfDbmw++ihPFeBwDMRtQhd3JUJE9yE2bYWHAAjBKFEDYz1R100CanfUczcl3uQDmmRbddqOE4UOoGigEs9khcP563t2xW3h7fXEE/eipt4HXAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yIG/Gq5YZPbCV92/HN+i3wEWEJiFnj6txDrJHePjssI=; b=PRn42Kud+MLQPYTNFFQI1P+isrIYgAcPFnfZdcOeytFvu/cqkI+DA96w4YcQ+cCH3K6Pw53yTUhUiMqqxCr7ieFjuwk0l1xKS2BGu2B3ipVf+24lcepxLSWXUjkvHrekCVRka7lV+CNa34bLKyieeByLWZFHBHy/z759LwEm/Dh2YY1Y2DKTOh7xC0FrMIgI6LPZUlwrmk5HDWmKKz3yh+/nC04MB+7rBjhsuGJeN5xW76YzbnqCDzH6SDXIN9uFp+pZkZlPga2YXJtZFMZa45ptCMm8GPP7XIPpmiImhFNKdoACxqmqGZJAcWLKrNZbVn79z/dHbAH7ar1KdcSY2g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=kvack.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yIG/Gq5YZPbCV92/HN+i3wEWEJiFnj6txDrJHePjssI=; b=Y9GzWAjk0zyDHTmj9C4+Ek+E8jyGmU3ZtOtLpkL+FI7Jm/AT8wJBosRQpNqytJN4uDZeAJj0fGczqA08MJN+p0nwNNGKCYeXEp5t9DVvjIS9rOL/XDLz2E68bCDwGsx9V96g8KwXbpUDL2an7l9Vo78X4Mdewhe3CcAH03ZH/aEoY56fe0MjWcp9d/5AkJWMNpRbTdI8BbgInR99kj9PP69cM/kEoIQ/Ma1ulrojKJI0aNaFWVsat6u7X0gsPRqtf0L4rSOVmjphEY1TiKoq/pi3AUkhc61EriaxSa2Rf+7WJ8m60GdWyT5hp1xKnK41Ekas5MD+4/XLHruhsYLuRQ== Received: from CH2PR14CA0013.namprd14.prod.outlook.com (2603:10b6:610:60::23) by BY5PR12MB4130.namprd12.prod.outlook.com (2603:10b6:a03:20b::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6813.20; Wed, 20 Sep 2023 14:02:46 +0000 Received: from DS2PEPF00003443.namprd04.prod.outlook.com (2603:10b6:610:60:cafe::e8) by CH2PR14CA0013.outlook.office365.com (2603:10b6:610:60::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.29 via Frontend Transport; Wed, 20 Sep 2023 14:02:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DS2PEPF00003443.mail.protection.outlook.com (10.167.17.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.20 via Frontend Transport; Wed, 20 Sep 2023 14:02:46 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Wed, 20 Sep 2023 07:02:30 -0700 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Wed, 20 Sep 2023 07:02:30 -0700 Received: from sgarnayak-dt.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41 via Frontend Transport; Wed, 20 Sep 2023 07:02:25 -0700 From: To: , , , , , , , CC: , , , , , , , , , , Subject: [PATCH v1 2/4] mm: Add poison error check in fixup_user_fault() for mapped pfn Date: Wed, 20 Sep 2023 19:32:08 +0530 Message-ID: <20230920140210.12663-3-ankita@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230920140210.12663-1-ankita@nvidia.com> References: <20230920140210.12663-1-ankita@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003443:EE_|BY5PR12MB4130:EE_ X-MS-Office365-Filtering-Correlation-Id: cbb806d4-28ed-438c-0c44-08dbb9e24445 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: O/SSvjLpNYBbgXSacNhW9nrr0LphHgHDkpLTK4rTXsP3MT9kYoUOF6Wie8AIIDQ5m/36KOfqfOi9Yk1iF0BwezREN/+oQRBvZSo205Wl4sgLmffcOtGrln1kSzyc/mA+Iy/sPxWYXrZiO0lZxufXNZp5NRSWVco7CgmzEt/vWxe2ApXtOhEFMT/T5W+ubtNT98dp9EMvQC/ME9D5gTkqShzlq5+/bdUfnsWYgd2XdbHqKjSx0Dhp+sHRIfehuso3ePJ42c4ElEpNnpV7ao9Q5Upx7s8RxQul1TA8/an5AtzXcXYfzvwj/SVgNGs5w9Nl0eZkpjalk1064th+nRnv+neERZwmnqBs4UD2dIAboFun++qt8//YV36Ghg9/IHcffIJgHRplcDGyEUBLeSbXtURyFgBdFId7o67/rNUQX6E1y2E174Y/StcnpPCPJMn9mTgcknAFkPJdHXRRLE1sVD0sl32WVc7p3bIlACo33edc2CLjPcag9QZZQKGEDqq2qTKKDbyTKyyA8vyE4eAsqRvcitUFh9ywFgxJdGRsPTo6+e1+EtWM131ha6v9K6Nq1CZQf7dilWSxrAJHs/l9UGCE9tm+Ye6/CO0qT4K0Pr9vMrypfDmNleamcL2x9umv41r0cniN+hVA1cNWHkVyAC3uzxCcOgP57kFBSDgy2i4TAKvKkgEMrudLwf5ZBV+rVadwlbZtxmViQ6JzNFfafjGv/khT/o+vhFcXFqsWiDzmDnJqSQme5Ivddbi2jUp9cD0QXgTKmZ3spAujCCn4Xw== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(346002)(396003)(136003)(39860400002)(376002)(82310400011)(1800799009)(186009)(451199024)(46966006)(36840700001)(40470700004)(478600001)(7696005)(6666004)(83380400001)(47076005)(26005)(1076003)(2616005)(426003)(336012)(2876002)(70586007)(54906003)(70206006)(316002)(8676002)(110136005)(7416002)(41300700001)(5660300002)(4326008)(8936002)(2906002)(40460700003)(36860700001)(40480700001)(86362001)(36756003)(356005)(82740400003)(7636003)(2101003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2023 14:02:46.0617 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cbb806d4-28ed-438c-0c44-08dbb9e24445 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003443.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4130 X-Rspamd-Queue-Id: 1226D1A0186 X-Rspam-User: X-Stat-Signature: 63owknygsxa1m1zz33d5drosu1y5bxtz X-Rspamd-Server: rspam03 X-HE-Tag: 1695218569-874755 X-HE-Meta: U2FsdGVkX1/ViUqXwUCOZbee+h3V1k6mnOdLiA0XzDwjHUxqiMWznIxX0o4PA2gfTMxxCACr4v3ueupLe8wfbS82G0YQ/zmFZEfAgIyapca4obcK0NM4CR8+UInJbAWtE1+IkK2hDLMsQGkYOaXnJFyYPU1aKQfSzZnhnNu0YUUEcBVBHdRtcXt4xmoqj+V+EIQGO5glP6EoUN/qTz2UVgILwbVOa4zgxXMN8rP13TB8jEr+ZPEwOMUQsiorFuUeUajNS91BMfVJ9ZWovyKnWmgAb1fjeBHWnceZ3U7eEtzfll+DM40Jl4ck/9CtjW0C2GgmsAQVpHjCCBTGOMl8Kxt8xlSqJSz7OAq3lBS4OLdElpOD09+Qsx+WW8eRdIUzwuFU1a7T6CxH5+W/qjZXMqT/6NutHezForiw/XIo2qWYNgrFaCT2siOCF4YFvlfby4GXKEvLI0v5sfxbNBc1iV9WAtSV+qj6PSTR8jbwEXbLCEvP0/OlUZ0qLTdS2a5nuRAfC1X2esU3TXHip+PtVwnxBu5fzxeKUB0BDwZgu+Rm7VX3WZQo1AS9nloUHFsDpT7OYr1PomjhWefYHvfrCBI9dOcBNXYtoqlBaxpcIqI4eXb94RhFTHkPFG3VW7XM1e1g6H5g+8Nwk9cf4ME3Gn078lJxZ0racsii02QfG2BjZa281wqYUrj46ERGD9/gPvR2ifHOiMSQIsZifAZqJueeq5lO84k5U/p2XzoKr9mVV/RyCriNKCBG1zTumeiioLCbRKDNIvRJDA2X0UIFGGj2FEV9a62lx9CbJYbwPMxtri8+qxDMPLugsUwW/QjrWWTLhoIZVeFHOlkChHfI8Nv1kM9lYohXmgHD8u/A1tihu0YZjKhdcUpqjLlfxIaxaMnyDghxx936Ud8mB2Nwo4W7zrPlSrsRQXyMm4EJA+dTsjjFpUO2hguzIFiK3P7LQ82q6F1fQnlgQERNDJk +p8hHzxO G4p+SwMq8zkihHHwKBUBNeJNipw7YpUKqWCEbZsolO18BBhWzxQAar92g42jtykUvwXOTKsBSdEuRxsUk2sRWJMGY14G/NOQFEsc+BKZ0SNt9tyPXaQAMm0k39VOW9KfLj/egs3efFylw0uGckci6LjDdzz9S1gp3tDIaxH5cUU1MPdQYFGyAwhd7loJIiN/86DQorv9dfpKrPlbcfy4RHNLU/nBKjxgOETYtYrPp5rkIdTfPJGsqYE/ED007pBW2Jq42iDGrRoIyb/jpFe8alN845Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000426, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Ankit Agrawal The fixup_user_fault() currently does not expect a VM_FAULT_HWPOISON and hence does not check for it while calling vm_fault_to_errno(). Since we now have a new code path which can trigger such case, change fixup_user_fault to look for VM_FAULT_HWPOISON. Also make hva_to_pfn_remapped check for -EHWPOISON and communicate the poison fault up to the user_mem_abort(). Signed-off-by: Ankit Agrawal --- mm/gup.c | 2 +- virt/kvm/kvm_main.c | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/mm/gup.c b/mm/gup.c index 2f8a2d89fde1..fe469326dbe6 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1414,7 +1414,7 @@ int fixup_user_fault(struct mm_struct *mm, } if (ret & VM_FAULT_ERROR) { - int err = vm_fault_to_errno(ret, 0); + int err = vm_fault_to_errno(ret, FOLL_HWPOISON); if (err) return err; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 486800a7024b..2ff067f21a7c 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2731,6 +2731,12 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool atomic, bool interruptible, r = hva_to_pfn_remapped(vma, addr, write_fault, writable, &pfn); if (r == -EAGAIN) goto retry; + + if (r == -EHWPOISON) { + pfn = KVM_PFN_ERR_HWPOISON; + goto exit; + } + if (r < 0) pfn = KVM_PFN_ERR_FAULT; } else { From patchwork Wed Sep 20 14:02:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankit Agrawal X-Patchwork-Id: 13392839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5110CCE79D1 for ; Wed, 20 Sep 2023 14:04:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE8616B0179; Wed, 20 Sep 2023 10:04:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C986A6B017A; Wed, 20 Sep 2023 10:04:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B60836B017B; Wed, 20 Sep 2023 10:04:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A69566B0179 for ; Wed, 20 Sep 2023 10:04:28 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 80419B327E for ; Wed, 20 Sep 2023 14:04:28 +0000 (UTC) X-FDA: 81257145816.08.939E9CD Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2043.outbound.protection.outlook.com [40.107.92.43]) by imf29.hostedemail.com (Postfix) with ESMTP id 01DEE120341 for ; Wed, 20 Sep 2023 14:02:51 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=HPeXu5Op; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf29.hostedemail.com: domain of ankita@nvidia.com designates 40.107.92.43 as permitted sender) smtp.mailfrom=ankita@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1695218572; a=rsa-sha256; cv=pass; b=Hc3mm8tutHrY39ACvelhOcL0lkBP7bieK0J35+skdGn7GYyAvzdXgJs81/szEZZuVycfvo VmzCNdTDkqRyeSPcSXGtiA7ahkWoE9DuhQIwIi+z25GAWR4WvCXBQ8sqkZRWKLVK3m9baq zr39sUd7cGsS6kSzI//usv5ZkQ6zve4= ARC-Authentication-Results: i=2; imf29.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=HPeXu5Op; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf29.hostedemail.com: domain of ankita@nvidia.com designates 40.107.92.43 as permitted sender) smtp.mailfrom=ankita@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695218572; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nJPM6GxvrfU+SOFbh35kBojQcnerwuMNxRwCXwIw0ng=; b=zJZPMMKfwqJcG//pmt53rHdSCXpWj/QE/qDfOsZq6xrnHGBYR2gMeYFjMGRIsBvBwWykMB Md4DUxpNKsDatZ+aHjDDbYMkp4uGtLJzsvEpK4/Ns29EXWLv2qSG1ztaBhF1aRrqLzBRRE ueP3Cg7P+Sq2dWekYt8XLpdCHtW61hU= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZJxSk/iNQedNw6Mnfnxngnu1fVVk08fMh0Z47BfOGaFe7l3z7hHR14najhxKA7TPW+SOozua0O197QoQOq13Jj7EyxHbNT34bfY1LLeHDk14GKm08SaVI9FBc05BLDxrIr4AOm0aUcv99OghxZ063fEXZxuXlJavBOjMmqk1YqlkKi98bc9y6jBMIuSnbm59JIybqbrNq6USURhaU6vR7Y1ZRxG2CqfWc43yB/vr+YU1xgHPpWsNKnsMlF4Mpz168GAkmtAuGOjcNrd17F7RAiE97kkNxbo240qea9Yi9wS/MYx7H5DQWPgm/xGTVlAvwGAJY+WPVYKywr4Skl3ydw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nJPM6GxvrfU+SOFbh35kBojQcnerwuMNxRwCXwIw0ng=; b=mA9b60YLDiBRJerT+uMDDDxjsdQDYCZ31dUIdknam8od/h6aVRyD0ctCnru5w69M1jStKJh4/2q+kmKkDpJqoRqgjdNO5UWKwWm746XT/V6kebSQSKqMsE2Q7/UYKy5M3XKFEnFQwnmL9tNnYp6o27XQ6YCO1Cq5HUo04QnlUScdpoGTlnIU3DjpZEahvdN4lgrgd+cDoiH+spclcHMdd1LwM1SUznf8paD09ERnkqLpDjHvM06PkNhxuc5kIQLoHgrhLG83kLULe9nRXrXDAAllNYzUxnm2EAItShrs/MvK5orMuSi+N4Yd+hfB6WMjzPKN0HoY/RL+wYjDk6Eb4w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=kvack.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nJPM6GxvrfU+SOFbh35kBojQcnerwuMNxRwCXwIw0ng=; b=HPeXu5OptSu9mRJh0Gx3xjOL8bKgLmskWJm2uMbc31NDJa7mtIOhSslc8WvXtP5vvC++pB5YCVuQcWuvQQBYKcETd97WITMrAgy/pV04p/Ow76Z4SKDNp0U8yUrU8UeLe9Hh8umGlW5P++iOclV6PYzsZTX4VopMA9zcxUR+7SRZbqFcpddjvQHQB80rpTyKtfchyn81DRRZ63vCHpzPA4TIs05LkiU20ODzBOM9aTkMV5X1SnIBB754EBLejvP7/QrJnXlTQTL3NhWHWwapWlaUctCFZgwXBc4uRQswDiKo7AFFJQDOyJs5ky5knnOsoYKf11qcvzO4wACoW/QuWw== Received: from DM6PR06CA0038.namprd06.prod.outlook.com (2603:10b6:5:54::15) by MN2PR12MB4253.namprd12.prod.outlook.com (2603:10b6:208:1de::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6813.20; Wed, 20 Sep 2023 14:02:47 +0000 Received: from DS1PEPF0001708F.namprd03.prod.outlook.com (2603:10b6:5:54:cafe::d1) by DM6PR06CA0038.outlook.office365.com (2603:10b6:5:54::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.30 via Frontend Transport; Wed, 20 Sep 2023 14:02:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DS1PEPF0001708F.mail.protection.outlook.com (10.167.17.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.20 via Frontend Transport; Wed, 20 Sep 2023 14:02:47 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Wed, 20 Sep 2023 07:02:36 -0700 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Wed, 20 Sep 2023 07:02:36 -0700 Received: from sgarnayak-dt.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41 via Frontend Transport; Wed, 20 Sep 2023 07:02:30 -0700 From: To: , , , , , , , CC: , , , , , , , , , , Subject: [PATCH v1 3/4] mm: Change ghes code to allow poison of non-struct pfn Date: Wed, 20 Sep 2023 19:32:09 +0530 Message-ID: <20230920140210.12663-4-ankita@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230920140210.12663-1-ankita@nvidia.com> References: <20230920140210.12663-1-ankita@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS1PEPF0001708F:EE_|MN2PR12MB4253:EE_ X-MS-Office365-Filtering-Correlation-Id: 34cbfda9-142b-4468-32a1-08dbb9e24536 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: d0Jy4EnczyibCEUGgEINwvZfy91ADTU5AL9s0fYez1GCX+vuYydmwPJ3dW/b3LlSfI8dnreXrYIBlv7Kow/e4O2XfhGLnzDwNh10YzuNtWrx12bt/jcSYPn+jkcPOPQ2ohfaN4hBoYvoYlmkS1uvIpX+IRi2uVU6FjYw8tIiX9xh8ftELazBGMr4IH1wMjHI/ZNZIYjl8zyqdjEwbz8//6M1LyE1QhGRaes7OtMgr1b66Ff8pRjy2WmiEu+jlNahJoLi6AHaP1KV5lGgDH86k0hZ2y95KtZdPnPg+aPHIZJiOTKG2s96jrbNOsZ5++rPWrsQpDNiPEcQtvwuUub5RzuHKMZZM5dpM9uMKBF0GmVWJ7LXQzhUMHo2ZjsnKjL2Q7vQ0pZ0vfgHwZ7BEEwMPSDXURA6oysWjhxNqMZFOHjbb01hN/dzp0EtTWsJON2ndXVPw3zCC3qGgYP8jeuXEDCxlddmvvYJB/DVgt/qxdLYL+EOEo/eyJCNum/EGtBS9X9yQmobn9a425ADhBfLxE1OWUMG9m0RhBQyQGq1SwbWiZwPFNZGsCULHHC1nA6cBldzjHetQ7pePZ26v3f51fUAMQzIw4/P2zgoQOM06M1n++xdKpQidRxEFZLymItNmI6sEpIEZ6BN/RfAg0xnnIR/Lka4fwZhtPTDn6hKpk/rR2sNBX7FipuNS6tXO3XvTXwnVuA6OseZau8iCEjOhlcbn9Fu3dyLXvYVbPylYl3w24ektRy+WaP7L2pNRQ8S X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(346002)(376002)(136003)(396003)(82310400011)(1800799009)(186009)(451199024)(36840700001)(40470700004)(46966006)(40460700003)(7696005)(6666004)(83380400001)(82740400003)(7636003)(356005)(86362001)(47076005)(36860700001)(36756003)(2616005)(336012)(1076003)(40480700001)(26005)(110136005)(316002)(70586007)(70206006)(54906003)(7416002)(41300700001)(2876002)(2906002)(5660300002)(4326008)(8676002)(426003)(8936002)(478600001)(2101003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2023 14:02:47.6421 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 34cbfda9-142b-4468-32a1-08dbb9e24536 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS1PEPF0001708F.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4253 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 01DEE120341 X-Stat-Signature: gf9nhfw13a944amqedhzttnfq1cnoghm X-HE-Tag: 1695218571-524856 X-HE-Meta: U2FsdGVkX182ToAmDSmDyIZX7pP5PIAPccqsmK2aUpuWkoFPYJPvDLiU4qBNDTQoDXKNYWZ/qMPGjCbA2jFJDb5Lz7mLPS4iDL1DPKBHHcBnERNl9/3ZKmEtv7TUr7aSwcJ30tmNYl8azQFlRz1asRuejof8Mor+M3NS5oQp8rlnIY0/OItLOfHx+Z51ivFxbNiggxLdpptp7A4rJgJcTNztoJT3xaziy1DZYfQKhZOZuaaWgp92ESzR5KZFFHoLsWGjLskCQDbavy68iLB4Ik1nvVSlxrLbKKs2brMR6f5D16P8VqjsKB06qx+/3DpbZ0mEdCNDwYPi+bRvxCSNR5nwX0wdnuN6Gh7/1VdT6UFRlMOpFYAfVgwkTaKJx3P466QNzOSM+FfeKCx/OnwWebOQ1dHQxRalWGNEGiGU/SqTeYwjNHGU0An7B9jqNVmT4Nd3joAsIvjti2gyAt8BHcfQtPJ4TyMlKWF4ntr4oIK3QEUkTo/Ey9ao4M8jRmfZJEdUkgDsUlJaZoQwccI74nXvSa/lBXmCDq62oObfIXoFTf4TpN96YeX/cOrTXAn37pPduv1r1fuI+ag/2Vmv0PQhDWhMhgQz8XYvO/oFgNPbZqtIQflxKpFtxnfMAXIWyrCPFg5ONa5TQjBquFMySBdcYqBhfueGt/YOKFXK8JhwScOu0TCae+FhOgzwFm0QDY1s1pKiAilvuUFP7K91WOOLM34V9+OhK8pni+olYumS1t8HFgcCu66E4YoM9jVyG+jS7rV9RSUQvhf69Kb3Y8YvoYzwLZJLUHgqjMDY1nbXeVBXbRG+7kGVRc1Dna/OgdqSZmSMN3C8pK4bmzjH57eV8q2h7f3g5w3kE1SwZ1MQhHQC9wTwXEA7ILDF58Ko/nJEvC0q73oTSp9TCWlQJI3PBDL3a6cWKONHDHFjZTe+Yw8izcYvoAyG+9ozwCC3m+jX+YowB+/f297/LG8 EjLiyY7C hSOWQPtDjHDP2db1eIfm4rmnibx9NpNm6QXWv1fvepiroCNa1deZmpHfIRWR8cz+Ut9DU+GeOeXl3MSqIi6fKW39DinON1c804JvLtIS+M3YucXSOXRzS87sZG5d4BI3gDAhRqFF4w7RQKaKAR8u/fMiTfa5fKeuSDEx6fr/G3gTh9cO/kQCHs7nCahHKnlIkFchzmAboTPQlkD8ZZwXiB09o2keYoWDr7acB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Ankit Agrawal The GHES code allows calling of memory_failure() on the PFNs that pass the pfn_valid() check. This contract is broken for the remapped PFNs which fails the check and ghes_do_memory_failure() returns without triggering memory_failure(). Update code to allow memory_failure() call on PFNs failing pfn_valid(). Signed-off-by: Ankit Agrawal --- drivers/acpi/apei/ghes.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c index ef59d6ea16da..6ad1e4cbc968 100644 --- a/drivers/acpi/apei/ghes.c +++ b/drivers/acpi/apei/ghes.c @@ -457,20 +457,10 @@ static void ghes_kick_task_work(struct callback_head *head) static bool ghes_do_memory_failure(u64 physical_addr, int flags) { - unsigned long pfn; - if (!IS_ENABLED(CONFIG_ACPI_APEI_MEMORY_FAILURE)) return false; - pfn = PHYS_PFN(physical_addr); - if (!pfn_valid(pfn) && !arch_is_platform_page(physical_addr)) { - pr_warn_ratelimited(FW_WARN GHES_PFX - "Invalid address in generic error data: %#llx\n", - physical_addr); - return false; - } - - memory_failure_queue(pfn, flags); + memory_failure_queue(PHYS_PFN(physical_addr), flags); return true; } From patchwork Wed Sep 20 14:02:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Ankit Agrawal X-Patchwork-Id: 13392840 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0613BCE79D1 for ; Wed, 20 Sep 2023 14:04:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8C99D6B017B; Wed, 20 Sep 2023 10:04:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 879B26B017C; Wed, 20 Sep 2023 10:04:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 71AED6B017D; Wed, 20 Sep 2023 10:04:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 6019C6B017B for ; Wed, 20 Sep 2023 10:04:42 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2D6A31C9E17 for ; Wed, 20 Sep 2023 14:04:42 +0000 (UTC) X-FDA: 81257146404.26.868D1F5 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2068.outbound.protection.outlook.com [40.107.100.68]) by imf05.hostedemail.com (Postfix) with ESMTP id 977B6100022 for ; Wed, 20 Sep 2023 14:03:00 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=YK4+TSkE; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf05.hostedemail.com: domain of ankita@nvidia.com designates 40.107.100.68 as permitted sender) smtp.mailfrom=ankita@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695218581; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DPKCo1tinmHhjn1Xd/DXhRSf51vYCpNLutWbS38L36k=; b=x3eHokCOvAk7gs2c7/AXIGnVKeBn9L78dPD9BPQOfPIWCNNIeu4tliqCWu9apuQSmlOAN2 hPAkJF9CjXDSmV2/VfjdAjCJm7IDCL5L3ukhr0FxuynTFNiMfWS1mM/YlsbvW1pd+U+tFu NLZbnFgdJ1Er6lCO43O5nl1+SYcapIc= ARC-Authentication-Results: i=2; imf05.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=YK4+TSkE; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf05.hostedemail.com: domain of ankita@nvidia.com designates 40.107.100.68 as permitted sender) smtp.mailfrom=ankita@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1695218581; a=rsa-sha256; cv=pass; b=lhnJ+n0sUF+c+chU6W4T/QCWy8k/0Z0BsL6wppbnGnUI3nYKxgFkR9pXVP0Mi08OCqcSuj YNjaEqQ8UQk+JkYcGjlMJ9Pla2JsWtR+zkBlu+jdwcDNmCBqY3k0t8YVUau77Na2Cf8qLM bmyHWHDg1GNHScomJ9OBk54VUcfK4qs= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oXiUqxxcr+Z8eeSiVfmYXNG2dOYBPxEM+VSqzu54fVP1aVVaJq+TPSSmmKItPwITuoyFm/AKtQu/bkrIQdd8jpaJDTiTHDeebjH2P031TC1uNP6VxlCHjtievjpmP13CpCnY0tPu/BFOCDv2lpFpMfT6gu3kqzogLuXcEF9nEYG7sJ9AVRLN1GWGIt5lbJ+72lESeFqNdRG+VWbpkGvOnIBHzEFXCCfw0luDJytqMm0tgYIw4uIaUGsCRrNspuxb/13xOgEgkUU1anKCW4tP6V7stOdFoBRYgfeLfiNE2mwfj0vCB6WmfaOw+M8AL8Pt5vRWzlTkKM1KIxEVi+v8Hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DPKCo1tinmHhjn1Xd/DXhRSf51vYCpNLutWbS38L36k=; b=WIZroxPY/LMkz9O/sUlx8Aijp9yMeG3ZOadq5ydwZWxksPbh7MLBwLWKfPRawArVRydEWv+gtX4qaFXwaqP2+W6MaRipCbauNMLJhelIZtWjsy8OhCWvLetGwOnYDgjL9yee2xxkaxwgp5EJ2MLdXFkBUc7kmh0EbKhn/XRpkymDJz0Ew6E8qPYVMHuAGBMRKWeRQF4WbHTKCgyICM6W3y53LMuOBWrYL8UshbuWysVf2rY/eKmUoih/22WKlltxEpgM8vi6YHwf/UpP0jybHY0Ql5ZPdGR26IBhGZudHcZRDtORP7yqFC1xDEjllgboxfbM3Hi70Q2Ohh3tEDhhLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=kvack.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DPKCo1tinmHhjn1Xd/DXhRSf51vYCpNLutWbS38L36k=; b=YK4+TSkEDcC0AeBRGAdH2P1pgY8uzei5cZ1ERCiNBO0KJhDHLEmWvs5/8xogcqnZoBHZkRY8awB4p3uIhWcjzmgv8krRxUMypbzZ8TzDhAN3gLslGlts0SleA3Mzw0CnMcsoGxmDfYD/uMz65MBAVO9VxXxXhvcBEM9TeZOtKOOLBc8BN8DAmkCqLFI4ASL6/eEdk2j06oBUTWSPo+g0wPGMcAi5d5+FUWK/mQ9O6QYGB1wTB3lHnoLbYCR9y1uGoadbiT6qBIclE5MGJDS/P0bWgA9I7ZZBzGmtrUc6GoweB9jH9VbwKRs8OwhLYvCrewnhOI8dZcCoJm+AtqNu5g== Received: from DM6PR06CA0062.namprd06.prod.outlook.com (2603:10b6:5:54::39) by IA0PR12MB9045.namprd12.prod.outlook.com (2603:10b6:208:406::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.27; Wed, 20 Sep 2023 14:02:54 +0000 Received: from DS1PEPF0001708F.namprd03.prod.outlook.com (2603:10b6:5:54:cafe::2) by DM6PR06CA0062.outlook.office365.com (2603:10b6:5:54::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.30 via Frontend Transport; Wed, 20 Sep 2023 14:02:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DS1PEPF0001708F.mail.protection.outlook.com (10.167.17.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.20 via Frontend Transport; Wed, 20 Sep 2023 14:02:54 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Wed, 20 Sep 2023 07:02:42 -0700 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Wed, 20 Sep 2023 07:02:42 -0700 Received: from sgarnayak-dt.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41 via Frontend Transport; Wed, 20 Sep 2023 07:02:36 -0700 From: To: , , , , , , , CC: , , , , , , , , , , Subject: [PATCH v1 4/4] vfio/nvgpu: register device memory for poison handling Date: Wed, 20 Sep 2023 19:32:10 +0530 Message-ID: <20230920140210.12663-5-ankita@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230920140210.12663-1-ankita@nvidia.com> References: <20230920140210.12663-1-ankita@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS1PEPF0001708F:EE_|IA0PR12MB9045:EE_ X-MS-Office365-Filtering-Correlation-Id: f3e04218-c33f-439b-a2f3-08dbb9e24913 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2QGWyN9V69NSR3EDKUzCDxuoAZtMSk2eSWy4dr8k9MxQY0hYxNp/kH7e2XkOKy/uKDj7MxFc9tD8Os0A1MQjFKADkNZ9rO40EH7Qon6sxsbNNA+SHsjEkeVBUnQC4fwy1fA7VdHf0PzYweq5YRtTV9Xy5XDH7VR+OofFc8CfOtopEJprdiB7ptWclcY9PuhzwgOahMsRtcShMnt3qyHH8qRB2csfjjNWjLAAS5CzLo8g3ofRVJdnp+SteaVN5W/2kMjkZDjn72GjHlIgpHcMVdXIteyeW015yZkpPWlm8ZaGRtstz4K1WLZLhHjb5nA7vBTqMgcNJs+pcSr3HqitLV9eeOceWAHNob2o6LMWsGqOIo7+5qA8mPCxxHV8FO0R0cgsmGqKgJNT65zLY+U/4CGrQ05PgnDiCO61FVEbVhebGW7gl3l9eQudnQQpe7eJjMOhOJ2eVtn9wrhF8h3Jx8uDdRtlGuQrnjX1f1G35avs5W/5uS1he/b1qlkhv+rXvnefLNZlLWQh/cHgRUZbsysMEod6y0S9cX5bZ6u1etw0J2u4pA5cp5N8HVc+Qf+MfB2RxSYGpBXqz9kXcuf9w6NUYBT3t2DeAOklQ1747fyhCkEukrJuCL0n5ymY1fe2naOaD8WJRDt89uhA3ebirD9/68i8Bf+bkiC/l5HjXBblmNyvp6yqhtfLv7mlhTmRlXsk5YHLgmnU8EBMs2mltYfIVlL8yC3JcNfhRCJqKT4DMHB1a8S971H6BU5lmADBwq2M6Qw3wqKBinrNQfwKnLWpZCdJETmZjtdytFZvdrqoyEssYXzMQNe3ySBIgwyT X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(396003)(39860400002)(376002)(346002)(136003)(451199024)(1800799009)(82310400011)(186009)(40470700004)(36840700001)(46966006)(70586007)(54906003)(70206006)(966005)(110136005)(40480700001)(316002)(41300700001)(478600001)(7696005)(6666004)(8936002)(8676002)(4326008)(40460700003)(1076003)(2616005)(26005)(5660300002)(47076005)(82740400003)(36756003)(7636003)(36860700001)(356005)(336012)(426003)(83380400001)(2906002)(7416002)(86362001)(2876002)(2101003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2023 14:02:54.1421 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f3e04218-c33f-439b-a2f3-08dbb9e24913 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS1PEPF0001708F.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB9045 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 977B6100022 X-Stat-Signature: bcc66znhcpdtb7311c68yhtz86gebmtz X-HE-Tag: 1695218580-964938 X-HE-Meta: U2FsdGVkX1938u0YKV0T/E47ayImvizWsRTEhVzteal9T0Huu3SlfafX4wKlG11GltPltAOAnG8Zqm2gdcgDoRaGVjYSx1wJPSmLEIrjVvZXXnINUEDA/+knY3XvSRJ64ADPrOAPBafpsiVyhUaTKF96uUG8d6Lz5JEzLJh4CD+Dm7ISGl2cbQ3fZHD4+Xqp7kzL1o6mrO7ob5XRymY2pbHl9YIV6OWhXTm7PvQ0Sc4vI6zgkY25BI4It50KCCoDpGayg0rWpTAeef9EZhVZI90X1c8TLLjKWR7lyd9CoJhSdf2CHQUs307BnaLL/sJrEglqu4hv0qgeQmSNbAlI9k3ogj4aHo+IFcsmhSddthywxCdrtjbFZCi1vToeczExUTOdRISKj2EwoSz+ftaNNjamQEGjuDBj9aoMUgbUkpsZDU05Bw15RPBPuzBbDKKNXbNdvhKmqMvetKCkUHQ0d5m7eJlHiw+GsjNpbjc+RH+Wn+oWbX8vJiXXrybS3zAsoEVwaokPuhlrm3pKTTMjfBom+/SHPBRDKzxhs48J7kI7YCRf3jaRuF4GR42IEpsDsqnwaItrEMYukIDxFSVDBq2e83an0njdW2UuHSboNwWBPChunhTfyawg4hL5DjcfEJvLShfGLAAetuVdywVvBZC5n+gGD4g4HCMXpZiGcci7DgS8W8jEUeuooNYdmBldNPpA/JA6l4lwxaEUg1KRVNOyBNs4S1lziaGjq4rkyYeVgZFQIxV5LO4coHomxnbHLptsQmzTjPFQRez6uyUzIpVDhf/7/SBCZudQKPwHoUE+9RqYC0R94XlAMZbmZhFwE9DtZsLaeet4NqmxZ8QGQTHllaS9nxJ51/GhQeNCjz7Nu8tTxoE4Fk9HVkIRDgB2nX/qOZGfF0WwlzmHtecrM+75zoubqU0K2jy9a+EoA+OwptHYNDWbKF4dhMl+qq+HYS8TL+DyE3B7Win0hEk HcIemD+I v1wBR4HDIvVaV7GT/Kb9k81PyWH8ohTlz3T5freejxQUHx/LjrP6ssqapc+puZ4i74FHBYoJVw+pYrSFT7qQq2MIwobrnjwdSzw6tFN0XjRWufhDTmEX6jbGMGmTnxERYQS4mQWUpOT80C0lnCaughZUtSP5Jc/Uyl80iHnlqKT6KFocEYdGmXJeHuRtRUOYu+DF5iUzg5Zq2/yk9IJ0PkRbKlsTcgM5OXkqjQJvi6QBbrRCavBkZ7Un3+o4oDuYgi23KTA7p/qttRrqkMRjMZYoc+BMfkiPx3znLdB8jeVqyz4HA22sVyOEINMqH+9+wL4pwkXQEBJtrozM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Ankit Agrawal The nvgrace-gpu-vfio-pci module [1] maps the device memory to the user VA (Qemu) using remap_pfn_range() without adding the memory to the kernel. The device memory pages are not backed by struct page. Patches 1-3 implements the mechanism to handle ECC/poison on memory page without struct page and expose a registration function. This new mechanism is leveraged here.   The module registers its memory region with the kernel MM for ECC handling using the register_pfn_address_space() registration API exposed by the kernel. It also defines a failure callback function pfn_memory_failure() to get the poisoned PFN from the MM.   The module track poisoned PFN as a bitmap with a bit per PFN. The PFN is communicated by the kernel MM to the module through the failure function, which sets the appropriate bit in the bitmap.   The module also defines a VMA fault ops for the module. It returns VM_FAULT_HWPOISON in case the bit for the PFN is set in the bitmap. [1] https://lore.kernel.org/all/20230915025415.6762-1-ankita@nvidia.com/ Signed-off-by: Ankit Agrawal --- drivers/vfio/pci/nvgrace-gpu/main.c | 107 +++++++++++++++++++++++++++- drivers/vfio/vfio.h | 11 --- drivers/vfio/vfio_main.c | 3 +- include/linux/vfio.h | 15 ++++ 4 files changed, 123 insertions(+), 13 deletions(-) diff --git a/drivers/vfio/pci/nvgrace-gpu/main.c b/drivers/vfio/pci/nvgrace-gpu/main.c index ba323f2d8ea1..1c89ce0cc1cc 100644 --- a/drivers/vfio/pci/nvgrace-gpu/main.c +++ b/drivers/vfio/pci/nvgrace-gpu/main.c @@ -6,6 +6,10 @@ #include #include #include +#ifdef CONFIG_MEMORY_FAILURE +#include +#include +#endif struct nvgrace_gpu_vfio_pci_core_device { struct vfio_pci_core_device core_device; @@ -13,8 +17,85 @@ struct nvgrace_gpu_vfio_pci_core_device { size_t memlength; void *memmap; struct mutex memmap_lock; +#ifdef CONFIG_MEMORY_FAILURE + struct pfn_address_space pfn_address_space; + unsigned long *pfn_bitmap; +#endif }; +#ifdef CONFIG_MEMORY_FAILURE +void nvgrace_gpu_vfio_pci_pfn_memory_failure(struct pfn_address_space *pfn_space, + unsigned long pfn) +{ + struct nvgrace_gpu_vfio_pci_core_device *nvdev = container_of( + pfn_space, struct nvgrace_gpu_vfio_pci_core_device, pfn_address_space); + unsigned long mem_offset = pfn - pfn_space->node.start; + + if (mem_offset >= nvdev->memlength) + return; + + /* + * MM has called to notify a poisoned page. Track that in the bitmap. + */ + __set_bit(mem_offset, nvdev->pfn_bitmap); +} + +struct pfn_address_space_ops nvgrace_gpu_vfio_pci_pas_ops = { + .failure = nvgrace_gpu_vfio_pci_pfn_memory_failure, +}; + +static int +nvgrace_gpu_vfio_pci_register_pfn_range(struct nvgrace_gpu_vfio_pci_core_device *nvdev, + struct vm_area_struct *vma) +{ + unsigned long nr_pages; + int ret = 0; + + nr_pages = nvdev->memlength >> PAGE_SHIFT; + + nvdev->pfn_address_space.node.start = vma->vm_pgoff; + nvdev->pfn_address_space.node.last = vma->vm_pgoff + nr_pages - 1; + nvdev->pfn_address_space.ops = &nvgrace_gpu_vfio_pci_pas_ops; + nvdev->pfn_address_space.mapping = vma->vm_file->f_mapping; + + ret = register_pfn_address_space(&(nvdev->pfn_address_space)); + + return ret; +} + +static vm_fault_t nvgrace_gpu_vfio_pci_fault(struct vm_fault *vmf) +{ + unsigned long mem_offset = vmf->pgoff - vmf->vma->vm_pgoff; + struct vfio_device *core_vdev; + struct nvgrace_gpu_vfio_pci_core_device *nvdev; + + if (!(vmf->vma->vm_file)) + goto error_exit; + + core_vdev = vfio_device_from_file(vmf->vma->vm_file); + + if (!core_vdev) + goto error_exit; + + nvdev = container_of(core_vdev, + struct nvgrace_gpu_vfio_pci_core_device, core_device.vdev); + + /* + * Check if the page is poisoned. + */ + if (mem_offset < (nvdev->memlength >> PAGE_SHIFT) && + test_bit(mem_offset, nvdev->pfn_bitmap)) + return VM_FAULT_HWPOISON; + +error_exit: + return VM_FAULT_ERROR; +} + +static const struct vm_operations_struct nvgrace_gpu_vfio_pci_mmap_ops = { + .fault = nvgrace_gpu_vfio_pci_fault, +}; +#endif + static int nvgrace_gpu_vfio_pci_open_device(struct vfio_device *core_vdev) { struct vfio_pci_core_device *vdev = @@ -46,6 +127,9 @@ static void nvgrace_gpu_vfio_pci_close_device(struct vfio_device *core_vdev) mutex_destroy(&nvdev->memmap_lock); +#ifdef CONFIG_MEMORY_FAILURE + unregister_pfn_address_space(&(nvdev->pfn_address_space)); +#endif vfio_pci_core_close_device(core_vdev); } @@ -104,8 +188,12 @@ static int nvgrace_gpu_vfio_pci_mmap(struct vfio_device *core_vdev, return ret; vma->vm_pgoff = start_pfn; +#ifdef CONFIG_MEMORY_FAILURE + vma->vm_ops = &nvgrace_gpu_vfio_pci_mmap_ops; - return 0; + ret = nvgrace_gpu_vfio_pci_register_pfn_range(nvdev, vma); +#endif + return ret; } static long @@ -406,6 +494,19 @@ nvgrace_gpu_vfio_pci_fetch_memory_property(struct pci_dev *pdev, nvdev->memlength = memlength; +#ifdef CONFIG_MEMORY_FAILURE + /* + * A bitmap is maintained to track the pages that are poisoned. Each + * page is represented by a bit. Allocation size in bytes is + * determined by shifting the device memory size by PAGE_SHIFT to + * determine the number of pages; and further shifted by 3 as each + * byte could track 8 pages. + */ + nvdev->pfn_bitmap + = vzalloc((nvdev->memlength >> PAGE_SHIFT)/BITS_PER_TYPE(char)); + if (!nvdev->pfn_bitmap) + ret = -ENOMEM; +#endif return ret; } @@ -442,6 +543,10 @@ static void nvgrace_gpu_vfio_pci_remove(struct pci_dev *pdev) struct nvgrace_gpu_vfio_pci_core_device *nvdev = nvgrace_gpu_drvdata(pdev); struct vfio_pci_core_device *vdev = &nvdev->core_device; +#ifdef CONFIG_MEMORY_FAILURE + vfree(nvdev->pfn_bitmap); +#endif + vfio_pci_core_unregister_device(vdev); vfio_put_device(&vdev->vdev); } diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h index 307e3f29b527..747094503909 100644 --- a/drivers/vfio/vfio.h +++ b/drivers/vfio/vfio.h @@ -16,17 +16,6 @@ struct iommufd_ctx; struct iommu_group; struct vfio_container; -struct vfio_device_file { - struct vfio_device *device; - struct vfio_group *group; - - u8 access_granted; - u32 devid; /* only valid when iommufd is valid */ - spinlock_t kvm_ref_lock; /* protect kvm field */ - struct kvm *kvm; - struct iommufd_ctx *iommufd; /* protected by struct vfio_device_set::lock */ -}; - void vfio_device_put_registration(struct vfio_device *device); bool vfio_device_try_get_registration(struct vfio_device *device); int vfio_df_open(struct vfio_device_file *df); diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c index 40732e8ed4c6..a7dafd7c64a6 100644 --- a/drivers/vfio/vfio_main.c +++ b/drivers/vfio/vfio_main.c @@ -1309,7 +1309,7 @@ const struct file_operations vfio_device_fops = { .mmap = vfio_device_fops_mmap, }; -static struct vfio_device *vfio_device_from_file(struct file *file) +struct vfio_device *vfio_device_from_file(struct file *file) { struct vfio_device_file *df = file->private_data; @@ -1317,6 +1317,7 @@ static struct vfio_device *vfio_device_from_file(struct file *file) return NULL; return df->device; } +EXPORT_SYMBOL_GPL(vfio_device_from_file); /** * vfio_file_is_valid - True if the file is valid vfio file diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 454e9295970c..d88af251e931 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -361,4 +361,19 @@ int vfio_virqfd_enable(void *opaque, int (*handler)(void *, void *), struct virqfd **pvirqfd, int fd); void vfio_virqfd_disable(struct virqfd **pvirqfd); +/* + * VFIO device file. + */ +struct vfio_device_file { + struct vfio_device *device; + struct vfio_group *group; + u8 access_granted; + u32 devid; /* only valid when iommufd is valid */ + spinlock_t kvm_ref_lock; /* protect kvm field */ + struct kvm *kvm; + struct iommufd_ctx *iommufd; /* protected by struct vfio_device_set::lock */ +}; + +struct vfio_device *vfio_device_from_file(struct file *file); + #endif /* VFIO_H */