From patchwork Thu Apr 6 04:05:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 13202849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B25C0C761A6 for ; Thu, 6 Apr 2023 04:05:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 35BE96B0071; Thu, 6 Apr 2023 00:05:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 30B1F6B0074; Thu, 6 Apr 2023 00:05:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1ABF06B0075; Thu, 6 Apr 2023 00:05:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 0C4776B0071 for ; Thu, 6 Apr 2023 00:05:27 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A8AA91A0EBF for ; Thu, 6 Apr 2023 04:05:26 +0000 (UTC) X-FDA: 80649626652.04.359FB5C Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2080.outbound.protection.outlook.com [40.107.94.80]) by imf17.hostedemail.com (Postfix) with ESMTP id 38C8540003 for ; Thu, 6 Apr 2023 04:05:22 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=d8Va2jKE; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf17.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.94.80 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680753923; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=alrwpWzokdohJKiHFc8dXD2lvbqGD0Er9G1lfYJ1zxg=; b=uhHu12OTaNNjKkdjEA15aFo9QpihR6F9IcX2dKMUr9LcVg7a21ddOBwOJ7djaM/yjK0AeU z748rzeCWN/MDhnR3nX0wuaVttpWhZZ7UKgIekADw1wqlSOVS3hyDMN7TvcwsEgDlSPyGB lXNYU6ilTZdAFps2PY0aUFf5VJ8M01Y= ARC-Authentication-Results: i=2; imf17.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=d8Va2jKE; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf17.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.94.80 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1680753923; a=rsa-sha256; cv=pass; b=YeaoQLvcLFpuANeRHVTowD6vPPzAE30wd8vUoXQDN2bB+9eUra59/BrErf014LxzzhAxuk qTYyUNFAcnTbhCwgFtyQ6XhbGktxD5Y/Ffb+4McNO0XbShtXI4rO/8YhHF4kcjQU1PUI13 YtFSEfrV5eBl2WOY7QIduUp3GOQ4I8c= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GU4amnxqQKXP1mM86KLuB3mLjmjbnTEd49v1audCFGxmrsjejbz8W8PjNKh4sLlvSGtenbaXHiBRIwiwcHsaMLcA5/ivGqzQoyVgw49a/Gkzl3yrT7nQwb81vhOrpdx3jIgt92nT90j/gJtBupGBMR6ZNPyLuqfYrLDv1dz6+UKly1/5AiqbhSilROqmCdFcmfc8bzeCagO4Fso0IuZNV372jMUF1h5ZgUSJsYDaiu2VD8UlzE41Dp+fPClLt69pgg3R+e9CJ3n+oDy2DwmA38wVgHvND6znL2fxLVd5B7cxRGGl8G7qEywhW6/wD0AjgbUUETrKDrCzUkY9eJl3kw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=alrwpWzokdohJKiHFc8dXD2lvbqGD0Er9G1lfYJ1zxg=; b=JLn2yT5GYLh/V0tciIbD1fYE0mirBUDGp73AXf+LeBvtJirhwdYWeqUpSPfNKfxIj0CeVn+9RD64pySAKnyA2/UIMJqPLTaqYaACyWzCHR6IKZ0cPhvYxHsBtFmYA0aCk/AYvY3+BEFiqbtyfcH7KtnlHzAdvmNvV3xncCEIdEYetHvcHehoeak1u8Jo4QKTkZUt2oAv+wqVhVvv+WnH5oOBry2Y2EGMx5YouemlvIIpdiunzGA08tFPiTG4hCRkx0J4S4hRXCuJ45UXaNZsMHa/UJ3PYpH4cWZjfrTsdlJl5OLTO21n6uEWTLmZ6RE34o0LvHxw0hS70di1PfGthQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=linux-foundation.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=alrwpWzokdohJKiHFc8dXD2lvbqGD0Er9G1lfYJ1zxg=; b=d8Va2jKEZBLmNqGP4yF8p7fX/ef9F0jzrXzGTLvDdMOFtwSlXkFkAbkZ6ryTVDXfnper1LOX0thsMJ4XX7YcNB+evjz2TwApfvTnBZtydOFZdXQki2/0edgK3U5VRiQcid661MZEsWa1i4Wq2LIZv/+7OKmKT8vXla8m6ZGEcDzXstz+9pQwRhvS/wVWYKEi8oinFWGW0wEpgD6qD4HrLKAyicGuDDKn9PoXVn2xttUolWxe3Q/ZBUg70xscRKzsANvKMhSEBynoujHCpfpdon7RjOjs5WtTHwKaaHdacDk3eVA9TSZQim7V/P4IbSwsC1pcXRVv/oh1uFHgDYhcXQ== Received: from MW4PR03CA0261.namprd03.prod.outlook.com (2603:10b6:303:b4::26) by CY5PR12MB6598.namprd12.prod.outlook.com (2603:10b6:930:42::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.35; Thu, 6 Apr 2023 04:05:19 +0000 Received: from CO1NAM11FT061.eop-nam11.prod.protection.outlook.com (2603:10b6:303:b4:cafe::a8) by MW4PR03CA0261.outlook.office365.com (2603:10b6:303:b4::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.31 via Frontend Transport; Thu, 6 Apr 2023 04:05:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1NAM11FT061.mail.protection.outlook.com (10.13.175.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.30 via Frontend Transport; Thu, 6 Apr 2023 04:05:19 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Wed, 5 Apr 2023 21:05:17 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Wed, 5 Apr 2023 21:05:17 -0700 Received: from blueforge.nvidia.com (10.127.8.13) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.986.37 via Frontend Transport; Wed, 5 Apr 2023 21:05:17 -0700 From: John Hubbard To: Andrew Morton CC: Catalin Marinas , Will Deacon , Ard Biesheuvel , Anshuman Khandual , Mark Rutland , Kefeng Wang , Feiyang Chen , Alistair Popple , Ralph Campbell , , LKML , , John Hubbard , Subject: [PATCH] arm64/mm: don't WARN when alloc/free-ing device private pages Date: Wed, 5 Apr 2023 21:05:15 -0700 Message-ID: <20230406040515.383238-1-jhubbard@nvidia.com> X-Mailer: git-send-email 2.40.0 MIME-Version: 1.0 X-NVConfidentiality: public X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT061:EE_|CY5PR12MB6598:EE_ X-MS-Office365-Filtering-Correlation-Id: 815de773-0bf1-411d-c1ee-08db365422f6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: PBhc6zrjkfU1ffSCFTZo+l9bY9R4+anZMP8PKXCgC0h1TTz8YTSO7DFwxFNll2cDDdTQDKr+/6ugilCyPqAm91BbF5+Jeqdu2UPvbEFe/bwkK3xNNEoDqOAQHV6RA7WaqT4La7gvo8pvRsg8BCn1Y1KQz0QMtYz1rH8foM47TYhA0CJE1He2kdNYVOPV7IHWuwmOFGlMA90pI9bQtD0/WmozAGlz9zLpbOskpZYkUkwKJG8YP6o7qL6p0xmn69y0s3jfNyxjpf/hpxeU1VLGZFQMenb4P48kyWZw6CKEpsaRbJcZsasApBOJm64SxjkGvDevKETfLZh1d6dywqIC6SSzvowxkm0dpIgYuI2R6Wp2duyDq4QH9+SrRDpZ9jMe/VG2j6ZS5P36nr+y/QkfQCcDqYsMhVISKmeW5GexBWga8ZeBsU875+43KaEHNt3kAx/mhOiGcmkf/C/oLfJ63HkFdGUyDiz027exYyBTOuyEM2O8ZJkI4s0bRPOujideWWp2FboUNsjzzIbs+5acXkqapxi2dhIkpEYBlwRTwQs4CNZM50OEfKYfsLxWP5GQphbZpsdMoucNMbBTFcQiL4FPJFzj5msMMs6MKq35Cwc7LgcHHcd5gMqaBrNTVyE/j1NBXhNNgbDr5gsl4L3uH36BDZpgMqfyhqoeN3jzrNIJ01MFTVpoIL/H2u6noaDgf+fN2WI6HLsOjNqpXaPi/9vHQQdHhihLggEwjTzoUgkzg73g6yr8sOPZn7w8HlVB X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230028)(4636009)(396003)(376002)(39860400002)(346002)(136003)(451199021)(46966006)(36840700001)(40470700004)(36860700001)(82740400003)(356005)(7636003)(1076003)(7696005)(47076005)(2616005)(26005)(336012)(186003)(83380400001)(2906002)(7416002)(36756003)(426003)(5660300002)(40460700003)(70206006)(86362001)(316002)(54906003)(8676002)(8936002)(478600001)(41300700001)(40480700001)(82310400005)(4326008)(6916009)(70586007);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Apr 2023 04:05:19.4025 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 815de773-0bf1-411d-c1ee-08db365422f6 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT061.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6598 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 38C8540003 X-Rspam-User: X-Stat-Signature: xts3b7ysq89gpqgsgw17c4wausxbqd6n X-HE-Tag: 1680753922-768644 X-HE-Meta: U2FsdGVkX1/xoepMyRQr56t0rtokVx/hZKZGyHkqcInY001xfP2ISn/oQ4HOmgIl5DL4RoJDL99rulw/6iMxsziB+05PilIgQPnHud96OFJvpdeAD+0EPTK5/K5PEb/ahiKplQCKSIRXupfYanBBbs18na/tJlAwBvfxSLj3UgJlvO7Oa9S3LYdBt4Hb0yyjdqPWceBZNFeI2qu3JL4AyL15iZSQJ2cVJWujW3NVP/eNMCCLGgtFuBqOQ+Ds0++vTqIT1Se21JnMaqH5r9mEQLtQnC6GuEsfqafgZYdbZhfLHPS0nZgts5kkbs8fDTBEgujIllAwj/5wNY849RcqB369wrmcAGVAuni9vCRAvEfQGTcYfQBFAhjYNtsSHSd5SzUpDsXgx4CKYddma1KmfRso89gsrVMd6j5kT8AM8NhqlNcRu5r1/buiJxx2hQdCfEpgt9+9Onk3WsgH7AyXQoJlNzA6yekCf/LRtr/BrQXkgyHttVbQ9EVyl440FwE5ohU7Ny7CNZHJLsmaDmLwN9ZbXvcExda4xeLBo71bMwxv1WqI/XClk9KM5MfHnjKWvxtxwHVI5PsSPu9mCQmJuz99U1f4PHNpJnC3OO/QIwWtQhCKlRYkP48t7aYixD3N45oxCvR1BjoTeWS6hbftIUXC2zmAhoh3mHiuiPCz43PP+wcP7ebNPe3OeDCuxJ801kR3ZDtguoBf3aBo5uCBcRPXQkpTrVnLYhRMvXCMdT84jghz8h3t9zlLxBuVsDnrFeZopp/x5ML65xtMPULaFVV7zzwzOP7cLlk4O0+0/MfG12Hnm2PpUa2qJcc3I3v+aOtneu2xxwlHtmL6JnMizEqu510W6fvZWx0OysMe6VyolwjTZFYX8lQL3uh6t4BgLhwrRFbvoAndxy/kXrEjNdgWJmIn9z3OXm6L542snobWz3fhKSodwEEUEgDVb/xGZ5fYNQ1wObGav7ZwgN9 UaQQ5ICm b0big6ZiYaEej6c2SWEKCiW70ebM3UjvhnUhlDCjllIzJBHCOlBRHztrJuXgYyCLlywo1RRgBlZk//pLIa62R4iBpvqfDB76zCD//bHypsXHmL70HxypDYcfMZ2x0h7EJYcvRsq2JizGyLWXCydLfbabUrlS9DAxyRW5AfnSucuCjl2CUcwG1gLMP+XKwdCOe3KtJ9gIE8R84t0GAm5bpkiiWgRyhou7fbfbejG5zKhGO1JvndWKfjsex9CueD+MFvnWz2HeUd+CI3VrqkHa/A4CP49coK92G2OBUz0MaY/iO/FCoKgs3L0isCqQ3f5lOAvMV X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Although CONFIG_DEVICE_PRIVATE and hmm_range_fault() and related functionality was first developed on x86, it also works on arm64. However, when trying this out on an arm64 system, it turns out that there is a massive slowdown during the setup and teardown phases. This slowdown is due to lots of calls to WARN_ON()'s that are checking for pages that are out of the physical range for the CPU. However, that's a design feature of device private pages: they are specfically chosen in order to be outside of the range of the CPU's true physical pages. x86 doesn't have this warning. It only checks that pages are properly aligned. I've shown a comparison below between x86 (which works well) and arm64 (which has these warnings). memunmap_pages() pageunmap_range() if (pgmap->type == MEMORY_DEVICE_PRIVATE) __remove_pages() __remove_section() sparse_remove_section() section_deactivate() depopulate_section_memmap() /* arch/arm64/mm/mmu.c */ vmemmap_free() { WARN_ON((start < VMEMMAP_START) || (end > VMEMMAP_END)); ... } /* arch/x86/mm/init_64.c */ vmemmap_free() { VM_BUG_ON(!PAGE_ALIGNED(start)); VM_BUG_ON(!PAGE_ALIGNED(end)); ... } So, the warning is a false positive for this case. Therefore, skip the warning if CONFIG_DEVICE_PRIVATE is set. Signed-off-by: John Hubbard cc: --- arch/arm64/mm/mmu.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 6f9d8898a025..d5c9b611a8d1 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1157,8 +1157,10 @@ int __meminit vmemmap_check_pmd(pmd_t *pmdp, int node, int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, struct vmem_altmap *altmap) { +/* Device private pages are outside of the CPU's physical page range. */ +#ifndef CONFIG_DEVICE_PRIVATE WARN_ON((start < VMEMMAP_START) || (end > VMEMMAP_END)); - +#endif if (!IS_ENABLED(CONFIG_ARM64_4K_PAGES)) return vmemmap_populate_basepages(start, end, node, altmap); else @@ -1169,8 +1171,10 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, void vmemmap_free(unsigned long start, unsigned long end, struct vmem_altmap *altmap) { +/* Device private pages are outside of the CPU's physical page range. */ +#ifndef CONFIG_DEVICE_PRIVATE WARN_ON((start < VMEMMAP_START) || (end > VMEMMAP_END)); - +#endif unmap_hotplug_range(start, end, true, altmap); free_empty_tables(start, end, VMEMMAP_START, VMEMMAP_END); }