From patchwork Fri Jun 16 11:08:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7539FEB64D7 for ; Fri, 16 Jun 2023 11:09:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 036126B0074; Fri, 16 Jun 2023 07:09:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F01D66B0078; Fri, 16 Jun 2023 07:09:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA21E8E0001; Fri, 16 Jun 2023 07:09:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C74A36B0074 for ; Fri, 16 Jun 2023 07:09:52 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id A8E8C120C8B for ; Fri, 16 Jun 2023 11:09:52 +0000 (UTC) X-FDA: 80908341024.25.2107F60 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf19.hostedemail.com (Postfix) with ESMTP id 2E4D41A000C for ; Fri, 16 Jun 2023 11:09:49 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=Bf6hySyB; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf19.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686913790; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2nIHSSUSxuasnMPG57XtfkGpKEslBIgIjQykIV+BjZI=; b=farFL0WpVu2cc/xIzfA7v8bwp8SCQEeor1QDH/mYQQKuWSqIfbOsrBwpLYEcTM8/04B5lr zDTeGluSMEJjoHXbOyhuF+gawldNAgV9pUe3GrYCljPnVQdYQfRQi8ttwUoumsloH8EofT 0xtkOi5ckf2KWN60enEYIAo86z+JIrk= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=Bf6hySyB; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf19.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686913790; a=rsa-sha256; cv=none; b=qH9z+kFiqV9pa3GCmceyBhDRIbW5zzp91f9RdO7w1eMm4QSheK6r2wQWtVGNEhEWxFxPTH WufGqRUX9SEZ5pmv0oqEKN4IoESULbWtZmkunwU1Ku/n8MP3F8FjycTOBIjKMMAShyy6oS K/uiXVFtB4LU2avMaGE2w3Z9bajEMsI= Received: from pps.filterd (m0353728.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAowna018749; Fri, 16 Jun 2023 11:09:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=2nIHSSUSxuasnMPG57XtfkGpKEslBIgIjQykIV+BjZI=; b=Bf6hySyB9OdGKdq/zlFMV6Xr9cWAzlH7wJSONsWfT9w6IlizdKuFxEG2hXb9wFf3zYS+ I1kAHZvcNwMtve6upiULELVaAOB84u3O6n3l2/RnTX/TB5X4nl8eVyUUAdxUDgbsV/A3 q4ZVUzAmM4aKuXosZkVXIm+9oZxHa5D5YHTKnhSy+PjD67Rzhqt6ng90hPWKoSU0E3bs TIsfk261FQOOVFY8/PKZJRxJaQ2wGs7VLvoyXTfeUh9CF8Szmr3z2ntwcCkPXW8wlUqL SsFXhC5Z//cbIfGXvqlCSWAAMDy7+6OZeh/bPXYq0kDy+EC6ZsI1OTUZoiaygT418QB2 lw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8pbm0f7d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:39 +0000 Received: from m0353728.ppops.net (m0353728.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GApBnB019196; Fri, 16 Jun 2023 11:09:38 GMT Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8pbm0f6h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:38 +0000 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAlmZr010717; Fri, 16 Jun 2023 11:09:36 GMT Received: from smtprelay03.dal12v.mail.ibm.com ([9.208.130.98]) by ppma04wdc.us.ibm.com (PPS) with ESMTPS id 3r4gt5yhe5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:36 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay03.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GB9ZFJ28967198 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:09:35 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 479845805C; Fri, 16 Jun 2023 11:09:35 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3F82C58058; Fri, 16 Jun 2023 11:09:29 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:09:28 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 08/16] mm/vmemmap: Improve vmemmap_can_optimize and allow architectures to override Date: Fri, 16 Jun 2023 16:38:18 +0530 Message-Id: <20230616110826.344417-9-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: kYPxs0AVpUyxr9Dw0HjJ-6Dc32dedHBR X-Proofpoint-ORIG-GUID: RZSIYIkuDjT9w6LwUdE6d3PlYbkqnWEB X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_06,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 malwarescore=0 bulkscore=0 spamscore=0 mlxlogscore=983 phishscore=0 adultscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 priorityscore=1501 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160099 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 2E4D41A000C X-Stat-Signature: d8i4uced67cfa89y8ncndcc5tc6ikbco X-HE-Tag: 1686913789-478336 X-HE-Meta: U2FsdGVkX19+ZOLydahE9fhjcxCpxdLWvNEK5dq6taU7nsznlFK99xNunMuTQjdj2T+WaNVsdMtUzm64BYGe3J/3HTVzp12SN35KuJ+YYq9B1T4xo3CQNmVZ/LZSbixtnaVKdzlwdsDFHkBE1NOA0KpanCCAoUih/ZmIX24bDh48P8ZlckL6cfj01nCFxPmQlhM6a5G7wZQeFgmdhFP5IX1fyzuPgMai1WXjUUZQb9y7tPLGG1lk3URY1L+W5LGERly95ZsPx68tVu125FFzy7gSNZVzQczcdoC6wC6WroORL3pC1DODbd6JJWbcByh+Fj41nO3kQYvcEYpHOoJCofVdiodGw3HxGhDfMbMdTKAe4Egya5GGZZGrPYOktG2/1T2+JpPEyETYL8NaWw2eZ6gFvvO43gQ+hcaMRYgg8gVWFboPhZ2ostuI4qapgwC3myvOywB2COscr4iNCrj/BoZsbvZjnh/EHa8V6VolXBu/C6ywh6CMW4MTW+nthARZDexdyLPGJdy6azU7m+IjOLgUI8qUFS4H/blKi/VXrVP1YrWMgvYVDlqnWbaPFh6V1GHnwbI2FlUWgDYOgCdZMINkFgTLXwLOHyhwYrZ7z0HcfWMzBi2mpVv9FlmOuuN+E5HFWKh5lf2axmbHvX/p7H2Z62t6ts3bu51s+JL3xWty0EzJilF0wrAvRqAthIC3HfGqsOBQH8tpRi2Ipe0wAsNjuB0brCfC2f2Zbsp6Age8X5y4aDZhXpQOqvc6I81DEHnli82yTuxKy59ULz1mBrUyZdD+TdnpKK0+GwoSlIjA3Papyow/xbFXq+x9n7yP7KAZ+vV9Gb1DvahUMJ57MYzSmMrzP7GcSvlADjAGHQsVvm+vQhcu1oL5ScV042GfjgZhiuf3eBnwMyerbnHm1x+fi7lpXKGQEXfV5IgzwxKNOof7FCEoSup76Oepn1Z9MnuctzCFYMraXs2ozJu JiYPb2lq fXVSUelOq2EqOXHUEPyBhSMkbWqMYQUVAezKrTSM9ONzxgkcMpZYTVaA3BKR2nnr74YXzoW0yoabq3v6yenb+A/yqXZSYkoBBVLBMsTxAp66fTjIaJxfzcZ5FMKyMAW38tQCdaxwoYEF1expWdGpKubo5iayM2HJCySeBiYpNQcSn/ykmEcVZiYV2bkOC9OvT+A3I X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: dax vmemmap optimization requires a minimum of 2 PAGE_SIZE area within vmemmap such that tail page mapping can point to the second PAGE_SIZE area. Enforce that in vmemmap_can_optimize() function. Architectures like powerpc also want to enable vmemmap optimization conditionally (only with radix MMU translation). Hence allow architecture override. Signed-off-by: Aneesh Kumar K.V --- include/linux/mm.h | 30 ++++++++++++++++++++++++++---- mm/mm_init.c | 2 +- 2 files changed, 27 insertions(+), 5 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 27ce77080c79..9a45e61cd83f 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -31,6 +31,8 @@ #include #include +#include + struct mempolicy; struct anon_vma; struct anon_vma_chain; @@ -3550,13 +3552,33 @@ void vmemmap_free(unsigned long start, unsigned long end, struct vmem_altmap *altmap); #endif +#define VMEMMAP_RESERVE_NR 2 #ifdef CONFIG_ARCH_WANT_OPTIMIZE_VMEMMAP -static inline bool vmemmap_can_optimize(struct vmem_altmap *altmap, - struct dev_pagemap *pgmap) +static inline bool __vmemmap_can_optimize(struct vmem_altmap *altmap, + struct dev_pagemap *pgmap) { - return is_power_of_2(sizeof(struct page)) && - pgmap && (pgmap_vmemmap_nr(pgmap) > 1) && !altmap; + if (pgmap) { + unsigned long nr_pages; + unsigned long nr_vmemmap_pages; + + nr_pages = pgmap_vmemmap_nr(pgmap); + nr_vmemmap_pages = ((nr_pages * sizeof(struct page)) >> PAGE_SHIFT); + /* + * For vmemmap optimization with DAX we need minimum 2 vmemmap + * pages. See layout diagram in Documentation/mm/vmemmap_dedup.rst + */ + return is_power_of_2(sizeof(struct page)) && + (nr_vmemmap_pages > VMEMMAP_RESERVE_NR) && !altmap; + } + return false; } +/* + * If we don't have an architecture override, use the generic rule + */ +#ifndef vmemmap_can_optimize +#define vmemmap_can_optimize __vmemmap_can_optimize +#endif + #else static inline bool vmemmap_can_optimize(struct vmem_altmap *altmap, struct dev_pagemap *pgmap) diff --git a/mm/mm_init.c b/mm/mm_init.c index 7f7f9c677854..d1676afc94f1 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -1020,7 +1020,7 @@ static inline unsigned long compound_nr_pages(struct vmem_altmap *altmap, if (!vmemmap_can_optimize(altmap, pgmap)) return pgmap_vmemmap_nr(pgmap); - return 2 * (PAGE_SIZE / sizeof(struct page)); + return VMEMMAP_RESERVE_NR * (PAGE_SIZE / sizeof(struct page)); } static void __ref memmap_init_compound(struct page *head,