From patchwork Thu Jul 6 08:58:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13303376 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCA6BEB64DC for ; Thu, 6 Jul 2023 08:59:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6DF688D0006; Thu, 6 Jul 2023 04:59:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 690D08D0001; Thu, 6 Jul 2023 04:59:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 52F828D0006; Thu, 6 Jul 2023 04:59:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 46C028D0001 for ; Thu, 6 Jul 2023 04:59:05 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 099E6160C36 for ; Thu, 6 Jul 2023 08:59:05 +0000 (UTC) X-FDA: 80980587450.19.1EBCBCB Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by imf25.hostedemail.com (Postfix) with ESMTP id A7F7DA0012 for ; Thu, 6 Jul 2023 08:59:02 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=iG1zm2bc; spf=pass (imf25.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688633942; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wRpOvZEenjvDphBufXXJXjaJM9fWCo+EyUTVYUwsbTE=; b=ZUpMYz/BqyQEuAYn1yhFBKqhDoSt1xZDcfHwD9ZKpkyP3//tcCIjiUwurnn2oGznDsAiwJ 6yYTF+N7sj0Uf60Nti9MgAdstB5Bgak4kOhjIi0E8O3wM7ta0NTTHNKNgt8a5En3gmYoZw n0qxibMQIjhE9VTNQ1fqZwRZQh10EDg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688633942; a=rsa-sha256; cv=none; b=6e0riIBdw9QoTAYMxjW8UBXVLlUxY+7epQxZDS9oqJgVfaoZzk3d2FL5ziZZjjgSOrlv35 lWEjDZW5rHYxL2Ygjh9/jqKoMcbJggfGCeThJuaFUlOY14cyQ0hCgUEkDuL02rV+t9aPqz WoOSV8L+/EK1tPCJRxOSkN/FNWunaQs= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=iG1zm2bc; spf=pass (imf25.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3668mSRt012947; Thu, 6 Jul 2023 08:58:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=wRpOvZEenjvDphBufXXJXjaJM9fWCo+EyUTVYUwsbTE=; b=iG1zm2bcmvr0m3aEc3ZhzyeVEctsTzRpc4vXyqFbQteqEDyPDMxfJAuBcfVKVMOrh7vc HJ6wFCK9Wim7gx4d+THahjsKi9pnDR2ROsQg4B+96CgKwTXFI7A1QVJZ19VEDwqrVD6Y LuplTicjvt5CL/Fx0ESGvelSpXE4wj81HLaIi6sdvRgDhdoWtWLVEKEUeo20aPTcwN7y YupKnkwm6rec/UlYb3ltTE/ziA0t4/6726wU6nxIyvhXf0Bsez69y9RgSn6v8OKcpKkk owQzgyaHzDgM6aIHLi1qTMzYUG7Lg9/W5pAhOsg9TphdUTP+t1hwaAECNx14ut1jaCnt pw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3rnte408pg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 06 Jul 2023 08:58:51 +0000 Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3668oUkj024681; Thu, 6 Jul 2023 08:58:50 GMT Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3rnte408nv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 06 Jul 2023 08:58:50 +0000 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3668U1BB004414; Thu, 6 Jul 2023 08:58:49 GMT Received: from smtprelay03.wdc07v.mail.ibm.com ([9.208.129.113]) by ppma01wdc.us.ibm.com (PPS) with ESMTPS id 3rjbs5quwr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 06 Jul 2023 08:58:49 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay03.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 3668wmjR66191850 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 6 Jul 2023 08:58:48 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2DAF858058; Thu, 6 Jul 2023 08:58:48 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 73A7F58057; Thu, 6 Jul 2023 08:58:43 +0000 (GMT) Received: from skywalker.in.ibm.com (unknown [9.109.212.144]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Thu, 6 Jul 2023 08:58:43 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v3 03/13] mm/vmemmap: Improve vmemmap_can_optimize and allow architectures to override Date: Thu, 6 Jul 2023 14:28:09 +0530 Message-ID: <20230706085819.827761-4-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230706085819.827761-1-aneesh.kumar@linux.ibm.com> References: <20230706085819.827761-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: w85QwxJHUvyU5O4k89ARA3kbIUQVepph X-Proofpoint-ORIG-GUID: q7Yv54e6Sp5kY0GDyNPjEzDJrq-2NIvR X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-07-06_05,2023-07-06_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1011 lowpriorityscore=0 bulkscore=0 phishscore=0 impostorscore=0 priorityscore=1501 malwarescore=0 adultscore=0 mlxlogscore=943 mlxscore=0 suspectscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2307060075 X-Stat-Signature: hqmmpkyeup45unhaxw4qsb1ka6pj5y9b X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: A7F7DA0012 X-Rspam-User: X-HE-Tag: 1688633942-843042 X-HE-Meta: U2FsdGVkX19K1paxFMFcSLYqaUBqST0dh25955R9+hTnc8imnFOwPVg1tMZdBrkCyy5MTudLy1ye481JzAINI5Dq6NGVRaNfxm78wvhr21r5xFmY4dUrtPgaGPJdtEcLxqxf+RUq1qG2d6aAtKaomEoGz28C1y7NNOj26QLnUkOFdV1KSgsjPM5PGy03Juk0MxLkkH1GrvhRaiVKD/5uX77dB9/FADCACO6hAeasoXmh0P1pIeqSny16qzVzRs+DZ3w7wq+tbxT5zRT6NPEj7toIzv6JAEP+WIVKPWNpvYXZ8e4XmXCwszc5B7C1fGRjWag9X/ler8UlexaH2ozrKpiG0WJcD2VD1a6uU1EJlyRN20uBPRQUcg94v5plPpfzscHdf59LeN+Ow1ZlurTbxFO/+t3wpRX99+utFuUH+LZbEeMgcCrsPYK+CXysEWdNgtJYl9SZNaDlqfY8u4MaKVAFWZFonqxNp5eybQu6dpWCd+9XZgsXw2jlSzIv+XkD7ie/WVUWDqBv2VuI9ZSp2zOvToLNh3lbkqcw6QhedYqCEScWvHN4Qrldy78dfS8XcBCF83uvZgsQYbBAY0E03Q30QMym3/2zoEfSxn0hpboULEw+HJDyiNrmvdXgX7TUeXUBbFvFG6Eere3VzAl6Cw5z5gb/G/BGPcZKJGWWwXcEhIcC4SpJsVIZXYnxodm9vy37nuNdKHXDJMDcsyT5ZI8h5jKr/9f03qqae4IRjdonuSHPsOEpcPTWzodRt6dmsbFV7CTd1A+eRPD1zD7H9sPfeByA0+RKgM5D1GAi1E4u0FPsE+wt+aEvcYGt7mXvLRiqU7A3JJ7EOl4EEh02P7ZoUMWdf1QjbdfH2zvaslYQNNKT5VEOhMGafp2d8WegoPfib1rie4KSEIG9CGWRqZS6O3xg+wEh0K53ZwB7TEQ6oYt/o9CqUXtlHaNrbGopsL3xdOpzu360oG9QnXk ygVyuJu4 +Crsc+Gd6BfnFoGzW9jkaiEoccSaufMF7/m4PBFE3qPMxTYnp7gfdEd+c2T2pVpLlG45wcG/9bliSsaRfg3RhlHjPzmDYeYsybepbF0yPyV7VzS3Ir+4Ufq2nbYaLQxi2V42Q3WNKRKKDk4eNbgp5IEIlfA9IYEju673RHr1h+UbCCPxLSlLcH8l7WgBzJn+AyXWXwLjtHnIepTsINsFxeMkLy8AlrpLbkPpYCP9GQdo6SOwVJBW4JQk5r7F/pxlwgkX9fBqIkloVBLXXZIJJ2K9OBA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: dax vmemmap optimization requires a minimum of 2 PAGE_SIZE area within vmemmap such that tail page mapping can point to the second PAGE_SIZE area. Enforce that in vmemmap_can_optimize() function. Architectures like powerpc also want to enable vmemmap optimization conditionally (only with radix MMU translation). Hence allow architecture override. Signed-off-by: Aneesh Kumar K.V --- include/linux/mm.h | 27 +++++++++++++++++++++++---- mm/mm_init.c | 2 +- 2 files changed, 24 insertions(+), 5 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 2dd73e4f3d8e..1a2234ee14d2 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3639,13 +3639,32 @@ void vmemmap_free(unsigned long start, unsigned long end, struct vmem_altmap *altmap); #endif +#define VMEMMAP_RESERVE_NR 2 #ifdef CONFIG_ARCH_WANT_OPTIMIZE_VMEMMAP -static inline bool vmemmap_can_optimize(struct vmem_altmap *altmap, - struct dev_pagemap *pgmap) +static inline bool __vmemmap_can_optimize(struct vmem_altmap *altmap, + struct dev_pagemap *pgmap) { - return is_power_of_2(sizeof(struct page)) && - pgmap && (pgmap_vmemmap_nr(pgmap) > 1) && !altmap; + unsigned long nr_pages; + unsigned long nr_vmemmap_pages; + + if (!pgmap || !is_power_of_2(sizeof(struct page))) + return false; + + nr_pages = pgmap_vmemmap_nr(pgmap); + nr_vmemmap_pages = ((nr_pages * sizeof(struct page)) >> PAGE_SHIFT); + /* + * For vmemmap optimization with DAX we need minimum 2 vmemmap + * pages. See layout diagram in Documentation/mm/vmemmap_dedup.rst + */ + return !altmap && (nr_vmemmap_pages > VMEMMAP_RESERVE_NR); } +/* + * If we don't have an architecture override, use the generic rule + */ +#ifndef vmemmap_can_optimize +#define vmemmap_can_optimize __vmemmap_can_optimize +#endif + #else static inline bool vmemmap_can_optimize(struct vmem_altmap *altmap, struct dev_pagemap *pgmap) diff --git a/mm/mm_init.c b/mm/mm_init.c index a1963c3322af..245ac69b66a5 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -1020,7 +1020,7 @@ static inline unsigned long compound_nr_pages(struct vmem_altmap *altmap, if (!vmemmap_can_optimize(altmap, pgmap)) return pgmap_vmemmap_nr(pgmap); - return 2 * (PAGE_SIZE / sizeof(struct page)); + return VMEMMAP_RESERVE_NR * (PAGE_SIZE / sizeof(struct page)); } static void __ref memmap_init_compound(struct page *head,