From patchwork Fri Jun 16 11:08:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282622 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7BF4EEB64D8 for ; Fri, 16 Jun 2023 11:09:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A717D6B0075; Fri, 16 Jun 2023 07:09:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A1F908E0001; Fri, 16 Jun 2023 07:09:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8E8666B007B; Fri, 16 Jun 2023 07:09:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 7B7786B0075 for ; Fri, 16 Jun 2023 07:09:06 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 3C639A0BEA for ; Fri, 16 Jun 2023 11:09:06 +0000 (UTC) X-FDA: 80908339092.03.F0D2CE0 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf21.hostedemail.com (Postfix) with ESMTP id D14321C0014 for ; Fri, 16 Jun 2023 11:09:03 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=XVwPhqRw; spf=pass (imf21.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686913744; a=rsa-sha256; cv=none; b=tmOcruHPOWiuMSDPSMwwnCh7Jc8RMGk365XStyNFmNhr885N0maHXRmCk5heEUspKTX38P ebSUygCXHdmEP2QWu0BgdigxWyl3Ng+I0kPX2vFIM5uv3N+g42UpPO0xBjmYnEzyNJLCYm uUDRoD1kX+o4nY4SWkDEdlFxljNAdcw= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=XVwPhqRw; spf=pass (imf21.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686913744; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xtQrMo4AYLeio8end8gS7yq8+iUqWQYcit2hc/5pCeQ=; b=EQXyDgBVIMSzvjPhgSC4ZBojnNmKODhhXsugAlW22S/DgZAFugs5fRrQTTN6AvEpOdZdpd kcb5q6rblBTmwv9ftlAcVTzHbVmCizOICM3O325EliJZmHBfuBb6VasZvv9BMVqVVO+0Hc QMQQVz1m+mMNrDQSx/Wy9O++AreEIng= Received: from pps.filterd (m0353728.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAow1I018729; Fri, 16 Jun 2023 11:08:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=xtQrMo4AYLeio8end8gS7yq8+iUqWQYcit2hc/5pCeQ=; b=XVwPhqRw3p4SrJ5IcH5PoeqFn2RPfqqC6b4LTaX0eJG91/kiG0HdjUvLEhpnWgbYI+5U LtX9JHWfCTB25nhi765A0xBoJGS81s0j1Z4pXIXaFdcKbznlVEUz1MLsyQfEYMi1Gvt1 6nhFNf+XWrLLld8X1Wfd5IEiVOpmet53XStDVVLbOEYh2AWkReED1aeh0v/m1FvnnIrp AgGTMmfXUcvb7A8Yw3fuaHB2pzsSp36S4vt/t3Jp0Fa/JdC1/2LyhSVTVUx/ztFtAsAw NK9fjEpANgGCQtq+R2+x30AHseOx6JZzm4yePyKdL/4fM6oL9XRaBRryCHPGfRwDPgzu cw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8pbm0eh8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:08:52 +0000 Received: from m0353728.ppops.net (m0353728.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GApC47019250; Fri, 16 Jun 2023 11:08:51 GMT Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8pbm0egn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:08:51 +0000 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35G7QPPY014531; Fri, 16 Jun 2023 11:08:50 GMT Received: from smtprelay05.dal12v.mail.ibm.com ([9.208.130.101]) by ppma02wdc.us.ibm.com (PPS) with ESMTPS id 3r4gt5yjvf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:08:50 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay05.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GB8nYa60162416 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:08:49 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 661095805B; Fri, 16 Jun 2023 11:08:49 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B96A558059; Fri, 16 Jun 2023 11:08:42 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:08:42 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 01/16] powerpc/mm/book3s64: Use pmdp_ptep helper instead of typecasting. Date: Fri, 16 Jun 2023 16:38:11 +0530 Message-Id: <20230616110826.344417-2-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: ksD9Yy4yvUoitJrCWbkOKROPOgwte6dL X-Proofpoint-ORIG-GUID: 5vi4Ev013f7LObh82qNDVa2uIIhIsZro X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_06,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 malwarescore=0 bulkscore=0 spamscore=0 mlxlogscore=889 phishscore=0 adultscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 priorityscore=1501 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160099 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: D14321C0014 X-Stat-Signature: q8ufi6afhm61zqjayim8y5nti5m7x1or X-Rspam-User: X-HE-Tag: 1686913743-430307 X-HE-Meta: U2FsdGVkX1+IMgpAb7Wx2tSjab2z+ZMrWcCYB/cY3UkuobuhS+lTueV+EKmBSbN8J6g/WaacStXzMO+6V2L6iy+5NhCGAavl9QZnLf4+Wx03oC2H9pMAEpj0MOq/8CT94pIP0nJqEZCge3lRlAFi0Mw7cOX7EI9Ebs0k1itxskIYkbN7Q1ouOioch/1m44bFDC31/ktZl2FrWxnSfVCt8MgG1392mniBJp0AS6o/PI2yIR/gw8aIWIDdji9WwnU2MA2IGWbF62+0zRXzt9Wm+OUNfhsie2GBX9x86NVqtEYVbNm8ESIffvjbsdibTtA6YdPDyOlizki66AWUbb9WT3qNdWq08HRPNLVNOZtWV+mwHlYsSbGymmXIjw5WlJHg41HrDeMB7tL64HohVFLzw60ElII8T9vbTsHhz6FyT0DHd3pF0QET5B36II/6QFmLYLNputN9xqmRiTpkotfVk0FVmcaY3Q9gpsyc6Ae5xy3A8orVWfu7jpnef/+bCWeLAzKOu4XXMIWqPklRpcYAASuetXFYwo40Id1s107ztiB0Tm6WwBy+EjdHAV/4Z4aZ0rWrI5aIdyZ++D/dG8l5+jIfb0DZts4Nc+fvx88JeSCMp1AY18OjxFRcZT4Y7ZORW72F+KrztFFTyF0rkBL00sp2k6r8Oqezc02WkN3U8ZPI07H1Nd1z8Y1CiGjJANjkaXniC4bCaUaOExbxlBxOavwpmetADJWxCKZjw0oGY2cu2VEv4R1nMklcxA5KJGgkAVwpMM4geg3SDHZTiZE4EuHHMBlnXKXzBHW7kiNEup2H1CYJ9U1NEqG/nV3XjcHfpg/uiCkk+uVb6I2LLUsurs157M525bCv8BH3R1pFgBbmvuzm4Y3Z2Yi9v2v8vFtqDyO5rQedgHPH3G7wHFSeHWntAYC0zzzLyLh/elObudfVuk0M0rasyjZCE1DdGmWV9souAw1pU/ZbDaEcjuO Q/9eGJb/ mlRmC/MZEIeJQhRbnYXl2zZgHjaKFaxeD+fGBUHAtCUvAvmhAynrDM0Y+/pun6tG6plFpPtDL1FKm2U24X5tMCv+QliQVMESEZmkcL/GCnCa2i24P1SwvSChlllvMQXPLquM52LKeutfTKAb92GIXD/osHn7/FA6wboG7ACFp+YAU5ABs90FC1UvXvU4PeN86LBj69/YMo+JTeHoq/q8y2+JaAQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: No functional change in this patch. Signed-off-by: Aneesh Kumar K.V --- arch/powerpc/mm/book3s64/radix_pgtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c index 2297aa764ecd..5f8c6fbe8a69 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -952,7 +952,7 @@ unsigned long radix__pmd_hugepage_update(struct mm_struct *mm, unsigned long add assert_spin_locked(pmd_lockptr(mm, pmdp)); #endif - old = radix__pte_update(mm, addr, (pte_t *)pmdp, clr, set, 1); + old = radix__pte_update(mm, addr, pmdp_ptep(pmdp), clr, set, 1); trace_hugepage_update(addr, old, clr, set); return old; From patchwork Fri Jun 16 11:08:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81FB4EB64D8 for ; Fri, 16 Jun 2023 11:09:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1BDC36B0078; Fri, 16 Jun 2023 07:09:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 16DE86B007B; Fri, 16 Jun 2023 07:09:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 00E9F8E0001; Fri, 16 Jun 2023 07:09:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E64C16B0078 for ; Fri, 16 Jun 2023 07:09:12 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 94E5F140C1C for ; Fri, 16 Jun 2023 11:09:12 +0000 (UTC) X-FDA: 80908339344.20.410425E Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf10.hostedemail.com (Postfix) with ESMTP id 2AC0FC002C for ; Fri, 16 Jun 2023 11:09:09 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=s1WUfHz6; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf10.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686913750; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LnAST0ri6tuMUS6B67naBvkx2n1hStwtP2VkWo2LbwI=; b=YJzgjX/wJl7R7R+o/X9JX4xs38lMbk/7eBhZVKZPBZZm5BKQNY695Cx7EaHTxOyGlJHlJU gwgerA3YpzAatf7CC8dK0zSXyZ6GN1i9LvikVjdg3nRTvHUASjgFjsEKSCOmnUU3utJtHk RWfuiSHNi27Uf30nxvYtc9kbs3USHg8= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=s1WUfHz6; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf10.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686913750; a=rsa-sha256; cv=none; b=a6TjstlY8pn7/tyFQxIVgZO7zzysXPpiFKdb9tZfJLtYaNmR3Wl3GjGDMV0AHULeylKefc 8SXdd6AC64B4YWDNvfhn0SX9cjV1bseTduDEoasnjNDe8E96QEyj1TT1c3MbFcSEgs5CeB Sb38ERQF+YBvI8OqchJOINYcYhMxZdE= Received: from pps.filterd (m0353727.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAl6dQ026270; Fri, 16 Jun 2023 11:09:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=LnAST0ri6tuMUS6B67naBvkx2n1hStwtP2VkWo2LbwI=; b=s1WUfHz6hDx/YEMeoLQeVT8EWgcv+9p7uOrDBjPdnfqvJ07eb4MT9YLzfqupvekE847x RLmdjpNlEW5K/AU1uJFFhOZbdWjYovGUMrgoP4Um566KKtmOsMm8EwsMAQfHsk2Cv2Pz 1PIAgmlFR0PC932QL9kh7C+OAdFf0KHQ2XBM9S2pwbfthy0mTivrKtdNKLWyL0pmM1N5 6YV4AoDqkix7utmRktKV7vomyY/NMsN6KksJjg9kHpYXeFqIvSPK1sxoWItt9dx3WsNA peouFPESwiUuZGHS3JJT8hHEIZc3wbtmy/MXK+qWaw61U2sRdytR8nNQDkMtccWhCQFa Gw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9ngg42-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:08:59 +0000 Received: from m0353727.ppops.net (m0353727.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GAnGCC031056; Fri, 16 Jun 2023 11:08:58 GMT Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9ngg3d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:08:58 +0000 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAklnT014069; Fri, 16 Jun 2023 11:08:57 GMT Received: from smtprelay03.dal12v.mail.ibm.com ([9.208.130.98]) by ppma05wdc.us.ibm.com (PPS) with ESMTPS id 3r4gt5yfhy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:08:57 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay03.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GB8uf248038162 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:08:56 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5D4585805B; Fri, 16 Jun 2023 11:08:56 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2EF4058058; Fri, 16 Jun 2023 11:08:50 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:08:49 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 02/16] powerpc/book3s64/mm: mmu_vmemmap_psize is used by radix Date: Fri, 16 Jun 2023 16:38:12 +0530 Message-Id: <20230616110826.344417-3-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: aERBqponElN9irMQ6o7ahQLLC8ve-T24 X-Proofpoint-ORIG-GUID: cK2HMq7nYsEUtthtJd8SSOXaZ4n71xE1 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_07,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 clxscore=1015 bulkscore=0 mlxlogscore=987 priorityscore=1501 phishscore=0 suspectscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160099 X-Rspamd-Queue-Id: 2AC0FC002C X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: x9ur4t3ynzxtob5y9xqbq11x9y66m3kt X-HE-Tag: 1686913749-610785 X-HE-Meta: U2FsdGVkX1+QVDz0oArt6/7AjDRzduhXixxFMXaZFmtR1TiV1Yiv8VMCLY5n4b9f+maPwS2TzIhfqTPRgnJpE7h3EJiwV50CRPnLN5X87yhmHysAPm4c9WoLhd3oebOajIDPPO7xorN+F9VqnVYKabXYKrBMzvcus51Tlx0OTxNqjF2hhRpSRudQxoMLljai2ZxqRcHwSWE+ccbPEtg96f+cl8VfRVJo2oh429cxke2X3hk0wdExfMokXyXLrj42rKgPyL8gOMIlrUPaBYx4+BC88rLOFAdcwi9SJRsWGOpPIhrM4U7gIpgp6vJSJ1XIlAqism3016nDRPPKpYgSSQGHsOxazec9XmdxsCAuNegRaz2tgEqr/HiLsBwqOoXJ6aRJX0lj/58fVJwy/+M+V/XL84mDMKgCJbz3tBJoo32tXaUUq++jEpR9FJK8KfFMEtgCZKZ3mOWdBqLtm0cNYqQF3N3YXawzoAPo7PqGrfe8w/dCgVt/tbDYtqjwlH6ywfc/pj/USsB/wxUCbTEJRYahcUEfPZrI0g2tbJo/CaIWunuroHvCVC7AkKcnXHWWXjMOXcM9cwBUmwhHso52fMRaTYrlW79dKvelYQZ554jhYzawk3S55UW0et69+tGyItDKSdExS07Vs+GwvNd1zI3dE8dD5+s3BOZlfVebrZJRc5dM5sRFMpYnzsZokil67LlqoQJiXyTIxKRti1KjqWJmr49y6HBslnmQA9HQuF//Lv6bZpCY1xGctQ40tjj17iiiWqZ/cmM0Ol85QTd0GELGZ7/eOvqWqajhVUXvfY6iYn20yjdDhPQmCNzLrWU7uDj65hxydZMY5XFuQMVEjhB5MynlOPoCrP2WO/Gd1qH+IRP6OOouHVh9whUpFWjeUoqXypbD2uGyLuEu6sK3Rt2OyFGXvDr+qkWVcE3TAYpgtFFR9CwF8PnSQuKL0BZQmx2KO+nOlGj3eeUapcP Jdt/3Vzc qjSep1LFEnE2qLLo0SqCFQfw+AD4BuMbQ5Hb9eiFT8XGoEWr+Sg2Vv+QO5vq7c50YgDvRHx6DbTTKRbEXYnQ9glvPF/CpqSR8SZPoF/avyCUYSVkhf3guofIym92amm2MTph9dXWExj9dVyMLh80q6Ya1UgwzEV4jBGSVGWR9wv4nI5wXgnWriN65ISSUpjc+9Zec X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This should not be within CONFIG_PPC_64S_HASHS_MMU. We use mmu_vmemmap_psize on radix while mapping the vmemmap area. Signed-off-by: Aneesh Kumar K.V --- arch/powerpc/mm/book3s64/radix_pgtable.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c index 5f8c6fbe8a69..570add33c02d 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -594,7 +594,6 @@ void __init radix__early_init_mmu(void) { unsigned long lpcr; -#ifdef CONFIG_PPC_64S_HASH_MMU #ifdef CONFIG_PPC_64K_PAGES /* PAGE_SIZE mappings */ mmu_virtual_psize = MMU_PAGE_64K; @@ -611,7 +610,6 @@ void __init radix__early_init_mmu(void) mmu_vmemmap_psize = MMU_PAGE_2M; } else mmu_vmemmap_psize = mmu_virtual_psize; -#endif #endif /* * initialize page table size From patchwork Fri Jun 16 11:08:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282624 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA2A4EB64D8 for ; Fri, 16 Jun 2023 11:09:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3B8E96B007B; Fri, 16 Jun 2023 07:09:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 368D26B007E; Fri, 16 Jun 2023 07:09:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E49E6B0080; Fri, 16 Jun 2023 07:09:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0C05F6B007B for ; Fri, 16 Jun 2023 07:09:20 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id D4EEBA0BEA for ; Fri, 16 Jun 2023 11:09:19 +0000 (UTC) X-FDA: 80908339638.02.4CD8430 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf04.hostedemail.com (Postfix) with ESMTP id 641CD4001E for ; Fri, 16 Jun 2023 11:09:17 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b="Fq/G4oOB"; spf=pass (imf04.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686913757; a=rsa-sha256; cv=none; b=0FOvAL0xR0vcoAQl3+WiHhB0xkuXMYVIAb+NyflUXcDsm208OSQX2xCUDY596x1XdaFr5u 95fTSqYbbiUPwY19DH6qS3iI/YbU4PJ6BQS9L5MQXTzqvpUgaL9A77Khh4IhBmjfmhS14J RAkQaA0PojPCgU6gI9so9SPk49apTw8= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b="Fq/G4oOB"; spf=pass (imf04.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686913757; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=77K1R1/kGwWHCcLG2tEIMVoGfZHm1aKBUIsLmqWw1So=; b=8WUe7l52PpZQfQsZP7DT/l5kIMcoyNhqv2Lqco75g1wDOfirJqbsymoU7dXnM/EccEWBHN VZtupZSY1imu1JbUV8iv8B8eig/V8Wmf+uSCXuD1ssUo0XnIQvsSzUghT8lKReELT4/nBI 4pkeKNGq7sq/Ibq7YC3oWdxxqGrP3iw= Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GB8KDq015816; Fri, 16 Jun 2023 11:09:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=77K1R1/kGwWHCcLG2tEIMVoGfZHm1aKBUIsLmqWw1So=; b=Fq/G4oOBRR/X6XqBnIrEAbYzlbI1Kqg96GPFuSK+QRwR1aZcN4K2zgh+XJyFBLhn2+EA fxPsq0usRYCH1z68mxZwiP519e1HvcLDcqwBbYorXUAHdP7czvY9UK90SlpkbHUODQoL e1lPrfKPSuWdhiOeBSuzP+eNv1t8+hqFvYEAT68A+qND5cPwZjKQMVmA8Ma9lD5Ai8ho nZYhgqLJC49KgzGSR0/PAdq98PkMiOSGZX/faGOHL5tqjp0jQHxVO2WFC2/sQMxobDLW mGdGyMstgOntnhRJ5vRiMsykqm5nBf64N/BJCU3HKP5SsTj7M5T8JPOB5vnNZdE7vOYP PQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9m8a4s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:06 +0000 Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GB8Q0x016011; Fri, 16 Jun 2023 11:09:06 GMT Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9m8a4e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:05 +0000 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAMR7J003274; Fri, 16 Jun 2023 11:09:04 GMT Received: from smtprelay06.dal12v.mail.ibm.com ([9.208.130.100]) by ppma01wdc.us.ibm.com (PPS) with ESMTPS id 3r4gt5qhqx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:04 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay06.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GB93NN983798 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:09:03 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6E1B75805B; Fri, 16 Jun 2023 11:09:03 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3097A5805C; Fri, 16 Jun 2023 11:08:57 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:08:56 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 03/16] powerpc/book3s64/mm: Fix DirectMap stats in /proc/meminfo Date: Fri, 16 Jun 2023 16:38:13 +0530 Message-Id: <20230616110826.344417-4-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: Odp8cUWp3s0d4YuSsCiOdERwl9xp95pY X-Proofpoint-ORIG-GUID: s4EXAUrBlFL4zFELS2xfARnVFE1sJDJ8 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_07,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 mlxscore=0 adultscore=0 lowpriorityscore=0 clxscore=1015 mlxlogscore=999 bulkscore=0 impostorscore=0 priorityscore=1501 suspectscore=0 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160099 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 641CD4001E X-Stat-Signature: inzwpkty6ncmwbursmshpfpm7nwbygsd X-Rspam-User: X-HE-Tag: 1686913757-786462 X-HE-Meta: U2FsdGVkX18Nc36ugRd8ms6Is1rLQ0C+8wZqGuczMw3RKODFNdRxDbHPdwwgUrV5TP0p7kGbvTC/dLetAYtpVv15zTB/atwShJYfooavWo+N5MXvYfKrPs+pyDf09qytzVlZKyfiT4qHz9z1B6somG27Xy6eSBKkVwJhAwiTUnXpfP2pMjejKChpuP5+vNgXYW5+rctaUtUQSOx6YbJ8xjtNrz2T0F6NJH6uZh6v85ps887qeRf7yzlzTt+GX01xRR58ufKLdYecKcHpvs92lGuqogUHTpxO0beI/nX8n/+C08IpJVfZanbVT1dyojL9cwxAI7f+E7d8bek20dwQEd/ujiHgsEy1liH4rl2BDqkoeRMcVRSg6xU9O3izbO/NCU+iHQfJ6t60Y2i4tRCy1b8x1/pSjXaHUoYg1PWlEHS9ftaMUZYBggqXF+i8H1giXaZRvaegoLgvPjkjW59V0cDYfhd/Q5GhMCtnd0TZUDoZntduJjZ7tHD5tjugzNESwc2NGJA5mijHFSw/k45wv2I3nNK/S+W2g2uDmFNsn5I+tLf/WIC98PKh1V9pHY+LA0OoJxKwLHlgZO9lxPSlH67sb7LlDFmVkhYl9RsIwgtPTtFAwYUyppUf50Uq9tjhS0aAK/j1vVNgjwpkBImWrAyZe/6ZqEHKmSugHYbyneo3Feu+fP2OP2HCUZl+MlICBfxJ2cAnF+LPbmDdql+3xBUJP0cuc+9tcUmpZ38epOtf5pkEteekBoK66Q1mjOgy/jjLpCLpmK4HoJdx9YqOcw4p+xGyIVf6mMhmya0czfoyrwYsMH1zxwoDLS7weMBukiEESIMg7Y4AG8+KtLv8BXU2Mj8NK4IqajXyRpCCQS8wkRvzT7ZzwxwCJ6wLt3chR9DM+kGxL8Hp1oP40xSqpclW5uKxyHGxrCiJWz6VPVgdIee3VIacz4xrWpYj1InnwvQ3OUek8n3/Vub/2Ha 2rT5B37E CBi7i2kgcoZtubGnRlQW2iNvfZcsuewBKlKd+tjWfjwWNQjE29/gWcfioWLh9S3tyBkHMBJyWxCvI4zofOIGMdogEQMJNJwwhnYswGm+KtXy5JpyKZYsMmZP57eAFdpG6xrfNkMouxcrZ6s4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On memory unplug reduce DirectMap page count correctly. root@ubuntu-guest:# grep Direct /proc/meminfo DirectMap4k: 0 kB DirectMap64k: 0 kB DirectMap2M: 115343360 kB DirectMap1G: 0 kB Before fix: root@ubuntu-guest:# ndctl disable-namespace all disabled 1 namespace root@ubuntu-guest:# grep Direct /proc/meminfo DirectMap4k: 0 kB DirectMap64k: 0 kB DirectMap2M: 115343360 kB DirectMap1G: 0 kB After fix: root@ubuntu-guest:# ndctl disable-namespace all disabled 1 namespace root@ubuntu-guest:# grep Direct /proc/meminfo DirectMap4k: 0 kB DirectMap64k: 0 kB DirectMap2M: 104857600 kB DirectMap1G: 0 kB Fixes: a2dc009afa9a ("powerpc/mm/book3s/radix: Add mapping statistics") Signed-off-by: Aneesh Kumar K.V --- arch/powerpc/mm/book3s64/radix_pgtable.c | 34 +++++++++++++++--------- 1 file changed, 22 insertions(+), 12 deletions(-) diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c index 570add33c02d..15a099e53cde 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -743,9 +743,9 @@ static void free_pud_table(pud_t *pud_start, p4d_t *p4d) } static void remove_pte_table(pte_t *pte_start, unsigned long addr, - unsigned long end) + unsigned long end, bool direct) { - unsigned long next; + unsigned long next, pages = 0; pte_t *pte; pte = pte_start + pte_index(addr); @@ -767,13 +767,16 @@ static void remove_pte_table(pte_t *pte_start, unsigned long addr, } pte_clear(&init_mm, addr, pte); + pages++; } + if (direct) + update_page_count(mmu_virtual_psize, -pages); } static void __meminit remove_pmd_table(pmd_t *pmd_start, unsigned long addr, - unsigned long end) + unsigned long end, bool direct) { - unsigned long next; + unsigned long next, pages = 0; pte_t *pte_base; pmd_t *pmd; @@ -791,19 +794,22 @@ static void __meminit remove_pmd_table(pmd_t *pmd_start, unsigned long addr, continue; } pte_clear(&init_mm, addr, (pte_t *)pmd); + pages++; continue; } pte_base = (pte_t *)pmd_page_vaddr(*pmd); - remove_pte_table(pte_base, addr, next); + remove_pte_table(pte_base, addr, next, direct); free_pte_table(pte_base, pmd); } + if (direct) + update_page_count(MMU_PAGE_2M, -pages); } static void __meminit remove_pud_table(pud_t *pud_start, unsigned long addr, - unsigned long end) + unsigned long end, bool direct) { - unsigned long next; + unsigned long next, pages = 0; pmd_t *pmd_base; pud_t *pud; @@ -821,16 +827,20 @@ static void __meminit remove_pud_table(pud_t *pud_start, unsigned long addr, continue; } pte_clear(&init_mm, addr, (pte_t *)pud); + pages++; continue; } pmd_base = pud_pgtable(*pud); - remove_pmd_table(pmd_base, addr, next); + remove_pmd_table(pmd_base, addr, next, direct); free_pmd_table(pmd_base, pud); } + if (direct) + update_page_count(MMU_PAGE_1G, -pages); } -static void __meminit remove_pagetable(unsigned long start, unsigned long end) +static void __meminit remove_pagetable(unsigned long start, unsigned long end, + bool direct) { unsigned long addr, next; pud_t *pud_base; @@ -859,7 +869,7 @@ static void __meminit remove_pagetable(unsigned long start, unsigned long end) } pud_base = p4d_pgtable(*p4d); - remove_pud_table(pud_base, addr, next); + remove_pud_table(pud_base, addr, next, direct); free_pud_table(pud_base, p4d); } @@ -882,7 +892,7 @@ int __meminit radix__create_section_mapping(unsigned long start, int __meminit radix__remove_section_mapping(unsigned long start, unsigned long end) { - remove_pagetable(start, end); + remove_pagetable(start, end, true); return 0; } #endif /* CONFIG_MEMORY_HOTPLUG */ @@ -918,7 +928,7 @@ int __meminit radix__vmemmap_create_mapping(unsigned long start, #ifdef CONFIG_MEMORY_HOTPLUG void __meminit radix__vmemmap_remove_mapping(unsigned long start, unsigned long page_size) { - remove_pagetable(start, start + page_size); + remove_pagetable(start, start + page_size, false); } #endif #endif From patchwork Fri Jun 16 11:08:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DD84EB64D8 for ; Fri, 16 Jun 2023 11:09:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D30896B007E; Fri, 16 Jun 2023 07:09:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CDEDB6B0080; Fri, 16 Jun 2023 07:09:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B58F48E0001; Fri, 16 Jun 2023 07:09:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A4D616B007E for ; Fri, 16 Jun 2023 07:09:26 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 77DE21A0C84 for ; Fri, 16 Jun 2023 11:09:26 +0000 (UTC) X-FDA: 80908339932.20.6C20B9A Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf04.hostedemail.com (Postfix) with ESMTP id E677640014 for ; Fri, 16 Jun 2023 11:09:23 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=n9+LU627; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf04.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686913764; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3CRbUlsyj23kEwLD5Yk+bodTH+0qoV6U/+q1tWG5790=; b=JST/eCbHyFX9Ob6IYc9/3GA8+yHItyFursS7hQAr5hn6icCYCLQ1kLW+KyNcvxS0vpQJ8O 9WLpzquZ9UNVHjdO+rpMXZEkvW0j3BWDgNyZEBItgFlobZqmVk1YWPVEW+KdxeeqU9EchE zXHJx+ocfZPOE+Wo897SrauqPB8V44k= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=n9+LU627; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf04.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686913764; a=rsa-sha256; cv=none; b=vsRY+lwWCEWcr5t6edCqOg310sXwaZ/nrzhDS40/akcL0oUZumwqHYL2ksBMOFawLEZybp 0kQtKEA1P1uyNkJSRpTjx0UrnvE+mWLxGtytDODM6UYqFGQHRAVgC33/ql2B02J2TKFt6R EvkhoUT9SuvdI762CVn0Q/0H1G4orDM= Received: from pps.filterd (m0353727.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAl4wM026232; Fri, 16 Jun 2023 11:09:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=3CRbUlsyj23kEwLD5Yk+bodTH+0qoV6U/+q1tWG5790=; b=n9+LU627WXjYwgcTEGejqSRxEv+m9VCK1iQxgR7wBQM6MLQJQyCjG21fxmzAd2R2Ne9w uSrBwWgWeFVlx2vdXsoQXpyO3MqiBObTUhI5NG8x7YYKRe08TfnmlZN4Y9K0w8mojC8a fybBT3xcGcXm34/e/9/AS77WQGDKh5OUjW/s+ds5Cam+7HblKcwwj9plLa7IiM94fe4U rQYrkTb28uVE3nprnurwugJvK06wF0AgGtjsI+41gyDRnx1IuCwaq00DkDXpDky9dp9V b0WyEOVvKHZj2zPAleotAVWjGy5m95M/ieBci13+Bx9LpkaAxZi4m0y5kwBFmH6WDGdD RQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9nggay-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:13 +0000 Received: from m0353727.ppops.net (m0353727.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GAlVuc026890; Fri, 16 Jun 2023 11:09:12 GMT Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9nggad-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:12 +0000 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35GADA8Q010728; Fri, 16 Jun 2023 11:09:11 GMT Received: from smtprelay04.wdc07v.mail.ibm.com ([9.208.129.114]) by ppma04wdc.us.ibm.com (PPS) with ESMTPS id 3r4gt5yhc4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:11 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay04.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GB9ABD35389774 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:09:10 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 31CD658059; Fri, 16 Jun 2023 11:09:10 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 30D7358058; Fri, 16 Jun 2023 11:09:04 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:09:03 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 04/16] powerpc/book3s64/mm: Use PAGE_KERNEL instead of opencoding Date: Fri, 16 Jun 2023 16:38:14 +0530 Message-Id: <20230616110826.344417-5-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: regFUQ7mHQyBSD4iIRzSNwcE4y9xCHVy X-Proofpoint-ORIG-GUID: yEls3uj9Mwd25etv8ICU9HXImIr80LcB X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_07,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 clxscore=1015 bulkscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 suspectscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160099 X-Rspamd-Queue-Id: E677640014 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 659y17zfams49yo1z3rch163bieapk4k X-HE-Tag: 1686913763-594240 X-HE-Meta: U2FsdGVkX19rV/ZZhh9UhD7MD3RoL5ANixlBxmKeylrVc+x83Vc0YHW76DU2x27PPV3zkF0YVLi/mx8l82CJzKwbGnr6FZ1FpnhBIirAHw6xrqIBanVcCh0HlNMxBKfzHLFtz7bVIRF9cqSAtWUk3kKczFEfovpjLg9/6nz7Rg1JJ2iusW+DX6wBjPBuLoSczORQbEVeOCgO5bfUEd+ZpKICLqVzM7VxU83LRYZbkb8z/Bg8IgQ5h1jKrVr/NfNg1lrYJQUffAEIt0xdMmWpzigW551CJhO1QcCqk+mAYBTS2UG9baqdOLkcMau4hfIKc2+PA9XitnJSIccpwwYK+RQfA8Dh9TANkmPx6Khw+0RXp9Z/PqacxyaRP9jALdAe/mS3buOh7eg8kXumMdhXCL6H45DrAMydjk2ACjadGkxlnrXVpQqleDQwvP/Pa8+XRH7keqPfH/0lJWOzbBRqE9SmttByfqcD9jHtQrrTfAcwUUzrMt6UTd9dJthuu6YwLzNDUQcb0Jrs76I0DcppqlDsdOeq5ZP79tn6mUkfgDaqtdo7TglDIF8+ScV52eTOErOJNB4SM6aPqQG6FPyJWz+h3rvzoDAAnl2UBR5yu7DcTCG3TU+x5QZlpmunZ7GVtUdlz0fPQmv8MoHAEsMjO7dx52qgLH/YWh8Uhx6FPpl5k+YRJQbpF/DGMI1PO4ks1mlEyEH/nbqbY6BwS1uK2Qjjnx8/NBoP/v9iDjk6TOAwn6eSammQHtnaItCM/G7+9eObgCYbTYwL6oXIlESJ9yIZmrb0GxcIuQDObgFDaEGbYipkZeYW7bR1RF4fFOHInvt8YXL/Kpx2NgfKVAkAJVm+WEKVs40lYhX9RaFz6+ujDJyeS9j1GxC8YTnlW1f2FTHj7z7HXqUWz43GL8sosRBx2WGFsX/8fBCUYTHTA/u0JfuT1S7xLJNug2tiPRJdNPHrC6R21QHcYCOcpY/ jNMJ9yNr CIEFeUdW0EKvQWcIPb4wyXajUQEo/rTFeHN+I5KmWeMdnVDGQcHenPVo5B35uLSLdHIIpbE39qmlSFhsJccTdCEr/IA/CCn0AbIP7C0TYFBA4r6O3FTO7pHqfzGbU+0ppCkwmBTHgJNEzFsiyK2Jrr2C8QwS4MxY5DBOvx8keM2/fdX72PLl+RxkGJzxl9ndRyRDw X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: No functional change in this patch. Signed-off-by: Aneesh Kumar K.V --- arch/powerpc/mm/book3s64/radix_pgtable.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c index 15a099e53cde..76f6a1f3b9d8 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -910,7 +910,6 @@ int __meminit radix__vmemmap_create_mapping(unsigned long start, unsigned long phys) { /* Create a PTE encoding */ - unsigned long flags = _PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_KERNEL_RW; int nid = early_pfn_to_nid(phys >> PAGE_SHIFT); int ret; @@ -919,7 +918,7 @@ int __meminit radix__vmemmap_create_mapping(unsigned long start, return -1; } - ret = __map_kernel_page_nid(start, phys, __pgprot(flags), page_size, nid); + ret = __map_kernel_page_nid(start, phys, PAGE_KERNEL, page_size, nid); BUG_ON(ret); return 0; From patchwork Fri Jun 16 11:08:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282626 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 860B5EB64D8 for ; Fri, 16 Jun 2023 11:09:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 20B096B0080; Fri, 16 Jun 2023 07:09:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1BC656B0081; Fri, 16 Jun 2023 07:09:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 05CB46B0082; Fri, 16 Jun 2023 07:09:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E8CBC6B0080 for ; Fri, 16 Jun 2023 07:09:32 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A186A1A0C7D for ; Fri, 16 Jun 2023 11:09:32 +0000 (UTC) X-FDA: 80908340184.13.C136D23 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf01.hostedemail.com (Postfix) with ESMTP id 4C13340022 for ; Fri, 16 Jun 2023 11:09:30 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=ZfeIyk5y; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf01.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686913770; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=H+uaA5+1mvtJFerKJUXsEuKB7to6xvSS0R1azlcL2R8=; b=BAi1NXZ9PhdthpMAVMRCSs8BAzShlyysiOg+mt0hyPqFaGBU5Llm23JiLuYAx6ykMyJIVd phumGTvuZVsB4RwodvAVUsGOmaVhQqbW06YLs90rdHe5D3+d5KLXoAiZM1S5I75Ggk0GHW smTWHI3gNzuGIr+Xj2BKTtCR3sfgWc0= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=ZfeIyk5y; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf01.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686913770; a=rsa-sha256; cv=none; b=uYTKA5ca93+4a2abPiJSwTg9A/r5btFCU2JLYzJ8mriIYfCHOO0IfoHlpGu9QJKlEthqbY KyAh97WogCUMpPgY6e78tTBGUuDeECeTFZ3fMtlk2b+cs22Eayfg4MCdETWk4bud8KJzCG IhV0fcV509tnQolpFkR6gEmr27awqH0= Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GB8VJh016651; Fri, 16 Jun 2023 11:09:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=H+uaA5+1mvtJFerKJUXsEuKB7to6xvSS0R1azlcL2R8=; b=ZfeIyk5y5gGXBF6IU6e3xWxUf7yrgbw5MKBu3IPu++VbAu1R9X6Erdwq8MU/GpZb+TnT TwleDZHF+u6Otu5p1y+LXrZsDn9Jta5AKsmK+7SX+azQskf3R/aVO6Fs2Rg968Fnxq9g oVBO++wBI8rUvWcaFB1Wv0vtxaYN6H+SGvfoKF3EAtQ3sq3LRqBSxS4uhnq6loZyvmB3 hmZppvSO8QTvwTP5vmMJfNmUHa06yLLk+kz12ANTOrcajkfmrMjmC47E0JzI7v1/oA6/ +cc2NjH/KkZJaKzhEIrjNb14S8b5Tzg7zIhoM4mGSBJP358MnII3DHKoFLGsIk24UpOG Dg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9m8a8e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:20 +0000 Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GB987a018597; Fri, 16 Jun 2023 11:09:19 GMT Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9m8a80-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:19 +0000 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35GADBcH002749; Fri, 16 Jun 2023 11:09:18 GMT Received: from smtprelay06.dal12v.mail.ibm.com ([9.208.130.100]) by ppma01wdc.us.ibm.com (PPS) with ESMTPS id 3r4gt5qhru-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:18 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay06.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GB9HjK50463216 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:09:17 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DC48B5805D; Fri, 16 Jun 2023 11:09:16 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0588458059; Fri, 16 Jun 2023 11:09:11 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:09:10 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 05/16] powerpc/mm/dax: Fix the condition when checking if altmap vmemap can cross-boundary Date: Fri, 16 Jun 2023 16:38:15 +0530 Message-Id: <20230616110826.344417-6-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: hrPuD47xEHbXC1tgOQ9G6Kcrw85WGkYC X-Proofpoint-ORIG-GUID: IioFF6ydZQZZJRliOezZINBo_cugxYIx X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_07,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 mlxscore=0 adultscore=0 lowpriorityscore=0 clxscore=1015 mlxlogscore=999 bulkscore=0 impostorscore=0 priorityscore=1501 suspectscore=0 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160099 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 4C13340022 X-Stat-Signature: 63k4dpfyo7drxesxyq4ph3opna1cgqi3 X-Rspam-User: X-HE-Tag: 1686913770-935124 X-HE-Meta: U2FsdGVkX1/lO7QU7Ikinoi1okTSzXy4pWb3+os7MUdm8sXzuF879KHlM+6KTjNoEK+ofNWDB0ajIKAjI77oKBjmVkbC8x0gtB4RLVBRP6y/g9/M02urSm9t9t0+7Oqve1zcdDhUDBF/iUgFG+W/wXud7rGfRB6I4cv97LHvRRpcJoKN0vTijfPEN9FjXbxlhMmduxbFSw2GEPpYEbFKK6fV/PfAJ105gVn/c9oN4xPlEijkdLpI7RqGTXsB2O7y/tWRJ3nLpCiiojRtbRQSIgij5hXKRwXRchEocpmS17j3xg9wqtVLD7EqxVbRtqxoFdt2NCfkeuY1WlHuZeXPyWyqckRSLdd+00YXYbhSOCagvyZ5RXsZ/eeZ7CCzaCCzTGGjFct9Ysae1pAdsYpw/AofNy+SqevL/QIR+TWmk0e3R0lIlhw4UqvqlIS/hbvsDzOL/ch73RLkjdT0hr8nftf1eNgFL55iZc5dgkWRnRb+8LGsp7yGh43icsKpHRA15o6t2fVEY10mRfr8DvlIXvEq+R0APhpHsXprPr4ipUoEaEWJsehBNchc/jsz2jUsECPQONqYFhJdnYmm2VhpZN3CSLi12GtXISVz8JcNW1uGAUmlt0FggDoeu1G/racHYiy29VYDzWEfNQHRLFA9G27cIASFpGrz9Iou3r0EmFeqY5EluhW7zMuZpntJroocyuUa2Zf57MR0orgidd7cIetUs0xTvtGqblqcxv3bYUdloWh0TuX74XHNH0lbeY76W2IRqc2gA3+F01A7GZL4CShyMJqlDOgrzb/4P91PnSIem6rzZopNmgOFQAsKmjWqrJ/XC0ZT3nZxsxaCeJCbwgyxMJZywvWMycq9nxXo2b9xZEFLsMVwj3iGCTUXOxbOXDl1LrCq2MbUvbgOjE2RQw/TmDGoh0N+W7uXRKMFcId89f4c9BRETkpyS4QVelNaxQzVg+7ZfoPj+cSo+5c iM4/4hTn K/XZJE0/CDZ2bQ7qN5QjUgplYxQZvt+j98coix/oycpY9v2YXj7qmAnb6BgMY9ovZZYBwzE0FKr6oDpukJdmCYX9tHkXQOmRef1w8m0o17Rj29YkLh2cncho7GTz2fotuoJigFeqdNFpfz2g= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Without this fix, the last subsection vmemmap can end up in memory even if the namespace is created with -M mem and has sufficient space in the altmap area. Fixes: cf387d9644d8 ("libnvdimm/altmap: Track namespace boundaries in altmap") Signed-off-by: Aneesh Kumar K.V --- arch/powerpc/mm/init_64.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c index 05b0d584e50b..fe1b83020e0d 100644 --- a/arch/powerpc/mm/init_64.c +++ b/arch/powerpc/mm/init_64.c @@ -189,7 +189,7 @@ static bool altmap_cross_boundary(struct vmem_altmap *altmap, unsigned long star unsigned long nr_pfn = page_size / sizeof(struct page); unsigned long start_pfn = page_to_pfn((struct page *)start); - if ((start_pfn + nr_pfn) > altmap->end_pfn) + if ((start_pfn + nr_pfn - 1) > altmap->end_pfn) return true; if (start_pfn < altmap->base_pfn) From patchwork Fri Jun 16 11:08:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282627 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5D01EB64D8 for ; Fri, 16 Jun 2023 11:09:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6C8076B0081; Fri, 16 Jun 2023 07:09:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 676878E0001; Fri, 16 Jun 2023 07:09:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5173B6B0083; Fri, 16 Jun 2023 07:09:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 405BE6B0081 for ; Fri, 16 Jun 2023 07:09:39 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 11EE31C8F80 for ; Fri, 16 Jun 2023 11:09:39 +0000 (UTC) X-FDA: 80908340478.13.88D3647 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf08.hostedemail.com (Postfix) with ESMTP id 771D3160021 for ; Fri, 16 Jun 2023 11:09:36 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=j9Uzlewr; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf08.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686913776; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=N21LFn0bJw0zSofrkQfU13OVWvgTQgm6wU9sahNmz2k=; b=Kf6mVUpdnwDpXUkecr0zZql9/khlFnxi2ekZU+OUJ3j0Vm8p84wV0MKgUCTVoHNUd45gAB pfbBZZ1hUwyyQZm1Dzmfobfk96gbgPIdbYKX8MGLbXBzN8j1FD8yZy0D/Dnad3qkewvEVw 0hHu5jGfFsaF0laXZBzM+aBZD3LMBGk= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=j9Uzlewr; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf08.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686913776; a=rsa-sha256; cv=none; b=Izi/oUmGP4GsZuilFL/ZkgqJU+L2qYcXYfooGEPhgvb0bnm74qLDvWPTBIvyl66KyqSj8O XXJgVU+l8zhvfUPrQByIN4mfVibjOyiwqGA51g478D8qOCiyiKHiOzHBF0hlhuh8gHk7Sm 0ao3i2dskEfACv/AS5MDvUMwD1EkAiE= Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GB8BKn015694; Fri, 16 Jun 2023 11:09:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=N21LFn0bJw0zSofrkQfU13OVWvgTQgm6wU9sahNmz2k=; b=j9UzlewrJRErIYFUfcJMJFGpiFO85DRypxsnMU6gEolW3H7Mih7poMLxHszmGNY7WDkH Ldzc8+sSifk4YTMlhOmjvKgFaNzRd7uZ6nS1RSwCTo8/fpJlWbDjHXWM9NikEaOWMr2w fFLYGEVnr+BeRMIWQmJ5OGqHeO07nXPWypsOqwBt5orHgfa8NqqHFY4bTipmPhwD3nHO SEuRlvPkAArAkk1eXMcP8AQG4hJbOXduBoqhTchgsRapPTvNKHmHOBl3ZsDG7seiKt4v ez+k33WRSDcp6oqiJHJLQIrEKN23EaNqaSz4YzKuxLeo4ovFZCr3iRzfkGL81IXkYJaT Ug== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9m8aax-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:26 +0000 Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GB9P5H019087; Fri, 16 Jun 2023 11:09:25 GMT Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9m8aa8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:25 +0000 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35G9VjPn027195; Fri, 16 Jun 2023 11:09:24 GMT Received: from smtprelay03.dal12v.mail.ibm.com ([9.208.130.98]) by ppma02dal.us.ibm.com (PPS) with ESMTPS id 3r4gt6gxxw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:24 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay03.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GB9MuB28967192 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:09:23 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C6B565805D; Fri, 16 Jun 2023 11:09:22 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8C6445805B; Fri, 16 Jun 2023 11:09:17 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:09:17 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 06/16] mm/hugepage pud: Allow arch-specific helper function to check huge page pud support Date: Fri, 16 Jun 2023 16:38:16 +0530 Message-Id: <20230616110826.344417-7-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: gcZHzsUPhJ6vpRBn9biPPQH_ag3gnr6_ X-Proofpoint-ORIG-GUID: J_F5AjAuhX1jIBVYFZWa5-9r-All94eo X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_07,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 mlxscore=0 adultscore=0 lowpriorityscore=0 clxscore=1015 mlxlogscore=999 bulkscore=0 impostorscore=0 priorityscore=1501 suspectscore=0 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160099 X-Rspamd-Queue-Id: 771D3160021 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 96g16i8ac9zpgkod766sbcw7sm6xcrmo X-HE-Tag: 1686913776-943905 X-HE-Meta: U2FsdGVkX19pXr7ZpbDP43ipoqsIdZ3LoHilZ4eaDGV5GO3ClmOJucRkRKgPLaXjLeI9qK4jJ+F053Fl7PH4d20/Uh0IetIGpiSj2E3G4hEy8WivovcfWTElEV6vUdCSmDr99mVscTf7dU0pdwK05FuSm7B1U0Ay8V+M2uXpHCT1lmjhiHpOA8eZeQdUUtKTOI6NeC4cKnMQcehyaiXx0brooD790u5eCF9RjSjxlYHg1+LjdspJ3pFLyVAhrSfNiFOs3zD93jWPiIq0QJ4YTkKWDbRJJoAaoGrENTlwnfE4oONELBfclrpvSdqAwgfOMf00U1cwAkyUubvRuXVgKXMl71A4qb3q/0IAL5FW6zOVTzTFwdZF3p4vqGYi7bwHPQNdzy89E5zs27LZKp2l6ymiHPDCilFn+jNAKg45YS2KrvZPbj6B7KrDvpQxYJtvS7uUHTca04uC2h+dgoVlJU+Q/WAM/aTiE23dwPGYkEBlrmDKJcATRxo4pIIG2fetH0hZZgRG93VDnsxjMyM3FaxB5UI5/pCNl3P64jajLOasS5kqNeh5K4+CG7gA8aJcDN52unn0E4WK/QhupoVBKV+c4/xpwrKn4AANztG/w2FP6AS2Jr+8lRGx9R6xuBor34UVIvtenszn/v0SkpvW1YwBKZwRJyfU6TFiXeR8hSPT0Xiw1pGlK08FhmjchQkXp4OgPtYsOn231V/p9VgfDL1RlT6/vAIPm9nvytL1zMuTnr7uhsuQK9jZcPllAFSiyY1nupBSdCT3uarD1JzL8ZhoZY/G6vnIxoipjXqF/CwBCJ5CyXetL7NMhI/+Xzyo3dNf5V7wvu3yHAhVUezPitHAVEhcS+BNw4i2hiuUP0h/mV3TIJpIz+lWArG5mj1JI53tXPme4qw2B7/dCPUr9JEQcuYK7AIO8I1pR5I2V05OMqkmk1wq+j1v/Eo6h23xw0vW+MsnJ+xxTiw5wjI RQSiZ6i9 Gr63Li5AexgAbZK3HYih6FHaOgVm5iW3CXGvDscpZal6CYIeklyd4SMZouoQ3T9Xv+vBcFFf5cWQyZE1IZ0yowibFCfww4ArVk+nr64O8bm1BQXvYCcMODCteJBAU2oN28tgP4udmjBaGYmY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Architectures like powerpc would like to enable transparent huge page pud support only with radix translation. To support that add has_transparent_pud_hugepage() helper that architectures can override. Signed-off-by: Aneesh Kumar K.V --- drivers/nvdimm/pfn_devs.c | 2 +- include/linux/pgtable.h | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/nvdimm/pfn_devs.c b/drivers/nvdimm/pfn_devs.c index af7d9301520c..18ad315581ca 100644 --- a/drivers/nvdimm/pfn_devs.c +++ b/drivers/nvdimm/pfn_devs.c @@ -100,7 +100,7 @@ static unsigned long *nd_pfn_supported_alignments(unsigned long *alignments) if (has_transparent_hugepage()) { alignments[1] = HPAGE_PMD_SIZE; - if (IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD)) + if (has_transparent_pud_hugepage()) alignments[2] = HPAGE_PUD_SIZE; } diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index c5a51481bbb9..b3f4dd0240f5 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -1597,6 +1597,9 @@ typedef unsigned int pgtbl_mod_mask; #define has_transparent_hugepage() IS_BUILTIN(CONFIG_TRANSPARENT_HUGEPAGE) #endif +#ifndef has_transparent_pud_hugepage +#define has_transparent_pud_hugepage() IS_BUILTIN(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) +#endif /* * On some architectures it depends on the mm if the p4d/pud or pmd * layer of the page table hierarchy is folded or not. From patchwork Fri Jun 16 11:08:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282628 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAFA7EB64D7 for ; Fri, 16 Jun 2023 11:09:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 517D06B0082; Fri, 16 Jun 2023 07:09:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4C88F6B0083; Fri, 16 Jun 2023 07:09:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 31B958E0001; Fri, 16 Jun 2023 07:09:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 237166B0082 for ; Fri, 16 Jun 2023 07:09:45 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id E0093160BC8 for ; Fri, 16 Jun 2023 11:09:44 +0000 (UTC) X-FDA: 80908340688.12.C839664 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf13.hostedemail.com (Postfix) with ESMTP id 5E66D2001C for ; Fri, 16 Jun 2023 11:09:42 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=mzyfGpFm; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf13.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686913782; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IgFHfZ1KySuiFsDTbSIBfgisCKdqais3tV9iLIh71Vs=; b=heDJiHr8W1B1JWjYC2z0Kn9CGtjPe3jFmU0b4cOPQfwGs4taK4G4j8t99f4xtcQXe0omhv TyblwqhpR0VxqhdOrnr/rHnZ7aqdQcvBmhjVcnbc+ipHoCyUki8EU7cr4EzGzCB3cgL7HK lY3S8Z44TtcnXc6EOGbwEYenR7Pzx5U= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=mzyfGpFm; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf13.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686913782; a=rsa-sha256; cv=none; b=K10y39hEmX+dph+PAw1Wjul9F5Q+DpVFoJvTsJ8rsLe9lFwLgFKSrj47/pkLPEl5WKn3Ny A5HsmDol75Z1K0iqVjrgaKRpzgvabwODVm8bSYNwyHGx1pyzk5RVqKXJILCyime3tPRjW1 8pUkR6t5++o9Tyk3htmRC0Uu6X6yZ44= Received: from pps.filterd (m0353727.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAkmQD025344; Fri, 16 Jun 2023 11:09:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=IgFHfZ1KySuiFsDTbSIBfgisCKdqais3tV9iLIh71Vs=; b=mzyfGpFmT1Pch2wcBqn6wqSFt4rvnmNvEziRTRlvBlf2gSK1UwSCeZIOjLH56a88vOnk cofmx54awuupmgO0NKJsL3Lzi/XUmd19wMBjQaMvo/lYsMhgKhLaYnpnSLxKrSqzit7e B7Lfdj/r2SDOTyd+46luR5tz+hJLiiOoRUXi/zvxyXjmSSZBI2VitgnQiW3yan2Dry6b PKKyKVLUNtuo2LNzePPcP9FVoNcUwEMPfQP5bD1MTZlH1o7t1vD3qKcDJ92bON7HUPIv MoPanUKPAY/Og9ypeL6Ir022AkcvlAYPDdoG0jYd1usn9TPccSUlRoZUADM/4fZHPtEP lw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9nggks-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:31 +0000 Received: from m0353727.ppops.net (m0353727.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GAmUQC028781; Fri, 16 Jun 2023 11:09:31 GMT Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9nggka-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:31 +0000 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35G8oUp8030230; Fri, 16 Jun 2023 11:09:30 GMT Received: from smtprelay05.dal12v.mail.ibm.com ([9.208.130.101]) by ppma03dal.us.ibm.com (PPS) with ESMTPS id 3r4gt529c8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:30 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay05.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GB9SY764028950 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:09:29 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9BF7358058; Fri, 16 Jun 2023 11:09:28 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 69F6258059; Fri, 16 Jun 2023 11:09:23 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:09:23 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 07/16] mm: Change pudp_huge_get_and_clear_full take vm_area_struct as arg Date: Fri, 16 Jun 2023 16:38:17 +0530 Message-Id: <20230616110826.344417-8-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: GMvt6BQvPqImc8Zg_D1LAzg79RDphZh5 X-Proofpoint-ORIG-GUID: dG-IK40REO7GSnDV6A5S9Cqi2Ldrs4gE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_07,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 clxscore=1015 bulkscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 suspectscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160099 X-Rspam-User: X-Stat-Signature: ox7jt5d7q1q6q977ce65miennjkxbc5s X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 5E66D2001C X-HE-Tag: 1686913782-554263 X-HE-Meta: U2FsdGVkX188jTvi19KHRK5kqxfkXtob3ezotdAibW1Rt/4YHGdcIFqeaHba/0BQ36aulyXjRgz/Crk1fVnftSDuS7tkiyNnsnBArah3JVSKyk+wN9Fvab9knJ6mDLNFD5QccpcdAPnrg5l9mbmbN808UgPxfdG+qIJMQhPilYhPGZ/yDo5jfFDS74bLjbECKf+gLMBlqw7xS7lzw4RfklLrrmkWYnWd2xe0R4XrK4t0Il+hAiZEEslZYjszT/jF/IpcVbKdSM0lppf20YbFsjr7rOQMdJDkcNvrOg3JjI1QFezpIfyh3eLcjcX9X4EM9yr+b0kaaLq9x6US178FURaEzlP6sGh+OTx9W6Kwf3lfTtBuICQuNGtU+Z9s/ZzJVrds7uMlh2y9IFgEn3WEXZhsXXCN+RZ9AWQYwKMKG430+lVUwTwV7sDpotp7VFqka5HFY/KPBGqRb0IN8eY/QJXsIf+/zivPQVFCP0sLyocN73Jp0j17Oce7g7FVyt+I4zDXlJ+WawzWIf3DTjkSo5iAgmL6vXGMRWxO2Rgj51HnzamyfyFqE1inQW4Wg7NS9QWQsJJ/LJt595086ILj50kGFk+n7ll0con434NcMNjZscFDfBx+uS8rLNBHURu/+xJNo/XGK/0mdRaoF2AxFPUcnwqKtF38BoKt3KSvwLrzqmilnInJ4q+0Es4rKuypmOSQmpDRwTz9lHTLQAkW2M+j2PkZBOs3MuLx88MVwsBEUXMVMC4Qx1s877QxoR2HEXrsEsTnoGWiutIuziUQOF+5w9RlDEW9adS4lWddtYw6CuuJQZs/TtJnMY0PsITp1nR86BzVZaEtuo5/qijMaBaF+Ix/b38SNhbNvRnh3Rf6XP62K2jB6MI0Ei8yueWjdm92xA4F2IwRgOHm2lkNpXfnKDEQhn308waTKdDqhqa6Ah5BstVpG3vXh+ulNItkrfegIYOPGEqzewLFWgq Y9mivJgK h8E66SkzrswZmCwCnS+Cc7eq7VAF7uyBJkBF2u8pfwX7Qv/iSAaNOxX9LjzWui7Q89ecp0jG4zYP9gY9yyxS0gOnmH9zmlBFuQgdZqrnqKWPeUSfaAa77ZJzwGtjerqLQEtFRu9RrJxmFbaSo2lRZDnCebP8zWtSj3o4PPERQLoM/PaMkx2wjirvQYnBX3RiytL3A X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We will use this in a later patch to do tlb flush when clearing pud entries on powerpc. This is similar to commit 93a98695f2f9 ("mm: change pmdp_huge_get_and_clear_full take vm_area_struct as arg") Signed-off-by: Aneesh Kumar K.V --- include/linux/pgtable.h | 4 ++-- mm/debug_vm_pgtable.c | 2 +- mm/huge_memory.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index b3f4dd0240f5..2fe19720075e 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -442,11 +442,11 @@ static inline pmd_t pmdp_huge_get_and_clear_full(struct vm_area_struct *vma, #endif #ifndef __HAVE_ARCH_PUDP_HUGE_GET_AND_CLEAR_FULL -static inline pud_t pudp_huge_get_and_clear_full(struct mm_struct *mm, +static inline pud_t pudp_huge_get_and_clear_full(struct vm_area_struct *vma, unsigned long address, pud_t *pudp, int full) { - return pudp_huge_get_and_clear(mm, address, pudp); + return pudp_huge_get_and_clear(vma->vm_mm, address, pudp); } #endif #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index c54177aabebd..c2bf25d5e5cd 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -382,7 +382,7 @@ static void __init pud_advanced_tests(struct pgtable_debug_args *args) WARN_ON(!(pud_write(pud) && pud_dirty(pud))); #ifndef __PAGETABLE_PMD_FOLDED - pudp_huge_get_and_clear_full(args->mm, vaddr, args->pudp, 1); + pudp_huge_get_and_clear_full(args->vma, vaddr, args->pudp, 1); pud = READ_ONCE(*args->pudp); WARN_ON(!pud_none(pud)); #endif /* __PAGETABLE_PMD_FOLDED */ diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 624671aaa60d..8774b4751a84 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1980,7 +1980,7 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma, if (!ptl) return 0; - pudp_huge_get_and_clear_full(tlb->mm, addr, pud, tlb->fullmm); + pudp_huge_get_and_clear_full(vma, addr, pud, tlb->fullmm); tlb_remove_pud_tlb_entry(tlb, pud, addr); if (vma_is_special_huge(vma)) { spin_unlock(ptl); From patchwork Fri Jun 16 11:08:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7539FEB64D7 for ; Fri, 16 Jun 2023 11:09:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 036126B0074; Fri, 16 Jun 2023 07:09:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F01D66B0078; Fri, 16 Jun 2023 07:09:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA21E8E0001; Fri, 16 Jun 2023 07:09:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C74A36B0074 for ; Fri, 16 Jun 2023 07:09:52 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id A8E8C120C8B for ; Fri, 16 Jun 2023 11:09:52 +0000 (UTC) X-FDA: 80908341024.25.2107F60 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf19.hostedemail.com (Postfix) with ESMTP id 2E4D41A000C for ; Fri, 16 Jun 2023 11:09:49 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=Bf6hySyB; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf19.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686913790; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2nIHSSUSxuasnMPG57XtfkGpKEslBIgIjQykIV+BjZI=; b=farFL0WpVu2cc/xIzfA7v8bwp8SCQEeor1QDH/mYQQKuWSqIfbOsrBwpLYEcTM8/04B5lr zDTeGluSMEJjoHXbOyhuF+gawldNAgV9pUe3GrYCljPnVQdYQfRQi8ttwUoumsloH8EofT 0xtkOi5ckf2KWN60enEYIAo86z+JIrk= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=Bf6hySyB; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf19.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686913790; a=rsa-sha256; cv=none; b=qH9z+kFiqV9pa3GCmceyBhDRIbW5zzp91f9RdO7w1eMm4QSheK6r2wQWtVGNEhEWxFxPTH WufGqRUX9SEZ5pmv0oqEKN4IoESULbWtZmkunwU1Ku/n8MP3F8FjycTOBIjKMMAShyy6oS K/uiXVFtB4LU2avMaGE2w3Z9bajEMsI= Received: from pps.filterd (m0353728.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAowna018749; Fri, 16 Jun 2023 11:09:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=2nIHSSUSxuasnMPG57XtfkGpKEslBIgIjQykIV+BjZI=; b=Bf6hySyB9OdGKdq/zlFMV6Xr9cWAzlH7wJSONsWfT9w6IlizdKuFxEG2hXb9wFf3zYS+ I1kAHZvcNwMtve6upiULELVaAOB84u3O6n3l2/RnTX/TB5X4nl8eVyUUAdxUDgbsV/A3 q4ZVUzAmM4aKuXosZkVXIm+9oZxHa5D5YHTKnhSy+PjD67Rzhqt6ng90hPWKoSU0E3bs TIsfk261FQOOVFY8/PKZJRxJaQ2wGs7VLvoyXTfeUh9CF8Szmr3z2ntwcCkPXW8wlUqL SsFXhC5Z//cbIfGXvqlCSWAAMDy7+6OZeh/bPXYq0kDy+EC6ZsI1OTUZoiaygT418QB2 lw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8pbm0f7d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:39 +0000 Received: from m0353728.ppops.net (m0353728.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GApBnB019196; Fri, 16 Jun 2023 11:09:38 GMT Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8pbm0f6h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:38 +0000 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAlmZr010717; Fri, 16 Jun 2023 11:09:36 GMT Received: from smtprelay03.dal12v.mail.ibm.com ([9.208.130.98]) by ppma04wdc.us.ibm.com (PPS) with ESMTPS id 3r4gt5yhe5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:36 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay03.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GB9ZFJ28967198 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:09:35 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 479845805C; Fri, 16 Jun 2023 11:09:35 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3F82C58058; Fri, 16 Jun 2023 11:09:29 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:09:28 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 08/16] mm/vmemmap: Improve vmemmap_can_optimize and allow architectures to override Date: Fri, 16 Jun 2023 16:38:18 +0530 Message-Id: <20230616110826.344417-9-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: kYPxs0AVpUyxr9Dw0HjJ-6Dc32dedHBR X-Proofpoint-ORIG-GUID: RZSIYIkuDjT9w6LwUdE6d3PlYbkqnWEB X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_06,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 malwarescore=0 bulkscore=0 spamscore=0 mlxlogscore=983 phishscore=0 adultscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 priorityscore=1501 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160099 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 2E4D41A000C X-Stat-Signature: d8i4uced67cfa89y8ncndcc5tc6ikbco X-HE-Tag: 1686913789-478336 X-HE-Meta: U2FsdGVkX19+ZOLydahE9fhjcxCpxdLWvNEK5dq6taU7nsznlFK99xNunMuTQjdj2T+WaNVsdMtUzm64BYGe3J/3HTVzp12SN35KuJ+YYq9B1T4xo3CQNmVZ/LZSbixtnaVKdzlwdsDFHkBE1NOA0KpanCCAoUih/ZmIX24bDh48P8ZlckL6cfj01nCFxPmQlhM6a5G7wZQeFgmdhFP5IX1fyzuPgMai1WXjUUZQb9y7tPLGG1lk3URY1L+W5LGERly95ZsPx68tVu125FFzy7gSNZVzQczcdoC6wC6WroORL3pC1DODbd6JJWbcByh+Fj41nO3kQYvcEYpHOoJCofVdiodGw3HxGhDfMbMdTKAe4Egya5GGZZGrPYOktG2/1T2+JpPEyETYL8NaWw2eZ6gFvvO43gQ+hcaMRYgg8gVWFboPhZ2ostuI4qapgwC3myvOywB2COscr4iNCrj/BoZsbvZjnh/EHa8V6VolXBu/C6ywh6CMW4MTW+nthARZDexdyLPGJdy6azU7m+IjOLgUI8qUFS4H/blKi/VXrVP1YrWMgvYVDlqnWbaPFh6V1GHnwbI2FlUWgDYOgCdZMINkFgTLXwLOHyhwYrZ7z0HcfWMzBi2mpVv9FlmOuuN+E5HFWKh5lf2axmbHvX/p7H2Z62t6ts3bu51s+JL3xWty0EzJilF0wrAvRqAthIC3HfGqsOBQH8tpRi2Ipe0wAsNjuB0brCfC2f2Zbsp6Age8X5y4aDZhXpQOqvc6I81DEHnli82yTuxKy59ULz1mBrUyZdD+TdnpKK0+GwoSlIjA3Papyow/xbFXq+x9n7yP7KAZ+vV9Gb1DvahUMJ57MYzSmMrzP7GcSvlADjAGHQsVvm+vQhcu1oL5ScV042GfjgZhiuf3eBnwMyerbnHm1x+fi7lpXKGQEXfV5IgzwxKNOof7FCEoSup76Oepn1Z9MnuctzCFYMraXs2ozJu JiYPb2lq fXVSUelOq2EqOXHUEPyBhSMkbWqMYQUVAezKrTSM9ONzxgkcMpZYTVaA3BKR2nnr74YXzoW0yoabq3v6yenb+A/yqXZSYkoBBVLBMsTxAp66fTjIaJxfzcZ5FMKyMAW38tQCdaxwoYEF1expWdGpKubo5iayM2HJCySeBiYpNQcSn/ykmEcVZiYV2bkOC9OvT+A3I X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: dax vmemmap optimization requires a minimum of 2 PAGE_SIZE area within vmemmap such that tail page mapping can point to the second PAGE_SIZE area. Enforce that in vmemmap_can_optimize() function. Architectures like powerpc also want to enable vmemmap optimization conditionally (only with radix MMU translation). Hence allow architecture override. Signed-off-by: Aneesh Kumar K.V --- include/linux/mm.h | 30 ++++++++++++++++++++++++++---- mm/mm_init.c | 2 +- 2 files changed, 27 insertions(+), 5 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 27ce77080c79..9a45e61cd83f 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -31,6 +31,8 @@ #include #include +#include + struct mempolicy; struct anon_vma; struct anon_vma_chain; @@ -3550,13 +3552,33 @@ void vmemmap_free(unsigned long start, unsigned long end, struct vmem_altmap *altmap); #endif +#define VMEMMAP_RESERVE_NR 2 #ifdef CONFIG_ARCH_WANT_OPTIMIZE_VMEMMAP -static inline bool vmemmap_can_optimize(struct vmem_altmap *altmap, - struct dev_pagemap *pgmap) +static inline bool __vmemmap_can_optimize(struct vmem_altmap *altmap, + struct dev_pagemap *pgmap) { - return is_power_of_2(sizeof(struct page)) && - pgmap && (pgmap_vmemmap_nr(pgmap) > 1) && !altmap; + if (pgmap) { + unsigned long nr_pages; + unsigned long nr_vmemmap_pages; + + nr_pages = pgmap_vmemmap_nr(pgmap); + nr_vmemmap_pages = ((nr_pages * sizeof(struct page)) >> PAGE_SHIFT); + /* + * For vmemmap optimization with DAX we need minimum 2 vmemmap + * pages. See layout diagram in Documentation/mm/vmemmap_dedup.rst + */ + return is_power_of_2(sizeof(struct page)) && + (nr_vmemmap_pages > VMEMMAP_RESERVE_NR) && !altmap; + } + return false; } +/* + * If we don't have an architecture override, use the generic rule + */ +#ifndef vmemmap_can_optimize +#define vmemmap_can_optimize __vmemmap_can_optimize +#endif + #else static inline bool vmemmap_can_optimize(struct vmem_altmap *altmap, struct dev_pagemap *pgmap) diff --git a/mm/mm_init.c b/mm/mm_init.c index 7f7f9c677854..d1676afc94f1 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -1020,7 +1020,7 @@ static inline unsigned long compound_nr_pages(struct vmem_altmap *altmap, if (!vmemmap_can_optimize(altmap, pgmap)) return pgmap_vmemmap_nr(pgmap); - return 2 * (PAGE_SIZE / sizeof(struct page)); + return VMEMMAP_RESERVE_NR * (PAGE_SIZE / sizeof(struct page)); } static void __ref memmap_init_compound(struct page *head, From patchwork Fri Jun 16 11:08:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282658 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CD47EB64D7 for ; Fri, 16 Jun 2023 11:44:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C60F8E0001; Fri, 16 Jun 2023 07:44:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 976586B0075; Fri, 16 Jun 2023 07:44:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83DDB8E0001; Fri, 16 Jun 2023 07:44:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 71A8F6B0074 for ; Fri, 16 Jun 2023 07:44:58 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 3264D1A0CB5 for ; Fri, 16 Jun 2023 11:44:58 +0000 (UTC) X-FDA: 80908429476.26.8D31513 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf02.hostedemail.com (Postfix) with ESMTP id ACF0D8001C for ; Fri, 16 Jun 2023 11:44:55 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=Rg0zefpY; spf=pass (imf02.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686915895; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1NxvrTnWx0fd6TkC5h2RjLp00KHmM+IO3O6L7cY61sI=; b=uK9kxJHhAGvi9oWn8IUYNwprwiLernRf4tHN9zobEW8xRJlt+vqR/GXOvnYCIA2afDwSMQ KwtFRDCNxjfLh6IVe55oRkxuLV/EEmEzIS/a0cct1AIkO7Q6XIkHSzLynw3mBAir11wU2r kdP/bLOK45WT6N5wQ+ZYM+DnqqOT+Hg= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=Rg0zefpY; spf=pass (imf02.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686915895; a=rsa-sha256; cv=none; b=OZZUJ2IK6QT5ceu0tPctm4oKSLIDtWMgmHK7n4x8UAdBl1CEG2RVWpB8Rxvac34V+Sfh5s WKhcsv3j2vSpxEdzeIZif9PE1AYMmSkcrka+jJxZnTe5MeSaBVtloFAk8k2BDFPFIlHoH8 1dY++NoYfGbEcMLLxHy6y6jPnbVs9bw= Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GB89Ve015549; Fri, 16 Jun 2023 11:44:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=1NxvrTnWx0fd6TkC5h2RjLp00KHmM+IO3O6L7cY61sI=; b=Rg0zefpYgAFyJUK6vxVNJinXSR9JUBvc6Ky5MCdaVosIABHttzrKPCKuO6RIkl4YzohN XuDJhgvot9UURJtdWFjH9WQy3garnaM0ECzLSg3Q5vV1PUJ0NyP7D/ev1J98ol02WCpE YzYTGdDT1kCyy7zXedGTWNdNS1Q0o+OX6kIGEOO4pss09B16mjkQltX2eBB/8Z00JzvQ JLFOT5lwqLoq0CMmTm2bhRCEE5WYZ7oqix2vC2icRFUg/qqhxO2doIp2Kxdr212Gm4x7 vP2zOV4MFQ37Vw28JaRJEIhoPFVEQ8b7N4ny32TXH42iEGJ7RByKA1nRr12mRi0jNgsL yg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9m93u3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:44:43 +0000 Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GBfBId030991; Fri, 16 Jun 2023 11:44:42 GMT Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9m93tn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:44:42 +0000 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35G9PiKo030258; Fri, 16 Jun 2023 11:09:43 GMT Received: from smtprelay06.dal12v.mail.ibm.com ([9.208.130.100]) by ppma03dal.us.ibm.com (PPS) with ESMTPS id 3r4gt529ce-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:43 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay06.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GB9gmH2687546 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:09:42 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0C7A658058; Fri, 16 Jun 2023 11:09:42 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 03F375805C; Fri, 16 Jun 2023 11:09:36 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:09:35 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 09/16] mm/vmemmap: Allow architectures to override how vmemmap optimization works Date: Fri, 16 Jun 2023 16:38:19 +0530 Message-Id: <20230616110826.344417-10-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: GMpZt1GP3KN7acv2igOUSqdfrjzguVxv X-Proofpoint-ORIG-GUID: qZv9AyAbTlUNYM2GAQgBNx4vAn01YRIw X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_08,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 mlxscore=0 adultscore=0 lowpriorityscore=0 clxscore=1015 mlxlogscore=999 bulkscore=0 impostorscore=0 priorityscore=1501 suspectscore=0 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160104 X-Rspamd-Queue-Id: ACF0D8001C X-Rspam-User: X-Stat-Signature: 9o4t3ts885ahjd3jnc7ckio1gb8kziuk X-Rspamd-Server: rspam01 X-HE-Tag: 1686915895-206738 X-HE-Meta: U2FsdGVkX19NGceW8M4Qsz84cJZuG/JecZPACWD1Q0yiRsW4DHMJP/3n44zQ0HT6hNxBWnT1p5OY5qDACU5N7HKnn1Qu1/RFaFHF/jUYSTzU+RzIiQ07wo8+h96TvNXpILRFaEn8ZUDCBnvPN2KC5PhCjr28vAc+Cn1QQwLLGgwgwT8YDZr5XvOEAy+vpxY1FnfFxnBBk222UcyCB/9bJ2iWJZors5cTSOhvYlYBVS4t4z1dEKK59hyjqWOJcNyQhXJJt5Wt+ZoIXt9+RFeIRoAGog4zKhFq92L+zTdlAjSub1W1FhKr7/vmrQCyK1o92XN0itdwVQB/aK0hN7GtoWXf2V36V7fIEypHUyXUIaT3qh+7CaUDeBgfqij/Jp6YsYOp5a/707ayb7Qr1hMf0DHba0GUwduikCa1xOLrvCeR+eFhWWxk63keJb2uCRE77SqxCCiQ6bqSEjtDwSJQCN+vtljyRHoPJ0kz4Y71IP5Z9q0nbfQr9CgahG0ZF8rWoedRdeghE7Iboo5TAMrC+NfrL2ifpM/xMkiEhIzl2TqA03m3U/Ko9O2MtJxKEtemeqMMssMlpE1JKuxJ/eOayIW4KQWs7hwWRo5QxIaI7jARcN+S1qxb1KZv4JSh+9SUVoWti5NFHcWYkp5LWb9TF5rj98p5DCRqqWoDwwPWfSm/qb5HANsL75WcFEG6ZEc/fTl8jcCj6CFY1a0G96uH3053bAWctJRFnhnCoXQiiOELDUQmnE3+WyDmB/QBPid6M0s++1ZwGrc6vVZAqyTkhR/loJ6XeVaxujXZNBzFa42qmMR57wgIEJW+bi0ZdBc0CQKp0Z4xSDy4p0e28HToGxAgD5WCxS4m3qYUgFYjR0yXRargQuImGLOCApy/HKggyftCEViv+5Bbfkol/FPdgh5nD8GzPn1feHYqn6izJm0PDgtrep7UxJsOWIOlc5NRa6PGhERR3a+ZL8VldNW wV4lU8J7 QnEI5UBCGWPVD/h2bHTvnRlPQdbfbLVVMDndvBSglzxmTSGtheYYYXRl+PxUZ+aVC06DGjqBE5iKDIu0tVkHDLpiOTo3ZVZcLPb0bqupZU5OiXVABXrtHShAXhmTXHdD2AGOJyKI3SBpogAM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Architectures like powerpc will like to use different page table allocators and mapping mechanisms to implement vmemmap optimization. Similar to vmemmap_populate allow architectures to implement vmemap_populate_compound_pages Signed-off-by: Aneesh Kumar K.V --- mm/sparse-vmemmap.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index 10d73a0dfcec..0b83706c08fd 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -141,6 +141,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node, start, end - 1); } +#ifndef vmemmap_populate_compound_pages pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node, struct vmem_altmap *altmap, struct page *reuse) @@ -446,6 +447,8 @@ static int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn, return 0; } +#endif + struct page * __meminit __populate_section_memmap(unsigned long pfn, unsigned long nr_pages, int nid, struct vmem_altmap *altmap, struct dev_pagemap *pgmap) From patchwork Fri Jun 16 11:08:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282630 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77B5EEB64D8 for ; Fri, 16 Jun 2023 11:10:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0F9DC6B0078; Fri, 16 Jun 2023 07:10:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0A9B86B0080; Fri, 16 Jun 2023 07:10:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E8CCE8E0001; Fri, 16 Jun 2023 07:10:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id D6F936B0078 for ; Fri, 16 Jun 2023 07:10:11 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 52DB640C07 for ; Fri, 16 Jun 2023 11:10:11 +0000 (UTC) X-FDA: 80908341822.12.4FA7981 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by imf24.hostedemail.com (Postfix) with ESMTP id 7671118000C for ; Fri, 16 Jun 2023 11:10:07 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=ey8BLJ4n; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf24.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686913807; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TCoL5E0bYEmTZ0XGp0hkg5Sncgr9tImoJ2a7kXulZh0=; b=bA5TJwRoVpYytUCjOP/rGbbqsmHRpLJRBk+Zjcsqlz/wQiktVI/MUG6YEzibSU/FaiyrWP llt7FsZUjqN+FHPTFxdmtZgpWuUg4ed8R4IeeO5spdYC7HoyjIXEAmTIkO/Co72cEqRslc zESQkbMCt6ukb7OYo/B8KT1J5eoSlVo= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=ey8BLJ4n; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf24.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686913807; a=rsa-sha256; cv=none; b=gJOnCx28NCIVYpSbvdGI7yorZIPrmT0xKJqnOAGM/mqgbx9loKmKA4WPORIAhOU8ARQ6r3 URLQSpu2zu38UAtlIlKTQqitf1VK8b5wuKhqJqmEyRgt3k0zIKD5YovNIBUF+ZVdY44Mgy 2rEF8IGOXObEzPopAUlBQjoULhnWmAU= Received: from pps.filterd (m0353723.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAduYK011546; Fri, 16 Jun 2023 11:09:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=TCoL5E0bYEmTZ0XGp0hkg5Sncgr9tImoJ2a7kXulZh0=; b=ey8BLJ4nyDXDn/h2wRtLHeQYd6IMDuQDt5JmpRtWGXSm8BBFMQAdE/7v8LYEEukdXG53 3q0a12pA89s+KX6557q0PBbooH3RvIW1zRVLNbqw6/wilbjL6GrwNci3H42JAsaia2v1 +IFLgRO8VkdDy1+psroEkbvznplHTnZM1+wEef2wnwrSgg6B2VuweScNkkaN20cAldoh DouUm4R0u+NE26Ap3SgLJX/RRY/08sIE6LNooj5WlMD/F79W15wFLOl5NJ78nIi35ku7 MoSK0xuxhPetRVQNVSnAh32oE03/RnSsvToV6x2oXTBxBi+4+/6s73M7PvgZGa6QkDd6 uQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8nw4h0nd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:51 +0000 Received: from m0353723.ppops.net (m0353723.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GB08jv011610; Fri, 16 Jun 2023 11:09:50 GMT Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8nw4h0n6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:50 +0000 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAMR7V003274; Fri, 16 Jun 2023 11:09:49 GMT Received: from smtprelay03.dal12v.mail.ibm.com ([9.208.130.98]) by ppma01wdc.us.ibm.com (PPS) with ESMTPS id 3r4gt5qhuv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:49 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay03.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GB9mkx54198626 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:09:49 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A339C5805F; Fri, 16 Jun 2023 11:09:48 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D44E75805C; Fri, 16 Jun 2023 11:09:42 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:09:42 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 10/16] mm: Add __HAVE_ARCH_PUD_SAME similar to __HAVE_ARCH_P4D_SAME Date: Fri, 16 Jun 2023 16:38:20 +0530 Message-Id: <20230616110826.344417-11-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: GIvIKJ74U_PMZufrU9kb3Yx16WjbECTn X-Proofpoint-ORIG-GUID: zdvlEzfKit9YTXVRdmDfCXocMrhGxT50 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_07,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 phishscore=0 mlxlogscore=999 mlxscore=0 adultscore=0 malwarescore=0 spamscore=0 impostorscore=0 bulkscore=0 clxscore=1015 suspectscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160099 X-Rspamd-Queue-Id: 7671118000C X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 31j8er7aapa54mzpzrmowayii14jwnio X-HE-Tag: 1686913807-572537 X-HE-Meta: U2FsdGVkX1+gctklsAXPUm42s5cgd/DihJajdo6wluG7KHNFHvyPeV+Onda848jegBEeaWEDg4RD98G8CPNLSYCxO+PKInJQOVQfbrmYYfWNN+ztK5+2KMOQWGZcL+cEznJrLpiHNrzskjqcQVizS9qcBXGslQbt6nbgDFkFKNQjyJJ9jm+/yVi7AMFHP0fYPVOi/3zQPAh4xop1s+lnH0Ovb5xavhZQFnt4luDMd4UGMlZpOU90wi/iOhCxZS56wYuIIFd1g9DylyV8Rv1QQGwkqzaZ5R46lUpt5FB8fbzlEXbmilkRh/ovr88rVEO8Y5RbYN2Ykr3dLg5ppAgfTNhND9ZIs051tLE+qTR7MJfqgmqWGGGsOHQPLWD/fIhVgQI0rEYU7W+sdfIrRDay2tx3U8mDJG+vJb+dnrplorGTQUcJFfxqlcpR5gh71fMNHBsNQbvyzj7OXZAs89xsrOdUEFQ865KCW8WV4ilbZRTzFfddKVY1dKj008X35m2HYrRI5R0AZeK8zre/HH9axyONiNRHCJ+HZMtx/AA/rVijFOsnVefTYDzEQhj5JBvnIffT53cX+wRQV6pbl9Td0dYfW+HcgUxmpuK1niOQaABkoDY2nDfyHDt6D35ozV7GfFe9YSfsTx4yRatmdBK0Q3rkdS6WpXlAezflYR0CJEUYqACb4wph5+RClv1plpumnfP8mDaf5k61GwIOafrqVKmRljZcmy8OjZn6qCY7m0k51CNWyo9t77Q2vECjlOXSbyHOecRQAhZ5VMTnNIgSdMbkcKyuBgIIbH+xk0ywJGNoQ1Uc6iKK6JzjNxPuwcdG2VDCK0AgsMlxJhy3k29/rNCpnQcy91jX9QE1Z4E+kKR+sI5br8njDLc3eXUwvr4+LwxdPC8E5UNNiecn0rjw7LK92UC4vCShajIlCkv0xKB+6Cw0qh6GH8c9oGiDQd/MAGg+XUM0YG72eqYQ/yZ EvLXkRsj afkX/H5nlJWDVM/IY0y8pbQia5ayhX9je4Yxq0leyJCAJ4mI3nVCYzdyp+UkFRFOXOhIn/8LfPtO6tiTNWhxxVMqJ0R26CSJOy/AZSaZa69fh3G+r7e8TB/S/rszxORpM/Rh5r0MKFJggQT2mrLMQT2nT2/i4qwB5sk2YKpPy9OU+ZdukMxbgxJ697DFjNrDydt1x X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This helps architectures to override pmd_same and pud_same independently. Signed-off-by: Aneesh Kumar K.V --- include/linux/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 2fe19720075e..8c5174d1f9db 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -681,7 +681,9 @@ static inline int pmd_same(pmd_t pmd_a, pmd_t pmd_b) { return pmd_val(pmd_a) == pmd_val(pmd_b); } +#endif +#ifndef __HAVE_ARCH_PUD_SAME static inline int pud_same(pud_t pud_a, pud_t pud_b) { return pud_val(pud_a) == pud_val(pud_b); From patchwork Fri Jun 16 11:08:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282702 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5432BEB64D7 for ; Fri, 16 Jun 2023 12:04:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BE1DF6B0074; Fri, 16 Jun 2023 08:04:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B75D46B0075; Fri, 16 Jun 2023 08:04:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A0AD58E0001; Fri, 16 Jun 2023 08:04:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 8D4FA6B0074 for ; Fri, 16 Jun 2023 08:04:58 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 5700CB050C for ; Fri, 16 Jun 2023 12:04:58 +0000 (UTC) X-FDA: 80908479876.28.29BA5AB Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf27.hostedemail.com (Postfix) with ESMTP id CC17B40019 for ; Fri, 16 Jun 2023 12:04:55 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=rJ0CCkKR; spf=pass (imf27.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686917096; a=rsa-sha256; cv=none; b=3Uy77XhWNPDdY+TutezQtrgZLGSdZr7Ydu5hAjJqd++JtBxXVdxrglr8hXcIfAH0t5g3rE oQBezUETCGJYUGPY+BgjRs1W9D2wEB2I5NFIFlQYLlzsDYr11WyxL5I8AnF6VUu35FXRx+ Cn3cGyeoDQmtwuoubLZuIVMXPYFrqfs= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=rJ0CCkKR; spf=pass (imf27.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686917096; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=olZzUnBM/e0N6uqOS2BdEdJT2kmPhsk0clbOlTlXdCc=; b=nJymc5pByW9C8XH3erQGWoPk9nSCBw08+pF4wk4LOI29iq/rwURc3sPdniStsmxbimlE2D QnNGMTjy7wVRh1KX8LKlwiyrPPbjJzTgZW8USm5PF/KRXOQJJmLVI909kU9jtQwM9A8Dwa 12SPe+YX+xCQX3Wc1fefPQnFRw8Gr98= Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GC2Lkr022986; Fri, 16 Jun 2023 12:04:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=olZzUnBM/e0N6uqOS2BdEdJT2kmPhsk0clbOlTlXdCc=; b=rJ0CCkKRp4LNJYjT8Yerck+ryFBBBSdMcQEBlWrRZC5v7Nu/Cnt64ffdVCnqKOK+rHrD wAOyaoTs+Pxd97Ihk5EDYi/JuFG7yPvK5U8Vo4WSdZiQFrCkSE1qetfPcraLVitVNK/z f50JbH/e1xX/MoXNhBbeASxqUJrkmiGaW2W7oYonBxZqy3ISKx+EKvrENGqCzsDGGqzz oXwt52Sjzg0kvhF+UdyJaNg2yeV0UfDlFC+pTl9U1jxggf9csMJkjkv45/K1a+lH+DIn jyE7kC2pWdZa7Fod2CYI0FCF87VqgHF+NBXYljIb2ctjOCAoChdaXsIHyxsTGfUJHlN0 Uw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8qcs82a6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 12:04:42 +0000 Received: from m0360083.ppops.net (m0360083.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GC3H3G028054; Fri, 16 Jun 2023 12:04:42 GMT Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8qcs8276-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 12:04:42 +0000 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35FMGLok020557; Fri, 16 Jun 2023 11:09:57 GMT Received: from smtprelay06.dal12v.mail.ibm.com ([9.208.130.100]) by ppma03dal.us.ibm.com (PPS) with ESMTPS id 3r4gt529ch-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:09:56 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay06.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GB9tVf2687568 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:09:55 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8AB275805D; Fri, 16 Jun 2023 11:09:55 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 46A6358059; Fri, 16 Jun 2023 11:09:49 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:09:48 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 11/16] mm/huge pud: Use transparent huge pud helpers only with CONFIG_TRANSPARENT_HUGEPAGE Date: Fri, 16 Jun 2023 16:38:21 +0530 Message-Id: <20230616110826.344417-12-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: frKTnl-WpFRdIDjBFqJxtBQcZz3jswKD X-Proofpoint-ORIG-GUID: WMYlyHWFkL4rQ2q3O0j-Mohn2_cm_BLS X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_08,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 suspectscore=0 spamscore=0 clxscore=1015 priorityscore=1501 adultscore=0 mlxscore=0 lowpriorityscore=0 bulkscore=0 malwarescore=0 impostorscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160108 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: CC17B40019 X-Stat-Signature: 91cgjn3znmferaqo7hx4npx3t87krj7i X-Rspam-User: X-HE-Tag: 1686917095-307618 X-HE-Meta: U2FsdGVkX1/jw8ku2ainFAArtirNduKcmGq03IcApKVsM3ybjzfiA/lKpRB50SARbgwYEbWQiFovMzTx0jZd3Kkf0fl+BsbnuCDM8wmUQeaSChLb3Ef5zie8lij/NVLZc6gEjE2PhAzztKCtpOnz5HAyvStWfkJKDhGh5vuohV/oLHstf2tqYv4wGJeOgtDYOk4Z8QolfiTlGJ/PzlOCjI3GSuItieGoC/vGhnY3ulZX7oS94vVaCtUS/bYSi9h1TNPpQnnip/5z6KPuI1NRV+uvCrOkM7DlzBnUmNIYymguVvkPfbNDQAypgzCwaNWvu43xXFmRmD6V4jQMnyZ00Yhyf46OXdemOutMvMumLtNAmRaq06iwIsxfjtzv0rrBtkoxiUjV62CvgZKP/V8ftYzRRwy+biC2NhYAm5aY19FLfvKF4uvCgx0KnRCEMycsQaIJDFmGk4aRfH6FFm9gz0o8OdroAuLzpZMsZFRGg8ATHxreW/Ho/6OFWf3OUrtk8Wti/cpZVqGYDj0g0UzFnCA57w1f2ApQW6oD1DahIuTIx/1yCuwa7PDlutNt+PcZBzz6liAnXyGbh/hO7Q6CxK642bVP1QpxbTvlGqW+dVxywrlon1WeUB1/N5v40Sz2Wjro3OLKipWyq6N/ev17E/vlVGEUkc198p9+TJ/H+/wpRzlzfZgL1d1glTvoLGUz8YcVfxYRRh/DfXBWkbs8PygZPyHalZhM53MWCnIz5AWfL9dnnjb4QeYzqEcGZDSC9NQt7w8EOQRZdovCmjqAQj817jfM2xKiSYXNXKfpBZhIQ/XHtnNGTss1I6Y7tgASv/hMuXbSuiBGmSciNwX7bjrTFOjJ4qfWiJJ8GSrjyUlm/QZI6QnKKX+ALWqowY8DqNvtHFRhPAoNtNb290OqMx15usuBC7sFooSSTxKaEiZD+vN2yDOUNsKcjkTzz8B8slhjlW1o7vd87cbQEPM +Yq9JrZ9 /lI7zrgdANpNY9Ykr6Nr5OO+MIuoMwAnf2JJum+6QloPA9jDZUVic9VNkSYmIGe3oKSPQqUVn7gmFo0ATr2xgJ+w18Hjr0cZ6Kh+AtxBoPO8yT5n/b82nPmMxUy1yhcV9vsBVou8WW/dldL6EJEJP8HOsFJBzEFvFqIfs8gCAj5jygZRJdS7i3QhFAE5TTO6r1q/p X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: pudp_set_wrprotect and move_huge_pud helpers are only used when CONFIG_TRANSPARENT_HUGEPAGE is enabled. Similar to pmdp_set_wrprotect and move_huge_pmd_helpers use architecture override only if CONFIG_TRANSPARENT_HUGEPAGE is set Signed-off-by: Aneesh Kumar K.V --- include/linux/pgtable.h | 2 ++ mm/mremap.c | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 8c5174d1f9db..c7f5806dc9d1 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -550,6 +550,7 @@ static inline void pmdp_set_wrprotect(struct mm_struct *mm, #endif #ifndef __HAVE_ARCH_PUDP_SET_WRPROTECT #ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD +#ifdef CONFIG_TRANSPARENT_HUGEPAGE static inline void pudp_set_wrprotect(struct mm_struct *mm, unsigned long address, pud_t *pudp) { @@ -563,6 +564,7 @@ static inline void pudp_set_wrprotect(struct mm_struct *mm, { BUILD_BUG(); } +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ #endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */ #endif diff --git a/mm/mremap.c b/mm/mremap.c index b11ce6c92099..6373db571e5c 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -338,7 +338,7 @@ static inline bool move_normal_pud(struct vm_area_struct *vma, } #endif -#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) static bool move_huge_pud(struct vm_area_struct *vma, unsigned long old_addr, unsigned long new_addr, pud_t *old_pud, pud_t *new_pud) { From patchwork Fri Jun 16 11:08:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282631 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3438FEB64D7 for ; Fri, 16 Jun 2023 11:10:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C1D5F6B007B; Fri, 16 Jun 2023 07:10:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BCDA08E0001; Fri, 16 Jun 2023 07:10:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A6E306B0081; Fri, 16 Jun 2023 07:10:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 976A36B007B for ; Fri, 16 Jun 2023 07:10:18 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 76A19120C89 for ; Fri, 16 Jun 2023 11:10:18 +0000 (UTC) X-FDA: 80908342116.28.425B72E Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf20.hostedemail.com (Postfix) with ESMTP id F35831C0002 for ; Fri, 16 Jun 2023 11:10:15 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b="RUFGDkV/"; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf20.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686913816; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fnptJB5qGAcHT1CUlyBof6IMLSq55vUSDFTN16YlJMU=; b=EguKLXmgdu2vca3exHK25IvOxGJ/SEW+BUw/Tp67cJI0/Mm2gkIUWE3dZ1nGf3h8IYtzZP IKj4iQSrRnVdFlcas19Q21sQjz9TV3DIp6Ony6aQEXBpJUBv31JJfts3gIyUmlj8f8TNQN /ChdfapdCRywh0tQtD3voSxnXckLDRE= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b="RUFGDkV/"; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf20.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686913816; a=rsa-sha256; cv=none; b=n7sYSokr6txzX5Rq3l3k6bq6/V61DpBu36NpF5wzzxjYW7Ws8vOW06zllwUn1eBC2rfdxW WJoi5QEsLPe+sJv8VcskY/PGfrf7xW6HOrkWKfCGldAkUTOPB6zj2YOmyU9zzKiEqXLxVc BTJ42qxNy5UIvXOYJZiSDRf5OuRKK2k= Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAl7hT012077; Fri, 16 Jun 2023 11:10:06 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=fnptJB5qGAcHT1CUlyBof6IMLSq55vUSDFTN16YlJMU=; b=RUFGDkV/YOyUnQk7PIhvQaVSZdhcPhVgfcs0S6gaOIsaxTaCswznPk0c0SKhqCWB23XV skoS/+iIb2T6ABkvygfiyd+/h/ZOglgOQA+n6DV3jdB5mnY6oeJP6HsfGDJUOvUEkwkR CEX3iTXh376FU8ycWICkG8fhPPf00ot+Crgy4lqNCMcJX2eCRH7rDj3sbJ+86cZLC76V lexAf6aPRv94hGC/TozFxrcXl3ldx2mvQj1q0uhUua5HtNZkNeoQbMckeEFD7UcPYCEI r17r9uk/HTOG7lcjM5p1tTUwn1Dif6fw6E98hxP3LoHztb80exDbPdXWvPSOOsFPsxMs OQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9n8e7s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:10:05 +0000 Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GB59ko029868; Fri, 16 Jun 2023 11:10:05 GMT Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9n8e73-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:10:05 +0000 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35G8w3MO012063; Fri, 16 Jun 2023 11:10:04 GMT Received: from smtprelay04.wdc07v.mail.ibm.com ([9.208.129.114]) by ppma01dal.us.ibm.com (PPS) with ESMTPS id 3r4gt6s34x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:10:04 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay04.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GBA2JU28771058 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:10:02 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 87C3858058; Fri, 16 Jun 2023 11:10:02 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 454595805C; Fri, 16 Jun 2023 11:09:56 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:09:55 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 12/16] mm/vmemmap optimization: Split hugetlb and devdax vmemmap optimization Date: Fri, 16 Jun 2023 16:38:22 +0530 Message-Id: <20230616110826.344417-13-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: or_7neAhvWhjnojFDN9BtabCUIumjqQv X-Proofpoint-GUID: sXKKgXKZW4iyFjCUKvQCt774s8hcpCSd X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_07,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 bulkscore=0 priorityscore=1501 mlxlogscore=999 suspectscore=0 lowpriorityscore=0 mlxscore=0 spamscore=0 adultscore=0 impostorscore=0 malwarescore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160099 X-Rspamd-Queue-Id: F35831C0002 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: enjui8at44ksnbuxkjo763zbtyjfjrju X-HE-Tag: 1686913815-332555 X-HE-Meta: U2FsdGVkX19kvndadbtwSVeaHvsMRzETEW7bppnQbRX21YzU18LZr5vUKPvUDxW5Y5jUuJUDyIIdkImSNJLO1kYUM9Y8FohII3CELBORVtTVCdWD/MypoTXy6xUVJmq03HSSy21i8FEloUwwut8Um0Y2hn8nKkyFO8Jw8/T7muakE3rv6bXhaSIh+yH7DI8VGedf58DdUr+9PQHfVTsmaP5MsZM1kxWtbCTh1yXqIDHaJ/XZCTGFez/ft5YBLev5DaHbtsmLMUhC0XApD1b7hDdizVY/HfXW3+SSaruihv773QRq5L2s++zUSWwG58QS9o9eJymaLJhwHrpwmFxaFrbSrcJUJLp37ppq1AfFVYQ+OkmYa6kB4uSynYxtd4vYMO+V22Ywhl2AKcTcJJAUjxZOUcieILi9YuVC0yCNuQt/ltRi76BcJWwjEcR9tnOuemCa5oitQ9SPp9IfvPnERD259/KbLB9dKTlw+xedfnLDbO8UpBma15ecBctJ4TJa6vZD/uq0/GLYtEGnICyonoPvMbNVXuZbcGxOKdq709GPdYutom06nGgIK+Ly11Sf95T6nK4TATyTdJLTiKR/WhAJPlLSf/vMVP+J4wv/PFWOx0YBPyZzgnQbhhYdAavfVH2pv9A1xFL0dmZUgEN+syI2V+j83lsK+4CPBNCxZi2JBc7fEuyYFU9cLU+tspP0tJlFK3C1giisLZlv9BoLXfVoxo7XqCkw5ObFPYMRcJIUKUS8sqUkVSwy16FCkGybaPJvBfpNLJkZSNqPvpfR2vnR1JRw3coEArAFsbK+/a900eFw5qg4ACo/g6X/01JYl/EApOKRWx/NhHOBG705rBUn2TbzIRuD2PP6c702iBBPll8M5tPIBIbOyKTRtyU2ShhFIvYP8ZMNuf0clV1Ninn7PVERRVczvWKEbYX/Gri2nK0kPtcsaCVgnV32H3ibrK2pjYIqsF4a1RCw9lN /zOHYOLB 6a7+8k/i9zDmTfcfcHUEALuqU9zag8YWD7VCxb7Z/y5jHZTDTovXAgtkfxxUjY5dfnxTbfoSGvMqMvp/38SKjOvwZ2lBY9txT1aOaVhyuf4qFs2NlHmdEO4EyBqt5+VFEt2WsR38MwV3EsM0d8YHCKqInwHWWOFqDDXVuG9WadvBNNlE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Arm disabled hugetlb vmemmap optimization [1] because hugetlb vmemmap optimization includes an update of both the permissions (writeable to read-only) and the output address (pfn) of the vmemmap ptes. That is not supported without unmapping of pte(marking it invalid) by some architectures. With DAX vmemmap optimization we don't require such pte updates and architectures can enable DAX vmemmap optimization while having hugetlb vmemmap optimization disabled. Hence split DAX optimization support into a different config. loongarch and riscv don't have devdax support. So the DAX config is not enabled for them. With this change, arm64 should be able to select DAX optimization [1] commit 060a2c92d1b6 ("arm64: mm: hugetlb: Disable HUGETLB_PAGE_OPTIMIZE_VMEMMAP") Signed-off-by: Aneesh Kumar K.V --- arch/loongarch/Kconfig | 2 +- arch/riscv/Kconfig | 2 +- arch/x86/Kconfig | 3 ++- fs/Kconfig | 2 +- include/linux/mm.h | 2 +- mm/Kconfig | 5 ++++- 6 files changed, 10 insertions(+), 6 deletions(-) diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index d38b066fc931..2060990c4612 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -55,7 +55,7 @@ config LOONGARCH select ARCH_USE_QUEUED_SPINLOCKS select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT select ARCH_WANT_LD_ORPHAN_WARN - select ARCH_WANT_OPTIMIZE_VMEMMAP + select ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP select ARCH_WANTS_NO_INSTR select BUILDTIME_TABLE_SORT select COMMON_CLK diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 5966ad97c30c..d8f3765ff115 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -50,7 +50,7 @@ config RISCV select ARCH_WANT_GENERAL_HUGETLB if !RISCV_ISA_SVNAPOT select ARCH_WANT_HUGE_PMD_SHARE if 64BIT select ARCH_WANT_LD_ORPHAN_WARN if !XIP_KERNEL - select ARCH_WANT_OPTIMIZE_VMEMMAP + select ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP select ARCH_WANTS_THP_SWAP if HAVE_ARCH_TRANSPARENT_HUGEPAGE select BINFMT_FLAT_NO_DATA_START_OFFSET if !MMU select BUILDTIME_TABLE_SORT if MMU diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 53bab123a8ee..eb383960b6ee 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -127,7 +127,8 @@ config X86 select ARCH_WANT_GENERAL_HUGETLB select ARCH_WANT_HUGE_PMD_SHARE select ARCH_WANT_LD_ORPHAN_WARN - select ARCH_WANT_OPTIMIZE_VMEMMAP if X86_64 + select ARCH_WANT_OPTIMIZE_DAX_VMEMMAP if X86_64 + select ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP if X86_64 select ARCH_WANTS_THP_SWAP if X86_64 select ARCH_HAS_PARANOID_L1D_FLUSH select BUILDTIME_TABLE_SORT diff --git a/fs/Kconfig b/fs/Kconfig index 18d034ec7953..9c104c130a6e 100644 --- a/fs/Kconfig +++ b/fs/Kconfig @@ -252,7 +252,7 @@ config HUGETLB_PAGE config HUGETLB_PAGE_OPTIMIZE_VMEMMAP def_bool HUGETLB_PAGE - depends on ARCH_WANT_OPTIMIZE_VMEMMAP + depends on ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP depends on SPARSEMEM_VMEMMAP config HUGETLB_PAGE_OPTIMIZE_VMEMMAP_DEFAULT_ON diff --git a/include/linux/mm.h b/include/linux/mm.h index 9a45e61cd83f..6e56ae09f0c1 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3553,7 +3553,7 @@ void vmemmap_free(unsigned long start, unsigned long end, #endif #define VMEMMAP_RESERVE_NR 2 -#ifdef CONFIG_ARCH_WANT_OPTIMIZE_VMEMMAP +#ifdef CONFIG_ARCH_WANT_OPTIMIZE_DAX_VMEMMAP static inline bool __vmemmap_can_optimize(struct vmem_altmap *altmap, struct dev_pagemap *pgmap) { diff --git a/mm/Kconfig b/mm/Kconfig index 7672a22647b4..7b388c10baab 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -461,7 +461,10 @@ config SPARSEMEM_VMEMMAP # Select this config option from the architecture Kconfig, if it is preferred # to enable the feature of HugeTLB/dev_dax vmemmap optimization. # -config ARCH_WANT_OPTIMIZE_VMEMMAP +config ARCH_WANT_OPTIMIZE_DAX_VMEMMAP + bool + +config ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP bool config HAVE_MEMBLOCK_PHYS_MAP From patchwork Fri Jun 16 11:08:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282632 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A728EEB64D8 for ; Fri, 16 Jun 2023 11:10:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 42DA96B0078; Fri, 16 Jun 2023 07:10:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3DE316B0081; Fri, 16 Jun 2023 07:10:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 230936B0082; Fri, 16 Jun 2023 07:10:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 12EA76B0078 for ; Fri, 16 Jun 2023 07:10:28 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id DE094120C89 for ; Fri, 16 Jun 2023 11:10:27 +0000 (UTC) X-FDA: 80908342494.22.DFFC98A Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf30.hostedemail.com (Postfix) with ESMTP id 752678000F for ; Fri, 16 Jun 2023 11:10:25 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=CWyTEp0q; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf30.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686913825; a=rsa-sha256; cv=none; b=2zUJvZZS1zTZUOqWw6qjRtG5lSciuHPxFpDoJ5xJA/rCUEure8s1f3COrqYto0jbwifIUV TdO1VjVTCL9NN/Dg2gUo85GX/kTReRuyzk8hcjmp4obMljVrnGb+5tJqp1r039ptd7i0xP vusO54KL1/9vH3Q9Fm6Qge9LNVjGKnM= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=CWyTEp0q; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf30.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686913825; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LqJgFdj7v6/Kir5NbuyKDW+G4XcT7Diemwzy6cmsNzE=; b=Rw0FKvx7p2heodDvAA0sbhCk6ASZm3rs83O73TWaGhitpH3fdps9rKufLOwqjrTSKa4Tx0 to7uiU60Fo8a1N+LbYdnmJSCoXk3s388brboPwnqUwzT1K4XWM7bqL9tf6dqgoSmcFQD0M fV8+i797NKJoFROxRCqHxwOoYEY1Aoo= Received: from pps.filterd (m0353728.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAoxHD018756; Fri, 16 Jun 2023 11:10:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=LqJgFdj7v6/Kir5NbuyKDW+G4XcT7Diemwzy6cmsNzE=; b=CWyTEp0qYr73mL9fJXYXe/Uee7YRUseIAcg0PM91MKb335/gvxs0Fkg7TCjjYMP59CUw tbKGtTbD8BZA0aH2oBuEOofJ5BuhQuzVqjQx+73WndPNrQerAwb5Y8+twfIJVpyUVo6C tEc79NhL1z7uKRC945AgF1JKPVP/6d0E+RSAsA2rH57yK/H0fUfSaSI6NBmn5uHCTIs9 RUkxUZXs4D7DBbX2PXTy5nH+n4NjmZ6zfl2VJ7fNXbbt4GeuyEjiIIllBoiTEg6MKBKt s65Vi8HLTkJxXJRtUXk5g3uuEtgLFIj+QMoM3Xr1ABoAZ3Fj6aIVXbk14OjVc1IxtlVr YA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8pbm0fr5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:10:14 +0000 Received: from m0353728.ppops.net (m0353728.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GAucj3005020; Fri, 16 Jun 2023 11:10:13 GMT Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8pbm0fpr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:10:13 +0000 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35G9UMTR027273; Fri, 16 Jun 2023 11:10:11 GMT Received: from smtprelay07.dal12v.mail.ibm.com ([9.208.130.99]) by ppma02dal.us.ibm.com (PPS) with ESMTPS id 3r4gt6gy4u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:10:11 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay07.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GBA9fL38732426 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:10:10 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C3C3658059; Fri, 16 Jun 2023 11:10:09 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 54F425805C; Fri, 16 Jun 2023 11:10:03 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:10:02 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 13/16] powerpc/book3s64/mm: Enable transparent pud hugepage Date: Fri, 16 Jun 2023 16:38:23 +0530 Message-Id: <20230616110826.344417-14-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: OW6ZXt6qLxF8YJSPIpaQ5cdiBv--IRx- X-Proofpoint-ORIG-GUID: lkjZhVdcBFmXEnlBDHppvd-92Bt6e6dN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_06,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 malwarescore=0 bulkscore=0 spamscore=0 mlxlogscore=999 phishscore=0 adultscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 priorityscore=1501 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160099 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 752678000F X-Stat-Signature: q71xcc98se1771157t14ncikrbu5jeo4 X-HE-Tag: 1686913825-256363 X-HE-Meta: U2FsdGVkX19TUihh1i69PY7bkyjGf1nlzyEGQak6PDxdfKVYjtRgb7aHw1oUhtuJF8k7Mm/FQdyUBD10wkjl8EMWjl1/pDVQothFC56OIkHyhwHDCYBgubCxZ469hu5OunOzwXdgz9CLl4ptbUw602kMUNfNdnoPKtqLk5Sriu1cTsH7iOQmyCNhm+79JL8AOiTDHpmnSS1JXki/vCE5T1SlEUZlATJRlWnXBM1OBpqBzXr/wkf/2Xq350iTeYJI6xf9/KXd8kfQSTPDDc1OpX8GvLf8eqH7IfR776Vo4vDOgyfcL1qBpFMXotZy2/mCHrvzV2xVbdGbE/ZTfE8jxdwS1ZL2yXC66Z/Eo1+TQhhpZU8r6hB58+cj8mMBX2LXi8jEf3OkaDZLQondAeM+NEbUvYbuBu/gMdZJ5PEk26qDOivj32tBx2zzuxXrt+sMWTLTpVsSPV7gFx5+truCwWWnAcF1ATBlVkUHA1AywP8I/QRDOoSymh1uw+fkoQpSym2QRNGKKIWbnCp7xIVBE4JfIwFYv330JjX5jS4d2+szfDDnzV9iVJEdaQxzkWacYJjrfTAISX5XdYpfwOtd5n6MsnfMUr2yYv44hbR1sCLqeP1lsnCWws2jcvNWaRyReYuo2wvNbASmLwQTBJPStgNTitpKtUXZhp3a+5CmJJvwFnbjosT6P9/u1REW2Y4Z5BguDSsZunYrEIhEzr3LHFH9pvQDxRIWUjlz8aplI1fTadjtGINbjq0Iflnw1UT0sWr+pfR74SNtQY0l/Uk6bfQ+7/JNlTj79mcE2F65PIQnwiL3vxoEqEMbXwxMlKVGr+xHbe/jxef1bypjNff5OxWgxVjsfaw8pDpO1qGF8H3CYzNdMFdgZOZL8i6vC7pD9qQfgXunQRH40bMvxiH/M6XX+nfo30abPBbUwJm7QnTaolA/5TPiZCaP8sNZ7fRiqrvT4mVtfQF8wB3mztS JYvuA/P+ W3CEAqJixV6EpQJOPxJQ7lNSCbOZuf6Uqj93rEq9v9LCaXXi8WIMQb75CJr0fdJWppfNJkQuY0DSt+UPpPXedBGZ9HuB3ZGLKHPZpQd48ZI6/pdDuICBUSoGeKZqJbPmIxBTHecO1fLpBbkw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is enabled only with radix translation and 1G hugepage size. This will be used with devdax device memory with a namespace alignment of 1G. Anon transparent hugepage is not supported even though we do have helpers checking pud_trans_huge(). We should never find that return true. The only expected pte bit combination is _PAGE_PTE | _PAGE_DEVMAP. Some of the helpers are never expected to get called on hash translation and hence is marked to call BUG() in such a case. Signed-off-by: Aneesh Kumar K.V --- arch/powerpc/include/asm/book3s/64/pgtable.h | 156 ++++++++++++++++-- arch/powerpc/include/asm/book3s/64/radix.h | 37 +++++ .../include/asm/book3s/64/tlbflush-radix.h | 2 + arch/powerpc/include/asm/book3s/64/tlbflush.h | 8 + arch/powerpc/mm/book3s64/pgtable.c | 78 +++++++++ arch/powerpc/mm/book3s64/radix_pgtable.c | 28 ++++ arch/powerpc/mm/book3s64/radix_tlb.c | 7 + arch/powerpc/platforms/Kconfig.cputype | 1 + include/trace/events/thp.h | 17 ++ 9 files changed, 323 insertions(+), 11 deletions(-) diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index 4acc9690f599..9a05de007956 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -921,8 +921,29 @@ static inline pud_t pte_pud(pte_t pte) { return __pud_raw(pte_raw(pte)); } + +static inline pte_t *pudp_ptep(pud_t *pud) +{ + return (pte_t *)pud; +} + +#define pud_pfn(pud) pte_pfn(pud_pte(pud)) +#define pud_dirty(pud) pte_dirty(pud_pte(pud)) +#define pud_young(pud) pte_young(pud_pte(pud)) +#define pud_mkold(pud) pte_pud(pte_mkold(pud_pte(pud))) +#define pud_wrprotect(pud) pte_pud(pte_wrprotect(pud_pte(pud))) +#define pud_mkdirty(pud) pte_pud(pte_mkdirty(pud_pte(pud))) +#define pud_mkclean(pud) pte_pud(pte_mkclean(pud_pte(pud))) +#define pud_mkyoung(pud) pte_pud(pte_mkyoung(pud_pte(pud))) +#define pud_mkwrite(pud) pte_pud(pte_mkwrite(pud_pte(pud))) #define pud_write(pud) pte_write(pud_pte(pud)) +#ifdef CONFIG_HAVE_ARCH_SOFT_DIRTY +#define pud_soft_dirty(pmd) pte_soft_dirty(pud_pte(pud)) +#define pud_mksoft_dirty(pmd) pte_pud(pte_mksoft_dirty(pud_pte(pud))) +#define pud_clear_soft_dirty(pmd) pte_pud(pte_clear_soft_dirty(pud_pte(pud))) +#endif /* CONFIG_HAVE_ARCH_SOFT_DIRTY */ + static inline int pud_bad(pud_t pud) { if (radix_enabled()) @@ -1115,15 +1136,24 @@ static inline bool pmd_access_permitted(pmd_t pmd, bool write) #ifdef CONFIG_TRANSPARENT_HUGEPAGE extern pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot); +extern pud_t pfn_pud(unsigned long pfn, pgprot_t pgprot); extern pmd_t mk_pmd(struct page *page, pgprot_t pgprot); extern pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot); extern void set_pmd_at(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp, pmd_t pmd); +extern void set_pud_at(struct mm_struct *mm, unsigned long addr, + pud_t *pudp, pud_t pud); + static inline void update_mmu_cache_pmd(struct vm_area_struct *vma, unsigned long addr, pmd_t *pmd) { } +static inline void update_mmu_cache_pud(struct vm_area_struct *vma, + unsigned long addr, pud_t *pud) +{ +} + extern int hash__has_transparent_hugepage(void); static inline int has_transparent_hugepage(void) { @@ -1133,6 +1163,14 @@ static inline int has_transparent_hugepage(void) } #define has_transparent_hugepage has_transparent_hugepage +static inline int has_transparent_pud_hugepage(void) +{ + if (radix_enabled()) + return radix__has_transparent_pud_hugepage(); + return 0; +} +#define has_transparent_pud_hugepage has_transparent_pud_hugepage + static inline unsigned long pmd_hugepage_update(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp, unsigned long clr, unsigned long set) @@ -1142,6 +1180,16 @@ pmd_hugepage_update(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp, return hash__pmd_hugepage_update(mm, addr, pmdp, clr, set); } +static inline unsigned long +pud_hugepage_update(struct mm_struct *mm, unsigned long addr, pud_t *pudp, + unsigned long clr, unsigned long set) +{ + if (radix_enabled()) + return radix__pud_hugepage_update(mm, addr, pudp, clr, set); + BUG(); + return pud_val(*pudp); +} + /* * returns true for pmd migration entries, THP, devmap, hugetlb * But compile time dependent on THP config @@ -1151,6 +1199,11 @@ static inline int pmd_large(pmd_t pmd) return !!(pmd_raw(pmd) & cpu_to_be64(_PAGE_PTE)); } +static inline int pud_large(pud_t pud) +{ + return !!(pud_raw(pud) & cpu_to_be64(_PAGE_PTE)); +} + /* * For radix we should always find H_PAGE_HASHPTE zero. Hence * the below will work for radix too @@ -1166,6 +1219,17 @@ static inline int __pmdp_test_and_clear_young(struct mm_struct *mm, return ((old & _PAGE_ACCESSED) != 0); } +static inline int __pudp_test_and_clear_young(struct mm_struct *mm, + unsigned long addr, pud_t *pudp) +{ + unsigned long old; + + if ((pud_raw(*pudp) & cpu_to_be64(_PAGE_ACCESSED | H_PAGE_HASHPTE)) == 0) + return 0; + old = pud_hugepage_update(mm, addr, pudp, _PAGE_ACCESSED, 0); + return ((old & _PAGE_ACCESSED) != 0); +} + #define __HAVE_ARCH_PMDP_SET_WRPROTECT static inline void pmdp_set_wrprotect(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp) @@ -1174,6 +1238,14 @@ static inline void pmdp_set_wrprotect(struct mm_struct *mm, unsigned long addr, pmd_hugepage_update(mm, addr, pmdp, _PAGE_WRITE, 0); } +#define __HAVE_ARCH_PUDP_SET_WRPROTECT +static inline void pudp_set_wrprotect(struct mm_struct *mm, unsigned long addr, + pud_t *pudp) +{ + if (pud_write(*pudp)) + pud_hugepage_update(mm, addr, pudp, _PAGE_WRITE, 0); +} + /* * Only returns true for a THP. False for pmd migration entry. * We also need to return true when we come across a pte that @@ -1195,6 +1267,17 @@ static inline int pmd_trans_huge(pmd_t pmd) return hash__pmd_trans_huge(pmd); } +static inline int pud_trans_huge(pud_t pud) +{ + if (!pud_present(pud)) + return false; + + if (radix_enabled()) + return radix__pud_trans_huge(pud); + return 0; +} + + #define __HAVE_ARCH_PMD_SAME static inline int pmd_same(pmd_t pmd_a, pmd_t pmd_b) { @@ -1203,6 +1286,16 @@ static inline int pmd_same(pmd_t pmd_a, pmd_t pmd_b) return hash__pmd_same(pmd_a, pmd_b); } +#define __HAVE_ARCH_PUD_SAME +static inline int pud_same(pud_t pud_a, pud_t pud_b) +{ + if (radix_enabled()) + return radix__pud_same(pud_a, pud_b); + BUG(); + return 0; +} + + static inline pmd_t __pmd_mkhuge(pmd_t pmd) { if (radix_enabled()) @@ -1210,6 +1303,14 @@ static inline pmd_t __pmd_mkhuge(pmd_t pmd) return hash__pmd_mkhuge(pmd); } +static inline pud_t __pud_mkhuge(pud_t pud) +{ + if (radix_enabled()) + return radix__pud_mkhuge(pud); + BUG(); + return pud; +} + /* * pfn_pmd return a pmd_t that can be used as pmd pte entry. */ @@ -1225,14 +1326,34 @@ static inline pmd_t pmd_mkhuge(pmd_t pmd) return pmd; } +static inline pud_t pud_mkhuge(pud_t pud) +{ +#ifdef CONFIG_DEBUG_VM + if (radix_enabled()) + WARN_ON((pud_raw(pud) & cpu_to_be64(_PAGE_PTE)) == 0); + else + WARN_ON(1); +#endif + return pud; +} + + #define __HAVE_ARCH_PMDP_SET_ACCESS_FLAGS extern int pmdp_set_access_flags(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp, pmd_t entry, int dirty); +#define __HAVE_ARCH_PUDP_SET_ACCESS_FLAGS +extern int pudp_set_access_flags(struct vm_area_struct *vma, + unsigned long address, pud_t *pudp, + pud_t entry, int dirty); #define __HAVE_ARCH_PMDP_TEST_AND_CLEAR_YOUNG extern int pmdp_test_and_clear_young(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp); +#define __HAVE_ARCH_PUDP_TEST_AND_CLEAR_YOUNG +extern int pudp_test_and_clear_young(struct vm_area_struct *vma, + unsigned long address, pud_t *pudp); + #define __HAVE_ARCH_PMDP_HUGE_GET_AND_CLEAR static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm, @@ -1243,6 +1364,16 @@ static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm, return hash__pmdp_huge_get_and_clear(mm, addr, pmdp); } +#define __HAVE_ARCH_PUDP_HUGE_GET_AND_CLEAR +static inline pud_t pudp_huge_get_and_clear(struct mm_struct *mm, + unsigned long addr, pud_t *pudp) +{ + if (radix_enabled()) + return radix__pudp_huge_get_and_clear(mm, addr, pudp); + BUG(); + return *pudp; +} + static inline pmd_t pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp) { @@ -1257,6 +1388,11 @@ pmd_t pmdp_huge_get_and_clear_full(struct vm_area_struct *vma, unsigned long addr, pmd_t *pmdp, int full); +#define __HAVE_ARCH_PUDP_HUGE_GET_AND_CLEAR_FULL +pud_t pudp_huge_get_and_clear_full(struct vm_area_struct *vma, + unsigned long addr, + pud_t *pudp, int full); + #define __HAVE_ARCH_PGTABLE_DEPOSIT static inline void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, pgtable_t pgtable) @@ -1305,6 +1441,14 @@ static inline pmd_t pmd_mkdevmap(pmd_t pmd) return hash__pmd_mkdevmap(pmd); } +static inline pud_t pud_mkdevmap(pud_t pud) +{ + if (radix_enabled()) + return radix__pud_mkdevmap(pud); + BUG(); + return pud; +} + static inline int pmd_devmap(pmd_t pmd) { return pte_devmap(pmd_pte(pmd)); @@ -1312,7 +1456,7 @@ static inline int pmd_devmap(pmd_t pmd) static inline int pud_devmap(pud_t pud) { - return 0; + return pte_devmap(pud_pte(pud)); } static inline int pgd_devmap(pgd_t pgd) @@ -1321,16 +1465,6 @@ static inline int pgd_devmap(pgd_t pgd) } #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ -static inline int pud_pfn(pud_t pud) -{ - /* - * Currently all calls to pud_pfn() are gated around a pud_devmap() - * check so this should never be used. If it grows another user we - * want to know about it. - */ - BUILD_BUG(); - return 0; -} #define __HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION pte_t ptep_modify_prot_start(struct vm_area_struct *, unsigned long, pte_t *); void ptep_modify_prot_commit(struct vm_area_struct *, unsigned long, diff --git a/arch/powerpc/include/asm/book3s/64/radix.h b/arch/powerpc/include/asm/book3s/64/radix.h index 686001eda936..8cdff5a05011 100644 --- a/arch/powerpc/include/asm/book3s/64/radix.h +++ b/arch/powerpc/include/asm/book3s/64/radix.h @@ -250,6 +250,10 @@ static inline int radix__pud_bad(pud_t pud) return !!(pud_val(pud) & RADIX_PUD_BAD_BITS); } +static inline int radix__pud_same(pud_t pud_a, pud_t pud_b) +{ + return ((pud_raw(pud_a) ^ pud_raw(pud_b)) == 0); +} static inline int radix__p4d_bad(p4d_t p4d) { @@ -268,9 +272,22 @@ static inline pmd_t radix__pmd_mkhuge(pmd_t pmd) return __pmd(pmd_val(pmd) | _PAGE_PTE); } +static inline int radix__pud_trans_huge(pud_t pud) +{ + return (pud_val(pud) & (_PAGE_PTE | _PAGE_DEVMAP)) == _PAGE_PTE; +} + +static inline pud_t radix__pud_mkhuge(pud_t pud) +{ + return __pud(pud_val(pud) | _PAGE_PTE); +} + extern unsigned long radix__pmd_hugepage_update(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp, unsigned long clr, unsigned long set); +extern unsigned long radix__pud_hugepage_update(struct mm_struct *mm, unsigned long addr, + pud_t *pudp, unsigned long clr, + unsigned long set); extern pmd_t radix__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp); extern void radix__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, @@ -278,6 +295,9 @@ extern void radix__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, extern pgtable_t radix__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); extern pmd_t radix__pmdp_huge_get_and_clear(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp); +pud_t radix__pudp_huge_get_and_clear(struct mm_struct *mm, + unsigned long addr, pud_t *pudp); + static inline int radix__has_transparent_hugepage(void) { /* For radix 2M at PMD level means thp */ @@ -285,6 +305,14 @@ static inline int radix__has_transparent_hugepage(void) return 1; return 0; } + +static inline int radix__has_transparent_pud_hugepage(void) +{ + /* For radix 1G at PUD level means pud hugepage support */ + if (mmu_psize_defs[MMU_PAGE_1G].shift == PUD_SHIFT) + return 1; + return 0; +} #endif static inline pmd_t radix__pmd_mkdevmap(pmd_t pmd) @@ -292,9 +320,18 @@ static inline pmd_t radix__pmd_mkdevmap(pmd_t pmd) return __pmd(pmd_val(pmd) | (_PAGE_PTE | _PAGE_DEVMAP)); } +static inline pud_t radix__pud_mkdevmap(pud_t pud) +{ + return __pud(pud_val(pud) | (_PAGE_PTE | _PAGE_DEVMAP)); +} + +struct vmem_altmap; +struct dev_pagemap; extern int __meminit radix__vmemmap_create_mapping(unsigned long start, unsigned long page_size, unsigned long phys); +int __meminit radix__vmemmap_populate(unsigned long start, unsigned long end, + int node, struct vmem_altmap *altmap); extern void radix__vmemmap_remove_mapping(unsigned long start, unsigned long page_size); diff --git a/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h b/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h index 77797a2a82eb..a38542259fab 100644 --- a/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h +++ b/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h @@ -68,6 +68,8 @@ void radix__flush_tlb_pwc_range_psize(struct mm_struct *mm, unsigned long start, unsigned long end, int psize); extern void radix__flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); +extern void radix__flush_pud_tlb_range(struct vm_area_struct *vma, + unsigned long start, unsigned long end); extern void radix__flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); extern void radix__flush_tlb_kernel_range(unsigned long start, unsigned long end); diff --git a/arch/powerpc/include/asm/book3s/64/tlbflush.h b/arch/powerpc/include/asm/book3s/64/tlbflush.h index 0d0c1447ecf0..a01c20a8fbf7 100644 --- a/arch/powerpc/include/asm/book3s/64/tlbflush.h +++ b/arch/powerpc/include/asm/book3s/64/tlbflush.h @@ -50,6 +50,14 @@ static inline void flush_pmd_tlb_range(struct vm_area_struct *vma, radix__flush_pmd_tlb_range(vma, start, end); } +#define __HAVE_ARCH_FLUSH_PUD_TLB_RANGE +static inline void flush_pud_tlb_range(struct vm_area_struct *vma, + unsigned long start, unsigned long end) +{ + if (radix_enabled()) + radix__flush_pud_tlb_range(vma, start, end); +} + #define __HAVE_ARCH_FLUSH_HUGETLB_TLB_RANGE static inline void flush_hugetlb_tlb_range(struct vm_area_struct *vma, unsigned long start, diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/pgtable.c index 85c84e89e3ea..9e5f01a1738c 100644 --- a/arch/powerpc/mm/book3s64/pgtable.c +++ b/arch/powerpc/mm/book3s64/pgtable.c @@ -64,11 +64,39 @@ int pmdp_set_access_flags(struct vm_area_struct *vma, unsigned long address, return changed; } +int pudp_set_access_flags(struct vm_area_struct *vma, unsigned long address, + pud_t *pudp, pud_t entry, int dirty) +{ + int changed; +#ifdef CONFIG_DEBUG_VM + WARN_ON(!pud_devmap(*pudp)); + assert_spin_locked(pud_lockptr(vma->vm_mm, pudp)); +#endif + changed = !pud_same(*(pudp), entry); + if (changed) { + /* + * We can use MMU_PAGE_2M here, because only radix + * path look at the psize. + */ + __ptep_set_access_flags(vma, pudp_ptep(pudp), + pud_pte(entry), address, MMU_PAGE_1G); + } + return changed; +} + + int pmdp_test_and_clear_young(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp) { return __pmdp_test_and_clear_young(vma->vm_mm, address, pmdp); } + +int pudp_test_and_clear_young(struct vm_area_struct *vma, + unsigned long address, pud_t *pudp) +{ + return __pudp_test_and_clear_young(vma->vm_mm, address, pudp); +} + /* * set a new huge pmd. We should not be called for updating * an existing pmd entry. That should go via pmd_hugepage_update. @@ -90,6 +118,23 @@ void set_pmd_at(struct mm_struct *mm, unsigned long addr, return set_pte_at(mm, addr, pmdp_ptep(pmdp), pmd_pte(pmd)); } +void set_pud_at(struct mm_struct *mm, unsigned long addr, + pud_t *pudp, pud_t pud) +{ +#ifdef CONFIG_DEBUG_VM + /* + * Make sure hardware valid bit is not set. We don't do + * tlb flush for this update. + */ + + WARN_ON(pte_hw_valid(pud_pte(*pudp))); + assert_spin_locked(pud_lockptr(mm, pudp)); + WARN_ON(!(pud_large(pud))); +#endif + trace_hugepage_set_pud(addr, pud_val(pud)); + return set_pte_at(mm, addr, pudp_ptep(pudp), pud_pte(pud)); +} + static void do_serialize(void *arg) { /* We've taken the IPI, so try to trim the mask while here */ @@ -147,11 +192,35 @@ pmd_t pmdp_huge_get_and_clear_full(struct vm_area_struct *vma, return pmd; } +pud_t pudp_huge_get_and_clear_full(struct vm_area_struct *vma, + unsigned long addr, pud_t *pudp, int full) +{ + pud_t pud; + + VM_BUG_ON(addr & ~HPAGE_PMD_MASK); + VM_BUG_ON((pud_present(*pudp) && !pud_devmap(*pudp)) || + !pud_present(*pudp)); + pud = pudp_huge_get_and_clear(vma->vm_mm, addr, pudp); + /* + * if it not a fullmm flush, then we can possibly end up converting + * this PMD pte entry to a regular level 0 PTE by a parallel page fault. + * Make sure we flush the tlb in this case. + */ + if (!full) + flush_pud_tlb_range(vma, addr, addr + HPAGE_PUD_SIZE); + return pud; +} + static pmd_t pmd_set_protbits(pmd_t pmd, pgprot_t pgprot) { return __pmd(pmd_val(pmd) | pgprot_val(pgprot)); } +static pud_t pud_set_protbits(pud_t pud, pgprot_t pgprot) +{ + return __pud(pud_val(pud) | pgprot_val(pgprot)); +} + /* * At some point we should be able to get rid of * pmd_mkhuge() and mk_huge_pmd() when we update all the @@ -166,6 +235,15 @@ pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot) return __pmd_mkhuge(pmd_set_protbits(__pmd(pmdv), pgprot)); } +pud_t pfn_pud(unsigned long pfn, pgprot_t pgprot) +{ + unsigned long pudv; + + pudv = (pfn << PAGE_SHIFT) & PTE_RPN_MASK; + + return __pud_mkhuge(pud_set_protbits(__pud(pudv), pgprot)); +} + pmd_t mk_pmd(struct page *page, pgprot_t pgprot) { return pfn_pmd(page_to_pfn(page), pgprot); diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c index 76f6a1f3b9d8..d7e2dd3d4add 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -965,6 +965,23 @@ unsigned long radix__pmd_hugepage_update(struct mm_struct *mm, unsigned long add return old; } +unsigned long radix__pud_hugepage_update(struct mm_struct *mm, unsigned long addr, + pud_t *pudp, unsigned long clr, + unsigned long set) +{ + unsigned long old; + +#ifdef CONFIG_DEBUG_VM + WARN_ON(!pud_devmap(*pudp)); + assert_spin_locked(pud_lockptr(mm, pudp)); +#endif + + old = radix__pte_update(mm, addr, pudp_ptep(pudp), clr, set, 1); + trace_hugepage_update(addr, old, clr, set); + + return old; +} + pmd_t radix__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp) @@ -1041,6 +1058,17 @@ pmd_t radix__pmdp_huge_get_and_clear(struct mm_struct *mm, return old_pmd; } +pud_t radix__pudp_huge_get_and_clear(struct mm_struct *mm, + unsigned long addr, pud_t *pudp) +{ + pud_t old_pud; + unsigned long old; + + old = radix__pud_hugepage_update(mm, addr, pudp, ~0UL, 0); + old_pud = __pud(old); + return old_pud; +} + #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ void radix__ptep_set_access_flags(struct vm_area_struct *vma, pte_t *ptep, diff --git a/arch/powerpc/mm/book3s64/radix_tlb.c b/arch/powerpc/mm/book3s64/radix_tlb.c index ce804b7bf84e..a18f7d2c9f63 100644 --- a/arch/powerpc/mm/book3s64/radix_tlb.c +++ b/arch/powerpc/mm/book3s64/radix_tlb.c @@ -1453,6 +1453,13 @@ void radix__flush_pmd_tlb_range(struct vm_area_struct *vma, } EXPORT_SYMBOL(radix__flush_pmd_tlb_range); +void radix__flush_pud_tlb_range(struct vm_area_struct *vma, + unsigned long start, unsigned long end) +{ + radix__flush_tlb_range_psize(vma->vm_mm, start, end, MMU_PAGE_1G); +} +EXPORT_SYMBOL(radix__flush_pud_tlb_range); + void radix__flush_tlb_all(void) { unsigned long rb,prs,r,rs; diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype index 45fd975ef521..340b86ef7284 100644 --- a/arch/powerpc/platforms/Kconfig.cputype +++ b/arch/powerpc/platforms/Kconfig.cputype @@ -94,6 +94,7 @@ config PPC_BOOK3S_64 select PPC_FPU select PPC_HAVE_PMU_SUPPORT select HAVE_ARCH_TRANSPARENT_HUGEPAGE + select HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD select ARCH_ENABLE_HUGEPAGE_MIGRATION if HUGETLB_PAGE && MIGRATION select ARCH_ENABLE_SPLIT_PMD_PTLOCK select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE diff --git a/include/trace/events/thp.h b/include/trace/events/thp.h index 202b3e3e67ff..a919d943d106 100644 --- a/include/trace/events/thp.h +++ b/include/trace/events/thp.h @@ -25,6 +25,23 @@ TRACE_EVENT(hugepage_set_pmd, TP_printk("Set pmd with 0x%lx with 0x%lx", __entry->addr, __entry->pmd) ); +TRACE_EVENT(hugepage_set_pud, + + TP_PROTO(unsigned long addr, unsigned long pud), + TP_ARGS(addr, pud), + TP_STRUCT__entry( + __field(unsigned long, addr) + __field(unsigned long, pud) + ), + + TP_fast_assign( + __entry->addr = addr; + __entry->pud = pud; + ), + + TP_printk("Set pud with 0x%lx with 0x%lx", __entry->addr, __entry->pud) + ); + TRACE_EVENT(hugepage_update, From patchwork Fri Jun 16 11:08:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C1D3EB64D8 for ; Fri, 16 Jun 2023 11:10:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C6AF96B007B; Fri, 16 Jun 2023 07:10:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C1AA46B0081; Fri, 16 Jun 2023 07:10:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A94F28E0001; Fri, 16 Jun 2023 07:10:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 97A326B007B for ; Fri, 16 Jun 2023 07:10:35 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 6A1A3120C89 for ; Fri, 16 Jun 2023 11:10:35 +0000 (UTC) X-FDA: 80908342830.02.65187F8 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf19.hostedemail.com (Postfix) with ESMTP id 0BD6E1A0002 for ; Fri, 16 Jun 2023 11:10:32 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=D76h72uJ; spf=pass (imf19.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686913833; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6Jl6wriXJWVZUja7X5z0rce8AeFw3lmLCLsgACezTAQ=; b=sDiLIJbHkSSG+iLvLDdvkLRsO0Zgb6FyGfbR3Z3EvT24Kbjb57HxlHMOnjEYkgj7b/TkAl JHjMIuqbfHnU/Z0ryC/SVCDPyUb1nQ/8KCJYsThzU7tYvF+WqaifKBqZR/4TVQgW07imdu 19EGzhoGbGW7lktsrATvFG5xhDIDDMc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686913833; a=rsa-sha256; cv=none; b=MYU8QNmCoSGENIlb2dnDMshvyi9fL+r8+OEOatj7xGVtvcPi2NKmNxM6QuEoREYEVqE6wA kff26Bw26DPQTiQQUwoRHGPDKriSCa+mlacSvPTy/h3Qtkyn9JzdH3VwARkQ8gfLawLhPh b+wpKZ96kqhU44FHqVRvm/nXK0k8rqE= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=D76h72uJ; spf=pass (imf19.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com Received: from pps.filterd (m0353726.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAkrmO002627; Fri, 16 Jun 2023 11:10:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=6Jl6wriXJWVZUja7X5z0rce8AeFw3lmLCLsgACezTAQ=; b=D76h72uJrLLSmeAnThVzJ8wlYgIF3QKa66qwexHQawXVKixbpOaCHEmddxpVxRtyul82 tpE3aSS82QlhpZCHDqySw9hTg/bUo/6phNeSDU0u635h/LkN49LqigrU1YfWBYMbCQKw reFGrAZXCMKqtIdFZoY3D1/c3WckC8bjqDV5nZxTdAX7+JcdGuM85VxhO/sehPmQDfdx VdMXtXKGdG7dNufYkUK8Gwp6Mr3jGsBJ5PCjUL+GSBwir7/q9Lb6cQOQ7HRCxo1NmSX+ ijeA3u5IDx95B6iNO4IjZPwcTOZwfcbg0Sel9++X+DnnAu6CAcwz1qd/BC0w2fZ3OfFL FQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9prj09-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:10:20 +0000 Received: from m0353726.ppops.net (m0353726.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GAqPGa018636; Fri, 16 Jun 2023 11:10:19 GMT Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9prhy6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:10:19 +0000 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35G9lNgL027199; Fri, 16 Jun 2023 11:10:18 GMT Received: from smtprelay05.wdc07v.mail.ibm.com ([9.208.129.117]) by ppma02dal.us.ibm.com (PPS) with ESMTPS id 3r4gt6gy5t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:10:17 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay05.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GBAG136423106 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:10:16 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 88D435805B; Fri, 16 Jun 2023 11:10:16 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9723858059; Fri, 16 Jun 2023 11:10:10 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:10:10 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 14/16] powerpc/book3s64/vmemmap: Switch radix to use a different vmemmap handling function Date: Fri, 16 Jun 2023 16:38:24 +0530 Message-Id: <20230616110826.344417-15-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: Lkado8UONI6fm9TNSghPSkwV_A8dBuOg X-Proofpoint-GUID: xPOsUBAXPrrjF3qncdH_B3fXZxk4T3mj X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_06,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 suspectscore=0 bulkscore=0 clxscore=1015 spamscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 lowpriorityscore=0 priorityscore=1501 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160099 X-Rspamd-Queue-Id: 0BD6E1A0002 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: mmgj78ghph5fkcdt8nqwcrazdwt7xeyo X-HE-Tag: 1686913832-62357 X-HE-Meta: U2FsdGVkX19MLrimDutwdo5H1aDMZLVQ6qaJQNRvIu/yhEUqIArqBtpPTwz0iMOk72/vkbGJ6rNrgPsnqEGnUMvz1JcqhWMZB1UBnVLuVPiteQgw8rT8zLzR+F0Z5KNp1d2ZjAOBh4F1iIV8+dM9QAqMOE03QvR8Dg30Fbzp1cT/I49PoPTVErVDNT0cVSlwDg1Qe1WyQo8jGlh7kf3LyctWglgpHOj6FU5l84dhceNhbDJlAry44A2UyNp1RT66S7DGNYDdfhg6+vGhmiQlkS9/NFYOh7jXIRoAyLHh3LpnUwvjyUNfIevB1SFzxonsvIQdeB6rzyrX9CIxSEWnUH5b61e+6QAyr+674Pc9XwIkw+NgF+30smCW23dy/OSHrbMESFEexPknOV7k13Sjhkg/2xLw0kPLvOGRyTWpbncnF6Kmrqq8V1y+081beX856h3Wobs/6t62zeXW/s7SXOzt3CBl9Nqm5uJtqoXXgmPi6YJ8R9j+6XbDOfb3fnlOM2hRzSbWB32dMEHaqKbix1lWe3T3f3pc00mINc6pXK186W1pVZEuKaD/HOAIxVdUE14kZgzBG0A+hLl2M0eOE7ednghFFzqwR4+KF4rNtbnmHaLv3hQ2pKs2jT3IPHImVx40QykCKdww6uqMfQvSBJ+9Wt3DYCX5OcFkxZ0u02ZomeMqOfvoc1CTvJYJBQGpW8CWWVrS6koeoLo33yDCvx/yMNxDkN/8N5oVfL1POCxcv5JIggUVRQyqhf1bvFQErIwEPLCdMG4hoCaCXNX5eJmvw+mEkFD51AKbXcwPME5K9Qr2g2JibiuiKBALK9TvKfdz4Gobaj/Ad6trHyBv74b4ONucYSNdaSlmyF6PEb8pT03zNqIZfH+oUEFfYg3Pk+nyEYt8yR9R/TbJEJoof9mAnWJIUl59lFomx986aU7GnH/eNOCFVK1TY8R2TUh+FeloF6Hla6QkDTWIepD 1tH/N2rX 0+g2s78UXloz6TENsq2XjoelVlyUtXUB7pluFUPZzAAiAo5UhcYeW5p3nPuXxwnepLAiNE/Gkcyizp9+t7zyS4zVYyCMmDptS1LVfsKJmifrD3rhwz4O8Emi7xBMGahlqA9AqCnUJsyu/7vY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is in preparation to update radix to implement vmemmap optimization for devdax. Below are the rules w.r.t radix vmemmap mapping 1. First try to map things using PMD (2M) 2. With altmap if altmap cross-boundary check returns true, fall back to PAGE_SIZE 3. If we can't allocate PMD_SIZE backing memory for vmemmap, fallback to PAGE_SIZE On removing vmemmap mapping, check if every subsection that is using the vmemmap area is invalid. If found to be invalid, that implies we can safely free the vmemmap area. We don't use the PAGE_UNUSED pattern used by x86 because with 64K page size, we need to do the above check even at the PAGE_SIZE granularity. Signed-off-by: Aneesh Kumar K.V --- arch/powerpc/include/asm/book3s/64/radix.h | 2 + arch/powerpc/include/asm/pgtable.h | 3 + arch/powerpc/mm/book3s64/radix_pgtable.c | 319 +++++++++++++++++++-- arch/powerpc/mm/init_64.c | 26 +- 4 files changed, 319 insertions(+), 31 deletions(-) diff --git a/arch/powerpc/include/asm/book3s/64/radix.h b/arch/powerpc/include/asm/book3s/64/radix.h index 8cdff5a05011..87d4c1e62491 100644 --- a/arch/powerpc/include/asm/book3s/64/radix.h +++ b/arch/powerpc/include/asm/book3s/64/radix.h @@ -332,6 +332,8 @@ extern int __meminit radix__vmemmap_create_mapping(unsigned long start, unsigned long phys); int __meminit radix__vmemmap_populate(unsigned long start, unsigned long end, int node, struct vmem_altmap *altmap); +void __ref radix__vmemmap_free(unsigned long start, unsigned long end, + struct vmem_altmap *altmap); extern void radix__vmemmap_remove_mapping(unsigned long start, unsigned long page_size); diff --git a/arch/powerpc/include/asm/pgtable.h b/arch/powerpc/include/asm/pgtable.h index 9972626ddaf6..6d4cd2ebae6e 100644 --- a/arch/powerpc/include/asm/pgtable.h +++ b/arch/powerpc/include/asm/pgtable.h @@ -168,6 +168,9 @@ static inline bool is_ioremap_addr(const void *x) struct seq_file; void arch_report_meminfo(struct seq_file *m); +int __meminit vmemmap_populated(unsigned long vmemmap_addr, int vmemmap_map_size); +bool altmap_cross_boundary(struct vmem_altmap *altmap, unsigned long start, + unsigned long page_size); #endif /* CONFIG_PPC64 */ #endif /* __ASSEMBLY__ */ diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c index d7e2dd3d4add..ef886fab643d 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -742,8 +742,57 @@ static void free_pud_table(pud_t *pud_start, p4d_t *p4d) p4d_clear(p4d); } +static bool __meminit vmemmap_pmd_is_unused(unsigned long addr, unsigned long end) +{ + unsigned long start = ALIGN_DOWN(addr, PMD_SIZE); + + return !vmemmap_populated(start, PMD_SIZE); +} + +static bool __meminit vmemmap_page_is_unused(unsigned long addr, unsigned long end) +{ + unsigned long start = ALIGN_DOWN(addr, PAGE_SIZE); + + return !vmemmap_populated(start, PAGE_SIZE); + +} + +static void __meminit free_vmemmap_pages(struct page *page, + struct vmem_altmap *altmap, + int order) +{ + unsigned int nr_pages = 1 << order; + + if (altmap) { + unsigned long alt_start, alt_end; + unsigned long base_pfn = page_to_pfn(page); + + /* + * with 1G vmemmap mmaping we can have things setup + * such that even though atlmap is specified we never + * used altmap. + */ + alt_start = altmap->base_pfn; + alt_end = altmap->base_pfn + altmap->reserve + + altmap->free + altmap->alloc + altmap->align; + + if (base_pfn >= alt_start && base_pfn < alt_end) { + vmem_altmap_free(altmap, nr_pages); + return; + } + } + + if (PageReserved(page)) { + /* allocated from memblock */ + while (nr_pages--) + free_reserved_page(page++); + } else + free_pages((unsigned long)page_address(page), order); +} + static void remove_pte_table(pte_t *pte_start, unsigned long addr, - unsigned long end, bool direct) + unsigned long end, bool direct, + struct vmem_altmap *altmap) { unsigned long next, pages = 0; pte_t *pte; @@ -757,24 +806,23 @@ static void remove_pte_table(pte_t *pte_start, unsigned long addr, if (!pte_present(*pte)) continue; - if (!PAGE_ALIGNED(addr) || !PAGE_ALIGNED(next)) { - /* - * The vmemmap_free() and remove_section_mapping() - * codepaths call us with aligned addresses. - */ - WARN_ONCE(1, "%s: unaligned range\n", __func__); - continue; + if (PAGE_ALIGNED(addr) && PAGE_ALIGNED(next)) { + if (!direct) + free_vmemmap_pages(pte_page(*pte), altmap, 0); + pte_clear(&init_mm, addr, pte); + pages++; + } else if (!direct && vmemmap_page_is_unused(addr, next)) { + free_vmemmap_pages(pte_page(*pte), altmap, 0); + pte_clear(&init_mm, addr, pte); } - - pte_clear(&init_mm, addr, pte); - pages++; } if (direct) update_page_count(mmu_virtual_psize, -pages); } static void __meminit remove_pmd_table(pmd_t *pmd_start, unsigned long addr, - unsigned long end, bool direct) + unsigned long end, bool direct, + struct vmem_altmap *altmap) { unsigned long next, pages = 0; pte_t *pte_base; @@ -788,18 +836,21 @@ static void __meminit remove_pmd_table(pmd_t *pmd_start, unsigned long addr, continue; if (pmd_is_leaf(*pmd)) { - if (!IS_ALIGNED(addr, PMD_SIZE) || - !IS_ALIGNED(next, PMD_SIZE)) { - WARN_ONCE(1, "%s: unaligned range\n", __func__); - continue; + if (IS_ALIGNED(addr, PMD_SIZE) && + IS_ALIGNED(next, PMD_SIZE)) { + if (!direct) + free_vmemmap_pages(pmd_page(*pmd), altmap, get_order(PMD_SIZE)); + pte_clear(&init_mm, addr, (pte_t *)pmd); + pages++; + } else if (vmemmap_pmd_is_unused(addr, next)) { + free_vmemmap_pages(pmd_page(*pmd), altmap, get_order(PMD_SIZE)); + pte_clear(&init_mm, addr, (pte_t *)pmd); } - pte_clear(&init_mm, addr, (pte_t *)pmd); - pages++; continue; } pte_base = (pte_t *)pmd_page_vaddr(*pmd); - remove_pte_table(pte_base, addr, next, direct); + remove_pte_table(pte_base, addr, next, direct, altmap); free_pte_table(pte_base, pmd); } if (direct) @@ -807,7 +858,8 @@ static void __meminit remove_pmd_table(pmd_t *pmd_start, unsigned long addr, } static void __meminit remove_pud_table(pud_t *pud_start, unsigned long addr, - unsigned long end, bool direct) + unsigned long end, bool direct, + struct vmem_altmap *altmap) { unsigned long next, pages = 0; pmd_t *pmd_base; @@ -832,15 +884,16 @@ static void __meminit remove_pud_table(pud_t *pud_start, unsigned long addr, } pmd_base = pud_pgtable(*pud); - remove_pmd_table(pmd_base, addr, next, direct); + remove_pmd_table(pmd_base, addr, next, direct, altmap); free_pmd_table(pmd_base, pud); } if (direct) update_page_count(MMU_PAGE_1G, -pages); } -static void __meminit remove_pagetable(unsigned long start, unsigned long end, - bool direct) +static void __meminit +remove_pagetable(unsigned long start, unsigned long end, bool direct, + struct vmem_altmap *altmap) { unsigned long addr, next; pud_t *pud_base; @@ -869,7 +922,7 @@ static void __meminit remove_pagetable(unsigned long start, unsigned long end, } pud_base = p4d_pgtable(*p4d); - remove_pud_table(pud_base, addr, next, direct); + remove_pud_table(pud_base, addr, next, direct, altmap); free_pud_table(pud_base, p4d); } @@ -892,7 +945,7 @@ int __meminit radix__create_section_mapping(unsigned long start, int __meminit radix__remove_section_mapping(unsigned long start, unsigned long end) { - remove_pagetable(start, end, true); + remove_pagetable(start, end, true, NULL); return 0; } #endif /* CONFIG_MEMORY_HOTPLUG */ @@ -924,10 +977,224 @@ int __meminit radix__vmemmap_create_mapping(unsigned long start, return 0; } +int __meminit vmemmap_check_pmd(pmd_t *pmd, int node, + unsigned long addr, unsigned long next) +{ + int large = pmd_large(*pmd); + + if (pmd_large(*pmd)) + vmemmap_verify((pte_t *)pmd, node, addr, next); + + return large; +} + +void __meminit vmemmap_set_pmd(pmd_t *pmdp, void *p, int node, + unsigned long addr, unsigned long next) +{ + pte_t entry; + pte_t *ptep = pmdp_ptep(pmdp); + + VM_BUG_ON(!IS_ALIGNED(addr, PMD_SIZE)); + entry = pfn_pte(__pa(p) >> PAGE_SHIFT, PAGE_KERNEL); + set_pte_at(&init_mm, addr, ptep, entry); + asm volatile("ptesync": : :"memory"); + + vmemmap_verify(ptep, node, addr, next); +} + +static pte_t * __meminit radix__vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node, + struct vmem_altmap *altmap, + struct page *reuse) +{ + pte_t *pte = pte_offset_kernel(pmd, addr); + + if (pte_none(*pte)) { + pte_t entry; + void *p; + + if (!reuse) { + /* + * make sure we don't create altmap mappings + * covering things outside the device. + */ + if (altmap && altmap_cross_boundary(altmap, addr, PAGE_SIZE)) + altmap = NULL; + + p = vmemmap_alloc_block_buf(PAGE_SIZE, node, altmap); + if (!p) { + if (altmap) + p = vmemmap_alloc_block_buf(PAGE_SIZE, node, NULL); + if (!p) + return NULL; + } + } else { + /* + * When a PTE/PMD entry is freed from the init_mm + * there's a free_pages() call to this page allocated + * above. Thus this get_page() is paired with the + * put_page_testzero() on the freeing path. + * This can only called by certain ZONE_DEVICE path, + * and through vmemmap_populate_compound_pages() when + * slab is available. + */ + get_page(reuse); + p = page_to_virt(reuse); + } + + VM_BUG_ON(!PAGE_ALIGNED(addr)); + entry = pfn_pte(__pa(p) >> PAGE_SHIFT, PAGE_KERNEL); + set_pte_at(&init_mm, addr, pte, entry); + asm volatile("ptesync": : :"memory"); + } + return pte; +} + +static inline pud_t *vmemmap_pud_alloc(p4d_t *p4d, int node, + unsigned long address) +{ + pud_t *pud; + + /* All early vmemmap mapping to keep simple do it at PAGE_SIZE */ + if (unlikely(p4d_none(*p4d))) { + if (unlikely(!slab_is_available())) { + pud = early_alloc_pgtable(PAGE_SIZE, node, 0, 0); + p4d_populate(&init_mm, p4d, pud); + /* go to the pud_offset */ + } else + return pud_alloc(&init_mm, p4d, address); + } + return pud_offset(p4d, address); +} + +static inline pmd_t *vmemmap_pmd_alloc(pud_t *pud, int node, + unsigned long address) +{ + pmd_t *pmd; + + /* All early vmemmap mapping to keep simple do it at PAGE_SIZE */ + if (unlikely(pud_none(*pud))) { + if (unlikely(!slab_is_available())) { + pmd = early_alloc_pgtable(PAGE_SIZE, node, 0, 0); + pud_populate(&init_mm, pud, pmd); + } else + return pmd_alloc(&init_mm, pud, address); + } + return pmd_offset(pud, address); +} + +static inline pte_t *vmemmap_pte_alloc(pmd_t *pmd, int node, + unsigned long address) +{ + pte_t *pte; + + /* All early vmemmap mapping to keep simple do it at PAGE_SIZE */ + if (unlikely(pmd_none(*pmd))) { + if (unlikely(!slab_is_available())) { + pte = early_alloc_pgtable(PAGE_SIZE, node, 0, 0); + pmd_populate(&init_mm, pmd, pte); + } else + return pte_alloc_kernel(pmd, address); + } + return pte_offset_kernel(pmd, address); +} + + + +int __meminit radix__vmemmap_populate(unsigned long start, unsigned long end, int node, + struct vmem_altmap *altmap) +{ + unsigned long addr; + unsigned long next; + pgd_t *pgd; + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + pte_t *pte; + + for (addr = start; addr < end; addr = next) { + next = pmd_addr_end(addr, end); + + pgd = pgd_offset_k(addr); + p4d = p4d_offset(pgd, addr); + pud = vmemmap_pud_alloc(p4d, node, addr); + if (!pud) + return -ENOMEM; + pmd = vmemmap_pmd_alloc(pud, node, addr); + if (!pmd) + return -ENOMEM; + + if (pmd_none(READ_ONCE(*pmd))) { + void *p; + + /* + * keep it simple by checking addr PMD_SIZE alignment + * and verifying the device boundary condition. + * For us to use a pmd mapping, both addr and pfn should + * be aligned. We skip if addr is not aligned and for + * pfn we hope we have extra area in the altmap that + * can help to find an aligned block. This can result + * in altmap block allocation failures, in which case + * we fallback to RAM for vmemmap allocation. + */ + if (altmap && (!IS_ALIGNED(addr, PMD_SIZE) || + altmap_cross_boundary(altmap, addr, PMD_SIZE))) { + /* + * make sure we don't create altmap mappings + * covering things outside the device. + */ + goto base_mapping; + } + + p = vmemmap_alloc_block_buf(PMD_SIZE, node, altmap); + if (p) { + vmemmap_set_pmd(pmd, p, node, addr, next); + continue; + } else if (altmap) { + /* + * A vmemmap block allocation can fail due to + * alignment requirements and we trying to align + * things aggressively there by running out of + * space. Try base mapping on failure. + */ + goto base_mapping; + } + } else if (vmemmap_check_pmd(pmd, node, addr, next)) { + /* + * If a huge mapping exist due to early call to + * vmemmap_populate, let's try to use that. + */ + continue; + } +base_mapping: + /* + * Not able allocate higher order memory to back memmap + * or we found a pointer to pte page. Allocate base page + * size vmemmap + */ + pte = vmemmap_pte_alloc(pmd, node, addr); + if (!pte) + return -ENOMEM; + + pte = radix__vmemmap_pte_populate(pmd, addr, node, altmap, NULL); + if (!pte) + return -ENOMEM; + + vmemmap_verify(pte, node, addr, addr + PAGE_SIZE); + next = addr + PAGE_SIZE; + } + return 0; +} + #ifdef CONFIG_MEMORY_HOTPLUG void __meminit radix__vmemmap_remove_mapping(unsigned long start, unsigned long page_size) { - remove_pagetable(start, start + page_size, false); + remove_pagetable(start, start + page_size, true, NULL); +} + +void __ref radix__vmemmap_free(unsigned long start, unsigned long end, + struct vmem_altmap *altmap) +{ + remove_pagetable(start, end, false, altmap); } #endif #endif diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c index fe1b83020e0d..5701faca39ef 100644 --- a/arch/powerpc/mm/init_64.c +++ b/arch/powerpc/mm/init_64.c @@ -92,7 +92,7 @@ static struct page * __meminit vmemmap_subsection_start(unsigned long vmemmap_ad * a page table lookup here because with the hash translation we don't keep * vmemmap details in linux page table. */ -static int __meminit vmemmap_populated(unsigned long vmemmap_addr, int vmemmap_map_size) +int __meminit vmemmap_populated(unsigned long vmemmap_addr, int vmemmap_map_size) { struct page *start; unsigned long vmemmap_end = vmemmap_addr + vmemmap_map_size; @@ -183,8 +183,8 @@ static __meminit int vmemmap_list_populate(unsigned long phys, return 0; } -static bool altmap_cross_boundary(struct vmem_altmap *altmap, unsigned long start, - unsigned long page_size) +bool altmap_cross_boundary(struct vmem_altmap *altmap, unsigned long start, + unsigned long page_size) { unsigned long nr_pfn = page_size / sizeof(struct page); unsigned long start_pfn = page_to_pfn((struct page *)start); @@ -204,6 +204,11 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, bool altmap_alloc; unsigned long page_size = 1 << mmu_psize_defs[mmu_vmemmap_psize].shift; +#ifdef CONFIG_PPC_BOOK3S_64 + if (radix_enabled()) + return radix__vmemmap_populate(start, end, node, altmap); +#endif + /* Align to the page size of the linear mapping. */ start = ALIGN_DOWN(start, page_size); @@ -303,8 +308,8 @@ static unsigned long vmemmap_list_free(unsigned long start) return vmem_back->phys; } -void __ref vmemmap_free(unsigned long start, unsigned long end, - struct vmem_altmap *altmap) +void __ref __vmemmap_free(unsigned long start, unsigned long end, + struct vmem_altmap *altmap) { unsigned long page_size = 1 << mmu_psize_defs[mmu_vmemmap_psize].shift; unsigned long page_order = get_order(page_size); @@ -362,6 +367,17 @@ void __ref vmemmap_free(unsigned long start, unsigned long end, vmemmap_remove_mapping(start, page_size); } } + +void __ref vmemmap_free(unsigned long start, unsigned long end, + struct vmem_altmap *altmap) +{ +#ifdef CONFIG_PPC_BOOK3S_64 + if (radix_enabled()) + return radix__vmemmap_free(start, end, altmap); +#endif + return __vmemmap_free(start, end, altmap); +} + #endif void register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page, unsigned long size) From patchwork Fri Jun 16 11:08:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282634 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A05BAEB64D8 for ; Fri, 16 Jun 2023 11:10:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3B8DF6B0081; Fri, 16 Jun 2023 07:10:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3681A6B0082; Fri, 16 Jun 2023 07:10:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E26F6B0083; Fri, 16 Jun 2023 07:10:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0C8F06B0081 for ; Fri, 16 Jun 2023 07:10:40 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id D2BEC120C75 for ; Fri, 16 Jun 2023 11:10:39 +0000 (UTC) X-FDA: 80908342998.15.DFE27B7 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf20.hostedemail.com (Postfix) with ESMTP id C9D061C001F for ; Fri, 16 Jun 2023 11:10:36 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=H8PCBWz1; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf20.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686913837; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WEhl9BlLPzmq+efvwpqyCzmDovFQwSils+63Z9z9AmY=; b=ajFS+GN8Hf0iDoo5k93gJzKIjvFYIqAfdjpN+hjtnnuBOM9iR/2kfUXupLoo2p5+2Q8USx MUM1RbXXNezvtJ3ln0F9VZiFo8XAlkK/TKY7An8kzGLlawJ77OAChbl7gLEJ3btfHIiIKG MKlSsMzisaBkD+y2YuwBQNaKE2WHlIQ= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=H8PCBWz1; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf20.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686913837; a=rsa-sha256; cv=none; b=UQpxbqkmZEia8hOrVlT1NId9GmEHnSZw1mQkxOiRnn3dkF/emiwxv4t/UvMZ1yEtVjB1s8 7CnZ+opoEub723vjR7vlfkAMOuo+UH0vHLNWQKNBY2V25fUm49xVEg97s8sueaDl3rydXf 0daRi0s7hqwnVBd5GVTL/GKKDA97Tpg= Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GB89Qj015550; Fri, 16 Jun 2023 11:10:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=WEhl9BlLPzmq+efvwpqyCzmDovFQwSils+63Z9z9AmY=; b=H8PCBWz1MuXGcJCfl/P2EpbD5PA960goxF6h+WOLDqBRL1RRp4a/Fkyu9zEOQdnDHt9V QVevji5LoC8O2Pp4E+25Rgu+oDAbR/ew7khkfiFp7ZVQ5QeNK+Vj85Mc55oS2pWdzk2p TUYxszk6ihCFNYJvFDHE3esbNUoP+Q9J45mvL/alzCo4phHjuBurVB/pKIlSOw2cTjBf H+8ANCfp5graGMESMRccrm3z5G0eEGBQgp+AEkls4x+AxnYnPVBQg53SpSH9Nu6p3jw1 vmZyBsRZNDg8rVl2bHba4aoB1dzY6BjOO8ZJlcgODXdmo6Hx5tA7Jy6nCFZO3ZTchfN9 ew== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9m8b00-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:10:26 +0000 Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GB9WPo020221; Fri, 16 Jun 2023 11:10:25 GMT Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9m8aym-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:10:25 +0000 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35G8vEu8012044; Fri, 16 Jun 2023 11:10:24 GMT Received: from smtprelay01.wdc07v.mail.ibm.com ([9.208.129.119]) by ppma01dal.us.ibm.com (PPS) with ESMTPS id 3r4gt6s37x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:10:24 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay01.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GBANhu36242142 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:10:23 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 25ABB5805D; Fri, 16 Jun 2023 11:10:23 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5C9E258059; Fri, 16 Jun 2023 11:10:17 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:10:16 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 15/16] powerpc/book3s64/radix: Add support for vmemmap optimization for radix Date: Fri, 16 Jun 2023 16:38:25 +0530 Message-Id: <20230616110826.344417-16-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: xJJVcIDsEMLH2maCfqpkSczvf5cJZrdd X-Proofpoint-ORIG-GUID: ZObC27QUMOBmiQVGplfYRc_seUr5PVGZ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_07,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 mlxscore=0 adultscore=0 lowpriorityscore=0 clxscore=1015 mlxlogscore=999 bulkscore=0 impostorscore=0 priorityscore=1501 suspectscore=0 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160099 X-Rspamd-Queue-Id: C9D061C001F X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: ba1tjxtbmjusbqfw98a757jfjzqz81ra X-HE-Tag: 1686913836-391876 X-HE-Meta: U2FsdGVkX1+aeW9wyw7dfx6SzB5oDm7SizBL3WuEk2gmPwT1BI7V233IUSWGOw51Xcn7LZ7eQ544bY+yk1549iu9l1oiEXi5WdCUvb1lU56lUt3bHnEj3Wi+85oLsjh5qrASoNAfWe4HLivHOPpbfowLZKZS8peleswyHa1sZKrepCvXj5pR2LrHo56oVyTtMDZ3jEMm+H3fp/lRT5EHk/IMeWAtbauXFijQ1hzLBQYCpil3EeX2luU8o9Uhg0rQvg8ZWszvZDRTpBnuGSc81/E0w9G5RsdQ3+zaWpDh5bsK2eVNC067Hz6BrJdFqL+sDZUEmPA3dV6HseTEUq4GP4aU5R+yn0Dy1ZmviO1eEcXQHyysoxKK9pYn7vsFhPmwcr4AtQWrShsiVQJbx+UD/DA5MV6OL7Y3dFncDHm3wO30nfQoJIo4N7meod4RPFr4Gcy/2gsVaid/B5cUgo99AgOd9l1CPlaSjS5izRgQEDO3gY+yVwIDKDwZCJyXnTT/8FqiqN1/pW7ukVyHqLugzxRIeH1vrjiTOdxha3AAy2U9Fu6fKkQ3HcLoNl9Y5Ne+6pCqkHXqZBGmvTQUQjFWVzYQy9GbX3W9XPysshJn7NjgTChmrrRJrQMwXjgEP45zRsn0wq+vnVYhnL6DkKMSA4b6jK4BbRbythztszNq5epXvEYKiB4dBjgMdJfGb79qruAyTA7/Fv1Y8KepbK8BOMNqT7R9jHqlbZ67FzJWrc5Xcsz+FKL6cg78vCYQwbyl6xDc/qMCGQy4w+Y+ggc2oWXxR2eoVoOu+qwPBZahkLLyMOsJW4KsXvk1oZoyUu3LGhKpSvVVjeNEmoBW4Ukh89J5Z1M3mr+hvHlmD4ZLSx9luGOTL38EBVmKUTUAZWSXRjE8Tv14Y7w3TMeLg4gYNNiyv1XVwehELtNlfSuzFE2vcy+qtGO4P1T/mEqn6FSMUpq0D+IlBHdlmmUUjLI ax4FU/Be w1itivEYhhmo8LVORwYetLwB7FMF7KQ2CkahqWdXEVx8uNCc87FAuC96diEPVHT/4N+uTSpUM7d4vLgPYmf4hxhwzjLvRT/hUs+vafrOajaFV4SWXq1D2GW4nyiUHUphPZ01mzRXp2ZnjSVD4sER2jUMzbg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: With 2M PMD-level mapping, we require 32 struct pages and a single vmemmap page can contain 1024 struct pages (PAGE_SIZE/sizeof(struct page)). Hence with 64K page size, we don't use vmemmap deduplication for PMD-level mapping. Signed-off-by: Aneesh Kumar K.V --- Documentation/mm/vmemmap_dedup.rst | 1 + Documentation/powerpc/index.rst | 1 + Documentation/powerpc/vmemmap_dedup.rst | 101 ++++++++++ arch/powerpc/Kconfig | 1 + arch/powerpc/include/asm/book3s/64/radix.h | 8 + arch/powerpc/mm/book3s64/radix_pgtable.c | 203 +++++++++++++++++++++ 6 files changed, 315 insertions(+) create mode 100644 Documentation/powerpc/vmemmap_dedup.rst diff --git a/Documentation/mm/vmemmap_dedup.rst b/Documentation/mm/vmemmap_dedup.rst index a4b12ff906c4..c573e08b5043 100644 --- a/Documentation/mm/vmemmap_dedup.rst +++ b/Documentation/mm/vmemmap_dedup.rst @@ -210,6 +210,7 @@ the device (altmap). The following page sizes are supported in DAX: PAGE_SIZE (4K on x86_64), PMD_SIZE (2M on x86_64) and PUD_SIZE (1G on x86_64). +For powerpc equivalent details see Documentation/powerpc/vmemmap_dedup.rst The differences with HugeTLB are relatively minor. diff --git a/Documentation/powerpc/index.rst b/Documentation/powerpc/index.rst index 85e80e30160b..1b7d943ae8f3 100644 --- a/Documentation/powerpc/index.rst +++ b/Documentation/powerpc/index.rst @@ -35,6 +35,7 @@ powerpc ultravisor vas-api vcpudispatch_stats + vmemmap_dedup features diff --git a/Documentation/powerpc/vmemmap_dedup.rst b/Documentation/powerpc/vmemmap_dedup.rst new file mode 100644 index 000000000000..dc4db59fdf87 --- /dev/null +++ b/Documentation/powerpc/vmemmap_dedup.rst @@ -0,0 +1,101 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========== +Device DAX +========== + +The device-dax interface uses the tail deduplication technique explained in +Documentation/mm/vmemmap_dedup.rst + +On powerpc, vmemmap deduplication is only used with radix MMU translation. Also +with a 64K page size, only the devdax namespace with 1G alignment uses vmemmap +deduplication. + +With 2M PMD level mapping, we require 32 struct pages and a single 64K vmemmap +page can contain 1024 struct pages (64K/sizeof(struct page)). Hence there is no +vmemmap deduplication possible. + +With 1G PUD level mapping, we require 16384 struct pages and a single 64K +vmemmap page can contain 1024 struct pages (64K/sizeof(struct page)). Hence we +require 16 64K pages in vmemmap to map the struct page for 1G PUD level mapping. + +Here's how things look like on device-dax after the sections are populated:: + +-----------+ ---virt_to_page---> +-----------+ mapping to +-----------+ + | | | 0 | -------------> | 0 | + | | +-----------+ +-----------+ + | | | 1 | -------------> | 1 | + | | +-----------+ +-----------+ + | | | 2 | ----------------^ ^ ^ ^ ^ ^ + | | +-----------+ | | | | | + | | | 3 | ------------------+ | | | | + | | +-----------+ | | | | + | | | 4 | --------------------+ | | | + | PUD | +-----------+ | | | + | level | | . | ----------------------+ | | + | mapping | +-----------+ | | + | | | . | ------------------------+ | + | | +-----------+ | + | | | 15 | --------------------------+ + | | +-----------+ + | | + | | + | | + +-----------+ + + +With 4K page size, 2M PMD level mapping requires 512 struct pages and a single +4K vmemmap page contains 64 struct pages(4K/sizeof(struct page)). Hence we +require 8 4K pages in vmemmap to map the struct page for 2M pmd level mapping. + +Here's how things look like on device-dax after the sections are populated:: + + +-----------+ ---virt_to_page---> +-----------+ mapping to +-----------+ + | | | 0 | -------------> | 0 | + | | +-----------+ +-----------+ + | | | 1 | -------------> | 1 | + | | +-----------+ +-----------+ + | | | 2 | ----------------^ ^ ^ ^ ^ ^ + | | +-----------+ | | | | | + | | | 3 | ------------------+ | | | | + | | +-----------+ | | | | + | | | 4 | --------------------+ | | | + | PMD | +-----------+ | | | + | level | | 5 | ----------------------+ | | + | mapping | +-----------+ | | + | | | 6 | ------------------------+ | + | | +-----------+ | + | | | 7 | --------------------------+ + | | +-----------+ + | | + | | + | | + +-----------+ + +With 1G PUD level mapping, we require 262144 struct pages and a single 4K +vmemmap page can contain 64 struct pages (4K/sizeof(struct page)). Hence we +require 4096 4K pages in vmemmap to map the struct pages for 1G PUD level +mapping. + +Here's how things look like on device-dax after the sections are populated:: + + +-----------+ ---virt_to_page---> +-----------+ mapping to +-----------+ + | | | 0 | -------------> | 0 | + | | +-----------+ +-----------+ + | | | 1 | -------------> | 1 | + | | +-----------+ +-----------+ + | | | 2 | ----------------^ ^ ^ ^ ^ ^ + | | +-----------+ | | | | | + | | | 3 | ------------------+ | | | | + | | +-----------+ | | | | + | | | 4 | --------------------+ | | | + | PUD | +-----------+ | | | + | level | | . | ----------------------+ | | + | mapping | +-----------+ | | + | | | . | ------------------------+ | + | | +-----------+ | + | | | 4095 | --------------------------+ + | | +-----------+ + | | + | | + | | + +-----------+ diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index bff5820b7cda..6bd9ca6f2448 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -175,6 +175,7 @@ config PPC select ARCH_WANT_IPC_PARSE_VERSION select ARCH_WANT_IRQS_OFF_ACTIVATE_MM select ARCH_WANT_LD_ORPHAN_WARN + select ARCH_WANT_OPTIMIZE_DAX_VMEMMAP if PPC_RADIX_MMU select ARCH_WANTS_MODULES_DATA_IN_VMALLOC if PPC_BOOK3S_32 || PPC_8xx select ARCH_WEAK_RELEASE_ACQUIRE select BINFMT_ELF diff --git a/arch/powerpc/include/asm/book3s/64/radix.h b/arch/powerpc/include/asm/book3s/64/radix.h index 87d4c1e62491..3195f268ed7f 100644 --- a/arch/powerpc/include/asm/book3s/64/radix.h +++ b/arch/powerpc/include/asm/book3s/64/radix.h @@ -364,5 +364,13 @@ int radix__remove_section_mapping(unsigned long start, unsigned long end); void radix__kernel_map_pages(struct page *page, int numpages, int enable); +#define vmemmap_can_optimize vmemmap_can_optimize +bool vmemmap_can_optimize(struct vmem_altmap *altmap, struct dev_pagemap *pgmap); + +#define vmemmap_populate_compound_pages vmemmap_populate_compound_pages +int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn, + unsigned long start, + unsigned long end, int node, + struct dev_pagemap *pgmap); #endif /* __ASSEMBLY__ */ #endif diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c index ef886fab643d..f0527eebe012 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -977,6 +977,15 @@ int __meminit radix__vmemmap_create_mapping(unsigned long start, return 0; } + +bool vmemmap_can_optimize(struct vmem_altmap *altmap, struct dev_pagemap *pgmap) +{ + if (radix_enabled()) + return __vmemmap_can_optimize(altmap, pgmap); + + return false; +} + int __meminit vmemmap_check_pmd(pmd_t *pmd, int node, unsigned long addr, unsigned long next) { @@ -1185,6 +1194,200 @@ int __meminit radix__vmemmap_populate(unsigned long start, unsigned long end, in return 0; } +static pte_t * __meminit radix__vmemmap_populate_address(unsigned long addr, int node, + struct vmem_altmap *altmap, + struct page *reuse) +{ + pgd_t *pgd; + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + pte_t *pte; + + pgd = pgd_offset_k(addr); + p4d = p4d_offset(pgd, addr); + pud = vmemmap_pud_alloc(p4d, node, addr); + if (!pud) + return NULL; + pmd = vmemmap_pmd_alloc(pud, node, addr); + if (!pmd) + return NULL; + if (pmd_leaf(*pmd)) + /* + * The second page is mapped as a hugepage due to a nearby request. + * Force our mapping to page size without deduplication + */ + return NULL; + pte = vmemmap_pte_alloc(pmd, node, addr); + if (!pte) + return NULL; + radix__vmemmap_pte_populate(pmd, addr, node, NULL, NULL); + vmemmap_verify(pte, node, addr, addr + PAGE_SIZE); + + return pte; +} + +static pte_t * __meminit vmemmap_compound_tail_page(unsigned long addr, + unsigned long pfn_offset, int node) +{ + pgd_t *pgd; + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + pte_t *pte; + unsigned long map_addr; + + /* the second vmemmap page which we use for duplication */ + map_addr = addr - pfn_offset * sizeof(struct page) + PAGE_SIZE; + pgd = pgd_offset_k(map_addr); + p4d = p4d_offset(pgd, map_addr); + pud = vmemmap_pud_alloc(p4d, node, map_addr); + if (!pud) + return NULL; + pmd = vmemmap_pmd_alloc(pud, node, map_addr); + if (!pmd) + return NULL; + if (pmd_leaf(*pmd)) + /* + * The second page is mapped as a hugepage due to a nearby request. + * Force our mapping to page size without deduplication + */ + return NULL; + pte = vmemmap_pte_alloc(pmd, node, map_addr); + if (!pte) + return NULL; + /* + * Check if there exist a mapping to the left + */ + if (pte_none(*pte)) { + /* + * Populate the head page vmemmap page. + * It can fall in different pmd, hence + * vmemmap_populate_address() + */ + pte = radix__vmemmap_populate_address(map_addr - PAGE_SIZE, node, NULL, NULL); + if (!pte) + return NULL; + /* + * Populate the tail pages vmemmap page + */ + pte = radix__vmemmap_pte_populate(pmd, map_addr, node, NULL, NULL); + if (!pte) + return NULL; + vmemmap_verify(pte, node, map_addr, map_addr + PAGE_SIZE); + return pte; + } + return pte; +} + +int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn, + unsigned long start, + unsigned long end, int node, + struct dev_pagemap *pgmap) +{ + /* + * we want to map things as base page size mapping so that + * we can save space in vmemmap. We could have huge mapping + * covering out both edges. + */ + unsigned long addr; + unsigned long addr_pfn = start_pfn; + unsigned long next; + pgd_t *pgd; + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + pte_t *pte; + + for (addr = start; addr < end; addr = next) { + + pgd = pgd_offset_k(addr); + p4d = p4d_offset(pgd, addr); + pud = vmemmap_pud_alloc(p4d, node, addr); + if (!pud) + return -ENOMEM; + pmd = vmemmap_pmd_alloc(pud, node, addr); + if (!pmd) + return -ENOMEM; + + if (pmd_leaf(READ_ONCE(*pmd))) { + /* existing huge mapping. Skip the range */ + addr_pfn += (PMD_SIZE >> PAGE_SHIFT); + next = pmd_addr_end(addr, end); + continue; + } + pte = vmemmap_pte_alloc(pmd, node, addr); + if (!pte) + return -ENOMEM; + if (!pte_none(*pte)) { + /* + * This could be because we already have a compound + * page whose VMEMMAP_RESERVE_NR pages were mapped and + * this request fall in those pages. + */ + addr_pfn += 1; + next = addr + PAGE_SIZE; + continue; + } else { + unsigned long nr_pages = pgmap_vmemmap_nr(pgmap); + unsigned long pfn_offset = addr_pfn - ALIGN_DOWN(addr_pfn, nr_pages); + pte_t *tail_page_pte; + + /* + * if the address is aligned to huge page size it is the + * head mapping. + */ + if (pfn_offset == 0) { + /* Populate the head page vmemmap page */ + pte = radix__vmemmap_pte_populate(pmd, addr, node, NULL, NULL); + if (!pte) + return -ENOMEM; + vmemmap_verify(pte, node, addr, addr + PAGE_SIZE); + + /* + * Populate the tail pages vmemmap page + * It can fall in different pmd, hence + * vmemmap_populate_address() + */ + pte = radix__vmemmap_populate_address(addr + PAGE_SIZE, node, NULL, NULL); + if (!pte) + return -ENOMEM; + + addr_pfn += 2; + next = addr + 2 * PAGE_SIZE; + continue; + } + /* + * get the 2nd mapping details + * Also create it if that doesn't exist + */ + tail_page_pte = vmemmap_compound_tail_page(addr, pfn_offset, node); + if (!tail_page_pte) { + + pte = radix__vmemmap_pte_populate(pmd, addr, node, NULL, NULL); + if (!pte) + return -ENOMEM; + vmemmap_verify(pte, node, addr, addr + PAGE_SIZE); + + addr_pfn += 1; + next = addr + PAGE_SIZE; + continue; + } + + pte = radix__vmemmap_pte_populate(pmd, addr, node, NULL, pte_page(*tail_page_pte)); + if (!pte) + return -ENOMEM; + vmemmap_verify(pte, node, addr, addr + PAGE_SIZE); + + addr_pfn += 1; + next = addr + PAGE_SIZE; + continue; + } + } + return 0; +} + + #ifdef CONFIG_MEMORY_HOTPLUG void __meminit radix__vmemmap_remove_mapping(unsigned long start, unsigned long page_size) { From patchwork Fri Jun 16 11:08:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13282635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16819EB64DA for ; Fri, 16 Jun 2023 11:10:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A96586B0082; Fri, 16 Jun 2023 07:10:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A47AE8E0001; Fri, 16 Jun 2023 07:10:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8E72E6B0085; Fri, 16 Jun 2023 07:10:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 79C286B0082 for ; Fri, 16 Jun 2023 07:10:46 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 544D3160B9A for ; Fri, 16 Jun 2023 11:10:46 +0000 (UTC) X-FDA: 80908343292.28.623477F Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf15.hostedemail.com (Postfix) with ESMTP id DF675A001C for ; Fri, 16 Jun 2023 11:10:43 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=U6yWGXqb; spf=pass (imf15.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686913844; a=rsa-sha256; cv=none; b=5mVFILZshWqSll18DZDWGXNya2l7QgsNOtPGICrrCneMpWzJoXn9tbyVAppyNdi/FADFLr YdkQNoJXZRphiNRvbXdLXXnDZuMMIEvlTlAtoXRJFn2X2rFV1cPb8ECiz61SU2uJgitzur lnRk1gBoD82QbBLNaIawVyADngFeWGE= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=U6yWGXqb; spf=pass (imf15.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686913844; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bXy7ekEYr/1/XXHTn4mE4T1FXjOwT918/Sm/a9WdL+I=; b=kWunxtlI8DcEF36lnSkeoweLyBIK5S/EO9yDuZHpcAQJVLcG4+n50PK8Xa8/xJEa1D/uIM C3anK16Mbugn6SBwVuGQg34ih7ilyDRLgHABOmQ2Wlq8z/CHi7CbkZ8FlJ227zAaqdCQNx ZjQHsPO6eO/JaNWMVezpOG02ZwkBU/8= Received: from pps.filterd (m0353727.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35GAkmR3025344; Fri, 16 Jun 2023 11:10:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=bXy7ekEYr/1/XXHTn4mE4T1FXjOwT918/Sm/a9WdL+I=; b=U6yWGXqbZRjUIJV8uncRffYTjkSu4/gM8Y4u+y0OPal6BSAnuqI7p34WAEeAD3zTzZkC sdL2SCJz9cd20jxSm7Q7j3IVoeiSc0MbBkhQ4jfosqbaLpcZyE8D+S6he34WSjBo2MaG lymBcQCwOqqtk5iqXx5NwEQ930V79im2pcceEjAeujvKgy0IujJ/vHrgvj6/rL6sXHSI EylFtGAF1S2pobV1GguGos9qe6Gnnom5tHKOtZg+Dwek0n0/TiqVtrmrc4G3HmTwTkSE clY7NMgzRYoeLMVXhl7O1trs3dm16XfP7Rar0RXoFeInxspdqP7Lb+mG8ooQC3IBh4g3 aw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9nghj8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:10:33 +0000 Received: from m0353727.ppops.net (m0353727.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35GAmUQI028781; Fri, 16 Jun 2023 11:10:32 GMT Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r8p9nghhg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:10:32 +0000 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35GASRwG014515; Fri, 16 Jun 2023 11:10:31 GMT Received: from smtprelay06.wdc07v.mail.ibm.com ([9.208.129.118]) by ppma02wdc.us.ibm.com (PPS) with ESMTPS id 3r4gt5yk3k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 16 Jun 2023 11:10:31 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay06.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35GBAUrH55050544 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 16 Jun 2023 11:10:30 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EFFEE58058; Fri, 16 Jun 2023 11:10:29 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CCD8A58059; Fri, 16 Jun 2023 11:10:23 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.52.24]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 16 Jun 2023 11:10:23 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Mike Kravetz , Dan Williams , Joao Martins , Catalin Marinas , Muchun Song , Will Deacon , "Aneesh Kumar K.V" Subject: [PATCH v2 16/16] powerpc/book3s64/radix: Remove mmu_vmemmap_psize Date: Fri, 16 Jun 2023 16:38:26 +0530 Message-Id: <20230616110826.344417-17-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> References: <20230616110826.344417-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: titP5Y9RHzu9FpseHNpM75xHP-h2c8Jw X-Proofpoint-ORIG-GUID: H2GG3xTaqP7O8WKpAjaaLvyyiU854HaF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-16_07,2023-06-16_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 clxscore=1015 bulkscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 suspectscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306160099 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: DF675A001C X-Stat-Signature: gpoibmygsxujueyq9oxiaijfkap9ug6a X-Rspam-User: X-HE-Tag: 1686913843-339486 X-HE-Meta: U2FsdGVkX197piTDqVL87Wox0rLuWEihL2dIAOENP92ATmFaG3SmYzslxAfQKCY6SLEk6ZG0ttql7FQYi8HJafVwMn8ZW5kacA13l7oFeJ0HCJJCLYxkis+F5Z58J63BmcN7yVLlquglhqeH33RgafGQwZ7+4jQ0hgvkfd3mdkYNq71K/OAiehAdV/P0aKcaeJECK3Zys/tftUMgM1C01SPGnCXimZzSLDtPh1N2w49+42QkNMreMRxpJDefPzM4izPEn57+LIb8oZAudzcqYpJ1GaOxY1m3+Km5dWXwF1AStinSwjmdtcl5sH5xNkAjAp2bwK8AEDXpW6k/F/3op0PlxEcQvWXhWyNWxEPtqJ4mBCeUAYIpGqUwZATHTwSKH/XBoWSAOnCDoYYsR0JKHAvNzNomYKbZCtT1WL9iY+9qLU2D9WQsQu5KwiseKDTlBnsmuLEzEn69MOKQnNpOQYzi/PvKjzyiWA55i8moTQzLrcxX7X+ju/S3pM9enrM7fG1LpCG5aILHh7f+xDZrN2RDJC+vTPHw5CvzKcVpYq5HltbETJjSM+L5MPq3SAfEtdLa2dhLf0tq1WcQjKw1kOZ950Elm4Y6A3rvWISBkDouU6iU3G0rV+ifRbwuu8rvHMlGL3XFC24i+cRmOWJjRCxqpYeP5Bmn3zPfbhlcS6PtKqD3eAR2DI/83KlGI/GytVlz1vr1TueYtrQMezh7NlrDclPRyq3U2mvX4QFCjvYYs2l5c9sYQT73ip0fY0vtsThcA4sLFpAbgWlOE7Ag9AeM8ALmPvpCcMcKblclTz1kVAqMUAMqBoSaC+apSQjWYhQwTad52wdleyT6lDd6mJTKimXFIzkFOTstK8mhsbwBGO5Xr+mSdmFvjty+nBMv+/Zthoun0DQBmk7Cm0cJiYbRpD9h97vyPqCfzaVNhlsRC1iA2TT+83sj1CCddJwTqbJOezKQpwYDR7TZRCy eGZtDBcr O9nj2BtJ28mnjCi88AC9foXxxOtY7COmv9DuqCehi6G5ozCR/T/vE+xhxb5yCNrhVqQeoyaTmOUqk/k0Z1ZR2INAE995vcouivoHq1bS8puJfbEvomNtz5D3+EtCQB1aTfL8jTcJ5n1fvoTI1hijf22U6s96+fQ8cEJdxS6rW+rAPpX4CM5pHjHO1enC4FGBFAly1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is not used by radix anymore. Signed-off-by: Aneesh Kumar K.V --- arch/powerpc/mm/book3s64/radix_pgtable.c | 10 ---------- arch/powerpc/mm/init_64.c | 21 ++++++++++++++------- 2 files changed, 14 insertions(+), 17 deletions(-) diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c index f0527eebe012..d57b4f5d5cb3 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -601,16 +601,6 @@ void __init radix__early_init_mmu(void) mmu_virtual_psize = MMU_PAGE_4K; #endif -#ifdef CONFIG_SPARSEMEM_VMEMMAP - /* vmemmap mapping */ - if (mmu_psize_defs[MMU_PAGE_2M].shift) { - /* - * map vmemmap using 2M if available - */ - mmu_vmemmap_psize = MMU_PAGE_2M; - } else - mmu_vmemmap_psize = mmu_virtual_psize; -#endif /* * initialize page table size */ diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c index 5701faca39ef..6db7a063ba63 100644 --- a/arch/powerpc/mm/init_64.c +++ b/arch/powerpc/mm/init_64.c @@ -198,17 +198,12 @@ bool altmap_cross_boundary(struct vmem_altmap *altmap, unsigned long start, return false; } -int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, - struct vmem_altmap *altmap) +int __meminit __vmemmap_populate(unsigned long start, unsigned long end, int node, + struct vmem_altmap *altmap) { bool altmap_alloc; unsigned long page_size = 1 << mmu_psize_defs[mmu_vmemmap_psize].shift; -#ifdef CONFIG_PPC_BOOK3S_64 - if (radix_enabled()) - return radix__vmemmap_populate(start, end, node, altmap); -#endif - /* Align to the page size of the linear mapping. */ start = ALIGN_DOWN(start, page_size); @@ -277,6 +272,18 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, return 0; } +int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, + struct vmem_altmap *altmap) +{ + +#ifdef CONFIG_PPC_BOOK3S_64 + if (radix_enabled()) + return radix__vmemmap_populate(start, end, node, altmap); +#endif + + return __vmemmap_populate(start, end, node, altmap); +} + #ifdef CONFIG_MEMORY_HOTPLUG static unsigned long vmemmap_list_free(unsigned long start) {