From patchwork Mon Apr 3 05:22:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13197669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB449C761A6 for ; Mon, 3 Apr 2023 05:22:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 84D0B6B0074; Mon, 3 Apr 2023 01:22:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7FA3A6B0075; Mon, 3 Apr 2023 01:22:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5FFD46B0078; Mon, 3 Apr 2023 01:22:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5093E6B0074 for ; Mon, 3 Apr 2023 01:22:58 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 29069408A0 for ; Mon, 3 Apr 2023 05:22:58 +0000 (UTC) X-FDA: 80638935636.21.A541582 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf26.hostedemail.com (Postfix) with ESMTP id AC593140005 for ; Mon, 3 Apr 2023 05:22:54 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b="hOua9Es/"; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=Hy2ugkgh; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf26.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680499374; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1N6qbt+qhUVgQy2JQLXyJCdS15y0cyZtppYubbAlJkY=; b=NOn/KIrUqetPM50qjQNA3rphvVpHK4eNR4Wu716nCV//vnhWCWdl0VP8+fypqrp2QGwnfH OPF7jr5RaE8NPBhMsVOpCyOHEajrUERHbh2rcrzhTJ8mvlewou5x4eiUd5QbgMwvcpP9t0 nInwfFkWK0emcrvpk00Y6JoHQtWRReA= ARC-Authentication-Results: i=2; imf26.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b="hOua9Es/"; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=Hy2ugkgh; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf26.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1680499374; a=rsa-sha256; cv=pass; b=Vd0RmAaV8TX/LzbiPRXuUO7TKNlln1gAcjkDMK0qyWTdvEQJMYYY40ALhRGm84FfGJsp+8 iIvsaRU7Ias7rv907XPT1FPu7xwsZkwDBFQ31WebCN3wi7TZ+ertmZHYhT+awkhueKDbSm 0Ugscw6xMYp2P2T5hUKhsWLvxyaVNPg= Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 332Msls0031744; Mon, 3 Apr 2023 05:22:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2022-7-12; bh=1N6qbt+qhUVgQy2JQLXyJCdS15y0cyZtppYubbAlJkY=; b=hOua9Es/E4R9NqhhOatqoqZs0Rk2TIVxG8ieeM+AIaeT8PTbTG4f6aG5P3XYQebQckMh Q7tqwvhoRemGGUJtPBiaXpXX4V/HUdTJ+9RvIUoIYlgUMuFpztsuaRrXhrTL6sm6eK3m j4qgPoHynHSCmDO0iGTwMEh+j7jopfyWrGGuErQ9vmx7vVm1JGrQvuvLcc7LRxohv3m/ 65SDlRj3p7Xl6yO04aSSGdpOBlVU11Dchg7rBGytx0iCmLZYnsHog+6cgqKr99MpN9P8 8idfndT1TjaLUodlXT8BYpXl1BwWbJWtWqL5Z7ArLAzN2eWJz2a8/TsglHor0XvlvMkS cQ== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3ppb71j7h3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:30 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 3332Pl6K014039; Mon, 3 Apr 2023 05:22:29 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2174.outbound.protection.outlook.com [104.47.57.174]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3ppt3djufx-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:29 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cgT8+ZSkRpktst4jWPVexS1fiEbuJ1OI+a6c/rkVxC6U0BGuhmHoLA7Fy5+gxiZX1o7ove0JeJoNkyP8OSIJFmLdevl4jCBVtNUfNdDKuoj2ECLJ+6FCF3bKZ3+Kkjn1cA6ktMrwiPgCIuHj6c5y9GrpvbJ0lOouJatJ9oumUVvRQ2Cl2d2fYHH1GkWY1EF11S9apULhbUB5L9j3LKScKDPRfMg7GZhTgq4n2Gi5w/0yo1CAXMAQv/dN2peOduXGuyTxr9nvaVA8sJWFtnDAaDKnvcivTsdr8VYMDJX4VNrvIgCm6i1vQh3BDj6wC8fTHg7AwIvJcYgwEF2btEa/Bw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1N6qbt+qhUVgQy2JQLXyJCdS15y0cyZtppYubbAlJkY=; b=DPwVOGIGxn22GTFqi7mQAzNX0gJiWj1Ft/Ehqp5+9ArhjiEvq+iDd0yOR3XRsH+3mOZUndGU3hD9TZ6DeVst7kYoq1jIPSCyjp5YfdXxQwIqeod4lmKlXWuJrmHwFQ7O7jYdiO61UfwU4dWj5iSQYrp0YsVnl8LIErgGA7d4yHTtmxvgsPNHzMY7ruxi1HVuavqH3hkp1ESnZguUwUNaUdgIhO4VmdPwO4Kb0Ufk+7ShnYWZMB2doKdipX4gv/Hu0TRgMWXd6qYShY33iYmJsj9O7GHvTpXJxgn5WEwXDEAUdwwNLoXqPJUUDFo78opItjHn47P0p0mJW5Tyg/9z9Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1N6qbt+qhUVgQy2JQLXyJCdS15y0cyZtppYubbAlJkY=; b=Hy2ugkgh/AFwOS0PC/TBdVB5Hmafo3mRQA0a9UhGsOSgIVsD5vG3FifpRaW8hATHCe1RSAJym1FN+01vEBCVZis+ayoopfIy46YB66jSKtkNYeBirdwGDgrXy5cFlkcG9cy2UMkuAD2ibjM3YYTGa4FnwNPSXPjj5oiHDmwUK3E= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by SJ0PR10MB6398.namprd10.prod.outlook.com (2603:10b6:a03:44a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.33; Mon, 3 Apr 2023 05:22:27 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6%8]) with mapi id 15.20.6254.029; Mon, 3 Apr 2023 05:22:27 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: torvalds@linux-foundation.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, vincent.guittot@linaro.org, jon.grimm@amd.com, bharata@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, ankur.a.arora@oracle.com Subject: [PATCH 1/9] huge_pages: get rid of process_huge_page() Date: Sun, 2 Apr 2023 22:22:25 -0700 Message-Id: <20230403052233.1880567-2-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230403052233.1880567-1-ankur.a.arora@oracle.com> References: <20230403052233.1880567-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR04CA0267.namprd04.prod.outlook.com (2603:10b6:303:88::32) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|SJ0PR10MB6398:EE_ X-MS-Office365-Filtering-Correlation-Id: 45791c6b-8e75-4f74-8949-08db34036a2e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YD1Wo4q6FHinxM29vI1SsUONfNcIqT9VEQ0j15jqAUFM/+cbXdp4DWdXlZHyKcLQsbrFBn5s0nqALWiE6HEEvJccQu4cfVJ5l3y5ukUiX6qYZX2rtXREyE0L9R7bNEFOAZgnEO1Py+HdkN9qTQNxxRVKGfnQJ+Eeayz+XBlv/9G2GVVdtCGeXj8TWS83KxHhuecD1l14vYQJ6hSh7+/+79sMvvdlWO+x2XqIA/yZgo7SZQva0r8lVNRbb5WDvqHhtCQUmBk1PMtTAzu5U97KHYIgTbjVfSYxOJNuPVvfGCQBji5S2+Sg7hz3g3Hgw32e58YVw8oxAAI2q/JJV8qzwJ93gb8eOpKE/2g7WGkuZFT5VrEgUffc673O3l2tOjsOrDWPGeotzZLNOynELxAk4iYb2Zlp8fCwx42Wr+gzSmIKxRQFEbJaR27Ns51bn2IXToSs/4Xruj+Jf8zqFZnTZ/1VVhPNWKUA4eDpn7xWWsJcbi6Tpu9KWgtwe5JzeWgfJZorDWt8AHOg8BBX56IyM8SGGvHqy80Bv9dXP+9lPBCBGejQeml+FfAQEHsJEOihXo06jN0B+3lkDmvlxzH+I0xhu3fZclILX9tiszBa8DFXC30ePmmkoXDpdgUUfPMH X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(366004)(136003)(376002)(346002)(396003)(39860400002)(451199021)(2906002)(8936002)(38100700002)(41300700001)(7416002)(5660300002)(103116003)(36756003)(86362001)(478600001)(316002)(6486002)(966005)(107886003)(2616005)(1076003)(6512007)(6506007)(26005)(8676002)(4326008)(83380400001)(66476007)(66556008)(66946007)(186003)(26583001)(14583001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: xLIrrbNyKGOhvDeQ72mC7nMogOYXdEbwUTXgTaGJnbwqAAdk5HV8hqRh1w1PYb0nFE3Z+T7Ujw2t2NcGqeW0+FTrzKQz6iv0QyDmy9MGkdmAuEbptn6Zm1zXRXGUogXxunRpf9R6bIQB+X1jRInCrpiX7SqnozmzKL4jPFEMwl5Zd2ssoOHpKhw82NI19yKK4irvB3oxUuPqy+3emd6igEiBGuYIG+tC/8GqHPFNLYFKSD+wkU62LC+cNpTCXABdhMMu1jvg0o9FKaveaFFDOHpfWuFBDAwyisYaMd8I6qZ7ZIRNS8vkHnBDmcF9aVNFcWxqYPuA5LQHb3T42kQrva+Pqy1RkIrk33VdMpcjFADhv7IoH9Qms/iM3phL43nPvZ/kpS0H4TBLg1odXUoOqEiIpbZxcIcJcpVT/aggxXoD+IAlBfJwzutINURmwBDdfLLesSZdxi6uEoAEzx8T/1NkPZIw5tZslAiphFbivg8mYnRLyCPpIBqkdebXTkKR4UtktxKaiYyta7KaWkfOv2PSCySB5MWlYpifmgNPUza/bEBW+FeAXQlGmxXsGYI2OCYVJH5ukTXKdDnmexTJfHTPjQARNH6pKHJVOYzO1h5sz5oHsF5dvCuOHRYgDzNZlZLYS2zy0Nc3iwn4omnDgPyF1gHzsESHyO9GWe/h7lPpJUCIDyitwvNHSh+vZUmMNfzGAPwwowV2Y1uLExv/wDPsxjpyK2hDXbbqzgIUeAXaZ9A7f6CetEc++lpacTbyT15ElW0uOIc+RG5x1WMlfs8e0qGHQKnHWliBdu60v+1o1megK83z8+blplS/H5iPK7eO49+xwizjhOFmWxJW3FNipsrooeLm344j1xrFgLa8yJ7HMg12xbXKo6Q+TMJOmaXddr6i+sGXjgE9YkBsle7I4JXK3Qegg12Zpc3KFVS/ANMcfLVgRqwhZBBA8ug6ewOZ0OD9ic2rZK7wayucB2u5gT9yvs0VCjJb74vkFVpAxquEJwJvHev9wPHABfrJra6spzS7xtRIF8VxxRi8eaLJIf/+w8AzemMfythu+O/Em4lw2s9HB9kLbDVpcDqDYRouG64k/IVEzFD/S0zVUETYOunPDIoVql+I1J39pJSu39ExyQFnHnDSRMxnkab/5UlW+eoTtRNhXcfaMASii1OFY8AKYvCXmLxAaa9wLyiVFLDvEHVrXr4jv9RQbtyRtUNuzPTgs3Jqf71jnRI8yVHQS00iHJddYDkE2YtoeXLGbFtY7490WjGWYMmUDwXBTmM9SWncqBi8MJp78wdH4DrMsDawTqYaZChHaUV2u1IfSW6giRWQLFW/BvqJ4PNeSjLQTStA5Ui0DC6XNcgatxrFnY6E2DFSCx1RHTDWdjNxgF1xwbPyPsp8zUNDcdZOzNI5r+AHQlKsB+PpCJHbl+vilfrxl3M0xEnqGm70SRx9VSlV3hyzbgxTD3Q/bsvft307mOHHW7O3/yQdwsuOQmBfNFHsTGmkbINSu3USC2Sv7BxXnGMmjvvKa4TL90KWKqXUg/xY4sfO80kLIwnY2ELSmWx6vNt4MMzThOvcVdyFUP6oRVenUXaylgGgrbaO X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: V3NALSOKYpCCpvh/qyzIZhyKL78AqK4EIAxQBe3+nkeExfNerw+O8GYUl4gmcSI9PV83/vqx0AJa1NJox8G4NXX3KDoGkbVZdvHqhxSDIb1MnXgzFDSwDCx27i/mQsrhqgMS/Pp3Wn2C4NcBzi8+C3Km0ZoyaYdQ00yOso539jEdaV8onZ64oQLO3F9qWW+NCHFxkSn4hQD8s2w+F3aKJNNkbGLxoo04yVduGatvpr+AQW6suF1agXbBlIVflUxJF8t9x9kM5S2bEkTZfOXCg5ub2SNejfy+YPmKRsCIPAB+03BbXliRRWnoNEwhzhaKQjPQjrKrOQVitx3UY08La1ZmEpi5+Pxfb8Mr9RLT6+GB/1gRO4bReFNKt/pUxWjOBO4FdzdCe/zQp84bt9FquOboByLpnz8Z3LML7vLHH3m90SsDUu7q3j0e0YFHeg5RxGxZ844qdEsiXCPUVxdH2UeCy5tJzfTDHQolIjaeZ3Bhj3gtoSO9I88bKndBsXM6MB+t99f7EwMmgm6ocqmrL5ulDO2XdmVpR78BfI9FhSLqaL/bRAjIoqwtGiYajhim8WBNscUbJs86bF+EznsXRjX3w7CQ+kD6LxJSnK5krHSdnguzUYvJhIHzCsPIOt6D7P1LNPQrQIvlwTPDdW/RSeQ6vAqmygaZup193b6vmQA+as9duYhJdtLPoZFxTfb6v36ctWnanMulzS9yx+/k+/1Hvu3V7lUviYgIqbNtaj2M58xfTAS5Y3fLw+b2zWersQW2Ffs7QeMS7FMkx8REuB/Xe7iC2M46I3hvjJmqctO7SUeRqkPMQujY/qhNri+MsVzZqIpBS51xOAdqs9D95ejv0xLfqofBW64CuFI6pLIART2Y7Dtl2mnGSuvPEZKVJEL+F4HbEKTK1FXxyg4jUvZW2bXKIzChhBMwyrm4Um+f1hfPx8WG/wOvKlDfojqGeO6SI7lrGxh0MN2EPhg6kIjTL6IEj7zhAxsDqHPv+REBnytAU9Mk7gNzNNYHjxYfIEg53oCgmfGsipiEdaLVukW1zXi+npMBsTtZSOqMyIZCwIoEFqgQSDTLUt2cy8nETOWhCNPxFJ/oB3z0M7mWJZWyphwdNpB/dG6RK5/3kl+nLJRN0JAbPgm6wUc2JMmAGT/Bse76zDdngZZScmgylI1m4mC4fPvuQ/3eM6YC/wU= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 45791c6b-8e75-4f74-8949-08db34036a2e X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2023 05:22:27.5865 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QocwTopNBygm3UVFFOScbZPlsMlf7MXtjZkZkqwCu5BuBnY9oSMBt0zKthnkf45CgCdJ7F9e6AKy+TyaJX9fK1tThqOLe03lq9uozJEXAzw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB6398 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-03_02,2023-03-31_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 adultscore=0 suspectscore=0 spamscore=0 malwarescore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2304030039 X-Proofpoint-ORIG-GUID: qoiaRtD32JN3hRfot9KfEK_SPa1ULIEs X-Proofpoint-GUID: qoiaRtD32JN3hRfot9KfEK_SPa1ULIEs X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: AC593140005 X-Stat-Signature: jjy1ikbsmoss65j3gbp5idi4iqtdgwgs X-Rspam-User: X-HE-Tag: 1680499374-335287 X-HE-Meta: U2FsdGVkX1/w9olnhNH/7MUYUo6CLsde1LbWdKrANeeqob8t91IeL756w6U/ZcjEHLvhqMkQg09lC/RdnUg8mDADQkFjMPpoo9YWHK9nphaltWNrYve6SbzoiqoIT65bsW+jbY6g/vnkWXFALyAFOdz5iSGr3WkdriX/Uve4ipui8MLJGQFof+Bt+sq6G4p1C9LHdt0IBbgtL2at3N4R1W/qSCGDgZZ6M12vQsFkO7L7MI6RoEv8+9Sfvv4dUGI0HlDmabGetVhWIz5ObkSdO0Y8lZXkDLvsQVAmUPoTsLRn8pT6S1kkkf/ZktXmwUUK63of01QNF4zzAn94Z2CExvuhiZ8+MYgsEzU3XatT8cng5SVClx9V1VkoAKc+Wr+6porHucL+e/6eCOO0qAL0hkABSJW2RQLYw2mBuYMLK5q+RwS7wwht9zClyLL3Uwd9mDn3paCynWkN6gZujz4Y8iwU5gibxiHUvyoKhwrxSwczahUkDqtsX8MPxxx/5DAd0a9WZCiKs1Zutp0gSYgwzuHcjXW9MN+tXVwkVZyzcfJyGME3oraN2+VBECa16iCP7q05pkegbIeqNerGvXDF6D1pnHOi+RmMbQbuMnmdeqvcSwFxJBF5P+xtgc1lUBjz9JR3q10ky+FIHWNdw+rp1hGFZagz2wXIN/BdBzCvE6Dgz54A+obqKMxdshU1oWj7uQRCoHdFs4PaznaPm1K+wA/ikhRzPnsI3hLBBjC890oT9rGKS7xuQWwfn8ME+ifAbWwRFtEw/DvhzOlkbX6Kibfv36TeH3fjF7wR5+TOPHywGK3rkjv9jsxoWk6fAdHEmiODRKBnIFk0FFqXfi1cWYSEA6tiC/yq+Q0WrOvTYzYfulPZ9nD6+CCbkODHj6nZzcrihlM6YrghhN/FFsQGcyqndRUX/8yqdu5P43b9ON9dzcO4FHFpClbJhgYhO9BNnBgGapl9yGpopejSHTc T6h6I1Fl r+NDZqePfP0p8663iliyXXQuvksdpmpjbYF9Hbu8vcmELNlTJ7RPXCtsUTISpHWv2rjgwOnvVH+dMo4YoP8Fxe6wVpXaXyt1G40FD/fSMUVlRHyVHIIbK04KfTe20z4c7U6HD2nuRgqc3JGsKlpyduxV29eRiqO9YTcuf0OG1MAsQrk6eefu+Bhkp2swwbn+jGQ7qa+dhrYNsX7OOCsIN4tPW4glWLJgSkMYue7Kp39PLOtH/zT84e1Q/9Wf6MkaYy2KlclpBhUBzCa3R8RbXJPEMAVF9AkP++mvZSqct1Fs/ayZy/9+jEFAEMnZN6Gz0nGhinwGkyOXdVNno3J5Qwxsxv/93o6CpQItlD3WSmfQgfsD+T2McUi4hQYdthuJTxAzTT3hT8wltcxVlnd7fRzVIa+1kbmTb1KRlugUDrPxrt2qCDU4bBQV8DZE/mPRmmGUk8vOJt9IvoGdbcerSzj2DRSx9odw/jJM25XN1wlLsR0spUFxUeeMouS/RS3g8ggFJNe88Q8u9r1KZWuoPDulv93YigkBFeYsFg3Hh3O59Mp0AIDO1cZJKxA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: process_huge_pages() (and the subsequent call to process_subpage()) translate between base-addr[index], struct page *, first when calling clear/copy_subpage() which then translates to the parameters needed in clear/copy_user_highpage(). There's no runtime cost in doing this, but it's unnecessary complexity for something that only has two users. Accordingly, fold process_huge_page() back in its callers. Link: https://lore.kernel.org/lkml/20220606202109.1306034-1-ankur.a.arora@oracle.com/ Suggested-by: Linus Torvalds Signed-off-by: Ankur Arora --- mm/memory.c | 126 ++++++++++++++++++++++++++++++---------------------- 1 file changed, 73 insertions(+), 53 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index f456f3b5049c..d54bc27a35ca 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5629,63 +5629,13 @@ EXPORT_SYMBOL(__might_fault); #endif #if defined(CONFIG_TRANSPARENT_HUGEPAGE) || defined(CONFIG_HUGETLBFS) -/* - * Process all subpages of the specified huge page with the specified - * operation. The target subpage will be processed last to keep its - * cache lines hot. - */ -static inline void process_huge_page( - unsigned long addr_hint, unsigned int pages_per_huge_page, - void (*process_subpage)(unsigned long addr, int idx, void *arg), - void *arg) -{ - int i, n, base, l; - unsigned long addr = addr_hint & - ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); - - /* Process target subpage last to keep its cache lines hot */ - might_sleep(); - n = (addr_hint - addr) / PAGE_SIZE; - if (2 * n <= pages_per_huge_page) { - /* If target subpage in first half of huge page */ - base = 0; - l = n; - /* Process subpages at the end of huge page */ - for (i = pages_per_huge_page - 1; i >= 2 * n; i--) { - cond_resched(); - process_subpage(addr + i * PAGE_SIZE, i, arg); - } - } else { - /* If target subpage in second half of huge page */ - base = pages_per_huge_page - 2 * (pages_per_huge_page - n); - l = pages_per_huge_page - n; - /* Process subpages at the begin of huge page */ - for (i = 0; i < base; i++) { - cond_resched(); - process_subpage(addr + i * PAGE_SIZE, i, arg); - } - } - /* - * Process remaining subpages in left-right-left-right pattern - * towards the target subpage - */ - for (i = 0; i < l; i++) { - int left_idx = base + i; - int right_idx = base + 2 * l - 1 - i; - - cond_resched(); - process_subpage(addr + left_idx * PAGE_SIZE, left_idx, arg); - cond_resched(); - process_subpage(addr + right_idx * PAGE_SIZE, right_idx, arg); - } -} static void clear_gigantic_page(struct page *page, unsigned long addr, unsigned int pages_per_huge_page) { int i; - struct page *p; + struct page *p = page; might_sleep(); for (i = 0; i < pages_per_huge_page; i++) { @@ -5707,13 +5657,48 @@ void clear_huge_page(struct page *page, { unsigned long addr = addr_hint & ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); + int i, n, base, l; if (unlikely(pages_per_huge_page > MAX_ORDER_NR_PAGES)) { clear_gigantic_page(page, addr, pages_per_huge_page); return; } - process_huge_page(addr_hint, pages_per_huge_page, clear_subpage, page); + /* Process target subpage last to keep its cache lines hot */ + might_sleep(); + n = (addr_hint - addr) / PAGE_SIZE; + if (2 * n <= pages_per_huge_page) { + /* If target subpage in first half of huge page */ + base = 0; + l = n; + /* Process subpages at the end of huge page */ + for (i = pages_per_huge_page - 1; i >= 2 * n; i--) { + cond_resched(); + clear_subpage(addr + i * PAGE_SIZE, i, (void *)page); + } + } else { + /* If target subpage in second half of huge page */ + base = pages_per_huge_page - 2 * (pages_per_huge_page - n); + l = pages_per_huge_page - n; + /* Process subpages at the begin of huge page */ + for (i = 0; i < base; i++) { + cond_resched(); + clear_subpage(addr + i * PAGE_SIZE, i, (void *)page); + } + } + /* + * Process remaining subpages in left-right-left-right pattern + * towards the target subpage + */ + for (i = 0; i < l; i++) { + int left_idx = base + i; + int right_idx = base + 2 * l - 1 - i; + + cond_resched(); + clear_subpage(addr + left_idx * PAGE_SIZE, left_idx, (void *)page); + cond_resched(); + clear_subpage(addr + right_idx * PAGE_SIZE, right_idx, (void *)page); + } } static void copy_user_gigantic_page(struct page *dst, struct page *src, @@ -5759,6 +5744,7 @@ void copy_user_huge_page(struct page *dst, struct page *src, .src = src, .vma = vma, }; + int i, n, base, l; if (unlikely(pages_per_huge_page > MAX_ORDER_NR_PAGES)) { copy_user_gigantic_page(dst, src, addr, vma, @@ -5766,7 +5752,41 @@ void copy_user_huge_page(struct page *dst, struct page *src, return; } - process_huge_page(addr_hint, pages_per_huge_page, copy_subpage, &arg); + /* Process target subpage last to keep its cache lines hot */ + might_sleep(); + n = (addr_hint - addr) / PAGE_SIZE; + if (2 * n <= pages_per_huge_page) { + /* If target subpage in first half of huge page */ + base = 0; + l = n; + /* Process subpages at the end of huge page */ + for (i = pages_per_huge_page - 1; i >= 2 * n; i--) { + cond_resched(); + copy_subpage(addr + i * PAGE_SIZE, i, &arg); + } + } else { + /* If target subpage in second half of huge page */ + base = pages_per_huge_page - 2 * (pages_per_huge_page - n); + l = pages_per_huge_page - n; + /* Process subpages at the begin of huge page */ + for (i = 0; i < base; i++) { + cond_resched(); + copy_subpage(addr + i * PAGE_SIZE, i, &arg); + } + } + /* + * Process remaining subpages in left-right-left-right pattern + * towards the target subpage + */ + for (i = 0; i < l; i++) { + int left_idx = base + i; + int right_idx = base + 2 * l - 1 - i; + + cond_resched(); + copy_subpage(addr + left_idx * PAGE_SIZE, left_idx, &arg); + cond_resched(); + copy_subpage(addr + right_idx * PAGE_SIZE, right_idx, &arg); + } } long copy_huge_page_from_user(struct page *dst_page, From patchwork Mon Apr 3 05:22:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13197671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7C9DC761A6 for ; Mon, 3 Apr 2023 05:23:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2054C6B0078; Mon, 3 Apr 2023 01:22:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1BA5E900006; Mon, 3 Apr 2023 01:22:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D032C900004; Mon, 3 Apr 2023 01:22:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B0BBC900002 for ; Mon, 3 Apr 2023 01:22:58 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 8B1581203D7 for ; Mon, 3 Apr 2023 05:22:58 +0000 (UTC) X-FDA: 80638935636.22.D7A8A0A Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf16.hostedemail.com (Postfix) with ESMTP id 4094E180004 for ; Mon, 3 Apr 2023 05:22:55 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=x12Br4NU; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=gZnsbqt1; spf=pass (imf16.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680499375; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=A+gbjMRQ73fBFl4pqWuQodz1X3vjS3hPwXVwaIKhxTY=; b=bvEQ4KwWUJo8WkYkPII5m7RS8cBlFLDwR4xQLQyERiInD81HmZOwNvdzksWOkbfvC2tbKL EcdY1RzugBJ/gjCCFj8GanlZGE7KOJ9FBz3b5rsc0wZfAxml+Q60VaX/ZHXlFFjoqvCVKn O0jXkVNbZDIxePRVObwI26AoGR8CbZ8= ARC-Authentication-Results: i=2; imf16.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=x12Br4NU; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=gZnsbqt1; spf=pass (imf16.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1680499375; a=rsa-sha256; cv=pass; b=w4IXxgRFoD6BEG28EEd5Plz2Mv5srMCWtfkrvA7w/HbjVv8MLls2eo2ayeTHWnRh1QyfYc TsSV27WvG/qvN0IKkzc2XgBRnYPGhk7hP4Nuh6+FjcHPTResZGPTe/H6kZlOLU504n7aeQ g+E2Ho4zKa4589eAY15RH8IZqBa4jTM= Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 332MwUai024750; Mon, 3 Apr 2023 05:22:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2022-7-12; bh=A+gbjMRQ73fBFl4pqWuQodz1X3vjS3hPwXVwaIKhxTY=; b=x12Br4NU5hCMefRthWdz/zXcq/h36XvBZA48N62EX/+ton8v+E3dG+tJGyTl+HWC5CYa 3R/xWCSHtFlGQ0TvYHzP+HCDkeqyMDCmtxNcJQR3tjyg4OaIHP/X8EPGwKzmn589Z+kV o895JU7vuzXpwWg5ecYA+4wU6UsmXKKq+bxDCKRNHDsDFa5zq0WdqQr1LbybRdn3fvYW 4ZrSZb6OZF62Xjpl3O0pBAehieMZIDnAebyoLvZhTYbBSr2axYUAnyI4kMiwlHjZYivP S3Uv33xF03xhMj6Vm500pHUJHyY/WiCm00wEO6qKDvE0SmgD7pq4fwa+lX8OOJdeW+iB KA== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3ppc7tt43u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:33 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 3332axZw001034; Mon, 3 Apr 2023 05:22:32 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2176.outbound.protection.outlook.com [104.47.57.176]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3ppt3eu93b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:32 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PK38zv7jQIUEySx9bR3gfkcP96Y/lsJNJ/2lLjaN4SCHIAK6YNGdOCeG9fWdPGIT94L3bSiq9gEePfG9musfn7D5ep+hELd6pHwAbBa76odzDVgwkeqQ+SW8jBwEeN2xPXuDROqVs7HzPbYgJTAXX1X716tXTJ87TTOCbzj1xkD56r+Bj6Rf4GT7P5smXznd3AsW18NMzKTpOD08wzNxuq8Is97dY1m1J9zyMqz0/+9LGhnY++qe4SKzB99y0rQlLyPwkX+958qqGNm4w8+ZtPueeeg3p+8itmMZUvUTY/u2V9DMXBSuqNA6vGThZR6ZjS1+85IOD16Ac2KIdJAf9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=A+gbjMRQ73fBFl4pqWuQodz1X3vjS3hPwXVwaIKhxTY=; b=NqhwkBJ3T/wnZAMZD5mxvms9F0PXji9OnAwpss5ihUcnnVHufroYwY2SI+hRZbVsjR0jSkufPUAQvu18d4edzGm6DeZr4wBPoqxdJ5uRyNSz2FYrhHP0GBt/6gzXEu0OkH3Hxi45LqU++QB+w0LEd/7T54t0A1It0xYjpfXA1E+SUEGyiNFEQy/1mtr21GkCtsbVvldgJpwDrFdxKY4x9J+xT4PwIW8SAuJ5lXRkMfJQzQ+c3l8OsNWlzwFw3dO8g+wBMYSev8Mf/WrTRD4rMKYtNekUt89lFXejm5MlhRL3h8tbbaS5HfKCJgzs2uEa+Cn5RKsQjTQv5GdSZMwTnQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A+gbjMRQ73fBFl4pqWuQodz1X3vjS3hPwXVwaIKhxTY=; b=gZnsbqt1OxjTPRecjJxa+fqCRJ3Tf4P/dVWfyJ0tpvTBW9djklyptSjocH+6t7VzT+SuNxkbV3bQtoHx7X/uusH3+bfjsYWMos/rh/KY3sd1HmkMdSoGV5K3Y9iCG4vjkt/OrGlrqY21ih96ObA3+mRhMdzUOf1nKO1Pxe0zUHU= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by SJ0PR10MB6398.namprd10.prod.outlook.com (2603:10b6:a03:44a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.33; Mon, 3 Apr 2023 05:22:30 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6%8]) with mapi id 15.20.6254.029; Mon, 3 Apr 2023 05:22:30 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: torvalds@linux-foundation.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, vincent.guittot@linaro.org, jon.grimm@amd.com, bharata@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, ankur.a.arora@oracle.com Subject: [PATCH 2/9] huge_page: get rid of {clear,copy}_subpage() Date: Sun, 2 Apr 2023 22:22:26 -0700 Message-Id: <20230403052233.1880567-3-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230403052233.1880567-1-ankur.a.arora@oracle.com> References: <20230403052233.1880567-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR04CA0266.namprd04.prod.outlook.com (2603:10b6:303:88::31) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|SJ0PR10MB6398:EE_ X-MS-Office365-Filtering-Correlation-Id: e6a7e217-89be-4ce1-1099-08db34036b92 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hyOmS+UFryA5XzOiwc0PreKDQKORHMZ/T0+Z6zKBBhDZP6xUWuulAkf4F2T7HUUdU+MbLk8dLNHOyL2/pLq4VDin1P+t99ziABeDAjdmUtX1fqbZ84ydFA8Mf1AmaTRgePtlIzAGMkOF9nWhaZWTCfBRwRn0ZxpB8BOIztkhN02ZeQSVOIAIBAqe7TRnDJMVcxrAjz2ty2RLMmro6KTgoYNlPz25jN1T5ZkSq3/1Jmt89NxonohqRT8D7thWSslUcsdVd/k7E6Obh1rRmKa8JCQgHQpgfPptdCzPNzs6eYf8YOJzQbouQi/hfGrb9KTXNQiujSt4ERvuEHxx6EnZzq7Vs2wwJjvdG/wu36MahN4XqTrsd3PwzipuzNQ/J9BLMBGwWdklHcbvi+06WDofvBWXZUvsPCgHp0qIjy2rYDsewD0JhaT+eRy0tPhDZzaft0LAIr0G4YD5vgMHIaOrCeV9vTpsxJgkUwG4OmV8w2Ui9Wed8wFtJmskmxZ0qT/W5/Qfd1plm3kBhy7AAkh23ze7XO90/lNBb1mrwVdCOg/5V2OT/NOsgN8YftfXYdoMvyFz7umRBPKisXNxQwF00zK/jVmyJcFQkDdRoOU33t84z2rmAxYdOCNndRRdyno34BdKCka0MJDv29/19dRKKQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(366004)(136003)(376002)(346002)(396003)(39860400002)(451199021)(2906002)(8936002)(38100700002)(41300700001)(7416002)(5660300002)(103116003)(36756003)(86362001)(478600001)(316002)(6486002)(107886003)(2616005)(1076003)(6512007)(6506007)(26005)(6666004)(8676002)(4326008)(83380400001)(66476007)(66556008)(66946007)(186003)(26583001)(14583001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: j0GKmuPQY6MndwSvByy3LbGQ10SgoBLdvUkrTeVi4lfI5ahCqdVRGXHtJH8IobDGgh3plyLEKF+ZDdwqCySZRx9YvhY7H8GplmD10B/hZ0TLwggl9lu9GJ2HUbsjD2G8mm3hSS7p0oo8cBJyInE/7qmZK8sHAgvNpeQI6tq6O5mCwa2IY/JbOaHxDEE+2R+In9N53UCFb81bNqT6y+SOr2+dCDCur5fbPKuiF5lXu+rnR4WH4bHALYa4P2VLg1hTFQyN1us4l8a8ZaKL3ohobCiNL256715/gI5O4j23+k6tyLhNfnKX9ey62NwBuBeOrt2456IcVU5bJxw+C4LHAnXuKkaA8ZKKPECZ5iVndpgQQwng5Zzj3c9PfUKX2Kzn9KorwsVl23SOY6i+x9TdmZmv/LDH/N1Vnt9hrFCsJLeAX3Aq3lpeldhiYcxFSFzi7LLM1/u/CvbCgoipZFsfX5xFf08M5ipHv5n92AvKfk18m5VWfFZ+ZsXfIatOY62ov91ggRrJElTA8YSGUUXbJMshpNJMc7ZXzqqM9/S5xFn0BKWf3rbZghJN71HHGP8so5B9o9GnBUoIragBNNRLaQrhXTPIyYTF7QIVcm5z0bKRyt+0/eTedLG3X2bzD6WPd4q3OKym31K5OH5gmZI2ilgA29Q1Z+2j8WoLb+6UJvuWOfLx27HER2w6GsqM8A2ekoMsau9FXnvh7SVkIMcyCttW521Mho5YRrv67yPdeyfV551e0cgujGmsKq//Xo12C3cixP/RFvqAVU01WbdHvezZBOwl//2cHKyUcdqHpxhgHTMwwZDnx8eBy17+3LQjJaZ58yqZyCKXnj+KQOVHlVJC34cK028zD0C+fcMpEh5aY8zrkR1JpEQnul3tVkiG/wXEalZX4CNliASaavRC3yD+pSwYnm3nLmV6bw0jykTjtTUMTSEzlxD8gnz30GxvVCwzt12YzVjXMoDZ/sjiRW8g53ChLGG1cSHHEznc8OyATgDQqksnyebOSGoL2gkAymU1RX9CbuzB+sh8UGf1jwTQ27te7v15Atl+YZiX582isRJsJH3gIIEod2/+Cxp0H5KgOJFr9jACfb2J1Gc2LywOm+wJrsCcJ/1nAZn6dKKxlMVA9twZfivlJozfF8I7h10yg/agx6Nec6RO/keO/Cqw9S8bq44U+rGrL2lxzxauLFmVJiESyGHAJAEFAXAsow/gUX+eeksXdREEPjVO9OW84aZNFr8oy6BdinpLegmw+8ImWpjB1SLVJT500o1A4WEJzevGvv4gGyunmmVy9bH6/w909/3zgiAFY7SBWCsqawIZyXkdnWPV2hEV0U8PSmpjp+V/qObAXsm1vrASPYO749JN13Oiot/j2PmkzUUKXDSEXNF5gUaTE0e2YiWUgTPKbrwUzoicqKaV/yKvzNK6jAd3ivrNLwtGCiZ1Z59XYi0DNvhuMaCg+7K5HVdDXwJzW8DCv7IbyJIxkaHJLu14MkIPG6Ithez8PgytJ8egQZhDJJdCl1JlALSylGtm7dArE/cs1IwAWL49QbDlMnRVu7xzae3LCyq9MavaFWIs04TvhpjnvehG1fz/qImr X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: kNquaCKMtyHag+0Yg+H3/LTeiyLlfuLyHzIqhWalNpR7d+qq3pLDIcdy/L4O1ApYlf75EQBlhv0LfBW16Vq0bDmHma2r5EAE/LY4JOhV0xeAR6K40oMhlDyCkw2fX0/1QaUzHBeX01GXyGMEdFtVQTXaPDnbFdY/YfTllVPDS0HlFEb/7WiU5EKf65zhOj4mnAt4vGxZaFrz9Hx07fU0J0y/0jistZ9SglfDcPdQ+L7rjYpdMuHi60Rl0qDbaGJ6nhU91sawHkT9oYPfvT/K5Zb2Ir6F0VQgki60jP+uc3/qtKQfdzYwnaWJ3RfOpZPXzqQSpm8oyTv0j3Fe5jQWDuzXGbSdu/exjYDiTyyBcjU2f2G42IvkGQLGQOJEHDfvSE1yZ2GZGedJlrJEC4FKSvMqEObOVG7HEU2rpy+pE1Ojhq5EtOMtL5uqn0Ap3oq1A9KgG2WY9LhVh3I33FKaJgklfDW8lXmsi4BVrOA/PBEVnv83h32Uhs2NNdc2ZfdfWMFf7Q1xJF6/7oUcNntsoRAvp+GgxYfd72CfQ+psLollm7d4C5R7Ww9Mq3lNnMML/YOjy8NDjvb0zBuGWd6WNudqs4ZLmyYCMCXiV5o4gJjEe5LdvFZdx5rGlE9ZRMRQKKt/Q385F8NhM2pxZvI18kzI0XfeOZE2CKZGzfBqPWrwzocg/SZ4bK8dZos9s07uUHtd7P/BQObfob9NXlzHW8u3/vavcoeAYH5CpQCDlqo05L/HrXdj577Kl/R8PRG7S3Y9ezxBoT0yefaOhGdNueQ1gw+qr/aR2Uh/J9bmsAQh1TlSHjiteOJD5/sn5b8x6og8UV8oteFFjgCgn9kc89XBkXDRBosn5M0FQV7jokEI6TUcxW0Nsf4TX/qljbhsYPlCz1J1XSMsLXXShJdbgTlHZp0XGDGmXU4bJbzPA466Bv33aqWv+IkbkcCLF5psiX5y/R23HgrbBYxueagRmn816GiJcsDdy/4clV/EwsqQtCoxZ5wKRUWO8t24PdEDIKj0l4gc1T4O5WbAo1nVkhSEbaRCR0CIeIwBbpSJ1/JaySB35lXZxjSa0tD8t3+LI3X9pi+Dim7YbHAkiPpl+uOdpjFz/idLxPlsRHNftrImTawF0jIrCNaAfg+vZ8dTtPR9eNTYm+fyEYMrjtmgp49NOM+ZJpQGrQu+y2DgRnc= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e6a7e217-89be-4ce1-1099-08db34036b92 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2023 05:22:29.9656 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SM97iqTUjbDjuwc9MF3gvVh6t3meW7PO7sazvOEB2EbZ3LrjlimRfaySfs9gjiCEeF2Ms5n40C8pz+khJZt2zTTP+KAzOJxWAbF7tcRazPM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB6398 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-03_02,2023-03-31_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 phishscore=0 bulkscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2304030039 X-Proofpoint-ORIG-GUID: YPAMp7xC3Fv8wlTX6rHaL5Ti_UiHFRq4 X-Proofpoint-GUID: YPAMp7xC3Fv8wlTX6rHaL5Ti_UiHFRq4 X-Stat-Signature: a4gwy58aiu3brdgx714ecka3196z48ms X-Rspam-User: X-Rspamd-Queue-Id: 4094E180004 X-Rspamd-Server: rspam06 X-HE-Tag: 1680499375-204421 X-HE-Meta: U2FsdGVkX19D39XghPnh//z+v3NXSSlAkX+H/eEmq3q6nczOCr9fyTNZVgy6WDJCZ7bNqfHkpVqozRiUT1be+n2Nw6Yyjx+IOYLk8EpsAO9MVvMmjjpzVMcE7yviiTbQOEtdWpzMd6OfmkkFuWos/maKrPjQzipoqIu7yQoJjWYQaMiZcAFXFTlQ2m67++1nkMJD7IQ5StL74mTo3j91yS5hXKXSLOVRAiwwesJASaafKOnCEga7qdz4HQc4Drzax74UOhxaOk2p1Ym7Tv7y5fyXcn+G3AaUFZjK/dMhNuAiZRGyh1BPyc8IfjYr/cURbdyA5LInyBMmRGXjihVQm0EarMiL+nmKf0vxrJv20P14uZFk6BHyspk5Ys8AZB7NHNUr3LzL32NyCPnLatjJJdqGQGxomok+JnwUCezKmv+hGAY8rRmnB5Jkaz7YPeCohalGqz/O3Ca6FAh0dLqxk5gIvdsZUFOdqQOHpg8WJ72j2ML5FyQLg569HHdFfSAHimatIcWZHstCQ5uViENAy26hMng+D6/rHof1CU0Ape46ciZ1Hm4scg/OXf3gk05QLnSyM0S/RrX2z+va67jEbVZYJNMPw8NAZl0zY0f4oGJuOoM+gXSi5XJPvYtQiPbltlyyNa64SVL3EcLL/ZbfNDydGplv0CwmUAIljxGZEJFrVmNcjQ1wjYSThCvTP7elmv5lq7SldRthXkiN71FBzHVA1DInRaVt5xKPBPEN11ratBf7RwbZHc3mkxxGd730JsFPjPG7VieVUkG1JibVFGQcUMjML8bI2GpINDQBYKNMu9wDmfJfsZHpCCBBzsI3lIxCvThWNzGA+MBkGPtx0H7LyG5kL7dBmEphwt5ZPBN3OiGBM2sAfvyZZ1H3X/3lVISeYxa1Ae6q6WI9FFdDaXRkIxZMo9jpXGENnvIbziF1fw0XAj+poP6y6cROb25mKUCngvH+fioX1Sdxcjx knpSNprM CVs8Pdwr68uFDmf4ivMG+Z3ikUtiKbl6iY0nG27/4bl2obIo1TmKfxDBbYmY5nw9SXbPJfezxei6W642OFOnD1QCMzXcT+oDNzLuFfkqwMmo2+39fqqOifXNANsh3f/n1SuzS7vZAAP/yYwCY1lDMn/SZPSuRu308HHeKmQE983pX2Pn050bRuvECHD6hIOi1tFww+0q7CywwScmrlfdd4LTUpLVn+wm8Ge8TDeCBNC5J8Yw8LaCzdzwaVFCur7/61qDWC3db82ntg+nNmhBrnDMgnVJGKiuvW0RjHrGeOueExaAarelwxIR1YB0izjiymjf62rJa2ZuAtUy1kvlHYB9O94tSCA2jjkZdDIwimMVHOm9U7TjrRtp0XkEtqu8+rhhJHni5VewWOPvuhv1ovhL5t5+lQEYm/NFnp5f3yWd0U7F6Q2yydtDghXHl2NL7EpFG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: clear/copy_subpage(): static void clear_subpage(unsigned long addr, int idx, void *arg) static void copy_subpage(unsigned long addr, int idx, void *arg) take as parameters: an index, a post indexing virtual address, and a base page * which is then resolved using the index. Instead just use clear/copy_user_highpage() directly. Signed-off-by: Ankur Arora --- mm/memory.c | 46 ++++++++++++---------------------------------- 1 file changed, 12 insertions(+), 34 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index d54bc27a35ca..6da97e6c7d21 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5645,13 +5645,6 @@ static void clear_gigantic_page(struct page *page, } } -static void clear_subpage(unsigned long addr, int idx, void *arg) -{ - struct page *page = arg; - - clear_user_highpage(page + idx, addr); -} - void clear_huge_page(struct page *page, unsigned long addr_hint, unsigned int pages_per_huge_page) { @@ -5674,7 +5667,7 @@ void clear_huge_page(struct page *page, /* Process subpages at the end of huge page */ for (i = pages_per_huge_page - 1; i >= 2 * n; i--) { cond_resched(); - clear_subpage(addr + i * PAGE_SIZE, i, (void *)page); + clear_user_highpage(page + i, addr + i * PAGE_SIZE); } } else { /* If target subpage in second half of huge page */ @@ -5683,7 +5676,7 @@ void clear_huge_page(struct page *page, /* Process subpages at the begin of huge page */ for (i = 0; i < base; i++) { cond_resched(); - clear_subpage(addr + i * PAGE_SIZE, i, (void *)page); + clear_user_highpage(page + i, addr + i * PAGE_SIZE); } } /* @@ -5695,9 +5688,9 @@ void clear_huge_page(struct page *page, int right_idx = base + 2 * l - 1 - i; cond_resched(); - clear_subpage(addr + left_idx * PAGE_SIZE, left_idx, (void *)page); + clear_user_highpage(page + left_idx, addr + left_idx * PAGE_SIZE); cond_resched(); - clear_subpage(addr + right_idx * PAGE_SIZE, right_idx, (void *)page); + clear_user_highpage(page + right_idx, addr + right_idx * PAGE_SIZE); } } @@ -5719,31 +5712,12 @@ static void copy_user_gigantic_page(struct page *dst, struct page *src, } } -struct copy_subpage_arg { - struct page *dst; - struct page *src; - struct vm_area_struct *vma; -}; - -static void copy_subpage(unsigned long addr, int idx, void *arg) -{ - struct copy_subpage_arg *copy_arg = arg; - - copy_user_highpage(copy_arg->dst + idx, copy_arg->src + idx, - addr, copy_arg->vma); -} - void copy_user_huge_page(struct page *dst, struct page *src, unsigned long addr_hint, struct vm_area_struct *vma, unsigned int pages_per_huge_page) { unsigned long addr = addr_hint & ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); - struct copy_subpage_arg arg = { - .dst = dst, - .src = src, - .vma = vma, - }; int i, n, base, l; if (unlikely(pages_per_huge_page > MAX_ORDER_NR_PAGES)) { @@ -5762,7 +5736,8 @@ void copy_user_huge_page(struct page *dst, struct page *src, /* Process subpages at the end of huge page */ for (i = pages_per_huge_page - 1; i >= 2 * n; i--) { cond_resched(); - copy_subpage(addr + i * PAGE_SIZE, i, &arg); + copy_user_highpage(dst + i, src + i, + addr + i*PAGE_SIZE, vma); } } else { /* If target subpage in second half of huge page */ @@ -5771,7 +5746,8 @@ void copy_user_huge_page(struct page *dst, struct page *src, /* Process subpages at the begin of huge page */ for (i = 0; i < base; i++) { cond_resched(); - copy_subpage(addr + i * PAGE_SIZE, i, &arg); + copy_user_highpage(dst + i, src + i, + addr + i*PAGE_SIZE, vma); } } /* @@ -5783,9 +5759,11 @@ void copy_user_huge_page(struct page *dst, struct page *src, int right_idx = base + 2 * l - 1 - i; cond_resched(); - copy_subpage(addr + left_idx * PAGE_SIZE, left_idx, &arg); + copy_user_highpage(dst + left_idx, src + left_idx, + addr + left_idx*PAGE_SIZE, vma); cond_resched(); - copy_subpage(addr + right_idx * PAGE_SIZE, right_idx, &arg); + copy_user_highpage(dst + right_idx, src + right_idx, + addr + right_idx*PAGE_SIZE, vma); } } From patchwork Mon Apr 3 05:22:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13197670 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E43FBC761AF for ; Mon, 3 Apr 2023 05:23:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E4AD56B0075; Mon, 3 Apr 2023 01:22:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DB73A900002; Mon, 3 Apr 2023 01:22:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE0686B007B; Mon, 3 Apr 2023 01:22:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9EC856B0075 for ; Mon, 3 Apr 2023 01:22:58 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 77F241A03A9 for ; Mon, 3 Apr 2023 05:22:58 +0000 (UTC) X-FDA: 80638935636.17.241B2F1 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf17.hostedemail.com (Postfix) with ESMTP id 3A49540013 for ; Mon, 3 Apr 2023 05:22:55 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b="Umr8tG/b"; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=gb4IdZuB; spf=pass (imf17.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680499375; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ohlWGuVeTFKX/ILJRWBRlxXLpQTPT1nzsZWWTdMkjxA=; b=sZVcXGhYBaute4++xzrdhRuRrPaEk1ImhYOMPgjEi1rGAhbIEZVRQZktImvWcGSCqDE0hq 3WtG4g8/0fxptty7fHXQL8yLxBqxnyBgDb4X2JQUnwc0DjGO0728XbLX5LK1EIKg4FSQu6 D6Ghqszz9m9rH+rpiuNllCgkBNu3/5s= ARC-Authentication-Results: i=2; imf17.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b="Umr8tG/b"; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=gb4IdZuB; spf=pass (imf17.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1680499375; a=rsa-sha256; cv=pass; b=LeHESKuHGGzDWajV79aAAsFB9Fft5eTZ4BLRf0ZRqRv/is225snAOl0+WsEqUfZnR5iXab R4jSPZBbB1Oz+GyyhzVzwTzcJOkFeQwWvXjxB4WYyYjgnWW3c1n1dPgW8Q/URvbrtBDZ37 MgU+VZ9u9zwrotJlQQQP9XDUpVeEEf8= Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 332MU6pK007758; Mon, 3 Apr 2023 05:22:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2022-7-12; bh=ohlWGuVeTFKX/ILJRWBRlxXLpQTPT1nzsZWWTdMkjxA=; b=Umr8tG/bSTqokt31/SwS8H6nVRyzLETUEd1pJhSfI38+SSBs9QM/3Kc43v9g1rLi1HWA CqJzKOSRWu7GYItYVXUu/o3vk5gcQ4/CChpYx2c1imXvcio0zt1QktC+wtdNcDRuaOAs oQaoeRmiro54Q8g/pKp2D0FzJlb4He0KZVcQppF9yJBJ/NosTL6gg4vg7ZgvMTJ6tCXF Hw+OgIiYdqoMs19OzftY2jdFYSxId9VzxdQufIPDPSdTyN5OXPY4I1WHFItiKkDanh7b eNP2QFmnc4lE0fh12qxcxnF93537u+cb2T2NEkFVQ0hjfvyKI7IAmTDgPA29v7YLqWeo wQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3ppb1dj5tg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:36 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 3332iZTo038319; Mon, 3 Apr 2023 05:22:35 GMT Received: from nam04-mw2-obe.outbound.protection.outlook.com (mail-mw2nam04lp2168.outbound.protection.outlook.com [104.47.73.168]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3pptjptnh1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:35 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BTkekeb43YDGaytj8ibdOp8kbxQRmitOIpWZBjfAWkYAMpHeWKdgvyPNkf/GrJUDLUswOpW2FHWzE2Q7/RerKKsxPIqrkg+5TPC7CnkHhftMgdnCbAOcnO3RwFi3ydw5CIfap/7+1ItAVcC6bAph0XVZprTCBV0bxdfFDS9B4C7woqrB74DPbq8ISmkC+KQDiyEjMot9Ctjs4CobgYEQ2u5hsa+1H/z7CcbH5ruc7LvDU8aQRxSqc29hFxySL4o6AbolR/79vgALB9KSL/i2aMuCmfljQQUH03xKi6mqncnCJDd7cHoxh3Ktt9aS7oEOA6E1lbjFsUwFUnC0ALOBOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ohlWGuVeTFKX/ILJRWBRlxXLpQTPT1nzsZWWTdMkjxA=; b=T3+lwwtbDWIuxosZQr7mbu1drV8m+YEFobi7qFxslRFa2qOUf44H4+Vl/iTljJ0JXfVyMW5lpRDgYwLPiOjZjZAdf8ee8d2McwO+CMuTN7kg9wa2yxChqSCmgHXzvDQ3N9b7mLhPlcSct9n3zMjkHK16BWLeVZvv/4wA1jOtuAopSAln9XzGYiTJ7XLdWWNNKg4hlmI2PymAAjuuxrWkzsK3pKx5CSPofIE5w80MRZdFgV5c5bxesJIMYHhgD4/rnjR5yBInsBGMsVRNxykffG401+2Aabh1PIw1t07M2h86eT7W3mh2mKqdSdvaB0CFRayL+hRB08yL8VAxCissVg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ohlWGuVeTFKX/ILJRWBRlxXLpQTPT1nzsZWWTdMkjxA=; b=gb4IdZuB+7iEcqZpYLUj+q87gpc6O7iai0Da2r+tnrXyv7oVSoGzvjnVgQ5jKwNRxG/xVXEjazyXNqvXb11o91NvB3btsaVuLTDlU+abbadzphxZ31hERaxAaftrLrjB9wR0h6TJldYpg1+49Jqm5KuPAiNlVqoxiVTsixWkI9Y= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by PH7PR10MB7106.namprd10.prod.outlook.com (2603:10b6:510:268::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.33; Mon, 3 Apr 2023 05:22:32 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6%8]) with mapi id 15.20.6254.029; Mon, 3 Apr 2023 05:22:32 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: torvalds@linux-foundation.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, vincent.guittot@linaro.org, jon.grimm@amd.com, bharata@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, ankur.a.arora@oracle.com Subject: [PATCH 3/9] huge_page: allow arch override for clear/copy_huge_page() Date: Sun, 2 Apr 2023 22:22:27 -0700 Message-Id: <20230403052233.1880567-4-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230403052233.1880567-1-ankur.a.arora@oracle.com> References: <20230403052233.1880567-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0130.namprd03.prod.outlook.com (2603:10b6:303:8c::15) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|PH7PR10MB7106:EE_ X-MS-Office365-Filtering-Correlation-Id: 01f37a97-c6f7-4797-e175-08db34036cda X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: g17nSWXUMfqWTm+Nexe+KttTvJn7LzjMkTOkxboC103/imibfWqRP+dRUAfy0MFiZ3jNho34I+Z5O8FmN3+0FXwMboZQuNpND9xd/2U0kbhed3oxKtcIp4ZFh52MnagA7VWrrRwHKIl/xpiEIZ12vK3J1DX0GnWPqWh6zgr2LMTQo9zWyWIAzJVhb12yWJytl4fhcgux6yreEErmcf8uvCi2b3niF+QVn7zLT4mduWYNbLPZzMThBR67aD2H1vR5GeQwB+RF9UbskebL7t1qCW9TWxSsMXqur96G7BDNzHVjpgLxJ7ql9od+TJP8M2VgHc3smjnMVdYu+3pZwb0bcnj5mYGillS9r3N6WudP+FX9Y2EapwbF1u1mTt8ntNF3wYUb+bjE3xD0wQC0jDYXYXkUO+zPcMMEHt5m6XQVhkika+qZR5KO0BxaPTmVMDL8wcHY+WPI19zyBmt4UNcxJV7J7jvCyoPwRw7m4a84JbiDFduLtwoamA3Yrmr0z9SNK3ZPKMSkmkvrWUQUks3/XvGAIqUwWZNVWPW4z4FKd5JguCbsoG/Z9py0JoQqvsuQ02F/29r558OAGgvOQzACYdo9GVkOh8bPQfIJmmjscAc= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(39860400002)(366004)(136003)(346002)(376002)(396003)(451199021)(2906002)(86362001)(5660300002)(7416002)(8936002)(66476007)(66556008)(66946007)(8676002)(41300700001)(36756003)(4326008)(316002)(478600001)(103116003)(6486002)(186003)(26005)(1076003)(6506007)(6512007)(2616005)(83380400001)(6666004)(107886003)(38100700002)(14583001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: wlsYQxBY0Nisi8aeBxFdIVyAH4j2k7GBLV9PHGjuPzZYKNSHkfVq6//mQkp9vW5PW+R1Z6w/9s93iHnlLWjJAB3bQRbws17+WSQJND4Gfp3PvWRztXQWPMq8+qUW6tyAcNjZHf1OZnH1Cy3oMSRyLxI4kYk4wfCTroMT8ze/XtV9EWfq/qp/IjqEA/tPRrx+x5Vym+NwWEKG3UIvy5arWju7R8QKENBb2HAVdasmSZjAfAY5vp/E0iBBZb9KoCBuMfz743vKekPiT/mBe1h7zn/nVzXjty0x5wdo+QieQVpPTKFXKf2ha6jcwMMIpBZ5ML8pkoU+L5vXyhRMR4Izw8oNR2hqBNuuIo+4EGalOeK4SvGqIn0xFnmsj5MKXOgJBy9nQd4HR6ZIgKYrYGvsIuE6I/WE4t2ilh3JMBE2L/KoeJJtJ7hHaTIvLP1olsCQR14dSkpo47CoB3PbNyU8oTL0o7h0+gsGe0GageCqBcrxaUpQn867jMFxwdAhoUtXSzX6Tr/i4+SFJ/jvd9ibHI5AT6MKUp9K4q215mC7kKEbjB7M+jWXfQLsRLdcL/WTj037CA3mhYBi1FqRS0+gnEhOla/JL2FtudCOIxfPHEdcZp72HWiweEY3XKlO44LqSmHue/FRTjK7ZRznmCg2fBxFNx5xmALmkQI/wMwO1DCYPP+BGX7BCgMegE/qKeYRPpg/dNEcLJPcfGAonbamqehrkRW59fqafecZPLGh2F8bFotgqi1Cfp1lgojvzpTkq2XhWl+b3XRoDr+sfvsmBg4d/CtABr4lweIjsTgchSxPuz0Q6+tFiBuUOSoR5CFpQoLefbmrHvJEL/j2tE5goCBcxzyVl6iChc0+nBn4xW+AQFftwloS6W1RmVuVoXGjjI6Cm5FbxWr6l2H+Y/Kn3i6iap6XDVrz7R+eUc+18PpuAY6gFYGLs19L/bSxMxwTRVe+AWAssddLrKBWtsWpnJBawijj+p0+/dS0QIRO+VtqMue2qtYZPJhs5y07xGAOn0fRBvQVOi3kA8aQ5+CHNZCljyl4JZFEIsP3I6UenBNVfBzNmyIJKs7SNRoCYM9ljDzSFr/zV4xAG78VJpdWgoh0uOj+KYEkaEfqMdjpRh5bDBL4Vj0Jy/k2K2L/kPVGmJlWfqKR38q28UgPQk2Pp7CZJ81Q4Robaxou0AzCa043/vZl9CduJgc8UzQYr0ZC8wFLxyJjCUoaJVFKo/lqyswWZKCxWHSFTlbyISZD/qkZNX7v6qwn3yPqRhKSKsG8ul6gfa3YnS3ZRYokuJSmRRE3iPaiuL/W7eIVC8STW9jiEZf4ZBFBxrEIR+jckelBBTmsGwtBSnjI+feMtbOv0OBUNa4SkCROyMCP+DPg8eoRIm8bIKF/dZZ8HWP4OFVjoP8DSteky1ydF32A9awPOpyzk7VbIU5GRfvESJRXBYVcuy0b46mtXVPDgUJlhnwBuiyOBrYrZdqUiJUUmSHamGjlzUTfVfGMFdKzzkZai6tPg0GTykwCUG+8SiA6m7cNU53mVB/FpyArWZiRNS/Fh4JZ8mZAilp+HIZZwblQlgV/RKPZ6ptPlPjkDZAlB1C3 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 6e0FLCQ+TObGhDfhFPzRPTGMNfJoow+YEhi5Vx+XU3Ub0tJ7l5c6qDftuEAOc3SaPxFK1uzl/sz8J/g5gCAKslwZZWnpTM0IbM3ivxuPrYYrz2dv6CdfVwje/HI9xQayT3Nh65GVA3i/QEO111D36XV0uUAvBK2rontz27fxktQMhHloAoqw4k9on4JYju5AWM/9YzTeg0q3e6FuUgqYsYXoLmKzPZU9DavapHU71P2FczNhLxN9veKWYAPYETmYBvhEn6YzHGnLW/U2OKUYGQaNq7oYzNYCaf42SzCTBTXitW8eo3suuTdjqtVLHYiRLIM15JOUYQERKF/K/6WLHLCtoWp6dhkpXUlCiRkKigpPdSCt63i9G0BmLBQnjJ6yVXVK6BdUIZPcmg+b38aWQ6xl/u9M8jBil622+w27l9cvmszEB7hye34l0rXilTbe7ke/JvzglVPvTBiyVIna2F7zpmovGS56KCsJwF6ZZvnbh2qc9VnC0YNAWeoEBlz7Md9McaMTZJOHf8V9HFNqmuQfhDaiU3xXwEkcymafwQbHx+coINTBWxHUZbgDmzlNsPDVQI+yp7Cg9m2Ua6NRH6GtLsorr149K81AakdEKK6c1m2oG774e2Buasg+58kl/6+sYU2MghvWMs2cDVTd6vvT3Vwqvex70e5V9NTF5XiYV7DwmAtF8EF2/3HymWef9Fyqt3BF9gfxxz8OouZwxx6fvtUdUGR8FWf7nHjZvYmDGSRMe7alpM4448LGcgbL8MVIIy0Mx7P4CQngz8vGWVPYupevmV+4A24uV2xcZLdcKLOF8Fhfhbui/E09JJUZfcVdzVhdtFaaATrL2qwJ/bVqwvbWh21ZvQJKkmmgg9844iwM/gnrM2qYFzIUrZkuRKsWQmfpNVbcD9Vftl2mdoAjvNHJpfo3uPHC/IHtNhPsJP9kRRwiKjbLbm1c9gonpEplWRSwpgx34yf/0kzR9FrfcOsrhYVIvB1AjJTOqu+QOus6gPXqXa27eY/Hpu0fdo/fPTtPGGyNPf9yGRW+SZdxB4Rv60p16jWjHr0BJ0L2V3UVNjN2r9Ia0YNO4ynT0L/749d9GhbtxDZ4JrR0ScPUYwPgShHE2rPK3cNLJNYhl5ig4z8gZUrzvRE08ea8tq3z9csweudX6rPwbaIN7f8Z9n8Ex8mIgccvnOPIG88= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 01f37a97-c6f7-4797-e175-08db34036cda X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2023 05:22:32.1264 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rNJXMkL48RPF+xFdQXcLCh0so6eMyDa5HL9gezr45m7jf5XtCTvjoS1tnyCjf4dVMTs88RSIBfpiVA+VbXMR13gPNT6KLzT7YbKEoc9Y76Y= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR10MB7106 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-03_02,2023-03-31_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 phishscore=0 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2304030039 X-Proofpoint-GUID: ewEv5zCY3Gyz7aBBO5lxVpjQuKxkAKo8 X-Proofpoint-ORIG-GUID: ewEv5zCY3Gyz7aBBO5lxVpjQuKxkAKo8 X-Rspamd-Queue-Id: 3A49540013 X-Stat-Signature: 78q3n6fxb4hpktzpbz6yrpe3zk11fft8 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1680499375-205161 X-HE-Meta: U2FsdGVkX1/WHMdpjtpBD1FkIsXZRT8T86uMfxFHaNcllM/BHeLD5a+i3OxrFXkOMGrmEDrGaiuo9HbeCfHzOP4QMmnhMcxJ2aLVzOFnWA/+3Gr0I1RFodLJMTvN3IRKa2qzuWo2KGpKKMQW/9rhqsjN9Dpp7162v22Cg3iSIj+BsVg4+xL4Es4/MMSQxi4IKCDGi2VWNnM62r5eLHoo9iy4fxFI5YGabXQXAbLKDRyQgJ3Jtchr3VtlvE58B09QvgugvI/E+Q5TimVSzO9C+KKG1H0kn18RS9L60xb0/anCZp44nGbVm7+t2bNTJkdgf8PfmiVSR+oFf9/dfoaapTdpLNh0hxEEAkAFVyPH471T6NPAZjl6zijDlNr8H6124sIAivDKjIe1wCBFFsoy8aa2Dc+6LrAqzOm2+1moGw0xzMlAeuliLzFvrk4tGp5Yd+HuKmcNnR+1cZQIjilViAZ+wQ2R2hn7ZKgJe+TRuwi6tS5V6NyVhvRgf4h4kdy9WBQUsu3Eff/F6ZkbkFkkdFOFgYpPfxo1lwSRGjqewcPkj+H/POPh3tPf/CtxKtu5WKoAUZFF+E2cW1KH47PI5/ACENEvnLGzT2p8BoUGTf4fbAtdDiO9VoNdGepPHmHGZ7tQcyVCQB2yt3WKp3HeY1XaMWrZO7gFX0DeIQhWbl7FM8fHgPeLUtEUVj0RPoFgrlEg/u7ks1FwiN2RDygciWpl+vSulCSexysrxtObO7LXmsV6cGo3W2JfUzICfEgoOOOspngio4bGIxnY38pWZASar4XXYQO9cU5gQynqECtKeIYQHlO9fNLp5+7VY0/N9pMVEI8zsqVLFGa/eeFL+8r6gha+d41LMwr1TvWponDmKwGKwz79op7KZq3FIWW/Exc/vfKM/a/t3nlUpPA2iDiQ0KUK0Lp/TYXDxfkW1XkWuo+f3Opd5QjhUUVpjNnGIWKnkt+UbzIL+54Wkpx MNbetSMi vYIouc7ZH7pP0qww5l1wNbS4hgPjwep4GP9RdVX7wjoU/NCV1hv35HSKnVEsen5ira7fdL5Bb1QfNh8eQgAYOHtufY8IZ8cdwP/5Qb0sL9VOxD+uSDbmsUzJr95GZLiMtYz38UvqBtVO7IcqCrnazsZ/+urGko9Q/h6Z41tu20SH8t10b4K0ulRXes5zvNWEdTi8uHU22YHOj0IAsNAb/WUandvU7wgJc/LAzKbrUCWyK0G0TRJlmnqmep6dkye9v4PsgIJpl9fpPX3oWCqJ+2sqBRVj5nbKDlviMdew9KsAVv7st6Ne8yejiBktFUb7OWI+oeI3pykFZtPSBIrC/qRwynfFxWbDLK0xq0GVzi84tYLHtZx47nvYpY8oiJoMBfnNLcRgN0cYtU2qSmY+0FwMJkhWAgXWGQ3Jqe2XUcVkq2BWpMAetTEWdXZnPOlmMCK8BZAkBrL7F5oby55y0ajxF8Rvymj+K/lMdVXau1/qtf8/tpIY5ybu9+dirYDZELAAx X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: clear/copy_huge_page() are constrained to operate on a page-at-a-time because they also handle the CONFIG_HIGHMEM case. Mark both __weak to allow for arch specific optimizations. Suggested-by: Linus Torvalds Signed-off-by: Ankur Arora --- mm/memory.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 6da97e6c7d21..9a6bce384616 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5645,8 +5645,12 @@ static void clear_gigantic_page(struct page *page, } } -void clear_huge_page(struct page *page, - unsigned long addr_hint, unsigned int pages_per_huge_page) +/* + * clear_huge_page(): does page-at-a-time clearing to handle the + * CONFIG_HIGHMEM case. + */ +__weak void clear_huge_page(struct page *page, + unsigned long addr_hint, unsigned int pages_per_huge_page) { unsigned long addr = addr_hint & ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); @@ -5712,9 +5716,13 @@ static void copy_user_gigantic_page(struct page *dst, struct page *src, } } -void copy_user_huge_page(struct page *dst, struct page *src, - unsigned long addr_hint, struct vm_area_struct *vma, - unsigned int pages_per_huge_page) +/* + * copy_user_huge_page(): does page-at-a-time copying to handle + * the CONFIG_HIGHMEM case. + */ +__weak void copy_user_huge_page(struct page *dst, struct page *src, + unsigned long addr_hint, struct vm_area_struct *vma, + unsigned int pages_per_huge_page) { unsigned long addr = addr_hint & ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); From patchwork Mon Apr 3 05:22:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13197673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB122C761AF for ; Mon, 3 Apr 2023 05:23:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AE2F26B007D; Mon, 3 Apr 2023 01:22:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ABABE6B007E; Mon, 3 Apr 2023 01:22:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 88800900002; Mon, 3 Apr 2023 01:22:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 58B706B007E for ; Mon, 3 Apr 2023 01:22:59 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 32A1FC0747 for ; Mon, 3 Apr 2023 05:22:59 +0000 (UTC) X-FDA: 80638935678.18.5AEC3E6 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf19.hostedemail.com (Postfix) with ESMTP id E4F181A0002 for ; Mon, 3 Apr 2023 05:22:55 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=1WOqz4d7; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=n6sXqHMa; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf19.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680499376; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UjM07Xo3SboE2ZDIKJWFIgfW4cJRS+1QtuqFInHyTCg=; b=NHFvLT1BVNPxhbmM1+EdT47laPmqUrfFXI4RZmz93B5zTGocjeAEQt9zSZV2NaNefEQlFZ /BrpHhtZ9K6cBWw9Js8xhUCk7SQ+xP70vg6nTMuAQi9TPVi13Yv9bLorDxlFX8lGcRbnbd qMB2cSq0vV01kT4azm/2M1zlfltF5IE= ARC-Authentication-Results: i=2; imf19.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=1WOqz4d7; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=n6sXqHMa; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf19.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1680499376; a=rsa-sha256; cv=pass; b=tEBEKeK5/yxAw/Tns0Wv9SVPottCcQGE9sVVP9v5HCyOMoNY9ksP6YCwA1LO5hVGeNwern jahSVVeCGz4oCfsVRcgRgmGeyVDbXvszHK7vEGqJPtTulqoy1cvl5qnRkXXDxwMfQY9yNg djecgJ1kAvB+9wRAKwKxemYg+Oz6Sk4= Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 332MM9Mx025427; Mon, 3 Apr 2023 05:22:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2022-7-12; bh=UjM07Xo3SboE2ZDIKJWFIgfW4cJRS+1QtuqFInHyTCg=; b=1WOqz4d7LZ6Zo//oL3jSMkScTNg8MkljM3a2AvUGDRL9NhXxfqwq6za20/VP8Rt+BveG WwSOv61eHLje0XWyl7g/KxyPTeLKcPmtMUcgIFEEgyiUiabzc3qyqlLA54RWZwk39pE4 Hp/F/e5emxKhiPkwlKrNUnawXgj/aT8aC+HtKvsyMyIuomzQsNKONRa7mEGhzRgaOadU YcP+bAXBKdCxZwyep90VRVBKJgcF6Q6U/6PdONgClE5F0yFTtEzrqj/zun136DGpwSdE 5iAJV/qf2UUIN2E1INTUl4BZFQcOQHy9yDRoLrY9ic6rp5Q15hQE3qIvl0mEzTGdWpYg Pw== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3ppb1dj5th-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:38 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 3333Uv6Y038324; Mon, 3 Apr 2023 05:22:37 GMT Received: from nam04-mw2-obe.outbound.protection.outlook.com (mail-mw2nam04lp2173.outbound.protection.outlook.com [104.47.73.173]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3pptjptnhk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FwI3s+LsyusrVjnaFPv2Y/9rT3KrXGz3nEkFzYrxeiTsnBdSTjp/6CjUmfHcVNQieDLldaTL+BjEzNk0X9JyJ4l1YN8BII3sfw85XNRqRiWSTSmTdQhTpqngCJQ0svNGurkzEWjsQazl4ehJ3nNhHlZSXrvZ+4TdXiduLByVw03Djj815SpLdetgVS8KlFIV4MpzCDl+L18OxaR+6x6MD4oG2q2g4KF0RAh/KV0DkD6mrNJxZQ3N+uZHYvQMPOFCFBX1QpQ8X9tlxA/WJMY/laBaIzN594ktvMTX4TDTJIsqnNywPK83UCgOZBi8ImNrT7CA7Gr3hD1N8md4iNVjCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UjM07Xo3SboE2ZDIKJWFIgfW4cJRS+1QtuqFInHyTCg=; b=BB1EaJyKAaWcDrRfS5W9Dqz97Qe7zjeiSwxNWKq1MREshAbkEk7xGVK0S/gSgaoCtVtEyBg7cgi9mHSCrW6axUt/6LGd1tv0ijJVME5XVGuAaAY2UYRbY3Mp5ZvEX+zCnwdA0Z4OxXcFkn/DuNfcX7GRlMNDRBVadO8mLBNa9NabalGV7jM3wotuvGiIwdqJHRfLs35gXE//AFIgFzIby0IwPM4aNZhyM8A2Em9yICs1thqByfGQPx6I6862YADtWIG3VA1qDqNI8KZcVSlRVoFYYIrveTQAHZpIHSGU+gCz7YK+hm4QAS+XIjgSrvQ3h09H8p1RR0CGgWYEz/A+OA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UjM07Xo3SboE2ZDIKJWFIgfW4cJRS+1QtuqFInHyTCg=; b=n6sXqHMaO5hzCTlLFo8CTQFiIk8a/a/x85w1hx0YUnRws02gna4e7JWmCr7AwTxFAUcGU13b82WatIQDvsNyU1htOOfd01dEnx79PsbLdLp1N4hqBkf1NZflzM9bWMk8D+rLEwEI6MCa5uD0Ig8rZGn95VBl+udUA1Yjgc2lHxo= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by PH7PR10MB7106.namprd10.prod.outlook.com (2603:10b6:510:268::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.33; Mon, 3 Apr 2023 05:22:35 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6%8]) with mapi id 15.20.6254.029; Mon, 3 Apr 2023 05:22:34 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: torvalds@linux-foundation.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, vincent.guittot@linaro.org, jon.grimm@amd.com, bharata@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, ankur.a.arora@oracle.com Subject: [PATCH 4/9] x86/clear_page: parameterize clear_page*() to specify length Date: Sun, 2 Apr 2023 22:22:28 -0700 Message-Id: <20230403052233.1880567-5-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230403052233.1880567-1-ankur.a.arora@oracle.com> References: <20230403052233.1880567-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0247.namprd03.prod.outlook.com (2603:10b6:303:b4::12) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|PH7PR10MB7106:EE_ X-MS-Office365-Filtering-Correlation-Id: 1f2f8e1c-02d5-4634-cd1d-08db34036e7e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WffUYm/xCiTWr3T6eBIlnT5ip5e6/1255qgOJCqYMQU1Dbqz8+vjgF8lRUmDYweMNPybA+AhbJC8VGgnyroj2NV+ZudsVlg0aD5+HVm6s7lY1xpFBcPx81m6eaiNCpWgYBPyc4EdRPadCeBjWWjroUUObsxOp+/UTim+ZivcayyGP3qW6bJli2hP94ymhS/DRz1dV6bhKgW9KGW3nI0oLs3E0/7sDrWUnUDHHbYDYzWssM0in0DKbX7ZULzgi+gp1nJxKR6Kvj8ne6/x3lI0WhjGpNRsILD9PXC+zG9jw4lWq6KqgQds2SjkAuttGRPx7zNe7zIX6qUF0qbYXfBgXSo0HOwYnSAGUx5TZ5DuJl04/NrIvZ3YBkjEGs3gTm4w0eM2MOSdBhjt9O7yKdSK9wS8QiDNKOjrmMWBNKGZc27eMDQHh6FhGT2AgIxcF3WLzk6Z6tbwJDFCqTYv5tGIS/jU9bPbaTzHdNwWOna37YZFx7m9uVaCjANMlLkWD5MhaFN4ammv7cxZ6nX6UaYJJH8gtEOx8fFixwGyI/ENpmt90OPowiHDnH2bAO7gIaQh X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(39860400002)(366004)(136003)(346002)(376002)(396003)(451199021)(2906002)(86362001)(5660300002)(7416002)(8936002)(66476007)(66556008)(66946007)(8676002)(41300700001)(36756003)(4326008)(316002)(478600001)(103116003)(6486002)(186003)(26005)(1076003)(6506007)(6512007)(2616005)(83380400001)(6666004)(107886003)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Yx05J4aPr+IqbvHOpvXk8ZEo8RheJNIi8S3M8yMX1PhG+wVxEpvFRvCkX/I0p/aPxICamVJ4bCIPuVwNf6SmJCbioX3TtahQIlZegGNjon9cYJEP4y3rFx6dfVYYUVxbZW3qaDH/JZmdvsHj4DRJhI+7Mxu3e5Wgyu2iTun++LGIjhljWSvPBv2bc+B9vk+C7s2Ncdpjkta7e6y+EG2jFAP3zCFCF7earEul2wJwUQ5neJq8Oy3Iip89LOrkdagaocVB20+XuUsKHky4tFgm4CL3JfV1oWVKQctLP/K4QqxcNmyLo430hxlmso4qu+PwnhdpZmsEKV3eMA/0uPwNypAWI2K0kb5S9qVWKnU2VuU4d/WW5/ci++QdEDYxIgh6ekqJubcNMWRhQR+hgJ+zOo9gVINXGrO4RQ46Vqg7+7ZF3b2P12qJyJg1IiHx3E1GHDfUM1IGGc+tkTD+1XvnaPCjSYD2SIH4m1oxNaSIcAJ6ja1/fPWbUApPJQnyjKkoUbdEHejG/p/Y8kD3OzWF//HAU5OWLyY8cOxCRbZav7jI0QSu1HioYDP1vKwYjfCBpf88NT5yBYjWB7QRKyTlfGAzgD5lGEn3fzJ6yK2kcK+Oony+p6KyoWg83AObNo/KEQV7/B/AIr5SY0fqlzog76J4xT5nv250wvaCkUQ2CIAgJSgFR2SSJfHkEUzfdQ7EY5jqrP9RyZm1paIASiB1QTFLq21SxvQWW8dNAwuV4hyDKUcIyZpOhmbAhPg3zeHAqWBPyW439F/azPXSm78lCW4Ah2gGrZwCuo1dpVQbiiARmZY6gxAUpO7VE/9hbwGZJ7wcmeN2JD3Hacr6rhlC/rie+3qnh4vDaUGPhP2iriTzArAzHCyELI4hvX2qhoBD9sih0zEhAS0FFlhr8x7eHA1XrLX+R/IsBRDxvv1aJavyBQAgCRsOO2nLJOkODbK1TyMKPsHfF9pNTzRSUDoEZqS2fBQ3pmstevurkwT6RaWVthxhnOQCnSFLbkVjy24fem7oiiPRoX99LaE44zLKCFSVEA2hxnm35ls09PXIylHTs7AXWXpNognsIGFOfXzX0RyRE4OrX0GsPhCx+OFe+zZEQH4176UFBpBhQvYfRg/WsPc2JDDkphILFJxzkg3+fQ5m3ON1M7Qo6T1PgepMhwyBAnJnjR/y4MvJcM97VvLS0H6os8zSDe+BV8+ULdGtoZeDlPuv0J7v68xzD3Q5QKcG/1fw0MfWQDjbzd13jQ00UhA5wDg+v3x22iMp37a8l8vkQ1lfljoq/P1sqB6JIzPTIcznG4wPe/yCQnV2YXWaG1THOA0i/Wd7Kfya2vkExOQZduwOTvCEeh4GoTR/All2o/4irtvv8WP7MLqB5YYrRN73sYR8fGz49+/+DZuWGm96U2uVK8KaI6xkubdmh/qK4k40IPZHAjrYNpglxONXvCg+dvGugdujKHPYBqJAZRKbqmOoB1BwK57egGt4pTNmKmZlNzXPa9peJ5w2ipXLJklEzzPljhvdet+vi5NBouyeNDhb+Xfm7+0Kk87s/WMGAB7sdATzrEBFRdUxqADA4/Y1hMDfVcI7KgIHE5iD X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 66vfYUq6go70SfEIgOxUSAfQFij07W6OPe7FvVXIA2JL9SaJN1lsQ0P9kN6Y4rTu7ndfTOxse2dvOohodIA0Lq1+wzaeA82Ic5CpC/V6zC9cqqwHTIzEt4EJZbn4TaS1p1XavDY7doWOFz/89T5/DxsEmB68sjeWtuEl4xtfHPR1x9uUr2kTTWu2U45MzhW92G+krl3TbfJadq3R+jiaLgtaUjmW0yCb7STlc3NJ8/LSUeAtQb7dcH2AtRct5XUpfIv0BLKvAtay+6yDMfkWGbB6OH/4FkSNpHGQkV/qiBV9Ni/COzfQls5S86Jv6U9MfmZ0QV1tTVbdaBf0NSwF3dGFWH/lorLy6zGkz0kboKyq+NXyzTVVTboASyNFjtT3HeF07ehHnItLhREr9jw8YXBE20fRKxnf+zbjuQuZZj6H0mgNslEyYe+5u/6UatAdn65rqksMKzGpe48rDrWOeYKHv9jNClhiOLZicGTVdsXajQZuc8gmIgec3pTJLz5EuCxm/vHmPEPWas4J5KLpqTbEIfuqmzl+zykPLLX7OIBWs3bWg7qQ+srAP955viutQkVSx0Pw9amMJTUR2zj0XoYFzcBsuNXAnp1Aw1u36iCQyFV/+IPGFLFni6+RRDuQiN1MoPj5qiu39aFIiA5FHQfo2wt122S/NbMr5U0DZg7rhhHk2s1NBg4MitwNQeXRhBhNddOm99i8Xa3/WcyX+00JmXUs9dgY+CAPKUI/+ixckPkqcAlISt3mN58Uo2xE9w1vmCvjMplz3v5GdYd791XzL7ykK11UZxfLU8Cu01DZiFO1wKzoR21vicK0lb57QbDmiy4iOyokuc+dVpijPTwZ3KI5qE7bFtmmzTetWq5oTNzhzQy3STJhv/RwZDLXSvaO01SLpBh/9iHHZNYDPkjPnTG3O7muGBArh/ySj1HTd9Ttioqx35dy8dKNIrBbDl8dItR61iLzacZ6lo/AcxJjlgRNrQgc61QTKOxIu0XGjJaYaRp2SRfWlFO/z6wJlT7z7LaQnMnm0MauNJ9rB+JoCzXJ0wD2EVopg8rkvF6aQFvbqRIsSzILzFk/IJjYh03ex/xil9SDxMYNTv3Yozh1NIG8UPsflZ71cNkitpIa996NfsTeaJXGrEsBaUDU59AYXH4oJylL+KV5RxnMAschuljt4yZ0dq24P4FBUAw= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1f2f8e1c-02d5-4634-cd1d-08db34036e7e X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2023 05:22:34.8234 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3va0khPeFN/HQs9xoXsiigLDI9gJ2EtiUo0uLwscOI4GgY64IrUkLy63zeq0OBaKYqG3GTbBxGYyrzeqRqpovULADgJ39+1Ezviu2Oqk0yY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR10MB7106 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-03_02,2023-03-31_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 phishscore=0 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2304030039 X-Proofpoint-GUID: rLVRmwNI7i5to-8xeaNRvZvYUnveTt8w X-Proofpoint-ORIG-GUID: rLVRmwNI7i5to-8xeaNRvZvYUnveTt8w X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: E4F181A0002 X-Rspam-User: X-Stat-Signature: im4p9dx7k8dq7px1hetpxompyyrqemh4 X-HE-Tag: 1680499375-278196 X-HE-Meta: U2FsdGVkX1+rcmJqgg9oi1lkuXkFVkdwVDTeP4lUvMmW+sKmQJfcXBX8PWzCnFDuknZfWBxs28c/sm+3VvQHIzRyEJL2VI66D21tTq4Qpfz7oEi1N7VWg/tV6vymPqoDFRYgKgP/QBjUO6LaHhsD2f6CecnAOmljSCQ19zIKiWsSvPUK1ZYlErSuksqm3bLbI7it+DTpiNRXsUtIZ5Se2H/a75IppLtjsGSor/LIAPU3gTv0yzu0fJtODeG4/onDPHEu3sH2qpel1zNg/WIDcmuJ/l/NS7xziP0LYKWxC3R6wWlLk+lh+r/mrtB/fTS4dXv+l28eMU4e3By6rty+3JNdoVJVMruvyQX45c97fwr0vEAQVqvBim8H3vE4/Euj4q9TZqQdueCmBs4cDmHkply3j4/NVmmkO/m4HHCH0LXcogHwmVgxnCiPoBgr25iTe3m/EkznVtBufT4iGkOyMJM5UyKcJ/l6fATeAMigqZ5E4in4Pf9kqA9xnWex4tv1d9841Uslhi/so+nS8kBiFfBGSyTRaQoBTEBGECTOVQ4pJiOw8O9BOjgj0hCiEJEe7kuPKIXfoIkkC+EX50HjhbYhl5qEZmlABn62lLfalTVVZ1R1c8gbMlprMv3jrhyi/ob9sQJLQIEKZHcEuJJek8LkLm+aB3lmVAjm1cp8pfomTCfNhJkr+IDMAPXC5zEvfq21BwUyxty2jGi2GBv9bFoRlTGdfAWmVXxlvmSNIt2ZakJrLV5682Xn/XAHjnNg3iCu5jmOoBRMz5DpocRTKj4b3v9AlsIXJSAg3cGQbkgfqiMBl0KqQZ+1GhrMMwczUjFj5DjRT08J8TGTR3RmK82PuZUaCNxnKcDgy9p1Wk5SdWeq+vzoEr4It7QBpwOLQB9PdIW6X/vQHtryQnIhcVS6ltsYQlxa+5W3xgNNPPlKQhBAfC7xXGjxQQ7FnZsI2AXgF6f+SsGAra+OQip Y91Lj2Su sh1ejD18f9cQymHirkI8PMmTRhhrH5uykFFzv0rQULEyhw63SD0nIoEtHED9Sye4FeBAAxpqBe69Vr/y2U3BXTeHpPknl4sJVZvJCyBuFlNRTUuZoOO1gIezan+Sab5AKTfDcLATmu3ZOWBnqTtj7RMSRsjZha7Jrvus8LFbiz2NBRYBWXQjDMCmb4LE8e3eU2aeaGgDlpM5JkvcTtaNorjWtNd93KspVI8X1kFCtT1XZIgUMm83AW1GeVeWeT/nTeSDnUybjtHrch1fkzNMdI5jN1eLGmKsU3iTC+jiXfY6uhjK7AemKDbQN9UrrSZmspI+FKBAdP365pD/rexQHSsrwGBjKms+2VIeWRrEugwoL9+wLrNzUl+utku2T0SmuOzBT8HKqEnuMl9wDwnuplyy3qM0QfeaKnEafNW74rq+/1e7c/4yjHDOcKOAYswOg0kE631IxegYW1v92q8OvDCqBaWczuLoWP0s/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Change clear_page*() to take a length parameter. Rename to clear_pages_*(). Signed-off-by: Ankur Arora --- clear_page() is now defined in terms of clear_pages(). This means that all clear_page() callsites -- which are more numerous -- would use an additional register now. Opinions on if that's worth optimizing? --- arch/x86/include/asm/page_64.h | 18 ++++++++------ arch/x86/lib/clear_page_64.S | 45 +++++++++++++++++++++++++--------- 2 files changed, 43 insertions(+), 20 deletions(-) diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h index cc6b8e087192..7ca3bd2448c1 100644 --- a/arch/x86/include/asm/page_64.h +++ b/arch/x86/include/asm/page_64.h @@ -39,22 +39,24 @@ extern unsigned long __phys_addr_symbol(unsigned long); #define __phys_reloc_hide(x) (x) -void clear_page_orig(void *page); -void clear_page_rep(void *page); -void clear_page_erms(void *page); +void clear_pages_orig(void *page, unsigned int length); +void clear_pages_rep(void *page, unsigned int length); +void clear_pages_erms(void *page, unsigned int length); static inline void clear_page(void *page) { + unsigned long length = PAGE_SIZE; /* * Clean up KMSAN metadata for the page being cleared. The assembly call * below clobbers @page, so we perform unpoisoning before it. */ - kmsan_unpoison_memory(page, PAGE_SIZE); - alternative_call_2(clear_page_orig, - clear_page_rep, X86_FEATURE_REP_GOOD, - clear_page_erms, X86_FEATURE_ERMS, + kmsan_unpoison_memory(page, length); + + alternative_call_2(clear_pages_orig, + clear_pages_rep, X86_FEATURE_REP_GOOD, + clear_pages_erms, X86_FEATURE_ERMS, "=D" (page), - "0" (page) + "0" (page), "S" (length) : "cc", "memory", "rax", "rcx"); } diff --git a/arch/x86/lib/clear_page_64.S b/arch/x86/lib/clear_page_64.S index ecbfb4dd3b01..6069acf6072f 100644 --- a/arch/x86/lib/clear_page_64.S +++ b/arch/x86/lib/clear_page_64.S @@ -2,6 +2,8 @@ #include #include #include +#include +#include /* * Most CPUs support enhanced REP MOVSB/STOSB instructions. It is @@ -13,18 +15,30 @@ /* * Zero a page. * %rdi - page + * %esi - page-length + * + * Clobbers: %rax, %rcx */ -SYM_FUNC_START(clear_page_rep) - movl $4096/8,%ecx +SYM_FUNC_START(clear_pages_rep) + movl %esi, %ecx xorl %eax,%eax + shrl $3,%ecx rep stosq RET -SYM_FUNC_END(clear_page_rep) -EXPORT_SYMBOL_GPL(clear_page_rep) +SYM_FUNC_END(clear_pages_rep) +EXPORT_SYMBOL_GPL(clear_pages_rep) -SYM_FUNC_START(clear_page_orig) +/* + * Original page zeroing loop. + * %rdi - page + * %esi - page-length + * + * Clobbers: %rax, %rcx, %rflags + */ +SYM_FUNC_START(clear_pages_orig) + movl %esi, %ecx xorl %eax,%eax - movl $4096/64,%ecx + shrl $6,%ecx .p2align 4 .Lloop: decl %ecx @@ -41,16 +55,23 @@ SYM_FUNC_START(clear_page_orig) jnz .Lloop nop RET -SYM_FUNC_END(clear_page_orig) -EXPORT_SYMBOL_GPL(clear_page_orig) +SYM_FUNC_END(clear_pages_orig) +EXPORT_SYMBOL_GPL(clear_pages_orig) -SYM_FUNC_START(clear_page_erms) - movl $4096,%ecx +/* + * Zero a page. + * %rdi - page + * %esi - page-length + * + * Clobbers: %rax, %rcx + */ +SYM_FUNC_START(clear_pages_erms) + movl %esi, %ecx xorl %eax,%eax rep stosb RET -SYM_FUNC_END(clear_page_erms) -EXPORT_SYMBOL_GPL(clear_page_erms) +SYM_FUNC_END(clear_pages_erms) +EXPORT_SYMBOL_GPL(clear_pages_erms) /* * Default clear user-space. From patchwork Mon Apr 3 05:22:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13197668 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB48CC761AF for ; Mon, 3 Apr 2023 05:22:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 034DC6B0072; Mon, 3 Apr 2023 01:22:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F26216B0074; Mon, 3 Apr 2023 01:22:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA0BB900002; Mon, 3 Apr 2023 01:22:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C7A0E6B0072 for ; Mon, 3 Apr 2023 01:22:57 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6CB80AB7A8 for ; Mon, 3 Apr 2023 05:22:57 +0000 (UTC) X-FDA: 80638935594.26.54E362F Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf22.hostedemail.com (Postfix) with ESMTP id 0EA0CC0011 for ; Mon, 3 Apr 2023 05:22:53 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b="Dn/H6HZm"; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=TUfpdY2t; spf=pass (imf22.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680499374; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gW/w4UBNoRyDr+fj9Sa/039P6gNwlfFSpI3amaMp3cc=; b=Qy5K8X/fz9WTicoGTrqRTHokOv2gngAQvRJ1bRfHhZXaa7QlLCvWxSm0t4t3i3NzV285wr cF3KBMmjRweASI53Sd8MU1koEUvlNORwNJaeZwq1todZ9QmTFWwhuM64hIK06UeXCyJGn0 NLDfxHPOeAsi+4zjuo29UIwL3LbbzsM= ARC-Authentication-Results: i=2; imf22.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b="Dn/H6HZm"; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=TUfpdY2t; spf=pass (imf22.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1680499374; a=rsa-sha256; cv=pass; b=cWC72u8qgWShUOrkKOdYuBOvLujy16rWQPEFKlh8jP8C7EfPjm0USktF8nsDe+buX9FkhM EFA+wRfPP388QnVLMR5TtWyu6TvNYMvoMeRwXgZJlY/cT2kuW4abvLkOAWHQJUTBL5WXlZ KrzLLKAA+DoKv5vAXmg6a+9+0XZMVks= Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 332Le24N009559; Mon, 3 Apr 2023 05:22:40 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2022-7-12; bh=gW/w4UBNoRyDr+fj9Sa/039P6gNwlfFSpI3amaMp3cc=; b=Dn/H6HZmJlLesKaHwksiUDif7P9mM1uzZUQbaX7ujbmfxhyibCNFgboF/dDK/CBCwqu0 h2QbvTUK+n1rQXsU6YoJwzJ1LeBU6ugQ+POVfi1CCeMxsxllbTsYWD4mCJJSd4lcySed MOhTtxMnW7vR5UWkfGDoSLAy057orTt5kyZ43n1A+GITTPyEPxd59Sn0HlHnShjkZovn CiVBxPJvoMsr/WIdfyKci22EVc/saxmgQzKI3WT4783X30rPMcYK1ZM2zE6DcY2WshJv fU/nC5drYb3LNlu8W+NyxEg94i7NPfGtMOMTBVpnPXnexiD1f9bR53BpGqZTlUJsa4ur ug== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3ppcnct4s8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:40 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 33358gdI028272; Mon, 3 Apr 2023 05:22:39 GMT Received: from nam04-mw2-obe.outbound.protection.outlook.com (mail-mw2nam04lp2170.outbound.protection.outlook.com [104.47.73.170]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3ppt243jdt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:39 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YgCfHXKDhVlODPaTrw5tanTx1s4DBtDjp10We2gX9y1WCAYc5JSVdLm9NcTQL/PsmaXGpDcUIeCsglKey1HO7oTR1GND7JPpIIFa3tvq28/LbNZnnAD2w5ST1cRFqRL9+z4q05CutuvCMbsqci9OFMTO+mzy8DR8C4/HgGiARxf6+ghonpShoLolKP8CbK2BTn2ineBgD0ZhNqUHTr+HQAEtjpTt0jmsyCX8fhAPPctTrLuEtOAqF4Ce1BXWAAeT7/T2jkJ4b2ZNfSc8y4yHG8frlq2YOvl3EpPNj27DeBzUyexe80aPe6hF3loFZSCwut8f52zxZ8K4PTjCgc8swg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gW/w4UBNoRyDr+fj9Sa/039P6gNwlfFSpI3amaMp3cc=; b=Y0Cm+uvbhKlFfPzOGyQzMU+r4HcyxXl57tCCL7aBTcn8OhTeR/sJeuS5jgLVAX/ydBhX8nCHgVckAb1BX4T8MC4gfJTH4xt3VMLjGeMf3oF9QOEVLjCNOW865vGKED4loLRu0s1QD/gn1yEAq0Yy3hiYts0EdHP5JzGMHhBV4Tfq+aLAMngaUo7aYlPRtY8PN6CCuCdKBDG7gZKm8uvB2q6M86Nx7RC9uZg+2SV7CDdgbqxnQp/ygjMxLakNJ9G+LHvP7EgQbRmXUA9U8B1axiCyzMvWREOi1NOvHl9jFDf9zH+KcgzWR7J8SWX0A38cH/sLqwOj+n1PBnLTCESf0A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gW/w4UBNoRyDr+fj9Sa/039P6gNwlfFSpI3amaMp3cc=; b=TUfpdY2tQMSA5eHHsYl5p6bY6EywNRdeKZjsmC2fKFyqoqO+MyeqKU6E7BcHFQ0J8vgYDFx3uq9YVTH/6+vuT7DPTFuVtRW+YzLijajZ7yuiAQdxIgZWgBaHA0ydlOzWuMdtsPUYtUc4G00QHExCXVdL3+MAevsGEf4VjuIuea4= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by PH7PR10MB7106.namprd10.prod.outlook.com (2603:10b6:510:268::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.33; Mon, 3 Apr 2023 05:22:37 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6%8]) with mapi id 15.20.6254.029; Mon, 3 Apr 2023 05:22:37 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: torvalds@linux-foundation.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, vincent.guittot@linaro.org, jon.grimm@amd.com, bharata@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, ankur.a.arora@oracle.com Subject: [PATCH 5/9] x86/clear_pages: add clear_pages() Date: Sun, 2 Apr 2023 22:22:29 -0700 Message-Id: <20230403052233.1880567-6-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230403052233.1880567-1-ankur.a.arora@oracle.com> References: <20230403052233.1880567-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR04CA0289.namprd04.prod.outlook.com (2603:10b6:303:89::24) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|PH7PR10MB7106:EE_ X-MS-Office365-Filtering-Correlation-Id: 25fc5ff5-c6ed-4530-c68f-08db34036fd5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TyFwhO7FrdtV/0UFCTJWxkzDFsU/vNJIVTciCP1I/JqKVaSZYgA+1t8RsdQr870dbd/hjMNcGFzL+hEx/Kms9eDvv2is2SneGKgPGv4fM5ZTgLXXek3jyKFXemjKOcwa7X6MZF4TuGQ+FoAJee64573fiXl0S9D6dLEK0Sxck3FKaBWVBQmgVdDc/zwTZpUj5DxafWRRedqiiOnqiLBllWozYFkmZRublZsWyfMmEKgCU38csPOW/mcNX4p46/H40SB46t5li5D8zb6kU+XFVTF3oa/G9iYuC8QhbJpY7FmvoH5us+WB2t4dphASX1sriblXzX+5Mmgd3VMLYZXeCgSjEmm74pSXHXawL8bbMHmSUsNgHA7b3Fd6nrx403dkYoKCjI1vHk8efgs3xxitry4C5vVhwpEJfX4NpTUk2ysUL89IWfVxBpHOfBideFJEN1tUnJjNZKMnSWCUVOtAVFO+Ae+DP/vg0os05zAo/C/wYnTUBsGPlHmS3HAiPaCDYW9i1mSIU4E0tSS37mOtTPWSFz1xUrtNmc0ARDWfdfGqjDd+Ws/qY5iqJk6pKo5w X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(39860400002)(366004)(136003)(346002)(376002)(396003)(451199021)(2906002)(86362001)(5660300002)(7416002)(8936002)(66476007)(66556008)(66946007)(8676002)(41300700001)(36756003)(4326008)(316002)(478600001)(103116003)(6486002)(186003)(26005)(1076003)(6506007)(6512007)(2616005)(83380400001)(6666004)(107886003)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: jOjgWgbM6lFjlTtRLSnNCkEMEwDZYEDYYjHrEMfAGp5ldXwtbaEOmYfQ6KGUUJwYfijNkHMz9ie3i5R9PugOFcFzQw54MuEYUS60CXM3NreNkbxzH9Sg8u3fSGv94wGVXne0Jpi+bixg3hl+5LTaKpY2TI7Cc4521nc+SD4X+yEuRGUkfmR43QuYIIjLiH8ku6FZN+V9E5jhcDY2bMk8T9MjtyNCniA0+bKZXsrqlNzJAUD8szfVoRnBlOKAsijKYrAuauxHrwEmunX5DzYpaVyyVxYHP4FQJ8Bw35TpY1fISlkrpb0hA1nNtQxIedW+750L0EnAFpU6XwDFo36uN7JqX4F8G2KdKG8vbA7aOzp+LUbt9zTBT9MzB8NMnEwDegPJfA8ygAjxZpw0FdKL+8qyl1bsj9zu1uh40tRAVsuoPwBVXoqQf01/j6xhNIeDr+NIEtwm0uiTJf8WA4tlVHO/BJ7lLaGQ38c8pk42oDtIrwUqb+cRv0W+dEnUdSWltfeeA9ql/KR2Ngl1OfaV4uWVN1y8FcKBwDiteJpQeiMjosyTCpV4GRQPDCaptbUNfIIBjbWwH2xFrPcySmznDmQHKVRQaAYb3Lyg/4AfSzj4nIUkrYMo5XLij4USuBPTDO6qWADgDW7eyB3fZyIwiO6ltyUDkUrc+B5vbRzqh01ayOoWnhAStXgTP8W/kt89CczJKDnZplLkFDXPeCMXeyCttXTLYFgrWj7zCLMTveHUBDqR66q2X5Fc97l31nyIWjfiHn4GLepBaz56KRDsHN4Aoo+Tjl4BxDmVYo9DUi1k8Hl1CvotE1aUe/7n9YBOdn6SXpDaSpaqN/Ovjn/DslDaG76Ssl9jP1i8WexH5XzcJi0vBxP7fz0E8ncZAGfMikQExkRGXM+tvBzoJhmxcVaA8icU6Qn7zPguA3g9H5RukupY1w/PKUre22FvU5xCOFY47je35d8zSlUJJ0LNpqsy7gvZcgeB3mYy5kp2JiCmjzXnt6Jw26nC2ml5RIjtEpcIH93U0ETqIuFhbbYYDLsxQ3Edla+cQEDSIav3kG/sPBoP4tzrgHFkPsPj0SqyXPsnrIVE0gwZAdMst/wfR2IZEixV8sHY2phU0yk5/+DCOrZdK1ClwCuR5Qr6xuE4RtGIcGY2MxuUl5BgIDtu9A550lCJ4/I271lt7edHM0yiP8rWUz2VX5pgAY+2NiIcy4UWhyQ27UplYyEAdQdNrrDeY56qS5KmA00FpclsawSobjfXQYnyVu67IQknsKPJvn3g8ZFRb3DKDBpt/87+w2ZQCE30uhNA81TlpjPNOuVdOwoDiSPRqPYgRdP9pOQmBEElvfRLl27tWxCvyX8pF+iSyb2YPblJjpQYE4EY6FSHS0BBC0ZH1DxfK89FldvQBU8LelasydHrg4vox2QiotEpZnWqQo72cWkL52UW5oLRpT6qUOJhEhHQy0os2XU+c6iNG/lkY+72O5pcWUCYu2YmexHVRgvbmfOLJjoHAkCAj9eU4SfWPYzQxdflbaDUGRm34eWgJ/E7ZfEJ0RkF1NUc5WwzzzQipahcXjT5IKYOmuGBJdPzyr/mpMp1QhVc X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: YLWSoUs2F+TjUZ9vlm7/o5qTYL00NfPC7EIcgf0lJhB9bywpH/JoKBV3IssJWP5Hn7VtDYJlpajko/SsVKZVzs/0XlZaoET8w/Hao9RXfyqCtMjS8n1GJnIm8vD15iC8ZoxK9GXfRptzv82puZjq6zPMP3lZlOTx0MM12sJgp4PtaxFiShTAyPPj+aEnid9y/FBNwp1BMrbDstIeBsU0AP8i1h6eTibAKytVL/c9dQQ4V4a57pD8rFlReQk3Tq5YvAofAe8jCGDtdCjW9rGQs4guwb7BlCXqbKxq9UXF89siHX4U2x5AX5qTGZPyUOPaQLzGwn+sQiyy0zzs5HuiITV4lNWEDymCUmgYBqY2kKhSx4stWYtAf6eUNozZA9127tTrQUw/0uSr0VrzTFksLi3oUFW+nk3zIW0k/1DVbPgVFxNGYRWiQLLT2vgptFGQxdD3iR+igqjOHLvaw+PeDmE6l42vYidp5VMWoaEoJETr91CWiTuQ3ByvBYbR8rMjZf9bqZRVGL9XxbSgGUYzfKBniMbFivytFyaWOukn8IyixkI26qecaY0cyF1pKCumIGoO1+EfUGybyOvYhUVfpisbxZG449B5s5b6PQbGu3nQtqBBh87UzOx1vfBgHWgrLzxTvUC+j6qWEoX4E5H84GS9rnJuFF5DxVwwYUv8PLfTMap24i6ZXoCp56Z6XYMEdwXQGXYjRwW46++//HL0Z4nA1GOcHvNiOI836I+rS6QzLs6RGcsGK3yK3u4TVCPMRS8LosIRI5j1Psfe/yx29c/SlVKvkTaF3FSzIr8L+TGQSm1D9cI1Si8NcF9CpIl5CV9gMvVsKZTHjZeurOfpJUvyv1s2DBW0XIOlzcLPOjLusnKHpzVv+97aB2lQCxhGTI2qg/i++qYRoC8s/2cwG6gYsN/dSAGiLZYB0wQRwHvSuZcOc6kj7mBoKB6WDpv5ObFD9MvU4Ua71KNhcP4CuV6Rbzv5H4MBCOwcMQfgS89ywpgMaNd6E6dJlCwpG1F886oEcE6sl2n7VUtA7ZElvAWMv6dye5A6RIKrIoFV5Jp/R6WGpHOgU8DF0Mvw/axvLFIpxoM4cfihGGRvjcm3cNFjKsV+fNre3xG3K0wJMY3bzy2OiFpWkq75V6JUFVVzvITfLw95X/echc8gKfibkTcWanAMreL7SjEes4z+KjM= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 25fc5ff5-c6ed-4530-c68f-08db34036fd5 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2023 05:22:37.1043 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lFy+bbT+hkUU2yGypQNLEk/z1osGbg4aI0nNHM5E1Q1+V/ApZpOwebI0gzc9Zj7p6CDcnbTKWS1iPhvjSH61bzfhFrgiE+iSeUGwwvzo1qM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR10MB7106 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-03_02,2023-03-31_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 mlxlogscore=999 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2304030039 X-Proofpoint-GUID: ajXacZ1gH2f6bdCwT9bIRPZ4J4mSVB7H X-Proofpoint-ORIG-GUID: ajXacZ1gH2f6bdCwT9bIRPZ4J4mSVB7H X-Rspamd-Queue-Id: 0EA0CC0011 X-Stat-Signature: szncijzss754dwfodj6ysf7js58n3un5 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1680499373-404181 X-HE-Meta: U2FsdGVkX189tSeiburewnfaM5kdaqj5GQHpi9OiHF3PTpDWUobDRbH0wuGpx/sQEN2eCh8BBydeepw72cv97DMKzZwcl2+TLzIzeno1Yxtox8pJlSiNtzDp3NskIAQozu15eX93K8kaL9KWibMBcUiNPmARRiQtqhtSXdEXeb7ufe2ntRnQPDIAvNcSBL0AacaWs5A+cnzwogDez56YCDDjs980f/Q2rBcd2vIkbQaE/2Zt3UXQYhqL2v3jHcLn1YnGSyudI3LWkXX0QHdaZL2oDISaLc4kvAZ161xtffos0vP729t5oALEWXwiHFL0dub971bLWfnQ0zU5NJYj2bY7SeWPJbxM+B1JFYZpSBzYHoD8bepxoDR8w8h3g2H/Nik69EToD8eKbhUvCzFVCOJ4QS3OqkX742H3o2R2gWcgUH+qexmg4+/+RYgeLNSafKOQ87chB9X+wRbUF00k9UtzokK4FxSWLkbjREPELVEcb/sJTsVpAaju30ZTsT1QmUpqj6vzhUoE64M1m+UEYgUrHUGEq5ep1USPRlRV9ZDGOQcaciJ3O7V3D6oieMIf/l6RRkadyoV8sv+FwFeWVLhmEXpm0ORZTaxKLKfvcmld3tvlN38wWyUWPyOCpmgzR6K0kfM3GdJ5v4FpY7CsfpthVo61Go6aFb2cauU/LKqJSE/pznP4CgJfAhL5VYr9jx/LaLdkOVSsD34jVH8Q31KyPHdKUDQsMHdbcIHEywKIQoYPNZYw4VFbgm9d1HuLbSviZQUkOF8+VAyJVHATFR6b/y/FLp6wQzXksQabcsSHHFySXOcOcHJApaxe/zfgCVMcC0IlMdriX62KGHioFWDhCY7ehQk/UehAFYpv/6LEJPDhEHJVYXyhD8duDc+0C+B0J3keI4TF0IYntg68rhtazEywAUy/p32t2iWDAN/0r9abxQ9kf5jVzP8noHfqzekoBriHa/23wDHc62H 4NR4uog2 JY1kp0PHpx0q9OCCsc1XcIrCGOsgkcQqgUIfAgMHd+va1LYwc3pI4LtygXYEQk4OF3pqLcQaagBQj2SY6ijbOBEXmwgOcDfvkRESzEVb6xNzYT4wbkNgwKwfebnZYUFdrdkdywT+c0i944QXhP4mMrVpWKcbtcRvWrZmB2F3/TzbvHzBuSNqpqBMSpk4e9y1SBFRiCpyLjaNO0eQRcnrxtnO2JVxXmPCy1Z8qjqtYHa+Eg+23+bM8Jspk/vf9/J8lN286RGhXhOiJGm+qrNxL0srVxJov6Vt7i2KFOxV1kH8IoNKo17ZgoqKh02/CEyBPjCkbJ7yFWLIOoQjhlu4GSwDv/pKDWoaaH7Qhcb/hVQ63hPb2GXz3jU6KWl+ng4XNR3MkyBOhkdvuCUqOrb40u7v71Hb6sS2UH9K5WE3lxN90eu4PymnIZXRZCjhd7vQGhJI4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add clear_pages() and define the ancillary clear_user_pages(). Signed-off-by: Ankur Arora --- arch/x86/include/asm/page.h | 6 ++++++ arch/x86/include/asm/page_32.h | 6 ++++++ arch/x86/include/asm/page_64.h | 9 +++++++-- 3 files changed, 19 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h index d18e5c332cb9..03e3c69fc427 100644 --- a/arch/x86/include/asm/page.h +++ b/arch/x86/include/asm/page.h @@ -28,6 +28,12 @@ static inline void clear_user_page(void *page, unsigned long vaddr, clear_page(page); } +static inline void clear_user_pages(void *page, unsigned long vaddr, + struct page *pg, unsigned int nsubpages) +{ + clear_pages(page, nsubpages); +} + static inline void copy_user_page(void *to, void *from, unsigned long vaddr, struct page *topage) { diff --git a/arch/x86/include/asm/page_32.h b/arch/x86/include/asm/page_32.h index 580d71aca65a..3523d1150cfc 100644 --- a/arch/x86/include/asm/page_32.h +++ b/arch/x86/include/asm/page_32.h @@ -22,6 +22,12 @@ static inline void clear_page(void *page) memset(page, 0, PAGE_SIZE); } +static inline void clear_pages(void *page, unsigned int nsubpages) +{ + for (int i = 0; i < nsubpages; i++) + clear_page(page + i * PAGE_SIZE); +} + static inline void copy_page(void *to, void *from) { memcpy(to, from, PAGE_SIZE); diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h index 7ca3bd2448c1..42f6c45206c1 100644 --- a/arch/x86/include/asm/page_64.h +++ b/arch/x86/include/asm/page_64.h @@ -43,9 +43,9 @@ void clear_pages_orig(void *page, unsigned int length); void clear_pages_rep(void *page, unsigned int length); void clear_pages_erms(void *page, unsigned int length); -static inline void clear_page(void *page) +static inline void clear_pages(void *page, unsigned int nsubpages) { - unsigned long length = PAGE_SIZE; + unsigned int length = nsubpages * PAGE_SIZE; /* * Clean up KMSAN metadata for the page being cleared. The assembly call * below clobbers @page, so we perform unpoisoning before it. @@ -60,6 +60,11 @@ static inline void clear_page(void *page) : "cc", "memory", "rax", "rcx"); } +static inline void clear_page(void *page) +{ + clear_pages(page, 1); +} + void copy_page(void *to, void *from); #ifdef CONFIG_X86_5LEVEL From patchwork Mon Apr 3 05:22:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13197674 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70627C76196 for ; Mon, 3 Apr 2023 05:23:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B820D900003; Mon, 3 Apr 2023 01:23:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B3000900002; Mon, 3 Apr 2023 01:23:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 95AF1900003; Mon, 3 Apr 2023 01:23:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 82118900002 for ; Mon, 3 Apr 2023 01:23:00 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 5AC138095F for ; Mon, 3 Apr 2023 05:23:00 +0000 (UTC) X-FDA: 80638935720.09.BE0DB2E Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf10.hostedemail.com (Postfix) with ESMTP id 0C778C000E for ; Mon, 3 Apr 2023 05:22:56 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=lppQNTdG; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=rsOWDWhC; dmarc=pass (policy=none) header.from=oracle.com; spf=pass (imf10.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680499377; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j54SMO8aHZZZsgBi7r1t7d7dsJQz4KL14GWBankHCBc=; b=0TWWROGT31rVmgvFZ217/ivDdu6q7nMfw7U43/sgRNRrQLUeGfsWiAhKMYQQCbNenc38C/ s1bPfhLEGF8TgvJyt1EAdPOIbKLOWqnOzA7bGlb8YHT0F5qpH2fyEYWWHepx5mFowc3pW1 SvcO/zBCjzQeC7dQwkT6QRdMPZN0rUg= ARC-Authentication-Results: i=2; imf10.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=lppQNTdG; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=rsOWDWhC; dmarc=pass (policy=none) header.from=oracle.com; spf=pass (imf10.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1680499377; a=rsa-sha256; cv=pass; b=TJcjOZuw1UXhpvEc5Rc7AgyBLNsHsgGfCMYBV0DRm9a5C4wYzc8/zmnHp8GHBblNHKHmG+ 6vSEOY1vPhBnelWy84IIJHD+H8ZH6+30YmMWFSkvjNcJb9+7vZKDEygODSMBsb+H+epNo8 CkH7JGAqveHmkgHLvAQrihoaYJD8ZFs= Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3330qGbo020222; Mon, 3 Apr 2023 05:22:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2022-7-12; bh=j54SMO8aHZZZsgBi7r1t7d7dsJQz4KL14GWBankHCBc=; b=lppQNTdGZtE0ehbHr57NIjPQfjX3bwIRB/Y0grP86aC7vC6mTDe/8r5J/HiKIubZsiaK er7J1ohrWApuGNzCsHcCm+p2wIC9jAsy+Ipcih/ORquQ7dmNncTR1r9MQ0BAO6jT19Yk kTrPfLDvEwoiABFzDnh2ec1j8iagTtvpirk0VA8eO0+znd229SP31GlzZLGrgyiF36pA SME8AZFMRtq52+CgB8TYYt/VuiNJpiBtLuhAUgjj3I0r71nsO8ZKpO78InXHNBxggdmc mjVFWsF0U4Qlhl1FR15mUji7rWy6aJPYT59mNSqTrDKO9W/i+oHDvGlC+IeGxmXNX+mK VA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3ppcnct4sa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:45 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 3332NM3i028114; Mon, 3 Apr 2023 05:22:44 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2106.outbound.protection.outlook.com [104.47.55.106]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3ppt243jf8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:44 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HLx0Ec+40KiTSJJWUXk2QMZFsmqR1BwZ66ZexSTG1z+vBWlWGtzJ9Klhbi/mTiytvyOOvvReyiLgl9OPmsnFran9yjeTWVO2xjsQkx8x8SGFyfWCuz8Cm4DSJGx5alwRI1jOt7YjSTMInSZEKX2YluOuKGY5SA9nbjMkHEkcJtOp5BpbT6+QSkcyg1YJ6a6GS7nE07VpsJEwhYvmpWBzOh+TEAUZNwe71W6i3MBfA9PXmpxkbmyHvcm1jwF5TUcwZZqzd+hjA8FPGS/aRhwxOkLHHissxs4x7AR12/Y3Pl51O2c9e+qoePjWJNttCsEUAXtrx9HUZ8BFZmbVwC6sDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=j54SMO8aHZZZsgBi7r1t7d7dsJQz4KL14GWBankHCBc=; b=FbVPuUSqf8qDw6A2V1Czu3Mk993iYLy/sO2KUwpSVAEaRHq38bB03fsnQMxLO8cP8PmfbSXcUb5IedaoOwP6aCeqrIx1cwm9e3O6krjRXrpIJHRjRGNUKwR4vETIReNaZg7CMjU6OueXRfLA4wx7ssR6wijmWc3FWq57cqBghdHR3U9PuP+DpqqbiZJZnlvsRa4zCoCJc2NFDPJitoA4Nvicb93XDemsVpEWMzPGRW9gV0MgfysxhlwUfIsnu+5SVX+kdLhJEAgkCHEwmQP7S+iAWDC/SC8Kyv95QNJeOGMwHOBZ6aqtWSd7EMfOJEuBsD+ZzBL55OYfp3pRXFi05w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j54SMO8aHZZZsgBi7r1t7d7dsJQz4KL14GWBankHCBc=; b=rsOWDWhCY3QbgShl49gkoaEKpfnDdPghQYRs6hg1QOE2tNKSKNifGD5K9mq78qxsJXh3IaGsm42DIfNA/YP3LcXqaMxl+SeFWbYBiBqojXbvpOq0bjVONt4oo2PINsxwN9MZDTPwscNunHaEcCRAttbma2evFksAWXu6hHLGp80= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by SA1PR10MB6365.namprd10.prod.outlook.com (2603:10b6:806:255::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.33; Mon, 3 Apr 2023 05:22:39 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6%8]) with mapi id 15.20.6254.029; Mon, 3 Apr 2023 05:22:39 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: torvalds@linux-foundation.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, vincent.guittot@linaro.org, jon.grimm@amd.com, bharata@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, ankur.a.arora@oracle.com Subject: [PATCH 6/9] mm/clear_huge_page: use multi-page clearing Date: Sun, 2 Apr 2023 22:22:30 -0700 Message-Id: <20230403052233.1880567-7-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230403052233.1880567-1-ankur.a.arora@oracle.com> References: <20230403052233.1880567-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0315.namprd03.prod.outlook.com (2603:10b6:303:dd::20) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|SA1PR10MB6365:EE_ X-MS-Office365-Filtering-Correlation-Id: 2abb81d4-735e-4473-77db-08db3403715f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oUltraz0xtWPZn3Uj5/EngPUTJ+kcPgAYtWU08DjJwocetxXw/OPferKiVhzJd8mOl9VLy1c1I55KK0+WcTM0ayPI4vMsv6IAEyq8PbSrSpzj19Xafahz0UQWziyt5l7mLeUrnjL7gGMZZkBM3bx0ZQsD3SqJcyr4aLpkV01fso/hA4RJrWPfUDhXFaCEf4tIYchu/z0Mpuv//pzhDlx4iIi+lL1jVTthamtqS+G0oLrSMKFmECuk+QfQpc7NG28fMv3sb5z4MVzmG1Gs/VVkvi5d5fuZPnBlUlyKjWod6Bp/ERAsiyYhvimBj8CvMx/dxGm0LlLu//SPn4dy9789PhYrjlA3dSFJPepQ+UvlvgSe3zL3jyvfRfwvatbiCEhjjo0xjjvRcXBQLwx4tdtpytvm6UVP7fYmSfuHwMEX4Kvt2NlSGv8+sKM+FPBdri4JwqoSrSOYWhoDPNULI5zj5ojVbTWyzvbIcxD4y5ts/Lzq5MSUC9u+lXgCDVKvOSHUEdI+z+jGOQd6YZV74YP2E0Q90CXeF11U8kwEzP2QadtryhUZhCPJkbfEGomI+f8GTansnfpMDpYjkguVVwnJ69BeFTNPAInqigCtIjRFe0= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(366004)(346002)(136003)(396003)(376002)(39860400002)(451199021)(86362001)(36756003)(103116003)(2906002)(2616005)(83380400001)(186003)(6512007)(6506007)(1076003)(26005)(6486002)(107886003)(6666004)(4326008)(8676002)(66946007)(66556008)(66476007)(41300700001)(5660300002)(7416002)(38100700002)(316002)(478600001)(30864003)(8936002)(14583001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: LDwZaoxviroa5oZKWcjLk4+gKaJkaTgn1Tv7iWksudtQqRwBOXztjUo6ZBsYygygzZLMbpeW7v0BZm9oM3CjDfo2VQzMPKO2P+Oca2nF5RU7W4jrvcgLzNAnrWxbvzmulCvyAJUtiNSIL3EwH8hdru03fUFxhb199R6Z5KWOrU0+BZW3XbUGisZvYsBd4waNZcrzKNq6SGaTLaKYjWFB2YTOVzQxHcgvUf4QLAjkYKWLu2Ke2KE2219HPA1YxGgswDUHVPL+iqNpH5HPLkUfzGUUEPKgyl2b1k/9EppRYA9KeOfb1UfnAp77+dl4ZrZ3+m5KNpqGz8TTl3BbUWvAxZHOMHuhdPtI9+cx+ruLYcKewx7zb71lq7Mu7FN4JCfjomZIG1xuFEyk/938QyVCnOJAQYLr+dU+d/sOqWcmX+ACIXgQDBCSvv5icmXODcTH+3GlMHjA2DqgLNG/C7VQ2HV6fmEmR45GljiKGctMAxW8G+C6WhEBWneoWcZfLp/lceRDNTilqpgfJ8lP1pXPFOskFsJE3H5vQFv4mqpn43kQXNyBRRUbXkA/HNxF5m3DtRhuXqU4sFeWUH1zlz8RB4CiWvE1N3q6ZLfbB92N4CLw1k5AWZLZekidy73cHsNjySTyn57PCbEnR9sAJLNnOSV0TwEYShihHJP255vNTPO+pO65Y5es5kjwMtVOaZi9mAUDF/Tuu5+w9At5EQu846YDeKlBj6QxZmkezFr/SC3dKlp+81ovQA0z8+yyV/zj7lWwcZ9Qvutcc/CkQOPPS5e3IwBnzWT/ehgCf5HWFgj1eIqTkvDzZHa2tX3yTaNTJ2H3CJy7zms+YShFDm4i1NgWmfZty98kWs8kdbOvCpaivhSMgepE3mZAP1CiGF64rBWRZeUwZHYWD+GWXTsYXLv1RvXDJgATqEOfdcH2KKEh0EBpfaGdzLodNdiHvTUl9iMItgj2ayB6QwFRNAF2Os9AjRbbJY3D8tZzVQeT7FBcWetQRz8MNtmEyiQPZbWm7n5NvVk4htMRSlZAT8GlQw9iEsEqp3Egg3Ce8Bg0HOT6wx2suTrOB/ZXl4lOhN4DSOikKu012PW1i42wsNa3Gxr7U9aCDQXp6o6cFlN1UaTJ6XaEKJ0MHrwWF9dj7NTVVGNbO8QktpIXMJYi6WryCOMOHydJhJBW+o4klpAyylVicmUAv1usOu4dTOpS3W5A4dBaDSpingMzIZRsBYrFhR2VjtNBgg4OY83BzuqCWA8s/FDwgL9qWZarYHFBtWJyGNKTq9b1cDjzaguKfkxOIp7PS6GHqDvOHQGGgyNeC+SB5gHyKCMMFCSLKO1/l2NTtxYIbSpj/sQ6WOyhFpsTOWTgikcuAYXEMl3JrdACXzE9KaEQ1w87IzA2wXb+3vvljBpBdSsZAqhjmUKWrwJ7HJKLixyLdhH8UcnvyL/Q0q6MHtP5tBhpqSlrBjEiQHUFA2cxirpbGHcqtI0wYIuoJVmhszh9COk+O9DdZ2iWlvYvKj9+bCWty6+7fihtqDo8pg6IEO9V0QRpLayLgrdM80j11ai7ArMrw2ld82qrvIY4PEomVxvUeMTa1U/ypICh X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: jC5u5HHWWcM+U3N7pE9x+2Vpo1XxWleztbM+IsfBIR/ncJHRueiqIggpefANnBAaM+E0I0ExyL2fcXtb7pVfDjwvX8r6MzYI09oCbCoeg/zDheswn5f1dHRM0LNaxUoiMgtvQvMVnw6DF4Dt9fvATXmOq8QdBMjfVDYsrsQYl+txcqQmJCpMTwxYT0+ZaD6P3iCSyA7vKKllzgumCTQKqAGn+dXDXUgaWn2zFN+L7FYUfke6/NhVxH+pkpXsYToQbED9RB9j49Rz1gORr25/5ec9u2iLhTCjyvmWqay6DuNRT15NhwOJaOltskdE4Sptn+BIfONNqQ53sNsd1smJ0jhxl8+DAgFtIg4IkE2l8ROgznedsbqtZSWsqNGZ3gFvYLg9TaPhXJLSpxOqREkLP5exKWm3888r0qD0Vu3lbKzF5REQKnNvAm7T/oEfwuObt/ltZLheiui6HT10q8SoHgc+5XzEi5r7iZE6EZqFKd6zNpXHjSILMwSnFkyxjAQEwZ2UFvZWDlA9zCSK5+pRaQX36BplOQLqIQzQJCA2+UA+01bCiTV7e3Afsdu0OyuGAKsGrgTXjjfhmWZZ1WTd8As186F9ar5yZ4f8RqDE5quMhuw7fuOhVBc8N0Za8Yi19PNdzsOiXVAqQ5OHF/qpJnprzKbYzK1VdCgh5dxujI+0n9TeOYpZaoIGwvh4Gq7M8QWXJeXBd+9uBfh33aHBEoNz/wksaLqfjKsDl+ZY44Dladv3P4ag7ifiCrpV7Uo4lv4EvnffyaTunT4xi8gFVsAgvB9TzXQeR/w0Ln7QD5f6vYx49GflwMumf4B8ZqYYMYex15Njrs9V1XwErxNkpeeYzEmifaywlpGu9YZaNRRy4WQP/h2DuojZIWOmrnr/NBa9puV3ImDG2nVQ07kb1P7D2Be5K+ckmiZlIf3QCfpf4WcE1iqHd0MIMyJ69+n0KSAQf0HKW14xrbLwidIYIq+HyUtSJlMjr+a/SwetyznwByKFmxgYcFdTi1QDEsYczsOd569b3oPfjZJtmpK8fbHUt0vfSGF5IMnNnHmcOrCoLAjhNuc62++zr4480ve3vzSOLtKGfU5w7RfakQv/WMiLmE0Yw6MZafPGMCsI8h/zeNBZO95rpEvLjZrzYWhYtpTjfEOWeMDSGWNXukl1L2G/wvTeW85leIpkhn2A07Q= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2abb81d4-735e-4473-77db-08db3403715f X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2023 05:22:39.7274 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2MdWZq9BlrhY8Ydt9i6R0oplwyjsNqivR+6F0aE66sPzJMWuB23Eqm7AdmO16PfdzWUtDeCbR2o1BUC6B+Hr0dFbh1MJm3y04n2f/uhib4U= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR10MB6365 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-03_02,2023-03-31_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 mlxlogscore=999 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2304030039 X-Proofpoint-GUID: OzYmjE-GglT-w3DRsHWC62p0xNTxrbmF X-Proofpoint-ORIG-GUID: OzYmjE-GglT-w3DRsHWC62p0xNTxrbmF X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 0C778C000E X-Stat-Signature: w533nmh8pgdidxxxy38j1cz9jb98jcns X-HE-Tag: 1680499376-463918 X-HE-Meta: U2FsdGVkX1+b8/YhSKW7qDLP7aa/xNRlMiiyYeh/rAd0K/EvkuxvFnfAg8SVhUdiVX7X+YlrTrGqwKo+Jlr7jimPMFMOIMipTSpAVcz1oZO9VQwdnljh5hCX+CgrmQySzEMud/2UHZaD3THnjqYYpnFawE773bNumrvcQ54c02Sac0C/KsbSaUQEr+aAchgfe7p1BlEQh4h+3euVTnfxZMIUfesO99BhAIQaf2R5q2LbVkuVRvsPNd/6TeXhOJduPKpClXh6gAJ/PBPsd900MOkRBmsYjMA2gA6juNCtbzOHiCx1YHRKZuuOCnV7gYb/FtYiJAmPGW62VQuSs+eHBZnnzVYStjNxu1+UYk3gvNE1Xptxv0W5E5rFvhQeJHIrB7XkLi1qJYwAofOvVHeiheUPcD6DfHPJXKUxhDbiToLfO42uffoJCTXB6pkz8oNmtAKSk7M7rRg3aV0MD+WV1/ZsDkvswRDyCSr9jT/tsCFX5yJFb52XKjn3OvccMpV+i96ynptCqj+3bskdMGrxLdIsQNq9xNO+8rtlA0Pnu8Sq5UD1Lvpymyis2i7Bcu5p3+hRIlftaivcSJGeadL1RWJvW0bvGJqwHm0fs0CeDWV7XvePZeBM4jInU/ftJpbQZkeS8zEM5pcXTld0r5oQM0f+zNRbDMeUkQpIlP5emmZWn6hcDyMv93VfSJVtQ0HET0L41E6qydgflAQg1mDrH1g9c1gyDrSTnSFwsRzTRQSVa2Y2fuTAI6pYGOqlIVgOdvqlvJgEx721qgC1RBB8xgUIcvHfnqal48YKHMBd3aXJ2wt8X5dqMtvj/O1YifO/8N+/aOok7cTfjXl9XPAJXmmP2seoy7ZkCEm5OkvCL4GVR9twAy/Hsn/HSUskzoSeseNwhY6rjfIv6fhanl+9zKBpdMinHTT/wVyjh8cWkdlS1dtzMPo1Vw3swQPe6wn0RtyIwCo/DgGAKLn3H13 nlfNXOBI sMwfxoB9QZhHTt6TTdZwVRZ4+k/rlRKDhssmJY9sFIBamUTLDh8/HuGxIOgBu4joTrAZw6gSDEblQvdTdW7VjdJapsXCf2p2Qr5EDe3XerE2ZJMylaUQGZD9UMnBP8N8lB4nNgMFc/0y0p//c0UAq+Kuxk+injTON4qzVZF5w3+FrdLNJBRUQu0HQYmS4QBbAv+wun5fnlVQDOt7g65uzp2MMrxdPmz6bClNcutA4aFcJmo/UHBQZL7duBLZioPO3rAwMQDHyoZnqnd8Ahd9Zu+d4HpHM2k6ru9CdseUEuzhuqxFNSyqkhkRkARud1qIdCRWJIEVPOLmCuJciSTtrseonfA+EIUyNcBnOWQifAn1F3iWXi/zHcSLHyOOfB2Vi4T0km2xVtfT32JVvI0UatNUapNSxmHck303WKkwJzxRXeo0GWRaOKzIvDoamdeIee7b6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: clear_pages_rep(), clear_pages_erms() use string instructions internally. These, unlike a MOV loop, allow us to explicitly advertise the region-size to the processor. Thus, clearing in multi-page chunks means we can specify the real region sizes (or close to it) which is good for two reasons: - region-size can serve as a hint to current (some AMD Zen models) and possibly future uarchs which can use this hint to avoid polluting one or more levels of the dcache. - string instructions are typically microcoded, and would be cheaper if amortized across larger regions. We also execute fewer loop iterations (ex. a cond_resched() check for each page but those instructions are likely free.) clear_huge_page() now clears in three sections: the local neighbourhood of the faulting address (faulting page, and four surrounding pages), and its left and right regions. The local neighbourhood is cleared last to keep its cachelines hot. Performance == Use mmap(MAP_HUGETLB) to demand fault a 64GB region (on the local NUMA node): Icelakex (Platinum 8358, ucode=0xd0002c1, no_turbo=1): mm/clear_huge_page x86/clear_huge_page change (GB/s) (GB/s) pg-sz=2MB 8.76 11.82 +34.93% pg-sz=1GB 8.99 12.18 +35.48% On Icelakex we continue to allocate cachelines: pg-sz=2MB: - 701,951,397 L1-dcache-loads # 47.985 M/sec ( +- 19.22% ) (69.23%) - 3,239,403,770 L1-dcache-load-misses # 691.17% of all L1-dcache accesses ( +- 19.25% ) (69.24%) + 194,318,641 L1-dcache-loads # 17.905 M/sec ( +- 19.07% ) (69.25%) + 3,238,878,229 L1-dcache-load-misses # 2480.93% of all L1-dcache accesses ( +- 19.25% ) (69.26%) pg-sz=1GB: - 532,232,051 L1-dcache-loads # 37.378 M/sec ( +- 19.25% ) (69.23%) - 3,224,574,249 L1-dcache-load-misses # 909.02% of all L1-dcache accesses ( +- 19.25% ) (69.24%) + 22,587,703 L1-dcache-loads # 2.150 M/sec ( +- 19.38% ) (69.25%) + 3,223,143,697 L1-dcache-load-misses # 21478.37% of all L1-dcache accesses ( +- 19.25% ) (69.25%) Milan (EPYC 7J13, ucode=0xa0011a9, boost=0): mm/clear_huge_page x86/clear_huge_page change (GB/s) (GB/s) pg-sz=2MB 12.24 17.54 +43.30% pg-sz=1GB 17.98 37.24 +107.11% Milan uses a threshold ~32MB for eliding cacheline allocation, so we see a dropoff in cacheline-allocations for pg-sz=1GB: pg-sz=2MB: - 2,495,566,569 L1-dcache-loads # 476.417 M/sec ( +- 0.04% ) (33.38%) - 1,079,711,798 L1-dcache-load-misses # 43.28% of all L1-dcache accesses ( +- 0.01% ) (33.37%) + 2,235,310,058 L1-dcache-loads # 610.770 M/sec ( +- 0.02% ) (33.37%) + 1,089,602,355 L1-dcache-load-misses # 48.73% of all L1-dcache accesses ( +- 0.01% ) (33.37%) pg-sz=1GB: - 2,417,846,489 L1-dcache-loads # 679.753 M/sec ( +- 0.01% ) (33.38%) - 1,075,531,869 L1-dcache-load-misses # 44.49% of all L1-dcache accesses ( +- 0.01% ) (33.35%) + 31,159,378 L1-dcache-loads # 18.119 M/sec ( +- 3.27% ) (33.46%) + 14,692,358 L1-dcache-load-misses # 48.21% of all L1-dcache accesses ( +- 3.12% ) (33.46%) Signed-off-by: Ankur Arora --- Fuller perf stats for context: # Icelakex, baseline (mm/clear_huge_page), region-sz=64g, pg-sz=2mb Performance counter stats for 'taskset -c 15 bench/pf-test --sz 64g --huge 1' (3 runs): 21,945.59 msec task-clock # 2.999 CPUs utilized ( +- 19.25% ) 34 context-switches # 2.324 /sec ( +- 20.38% ) 3 cpu-migrations # 0.205 /sec ( +- 19.25% ) 198,152 page-faults # 13.546 K/sec ( +- 19.29% ) 56,513,364,885 cycles # 3.863 GHz ( +- 19.25% ) (38.44%) 2,583,719,806 instructions # 0.07 insn per cycle ( +- 19.24% ) (46.14%) 585,212,952 branches # 40.005 M/sec ( +- 19.23% ) (53.83%) 562,164 branch-misses # 0.14% of all branches ( +- 19.23% ) (61.53%) 282,621,312,162 slots # 19.320 G/sec ( +- 19.25% ) (69.22%) 11,048,627,225 topdown-retiring # 3.8% Retiring ( +- 19.22% ) (69.22%) 34,358,400,894 topdown-bad-spec # 11.5% Bad Speculation ( +- 19.57% ) (69.22%) 2,231,092,499 topdown-fe-bound # 0.8% Frontend Bound ( +- 19.25% ) (69.22%) 246,679,210,776 topdown-be-bound # 84.0% Backend Bound ( +- 19.21% ) (69.22%) 701,951,397 L1-dcache-loads # 47.985 M/sec ( +- 19.22% ) (69.23%) 3,239,403,770 L1-dcache-load-misses # 691.17% of all L1-dcache accesses ( +- 19.25% ) (69.24%) 11,475,685 LLC-loads # 784.475 K/sec ( +- 19.23% ) (69.25%) 793,272 LLC-load-misses # 10.36% of all LL-cache accesses ( +- 19.23% ) (69.25%) 17,821,045 L1-icache-load-misses # 0.00% of all L1-icache accesses ( +- 19.51% ) (30.77%) 693,339,354 dTLB-loads # 47.397 M/sec ( +- 19.33% ) (30.76%) 637,811 dTLB-load-misses # 0.14% of all dTLB cache accesses ( +- 19.09% ) (30.75%) 131,922 iTLB-load-misses # 0.00% of all iTLB cache accesses ( +- 19.59% ) (30.75%) 7.31681 +- 0.00177 seconds time elapsed ( +- 0.02% ) # Icelakex, multi-page (x86/clear_huge_page), region-sz=64g, pg-sz=2mb Performance counter stats for 'taskset -c 15 bench/pf-test --sz 64g --huge 1' (3 runs): 16,276.28 msec task-clock # 2.999 CPUs utilized ( +- 19.24% ) 27 context-switches # 2.488 /sec ( +- 19.25% ) 3 cpu-migrations # 0.276 /sec ( +- 19.25% ) 196,935 page-faults # 18.146 K/sec ( +- 19.25% ) 41,906,597,608 cycles # 3.861 GHz ( +- 19.24% ) (38.44%) 729,479,932 instructions # 0.03 insn per cycle ( +- 19.38% ) (46.14%) 133,969,095 branches # 12.344 M/sec ( +- 19.35% ) (53.84%) 412,818 branch-misses # 0.46% of all branches ( +- 18.97% ) (61.54%) 209,574,316,961 slots # 19.311 G/sec ( +- 19.24% ) (69.24%) 4,933,512,982 topdown-retiring # 2.3% Retiring ( +- 19.24% ) (69.24%) 20,272,641,267 topdown-bad-spec # 9.4% Bad Speculation ( +- 19.51% ) (69.24%) 837,421,487 topdown-fe-bound # 0.4% Frontend Bound ( +- 19.24% ) (69.24%) 190,089,232,476 topdown-be-bound # 88.0% Backend Bound ( +- 19.19% ) (69.24%) 194,318,641 L1-dcache-loads # 17.905 M/sec ( +- 19.07% ) (69.25%) 3,238,878,229 L1-dcache-load-misses # 2480.93% of all L1-dcache accesses ( +- 19.25% ) (69.26%) 10,560,508 LLC-loads # 973.081 K/sec ( +- 19.23% ) (69.26%) 724,884 LLC-load-misses # 10.28% of all LL-cache accesses ( +- 17.15% ) (69.26%) 14,378,070 L1-icache-load-misses # 0.00% of all L1-icache accesses ( +- 19.13% ) (30.75%) 185,562,230 dTLB-loads # 17.098 M/sec ( +- 19.74% ) (30.74%) 617,978 dTLB-load-misses # 0.51% of all dTLB cache accesses ( +- 18.72% ) (30.74%) 112,509 iTLB-load-misses # 0.00% of all iTLB cache accesses ( +- 19.76% ) (30.74%) 5.42697 +- 0.00152 seconds time elapsed ( +- 0.03% ) # Icelakex, baseline (mm/clear_huge_page), region-sz=64g, pg-sz=1gb Performance counter stats for 'taskset -c 15 bench/pf-test --sz 64 --huge 2' (3 runs): 21,361.22 msec task-clock # 2.999 CPUs utilized ( +- 19.25% ) 23 context-switches # 1.615 /sec ( +- 18.95% ) 3 cpu-migrations # 0.211 /sec ( +- 19.25% ) 701 page-faults # 49.230 /sec ( +- 19.27% ) 54,981,958,487 cycles # 3.861 GHz ( +- 19.25% ) (38.44%) 2,012,625,953 instructions # 0.05 insn per cycle ( +- 19.25% ) (46.14%) 470,264,509 branches # 33.026 M/sec ( +- 19.25% ) (53.83%) 194,801 branch-misses # 0.06% of all branches ( +- 18.88% ) (61.53%) 274,966,507,627 slots # 19.311 G/sec ( +- 19.25% ) (69.22%) 10,555,137,650 topdown-retiring # 3.8% Retiring ( +- 19.04% ) (69.22%) 21,206,785,918 topdown-bad-spec # 7.8% Bad Speculation ( +- 18.13% ) (69.22%) 1,094,597,329 topdown-fe-bound # 0.4% Frontend Bound ( +- 19.25% ) (69.22%) 244,462,123,545 topdown-be-bound # 88.0% Backend Bound ( +- 19.33% ) (69.22%) 532,232,051 L1-dcache-loads # 37.378 M/sec ( +- 19.25% ) (69.23%) 3,224,574,249 L1-dcache-load-misses # 909.02% of all L1-dcache accesses ( +- 19.25% ) (69.24%) 2,318,195 LLC-loads # 162.804 K/sec ( +- 19.35% ) (69.25%) 206,737 LLC-load-misses # 13.44% of all LL-cache accesses ( +- 18.30% ) (69.25%) 4,950,866 L1-icache-load-misses # 0.00% of all L1-icache accesses ( +- 19.26% ) (30.77%) 531,299,560 dTLB-loads # 37.313 M/sec ( +- 19.24% ) (30.76%) 2,811 dTLB-load-misses # 0.00% of all dTLB cache accesses ( +- 17.25% ) (30.75%) 26,355 iTLB-load-misses # 0.00% of all iTLB cache accesses ( +- 19.58% ) (30.75%) 7.12187 +- 0.00190 seconds time elapsed ( +- 0.03% ) # Icelakex, multi-page (x86/clear_huge_page), region-sz=64g, pg-sz=1gb Performance counter stats for 'taskset -c 15 bench/pf-test --sz 64 --huge 2' (3 runs): 15,764.52 msec task-clock # 2.999 CPUs utilized ( +- 19.25% ) 17 context-switches # 1.618 /sec ( +- 20.47% ) 3 cpu-migrations # 0.285 /sec ( +- 19.25% ) 700 page-faults # 66.614 /sec ( +- 19.22% ) 40,560,984,582 cycles # 3.860 GHz ( +- 19.25% ) (38.45%) 79,578,792 instructions # 0.00 insn per cycle ( +- 19.24% ) (46.15%) 13,872,134 branches # 1.320 M/sec ( +- 19.23% ) (53.85%) 119,492 branch-misses # 1.29% of all branches ( +- 18.80% ) (61.55%) 202,854,573,160 slots # 19.304 G/sec ( +- 19.25% ) (69.25%) 3,982,417,725 topdown-retiring # 2.0% Retiring ( +- 19.25% ) (69.25%) 13,523,424,635 topdown-bad-spec # 6.8% Bad Speculation ( +- 18.69% ) (69.25%) 18,661,431 topdown-fe-bound # 0.0% Frontend Bound ( +- 19.28% ) (69.25%) 185,884,147,789 topdown-be-bound # 91.3% Backend Bound ( +- 19.28% ) (69.25%) 22,587,703 L1-dcache-loads # 2.150 M/sec ( +- 19.38% ) (69.25%) 3,223,143,697 L1-dcache-load-misses # 21478.37% of all L1-dcache accesses ( +- 19.25% ) (69.25%) 1,777,675 LLC-loads # 169.169 K/sec ( +- 19.60% ) (69.25%) 126,583 LLC-load-misses # 10.77% of all LL-cache accesses ( +- 19.82% ) (69.25%) 3,333,729 L1-icache-load-misses # 0.00% of all L1-icache accesses ( +- 19.49% ) (30.75%) 19,999,517 dTLB-loads # 1.903 M/sec ( +- 19.38% ) (30.75%) 1,833 dTLB-load-misses # 0.01% of all dTLB cache accesses ( +- 17.72% ) (30.75%) 34,066 iTLB-load-misses # 0.00% of all iTLB cache accesses ( +- 19.09% ) (30.75%) 5.25624 +- 0.00176 seconds time elapsed ( +- 0.03% ) # Milan, baseline (mm/clear_huge_page), region-sz=64g, pg-sz=2mb Performance counter stats for 'taskset -c 15 bench/pf-test --sz 64g --huge 1' (3 runs): 5,241.76 msec task-clock # 1.000 CPUs utilized ( +- 0.08% ) 10 context-switches # 1.909 /sec ( +- 8.82% ) 1 cpu-migrations # 0.191 /sec 65,636 page-faults # 12.530 K/sec ( +- 0.00% ) 12,730,694,768 cycles # 2.430 GHz ( +- 0.08% ) (33.31%) 36,709,243 stalled-cycles-frontend # 0.29% frontend cycles idle ( +- 24.07% ) (33.32%) 37,520,225 stalled-cycles-backend # 0.29% backend cycles idle ( +- 9.87% ) (33.34%) 874,896,010 instructions # 0.07 insn per cycle # 0.05 stalled cycles per insn ( +- 1.23% ) (33.36%) 199,308,386 branches # 38.049 M/sec ( +- 0.84% ) (33.38%) 441,428 branch-misses # 0.22% of all branches ( +- 4.68% ) (33.38%) 2,495,566,569 L1-dcache-loads # 476.417 M/sec ( +- 0.04% ) (33.38%) 1,079,711,798 L1-dcache-load-misses # 43.28% of all L1-dcache accesses ( +- 0.01% ) (33.37%) 50,936,391 L1-icache-loads # 9.724 M/sec ( +- 1.29% ) (33.35%) 284,407 L1-icache-load-misses # 0.56% of all L1-icache accesses ( +- 4.60% ) (33.33%) 546,596 dTLB-loads # 104.348 K/sec ( +- 6.14% ) (33.31%) 231,897 dTLB-load-misses # 42.08% of all dTLB cache accesses ( +- 4.27% ) (33.29%) 6 iTLB-loads # 1.145 /sec ( +- 72.65% ) (33.29%) 34,065 iTLB-load-misses # 262038.46% of all iTLB cache accesses ( +- 44.88% ) (33.29%) 18,237,487 L1-dcache-prefetches # 3.482 M/sec ( +- 12.84% ) (33.29%) 5.23915 +- 0.00421 seconds time elapsed ( +- 0.08% ) # Milan, multi-page (x86/clear_huge_page), region-sz=64g, pg-sz=2mb Performance counter stats for 'taskset -c 15 bench/pf-test --sz 64g --huge 1' (3 runs): 3,655.71 msec task-clock # 0.999 CPUs utilized ( +- 0.13% ) 7 context-switches # 1.913 /sec ( +- 8.25% ) 1 cpu-migrations # 0.273 /sec 65,636 page-faults # 17.934 K/sec ( +- 0.00% ) 8,879,727,514 cycles # 2.426 GHz ( +- 0.13% ) (33.26%) 5,733,380 stalled-cycles-frontend # 0.06% frontend cycles idle ( +-170.04% ) (33.28%) 42,012,302 stalled-cycles-backend # 0.47% backend cycles idle ( +- 24.51% ) (33.31%) 214,672,610 instructions # 0.02 insn per cycle # 0.28 stalled cycles per insn ( +- 1.71% ) (33.34%) 42,298,268 branches # 11.557 M/sec ( +- 1.28% ) (33.36%) 267,936 branch-misses # 0.62% of all branches ( +- 7.80% ) (33.37%) 2,235,310,058 L1-dcache-loads # 610.770 M/sec ( +- 0.02% ) (33.37%) 1,089,602,355 L1-dcache-load-misses # 48.73% of all L1-dcache accesses ( +- 0.01% ) (33.37%) 48,725,812 L1-icache-loads # 13.314 M/sec ( +- 0.25% ) (33.37%) 231,227 L1-icache-load-misses # 0.47% of all L1-icache accesses ( +- 13.20% ) (33.37%) 280,655 dTLB-loads # 76.685 K/sec ( +- 13.33% ) (33.37%) 151,028 dTLB-load-misses # 44.02% of all dTLB cache accesses ( +- 6.64% ) (33.35%) 15 iTLB-loads # 4.099 /sec ( +- 6.67% ) (33.32%) 121,208 iTLB-load-misses # 865771.43% of all iTLB cache accesses ( +- 2.74% ) (33.29%) 18,702,209 L1-dcache-prefetches # 5.110 M/sec ( +- 12.51% ) (33.27%) 3.66065 +- 0.00461 seconds time elapsed ( +- 0.13% ) # Milan, baseline (mm/clear_huge_page), region-sz=64g, pg-sz=1gb Performance counter stats for 'taskset -c 15 bench/pf-test --sz 64g --huge 2' (3 runs): 3,544.20 msec task-clock # 0.996 CPUs utilized ( +- 0.21% ) 5 context-switches # 1.406 /sec ( +- 6.67% ) 1 cpu-migrations # 0.281 /sec 227 page-faults # 63.819 /sec ( +- 0.15% ) 8,609,810,964 cycles # 2.421 GHz ( +- 0.21% ) (33.30%) 77,420,424 stalled-cycles-frontend # 0.90% frontend cycles idle ( +- 20.55% ) (33.33%) 25,197,541 stalled-cycles-backend # 0.29% backend cycles idle ( +- 1.09% ) (33.35%) 658,146,061 instructions # 0.08 insn per cycle # 0.16 stalled cycles per insn ( +- 0.04% ) (33.38%) 154,867,131 branches # 43.539 M/sec ( +- 0.04% ) (33.41%) 167,531 branch-misses # 0.11% of all branches ( +- 5.19% ) (33.41%) 2,417,846,489 L1-dcache-loads # 679.753 M/sec ( +- 0.01% ) (33.38%) 1,075,531,869 L1-dcache-load-misses # 44.49% of all L1-dcache accesses ( +- 0.01% ) (33.35%) 12,835,321 L1-icache-loads # 3.609 M/sec ( +- 0.41% ) (33.33%) 55,282 L1-icache-load-misses # 0.43% of all L1-icache accesses ( +- 1.98% ) (33.30%) 23,287 dTLB-loads # 6.547 K/sec ( +- 15.61% ) (33.29%) 1,333 dTLB-load-misses # 4.48% of all dTLB cache accesses ( +- 1.26% ) (33.29%) 3 iTLB-loads # 0.843 /sec ( +- 33.33% ) (33.29%) 231 iTLB-load-misses # 11550.00% of all iTLB cache accesses ( +- 6.14% ) (33.29%) 170,608,062 L1-dcache-prefetches # 47.965 M/sec ( +- 0.84% ) (33.29%) 3.55776 +- 0.00738 seconds time elapsed ( +- 0.21% ) # Milan, multi-page (x86/clear_huge_page), region-sz=64g, pg-sz=1gb Performance counter stats for 'taskset -c 15 bench/pf-test --sz 64g --huge 2' (3 runs): 1,718.27 msec task-clock # 0.999 CPUs utilized ( +- 0.08% ) 6 context-switches # 3.489 /sec ( +- 14.70% ) 1 cpu-migrations # 0.581 /sec 227 page-faults # 132.000 /sec ( +- 0.15% ) 4,176,107,493 cycles # 2.428 GHz ( +- 0.08% ) (33.19%) 2,675,797 stalled-cycles-frontend # 0.06% frontend cycles idle ( +- 0.34% ) (33.25%) 147,394,527 stalled-cycles-backend # 3.53% backend cycles idle ( +- 8.80% ) (33.31%) 12,779,784 instructions # 0.00 insn per cycle # 13.14 stalled cycles per insn ( +- 0.09% ) (33.37%) 2,428,829 branches # 1.412 M/sec ( +- 0.08% ) (33.42%) 63,460 branch-misses # 2.61% of all branches ( +- 3.48% ) (33.46%) 31,159,378 L1-dcache-loads # 18.119 M/sec ( +- 3.27% ) (33.46%) 14,692,358 L1-dcache-load-misses # 48.21% of all L1-dcache accesses ( +- 3.12% ) (33.46%) 2,556,688 L1-icache-loads # 1.487 M/sec ( +- 0.89% ) (33.46%) 21,148 L1-icache-load-misses # 0.84% of all L1-icache accesses ( +- 0.25% ) (33.41%) 6,114 dTLB-loads # 3.555 K/sec ( +- 12.76% ) (33.35%) 1,742 dTLB-load-misses # 33.73% of all dTLB cache accesses ( +- 21.79% ) (33.29%) 45 iTLB-loads # 26.167 /sec ( +- 7.52% ) (33.23%) 90 iTLB-load-misses # 210.94% of all iTLB cache accesses ( +- 21.20% ) (33.17%) 257,942 L1-dcache-prefetches # 149.993 K/sec ( +- 9.84% ) (33.17%) 1.72042 +- 0.00139 seconds time elapsed ( +- 0.08% ) --- arch/x86/mm/hugetlbpage.c | 49 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c index 5804bbae4f01..4294b77c4f18 100644 --- a/arch/x86/mm/hugetlbpage.c +++ b/arch/x86/mm/hugetlbpage.c @@ -148,6 +148,55 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr, return hugetlb_get_unmapped_area_topdown(file, addr, len, pgoff, flags); } + +/* + * This is used on all !CONFIG_HIGHMEM configurations. + * + * CONFIG_HIGHMEM, falls back to the __weak version. + */ +#ifndef CONFIG_HIGHMEM +static void clear_contig_region(struct page *page, unsigned long vaddr, + unsigned int npages) +{ + clear_user_pages(page_address(page), vaddr, page, npages); +} + +void clear_huge_page(struct page *page, + unsigned long addr_hint, unsigned int pages_per_huge_page) +{ + unsigned long addr = addr_hint & + ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); + const long pgidx = (addr_hint - addr) / PAGE_SIZE; + const int first_pg = 0, last_pg = pages_per_huge_page - 1; + const int width = 2; /* pages cleared last on either side */ + int sidx[3], eidx[3]; + int i, n; + + if (pages_per_huge_page > MAX_ORDER_NR_PAGES) + return clear_contig_region(page, addr, pages_per_huge_page); + + /* + * Neighbourhood of the fault. Cleared at the end to ensure + * it sticks around in the cache. + */ + n = 2; + sidx[n] = (pgidx - width) < first_pg ? first_pg : (pgidx - width); + eidx[n] = (pgidx + width) > last_pg ? last_pg : (pgidx + width); + + sidx[0] = first_pg; /* Region to the left of the fault */ + eidx[0] = sidx[n] - 1; + + sidx[1] = eidx[n] + 1; /* Region to the right of the fault */ + eidx[1] = last_pg; + + for (i = 0; i <= 2; i++) { + if (eidx[i] >= sidx[i]) + clear_contig_region(page + sidx[i], + addr + sidx[i] * PAGE_SIZE, + eidx[i] - sidx[i] + 1); + } +} +#endif /* CONFIG_HIGHMEM */ #endif /* CONFIG_HUGETLB_PAGE */ #ifdef CONFIG_X86_64 From patchwork Mon Apr 3 05:22:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13197675 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB6F1C761A6 for ; Mon, 3 Apr 2023 05:23:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7125E900004; Mon, 3 Apr 2023 01:23:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6C270900002; Mon, 3 Apr 2023 01:23:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4516F900004; Mon, 3 Apr 2023 01:23:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 2EFC1900002 for ; Mon, 3 Apr 2023 01:23:03 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D455AC078D for ; Mon, 3 Apr 2023 05:23:02 +0000 (UTC) X-FDA: 80638935804.29.D4A25C1 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf18.hostedemail.com (Postfix) with ESMTP id 8281F1C0010 for ; Mon, 3 Apr 2023 05:22:59 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=QzWiclfT; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=pc5h1iDO; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf18.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680499379; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AWtO26WrUxcC8Dr9AfFmU8r0+3TZoPF8bVvYSgph62Q=; b=QX4dTTpBSxqLI/UUhjrQPKyI88kH75U2XCnkS85wr/C50ttfgqXFSSK+8FlXfl6UHkdJtv m/RzKFPiTVYuOfS0i+SdRNRYI8MoPMBJOf5v91N/I5QxjvEWpK5vomFlt3MGO20Af5TR9j HexVpjxuusi9fBKSpgN2i+ONjX4hEqM= ARC-Authentication-Results: i=2; imf18.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=QzWiclfT; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=pc5h1iDO; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf18.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1680499379; a=rsa-sha256; cv=pass; b=u68YjGt+ARoLE1xiU9mArX8rsUJUP6k9yGRXI5WVwpM2rIFF72v60jX3uMpvlIqpXrNa36 2W2RMMx8f20QypSD3WH/fYgBqOLIkOYzPjboM3GKG9H0BAG8bwJfSo6wrOIGu/e5hGe/lv IB5CPiQRWkoEglRHw3iDWCj7Fw0Cp64= Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 332NIh1B013542; Mon, 3 Apr 2023 05:22:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2022-7-12; bh=AWtO26WrUxcC8Dr9AfFmU8r0+3TZoPF8bVvYSgph62Q=; b=QzWiclfTa84OJnoMiqC/7ngDJfU7OilahzGorX6/iDroF+VOt9o5E7YvMa0voGN9GMGi L/T5pVXsk7jDRfAs7svSsRk7spEwhTQFau+963l3607okU487LO2MlMNcIqeid/QU+z1 JIAipmHfsYHs8URW3bpfWRiu2GNX8IT78Gb2nICU7cJLQBTPET8ijOcuhWy89YwgmqiJ GaelT/lv+yXyj+22lS1X1citlyXcHod7ULtxhQgX+WrIQVy6d75z3/GB/mMeHtkmX43g vGQd0ImiCp3A6cAsFjE7cP5HpE5c8fnv8GK7sJjkIv+N2ALnV8OES7PI1C20nQaIC4+V xQ== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3ppcgaj5a1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:48 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 3332i7pt013932; Mon, 3 Apr 2023 05:22:47 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2107.outbound.protection.outlook.com [104.47.55.107]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3ppt3djuqb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:47 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VMfTjRopugbMrhSsq/HHWJBSJZik/77H9g6QRhukxMC6LFF5yXoM6dkaDhJBX+3CkJtQ7TrzImxT6D6QeN1lPrni9zrpdXN+jbevC8kAg0jmWgw7k2X5IWjf3jnQnTGr/51Ap6iqVgEhDNnbbnhJyoSknL4UXc4UpXKOkCVe4b/eTpDtVBvqU63bhDeb9YNqMz6w5d/jcwV0vtRjlMMfnG8cKbhmmhXGQidA8vixZZJZEa+pmqBxMfWGW4H1gw0CFcJysCGvFy9E+CivKfp92KBsNZuRLVVhvS7wZKKWEJnc+EZHSgCDzvv46S5HpDPdcYxcWV/IkEyc6hqLYZ5+/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AWtO26WrUxcC8Dr9AfFmU8r0+3TZoPF8bVvYSgph62Q=; b=fpG7HBxAZfSNIkJ23oXjW8pfBDsAos6ZuomHiV3sgcsGHdvuPyqbuiVgNfFloS7Hyv34uLNInFhbYMVxYz4+DNEG4rSQrG51yPpjg20NH/6QGETJOVlL5FRIZb0kc8JFIHX78IvBNN1EpGYhq5SNyZ/sxo7GvHo/10Vn2M2S43qgsjiVc1TyCUo9XSjq9PQQzcjW6u6KC/f6PU+AGi2OB065AWblwpSmYGvpxazSFyCaJT21oCR60Bst8kWSzOSH1kTw00NVdlFAn2IiRqbIYLYTUMPX1birWvpyxi4GmEW8wG4Hsij9Nu7tto9t3bdeezeetfr8BTjIVXCqXRFyTA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AWtO26WrUxcC8Dr9AfFmU8r0+3TZoPF8bVvYSgph62Q=; b=pc5h1iDO5I63MTP/JVPka1qQH1fKok4RVjVA60P+OIRd7FyTXOAlx5KDtGE1ooBdQzg880bOxouff4eRuD5IB2IdLVINdht+W/suw6OhBNdx8cckUIppCxX4ntmfeGyZJqaZR3WgeIIi9FZJ2QU13TuIJybzKbjvzEkh3dGyiQg= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by SA1PR10MB6365.namprd10.prod.outlook.com (2603:10b6:806:255::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.33; Mon, 3 Apr 2023 05:22:45 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6%8]) with mapi id 15.20.6254.029; Mon, 3 Apr 2023 05:22:45 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: torvalds@linux-foundation.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, vincent.guittot@linaro.org, jon.grimm@amd.com, bharata@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, ankur.a.arora@oracle.com Subject: [PATCH 7/9] sched: define TIF_ALLOW_RESCHED Date: Sun, 2 Apr 2023 22:22:31 -0700 Message-Id: <20230403052233.1880567-8-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230403052233.1880567-1-ankur.a.arora@oracle.com> References: <20230403052233.1880567-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0166.namprd03.prod.outlook.com (2603:10b6:303:8d::21) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|SA1PR10MB6365:EE_ X-MS-Office365-Filtering-Correlation-Id: 0187daa4-2b8a-47ea-8b48-08db340372f9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: n+2pEnKRx9o049wD/LZjcRBZ5dXhBClqW6Shrj5mkMgFH/k2q6ngnOXQKb7FtFuMSSNO1In8SUP693EX4t+kl/1xHdXBvGqkGrsZmlgnzA0zSm6TGbMHf/S6/SDxYHqrwoMkVwY/wx8thwDWtg1A7YZlMg818SO1ttX44/7KYfMS2YeM4uaCO+wPSVWsFegPV1rLysE4oDM4Qwsqh3qMyVZDuhy7CT3wWyRD1Hur6hb+LGo3VlZIW9ybe7GVaIs6SWXhBGr3mH32eQRDGCtQdcTbzzOJvxXFOitwQmeezuvQGoEIbOKw7QbghRm1VWU7X2vO2NPlxkDJrQCRezyjslVb/SGrxwjiTdxNsVHL2rpaovryQROCol8+K3ixyUuH1SQJh2omXaSqlyCLQSs+PY3cVvoPctamXrNZER+prq36KY2TTZhDeDnQ8WfLRZOL1QZTvb+E8Bod58GvDNhgXXSeeqsbPjAOaFNWaJQmL3qhIuK/YbgJFxoNvwfi/mrtilrhsfoP3z3bsr3eQcXxjRJnFZ4PnoL1CDLT/xeT5x1Kdje/mJHkQ9za4OHGZ8Pd X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(366004)(346002)(136003)(396003)(376002)(39860400002)(451199021)(86362001)(36756003)(103116003)(2906002)(2616005)(83380400001)(186003)(6512007)(6506007)(1076003)(26005)(6486002)(107886003)(6666004)(4326008)(8676002)(66946007)(66556008)(66476007)(41300700001)(5660300002)(7416002)(38100700002)(316002)(478600001)(8936002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ahJtsEbR0Pu035xSPsiACxf6wPqKzntHhxV8lwYFiKOJ5IXr7E3AwfLMJCJ+kD5ILdKgds1RrLPz4pca2iteRIzAxmCKACDLf7bsLn74fffJqKInGBId2mmgcXwhOK4cg6dvJ8Q1Z2Thd3x99lymP1mu3U+vRuwuzqysWUQJY8L0CFDpG9zLMnvtHphNvXhu6lc75UmDaGBDo/h7FQeWv0OKKFp5NemLeo55/2Uhv128LExDWyyVlKAS0JX+4DadL/3g8/ultQ5Nr80NO/I6E31N5i45QBC8KaI52Kc73Z4zxjfyjjt5aPOS9WXusKpCiYNJmmFeZ5EXeBFARu5MNMWeKjsEpp6rnctQjsKixYeOSbCzsjivtv/FkcP4gf5w80BuKzGqjtL98xfTSGS1U9LXwLtq1zbnf0CsivCtONCG5ckAsNtqv5dG03yJ0mZhAh2ctgUZaPTyQqqRtQNA5jhNexXOnrNSGRx8WHx5f1930m0u0npTOAc6WnflCeCQk7pKsTZEbbzI7+v3L0sGFq/T6cOj5MbOfRw5O567vSGWkerDpJ0VTxFYNbUEgwgr5LNnPNAVMRfZoOjA9vxwRox9JSIFUpAaKXn1O7m+zrnuDjmU7Gj0hHs6aE7L4cYm6QHqNxaBYrz7B+aIppR/T2nMmL/f9ZZ2ZPnm2V57d1WZLziQlqvM1izLb1pcJXGr/1YhPSWDJ9Hn6e1VAlRua4e23/umZc9IO5WkQ+Ib0h9o13UEcqxyM28eP/q0eIGfCHK9zLR1WGsNiqZMs0/Llb/oMoqv4UVt1CN75XS5lbbuhC+CjdgHlYvwcQ7bff7WuIBOYc48PNJNVMt6Uv0Ir4xa288WrIqTWADdVAXqPgcgicS/6fHrRR/gwTopnxor+Whi3+OIFdJz7shHnlnlg/erYINnKNJAR7Oc89V02mDDK7agTqFLz4vau8PCSWEVG8oVIpLeVSY9XsB+b0M1Kd+L7y0l4B5ySMFI8ULxij1r+CUC258sXurQKqG0/zkk2uMckRvJa4evuDFyTH1XyjAEfCLqV1eUmK19xUl2gjNarNb5I68lWpkLhx/Y5NRjxjCbTuhTz3hb1eKB5qGZk/fsAgeW0zCkAKb9hSGXg46aV7SNKQN2SGIMguc2kruriOcbwW2wXqGSr+jNjCSXowHvfGTeV2Hrw6wyTekFPiFwYZxBQWnBYcDEDumJ+9WyEYl64CSAeKQqq27PX96S6necIp3E3stRrCbg6zNa6jNP7zvQjpWXA9wYS+ur5Zge1oCzIVFabrqtzAAX2HfPW4cSW1aKk9relzp+U1Z9zi6X+leCNqjd45IgMBal0FrOJ8vrCSXPAEZeKWccv4a8M2f7Fb9HOxawJVSHWfYX/gVqCfauk7rdJI5ice4DEGynXw+iEscDnwPzmX9GMtYLpXpfdUI8wHG8fvVkFajEzI5v+dDw8OLOCMhEt4uKXwRwP4sAyK6EYWL3ChofGkDrB2GBc3VtFu+dlk67XuHnOI9yApRfofROHoSUJTWxtJK6ZoyQvYEqmTixpl2SQVg+dQlSKpa4HkId99GyWUkNlUaw+arC8biS4dIav3rKwjiL X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 3r6ZE2sDejkM8UbH77cxDVrIGoYh+cGi9iyshzrkFpfEcbAwfl9Me7pPIyWWZZ/I1JIHpyzvQiGnb/IturQjewasADuxN6tKFrZqhuL1rXKqYJMvvqO54Vv01HuYeHw9v1mS3p3CynkSAnzWjgABRIfQdm7pgLq6PboArgi9bauSun/VWrhVk1qr4GpoGWHEg8kEo8lA3yLaWDX+Z4C2ZaPJYyHGOjXPbF3NvUrOyjk+0GShQN+2PsXBQVf4y4yvw7iltqT12DEb3Dp5raC4Ckzx9oOvNkK9UcVhkAWMw/A7u2lu+sUStulM+Ij7E7F8KyOekq5Mf2+Hc/tquCL0yqx5KmmxBQmA104R63YLinuXqe0V1EDmttVRKsTyK6+tSYSgzwlOugSxdIsOguPg6PI5bYzXO0cLN4yD0XDubdNxta3YjJlP/y50IY1ys5Sw52oRWJK24pWlXeRlwIs3fUuHKnkTNS4s9FD28/zb2z/VlgHI/vV0TVZELnlSxWnI89vrcbO37ZOXppDGNHBLXMwnVcBhnVwtdhG8iZHl+t6Q5JAPpIH883OhWs+ZA6cwkvmV9g0fg1ljwz1OqN0e6EZJANanebwfQe8xMG6Qt+SVLFr0WDdVV+dMSIODOdglfTAj1B4aJtQ0Pf6gAKFaW79hxqt++xtU9w6RSNYh5626h6pZfS81ZIUnPUFf6KULjMKI9+4vGMk9zr4LYnG8YQTLHXLJaik1EKHVLrag3iMHc1UrZMl3w48vnCAngyxljjUrMQYe6gchcxRUgN/PhnzfilydRE2BAIENFjwKoMBl76NUM85px6K0YwIbmZFNW4p+K7dteu7ZV1pDaXr05PTJ3JvV75U6lywb8vxU0aGQCIdjYRUqUonXRlVkReGw/qZzyGJhRz5TsUDb5iS3U4GbzDgIHtW3swFZ7ho9vSJy6Ls7P4dvfldvrErdXlPxERhsR9yBnwxklZkaCcts/KDNVs6TowdRLJgtawie3zjIO8I7dx87JNgDbm/Cci6Ub6OtCOxDbKvP68jJx8ZVb1Dp5Sa3lm4DlMdsWE/Ny4P83lW2RAOe78xafB2aRvH5Jlo909rIID9EJdA8dC51AybwmDtn5VteCXJmP5ec/1O3CqJjs/xKKvGbGkGnGmjWukJolvVpPIsopYFl9Y8fC1D78ex4+WSkQKA6eByvNTc= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0187daa4-2b8a-47ea-8b48-08db340372f9 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2023 05:22:45.0440 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: H02fg1kyYkymzqYus/XlB3WnUpEHO0gj3BRTGe3t9rTYEGIfgBgvzuwIjlyrWRV5JaeGqodX5vMi6AG69uauyrbUGbThIjkeWKF4QojsDlY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR10MB6365 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-03_02,2023-03-31_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=627 mlxscore=0 adultscore=0 suspectscore=0 spamscore=0 malwarescore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2304030039 X-Proofpoint-GUID: 3n25f6C0Vp3I69V4G6C_LKVTj8U4YkXk X-Proofpoint-ORIG-GUID: 3n25f6C0Vp3I69V4G6C_LKVTj8U4YkXk X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: s9aoiq9k87u57t4z3t6xuczbx3ibb14n X-Rspamd-Queue-Id: 8281F1C0010 X-HE-Tag: 1680499379-940889 X-HE-Meta: U2FsdGVkX1+lPf4kiRdq9m5CE513dKlXhKvGUSIekFmoux4ryB9Nf957B25jDZjtF3MGZtEs5vclR+0VVZyLDb/YtBx1RWXQq5tzZrjS1Ut7xyBR+klIItJ2VN7Z6dul4ds0YBP+Tr9hYrp1WpPLhdw8WKzVdYCJWEbEBbRXvNupO4Gl5MthnSpb8qNKfr8RXsMNZ/0uL1PP5dmxcvzVXAGU9FozH9MYPrT01a4E5eMzmnvLmuoBli2VcwFULfXi/ANDulRag3I8ErKUgBDI57TqAsSgEa6EjkVd+LST5+sDDL8DzGfqYO6lY4snaPYe7eT6V9Rtqnge6SdMXnDske84tZq41VxyVRaeQruBRaPCDFmckdpf3CB+ZlEvAzbY568fXXIdV/OO0yAlSJzp37pZxSl0gKON8+SeePhPT7177PAgF3agJ5dZxTaTYBpppX6Rg1z3lT12fv2HLblw+YS+25+IXE9kPpTXj36fmsdyPqB2HfEY9gOQYjAVtBfiSbybC/CmwRYcW3cuwusKGoXL5MrHnw4jukuhrDNZP9JZIfdfW3tjSCs8w9llh6uPgzBCklpu1Tm5+UhdJvGXoL0AGuWflFZPz4+KmnieFrfylSFCucDv6cukbqyE6ESDFgX3MQjpnHRXGbWit1mHwX6iRO6dn9Cb+pindEz2nRXhtF6HXnRq4IUEo8Z86yAmL6vLVaC9L8umx8fnMGEzkqXRjL3vpEWVM7qBrSlVvazxjPpORdxkv/j1G8LJ2i058m0lIkfQ/tMib4+TGzYu2KnerhNvXbj8uOCIWcFnfY2WWrCJgWYWnTV99VQUO595kBguKCQ1sSCHjmM0wbd1dWXewW6kEgRFxOwSEJ/vAazWK60+rwPiOqRbj2C20ghG5xSCUV+JGBxYeT2qnauHcemLKYP4j4EEc43lsVEDnc9mhZE/xBMf/c3NE7vZXHYU+zitM0uQkTguFyS7mQP TsVOtuWN AwXFEZ4BhMLzZj6AMsqtXjRB8qNelhwxQGwBtTU1N5dCUrgSLCCVQ+3kUNr4XTRFQ8aVuK1G2/1LjhIw8Gf5pdXq9bLPhpQyS9lqwHZclKhVem0CiBzqvqL2mpqXu4JraAm0K8qyok2pu3YYJgWMqns9oZlD4yA4jTtfEm3WtPlCW/9/0iQk9+0UPv7z3OFpETNetwBDbnEwMVRY4ibU+dg/81JVE/2tbL7yMpMGQi4QzO50S9+rHaAyoTuCy8MSKLFXBFxIYwcWwYEWzYdYr45iKrBBhV1fjup/kRFZAsfrYrlu+wbNt/VdaqpgURN7B0RfgiS3vTN4dMzY1iXkavdDAEbO8R+FDA2itYeZRGn9rxrZEqxgxAPWrgYgXxaMab79KTxq/GdI/U8t0KeX4mlcq9OefkTf+gxkZKec2qUai0uGvg37OqCKM/fBTmvHRlpFSklyeRnmOa9a65m6dR6tDTq/jdx58m6lRQ8UuI69o/YGynLDc6GzFFA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Define TIF_ALLOW_RESCHED to allow threads to mark themselves as allowing rescheduling in the irqexit path with PREEMPTION_NONE/_VOLUNTARY. This is meant to be used for long running tasks where it is not convenient to periodically call cond_resched(). Suggested-by: Linus Torvalds Signed-off-by: Ankur Arora --- arch/x86/include/asm/thread_info.h | 2 ++ include/linux/sched.h | 29 +++++++++++++++++++++++++++++ 2 files changed, 31 insertions(+) diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h index f1cccba52eb9..8c18b9eaeec4 100644 --- a/arch/x86/include/asm/thread_info.h +++ b/arch/x86/include/asm/thread_info.h @@ -100,6 +100,7 @@ struct thread_info { #define TIF_BLOCKSTEP 25 /* set when we want DEBUGCTLMSR_BTF */ #define TIF_LAZY_MMU_UPDATES 27 /* task is updating the mmu lazily */ #define TIF_ADDR32 29 /* 32-bit address space on 64 bits */ +#define TIF_RESCHED_ALLOW 30 /* can reschedule if needed */ #define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME) #define _TIF_SIGPENDING (1 << TIF_SIGPENDING) @@ -122,6 +123,7 @@ struct thread_info { #define _TIF_BLOCKSTEP (1 << TIF_BLOCKSTEP) #define _TIF_LAZY_MMU_UPDATES (1 << TIF_LAZY_MMU_UPDATES) #define _TIF_ADDR32 (1 << TIF_ADDR32) +#define _TIF_RESCHED_ALLOW (1 << TIF_RESCHED_ALLOW) /* flags to check in __switch_to() */ #define _TIF_WORK_CTXSW_BASE \ diff --git a/include/linux/sched.h b/include/linux/sched.h index 63d242164b1a..1e7536e6d9ce 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -2229,6 +2229,35 @@ static __always_inline bool need_resched(void) return unlikely(tif_need_resched()); } +/* + * Define this in common code to avoid include hell. + */ +static __always_inline bool resched_allowed(void) +{ +#ifndef TIF_RESCHED_ALLOW + return false; +#else + return unlikely(test_tsk_thread_flag(current, TIF_RESCHED_ALLOW)); +#endif +} + +static inline void allow_resched(void) +{ + /* + * allow_resched() allows preemption via the irqexit context. + * To ensure that we stick around on the current runqueue, + * disallow migration. + */ + migrate_disable(); + set_tsk_thread_flag(current, TIF_RESCHED_ALLOW); +} + +static inline void disallow_resched(void) +{ + clear_tsk_thread_flag(current, TIF_RESCHED_ALLOW); + migrate_enable(); +} + /* * Wrappers for p->thread_info->cpu access. No-op on UP. */ From patchwork Mon Apr 3 05:22:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13197676 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4946BC76196 for ; Mon, 3 Apr 2023 05:23:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 14691900002; Mon, 3 Apr 2023 01:23:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0AA7D900005; Mon, 3 Apr 2023 01:23:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D8A3F900002; Mon, 3 Apr 2023 01:23:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C2EA2900002 for ; Mon, 3 Apr 2023 01:23:08 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8AA811609B2 for ; Mon, 3 Apr 2023 05:23:08 +0000 (UTC) X-FDA: 80638936056.02.C164563 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf07.hostedemail.com (Postfix) with ESMTP id 5C4304000E for ; Mon, 3 Apr 2023 05:23:05 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=yq99iavo; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=oeU9coHO; spf=pass (imf07.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1680499385; a=rsa-sha256; cv=pass; b=pyr05/m8QG4wImJR/KCB796GcvHoVNJV3kpqKZTDg1dHa4QP+UQbyFrf1Oz17NtLwjpkDV siRd7wyBzVdgP1bC618I8vTxJL15xyNFZau3fjS7yqkIbELyOfkn8tG+xzbgd8Daa7cs2/ 3w9oy06qksnGigYDeN3BSz5dG9SahGA= ARC-Authentication-Results: i=2; imf07.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=yq99iavo; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=oeU9coHO; spf=pass (imf07.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680499385; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZJIcErO6gCpLr53xeyAlFo9Dh6PeuItoHvFCmPpopeE=; b=sE7or/9xOJUF/lANNNiK6CDyCnqrlT/JxZIlwuND5M7s/nqlP2K2YIUVvrktwrHFKkz7G5 oP7g+HK9QhioasSQoXhOcl9KUGjtS469j4vZqRhH7yRmPQQpEP0WJyme+LYGkBwDTUj5Sx aCPxQfAVdeKr9UL9DI+ULxBcELOvCMM= Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 332MwUaj024750; Mon, 3 Apr 2023 05:22:56 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2022-7-12; bh=ZJIcErO6gCpLr53xeyAlFo9Dh6PeuItoHvFCmPpopeE=; b=yq99iavokHx7dDj/ZMnuuCRoekfZAZl5KaNeqSlK04pxvsWqQEpkSwpc1RyYEgxtWazh Gsk6VAIjRbXsQsJvEZsBrs3oX3QqLbZ2SPxiisC8ksJVzMZVLdj58OiuSPV0AcyG/xJ8 rqB3jyVZg6gclK9PgSQCa9LOGwiC8Y8LUsYRcuzBZaKlpJKb1VOnJlK8b0hDRZLJaIGA OitEEg/3grDjfXy7YcCVOoIjOdJUjdgIm9O/Je7YwyqIi2UUoDjfmpxtgXPhcAIAGamf HbVqFcet6GodoGhyvmgvGTSjVmrrXUX6hKAdLlMKnafPhuwmDjRmCsZYNvtoHZNimaae 7w== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3ppc7tt446-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:56 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 3332iZTr038319; Mon, 3 Apr 2023 05:22:55 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2105.outbound.protection.outlook.com [104.47.55.105]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3pptjptnpk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:55 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=E3Fk8fHQFXbxUx4L75SPpKVNndtsTtMxpzFepb+u/SyfuZRHRuwZXA7348ZJc2ayXY4grV+WTTynN0Sj4XYwi3sUPK9ZD4pzVCX6s2T6AU7UdRJ5g70rX76EzB3deevgEXX1Y+dxPXHXv8mkF4KhzIC3cguMUrE2CeBI6WLIuOo7iBqWHTkVJxQaEGiayAkvL2z+/xhQnGcqzZf5fjUYl0m1a4CJQ2gvrZeZd1rdhSIZmqN/w0MUdZWAWFvXBKlQPXpE6PCkpt9mks9cgkcOoExZUpvmImin60rFapctfnfZ5hjZjoHggN7OYEv3JA7cEtPnZ8kBuzx+R8TtgfsAOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZJIcErO6gCpLr53xeyAlFo9Dh6PeuItoHvFCmPpopeE=; b=GYFfn4lWX/o6HACFvYBHIa6oyUp7J85S9A3QB5ybIBWriZiTwRivris9+CAD5QpEVXkXs+nF2CUhCalO0RdtUC7kdsy2lOCUFVB0goFe2CLjSGeaX5uDHAhTmc/8DpXHx8JOols44WpXEbouSu/1dSQsfmuyITMRhEWmA/jJ62qF82c18YpjkYeC2zt8FzkyMJTXdr3zT2uuy3PbOpd+7nCmbemZm62CJ5SXvEbCCDWiGjo0ZwaK1YIDkG6iJnZ/dipOsm/obuQTIqDjdXZQO77/8TMUJlcfI/ZD3EllKQPvM8AzLulCrmD9g2Rfh4eAaqWEIHBYkr79Okk1e0tPbg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZJIcErO6gCpLr53xeyAlFo9Dh6PeuItoHvFCmPpopeE=; b=oeU9coHOBgncybcyMmLuFEfDVxbWPi66XZWaOd/VM2sTKXVYT0B4NEQWTwpLvhaMMzT8CXY/COf8sInHJ24ll0CH5yYH/COMqcLNBBWT4r+Zrbw397BI1VHK3r3qrS4dFF/BcM8IQKy3xB8MD75L0wCT0j9uM6zAVVTfH6hIcHI= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by SA1PR10MB6365.namprd10.prod.outlook.com (2603:10b6:806:255::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.33; Mon, 3 Apr 2023 05:22:48 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6%8]) with mapi id 15.20.6254.029; Mon, 3 Apr 2023 05:22:47 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: torvalds@linux-foundation.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, vincent.guittot@linaro.org, jon.grimm@amd.com, bharata@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, ankur.a.arora@oracle.com Subject: [PATCH 8/9] irqentry: define irqentry_exit_allow_resched() Date: Sun, 2 Apr 2023 22:22:32 -0700 Message-Id: <20230403052233.1880567-9-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230403052233.1880567-1-ankur.a.arora@oracle.com> References: <20230403052233.1880567-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0078.namprd03.prod.outlook.com (2603:10b6:303:b6::23) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|SA1PR10MB6365:EE_ X-MS-Office365-Filtering-Correlation-Id: 1bae760d-8984-4fbf-13ef-08db34037643 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /JMEgRLdVtjfsAbqIduYdVtaKETVJDnsWFbwxXSnLwTHw4Yb9b8Y5huwV6UPRr8sSAwGlLLN2IdBDFqxJ7tZkqTna5OlDEkJHe0vl7anwNP2EV1gxtPGt9ywVb30esqGvOAt/Nj/AMpIWY2FTtrRUhyFTBpaAIXqtbRnH+ErC0dajgKPDAK6ED6J8LSWdAwEaYOiSNpj+FFxVNo2XMuxfBEIO7OLFhJAmXxaBCUc5THRjQHI6BNRKHRnsE3PuXLk8ZO1+0Jr/HMzPhO9FcRyn5fnDKJ/Q6o1WDFVBCusNGfY8EmY/gI27codm129pJV0jzx88rSgCCgQhe99CwI13yW3Nvi3LZOB58PIhXHYL+9NMKoTCa+IX6ShBMc12pPjI12xflIMSvl93fA9+N+sGyAe2Srkh0alfuCH3clTBtqJkQ6kvnQkcFmMYEZpaS7D6f0NQjkzUIaBCuFVOnAjm18Bb5HjKDPdm2KU44XHrchWlIRWK63hQW0b6XexPUlUYo1d7mbx/ii4AsWfoDR4eGrzg0h29rarn3Wf918MLz4uETNmLgas8eVuMxmOG6VI X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(366004)(346002)(136003)(396003)(376002)(39860400002)(451199021)(86362001)(36756003)(103116003)(2906002)(2616005)(83380400001)(186003)(6512007)(6506007)(1076003)(26005)(6486002)(107886003)(6666004)(4326008)(8676002)(66946007)(66556008)(66476007)(41300700001)(5660300002)(7416002)(38100700002)(316002)(478600001)(8936002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Aq/HuEd+N7dXNtUvPz7JE4EYv/PgBz8EgC+c9JG9OA57ctbp12SGWCp9Lu2dXGfhRPb0G57ZWij0+Qrb1dIoWUT4/q7MuD8bsbPCzsBQhiJhrTaF79fvvFT5zeBUgTvKDvCYQuxMyx4l7yw2jxullhaHVYJqhV8xZs8GgVjXUxNU+ItWlJXf8eHW7yA9tT3T+8wVqvLNcdFtjcTYbmnmfI3tXo+K/0UqKIM+wA2ZtlDNwqirpbB/yR/RDlSGg367GenXLCo7rD5Vr4+G5NLz3iDMitJYMGndeuG+af8XzMViIC2mC8pEFZ0ojColm2FxHG4gR2lrgdl0n6o5XckhLQ2SEvHhmqgCWbOeb7bLmoR4XOASez4Lp66G9OKuHu60kbEUm2bazWHXaIPuZ5RIoKHD8TFNc2LTkPvXi3fnWIXQ8Nu+D+P8FsVPzJwedOVi/kHsJHuRQvD6es7+jq/4dBcPewO01PJ8RIHD3czOP4D42UHDIvkM6oSj6bGKcYwdo8BdP+7qHM0V1iZLNUilYdGn4vHitf0szzK07yUojt0UifoAd4yFKUwUEU/dm9jyr0oWH09a4M6LPeQL6cEoOtO5JU7V7egCWX243h2AT8qJWnkSlqYR2RMs5b5bx7nDiJYzpmW4kSMKKSBIC3iSUhJKRJGw3yCeJtHidZHBT8Rq5KTH/5N3dfH98R3EW94tgpVdiPMnrYqABR3zkDUZSVkt8sbUwuCqe6e5TIOgDSXO/I/J7epPO4HAP1xOhKRbIpfGjSNvEgL8uTO4L4Eqt4beHw2LkdpunSQc2F2sLuffIgDD3T7WF3OHqjudSB50oFW/HDzZhbpR33lOOq4BAxAgpZTFSMhBHIRLR2H6FecQhbCFctFfvb6nWvB17O+oy5MtPTJ7W5zffMkZ2LTS6R9p+E+CLkF3t/xtUl4KtYrqNaW0QxGwicZH0yvIJWpy2PBWpCdomOzqns8qd1De975GM8TFsb4u1AktJNOQnLoyKKmHZ/npviHAuPVoOxxZcIDR7vTa9nU+LjbaAXiebwZbVYnc1tzg8OqGjptmmngOcHK0pVU3e8uatkEuBejWMc+FvCDARdqyHEuCKTfcGGrZoJxjE0MG6V3RIY7139eOX67xXru145aHIvOwk1P5FD4iYjbjPuKwaPljGnrwChNO0fUL8sA+gvDkHZSyrvNO2Y4tDiKwWny9wpOsoSFhlogZi0yn0XjiZfsrANt6CLrq6sAX18V7dRY+C21tQU75fPNsLMokIgbhRCickICel1TKsxVdEdx3WjrUk1CPPnR/CgtSHUn6BQiWZIzUIRpVAzbcbw0/fD4oJQN73iwCkDCsFYm1sPkjrInKP+C/WjI4jq7ZVO4QZm79lWvgDeaMXRd43Olz6gNkGi4mOFbNFQZpKrIeY5DYLLWtWyCIJYY2tHvOJcUttgg4hCByoMKKD7CSxwQ5Xent+9EwJdobL6L/iU9QI1HXkjXnv0okKV22DLe1hrt+p8QlXc6ziSO0LQutYGsUXImbwkEpVDO22cQl9aDhj+b6AtUBh6coJXFBZUV0Ub20Hbuy4qhmkneP0sVy5/0qra4pfKwEkgOL X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: Cbnia/eFO/wgtFWUA7cTl+DzNH4JwlXTsRuR2KXvyvnnt0dQE++1XfH6ZEV/uN8kNb7CLsKFmptrmLcG8pTYgvQFnmfJdrzarrtCvBVymrrLbHMVXyoN2Y0KAXpd6+SCJubhawJcqg9jyI8z2WGrazBLD2D0jahyIwF2xVrsa5N+ZgG5XjNTdHPY5wpUmjvR3my4u/UgK3nSp/k13wIhvpTPSxWbkUqSouirxIahMY1/sYrvcWL9BmGXkrrlTkuelwr5hXiKGB4rcF16LY8ZRxkvaWZJSstQK9fRb2e79HlkBZ5O0MycnqA9bMt6tGzpcQra0lV5kpKav4/ypKgDLmK1ieQ4+pKEhIPfMSzbL1OXbQtoAYjJgAW97bDcbhCho0U4nclIjN9uwbo/Cju3LpFm5zZxDDsVWhIqwn2Ur4NIF/qk3uiQ1JYfU449EZkzUPQKdfPX3Q/L69NQ8jZ8pxxsDpheEUsdzlARblva8p/rSXYr9e2oSwwZFwwR3HrmecsOJTDwzSMhMbPHRQsPwX92Tbeq25wKh4v/SXxs44SowvUHRDLTBDVV6lTJs6dV5fkWcZf6WEPKnL1P/JTc6/62ArsIR3ypQXrvQZMy1hMPS04o6zHqgRNlZ1Nwb0ZHTA0h+sUEAVh2BeEiUS9NZUx0hdwtjMEYl501cL6NafwLZTivngAMBtQvIfm8hBx2HdjbrXLhxMHYYRjmfjswWhg9IgSdnVLbM8YBH+P242AwbOQdOOoAtaud/iI9wnqkfJkIuOEfVivmqVnMKz1bRPwS6SldOTkY0+lnzLOWPLpssQtOEZzUDPIMb5wkBiH+weR8b1w+Cr5eRElKpCK5CYIC4xzMxvs2s8bfezvsxQd2MACCa37ciHqOdIdJ4vyxTDWJ4zy5toJsKc6wi1i8mR96DKWX8UUdWOD+ZWUvKE4ieR0QPq1mym6fQhGHwIaZd05KGAS6DSd8GgtyruE7HLufinVjrDFyD9qNnJiPnqfX4fnOyPX5hzy1Z03sh7VJnJ8SSVxwfDHggkuoBmM+iOrTMTqk2Fx0ro17oxi5MyU7ZQrK+TYjvSqOASCL/eeW67R8w+CMKgot1hGzZaqr0onuvxOAYeBJEOXUj7umwLTts6xWWJWjdBFmT+q+DiCyW04ejvSJrjzOQmEl4pPRcy55KBS5LhT0rmQgmDicDdg= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1bae760d-8984-4fbf-13ef-08db34037643 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2023 05:22:47.8596 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tDyBYPmBhG9E/ZhXTWDy8nevhTrDYQ2xwFEfFIflBN0TDmISNcR80y883HbWBEqB1/K9lCEqqji9S+qnPrpEqO6lNqvE6ujFguDmRECddm4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR10MB6365 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-03_02,2023-03-31_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 phishscore=0 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 mlxlogscore=879 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2304030039 X-Proofpoint-ORIG-GUID: f-a-V6JSgdGnspaNV5qnhzyl8asrVf1g X-Proofpoint-GUID: f-a-V6JSgdGnspaNV5qnhzyl8asrVf1g X-Rspam-User: X-Rspamd-Queue-Id: 5C4304000E X-Rspamd-Server: rspam01 X-Stat-Signature: 9megoxp36jqc7w45zu8ztouth77y1rn8 X-HE-Tag: 1680499385-576729 X-HE-Meta: U2FsdGVkX1/7SuAURq3Uknyd1Zxz49t2GB0xcgc09gwYyYu5Glvk5VEguy5R+At3EKYBhnkkMzVQInR4bNnIF3h30/GqAgK+4Kfgu2of7IKZGmasfELFetlLeqltEpKRq2p0S59Ndp5WCsmLOhQbToySUYDMThxi07l69YnR9u+VUIFSVnetjt4/hexFP2/7DtaD81QYiQOSzVMqVZPuIKNG57NhKWLuFUQ8ve3lUXMan+aTc3EWaprpaePVxrxmM0cp93N55xNVABOK0k6qplGLfawnsTRfzcuM5KEKF/8Nf9GeACvojzr2SLiqzihZMcC83jhW7XOfJzoazT3WWeY+9wTkoMUYO7hjX+7bhGE1fvV8LeAm1NRFrf/F0wdu2mVZV+oL98v+vwMa7JFMM95Qx3fljlYX3cm4L8wbbefGV4BYefjnQE0jBK9zuBuMfinLMnlqyBFasRI3EdGInm0UJqRvLuRaw94X1dwHghlLVH8Pj977sg92UV/eh0MuOP21N8a/LnPzu4Ea+5N39wRMuCJGCoXOWSocbcwN0P3rwaoeYPiVHX9+N9ppM+R9eoZ4ercM7qyyD5QQH+cn1ZKcWB/G2IGdHnytiZ+Z/xWV2LoqYtIMJB61CeZNKFqrJBYX3aI04+9zuGI9m9gcdRWl+iA9Yxy0LmN2exKrGCyMTvzV/FjA5FmcaVg37h8aN58R0VZywjhlz7SY/21VwqWHqPYJr8p+xxTxR0hgFvckJkT5lSMHCmDkXDPT6yEAwiZnZsFmMm1gzoefcFt8V03xiQ0ySAdWxV8eM2cm0pdEilc9YNZX9pB/8uIXtvkxlQL0iPsvDhrbvW+iarjYxtQhafqOEiYIlQzSOIlH+nY4/bdb/6gDfIHsLAvqx96/hrIBkH1RDz0rWZS747Dv8A8AdzwgC9hF/XYppoBHBPphkVjJkpJHZRiezE3uX0OHERtyQoBsER8yumJL6Dc sM1HrBLW 0iLrY9vli86tMwI3mmRY+ahV9nnWewIwWEha2ODTSV04xc04VfVzxy63avdAUhQhjUC7ErxeHYoGS5DrosTgKXiZ5FPqvZVORVyHX83R/YNAS6JB6Q0kCqQF0Azg9Yb+V9oykZgsfbVK5E7WKbC/Q3O1AztPmJDpMpxfHVHJa7ZLfBcQJXnGfWsnxrQgCg3WGMvsyvy60ZUrRqa1sZQizoXU8ibeXR6gG16iwGESR1HE4woGNadspvogXhB+PmOmomOuAS5/jAUlwACw5AtLsekvVJb/CtFLGp7A8Azxpe8kjdXj8T0PKYyU6Jptl2tL1owhGb3EI9Gfl5hXXrQDmjCmNLu54EF75PF089Asx35LWGuMR53XzXZ6BQT4RBlata7Bv4aJ23sIapIVOEazeCN9w9Vi8SetVX9PiCnQTTpekN9ck6s/DYqMEo3D/Aj0YIxkb X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow threads marked TIF_ALLOW_RESCHED to be rescheduled in irqexit. This is only necessary under !preempt_model_preemptible() for which we reuse the same logic as irqentry_exit_code_resched(). Signed-off-by: Ankur Arora --- kernel/entry/common.c | 8 ++++++++ kernel/sched/core.c | 36 +++++++++++++++++++++--------------- 2 files changed, 29 insertions(+), 15 deletions(-) diff --git a/kernel/entry/common.c b/kernel/entry/common.c index be61332c66b5..f1005595ebe7 100644 --- a/kernel/entry/common.c +++ b/kernel/entry/common.c @@ -390,6 +390,9 @@ void raw_irqentry_exit_cond_resched(void) preempt_schedule_irq(); } } + +void irqentry_exit_allow_resched(void) __alias(raw_irqentry_exit_cond_resched); + #ifdef CONFIG_PREEMPT_DYNAMIC #if defined(CONFIG_HAVE_PREEMPT_DYNAMIC_CALL) DEFINE_STATIC_CALL(irqentry_exit_cond_resched, raw_irqentry_exit_cond_resched); @@ -431,6 +434,11 @@ noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t state) instrumentation_begin(); if (IS_ENABLED(CONFIG_PREEMPTION)) irqentry_exit_cond_resched(); + /* + * We care about this clause only in the dynamic !preemptible case. + */ + if (unlikely(!preempt_model_preemptible() && resched_allowed())) + irqentry_exit_allow_resched(); /* Covers both tracing and lockdep */ trace_hardirqs_on(); diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 0d18c3969f90..11845a91b691 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6500,6 +6500,8 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) * - explicit schedule() call * - return from syscall or exception to user-space * - return from interrupt-handler to user-space + * - return from interrupt-handler with the task having set + * TIF_RESCHED_ALLOW * * WARNING: must be called with preemption disabled! */ @@ -8597,28 +8599,32 @@ EXPORT_SYMBOL(__cond_resched_rwlock_write); * SC:preempt_schedule * SC:preempt_schedule_notrace * SC:irqentry_exit_cond_resched + * SC:irqentry_exit_allow_resched * * * NONE: - * cond_resched <- __cond_resched - * might_resched <- RET0 - * preempt_schedule <- NOP - * preempt_schedule_notrace <- NOP - * irqentry_exit_cond_resched <- NOP + * cond_resched <- __cond_resched + * might_resched <- RET0 + * preempt_schedule <- NOP + * preempt_schedule_notrace <- NOP + * irqentry_exit_cond_resched <- NOP + * irqentry_exit_allow_resched <- irqentry_exit_allow_resched * * VOLUNTARY: - * cond_resched <- __cond_resched - * might_resched <- __cond_resched - * preempt_schedule <- NOP - * preempt_schedule_notrace <- NOP - * irqentry_exit_cond_resched <- NOP + * cond_resched <- __cond_resched + * might_resched <- __cond_resched + * preempt_schedule <- NOP + * preempt_schedule_notrace <- NOP + * irqentry_exit_cond_resched <- NOP + * irqentry_exit_allow_resched <- irqentry_exit_allow_resched * * FULL: - * cond_resched <- RET0 - * might_resched <- RET0 - * preempt_schedule <- preempt_schedule - * preempt_schedule_notrace <- preempt_schedule_notrace - * irqentry_exit_cond_resched <- irqentry_exit_cond_resched + * cond_resched <- RET0 + * might_resched <- RET0 + * preempt_schedule <- preempt_schedule + * preempt_schedule_notrace <- preempt_schedule_notrace + * irqentry_exit_cond_resched <- irqentry_exit_cond_resched + * irqentry_exit_allow_resched <- NOP */ enum { From patchwork Mon Apr 3 05:22:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13197677 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 127D3C761AF for ; Mon, 3 Apr 2023 05:23:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 89A27900006; Mon, 3 Apr 2023 01:23:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 871AB900005; Mon, 3 Apr 2023 01:23:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 60041900006; Mon, 3 Apr 2023 01:23:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 4A15E900005 for ; Mon, 3 Apr 2023 01:23:09 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 18EB8A032E for ; Mon, 3 Apr 2023 05:23:09 +0000 (UTC) X-FDA: 80638936098.27.D858448 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf15.hostedemail.com (Postfix) with ESMTP id CA45BA000F for ; Mon, 3 Apr 2023 05:23:05 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=VAXqkuIL; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=hrxFOJGx; spf=pass (imf15.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1680499385; a=rsa-sha256; cv=pass; b=fm05rZJC6UdPZRqTn5JfNxekEziepK9ty88KiwWZwzfajHQo9KKEnG4r6b/53rZmWg7EZR NQexTp9Xr/G+ba23MpCJ/JWVjbmbE9lG8rlEBukvV8zEbCbBFXfpBrZJ+rmtO5pqgJfb9g ydTk7GoP08Uv9RXJUpZJ3hIxu06sg3s= ARC-Authentication-Results: i=2; imf15.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=VAXqkuIL; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=hrxFOJGx; spf=pass (imf15.hostedemail.com: domain of ankur.a.arora@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=ankur.a.arora@oracle.com; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680499385; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4gImNjdBdcopMpLrPUI6c2EyH13eycBU+Tw9ntgOP6Q=; b=VEBaeWnTrSEMock3tAs5SUAnqT/KHxT+Mt7EAvVY6UnG0/W5yS6A28IKVXf4yAS9yAjgeK mg7tEfzy4JZFbobWbR7Nw8nG0RgNCxs6qMxzuGrquZD5UM9+GQfIPyyeAqRhPFc3/ret9i /p7HYCKiTRii4cwEaz4UyFx4gpNu9IM= Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 332ML952010090; Mon, 3 Apr 2023 05:22:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2022-7-12; bh=4gImNjdBdcopMpLrPUI6c2EyH13eycBU+Tw9ntgOP6Q=; b=VAXqkuILTv1fDmRnW2ObaZ5Y3mpmAS5G9sXXZ18ssmMK2aENP3rweY+kmyUosGNQ91ki 9fXkBRil/1wOZlhtaU+QMCN2UYI1MgdRzJUh14rfk5kpsSE3o8ZQC/r5LW/DbF5dTqp3 nvQBeK2nINQTlk2bji6xt8VMjh6ToTPKO8lo7F5nBYswLftjl/wWPCqoJZhf4DA3xlMb Ls4UMQsaP11TpQwNUwnK9h2pPcYfvN8MkKF+pWZBd323QnZ2R3F8VsbCGpKeNsRgn/cP kSniV524vjnqeP+QpNA5PN0BivXSAqtrW7Zh+0vfk0oVeu7gfq6N2Mk/AdSb9Oj82LB1 ug== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3ppcnct4sf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:57 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 3332iZTs038319; Mon, 3 Apr 2023 05:22:56 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2105.outbound.protection.outlook.com [104.47.55.105]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3pptjptnpk-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Apr 2023 05:22:56 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n0ihLWkM/rbL+bJdATRKrvBavYnXp/Zp2f062BDmqwkm9RJckaDHIllhye01eOecD3q6fhbh2R6D96F5miAW0g59PxH2xog9kboGfq72EZm7zqynDB1SeoIeWESZ5QqFgvXn55vXSxXWvR8cQqWuGhFiGL2mtkyldqBuC5udI1X3UhT3CrUpnZ7OISs3bYNRHOwipnicgdAEsEakEEA2GZqYhNqJT6s4GQ0dmslb5ZfvitXlkPnoT4lsiFgs8kNPeAFgkGvQDcUBNkQ8W/TmXBPY9DmK4ZVbXYtXlHy7UM51xPpK/Sobsa9ILXBO0FDMV1ryteKaTvyHQQwRPAsXMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4gImNjdBdcopMpLrPUI6c2EyH13eycBU+Tw9ntgOP6Q=; b=lv91NjVV6ySkt8VungP3ENIpnfjso/xqzrEihbXLS3HQFet6kHAB34nZ+kG5h9YwT371pN/wInALslUkqbhc3VbvtUn21wZP0ABP14lki5Kh54Ig5kAkmLdkGY3sHd9gHNzcra1QUbEAvK+ugrLhim3J5JBYHPqac9+6e6i63lSvuIh649pFOF1/+QGFtf1Vxflwx0J5H/xSudzhso9QSiY4qSnBg6le3ClLoWlHBUdtW1fhwkQ0sO+jlKrp2w0OT2O0xfd94yG+wVjlGv4JNSDKy/DIU0hGntHtmCcqCDIx6FYzG2VldaZI0+K1YZbc+YC4Y987qRAgz2sfYRuO5A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4gImNjdBdcopMpLrPUI6c2EyH13eycBU+Tw9ntgOP6Q=; b=hrxFOJGxUlMf3BtPCEx23f6FMeCKATpfuY9g9kFZjJ2AC7RwqC3kWnXghkSFwEme0Xfu2cYNr6aHsf3KBJHATzZsxw4BLqUbK7gwiukjVUwX9kwczch5q9mrja97kQaMFip1GOyqAbYb16i0k6T6d2psLJx7ZshYY3RuYxMyoOY= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by SA1PR10MB6365.namprd10.prod.outlook.com (2603:10b6:806:255::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.33; Mon, 3 Apr 2023 05:22:50 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::986d:1a90:f60a:53b6%8]) with mapi id 15.20.6254.029; Mon, 3 Apr 2023 05:22:50 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Cc: torvalds@linux-foundation.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, willy@infradead.org, mgorman@suse.de, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, vincent.guittot@linaro.org, jon.grimm@amd.com, bharata@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, ankur.a.arora@oracle.com Subject: [PATCH 9/9] x86/clear_huge_page: make clear_contig_region() preemptible Date: Sun, 2 Apr 2023 22:22:33 -0700 Message-Id: <20230403052233.1880567-10-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230403052233.1880567-1-ankur.a.arora@oracle.com> References: <20230403052233.1880567-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR03CA0003.namprd03.prod.outlook.com (2603:10b6:303:8f::8) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|SA1PR10MB6365:EE_ X-MS-Office365-Filtering-Correlation-Id: e955a7c0-ed1c-47a0-8465-08db340377bb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: EbL+kGjbKMFebho9UeIYDAfwzB792wIeFWpV1vh9sGs83OIYJEydEXYIHVSMhBvRHES+eVIMNRRr79X+Vu9k8NJR0nTHtqmipzbeIt/N45SxqXEE1HkqOpsw35vLkg0hhw4O11CfW6Tjm1Ydw0iNXnYAc9VaVf2vx31yaRT3MtC3LY+qkreaWxsx6ArAKfEdpYrr5JZ2yEYvL5X/7o7u7fAh0E307GMLx5c/ZRqoeAJxdaDo/6vIJYLIaFrciCZq3+24PUK85Ulc3aEp2TVDoaw1q5isZ1T5Ytm21/id2qmRrT0wVgQFgQB0mK4yaMtQUdiBRVyTIdujMCX4S0M2Fry1pLdqhHLMbbr7sRB9lMtrmfTdEnQbnWRXLQuKvJgoFH8i0BzHwW01jgtEW4jXvUh5PQTSAyP6JXXIaXCcSUu3UmB7yiOzj9CCqY2gCqpfQTmDM2VME3yA1oDPVWfEQWGsCCIc5MzSRdzQiK1sqASy817NjtfCyQ2s+Wt1V1vtPzH7qYYJIMTo0p58LbCJI4mCdnCVnEuhJaVOqWD3zjz2ObXyXDEGwnTeU0C7l6hFRXZiCMCHkicNe/I/DAvZp1hR7NG5/8NW8iL7xvfIW10= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(366004)(346002)(136003)(396003)(376002)(39860400002)(451199021)(86362001)(36756003)(103116003)(2906002)(2616005)(83380400001)(186003)(6512007)(6506007)(1076003)(26005)(6486002)(107886003)(6666004)(4326008)(8676002)(66946007)(66556008)(66476007)(41300700001)(5660300002)(7416002)(38100700002)(316002)(478600001)(8936002)(14583001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 7VYrq0RaXD3bgzxE/BXFfDzqbIwOC0DYaCtJqLH5lNpBA63lLnaZWuLJ3tPilkdsy+ZJFMRFm14jYuksDPqGV3NhE+nFW31Gx0vaEQLgYG5DtB1/4/LWNZFKZGBitzCcqxJY57b/BuuxWh4BCdhvg+TjWkuLbzXdsBSWEVOqlYMYmHTLUwXSAPMPzDA/+pKObtOnrr6tHfd4lOc3IdhkyvzaW4lWQD4t/cffS/Ml1DwEx4UrywlbU1Hj/kHXjuyT8VqiKXq0ai9DOg/0HYrndsVHXos9ka/sueJbJl6yD/YRpPi6+cgLSv1v67QlGzejM5H6e0HXF83YfzsXJ96/y37iRufWvXtmOAmanWK2iPi394NaCS3+viVN2qgCB1VFj8qX1hXLn9GWU+KcmiDpo0x/kWRg/SKal4I3Z1vl3qSBuHi1yTPjq8RQzYQfDSwVPGAk7ZqhbIG8BZzoiKsNLiAEgN2vsScLdmHgOiYhN2csxq3lIUNqtes7FGzCU7SkCtYlaK/ua2uzW7M+B1nwETsBpBB2/iont+xIF0Bwbt0ze9z6YaCAjZiIsnDXlK3hZxzBYWZF03ihM20VNuhStgF0JpzrWWKcyiR1JheGP5boZkKHOQufFHpTkjpHWmXF63eXYLTtAzXw1zdOZKpjDGG2huHj9+xoUaM7IIbryj2eRy36xVwSOtpyLorUAztf8Apf6RVje4ZqegPnqIop15cCy5soXu3hfWIQwzulUZCd+WMCQbyfLzHkIJ3FReZTaJablAyOkhqT+26m4mahzcyLVz0U/Vc92pDcE11fGqxQTiVb8ThmjW33PvV/cT6wN+aTCPVYMNzyBjkTXlGpnqsX88hI2FTB53OX1ov08ltA+WmN2SmF6Xz3DUbDRxBkyhHo/GMufWTcWlPjKMiPap1V8VPG95D8w/WyuDEp4BXi5QgFgzd9jVjP0+2no3bSIIXyzkmV3yRpZitM8bH0hmZVJdg5UhdZJBCmD+4poXmAvgeTTmfM0xdlLXqhtbcVARD/MDq7/yhZ9vfbioALR7EjaILBzKIfbrgESsznglPrE5FOo624UjsAYCSeBvyE4Oc42zVx4XZ9SQKAOGQ1BfKGEqN915UW0NpKeSAWqD4CAqif0sv/EEHkxFa71mfJ1cyIplzf5HHgFKFAPrU/+hewPly9TKqOjQVdi+aH0hj8mW9KP/Yc1czmdf4XlgZGp0XvDmfEa4h/SW6QlXs74DUtt1XSa3rSXbGpkUjQcl5yp+q0GGDnI7bbrcVWvnpPcO/KHwBfrQLfxcaVMDTZtvW3EDxyYqlRG3knO5MHmgN37sUf2t6aaEIiYjle6HGXVS35McEoImJlH/k0Vd3cJZWJUUgnb1PIbteM0l3KFT8GAf2vbZBv8jtN/mueAIluhCi8xNFqirWbvdJc5W0oMJuB1XeHB9LxwU3VfZk6Zy5lj6Cm1dL2aMww+S8vQS7aBj1ylgqT02YOlm+G1ovPao3R2e+hBgmqyseJ9pXjE3yqu9govtzFf3ANq6lsGcvx5Rt/cc0rRSGNAYPlyPN79zPXp/56H8nI5DmgYmnRHjQMofC2oVFr6/CwF2ekJkW1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: zr7CdQo1TrKnvgmfFDz3j1d2Anwbz6bLRpTr+VoRl9+nwyQaWpYiomWGBkNGmt0NUftprS8cjk3KoubFfszkqIss2U3xLl9O6CpJQs0ZdlGA9yX9VHUFIgD91tdvLaJ+LPHJylUYK0YrB3ii/R+LmJ8XN/KssKle/J8x7VYaAvnE/nAWfVk+aP7sLnt6DqQxvqA/ldn26Rsm11Xw8JOxrBpnO4hDetLXuztPR89cD2IttP5GzoZaHkB6H8qQwrS5BNuLkY0543Qew2FB3sYTKsLFt6D4b69uQIDKRaE/VoTA53ToVyf3cJjUDXeScPzVYnayT4IdHbrETHe72U3No/NUP4lSfr0GMjKiRCIh2Q6tX1c78RLaeuwIG3tuGToMY22D1tZvgxlae5U/D22tvdBbB5pCj0zBEVSO25PVshq+dYHgJD9dJwtVKc3cZGeKjbbh23Doj0wFK4PPJTX2InbSVk/s6TNgqhVauNFnTIovP40U+Emme9r43WH6gc07gcyzOsx7STmKWRZdcZQ5TRJdO7Yr2tH3iE2BEJXvT67wHGPoadidRC7Dl2zqplJa7cAu5KkVotwPFUPxNsy5EFeXET9/2BUGbzwTASTVVtqwruC4k7ObwzevzvVNm6Kv6+QeSjJvih1Di8YILSSF8G0HuNGLwwWfSlzKip+PONfJp3oAMYzAITu/QxdpsbbzeAAt+4BdxoqGs02NBfAue1MyxOheYQbZlfgaS6N6MG2006KlSFOgu+oAK5/wzsfOXxmzhmp3mSXm9SIgqHgQ8oRs8biJ3hjoOemDPfOQqFsMqEcfv6uJP+e3Z075BjKjyCvXCnmsrPcwe5O4Wa/lQ4lNXDnrz/R35bFHcjkXtl3nQiYI8B3AZ64+aJ/bXhcC2ECr6RwEMb4TwLSCL9OJJ90CxxTYukwfcX4Poc6ome553Z6UX/SpohcMJ/sR7t7D5D3Qj6QlqWFVVG61hYcu/Kcanazh+NBaxSTchoElgq1ntzk5HecvAJ9Nzr3rglZA/3QaD9CD3iOLsmMOqmXkyMdqYIhXyd9YYGo1aLYS7hsoMIXo6DwFL8Et/B7wM2Gdukn2RONgDs8vvxKEUSvw8K5gmy6ZrqqqeQm68rykVCvGS5cvD6HkrHxk7d5DlhnLaV0L3FaPhzXcaJDKZDBeFyKye0K7ju3oztBdh7LZ9uI= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e955a7c0-ed1c-47a0-8465-08db340377bb X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2023 05:22:50.3049 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pToPetuEUS3W64nx5gmBwlvkqB4VZHXRQ5V0Ff95DELUVJ0dIXKXQ2JbahRL8UPalC6BtBeRJbS50q2c4MrN1LLDBTrGYYyWX/4rkAerLAk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR10MB6365 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-03_02,2023-03-31_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 phishscore=0 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2304030039 X-Proofpoint-GUID: JqiodZDSs7DsfTtPyIchBXU6qHB8E9Ma X-Proofpoint-ORIG-GUID: JqiodZDSs7DsfTtPyIchBXU6qHB8E9Ma X-Rspam-User: X-Rspamd-Queue-Id: CA45BA000F X-Rspamd-Server: rspam01 X-Stat-Signature: 1k9cn5gk8zy87ks4r35ir69rkxu36ein X-HE-Tag: 1680499385-24701 X-HE-Meta: U2FsdGVkX18cZXbSMhmN6dhxgP4wlZdMWfh4K3Q92zIdK5/RCBEUAMWtw6zgw58FCQBcVqzcPcs7hyB9NSoiqVomQgxbhylTHqs57pQXd/TczZPXtJDvSxxeXh5mEy4ziFI95f6wvhtoC1cCQW1kSJRTJZPkHPq7yIkzAQsF+9R6zlFbJoPosBzKuJ6P3Xav2AjqbY1G8Rd9JvQzBQSf0K/4MEnmMBYt4bYaLZwNG7v9O/ZaKJhF5+enEsbMubhnu9ZJEGkTdfs+nXJGHBdIXhq5k8il8MZyHoljS7TW6qMwYjGYaAtcndJNcn3dzIOf5+kW0JY0XCc3q6rouKHE2O/cnZ3z7vi7sFBVZ7ImYQC02HzUGYV/aLs9VqQsNECV3fUfQEUdVCrudOpJ3MctrB5jnEauY29dPi3EBcX2mG2FawFO2A5JVKeNhNXQTuxO/RJgxBl0SsVZwadhMgQt0DlmOIlYKeHX+16p8DnLRnq455H4ZQZ4zuc/PIJo0ZekgbA8CzjiyFVPLvGYztC2eEwr1TSyi9t8Ba/M48sFtRIXaYJxbxas8LsQ73Scfemf6lDqRj8rLGw2OEICLaRh8VeJC1gFxMqGD13Youl80bkfvcyD05fCoc2OuKZMOqSaxhmuQSF3Y8znYXzohVesnL9Lmmc8h9aEtcJBbp6UVz8mSYMxKsGdtP9kMEtUcYRQdBliJoQoNXaNF4EEgIzT3OOuugf0LCdVSkZQcwKbzkxUEqceK3wU289pqwdf1V0/FJRYmsTWcI5Sp6VwmTgwSwB2YztK0aQHqDZNB3ZpkVPjoeOLfA+Dfyc/pOcYWw6RSCOfBp/WkUVbUc7ZenQqjxgYBK6mVMlxBfygfWi4yDaqjWoP8X8dvp7mgwjzscsw5vh3No6+27DN3/Iymo/UMcSJhEEQbQXuRoRpGNPMQHu5JGo0UBVqbkz/iNzmnjNGuC4usnBlz1DBzGs/8eP 6do8OIFV scrzQIvkOXKXNvf1PBlPyR/f64466169COhMz+L6RRCOyWabIMjwxHVzh/U4X/iyWWWTNx7e0pZbPLa59Ktjlnwr8+D7w7zdQHtPT4hMqbgz91cRk4SodJzLv/QoWuvK1zIYK+71ZEsDm4eD9FAlcstgtc4gJ7acfhAlkg8fIQyinpOXw70Oc3Kx7NP8LiMnMpl/IU5gcxJ6BEo+mGUmi4wTRgsxpzf6eCU9mpLGLMdD3Mb5RKwj/1OCouJtwZhyBd4loXjWhSTK/VsVOSm/xm+4dnT7+XyGegULCaVuhEcwppbmZDevjKzkkmp/e1Qf6k/PUzLhTK+2x234d+gjDcm6xPIzKXb0RfoijfIqUsp1ply3QqfJto+JTML6k1Gyt7dO4EFuTRIVo3zhSOUpiNkAMNjyu44qODB7rvo9CeKjeJ5y4GbMYWyn3jiM/E+dJv4xZ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: clear_contig_region() can be used to clear up to a huge-page (2MB/1GB) chunk Allow preemption in the irqentry_exit path to make sure we don't hold on to the CPU for an arbitrarily long period. Performance: vm-scalability/case-anon-w-seq-hugetlb mmaps an anonymous hugetlb-2mb region, and then writes sequentially to the region, demand faulting pages on the way. This test, with a CONFIG_VOLUNTARY config shows the effects of this change: stime drops (~18% on Icelakex, ~5% on Milan), while the utime goes up (~15% on Icelakex, ~13% on Milan.) *Icelakex* mm/clear_huge_page x86/clear_huge_page change (mem=4GB/task, tasks=128) stime 293.02 +- .49% 239.39 +- .83% -18.30% utime 440.11 +- .28% 508.74 +- .60% +15.59% wall-clock 5.96 +- .33% 6.27 +-2.23% + 5.20% *Milan* mm/clear_huge_page x86/clear_huge_page change (mem=1GB/task, tasks=512) stime 490.95 +- 3.55% 466.90 +- 4.79% - 4.89% utime 276.43 +- 2.85% 311.97 +- 5.15% +12.85% wall-clock 3.74 +- 6.41% 3.58 +- 7.82% - 4.27% The drop in stime is due to REP; STOS being more efficient for bigger extents. The increase in utime is due to cache effects of that change: mm/clear_huge_page() clears page-at-a-time, while narrowing towards the faulting page; while x86/clear_huge_page only optimizes for cache locality in the local neighbourhood of the faulting address. This effect on utime is visible via the increased L1-dcache-load-misses and LLC-load* and an increased backend boundedness for perf user-stat --all-user on Icelakex. The effect is slight but given the heavy cache pressure generated by the test, shows up in the drop in user IPC: - 9,455,243,414,829 instructions # 2.75 insn per cycle ( +- 14.14% ) (46.17%) - 2,367,920,864,112 L1-dcache-loads # 1.054 G/sec ( +- 14.14% ) (69.24%) - 42,075,182,813 L1-dcache-load-misses # 2.96% of all L1-dcache accesses ( +- 14.14% ) (69.24%) - 20,365,688 LLC-loads # 9.064 K/sec ( +- 13.98% ) (69.24%) - 890,382 LLC-load-misses # 7.18% of all LL-cache accesses ( +- 14.91% ) (69.24%) + 9,467,796,660,698 instructions # 2.37 insn per cycle ( +- 14.14% ) (46.16%) + 2,369,973,307,561 L1-dcache-loads # 1.027 G/sec ( +- 14.14% ) (69.24%) + 42,155,621,201 L1-dcache-load-misses # 2.96% of all L1-dcache accesses ( +- 14.14% ) (69.24%) + 22,116,300 LLC-loads # 9.588 K/sec ( +- 14.20% ) (69.24%) + 1,355,607 LLC-load-misses # 10.29% of all LL-cache accesses ( +- 15.49% ) (69.25%) Given the fact that the stime improves for all loads using this path, while the utime drop is load dependent add this change. Signed-off-by: Ankur Arora --- arch/x86/mm/hugetlbpage.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c index 4294b77c4f18..c8564b0552e5 100644 --- a/arch/x86/mm/hugetlbpage.c +++ b/arch/x86/mm/hugetlbpage.c @@ -158,7 +158,17 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr, static void clear_contig_region(struct page *page, unsigned long vaddr, unsigned int npages) { + might_sleep(); + + /* + * We might be clearing a large region. + * Allow rescheduling. + */ + allow_resched(); clear_user_pages(page_address(page), vaddr, page, npages); + disallow_resched(); + + cond_resched(); } void clear_huge_page(struct page *page,