From patchwork Sat Dec 7 16:55:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13898338 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDFE5E7717B for ; Sat, 7 Dec 2024 16:55:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3661A6B0338; Sat, 7 Dec 2024 11:55:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3163D6B0339; Sat, 7 Dec 2024 11:55:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 18FAD6B033A; Sat, 7 Dec 2024 11:55:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id F088C6B0338 for ; Sat, 7 Dec 2024 11:55:19 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 70FA8121D62 for ; Sat, 7 Dec 2024 16:55:19 +0000 (UTC) X-FDA: 82868762844.25.1A998A3 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2083.outbound.protection.outlook.com [40.107.96.83]) by imf07.hostedemail.com (Postfix) with ESMTP id 0050F40015 for ; Sat, 7 Dec 2024 16:54:57 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=V47MbQOW; spf=pass (imf07.hostedemail.com: domain of ziy@nvidia.com designates 40.107.96.83 as permitted sender) smtp.mailfrom=ziy@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733590503; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=91j+UUl+idUfk8C4PcBDPsKC/XMGlAuu8q1FPv8PRf8=; b=KkN6wvSre+ECgxqWmPAI/E4Sq3hzzs2m3WnnnLuc2LeQkmCYzaKBx1WwE9eJo1mgQUR0b7 uKe3KQHMVqk13dmncs+6yP+nUpQx3etO6DrYXDy91YmNCg/bYuApPpMzZNi1dnQIdZeBFI RZUIIWM3TTXn9bPDzD85bvwWRS7hwN4= ARC-Authentication-Results: i=2; imf07.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=V47MbQOW; spf=pass (imf07.hostedemail.com: domain of ziy@nvidia.com designates 40.107.96.83 as permitted sender) smtp.mailfrom=ziy@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1733590503; a=rsa-sha256; cv=pass; b=JxvcBlRry5iwptTrhim7niVhMIIPP+c3YUkyz7818wpxsLJen3H+PYrXvIcP0C/ER2eDe4 to/FlaBeyYU98nEyR/9G91gh17JWSwx5Y5Ml0BcLc4dYRT8Szad2AqLaiifKkhFY36ZJ7u QCRmuE/MidVNpk5JCeMaiApp3yTsO0E= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=g43+6FPxTHKH97biJc+nvbm8RzTsofVNLdC0RqBlh6FbW66qQeuZRRygGmHdtA23h16ZUMiu4czPC+10ZEgUjkV1mjeTK5JCXKNG5ZAjBSNoIJVZUfWG2nNvKOWZ91XBpYGN52oFJJOpLiKh8hHbfPs9nZ69a1lfRqUBCP/yFDNKqIc4vlUn4AHcB9MqnQNb8pMO+tOx7aa6de9kPJp3jpdJpp5Oy6B+dCUzmPDFQHP4RlaQOcvPxJw5DRV1LDWska+BUA7mwPwad6P8j9+fdEpLE/IIcDcoeGehyNuKETAKqaj5H5ZBlK7DmFmJruzOV0G0EqhAY4nCdbmOaOYVxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=91j+UUl+idUfk8C4PcBDPsKC/XMGlAuu8q1FPv8PRf8=; b=mLP93/Rhg/Iw04TxgHAFwfJ7RHBRbWW0HZMw3oag6OfgxYmPxh5SpSvi2UdeOcOkYVLPGpVa/Vkc9L+5OU/MCx1AbqIV+zcrXgRgJ6XPILqGMqIIzmrUrcmjBU5sStWcBmRXIZX5jskhP4JCNpLDzIxH7TuMGyceufnRjJB6emzwfZHzj3cBZKR9/HknAYX/Zrm4z1f2WM6LhNKQuT+FhA1cocobANGEZ+T3GFQ7kH9fUMxJuEwpNREKx9jXbr6iQFY0i+h2v4KepBwDUwFZL154KKJtvB0tMv2K7PQsioDvfoL7JSPGqJyIc+GkR7PooBOaVe8yDcRqqM8JYebgzA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=91j+UUl+idUfk8C4PcBDPsKC/XMGlAuu8q1FPv8PRf8=; b=V47MbQOWyasGPmL8ayyfFSyb8+RHXMhO30Qn5mT9Ec0HHsT88G6DM9p5THbIkXJt+L/fHnD1QxglxnPehW2oXuCg+7/jy12JW9QBdDCaYLqJaGZoA9wwKhBiX5Nx1i/TlD9xS9J5R9mMQL7aqyFrG7wR2Dnr9rcGfqbqNXxAIdGr+mGt5/dlRZHxEdnH1u1ezE8MRwxBIAqYHnQoHsWU8F1rkWKR1CdeKzes27fcQZ3QQ6J+vvAGTzeAb4qgSeGcd7tjFDMOm01td2R7YgNHYBJva2nNBEjh5F8wZWaeGKuqoNxM26wRESbUbh2jou5actS7h+f0lJLAtgfvPuK+xg== Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by SJ0PR12MB6735.namprd12.prod.outlook.com (2603:10b6:a03:479::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8230.12; Sat, 7 Dec 2024 16:55:10 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%3]) with mapi id 15.20.8230.016; Sat, 7 Dec 2024 16:55:09 +0000 From: Zi Yan To: linux-mm@kvack.org, Andrew Morton , Geert Uytterhoeven , Mathieu Desnoyers Cc: Vlastimil Babka , David Hildenbrand , "Matthew Wilcox (Oracle)" , Miaohe Lin , Kefeng Wang , John Hubbard , "Huang, Ying" , Ryan Roberts , Alexander Potapenko , Kees Cook , Vineet Gupta , linux-kernel@vger.kernel.org, linux-snps-arc@lists.infradead.org, Zi Yan Subject: [PATCH v3 1/2] Introduce cpu_icache_is_aliasing() across all architectures Date: Sat, 7 Dec 2024 11:55:03 -0500 Message-ID: <20241207165504.2852058-1-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 X-ClientProxiedBy: BL1P223CA0005.NAMP223.PROD.OUTLOOK.COM (2603:10b6:208:2c4::10) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|SJ0PR12MB6735:EE_ X-MS-Office365-Filtering-Correlation-Id: e3886f22-a212-4934-d62c-08dd16dfe8b1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: M1tUwmlTVXyR8rf5YZ1qPQGr9tPtkknJ2OAc0QV/OJlTju0MFWf47c3TXPQDgX9b496zkigK+5C8Kvlkcrr9b43EhW79oH/VKPUCSAw75RUfcCLDK02+f+S6I55up0bFnZt8AGLXYSiKJJ2YUAuxF3XuXjSQUAgjj6x0ZNgWqzudmWuUBv23iLiTEZmVUGJqzSQTBUsj0ppoaajJWq1SJS7V4qObykfADektQXolGOHRUjl49X/yx2Bu4UINEJvhk4hwdXyBWixxLOqKQde4kDip6LpX8F+vR37ErTKsGz0wuU+ip75TBeZf9ZpqIqzixbAqJ6j6kIflt/T/MvwErbpcIVEFksj7Cp6vbKVldLC/1OsBgPALVnQ6JevYY6LIpQyvieL1+1F/nguohcTTMpvuv7u6Z3gBQR37yyeP3zNnWD4vBR6B6ySNhgD5v5eeO/pvDRl7cmsvpFhryW8eVCHaaBkCvmepvxbGI9b5r0q0qL2FlviheNWfUkX/ZGfEF0/OXBsQz5h2uHVabqLfcEOd70c6i2+TnPdL8NunoTAdvvPaiQ827pButoqoy1Sh0Xw8zcZy6JqxcxdVWDxoDMyBUxIg9emCQp6i/lf2zjrcR2wJW4MXCKRRu8CqTtbsyez30dndj5xoFHB4H1VcGj1Xxr2lkm2UwTsq4SLBTPKY8FYCY+sbZXcR/B81/vRobcYDzOBrMuwnwoVG/KYUfdAjz3958mARjXl/F7mj/BaYe8/Gr/nABlD3715+862A363U+7rIKccaBVxurdMN5Zsp5qvsku91xRx3GbH8ozyDmGqOHBI023rBe2f9MKPqH/3ypZAuqrl49MkaOauO5iYKeK75JkMZjKXVGdDrlQlTAFbcrzcroctCdNYIEgRGCg96/HAtatmfQTFBL97jlDvoYyle83YNlZD4fSsELAWNJ/MRNryVUaJhqmvs0DnVSos74VGUYHBqEutIf/t/r8FlIAfa2GvMBD9fdtSNJ7cVjpvbsBFckU52j5LFFxE3ZDypW7LvkTTnCnRw4ePH0p0suTo2zuT6q0oZZ08cIfi0mJ2BB1MqCeEP1NcDRkeVDz4mQVu+KFA8kVoMDDwjlImgA1t6xVDME2/vifRnE9vYkH6P5dAqVcAQ9qJWxsOHyqjua01EBJYfDbN+kwW6FL0ae7x7A1sznf7Kn5sbXXjKKoRX9R7PvIVsqxeHc6Z1DLe+229cZSCcZpZ642cxTm44+IDUPPmiZT/RVQgiNmDt4tOtvJnBodYy3lNLZcgVfy3GKfCC+mQ1uDlD7H0SuDGXPRjgZ9Cd2gmGZ/w5/VZfv4XA4h7qVaeCnCKS36FZMFgH2MQr0jLtq6rjl5CwDQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Ai+mS0QPDU/qlmtVMKrtIkkXNm5shfJRKSaF17U2nBw4ynq4HjekfoChoxcAC8F4NZR85ATCf8YbbRlVerNf8lmshYUtvr1umyIdfqpZxvgqZIRIgpMJ+o9/eS3YAhOZEn/K3uyeWWSMiRmaLQTqpklamKMqQiccQnw6z4zp6FtIH48kTcN6RCczZi4ui+dRqdO8D1kmM0b2QwxnitIGZcAllZarTtvDpzAUvSs13Ez2HdEwhxMT4Hm90Y/Pv+oulHwd+wTV6LM5ErVZH8fH4Tsh6/9of5HIE+yPVUhblPNEpLfrUajS3VApAQ2RK7eRaFiYbytoQ4quHIFMjKwSUu8Ngz04LO9cfWmh7Huuwe5lT0K9tjOZnuzwdGJjxTi4+DQtr7j0i/U0q7RYHovRx68pfCYZ9T6p3mmLk0deyqm3z5xqrE1BcK7WN6/KOObziqVE/G5ma6i0gRLnJKLxpYqID6KkOntLWSuSfFhiOLz68FmKJ0xJqUdi5WdBGvUoz3DUv7ra7umLUkv7rTsTkzEyWVLatQIPu1z5ovAye3TlNMzPdElfA43sMXf11ATAzVlIhiV05WWe/YjtCAi2bc/eghT91lvX34brCrWYgmVarJqg/999MCkUSoJW8PSytpK9WiQXijcYJtdE+/fSE1i1qKQ5Yog1G4hPU9TRkBKeOvoUhexp2Cefj87dRybCnJgwnSit3OkplS7NCIjPY7F3wPy0AD5Z4PQHaI/TrKUE58JQIXaLLX4OmUgj8k+dR/a4315GUZ8mzgK6aaWHQVIWUea8s3GnmtCVZoTcqNIurDmfXd0ov+UQ3lReWTFLXnyovCK2Fb73rWu6ZnvOA4ZH0lJooKrWbB7Kb4VtUgMgqNqQUk6SX63vIv6L+/htfEIs4bTr1Y4DwV22awGtzNRiVnM3dzpc4kHUcBX0/R7HmQo71tCHG31wkXcQr6RN5NnhqNVZxnp1zMnxTdvTNNt78XGJRnm4cJ+xAtulGlydRcKMRuO5NT+f56WnL3/tBC/v2sdHBwFjLCu9NkFg39nrbMdkrQLGBTBgrjmNFixmtAuDP6GBHpJUrY9zM44WMTYT98NK32QuNS19rARIjFdiMPORUbQmbbvz+e9QVMZ567pQIGQYVkJJBp5FJ5KVblzBticvbDFHYgziUR2s8dQZeCX8e2T0jQ+z3Ju/npWl752OswBNDQqKjXDq+hBP7ity+xo9pVeEIwrOz+hQjMtHkIBfg9JipTEt70HJa18+Or+g7snQzympo70+geHgb/Pz01IiEmiNxuNuJT5YVbI7E6QBBk/HMB7dY/79WEM53c268kcTDFF5+dWqwKptUikJrG7PsbUm/1n6QucJ6iFF3xqzP+V38/b2sX9LkiL87Yc6Z2pa0I+1+VZQ0/c9mEQLcxkWj98SWztjtM3cKXUZ5co1gQ751qFqBV+GMrqQGzsqwRQM+zl6VBWbg+xycja9vrTbGWKCYzywRX+NqZv3vaI/kZPxdmyoyEFmcHQl0gQ9YlTPlQzD0feURJNRR/36LwcYMc9uiztcGYv5Hz54wehcucyCJPdHMd0RH9u5zakQYqNkB/4n33AGkVm0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e3886f22-a212-4934-d62c-08dd16dfe8b1 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Dec 2024 16:55:09.4966 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: JbnLDO/X15uEFtsOEmAxTDXDBsGj+9RVqOXbXeJNabl05NAnhsQJSTeqLDavq6nn X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6735 X-Rspamd-Queue-Id: 0050F40015 X-Rspamd-Server: rspam12 X-Stat-Signature: j6cbfcreh6ep4cbi9oq8wjufhtce86sg X-Rspam-User: X-HE-Tag: 1733590497-755457 X-HE-Meta: U2FsdGVkX19pxHgkvgnf2vhZqFE97Iamruuw5wG3GQprgJXhQ0GpYlta2EHcl5hCW6seMaOranLVZMM3XfoQzUM5pXCq1dF7psBbwSGvGTE1fa3aQ1XLqdg1t5i+3azSbjFbvxGyAz5KXPb+AjKlvmXg5ECkO5Vx5DkyafTn3998FFx2mnm9J3q4ws5s8vx7/ZS2ku8cqJtb6CyN4ZS71MtoihZNuIMMVOauKyxP/K7W6Qzmv0eXbUZXkwyYmflHJAKPRSclfHu813i6Ku0dPcN5qkpAOiuJ1ao7T7bfeXRX8qTAL7cyjW0RvjhdCuK0plH/tDh0OVse/Sd05cVsMKqAQ243Sb7YVahugIv4fHLqB2YV4K88FpWLbsMrwmGFNzN8Bq3rfLTxDu5FKwt0J465vrkTPNaT9Hey4TDTgv9ii8CRxCcnVtFovLKYipfIVa/dsbOZixbZOsuffyxQW2mkv3BNQrTgVCfH+DzblErWlaQygQqiVe05l86JlQVV+Bif57R2jxqgGMc6BlRClDAjuTdWse3hYDvpPtVOnl9a4ko3l5dfrVi8j5Gt/9vA5cxPgYUDVl0QQAuoSsTr1CZQbzisWbEakPhnnYhTOvpAS6ZvMkiYVk0Pe+e/uA4YrLa445RCbTo0fwAocP5kYrUacpoV9siulW8RuCwUoxeQ+5errq2b/B5Enhpj2ZnNvH5Q0Zx7VQDujl4gK4VXufk82t7TZyegGLDshonhHWFkjkdSMncvMkVOadp+nzIKC8rbl0HQPOu6rNZ/boqr4wkr43cnx/4Jbv/w9ovqQjZpsolFFxxGA4C4JDxW7lQk6rKP+skksP0fatz1yEvkcc1sPdI8a3pUIVSdak8iXcs7IyxutUdIeDoLGz2gToOb4BXA7s6sShRErhHUv9Jx+W3K/AIUHi0kIEWPhxVz8cq3VdzdDR4jVoObCXBk5emEcThu8eTPVeBiAuT/eSD Od1cZ9TD 9IczqrYOLYae47GtOlcouyoXu+dBa4ogRV4kobWawZFZjD5BUf6jdtwNzRxqeuj1Jdm1HfBd/qs1fn/rvFjIty5atwdXCCJioc5JIEYW4WUVzNP2wS2R/xWsonRiAX14NzQIw2FLohe0qVlntWtUVSV97+zoMVeIOVglaSqTGpmGW4Q8YjQ98PraYLCgxnwrqYRZkkGs6noe/iarYZqkctXDr5cgiSKfALlucGrhZ2m9yWsCbVoG/vo/kEQqPUKYCeDOE/2K8m4LUH+0I3j4sOByDjl4Nb+vOgjlxcbG4UlTHqEEAb4m3FB7GtBVjhDTCXwbg3RjXr/5NRA1RvdQT2vG1Pyl0DvobKKXz9jGyJHCq0IsBHrtPmspV7J3flW3dYPAKt+fua+w3MKPNKFLdbs9w6MI/TARwsVFmOlIDkYLtf2g5HH8EMxr+h7JF8cr7NBt7 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In commit eacd0e950dc2 ("ARC: [mm] Lazy D-cache flush (non aliasing VIPT)"), arc adds the need to flush dcache to make icache see the code page change. This also requires special handling for clear_user_(high)page(). Introduce cpu_icache_is_aliasing() to make MM code query special clear_user_(high)page() easier. This will be used by the following commit. Suggested-by: Mathieu Desnoyers Signed-off-by: Zi Yan Reviewed-by: Mathieu Desnoyers --- arch/arc/Kconfig | 1 + arch/arc/include/asm/cachetype.h | 8 ++++++++ include/linux/cacheinfo.h | 6 ++++++ 3 files changed, 15 insertions(+) create mode 100644 arch/arc/include/asm/cachetype.h diff --git a/arch/arc/Kconfig b/arch/arc/Kconfig index 5b2488142041..e96935373796 100644 --- a/arch/arc/Kconfig +++ b/arch/arc/Kconfig @@ -6,6 +6,7 @@ config ARC def_bool y select ARC_TIMERS + select ARCH_HAS_CPU_CACHE_ALIASING select ARCH_HAS_CACHE_LINE_SIZE select ARCH_HAS_DEBUG_VM_PGTABLE select ARCH_HAS_DMA_PREP_COHERENT diff --git a/arch/arc/include/asm/cachetype.h b/arch/arc/include/asm/cachetype.h new file mode 100644 index 000000000000..acd3b6cb4bf5 --- /dev/null +++ b/arch/arc/include/asm/cachetype.h @@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_ARC_CACHETYPE_H +#define __ASM_ARC_CACHETYPE_H + +#define cpu_dcache_is_aliasing() false +#define cpu_icache_is_aliasing() true + +#endif diff --git a/include/linux/cacheinfo.h b/include/linux/cacheinfo.h index 108060612bb8..7ad736538649 100644 --- a/include/linux/cacheinfo.h +++ b/include/linux/cacheinfo.h @@ -155,8 +155,14 @@ static inline int get_cpu_cacheinfo_id(int cpu, int level) #ifndef CONFIG_ARCH_HAS_CPU_CACHE_ALIASING #define cpu_dcache_is_aliasing() false +#define cpu_icache_is_aliasing() cpu_dcache_is_aliasing() #else #include + +#ifndef cpu_icache_is_aliasing +#define cpu_icache_is_aliasing() cpu_dcache_is_aliasing() +#endif + #endif #endif /* _LINUX_CACHEINFO_H */ From patchwork Sat Dec 7 16:55:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13898339 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A554CE7717B for ; Sat, 7 Dec 2024 16:55:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 340DF6B0339; Sat, 7 Dec 2024 11:55:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C8E56B033B; Sat, 7 Dec 2024 11:55:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0CCBE6B033C; Sat, 7 Dec 2024 11:55:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id D706C6B0339 for ; Sat, 7 Dec 2024 11:55:33 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 942B0A1F04 for ; Sat, 7 Dec 2024 16:55:33 +0000 (UTC) X-FDA: 82868763306.01.8723C6E Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2067.outbound.protection.outlook.com [40.107.96.67]) by imf17.hostedemail.com (Postfix) with ESMTP id E087140003 for ; Sat, 7 Dec 2024 16:55:16 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=ZHnwWI8t; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf17.hostedemail.com: domain of ziy@nvidia.com designates 40.107.96.67 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733590512; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lFymnYIwzybc1lYwKoXrwnk6I4MsTHm/oq5M30dD9SA=; b=MBHeb8H2ZEkHB7O22Cj2WKlgNxTdyASWxk/q4oYxU/EB308vd38EMHEiPqxbnFbDFshf+y vrVz1VB8ua3oTPdNCy8CxW3f/c1lBo5EdoXVefGcTqDy8cUJ5NY65gBAivm+AJ+l2NeFOO YmOrCTUWpB3+xmvvcI6Kk53PAzhsMiU= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1733590512; a=rsa-sha256; cv=pass; b=RMEN9i3zTgoJbMBg0eY0o4a7utuHIwJ0VyaevFwf23llxE6Y6Wx94vLy2vYl0AI/f0hBbf H0Ra/t+XoFCwiTff7JHA5J67nShOmEMVEXpAenaVjHbj596RsDf1aa/u1cG2pUFcJ05ljy di75BzaHprHoSVMd8+ry6VVKex3VCpw= ARC-Authentication-Results: i=2; imf17.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=ZHnwWI8t; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf17.hostedemail.com: domain of ziy@nvidia.com designates 40.107.96.67 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KRDU83OkNRVGutl0w4MmMFptf8/NmNjBGb6n6fli3OL+8O3msTsRQoYxsWMYO2eUd53+GjyxfIjpQX/i5bH0NEpcaEDdTEjqlksmGas9hCw/R44UKA6lsZRtyvcIvIBG/5mebMOIpHaIEN9Kud5v8l2AkD/2GirOPzsrHBeAow+WxKSsfxRY8qjy5232K8aPDPhwnoGPDtQi1O9yYpKdIpdZi7+mqSGGUEvZyx1dj9CdOA081zU524KYPYYcPFWTDDbGfxEHpliWfwX75BSuzOUrjTp7tGlMtFFtUWg6+zYgQ6lTjaYZ+Dbda0SBigvEQUODCmwdtQvW0C0qwU0i1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lFymnYIwzybc1lYwKoXrwnk6I4MsTHm/oq5M30dD9SA=; b=WfMpszucI8OD2QLMmQdUCjyXD6UzoDuurnWr3Rihe7bXuEqWIc6VX/5z1XcieWIp9aeajr8BmNaZeK/JeboBJzYV4LRdWtcFCYFTT4o3FvfFcwxhoGkLfRdezl2pcy8b2W2Hnc2jY7V3hGXzHcj4YA1RU1ZLOd7Nbezqw92Yvczw0MMFebhU3/6tO3gPaKOWIiqc1MpmCustJX66oFoPUcuz2W316krF0kxIKYNcciXS3/VERFJ6WorrIHsJ6BWtUznFjrzMpKj7Jlu8hL0Ij1jPzgNWpst/yGKHxHLQHjhX8FGCJ8Hmvdjxqc4HkZz93NN9t6hD/2qDsYaBQ+vj6g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lFymnYIwzybc1lYwKoXrwnk6I4MsTHm/oq5M30dD9SA=; b=ZHnwWI8tw1GAvxMP1xppA733jAVQ/gm7+QTbrBOk9AAEu3RFUJmu0Ci7l5bUTiLaVNC2nj1PFEj/+q5ggig//LBcVaZ5z8mxw23mvtCskNZtEm87TSvyCZafausfSp7GDK6AY0Toduw1S6l0lDKuxt09MhwbcJARaqP2d5RVChhK/ZC30071acM9qEJf2tvgsTdRjlr6FaaYBgd0ePsCCV1DMprDhK0Uf0t3e8jbDC2VEvwjTrtqtqW/+VuRKEBHMbcjxbT+Yw+cKe9eumOzrTdG8VF3AIoTVJIqvt3d1BBVOR9B/tUD4ktTWEHhrvz/9WcMg53VmWtKDFKg+fZjyg== Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by SJ0PR12MB6735.namprd12.prod.outlook.com (2603:10b6:a03:479::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8230.12; Sat, 7 Dec 2024 16:55:11 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%3]) with mapi id 15.20.8230.016; Sat, 7 Dec 2024 16:55:11 +0000 From: Zi Yan To: linux-mm@kvack.org, Andrew Morton , Geert Uytterhoeven , Mathieu Desnoyers Cc: Vlastimil Babka , David Hildenbrand , "Matthew Wilcox (Oracle)" , Miaohe Lin , Kefeng Wang , John Hubbard , "Huang, Ying" , Ryan Roberts , Alexander Potapenko , Kees Cook , Vineet Gupta , linux-kernel@vger.kernel.org, linux-snps-arc@lists.infradead.org, Zi Yan , Geert Uytterhoeven Subject: [PATCH v3 2/2] mm: use clear_user_(high)page() for arch with special user folio handling Date: Sat, 7 Dec 2024 11:55:04 -0500 Message-ID: <20241207165504.2852058-2-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241207165504.2852058-1-ziy@nvidia.com> References: <20241207165504.2852058-1-ziy@nvidia.com> X-ClientProxiedBy: BL1P223CA0021.NAMP223.PROD.OUTLOOK.COM (2603:10b6:208:2c4::26) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|SJ0PR12MB6735:EE_ X-MS-Office365-Filtering-Correlation-Id: af7466e2-8880-4c69-543e-08dd16dfe97e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: zlI+7oHK89o3KMZqk96F14Hc8k8mwdZvpKJKp98yRmJQ6CXx3fLNmldTzaXiyxIOAxf7nqOSofZdMr6gLAkpz1eiddHwP5WxVzidt/FFNLSXdlz8ioLaYMgyc82v0JaoWWt901yVJG/lyhNAHigPVHXnOe/Sc8zwr/GmM9ZUri+t/rzcMZu6HhNDhaJJ4Gjlzkg3mScWkqTP68MJt6De3VhPO177r/Ebxd0ZQDbK36jaBiEBOUcyuCDKSyj9cXv83c0plxdpUM8+dTM/WV5fbL3p+f0xw1oag98M7PxHawTFK8DYI9eIhjWWnJXHTSH52cCe5FuCGTu4rsTfsKZMVg9LmAqcIyIbwH2p+NGBtwt/jcPulvFK3KOS6j2KeYpFFTJYcoYIMu4lOFGjkLoP7qje3HY68Lpe+VxfAcUhF+7GER72jrKk41opT3rcv1vanoY6to0G1jzaiI4On9SVkG39s0lcrmAncdIXYByYgINVVKNuxr/F6WjJ/VY+hg98cYj1DVn4zL+wElCpebeS6Ks+91Rj/qTpMUYIHPlJnpp/5M2qToMIW2o5Eo16jg6t/3OevTjGvrEzPmySQYfazMRA+kRnM4X1iZdlTtblXf371CXXuTu0KNxxU3tYOXaoZJM4FAEVDe1/FU9h978WJuCbjK5P0Jgp5Y5LfMglgo8Zjdrdwt3MACCDQaLwZUjMTevCyDmx+i/UHYqSKjeHEIzhv6+Qsuc/LPLHAFP7l1h0eiEm7ULUXqAuZndkw6EGxpI6GE03JQiO8/vr6A9oJnigltzuXOALHEXaS8kZHkPKnTxbEsyot63gs4BdjvaC/JVbNUcsBJCyL3JzrJ77riEmWOvcI6EqRUJpRk3GFZuX1+tt4YtQDdtifjukj5nXyW9tDDBVnSSgz+9yYmO+LO0XeFhWFJeyHz37w1vVujBpnJS3BgqVgSde0IyZ3m5LBYbZwKttzVHd+GU2CfXavp74swRlXVTCnOsRapMEyLJYM955Y+gGAC7Wnkr2HywyWX+ZVChTed0igEIUTVfWaOvwGep5yDVN/xp1aSr2L7yZir9wepmTyRQD/rPCpSu1z4rlnDRRPQgSNul1rlf1YPPbUrHPZv+8eCbjD/CkD2gWOXL5mr1vCMbb3lVQsbM0X/cGqS0u1tDH0TdJ1Tvp1D08FqLn/p6GsiXuXwY/Qnwsv0lP6lmBMY3+4/fAqq/2QJG5arrBgx4qhFYPrj1BdpFIhXjDBV+rXCdo/wULzUZ8S1iY1BSipq9k1uOFEupgRifdx1Io679fPMoG69gVqtUVJdn20wNJO1SxRFTIvEle5QICDx5lgVcLuFAG41GIdWbBG51Mhl4ix0aIMwdtj2z9PMEqwpNIFMKDJ2Q6BSY= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: nRZIhenbTdBVJSHcopmStKGcc4DeMe+3FUunW6jBSAHcJ3XTKWrcFZCxtyxajehpGEVv8hpsZ5VbritZGDseP92+6zJurde9e+CndBCSnPmvm4jWAAOxCDf/fOoXZQGShgsrh9DKWrxJGA/+y9LbNqETez8HawkH6xblFOTogfjWFsnoHR41NOsKyaW3rR9xIfR/LhoosMwcvYCSyozvOpQIbQYuXryPQPf5a9XihbJAMlYlhvbvgo8ScLTNFUUTpyA8GNg4VxR+q8+7ePTusJyhcPqpr0OxSn5VpG1kCTbsojovuxaTF8QsUctIf208b0FiUxBVZN6auHULJcxu/hbzR+1dWpghVqBHue7lZnbgoDuWTjxP2Pc/nlgRLqXCix9GpVb0kB617m6Pf/VMtYBZqENobFz25g25oVz3tt+2JQ/avBcVPkrZVP3+lEaI+zW4JlBYQ99dC6m5N5GRNjR+/FsX3gQ6Ct7DihuJPk3T+BhwwCdk5c0VC0HOLA3b1V+rd821UCsFgJzirCq8duaxLvV3Z+J/l9BKL5X/tywuxNCYp/kdb3lKeH4/W16WRgeSlTsA2dQ7anxRM2fkm/nii4O05InPo9UdX+nVmefNVMavaQFsU02Mjc/gwWqVl09EUyzm3w2QCIhSD8pe+ImZo8cE4ih7pMWkiJOqCp+Nx2tMQ8rKAkmE6W5rL+EaAgJxEwmJP7nbTEyXqcHT5jQINkvTb8dwNFAAHFXLoQyTXjGfkY6BFTU1yTmk05GaYYsUsBrGShgZ7MVnXlK6rTIHC6DQ8zv2yjRO+eevFA9hvU4zwuGs6ulssD8jp2OWvhs6PzqgVM1YcXa2+UxRH1fmADOj7xV1kDI33UIHr2D1+niCNkw89Nmql8F/kXRXYL+E++G75G8uP/ItYsJ3T0vP4i0F1S/WmV5u7m/cI4l+0ZCQ7DQme8ph1KNFKt8cUQKZq3eMDS1P/84j0j8jH8qVkboDVP7uuEtIFtGJWnm6cKg02Po/1LlYHyrWLIAEc0dY8rIuGriCVcu4Db7xIyIGzCtjMrZd4nAUMnFiVCZQdZb+2C9qeyFRX8eYqrNVgYKqYqfVJsPZftHuymPb9tBGw5VlxGpHIDxTyphFe3O4yakEhkqBpMxG7Vyj9SZG2ZEk5lUtgF2cai1Mk35CNgdOO/QWtD6DadiOmz7wI2G/DfQC+8T2QSuC4xsdYolpJsY+yWR2x7KfuPOekC/s6LdsIwotaRMyXpR/siHEa0biOtNHxR9oTIHCHIdZ5ezG/ulU7amHOzHpDv8WSzCvtzP0tlQSigWa53gEmvw0hxrKOWm/0dKZvwpBLxUo85/mv0zIvFSjqg5qfPJf1WDNRyya08a5UwD8ZOb/Zztxa56wH2JBG8ghQtr6WbHifPoLUdRv3t3W9KCMUXcY6AIsDBqhVg3F0CSyUeic5PHDh3fiCbv5toJIMb+7nAgqNtqpaslT8AZOEaz4dUnitp6h5Nr9jQ7BIlbOWx11CMAn5n8qtco4c9KH1Y7e/xm3xvUr/VdwuzTD5eU4I/5r90dCH1fxRLu5Oa/hD0K5NX62S875vqcJrLpyR5xO8+jw9aXF X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: af7466e2-8880-4c69-543e-08dd16dfe97e X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Dec 2024 16:55:10.8389 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zJPdy7woxaLIwUhFOFwDNMQwlkScUHWjgGUAfWRhYEoV3vNRimhLVbBi0A1iPnpQ X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6735 X-Rspamd-Queue-Id: E087140003 X-Rspam-User: X-Rspamd-Server: rspam07 X-Stat-Signature: 1xqjtm8nuozt8raunk7z9owswpes81ap X-HE-Tag: 1733590516-376966 X-HE-Meta: U2FsdGVkX1/D3Yxqj/N90Rxl7sAPQ3z1g++4gxNwvvFghEcGkzmxoPjKUF/3rZtbTkWcNHU5VwStWu3WOlnXkP+kzPhAMVTpKY44DN4xDgWGY1MCv+plC2WqCzTgqHBFiuHiIhQ1OLbvEGSDsEg8/Y+wPazJD4NbH8ACpSLppM0TY232P0CQ1h2mcEohwkMqyXRB8I4aduRVKVwqfv2MmzQUFGjNteieh7KJIfqcPKIj8XT6iMc2kgk27pdml52tE2Sx+aF8hwOTtaIeVOeIt3SAA2rjY6CVfVGfskUjHl7FvhLYII/dL3xYGmKU3DqnLrIEneKr5Z77GfDFksxbFlCPm4NyzGZFf+DeTXYYxzolhEFb9hVDlJOvqnOXbQiPUU8xoBMHlh/4F2y5/v9Bemf+EHWDTXcU4BOoKqZdjo31+VKLCt6Zd3tYSUPSyEjSbaAxQY+2NYXtfJjMqeJ930xjV/nD3tyCg3gKZoMYkSErdW6CpBBYVfXY/zPhdsKoVCibZLAgj+dfZdDk3naNY/QSA78P8EgmqYXiHQKQj3vFeHshii7ezQe9UNIGgd+ko9uig7fLCyv83Je6WWxsECWKa4AQcUC3OdRTG6eP1WAGzJDsOJcEsJyaSyrTfOuOi27DonSIKnNXyoGjXjt+flx5vW2hxWQ3g1nKEc4uC4f7Q6/ITFlaYrXlsVEvjXfs7Yoq8sNlEf2ZYIBNGB08SlVIeLnM6rBKWh1EvVJa6uYMB1QVav/895sLZmzc6Y0aowhVqtQLC4jiEDsI7ZLzRiAIKzQ0qkmRntY1/MX+IxxB8TWjc5mv7TJqLMx0srB3tWV1dqVUeqfn5knox8ONCocmLMdQwDOSa8nzOiZZpTeDK62tKa8KfzlGZFhMXeziJAwGcFNcdlZGxun1dWBTe/N31tk3Ayz/3bJQ9g9BeVzB5ErsF2VDunQ7FOYiWRRRUxVwsttffHD8XlHiYwQ 0kKMlbCB O0ksNDNG8FLB7ZDZcDUxxFSjA2MO/E8maLT/2tO7XFp/mGsnFQcqoDIlGrtCxj+WmVmyjGS7FPvPcrx7BGbgywIAaNjeg7cD3W+rJz5rk7xitwC9SACSKSvBKZi6aF8fXSHcprheNZAuzqOTT3Q6/KMVbWtWH1912cQLZ2eNdpp/HIFQ3pMRYHKjeDOpE3w5TG1wDboaZCA+szRV0sqRpV+1VairwkCT2eDHr/jfkqzTKpuSW2Um8KDoHuKKbRYPEHohomBWHebME5MG3yCHtmF/LIn0r1aSZpNQeDVOvn7+GwV1paiDmgJECCOUra9SYKRALTI9wLaJQuB21pyNjKznWDsiHHKBGd3fx34LHBh/S5Uy4MW4lPfAEFWL0Wog1MzjZhaeVp3heXdm1SUZ0GbYnXPbThJ/pY+VCx72PGGgRp0t5/VSi5JhR2LRzfx1Nad70QL17D/M8sceCqBC5pkIWemuGvXq0grmS/ByvVe+t1kA5ah/TyJD3b/7RvFwyUtnup0GQceAxBZ8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For architectures setting cpu_dcache_is_aliasing() to true, which require flushing dcache, and arc setting cpu_icache_is_aliasing() to true changes folio->flags to make icache coherent to dcache after clearing a user folio, __GFP_ZERO using only clear_page() is not enough to zero user folios and clear_user_(high)page() must be used. Otherwise, user data will be corrupted. Fix it by always clearing user folios with clear_user_(high)page() when cpu_dcache_is_aliasing() is true or cpu_icache_is_aliasing() is true. Rename alloc_zeroed() to alloc_need_zeroing() and invert the logic to clarify its intend. Fixes: 5708d96da20b ("mm: avoid zeroing user movable page twice with init_on_alloc=1") Reported-by: Geert Uytterhoeven Closes: https://lore.kernel.org/linux-mm/CAMuHMdV1hRp_NtR5YnJo=HsfgKQeH91J537Gh4gKk3PFZhSkbA@mail.gmail.com/ Tested-by: Geert Uytterhoeven Signed-off-by: Zi Yan --- include/linux/highmem.h | 8 +++++++- include/linux/mm.h | 18 ++++++++++++++++++ mm/huge_memory.c | 9 +++++---- mm/internal.h | 6 ------ mm/memory.c | 10 +++++----- 5 files changed, 35 insertions(+), 16 deletions(-) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index 6e452bd8e7e3..d9beb8371daa 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -224,7 +224,13 @@ static inline struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, unsigned long vaddr) { - return vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr); + struct folio *folio; + + folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vaddr); + if (folio && alloc_need_zeroing()) + clear_user_highpage(&folio->page, vaddr); + + return folio; } #endif diff --git a/include/linux/mm.h b/include/linux/mm.h index c39c4945946c..72d644cc8d9d 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -31,6 +31,7 @@ #include #include #include +#include struct mempolicy; struct anon_vma; @@ -4175,6 +4176,23 @@ static inline int do_mseal(unsigned long start, size_t len_in, unsigned long fla } #endif +/* + * alloc_need_zeroing checks if a user folio from page allocator needs to be + * zeroed or not. + */ +static inline bool alloc_need_zeroing(void) +{ + /* + * for user folios, arch with cache aliasing requires cache flush and + * arc changes folio->flags to make icache coherent with dcache, so + * always return false to make caller use + * clear_user_page()/clear_user_highpage() + */ + return (cpu_dcache_is_aliasing() || cpu_icache_is_aliasing()) || + !static_branch_maybe(CONFIG_INIT_ON_ALLOC_DEFAULT_ON, + &init_on_alloc); +} + int arch_get_shadow_stack_status(struct task_struct *t, unsigned long __user *status); int arch_set_shadow_stack_status(struct task_struct *t, unsigned long status); int arch_lock_shadow_stack_status(struct task_struct *t, unsigned long status); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index ee335d96fc39..107130a5413a 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1176,11 +1176,12 @@ static struct folio *vma_alloc_anon_folio_pmd(struct vm_area_struct *vma, folio_throttle_swaprate(folio, gfp); /* - * When a folio is not zeroed during allocation (__GFP_ZERO not used), - * folio_zero_user() is used to make sure that the page corresponding - * to the faulting address will be hot in the cache after zeroing. + * When a folio is not zeroed during allocation (__GFP_ZERO not used) + * or user folios require special handling, folio_zero_user() is used to + * make sure that the page corresponding to the faulting address will be + * hot in the cache after zeroing. */ - if (!alloc_zeroed()) + if (alloc_need_zeroing()) folio_zero_user(folio, addr); /* * The memory barrier inside __folio_mark_uptodate makes sure that diff --git a/mm/internal.h b/mm/internal.h index cb8d8e8e3ffa..3bd08bafad04 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1285,12 +1285,6 @@ void touch_pud(struct vm_area_struct *vma, unsigned long addr, void touch_pmd(struct vm_area_struct *vma, unsigned long addr, pmd_t *pmd, bool write); -static inline bool alloc_zeroed(void) -{ - return static_branch_maybe(CONFIG_INIT_ON_ALLOC_DEFAULT_ON, - &init_on_alloc); -} - /* * Parses a string with mem suffixes into its order. Useful to parse kernel * parameters. diff --git a/mm/memory.c b/mm/memory.c index 75c2dfd04f72..cf1611791856 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4733,12 +4733,12 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf) folio_throttle_swaprate(folio, gfp); /* * When a folio is not zeroed during allocation - * (__GFP_ZERO not used), folio_zero_user() is used - * to make sure that the page corresponding to the - * faulting address will be hot in the cache after - * zeroing. + * (__GFP_ZERO not used) or user folios require special + * handling, folio_zero_user() is used to make sure + * that the page corresponding to the faulting address + * will be hot in the cache after zeroing. */ - if (!alloc_zeroed()) + if (alloc_need_zeroing()) folio_zero_user(folio, vmf->address); return folio; }