From patchwork Fri Jun 9 16:29:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tarun Sahu X-Patchwork-Id: 13274171 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79408C7EE25 for ; Fri, 9 Jun 2023 16:29:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EA94B8E0002; Fri, 9 Jun 2023 12:29:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E59356B0078; Fri, 9 Jun 2023 12:29:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D208A8E0002; Fri, 9 Jun 2023 12:29:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C30166B0075 for ; Fri, 9 Jun 2023 12:29:31 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 69A1CC02FF for ; Fri, 9 Jun 2023 16:29:31 +0000 (UTC) X-FDA: 80883744942.22.1FF0278 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by imf09.hostedemail.com (Postfix) with ESMTP id 3D83214001F for ; Fri, 9 Jun 2023 16:29:28 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=e2i546JP; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf09.hostedemail.com: domain of tsahu@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=tsahu@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686328168; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=L/xGwhX2AoTt7EqDkVYEXtV8BmGP3M8VcDdHfDlDTOo=; b=bPW2gWFq/k9xbYVrlcrpPtiZu4L3/HMvFst/zRLhJG/BLInTIx1+0G8718jHYHEJ2ut/hw 2MRwSNP6jdIuLuzpJtrm0l5wb4MDghZyP4f6L1ytQtNFSmZ6SMQ9Ow0f0SBSBRfO3geOUn JMS5R7Aq92bfXqymUtBnRypPe2xuxoI= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=e2i546JP; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf09.hostedemail.com: domain of tsahu@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=tsahu@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686328168; a=rsa-sha256; cv=none; b=kKbTA0tp/yqbMFoLyPQ0gL0smgxv8Ai8VVFVbfSI3EjVBrknszgr95jYsWZShxKjGnYMU9 /Z4+LleNR4FoYBtkJPRDCTItp4tkLKGjT6KoHkVFhvhom9sLadnz15OOlgtIKUkNo2t+El E7lmJOC8/ELaCh/iF7eW1xFrxaKa9x4= Received: from pps.filterd (m0353722.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 359GNitX031946; Fri, 9 Jun 2023 16:29:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : content-transfer-encoding : mime-version; s=pp1; bh=L/xGwhX2AoTt7EqDkVYEXtV8BmGP3M8VcDdHfDlDTOo=; b=e2i546JPXmvZj+fZ6iNoyn2N8m0y5grJ95Jf0Xs/MOS9CoVsAp7DGDjGHQRlNz1f0RPj BCtj95bzAfcJWxJDLLaK3Ki5vLcr5OX4MYajfdL6hNyHsleamYLooN7eIMQ583a+jzkX NDLu7qjanToUL5t5hEhEKpBrQoAubRydd3YwdeR6IS92RTgwnbrTSoIOVace8UJvd99A GjwAkPTr1FZFzrJ8z/aS8OZ5gHnKgBcC9hp2kQ7HCMJeCReP5euzrQflfo7jf7xp43QG vLLDxm6NeaqljK5rZ+WSwVm6qf94E0yWLGLV3iIjkOELpE68rdYSaDHMreFo/qBk2i5C jg== Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r47jc032h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 09 Jun 2023 16:29:20 +0000 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3599SOsk021810; Fri, 9 Jun 2023 16:29:18 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma03ams.nl.ibm.com (PPS) with ESMTPS id 3r2a76a22f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 09 Jun 2023 16:29:17 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 359GTEpF21234262 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 9 Jun 2023 16:29:14 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 828C220043; Fri, 9 Jun 2023 16:29:14 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4843020040; Fri, 9 Jun 2023 16:29:12 +0000 (GMT) Received: from tarunpc.ibmuc.com (unknown [9.43.86.205]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 9 Jun 2023 16:29:12 +0000 (GMT) From: Tarun Sahu To: linux-mm@kvack.org Cc: akpm@linux-foundation.org, muchun.song@linux.dev, mike.kravetz@oracle.com, aneesh.kumar@linux.ibm.com, willy@infradead.org, sidhartha.kumar@oracle.com, gerald.schaefer@linux.ibm.com, linux-kernel@vger.kernel.org, jaypatel@linux.ibm.com, tsahu@linux.ibm.com Subject: [PATCH v3] mm/folio: Avoid special handling for order value 0 in folio_set_order Date: Fri, 9 Jun 2023 21:59:07 +0530 Message-Id: <20230609162907.111756-1-tsahu@linux.ibm.com> X-Mailer: git-send-email 2.31.1 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: ILg9SmIf6gmUc4pFU8EEijKVkf_BPtjl X-Proofpoint-GUID: ILg9SmIf6gmUc4pFU8EEijKVkf_BPtjl X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-09_11,2023-06-09_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 malwarescore=0 suspectscore=0 phishscore=0 mlxlogscore=999 priorityscore=1501 impostorscore=0 spamscore=0 bulkscore=0 adultscore=0 clxscore=1015 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306090135 X-Rspamd-Queue-Id: 3D83214001F X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: niy8noutkfqtxxmowydyo4m7hi45resg X-HE-Tag: 1686328168-492143 X-HE-Meta: U2FsdGVkX1/tDpZwSOtbnJdaqy4LOaSe5ixhytFSDG2HnbSNpCyUGvQsG7p0zq6PnPhk9E603gx0G6PflYpsbcIfHCPp07VsmhKYneiRggzXgECDbOkbfUGu1TRhZM2ddvRhbL5HteDqERubiyIwPYZ6Pv7Ypg0BCsbiVmHHgneaOq8bC+s6Z50tV+gMvuCC71B9OvCdUvQKMjoJpIgOCnPtsMraOCM0f8bvgG422q5yQrqLQx10nP7chdmpoMi9jPtrm1bV9mqGrUZ7rNh8hj5QE68GUsoRPN7r4J5Ke8KRlOYM+tC3AOYJ1ejUot7pZx6WnldsMMnG7p8BEQnQe8eRb1yJjgnc2dCiksql519GlAiMhYFxjroTvbBQpWzalKXENZ0x/7XzbajCCi4cVtHEpjH8O5b8nnYOvXpCRHIbc/61O2x06yWlcr1mf2K74YFfQdkFHw96wVfKohm6mGOcaCDYV6RZz+PXpurTcSd6wu4b/NJbf+ZYwIQkAkgVNE4IMh2I5RvFXsihnfWCnLBRTQ5vRS0RwjcKLn+bynRGLfRxT1FLhm8VnRmpH+0ZTMMPKA58d+zLjAwyl+k9dqeOEwcARXNqjFBFlOIyzE+IUGaQwzXtfQoUF1GeuR/p+Xldao/u7cOe4wpMTG6XgoXG224YyJaLmU699UNDvpCzcv5XWw0Dzs4KFx4RryK6V2nG3RSkx3ZZgurB5grCxxdqtY2h8zkRoNa8O0AwMQyWdX4f0baANATW/9ckSMzk0LsQY1ZegQvUVpskCv87xbz1RR2+bSQj6Gpnb1R0tSqkHiUcEAVRnEN8MVua/sH/KD2QNc3ypqoEX2imdRI/YBro0b/ZTJHIXilQj4eRTK4xKeTag9g94CdJqU9ThseYVExQ1b68u0ElK2upEENrfIdZdwD6RxVWoNea+lXrQT3H9ExDhqXIPGACu4yfOVXutRy6WYM+PLZau23qGy2 fRCitLq4 7ljJ4BIP+X+ZB+gmbplh/8q0JmBJr0GwvzD9Q8JEkJ+O4eoppRHnLY2c6mlrHimN9o4drdOnyaVussemfecusz2bnt0vbsbU3YYxTfzExl0s35WlM4FfhVDwpV7wBTAf7zxO0BL21JyVCrLzY6R88u5ierB8IGEQGX/TimTqP4YJ87hcJggAVmIIyL8IQRdhcQgeZE3Kc9Bs6VIPb1RLiXdkt4A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: folio_set_order(folio, 0) is used in kernel at two places __destroy_compound_gigantic_folio and __prep_compound_gigantic_folio. Currently, It is called to clear out the folio->_folio_nr_pages and folio->_folio_order. For __destroy_compound_gigantic_folio: In past, folio_set_order(folio, 0) was needed because page->mapping used to overlap with _folio_nr_pages and _folio_order. So if these fields were left uncleared during freeing gigantic hugepages, they were causing "BUG: bad page state" due to non-zero page->mapping. Now, After Commit a01f43901cfb ("hugetlb: be sure to free demoted CMA pages to CMA") page->mapping has explicitly been cleared out for tail pages. Also, _folio_order and _folio_nr_pages no longer overlaps with page->mapping. So, folio_set_order(folio, 0) can be removed from freeing gigantic folio path (__destroy_compound_gigantic_folio). Another place, folio_set_order(folio, 0) is called inside __prep_compound_gigantic_folio during error path. Here, folio_set_order(folio, 0) can also be removed if we move folio_set_order(folio, order) after for loop. The patch also moves _folio_set_head call in __prep_compound_gigantic_folio() such that we avoid clearing them in the error path. Also, as Mike pointed out: "It would actually be better to move the calls _folio_set_head and folio_set_order in __prep_compound_gigantic_folio() as suggested here. Why? In the current code, the ref count on the 'head page' is still 1 (or more) while those calls are made. So, someone could take a speculative ref on the page BEFORE the tail pages are set up." This way, folio_set_order(folio, 0) is no more needed. And it will also helps removing the confusion of folio order being set to 0 (as _folio_order field is part of first tail page). Testing: I have run LTP tests, which all passes. and also I have written the test in LTP which tests the bug caused by compound_nr and page->mapping overlapping. https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/hugetlb/hugemmap/hugemmap32.c Running on older kernel ( < 5.10-rc7) with the above bug this fails while on newer kernel and, also with this patch it passes. Signed-off-by: Tarun Sahu Reviewed-by: Mike Kravetz --- v2->v3 - removed the copy of page/folio definition from commit msg v1->v2 - Reword the commit message mm/hugetlb.c | 9 +++------ mm/internal.h | 8 ++------ 2 files changed, 5 insertions(+), 12 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index f154019e6b84..607553445855 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1489,7 +1489,6 @@ static void __destroy_compound_gigantic_folio(struct folio *folio, set_page_refcounted(p); } - folio_set_order(folio, 0); __folio_clear_head(folio); } @@ -1951,9 +1950,6 @@ static bool __prep_compound_gigantic_folio(struct folio *folio, struct page *p; __folio_clear_reserved(folio); - __folio_set_head(folio); - /* we rely on prep_new_hugetlb_folio to set the destructor */ - folio_set_order(folio, order); for (i = 0; i < nr_pages; i++) { p = folio_page(folio, i); @@ -1999,6 +1995,9 @@ static bool __prep_compound_gigantic_folio(struct folio *folio, if (i != 0) set_compound_head(p, &folio->page); } + __folio_set_head(folio); + /* we rely on prep_new_hugetlb_folio to set the destructor */ + folio_set_order(folio, order); atomic_set(&folio->_entire_mapcount, -1); atomic_set(&folio->_nr_pages_mapped, 0); atomic_set(&folio->_pincount, 0); @@ -2017,8 +2016,6 @@ static bool __prep_compound_gigantic_folio(struct folio *folio, p = folio_page(folio, j); __ClearPageReserved(p); } - folio_set_order(folio, 0); - __folio_clear_head(folio); return false; } diff --git a/mm/internal.h b/mm/internal.h index 68410c6d97ac..c59fe08c5b39 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -425,16 +425,12 @@ int split_free_page(struct page *free_page, */ static inline void folio_set_order(struct folio *folio, unsigned int order) { - if (WARN_ON_ONCE(!folio_test_large(folio))) + if (WARN_ON_ONCE(!order || !folio_test_large(folio))) return; folio->_folio_order = order; #ifdef CONFIG_64BIT - /* - * When hugetlb dissolves a folio, we need to clear the tail - * page, rather than setting nr_pages to 1. - */ - folio->_folio_nr_pages = order ? 1U << order : 0; + folio->_folio_nr_pages = 1U << order; #endif }