From patchwork Fri Nov 18 09:16:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 13047891 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E33FC433FE for ; Fri, 18 Nov 2022 09:16:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB2326B0071; Fri, 18 Nov 2022 04:16:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A61AF6B0072; Fri, 18 Nov 2022 04:16:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 929AC8E0001; Fri, 18 Nov 2022 04:16:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 817C36B0071 for ; Fri, 18 Nov 2022 04:16:24 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 5149BA01C3 for ; Fri, 18 Nov 2022 09:16:24 +0000 (UTC) X-FDA: 80146007088.20.DD9B5F2 Received: from mail-qk1-f180.google.com (mail-qk1-f180.google.com [209.85.222.180]) by imf28.hostedemail.com (Postfix) with ESMTP id F19BEC000A for ; Fri, 18 Nov 2022 09:16:23 +0000 (UTC) Received: by mail-qk1-f180.google.com with SMTP id d7so3013167qkk.3 for ; Fri, 18 Nov 2022 01:16:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=3l1Y5HSD6/V1oMjNmRLOOT3WB8p/T/AI0RfUdUHqvkM=; b=jqFHAvcXPb8pPqSgsQoBpVV2smoWYv0AsTkrJ+kQJiaRB6AMS+3EqMCC/cEs362x4u /+p301ynz83gOJciwL/+JhWXLwlzDXQtAVxI0pJDDB9GKFE/gP/ZecyBXVb0vVwGcrua B2vrLedGOjYteL855XQk1CA0/49+fnprzI25uu1e5Bx/oqPl9AG+803KHDKtnT/PLG+k Un9d7IN9ioEtrMxSurRiCVuroEOut7A/yFcb7dN8ho/9Rkdo002NkBdMT4cXDl7VmAKN uEfp7YDyLrXETMesR0xnhm5rSncI0B+2YFfyWAA1DTtmX6CIW2s8s/afyp7n0DlEMUch nA+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3l1Y5HSD6/V1oMjNmRLOOT3WB8p/T/AI0RfUdUHqvkM=; b=ZPcUTN9gM4swuPiPXz8q4Otzsc5dvXOcQUvnsm01HNDAcqyaalMOgIH4t/M+t8foyd RwG/FPKbj3qamrVhw8PsOZONb2XaPIIz/ysOhEPNCEMdA0HNkaykHkuKP4kjifxwcRCg 963ts6RDX+6HcfuEMylRU3evarBSmoKXy5vpEytVMlIVW4Gt8SitYetgX7O4BYWerc1V iIFdaponGtbs/A7Q+fzDIEztejwvXsjUqqrkU3fooMiUo0lDwXxo9IZRLa5Oyw/0OaAV 5xXYlhuRl9npKbdMw+/PlGlgL6HbkxpA6rxCoUge1biyXN7ZSu4yZ4yu1cdSUlC8Hde9 TKSQ== X-Gm-Message-State: ANoB5plMaut8xpCOGcWqPqHgRJ8ca4XKBbLpX123q8bRUBXHK4/g/+Xn GfJtQYdOUlSmPe8aBNXUCDWHsA== X-Google-Smtp-Source: AA0mqf64Gro9oee+mIQaltmjl0aWGdxSPUg7f+VNTF3pGS8eJ15VWOe1XhqGaEeBSf5jYOQDXt97hQ== X-Received: by 2002:a37:6554:0:b0:6f9:f236:1b2b with SMTP id z81-20020a376554000000b006f9f2361b2bmr4845776qkb.299.1668762983133; Fri, 18 Nov 2022 01:16:23 -0800 (PST) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id az20-20020a05620a171400b006ec771d8f89sm2086377qkb.112.2022.11.18.01.16.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 01:16:22 -0800 (PST) Date: Fri, 18 Nov 2022 01:16:20 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Andrew Morton cc: Linus Torvalds , Johannes Weiner , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Vlastimil Babka , Peter Xu , Yang Shi , John Hubbard , Mike Kravetz , Sidhartha Kumar , Muchun Song , Miaohe Lin , Naoya Horiguchi , Mina Almasry , James Houghton , Zach O'Keefe , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 3/3] mm,thp,rmap: clean up the end of __split_huge_pmd_locked() In-Reply-To: Message-ID: <2f4afe60-40d2-706c-af21-914fbbbd164@google.com> References: <5f52de70-975-e94f-f141-543765736181@google.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1668762984; a=rsa-sha256; cv=none; b=cDTOc5w/KUIT6mjkqcJJs9sEFyplhK7tpoiQAnBcDA3ImI5uYdrblIFYv2mCmihJ3Qhspz mqZpiyFGcwZITpsVPOXqX0uJVdWf627FHOg99POXfkzpPs9Pq3UKbTeo0dH+BE8vnOsqE2 /nRQ8YS4C+yDr9CKGax2WlMJuuQKHrU= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=jqFHAvcX; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of hughd@google.com designates 209.85.222.180 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1668762984; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3l1Y5HSD6/V1oMjNmRLOOT3WB8p/T/AI0RfUdUHqvkM=; b=Qlt+RzbF/feuZeDdT7sZsCiUVBwTgUca5j3Pm+cQitU2NHOjyyXcM8LMhE1P0gu5FEKpP8 ARCG8XVvBdP5Hbp4cKVL+vF/IbCAMPg0ofX8vfZALvDXcNBlqiRn2nIGsjMIcXXmZyKRkN E7/Jb6KxFVlztKGM/P0/7unrBe+uG18= Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=jqFHAvcX; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of hughd@google.com designates 209.85.222.180 as permitted sender) smtp.mailfrom=hughd@google.com X-Rspam-User: X-Stat-Signature: d4s13dg6fkbcsw3zjbfbhc6bwrp4sq1a X-Rspamd-Queue-Id: F19BEC000A X-Rspamd-Server: rspam11 X-HE-Tag: 1668762983-438206 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It's hard to add a page_add_anon_rmap() into __split_huge_pmd_locked()'s HPAGE_PMD_NR set_pte_at() loop, without wincing at the "freeze" case's HPAGE_PMD_NR page_remove_rmap() loop below it. It's just a mistake to add rmaps in the "freeze" (insert migration entries prior to splitting huge page) case: the pmd_migration case already avoids doing that, so just follow its lead. page_add_ref() versus put_page() likewise. But why is one more put_page() needed in the "freeze" case? Because it's removing the pmd rmap, already removed when pmd_migration (and freeze and pmd_migration are mutually exclusive cases). Signed-off-by: Hugh Dickins Acked-by: Kirill A. Shutemov --- mm/huge_memory.c | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 3dee8665c585..ab5ab1a013e1 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2135,7 +2135,6 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, uffd_wp = pmd_uffd_wp(old_pmd); VM_BUG_ON_PAGE(!page_count(page), page); - page_ref_add(page, HPAGE_PMD_NR - 1); /* * Without "freeze", we'll simply split the PMD, propagating the @@ -2155,6 +2154,8 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, anon_exclusive = PageAnon(page) && PageAnonExclusive(page); if (freeze && anon_exclusive && page_try_share_anon_rmap(page)) freeze = false; + if (!freeze) + page_ref_add(page, HPAGE_PMD_NR - 1); } /* @@ -2210,27 +2211,21 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, entry = pte_mksoft_dirty(entry); if (uffd_wp) entry = pte_mkuffd_wp(entry); + page_add_anon_rmap(page + i, vma, addr, false); } pte = pte_offset_map(&_pmd, addr); BUG_ON(!pte_none(*pte)); set_pte_at(mm, addr, pte, entry); - if (!pmd_migration) - page_add_anon_rmap(page + i, vma, addr, false); pte_unmap(pte); } if (!pmd_migration) page_remove_rmap(page, vma, true); + if (freeze) + put_page(page); smp_wmb(); /* make pte visible before pmd */ pmd_populate(mm, pmd, pgtable); - - if (freeze) { - for (i = 0; i < HPAGE_PMD_NR; i++) { - page_remove_rmap(page + i, vma, false); - put_page(page + i); - } - } } void __split_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd,