From patchwork Wed Sep 13 12:51:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13382961 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FA1AEDEC4E for ; Wed, 13 Sep 2023 12:51:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DEA5A6B0181; Wed, 13 Sep 2023 08:51:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D735D6B0182; Wed, 13 Sep 2023 08:51:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C141E6B0184; Wed, 13 Sep 2023 08:51:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A3FB16B0181 for ; Wed, 13 Sep 2023 08:51:25 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7EC9640642 for ; Wed, 13 Sep 2023 12:51:25 +0000 (UTC) X-FDA: 81231560130.13.17A27EC Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf06.hostedemail.com (Postfix) with ESMTP id D49F918000A for ; Wed, 13 Sep 2023 12:51:23 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OI44+GwW; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694609483; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fcwyuEyPhUjibl+UPXTZ+YZfX1YncqC+LcRBkk4rhVQ=; b=287/Kev/NLNBv/KDzlzO+gftXZXjyHqwNb6dnPKiAyEyLHwvfqc5iPt7eceLJHmC506YmG WdF1yV1dhaeY7IgyezisH04shLVnzoGtQJt0uv1Kal3lo7lh8tNbJJu4urzhT1n+9Ophd/ 8EgwtpDzD4PjdDC6wenajYRtRNJoeVI= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OI44+GwW; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694609483; a=rsa-sha256; cv=none; b=3FXjyt6LLapaByKY0f8x0LRbfN5z73nQcUtocIHKaT8gZF9aPHbJ3yh7eH5Jkixe07L9Mi aciavhcPSIDC+IaEn9VQtX9PD57NEVaXijEiZ/2GSojl87CJdrxonjKro8DESawCk0oHcJ 7Bi6shXLvcXo5JotUY4ywoXVGZZvLgw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694609483; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fcwyuEyPhUjibl+UPXTZ+YZfX1YncqC+LcRBkk4rhVQ=; b=OI44+GwW42BRP21Kz5zKlTFCEY65VufyoAUEm5ENXOY9znlgEIbkeG9BcUyjnJ57pitClC 8s7PI7sZt+RgN8ru/TS7SoFIipEikVnv5T/3dgLAVp3RJsFi8bj9smfT5XfCbcLzg0iOAE 7CgoPpw6gf1fq8qZnRLf1BJjySGYvek= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-664-skkLD7dfNDCSC0zIh01gyA-1; Wed, 13 Sep 2023 08:51:19 -0400 X-MC-Unique: skkLD7dfNDCSC0zIh01gyA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3CA79857A9A; Wed, 13 Sep 2023 12:51:19 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.193.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8FF2D2156721; Wed, 13 Sep 2023 12:51:17 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Mike Kravetz , Muchun Song Subject: [PATCH v1 1/6] mm/rmap: drop stale comment in page_add_anon_rmap and hugepage_add_anon_rmap() Date: Wed, 13 Sep 2023 14:51:08 +0200 Message-ID: <20230913125113.313322-2-david@redhat.com> In-Reply-To: <20230913125113.313322-1-david@redhat.com> References: <20230913125113.313322-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Queue-Id: D49F918000A X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: bdt1asocc973nkkro95dygpjx69gd89r X-HE-Tag: 1694609483-248354 X-HE-Meta: U2FsdGVkX18tgq/x4+LnOnhy7v4j8PMQ8i3NL4tIPWDuPN+zaZAs8VL0n+w54TSIrUbGpBw85CiDGW6j3EQx7xPuN7k6gmPFO3d0yVf1f3jqNbxWnCSNKN+DTeQXysVq2WyHp77vynFLgbKdvQAIMIlOyJo88NTVNV1PNLH+QUHE6flXCNQU8Dgx4FtcpQMl51TBfoxLobiqEO8ADPvZLzncGn48hlivtDYqnMGzeLn/1Hji1ee4Ye6riv46dpp7f4A9MV9A6VrnbTZh9vjbVWu3PV30q12l298uIwdiFfBRdWlcl6bItBTqVFmARWdG0JQYFZMJ7wjyv57UVktqyQDKQXV7xyaGqQ518XSmtp8T0aU9h5CQNUz78Eg0yd0uCtrKxm0UjDfD8uGqfT6ZQErNirD/OuVCEEvq4Kwe+xLNTPDV/7QeUEze6RNsaz8YqC1GRa2/NLpal68b6eeBtL/cgG/S7MmSBkhPlvwoCYxOaUr2T7s1FVjmln6HxCC7oRnFRs49GPN+ThchoB9Koq2TxMnl4trouV8Wi2xO9iycALN3Jdem9hjW2zTds/VrSO8GfoToU64yuhMpUKmx1FoyMzpSo8FOgRUQO1D80/1BGneUQY/2NcJrWl7/ULz4//DJfDcC7xqikrbSd9b27sPTUgW6HWnAnGsZFlyXYdR9jKLwWUAZRamjvb2GlclOcdaHDcCkDBWGQUtid7rSoiBhkua9zAhVUpYG1AaKWOPXQIvuxwrFxv7UblMEqQ6zN0skeZvqLyVKXcdcmG2aXOI29uEVeLCpjpNC3B7ZLrR4JYAdIaCqH8UjtV7U+wvpYt/wqhoJG+aNskyGHn5nUjKUcPudapzeS1yRhF78HDEG5S7/pGPqCj1Irs1sY5bK2x8vM0JjitTJ+ap4IME3c9XpHtn6MWmxs5AOAogQXRqrJCfSLMW7bFqMEWP1XHWPOXBW2kwQ98DowgboeAw WeP6hgyq XZMynTWT+XjqOn+oMR3cYwuP/vPdKYWxvywBvoHBKW+UV7lHqwgqPzBlr+NStu23zFIgm8aSlTWvZ74cEYXW5hUjKtv+o+5nSePepjlkw5aPM+QeGFkZLiBg8NZg+Dv9Aeq3cd+RI5/BtUhJlFi+Kz3MAYYw7hcku3paEdghUVoUZSnw5ezDbt4qmdj9V4CzvZP9sHLD1tXwgmXDVDz7wwP+R8Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: That comment was added in commit 5dbe0af47f8a ("mm: fix kernel BUG at mm/rmap.c:1017!") to document why we can see vma->vm_end getting adjusted concurrently due to a VMA split. However, the optimized locking code was changed again in bf181b9f9d8 ("mm anon rmap: replace same_anon_vma linked list with an interval tree."). ... and later, the comment was changed in commit 0503ea8f5ba7 ("mm/mmap: remove __vma_adjust()") to talk about "vma_merge" although the original issue was with VMA splitting. Let's just remove that comment. Nowadays, it's outdated, imprecise and confusing. Signed-off-by: David Hildenbrand --- mm/rmap.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index ec7f8e6c9e48..ca2787c1fe05 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1245,7 +1245,6 @@ void page_add_anon_rmap(struct page *page, struct vm_area_struct *vma, __lruvec_stat_mod_folio(folio, NR_ANON_MAPPED, nr); if (likely(!folio_test_ksm(folio))) { - /* address might be in next vma when migration races vma_merge */ if (first) __page_set_anon_rmap(folio, page, vma, address, !!(flags & RMAP_EXCLUSIVE)); @@ -2536,7 +2535,6 @@ void hugepage_add_anon_rmap(struct page *page, struct vm_area_struct *vma, BUG_ON(!folio_test_locked(folio)); BUG_ON(!anon_vma); - /* address might be in next vma when migration races vma_merge */ first = atomic_inc_and_test(&folio->_entire_mapcount); VM_BUG_ON_PAGE(!first && (flags & RMAP_EXCLUSIVE), page); VM_BUG_ON_PAGE(!first && PageAnonExclusive(page), page); From patchwork Wed Sep 13 12:51:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13382962 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B3BFEDEC49 for ; Wed, 13 Sep 2023 12:51:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DC4CC6B0185; Wed, 13 Sep 2023 08:51:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D4F126B0186; Wed, 13 Sep 2023 08:51:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC63A6B0187; Wed, 13 Sep 2023 08:51:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A8EDF6B0185 for ; Wed, 13 Sep 2023 08:51:29 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 7B4FEA039E for ; Wed, 13 Sep 2023 12:51:29 +0000 (UTC) X-FDA: 81231560298.05.49C6CA2 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf23.hostedemail.com (Postfix) with ESMTP id B95D1140002 for ; Wed, 13 Sep 2023 12:51:27 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=DHIrp70E; spf=pass (imf23.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694609487; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Fi954WhfJTlnxfu/yTjvI+IMpqyRN+5DQHmUbjacTd0=; b=GzAom3y6rJLJvtUvTf4asQIrXRonWlXtkn+ujk9sJx8bKdJ6qPDq4WyvzxLkblYLDCZZgp MVrRyJ19wYVMCJBnOExI4gbLpJfO3RKPvn8IbK0TRtPh1b+v9MzH8jdd3fe4Xlhq4jDoal MNxKMjm9hAg6lzWI6hf8BKpc+lC4cco= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694609487; a=rsa-sha256; cv=none; b=zTaLnRJgxyX6CgQrVcHQ2AD+lFASG6gvPdr/6neVGdMttTpQhKYKak64wGsGbayrnB7H3S L5SayyYBD1ro6POHwXGPNPvjQlf207iuJsoT3h+aHJWBWtDKVjIW9mOJEzxaNVkHOcqiIs y2aQT4jy9LcgQycmyisLpZxDQjLAa6g= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=DHIrp70E; spf=pass (imf23.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694609487; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Fi954WhfJTlnxfu/yTjvI+IMpqyRN+5DQHmUbjacTd0=; b=DHIrp70EoDQ6DpyaD2mSvnjle614u/b8rDJyXulYt/coW7BEf6xiQcfBC744IpSPtlK1RL /pCbXZzYhXLZln7/c4BmdeSyUJx8Y8H/aPrF7Cq2+9hjFMc8wXlHUD+4OPPhHOZZQGuWBt X2Fyg/evVIltzpqfvxr7C0blENC0nko= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-60-IQGL421PNvK9ZlO5bNgMvQ-1; Wed, 13 Sep 2023 08:51:21 -0400 X-MC-Unique: IQGL421PNvK9ZlO5bNgMvQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4CBBA101A550; Wed, 13 Sep 2023 12:51:21 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.193.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id A48BE2156722; Wed, 13 Sep 2023 12:51:19 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Mike Kravetz , Muchun Song Subject: [PATCH v1 2/6] mm/rmap: move SetPageAnonExclusive out of __page_set_anon_rmap() Date: Wed, 13 Sep 2023 14:51:09 +0200 Message-ID: <20230913125113.313322-3-david@redhat.com> In-Reply-To: <20230913125113.313322-1-david@redhat.com> References: <20230913125113.313322-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Queue-Id: B95D1140002 X-Rspam-User: X-Stat-Signature: ftimi37it98d1kxpud6gm4arcpfx83hs X-Rspamd-Server: rspam03 X-HE-Tag: 1694609487-222045 X-HE-Meta: U2FsdGVkX1+P3fOos9oXvj1ZuPJAXEPvhMueUSIR0nOhgG3/RkiqpNMMUI6F/wuSJYbnY8G3UIxDMmCdjI4M/PF+C+oKKZPJejIOjQUFKf3OTzyUKJXUzOv5H/QCEipIlXU63Gxh4ki5kEHlJRoOqxVBMGcFFOE09v2DgslQ/c6U3qKTtuZgDYo0a3Tgl0RZAG7OjPD6X1wa/BdAhdljG1wwAz+vYXmQL4xSGBhPd9uVW+SgU/dxw1jIYKwxFSDccQ4t4RsAoXzcX7lOLa3AvQI+D8oTx+eiIbxyefR8StMviLZ6Z1yjFKxvx7rV9SCl107J8rBjSRM5I1u/wkKu1VRrBcdHG+ApvUvDBH9r3y0gVGYWkVS0ly224/GGez0cOPpMeOd8JnzQEYU4Qb7JDL5uydFILGtjI+NJ+V0XhngXlLw1QtIPBQa3j7P15sOplJI0Ck31DB9aGHSQ4rfAAy4A8Lj+yjDKPnZSgPw02JehGvL/6N53xi514FV3gIX3Yb2TA8yWdze5ozF8oArGAwxktd8lsKKTJqoEQ8E0rNjNQYwDfY4q38OmCs+m18nPr/hE4cWjzPghrxhcNxLiKDKtJXoOJgX6N6N4rI1Y3P99OqMV759x9Lbef4OcdDdvyfpj9Xd8cORCaJBomlFw8HSYX98u9yReZ8cMBwlKFBvDcbTqBv3GUfNeMRt4B0eDfkV/IiNdb4abrhNYzPM4U4kooh35cIqL5zGcjzU8dhcAeToqKV95YYtmDoM6VeCUSTHFsBFsUYsdthHJq0dSmXY/99aug6Im0doT0gLjekZS+71ttTEE8raF0lrQrqlhqo6Ef5jOz2LXwfZVnokwAybL7XnETEscAAM3LRr/BVis7ByYAOe632dFylS7uPH1cgU3QAhG8kaRUMNkjY9fCBT4aPRIGDmE56UttH1WTAtg+rNTB/XXwK+cJePAnEEH058ClSpr7EUQrjD+WGI bJDJm761 m18Da5qWZNBeB5ALOkONMQv4cdTyQHysx5dcd/PjmpJscZmvC0lHNntpLCrCHrIhzRQ4n/6giCfX2XkKYKd703TXA/DJdG5EtbQvw1cMN/yS0cKsrCwZPkHobFxs8VybdKKsDPUmtCKene1wGxYrRt3cXN1Gg1WUv11hP7uQbFwfHB5kpyLKHEYpy5FxbtKyrUbvEjrCLyflrmuLFZPoXtyBy/w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let's handle it in the caller. No need to pass the page. While at it, rename the function to __folio_set_anon() and pass "bool exclusive" instead of "int exclusive". Signed-off-by: David Hildenbrand --- mm/rmap.c | 41 +++++++++++++++++++++-------------------- 1 file changed, 21 insertions(+), 20 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index ca2787c1fe05..ab16baa0fcb3 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1122,27 +1122,25 @@ void page_move_anon_rmap(struct page *page, struct vm_area_struct *vma) } /** - * __page_set_anon_rmap - set up new anonymous rmap - * @folio: Folio which contains page. - * @page: Page to add to rmap. - * @vma: VM area to add page to. + * __folio_set_anon - set up a new anonymous rmap for a folio + * @folio: The folio to set up the new anonymous rmap for. + * @vma: VM area to add the folio to. * @address: User virtual address of the mapping - * @exclusive: the page is exclusively owned by the current process + * @exclusive: Whether the folio is exclusive to the process. */ -static void __page_set_anon_rmap(struct folio *folio, struct page *page, - struct vm_area_struct *vma, unsigned long address, int exclusive) +static void __folio_set_anon(struct folio *folio, struct vm_area_struct *vma, + unsigned long address, bool exclusive) { struct anon_vma *anon_vma = vma->anon_vma; BUG_ON(!anon_vma); if (folio_test_anon(folio)) - goto out; + return; /* - * If the page isn't exclusively mapped into this vma, - * we must use the _oldest_ possible anon_vma for the - * page mapping! + * If the folio isn't exclusive to this vma, we must use the _oldest_ + * possible anon_vma for the folio mapping! */ if (!exclusive) anon_vma = anon_vma->root; @@ -1156,9 +1154,6 @@ static void __page_set_anon_rmap(struct folio *folio, struct page *page, anon_vma = (void *) anon_vma + PAGE_MAPPING_ANON; WRITE_ONCE(folio->mapping, (struct address_space *) anon_vma); folio->index = linear_page_index(vma, address); -out: - if (exclusive) - SetPageAnonExclusive(page); } /** @@ -1246,11 +1241,13 @@ void page_add_anon_rmap(struct page *page, struct vm_area_struct *vma, if (likely(!folio_test_ksm(folio))) { if (first) - __page_set_anon_rmap(folio, page, vma, address, - !!(flags & RMAP_EXCLUSIVE)); + __folio_set_anon(folio, vma, address, + !!(flags & RMAP_EXCLUSIVE)); else __page_check_anon_rmap(folio, page, vma, address); } + if (flags & RMAP_EXCLUSIVE) + SetPageAnonExclusive(page); mlock_vma_folio(folio, vma, compound); } @@ -1289,7 +1286,8 @@ void folio_add_new_anon_rmap(struct folio *folio, struct vm_area_struct *vma, } __lruvec_stat_mod_folio(folio, NR_ANON_MAPPED, nr); - __page_set_anon_rmap(folio, &folio->page, vma, address, 1); + __folio_set_anon(folio, vma, address, true); + SetPageAnonExclusive(&folio->page); } /** @@ -2539,8 +2537,10 @@ void hugepage_add_anon_rmap(struct page *page, struct vm_area_struct *vma, VM_BUG_ON_PAGE(!first && (flags & RMAP_EXCLUSIVE), page); VM_BUG_ON_PAGE(!first && PageAnonExclusive(page), page); if (first) - __page_set_anon_rmap(folio, page, vma, address, - !!(flags & RMAP_EXCLUSIVE)); + __folio_set_anon(folio, vma, address, + !!(flags & RMAP_EXCLUSIVE)); + if (flags & RMAP_EXCLUSIVE) + SetPageAnonExclusive(page); } void hugepage_add_new_anon_rmap(struct folio *folio, @@ -2550,6 +2550,7 @@ void hugepage_add_new_anon_rmap(struct folio *folio, /* increment count (starts at -1) */ atomic_set(&folio->_entire_mapcount, 0); folio_clear_hugetlb_restore_reserve(folio); - __page_set_anon_rmap(folio, &folio->page, vma, address, 1); + __folio_set_anon(folio, vma, address, true); + SetPageAnonExclusive(&folio->page); } #endif /* CONFIG_HUGETLB_PAGE */ From patchwork Wed Sep 13 12:51:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13382963 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD4B1EDEC4D for ; Wed, 13 Sep 2023 12:51:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4C6216B0186; Wed, 13 Sep 2023 08:51:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3FD766B0187; Wed, 13 Sep 2023 08:51:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 277DC6B0188; Wed, 13 Sep 2023 08:51:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 0C39B6B0186 for ; Wed, 13 Sep 2023 08:51:32 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id DE9C31A0A62 for ; Wed, 13 Sep 2023 12:51:31 +0000 (UTC) X-FDA: 81231560382.11.0F96603 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf24.hostedemail.com (Postfix) with ESMTP id 2BFC2180019 for ; Wed, 13 Sep 2023 12:51:29 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=iC+mDOYS; spf=pass (imf24.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694609490; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TPywS+lMvCdRAxxYldVBzyiy9uXPhmFWj9jbUK5aT08=; b=XoLDUhazgcHEb9azsVdSRmzVtn/PqGBJA1SXTo+EimSpPrDdphc4+i/fdQt04L6Sm+fuRV 1YZaOfIeGZ8hrWpwzKLgMokVJXocX7mBVfEy1cKrvzbe5uQnVdm8GomATOkwx8HeT1D6jL tupbupDGNU4krpSnxmQMjRtT3O2U8wc= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=iC+mDOYS; spf=pass (imf24.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694609490; a=rsa-sha256; cv=none; b=nz1ePfcKfIj2/SW3Uma/9QLNeyoAdvLdxz0oeyWESUVAIkQOL+od5CE516gaIMLgSz8uov 8GpiLe1D99vHzsCa+rhmTWOhiuA8oI/ufFULfdnaxjBr4glKYpcGTwCe75RmoS/0WR0OZt AEH+5RABXxmLWHkwedxC4eLBosMolRY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694609489; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TPywS+lMvCdRAxxYldVBzyiy9uXPhmFWj9jbUK5aT08=; b=iC+mDOYSGZ2M5QSMV9oPoL/zrYTcP94U71W9OrHfEl8JkrgkSnKP6dv94lvBAp0ieQN/+U sA63yt5NCLkJNushvUiQKEkOlZbdtWiPtkX0fLPKIaaRAqO2i723do8rgxznnyXK++pV/X D7AJ1LwGQ+T6BwXJOWnsJW1i7xBOlbM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-41-FbYiyCTvMFu_-aiiWy3VKA-1; Wed, 13 Sep 2023 08:51:24 -0400 X-MC-Unique: FbYiyCTvMFu_-aiiWy3VKA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BD850800045; Wed, 13 Sep 2023 12:51:23 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.193.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id A749F21B2414; Wed, 13 Sep 2023 12:51:21 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Mike Kravetz , Muchun Song Subject: [PATCH v1 3/6] mm/rmap: move folio_test_anon() check out of __folio_set_anon() Date: Wed, 13 Sep 2023 14:51:10 +0200 Message-ID: <20230913125113.313322-4-david@redhat.com> In-Reply-To: <20230913125113.313322-1-david@redhat.com> References: <20230913125113.313322-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Queue-Id: 2BFC2180019 X-Rspam-User: X-Stat-Signature: 9mp59qyf4ko63oxqbz7yzgbnih39ndp4 X-Rspamd-Server: rspam01 X-HE-Tag: 1694609489-695569 X-HE-Meta: U2FsdGVkX18CEh/IZNV/krAwlcQmDzoeAz5FX2XrL9h9PDrpHOrK02B4D7dfSI6mY22KWdDrAhwG7oWjT0I+eg0FvpUcaR/pz/XJ5U38uRbfJv9/Z0v7Tl2NnxsdF9eFn/QqhRBgyBb4yjwOUukp13m0V70xIAS0oQgAayAgfbcMdRghjYinPpGUJybY19WC/mx+gE6VhahCs3iFcZk8Jl/BLrRKTsdbBnE+0JIFsRx2dEEbzjbZHX5MV7fw8/Nc8ztJboqJQ3gksJDI128GygCQilQ9Lzf9xXahQ83zGq4/QDtq25i57HUCe6Mm8djcMVuyxvzplbzTSrT+wEsw9mY8bs/WVzJ2skQf3fAWwkdIAkpUxrmbA9hEQ9SssL/AjyYAim2oICjKrcDIgQecIZr3EEggRA7CQRVhZcpk57pouNBgbtuWIpO/USVk8SNmGuf3yBL2CPR0DM+CFVzKVnFI7KlHqGckfV6jxeQBwZlVaDstnxzr3exl/BnjSm/UdIoniKZUUEb7fqeEU5BoOnAhlUhhHx85FwWb2Cjqvt0we8h/7ZbNDHefni9/XcixUlGUo2dkLXi9vjc9J21RU2t+Hx95t4KuZAQxmGGDknSHx8bL/MShusvZ9duJstPJtXlhgad/lAktH3QivK44k7uTtCPv1zo7vP60HTOPCfhVLau2wuv3tATL+aXV+Hhh6992kjxHYwRe8IvmvfK1EfpQEg5j3xD0OxAerHkiRSBHV9EIkoNTwx8CuuF1XrdnACXwa/uUykD4xI2pMu2Fxv63msDvbimRuQoXMsRPARbb6M83CEVfOElRnCJebTrRCFvTj6FLPTjPP8Pkw9JTFGLT8KsovCl3uptkWlAQMaRBk6QbvYNT6qdQDLQt9zPMHiWXd38L1j5wEcm0xJBHtFd3tGJAqBSCOQDx0ZCcN6IM0nwCo4+7ChN17JqnLlAfGvEy2rAYysMfFkSyZzr yZUpGC6y UryFEU5oWSnRJwODHlv/R2ncVQq/bLuig38I8L7Q9hw/2g9Ef7xo+SheApAiVgR8eWPragDhT1wqlnzER3C20V9B4XfjGK219ruuZO72+z4wr/AyE2S/2EzwOY9OhZrKqOh+U6aWLUH+6AlFJclWs5CKueTV+y+c22jkMbU0l0BtsQrISzwjbY/gcCsHQ45EXoPTB3bQ1oY9GzEw2HRIzwZdKAw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let's handle it in the caller; no need for the "first" check based on the mapcount. We really only end up with !anon pages in page_add_anon_rmap() via do_swap_page(), where we hold the folio lock. So races are not possible. Add a VM_WARN_ON_FOLIO() to make sure that we really hold the folio lock. In the future, we might want to let do_swap_page() use folio_add_new_anon_rmap() on new pages instead: however, we might have to pass then whether the folio is exclusive or not. So keep it in there for now. For hugetlb we never expect to have a non-anon page in hugepage_add_anon_rmap(). Remove that code, along with some other checks that are either not required or were checked in hugepage_add_new_anon_rmap() already. Signed-off-by: David Hildenbrand --- mm/rmap.c | 23 ++++++++--------------- 1 file changed, 8 insertions(+), 15 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index ab16baa0fcb3..1ac5bd1b8169 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1135,9 +1135,6 @@ static void __folio_set_anon(struct folio *folio, struct vm_area_struct *vma, BUG_ON(!anon_vma); - if (folio_test_anon(folio)) - return; - /* * If the folio isn't exclusive to this vma, we must use the _oldest_ * possible anon_vma for the folio mapping! @@ -1239,12 +1236,12 @@ void page_add_anon_rmap(struct page *page, struct vm_area_struct *vma, if (nr) __lruvec_stat_mod_folio(folio, NR_ANON_MAPPED, nr); - if (likely(!folio_test_ksm(folio))) { - if (first) - __folio_set_anon(folio, vma, address, - !!(flags & RMAP_EXCLUSIVE)); - else - __page_check_anon_rmap(folio, page, vma, address); + if (unlikely(!folio_test_anon(folio))) { + VM_WARN_ON_FOLIO(!folio_test_locked(folio), folio); + __folio_set_anon(folio, vma, address, + !!(flags & RMAP_EXCLUSIVE)); + } else if (likely(!folio_test_ksm(folio))) { + __page_check_anon_rmap(folio, page, vma, address); } if (flags & RMAP_EXCLUSIVE) SetPageAnonExclusive(page); @@ -2528,17 +2525,13 @@ void hugepage_add_anon_rmap(struct page *page, struct vm_area_struct *vma, unsigned long address, rmap_t flags) { struct folio *folio = page_folio(page); - struct anon_vma *anon_vma = vma->anon_vma; int first; - BUG_ON(!folio_test_locked(folio)); - BUG_ON(!anon_vma); + VM_WARN_ON_FOLIO(!folio_test_anon(folio), folio); + first = atomic_inc_and_test(&folio->_entire_mapcount); VM_BUG_ON_PAGE(!first && (flags & RMAP_EXCLUSIVE), page); VM_BUG_ON_PAGE(!first && PageAnonExclusive(page), page); - if (first) - __folio_set_anon(folio, vma, address, - !!(flags & RMAP_EXCLUSIVE)); if (flags & RMAP_EXCLUSIVE) SetPageAnonExclusive(page); } From patchwork Wed Sep 13 12:51:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13382964 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0293AEDEC49 for ; Wed, 13 Sep 2023 12:51:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4C3326B0187; Wed, 13 Sep 2023 08:51:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 44C9D6B0188; Wed, 13 Sep 2023 08:51:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C7F06B0189; Wed, 13 Sep 2023 08:51:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 075406B0187 for ; Wed, 13 Sep 2023 08:51:33 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id C760C140277 for ; Wed, 13 Sep 2023 12:51:32 +0000 (UTC) X-FDA: 81231560424.03.7CA9998 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf19.hostedemail.com (Postfix) with ESMTP id 2F3AE1A001D for ; Wed, 13 Sep 2023 12:51:30 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="P7/lLqsf"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694609491; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iURFDA2RdJWU1H9IedgFZuHwYBnEAwvtokGRQic4P+w=; b=5RXzCqPuRJ0DW4enF8/PJRBGRZh4/SNYcJJdSHAJZrIg/3ygqssXX5n8IdyF0MNPG7rudz qZcyWsCRowihyCNSrTjis7DPb/bA2zV8YYMHsimFicQJvnKqZiMZg7j2UE1ZcR0uP1VW4p SPHCkgYOCor7InAhWImdxv6xztHFOJY= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="P7/lLqsf"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694609491; a=rsa-sha256; cv=none; b=t77NZWaVgD+/tw072g+iKAPcHNElGC4cdlXJYz5ZZmXVLbnszQHaUoTW5mWdE200CYHiJS DuhTF9lzkvHXSr87sJQoZp2OAaHwueq5vXirxEfRMzx9UcNgXvhis/d88phrr+n0aEVdNK FLWniYImPmp4KTsY78UhXqk1ReODxtU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694609490; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iURFDA2RdJWU1H9IedgFZuHwYBnEAwvtokGRQic4P+w=; b=P7/lLqsf6vWFk0ZOmZOsqOr9riYNo2cRU10YJ7PrKpe9kqeDqAyKzJwuqLDnLG0t1QCmDi sIcdZz1dyI5kxErfOOyBdfQ/5BBp3QGFMuB7GgQEI/rksIxl6f3HRCJfApCP0aUX+fObAS lbeYYIMA2ou76/ebauyMO7gwZ1qdwXE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-480-uy6DAjTwN3CYPzjjmN_0TQ-1; Wed, 13 Sep 2023 08:51:26 -0400 X-MC-Unique: uy6DAjTwN3CYPzjjmN_0TQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CB2A58030CB; Wed, 13 Sep 2023 12:51:25 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.193.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2A94721B20B1; Wed, 13 Sep 2023 12:51:23 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Mike Kravetz , Muchun Song Subject: [PATCH v1 4/6] mm/rmap: warn on new PTE-mapped folios in page_add_anon_rmap() Date: Wed, 13 Sep 2023 14:51:11 +0200 Message-ID: <20230913125113.313322-5-david@redhat.com> In-Reply-To: <20230913125113.313322-1-david@redhat.com> References: <20230913125113.313322-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Queue-Id: 2F3AE1A001D X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: pc9j3z7swjqiz6p5zzp4afpbyfuw4wek X-HE-Tag: 1694609490-425938 X-HE-Meta: U2FsdGVkX18G74s2erUOtIubUrNzNFAXgfeKuMx+vbWaiJDjMCh80SLw1zLmZifUqnkJIBVGHyEHXHfApLjvNyVRFp+IeD/EO5qUZSdNhkhbRHkvmNoTZWnCW+O/lIY1KjYMxWDEMu49kc2GmnsA21xZtfVQJIsIi1w5KlGVz8eSxlAF0BKUfElL9GmQcK4cVz8Idc07HT6vJPg/LUAWUvPgyUR+qaGyDwu1RXiB3whhLZgFPk4ei72oAt6WApfXNrIOwDClpSwdcxFpRvOAiMIxUKVwZeYNQLz8xWPp2/crx0ggHZox3TGXV5DS7+X0IEdsLu2RkahRJxgkG760+t25LXS4nndhQXYbvqFR7O/cFK+qaIMWAO8IL/YqomqTGJ9ulPEG/l3U2Ex6E5d+eIdhfFPeMnRE5sgyV+n/XCwm5Kl8cvIdp7ipNNRjRIq3RiojsLuwzg64ZMP5B+i9jQcrSBq2Un1x/EZRxRjMkHF8hgZZM7vfG/MqloDkbcwjT4YGDxiK8hAJeiRObzA2CUcgFd7yNcfhwdGuqwlr4WWB3SpODyakyRWHu/RNS0JYjuqsqKOxIpMh9g8pI9HO+8s6KwBnXxSPsNu77j/5uTpt3PLyPMGjTS/GWDanaTTiCZobL9JAvA8uMkR/sn0TFJVe5RQ0c0Y+oBopvG3CT1um3H8o57KHIfjRVZzLkqXGA2hjxIqYg17RTFUCBWGKr1RlJRQESm3NmXjHG8vkuWc02r0iSCIMaVT02Af8bwwhuDNWy5VeUosdT0caRsBfAlz2zYXvngf1LIDuePOzqs1GOzL0j5nb6uSAy2L99fZmBtR6FSQTd5pw9Vr4J3Mc14CLnbMcA3pRV4KEn7VLnwCIsDKK6yskHXqcOUIkDjDiP7oacyZ/IGhzCFVCx30+IELXhckVOKapQ0kAsHa0wVP6OfRSwft2BvtC+vpBYDlrrObaily9OJCP0SUVGao jVsNhYQn RMPaIQ6PulVePLzSXGrV5HzDF2E93jnb2zeeNZf4MSHz7ILycEPFg3v1Sfl9LEdpN53R1TLEQV8fwa8TbMxN0l2ubRF9vQG8bQq35LIa5Sz7GbrLwonVX9XSixwQlgY/YwoLqOZOuL//fbC+KdVkW5sxp8Td33M67orr858OLzk9HAT5KduZCBO5FdpGPkmohW5ZZOKkvjaJLXlGToyGepmfNaA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If swapin code would ever decide to not use order-0 pages and supply a PTE-mapped large folio, we will have to change how we call __folio_set_anon() -- eventually with exclusive=false and an adjusted address. For now, let's add a VM_WARN_ON_FOLIO() with a comment about the situation. Signed-off-by: David Hildenbrand --- mm/rmap.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/mm/rmap.c b/mm/rmap.c index 1ac5bd1b8169..489c142d073b 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1238,6 +1238,13 @@ void page_add_anon_rmap(struct page *page, struct vm_area_struct *vma, if (unlikely(!folio_test_anon(folio))) { VM_WARN_ON_FOLIO(!folio_test_locked(folio), folio); + /* + * For a PTE-mapped large folio, we only know that the single + * PTE is exclusive. Further, __folio_set_anon() might not get + * folio->index right when not given the address of the head + * page. + */ + VM_WARN_ON_FOLIO(folio_test_large(folio) && !compound, folio); __folio_set_anon(folio, vma, address, !!(flags & RMAP_EXCLUSIVE)); } else if (likely(!folio_test_ksm(folio))) { From patchwork Wed Sep 13 12:51:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13382965 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A520FEDEC4D for ; Wed, 13 Sep 2023 12:51:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D75BA6B0188; Wed, 13 Sep 2023 08:51:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CFF1B6B0189; Wed, 13 Sep 2023 08:51:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B28B46B018A; Wed, 13 Sep 2023 08:51:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 99B406B0188 for ; Wed, 13 Sep 2023 08:51:33 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 78CB6160331 for ; Wed, 13 Sep 2023 12:51:33 +0000 (UTC) X-FDA: 81231560466.25.20B10A1 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf20.hostedemail.com (Postfix) with ESMTP id B79DB1C0037 for ; Wed, 13 Sep 2023 12:51:31 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Fl6MNm9Y; spf=pass (imf20.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694609491; a=rsa-sha256; cv=none; b=HYKhmPpgQuUFsUSLDzBaEnWfB7zlyS04liX6TL9/G3ZSCKKfTDQRGNlslTj93NDLEUVmA0 iSzh4AOcAEvJMQymq3aDr168QC5Dz2aXPFLdETVQzr2p/0gNz54SE8eqGHXtJRJ2OxfkjY lfbcQy+CBdXMmO/Bmpkc+QVDg7QF27c= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Fl6MNm9Y; spf=pass (imf20.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694609491; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Z+qQSwZU5O6LoNzg+Y0KcgzyFROGp1piGCQLbieshoU=; b=BqZoCR6ljH8WLXvREbzEfXUptYs54ROFgmA/m4jnQg058F5l2l7eljiUkGuPafJy0xox4d Kdl+oSRsqK3jgpYFR6dTo6jYnDJjZ5Nwl9aTYygkUot2qDflQk3trgx8LCguxfVHi6kvEQ bUKnCU8Kdjmfgj1lJecdQ1ldhBNbLIQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694609491; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Z+qQSwZU5O6LoNzg+Y0KcgzyFROGp1piGCQLbieshoU=; b=Fl6MNm9YhWAC35q4yRQMYFdTAKRNhljX8wcvRVfdbWwXYutitKbnaOh5P/+DUDshyOTCxB Xd+uz+EN+ul9YciGxhN8QNenLkY77kGYB53Gx6F+TkfXRIsDe1X9fw/qJWRov31VBu6zeu T/a6dbsY23gk70OfMsYfoT7v3J8p+ng= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-18-4zJ54y7iMkaiv39wEbMctA-1; Wed, 13 Sep 2023 08:51:27 -0400 X-MC-Unique: 4zJ54y7iMkaiv39wEbMctA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6E9678011AD; Wed, 13 Sep 2023 12:51:27 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.193.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1F56421B20B1; Wed, 13 Sep 2023 12:51:26 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Mike Kravetz , Muchun Song Subject: [PATCH v1 5/6] mm/rmap: simplify PageAnonExclusive sanity checks when adding anon rmap Date: Wed, 13 Sep 2023 14:51:12 +0200 Message-ID: <20230913125113.313322-6-david@redhat.com> In-Reply-To: <20230913125113.313322-1-david@redhat.com> References: <20230913125113.313322-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: B79DB1C0037 X-Stat-Signature: yyc1a186mjid1yjz9g7n8g8hmpgtrkio X-Rspam-User: X-HE-Tag: 1694609491-38975 X-HE-Meta: U2FsdGVkX1+WMWpzvrdGr6+Rdz0rGG77tWxkoohRhblWaR5cuyEDSJgRoZoGLsFSZfKchLjlDLhsbryfVzHv1nCT7L4DNgZuoaGtmFkLkxO5ACrpcBA+jjKqoxJjkv2tklWtnhbwFQr+XTin/j/qAky1ppmA5/V0MnbTjAPtltlv0syk1KLmI6OtIv/5NU5/Od8nOPnFm+ya8sRIo/ygR85kLiN9aYYXRMJ9nikLLcaWHgrt88AhMu8ZBdR4dyLNF4EnO4/Mf5kofv3/DhjKYxF8tJ/nRcbs1gNnJWekgHwa4rDkt2JL6fRWqqt8OsgGJ+8Cb7s3eEhe2Zrbo9xm2RT1IpZfm770okKTK0R7cv0EgzOaSwGlPEl+015anA1tS8bGTM4+1/DqAagcnrcpDRyLhx4Z9t4DafKOW+BsrVEMwE8Nvivx0XjGdDDFSYKqV7mg2PtYGCk7Kusbjgj4ZZVgJ7RXSsG0iHgvOV/V0dCrZIGraMr0uiX7+7Eeg8DLHje5GuweJC6ImOY+WTCdyEXRRA8oyM8omh+Jo64YXD7KJZ7mEtqzGhPS7332YgHI3igcQ0AJj1s199MiS0+P9gkvmdhEWX4jNQvm2pjiU58fyiE3FBr9o/9O3leKAtbysduor7DR9Rf+/0QfINKnnjt9ekuVm/MREi0uISUTA56iDRsa9nbDr00rvltbafV+pF4Pn7B1hkxISwlxTzy6Dvof2mZFk+q7U9Xwlxc286b9mKndQfxYI1eJmWP+evTjOZkq3p+yk4t4qaSCiHoo43QyP6vWRusQS5aTBIe8H/8LH7wmR3Gm/a2DclBum4AUDm9x2gSKex6NJDEar+OSwH8AJIbNrVB+cbHgSH24SQo3r06HVBdJqc4FAnCc2GWXAypt2QHEvPEXN9se6/bRPGfiHl5KNiHMDRzPMLWRRRav7c0uqYdChbo6kXD1ZCbF4xvUR9xIjB+61aV48sR +AyvBdfi atdtYilhrfSwUq4EHkE0+u4iWzb9MjKVr7P9f0olkwE4SQj6FKNZN4OKnVbkhbUXPfRK8BZSSDQ4UAjFqwKG90YAYYAJHFU5U3wlO05tWvk/gleYsUsqrgU00N7yWCHQNs1NDU19BpTz18rZTggPzbMNvHD5Rw1+wuJIMnbAZFp0pmLI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let's sanity-check PageAnonExclusive vs. mapcount in page_add_anon_rmap() and hugepage_add_anon_rmap() after setting PageAnonExclusive simply by re-reading the mapcounts. We can stop initializing the "first" variable in page_add_anon_rmap() and no longer need an atomic_inc_and_test() in hugepage_add_anon_rmap(). While at it, switch to VM_WARN_ON_FOLIO(). Signed-off-by: David Hildenbrand Reported-by: syzbot+6e4f59235036c3c2e296@syzkaller.appspotmail.com Signed-off-by: David Hildenbrand --- mm/rmap.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index 489c142d073b..10d477a0991f 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1199,7 +1199,7 @@ void page_add_anon_rmap(struct page *page, struct vm_area_struct *vma, atomic_t *mapped = &folio->_nr_pages_mapped; int nr = 0, nr_pmdmapped = 0; bool compound = flags & RMAP_COMPOUND; - bool first = true; + bool first; /* Is page being mapped by PTE? Is this its first map to be added? */ if (likely(!compound)) { @@ -1228,9 +1228,6 @@ void page_add_anon_rmap(struct page *page, struct vm_area_struct *vma, } } - VM_BUG_ON_PAGE(!first && (flags & RMAP_EXCLUSIVE), page); - VM_BUG_ON_PAGE(!first && PageAnonExclusive(page), page); - if (nr_pmdmapped) __lruvec_stat_mod_folio(folio, NR_ANON_THPS, nr_pmdmapped); if (nr) @@ -1252,6 +1249,8 @@ void page_add_anon_rmap(struct page *page, struct vm_area_struct *vma, } if (flags & RMAP_EXCLUSIVE) SetPageAnonExclusive(page); + VM_WARN_ON_FOLIO(page_mapcount(page) > 1 && PageAnonExclusive(page), + folio); mlock_vma_folio(folio, vma, compound); } @@ -2532,15 +2531,14 @@ void hugepage_add_anon_rmap(struct page *page, struct vm_area_struct *vma, unsigned long address, rmap_t flags) { struct folio *folio = page_folio(page); - int first; VM_WARN_ON_FOLIO(!folio_test_anon(folio), folio); - first = atomic_inc_and_test(&folio->_entire_mapcount); - VM_BUG_ON_PAGE(!first && (flags & RMAP_EXCLUSIVE), page); - VM_BUG_ON_PAGE(!first && PageAnonExclusive(page), page); + atomic_inc(&folio->_entire_mapcount); if (flags & RMAP_EXCLUSIVE) SetPageAnonExclusive(page); + VM_WARN_ON_FOLIO(folio_entire_mapcount(folio) > 1 && + PageAnonExclusive(page), folio); } void hugepage_add_new_anon_rmap(struct folio *folio, From patchwork Wed Sep 13 12:51:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13382966 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DE63EDEC49 for ; Wed, 13 Sep 2023 12:51:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 932436B0189; Wed, 13 Sep 2023 08:51:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 894C96B018B; Wed, 13 Sep 2023 08:51:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70CE16B018C; Wed, 13 Sep 2023 08:51:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 559DC6B0189 for ; Wed, 13 Sep 2023 08:51:35 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 36E291A0DAA for ; Wed, 13 Sep 2023 12:51:35 +0000 (UTC) X-FDA: 81231560550.09.44D6F30 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf06.hostedemail.com (Postfix) with ESMTP id 70778180005 for ; Wed, 13 Sep 2023 12:51:33 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=XjRh1VaH; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694609493; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=t7xwwUMm6Z1sTtxTNKBbcilCnHU1Bmn/mrplLkXOKnQ=; b=k+bWJRMVbsi5yl0+o1bko159uAZL75re/uONfWLpCxluPkK5PyHZZWpoBzR8b4KgVHYZ6Q pKj5T1GU0jiVaRSObjTC3U83EiKYo6ElkfXX0etmCU85TjXtArgL6YLfaLCVPWaFyMKXsw gwej0OL27HkZyHTPnYZafmKKqAlEJwM= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=XjRh1VaH; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694609493; a=rsa-sha256; cv=none; b=34y4sKVwc/6Udd37wPVdnwzjB5XRkS6jBh3wPvyXD31Jg4SI8kBqkK59p943a6l4kSXAYT +4ZOIE7TazpNsUEg/uC6QMDORqEfYwhSZQcXHKlWV6hddZRPHwegxBA3Wu9AiKau9+JIol /atHLohWVa17+vNgLGuPoinGJWrJLbg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694609492; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t7xwwUMm6Z1sTtxTNKBbcilCnHU1Bmn/mrplLkXOKnQ=; b=XjRh1VaHkxphM4qrtl5pZCD5YZrVeBkTtuXJ2mFaj7A0cLGZEDtzoflWtxzEYyJ95oxgxb sYWJJoIslHmcWZcvuF6wWJhHpPS/q0H5uScJqP2hesramQf7thyUMIUT9iHgfqupYBcucn KpQku2qKfrQD+M/yRcYGJvtu5G7cPZU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-389-tgr1yN5WMfuxKuox-CRlxA-1; Wed, 13 Sep 2023 08:51:29 -0400 X-MC-Unique: tgr1yN5WMfuxKuox-CRlxA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 06B9A857A9E; Wed, 13 Sep 2023 12:51:29 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.193.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id BD4432156722; Wed, 13 Sep 2023 12:51:27 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Mike Kravetz , Muchun Song Subject: [PATCH v1 6/6] mm/rmap: pass folio to hugepage_add_anon_rmap() Date: Wed, 13 Sep 2023 14:51:13 +0200 Message-ID: <20230913125113.313322-7-david@redhat.com> In-Reply-To: <20230913125113.313322-1-david@redhat.com> References: <20230913125113.313322-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Queue-Id: 70778180005 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: zhntjzgjccheabnxfoopmag5deapzy5q X-HE-Tag: 1694609493-64509 X-HE-Meta: U2FsdGVkX1/xSHL2hOh0ByufBo19NHHtRXh4MvHwlLAmcT/3mnSEQESPq/s+ibuwD9evUN09IEiF2BYL5Ym5mpyVn8Dm7OIqSHyl5XCC1FlTKhr/x+94kwQUryKRH0WgCOr+pwuJf/oHBt1Dl3oBo5TFbkIH2gtGXV/g71LV6vcgIT+rfE9DT51NV1J8QV8gWGDVVtPtziINVXSN2SH14mVv48nfXVcKvhRJBS8XX4PDmZ49U7u22wL+bhanYmnwBsx01zlWQzIvjHTrb5Z/KN99TIX+wrS30SHnym9l0PwNvI1bWwIB8XnagMRh3l15lCtSa5MN/QPBwthVNGjM9lj9uI6lKl+on94q+E1Sz5bA1bhOV2/TtD3WelXzO1zHK8tRxcs3DAbyzDydlM4Yc9N5t+TOqcWLeG+TkLRikgUzWaq20O6u6PfNQFVkmcQt2CJnvlNG5rCAGWOraBoAP+0aHb2ufT+lhkEgj7BmUwbFLj2PUZrc0YDSjp868Zm1BJYT3hLnKiHclntsROHI40CIQUgCTsB9FQn+7RnRjOIvjPjPUAWQQrZQJhPYjfpqgVnwSsTKclMJEly+7VOh0S6k5QZ+8b31WnozasT7PDLvpE2SXx3S/uJ8VN6evREWU2PtbI+Q/DwCuc9pYxv2IXuBwvNyVOInS04oPpNvJU8CEz3lk1gjlMtw0E94JoRtrG4cmc+oJnlBBiKVjYv5k080Z37k33+FpRSnLEOQbDtOTLPd+q1Usr3vxeIqHiJPmQdqyuPs40Lb30EAN+2vRG3Ph/IMKFKQ9h+PO0Qlpg/0yajlXanBfm8p3NrJ8IXPKuf0ceJPUVKHcPH2nZ2166vCyQ2UuTAAmnXbvn9lm0qPkC/gWfd0gj07RXzzeIk7WfZmyekELUsX0gnjOwxdDtfsUe0vuoXyH6rbfDHFBGFd1inNG9EzDxdA5Nno8ZHJw7VzIWhUhl/bVbovBTT jeXnqOgB tL7+aM4HzyIPvd7dDhhRI1MgA4MEPpTeamQylB+/QE8/UCkw8MhtDVwGrBuH5dEPyDGwDdDzOOftKUmnSe4W+aPmTs9mquMJ1LDVXt4dpAPkoEIQJhtpkfX+XUnTztyn8iW8nvppQCUtEAOPQ1oo+HcDA93uJMfZpaTTLw0RcEXMVR4FjV3Hwz8Lr7FSvMJfXwiMvslX7qz/+3YxVU2cOvNkpeQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let's pass a folio; we are always mapping the entire thing. Signed-off-by: David Hildenbrand --- include/linux/rmap.h | 2 +- mm/migrate.c | 2 +- mm/rmap.c | 8 +++----- 3 files changed, 5 insertions(+), 7 deletions(-) diff --git a/include/linux/rmap.h b/include/linux/rmap.h index 51cc21ebb568..d22f4d21a11c 100644 --- a/include/linux/rmap.h +++ b/include/linux/rmap.h @@ -203,7 +203,7 @@ void folio_add_file_rmap_range(struct folio *, struct page *, unsigned int nr, void page_remove_rmap(struct page *, struct vm_area_struct *, bool compound); -void hugepage_add_anon_rmap(struct page *, struct vm_area_struct *, +void hugepage_add_anon_rmap(struct folio *, struct vm_area_struct *, unsigned long address, rmap_t flags); void hugepage_add_new_anon_rmap(struct folio *, struct vm_area_struct *, unsigned long address); diff --git a/mm/migrate.c b/mm/migrate.c index 123fc4dc0bc4..d4e8cbf0a704 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -247,7 +247,7 @@ static bool remove_migration_pte(struct folio *folio, pte = arch_make_huge_pte(pte, shift, vma->vm_flags); if (folio_test_anon(folio)) - hugepage_add_anon_rmap(new, vma, pvmw.address, + hugepage_add_anon_rmap(folio, vma, pvmw.address, rmap_flags); else page_dup_file_rmap(new, true); diff --git a/mm/rmap.c b/mm/rmap.c index 10d477a0991f..789a2beb8b3a 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -2527,18 +2527,16 @@ void rmap_walk_locked(struct folio *folio, struct rmap_walk_control *rwc) * * RMAP_COMPOUND is ignored. */ -void hugepage_add_anon_rmap(struct page *page, struct vm_area_struct *vma, +void hugepage_add_anon_rmap(struct folio *folio, struct vm_area_struct *vma, unsigned long address, rmap_t flags) { - struct folio *folio = page_folio(page); - VM_WARN_ON_FOLIO(!folio_test_anon(folio), folio); atomic_inc(&folio->_entire_mapcount); if (flags & RMAP_EXCLUSIVE) - SetPageAnonExclusive(page); + SetPageAnonExclusive(&folio->page); VM_WARN_ON_FOLIO(folio_entire_mapcount(folio) > 1 && - PageAnonExclusive(page), folio); + PageAnonExclusive(&folio->page), folio); } void hugepage_add_new_anon_rmap(struct folio *folio,