From patchwork Thu Dec 12 18:33:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13905798 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A686E7717F for ; Thu, 12 Dec 2024 18:33:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7FE126B009E; Thu, 12 Dec 2024 13:33:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 786B16B00A2; Thu, 12 Dec 2024 13:33:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 600E46B00A3; Thu, 12 Dec 2024 13:33:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 4153F6B009E for ; Thu, 12 Dec 2024 13:33:58 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id E43ABC063A for ; Thu, 12 Dec 2024 18:33:57 +0000 (UTC) X-FDA: 82887155316.21.66DD8C6 Received: from mail-qk1-f180.google.com (mail-qk1-f180.google.com [209.85.222.180]) by imf07.hostedemail.com (Postfix) with ESMTP id 118684000A for ; Thu, 12 Dec 2024 18:33:26 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=FSUfA8cN; spf=pass (imf07.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.222.180 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734028424; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=ZqbWz5w80/Ao2+q+hylk/EZt96xoYzXRfCQenTqI3gE=; b=w1RHBkLpns/fNk65dDMW6sfzvV/jOqPfbwA+CjpOiUwGzHLu+mhuVUN0FADv7xyrBlUbzf te2Kgzbed6uBAz0jzuG+mWUBT/ZbtSC3hvFajsce0OVu94UXwcnsjkWOmQLa0LE+FFobDc vaTGsN6CywdbJziV8U557v7SBK4uLMo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734028424; a=rsa-sha256; cv=none; b=5HCLXOcTHJ9UflbfimNYZnk0VlM5YT34kIz8Dr8690cGy+QYgpAmIoi323vkuN2fVY5Mt8 seeap10CXXLJm/xVk33m6eoraiL0/LdK8waa0PauSDcxSngUPepQ4tp4RYzPEY0SOtNPQt rHuyeCZKmqp7ffgtLtCGnxv+JGEwx3A= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=FSUfA8cN; spf=pass (imf07.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.222.180 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-qk1-f180.google.com with SMTP id af79cd13be357-7b6f8524f23so55800685a.2 for ; Thu, 12 Dec 2024 10:33:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734028435; x=1734633235; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=ZqbWz5w80/Ao2+q+hylk/EZt96xoYzXRfCQenTqI3gE=; b=FSUfA8cNQlO84Dfeuv12dOB0RaGZedgPivSq/dBt7+SRNznyYanNpAuFIhsOCR0WRt c6XzpgljzkraquCeiHOmC4IPyJDuYTIB4yqKxXMxzk4IvYbgCzsP1og8sl+jZjCyJOPw Kpa5fjaZ3V4JHtQ+Hy+l3ZnQmFAVGvGvdxCp1qWI8F8LO2ERRZ2yAMd2nejf1mVqzpgi X7yRiHX/dViEOF+qN69zhOqqcyh9SXQRP/fSYMEvpjOBsbQEaqjVi63ZNwPQI2Fr7bOd RisFFY/wMQ3v9Eg+afZ+49GowyfvqF6M8MjlqcKJ7TRGa87OWQ8BQmE8Q0PpdsQ/END3 rPsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734028435; x=1734633235; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ZqbWz5w80/Ao2+q+hylk/EZt96xoYzXRfCQenTqI3gE=; b=Nre7+M/Xy7EBX36EQPodFlOhSszSGK8HnClTbgDE6j/9J0o6p8zkWqBdrJgGPqFi4p b+pQPT3hADpJsViT9r7dZNyRjNBRo384OrfxJebHU/6YE88Iq56CmgLqj8WxnesxMXuV d4VzGIEXaoVmojipIBANF4i0fN8ie8jeXEPAcZKRwp7PHjkgI9ZlF7wZBQn01ZJNtR0w SzoQWujijqQbExFG0aa08JpZrlLNf0rciCZsc+xTHdTrYlGli1enHxIp1d/8RkEZb78i K5MorcdEPpqwrlW8cckuDitkHWK83FLnh646XrGTPCK5hDTIacS37KSfWq8CciNhGxEY LtWw== X-Forwarded-Encrypted: i=1; AJvYcCUJ0RUe2WubmOz9DgfueYNQjSBEs4z6JoUxv/7DUtZxy5xHez0+rO6EpV5zfLTk9B/jnmgTKwGRHQ==@kvack.org X-Gm-Message-State: AOJu0Yw1VqFT1PZcVoq+fWuzRWcvhYCDpuWjfJYdsJ/PPf0kkw8vHLHu RVkJMU6+lK8UJl+8zKFKvALwQlUfwwI7Z5zi15m+FyvGEG8niLCn X-Gm-Gg: ASbGncuJGPCQYBcOYLV/D3y2CZxXmxI+LSfJYGq10zGp68a1x6hGyPRPxxOJ4dy55EP 7z7pwXYWHdXITGejjn3+OE42qODatpD5exd0Bd2+omH68GapnGe6sbEtYV2WrZRgVh1e2JzaHxW v3q7b4+pFJVk8EiruYYdIKGcFBa1BCooXGER0thoqReonnXXw+kPpSnXCZA/XHecPUNzl0Pinpd 5e8D+znyW89ktCkJq8zL6AxJqegfleDCeyxAGKJf8fuYOSzOuq2uQ== X-Google-Smtp-Source: AGHT+IHW4lqyDD/WZ9MZc2ZAevw778ZC2oWLKzpYfIVQPXRhxGqN/JXNrTHqvV6y9ff/9LhoNEnlKQ== X-Received: by 2002:a05:620a:40c8:b0:7b6:f110:43d5 with SMTP id af79cd13be357-7b6f893016amr223205185a.18.1734028435127; Thu, 12 Dec 2024 10:33:55 -0800 (PST) Received: from localhost ([2a03:2880:20ff:c::]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7b6eed3d22dsm157188685a.31.2024.12.12.10.33.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Dec 2024 10:33:54 -0800 (PST) From: Usama Arif To: david@redhat.com, akpm@linux-foundation.org, linux-mm@kvack.org Cc: hannes@cmpxchg.org, riel@surriel.com, shakeel.butt@linux.dev, roman.gushchin@linux.dev, yuzhao@google.com, npache@redhat.com, baohua@kernel.org, ryan.roberts@arm.com, rppt@kernel.org, willy@infradead.org, cerasuolodomenico@gmail.com, ryncsn@gmail.com, corbet@lwn.net, linux-kernel@vger.kernel.org, kernel-team@meta.com, Usama Arif , stable@vger.kernel.org Subject: [PATCH v2] mm: convert partially_mapped set/clear operations to be atomic Date: Thu, 12 Dec 2024 18:33:51 +0000 Message-ID: <20241212183351.1345389-1-usamaarif642@gmail.com> X-Mailer: git-send-email 2.43.5 MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 118684000A X-Stat-Signature: hfsxdt1wrzsoc1yg93y51e5zzo978kjq X-Rspam-User: X-HE-Tag: 1734028406-193689 X-HE-Meta: U2FsdGVkX1+RCK4DdZOIKNomjTCn75PNHIHVh4CZ4gMNqFhIjN0xuYUVWC9ZzhOwBPRq29YewPk4StHrwR9r6Oas9TZ+xqZM+Ouv9MdhE8qJUAIF31ZXTvAfkqLxEcPlWDzceF0icagM8M+rMOstkNiiS+K7pBS4iuZ5EWEnEHujD/yf+XI25uXAPPpF8X0J/37tR4ZHsHpxXiVs58XtACbrRVKp/tFHTw69a5T/g/Kiawk/jmITqLipvGFcTLNB5Jdpi6wC1ALuRAQfpQPrQnbGgAyb/QcgC91U/awl0P2Sso0+d3RvfoZIPafpZlP4BW0zBGStQRag+17CUvbcdyNZRvnLtAUkfixR8eiu5Ts0aE7sSpz66F+sniEXJblGdb8VpN/7hTVGUqCXwn0lY4xZSRcHFtujfv+uq4xySHlzLL6sjTHGngK0vRahuNT5Z/J1PnO4PYkiecneb5dgFG80cegrUy3OicCM+vrCj6pH2IZfA+dJslyLBmxogcaKKvhaAMcNi4CZd8yjnVldmnIh/gf3/cs+V8xpMrr6XWjuxnxLI1QuVOxjCSykgeARdhNpMG1Cbh27m18JSfuKAxAwhFj3dLkuB0QeuXWEaaHi4PoMaa7kjZHNf+UDAkzAV5rkCYVeX2zZpWtamPVp2LGxCjOLvE1xFKx/0kOjHvx3XgRlD/VXmYqpyFQVvjqb6NHgw2Y1WTMsnooh+aANAGWr+AFG39KlEQYeMgEf1AEAwgZBe7snjdNch/3dAWHtm+fiwJxXA0LvUdIybDVRnMR+dhVBq3dNC9T0vc9CuCSH3YMxwRfpwB8wuimy+oHWiACdqC7tYSd7hfkhMyiRCq2x8X7JTrhZ7mxYg5l2mbVChYDLvze8outRezjD8yuDqkJG/8xb3+xApZP9hjwgcCW8R7/SIzJZkdt45xE9eNLWq+4CL54LEBaqArGSS//7pcGprUP0Cr86m6hprYz zx2efMRz c/+6NOArktIqGbS9MYl5tqdLs6Tp7QeM5o4oxsyFA/aKWs8NpGowOm5oyYQkBVaMYl/aW7M2muOBReoX+t6L5i9KYxiL/Fkvv+wFsf9zeCM6lcOE9A20lrZBfiMpCMfpmkIQ7HlPe0Y3qPFzaYPYis/m4VORrBanzoQkkuYo2b8XyQ6iFopig6oiQeu1hRWB1rQNNoxqZQjvW1AS3YfDjd6/+UGMFtO2DNj9nBuDkyVZpJsmlRqvtsa/PnXUCgQ40Up6KObMWGR7Ox7HJ3n/aMcy06ASvbdLlSh4sJ23bMuaZTvU9fOtsWS7TZWB3fhw00eLhXYrfslI9pBdHZs6WMYKiWUM4iTnoSqrIzURP3u3wFIJgihiSLNLJbsqz3zKsTAoaJzoLoyjeIt1UH1zS+A/RYo6aNaX+MMSXVWSefcgJeZnS5okskpfNL407MwduyRvRUrT0cc3CrTw0w1TYnsISihZ15Yex4qqB+DK8gOJIbu3ZfR/bL4UvYrQ3QGBm1j7ZOPfZnxzrZ3eE7CtpKoO0/Q/a0+0yIbVHTBUqauOvT+3pqNf0BJGkhA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.002777, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Other page flags in the 2nd page, like PG_hwpoison and PG_anon_exclusive can get modified concurrently. Changes to other page flags might be lost if they are happening at the same time as non-atomic partially_mapped operations. Hence, make partially_mapped operations atomic. Fixes: 8422acdc97ed ("mm: introduce a pageflag for partially mapped folios") Cc: stable@vger.kernel.org Reported-by: David Hildenbrand Link: https://lore.kernel.org/all/e53b04ad-1827-43a2-a1ab-864c7efecf6e@redhat.com/ Signed-off-by: Usama Arif Acked-by: David Hildenbrand Acked-by: Johannes Weiner Acked-by: Roman Gushchin --- v1 -> v2: - Collected acks - Added cc for stable@vger.kernel.org and link of initial report (Johannes) --- include/linux/page-flags.h | 12 ++---------- mm/huge_memory.c | 8 ++++---- 2 files changed, 6 insertions(+), 14 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index cf46ac720802..691506bdf2c5 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -862,18 +862,10 @@ static inline void ClearPageCompound(struct page *page) ClearPageHead(page); } FOLIO_FLAG(large_rmappable, FOLIO_SECOND_PAGE) -FOLIO_TEST_FLAG(partially_mapped, FOLIO_SECOND_PAGE) -/* - * PG_partially_mapped is protected by deferred_split split_queue_lock, - * so its safe to use non-atomic set/clear. - */ -__FOLIO_SET_FLAG(partially_mapped, FOLIO_SECOND_PAGE) -__FOLIO_CLEAR_FLAG(partially_mapped, FOLIO_SECOND_PAGE) +FOLIO_FLAG(partially_mapped, FOLIO_SECOND_PAGE) #else FOLIO_FLAG_FALSE(large_rmappable) -FOLIO_TEST_FLAG_FALSE(partially_mapped) -__FOLIO_SET_FLAG_NOOP(partially_mapped) -__FOLIO_CLEAR_FLAG_NOOP(partially_mapped) +FOLIO_FLAG_FALSE(partially_mapped) #endif #define PG_head_mask ((1UL << PG_head)) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 2da5520bfe24..120cd2cdc614 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3583,7 +3583,7 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, !list_empty(&folio->_deferred_list)) { ds_queue->split_queue_len--; if (folio_test_partially_mapped(folio)) { - __folio_clear_partially_mapped(folio); + folio_clear_partially_mapped(folio); mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON_PARTIALLY_MAPPED, -1); } @@ -3695,7 +3695,7 @@ bool __folio_unqueue_deferred_split(struct folio *folio) if (!list_empty(&folio->_deferred_list)) { ds_queue->split_queue_len--; if (folio_test_partially_mapped(folio)) { - __folio_clear_partially_mapped(folio); + folio_clear_partially_mapped(folio); mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON_PARTIALLY_MAPPED, -1); } @@ -3739,7 +3739,7 @@ void deferred_split_folio(struct folio *folio, bool partially_mapped) spin_lock_irqsave(&ds_queue->split_queue_lock, flags); if (partially_mapped) { if (!folio_test_partially_mapped(folio)) { - __folio_set_partially_mapped(folio); + folio_set_partially_mapped(folio); if (folio_test_pmd_mappable(folio)) count_vm_event(THP_DEFERRED_SPLIT_PAGE); count_mthp_stat(folio_order(folio), MTHP_STAT_SPLIT_DEFERRED); @@ -3832,7 +3832,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, } else { /* We lost race with folio_put() */ if (folio_test_partially_mapped(folio)) { - __folio_clear_partially_mapped(folio); + folio_clear_partially_mapped(folio); mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON_PARTIALLY_MAPPED, -1); }