From patchwork Tue Sep 3 21:36:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13789328 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A9B4CD37B1 for ; Tue, 3 Sep 2024 21:36:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 88F8F8D01E2; Tue, 3 Sep 2024 17:36:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 83FE48D016E; Tue, 3 Sep 2024 17:36:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 756D58D01E2; Tue, 3 Sep 2024 17:36:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 55DDA8D016E for ; Tue, 3 Sep 2024 17:36:57 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id F12C81202A8 for ; Tue, 3 Sep 2024 21:36:56 +0000 (UTC) X-FDA: 82524737232.19.04F200F Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf24.hostedemail.com (Postfix) with ESMTP id 45F1218001E for ; Tue, 3 Sep 2024 21:36:55 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=DwYPyyxh; spf=pass (imf24.hostedemail.com: domain of 3doHXZgYKCBUJFK2v919916z.x97638FI-775Gvx5.9C1@flex--yuzhao.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3doHXZgYKCBUJFK2v919916z.x97638FI-775Gvx5.9C1@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725399367; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=UmXnnh8rfcf9Ef3Qk2vhXxok9NGUV0TrSFmJAhvfZqQ=; b=1AjEEQwu04K7TpXx78o3D4rHqPnX15/qJx999cIFECRVGAjAhS5bBiC4IV0eUVh5+kq9J1 csI/LdfdSmtTGSvK87tCNxU8WW9Qdurva0Xvwk6Mq/5+ec3Jm17YP4rQJhYsODzbf+h5bw 4NOGW8la+Twd2QqPzfhiESLSxeGGJ3c= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=DwYPyyxh; spf=pass (imf24.hostedemail.com: domain of 3doHXZgYKCBUJFK2v919916z.x97638FI-775Gvx5.9C1@flex--yuzhao.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3doHXZgYKCBUJFK2v919916z.x97638FI-775Gvx5.9C1@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725399367; a=rsa-sha256; cv=none; b=IiE1HF77kmnx+Qu2Bzdip68HiqSOQfG67ySZMRu+6NPlU22+XkIxpjhN5UBcEBfXUFLRI6 PeDFPc8mApiPT6zScZUbQFbhqx6pQtSI9e3fcv2j78mIDi0kSbUsUSJW4RKM4kIEAibuuQ UNG5B9YIJvpvz64ABgfzFmieOPS5V3o= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e1ab008280aso5114705276.0 for ; Tue, 03 Sep 2024 14:36:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725399414; x=1726004214; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=UmXnnh8rfcf9Ef3Qk2vhXxok9NGUV0TrSFmJAhvfZqQ=; b=DwYPyyxhO/Ix+d113jM1lM8UFcf3YoSENMty7qa/+LhdSz3r5lG6yjYU7v950InGVi ykjgpgYqMIY8YIWye1XvTfDQjO2mjROlbgHS3Rs0R9kJ9hZ9aH+sgyjUFMRHTqaa83GJ afWF3YreN2e6SWbv0OnBi1wIFsfzWk2CZiMFcG/Se/i11PJEyTCIdQk9slm+KnK6eSkR oHqeyLHxWT3yTNYDyykrmVy0spHqNrGQAukKc0LmFcXMd1awP6Rsj3mGTox5d64+OZ5V owUwRR1ymsT0qg3J+ukJCxA+nEPPmXjhPe2291+e3W5PjZSDiiLbNYZpP5XnBu31iz5p 3BVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725399414; x=1726004214; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=UmXnnh8rfcf9Ef3Qk2vhXxok9NGUV0TrSFmJAhvfZqQ=; b=byxdsEJD6To1khmHp6K3bACtBglDmAalFWfgSfIHLCeeKW3EpYStcmY4+V4SmIuU2p +7or+vkiaonektPesLIw9RI9JsqgvglkgdeQ/2SMvYVHlwSlcA5YRx7wjXQr+wI4dOtj x2j02G5fj6NzKm7jRB0Hqv0sVOSy1Tq2HHYHd+r6S2+7NGubWZeRyNI6k85HlZVuFGqZ p+VCInHdcGOcGPHIxwqnGuUysPehhSB+zrGSF2aTYk/KbXGnw+EHfCs4rjyxLQPHCABa sxrB2u3+oUqTqb8afjj/TKhz1EcVNO/CxOWwulEf1+G4wAqSpaCyNeXo13PrweYl2G3u tM6w== X-Forwarded-Encrypted: i=1; AJvYcCVOyYrZH48bpsZZ0Uh9Dx2hdp8t1lrJrnZy/czz4oAFnNzEDk1cQNyU/Yyq2meNj38+1aB+nrvIng==@kvack.org X-Gm-Message-State: AOJu0Yy1L1nZR2VcmA8VDddo4IufruhjGKkI3CWS6YvhqjSBcf8+Ebux trwQ+ToQ48C+mXr6ti41S9U15jFuxQpHDdpJUH5WqvH2Dz7N6RV3xifcegvuxgrF3LpkeDB887k I1Q== X-Google-Smtp-Source: AGHT+IHe87noeWnjfFcEVgXXd4a8n6i/0Cooj3wFTObUDcV3v5LDVjuDx6ZfUiGUxPYz4jmQfYAlTZsvPWs= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:bb58:3d85:3f9c:c6c6]) (user=yuzhao job=sendgmr) by 2002:a25:e910:0:b0:e11:7039:ff92 with SMTP id 3f1490d57ef6-e1a7a3e8384mr25459276.11.1725399414110; Tue, 03 Sep 2024 14:36:54 -0700 (PDT) Date: Tue, 3 Sep 2024 15:36:47 -0600 Mime-Version: 1.0 X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240903213649.3566695-1-yuzhao@google.com> Subject: [PATCH mm-unstable v1 1/3] mm/codetag: fix a typo From: Yu Zhao To: Andrew Morton , Kent Overstreet , Suren Baghdasaryan Cc: Muchun Song , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 45F1218001E X-Stat-Signature: 3np1mmdgrmnfj73577h44wwyo3uj1thb X-HE-Tag: 1725399415-27130 X-HE-Meta: U2FsdGVkX1/yhVJckXVep19q+g73L0H+NwLwOJTN14qcnNezaf6GH8R5uPN44s0rbqSu5/aiOohK/qonsUbPBcUYfhRB3O1HdI1yZ356qx69IcpmBd8CNdvZ5m2duJ94JDX8Udiia8G7JJnLCPmBj6UeqCbxA6yQ//583o6Rwu3s1gGduWttQA6SVpgnCHcO06Vnga8ifaafrt9kstk0w4RlQsSj+4E4zVyLiNp3kY7756EcX6PCAWPsshnjqRrKHRIE6gHPEKXBKwklNjgTwMGURSmHfhFnU211zA3hgCfwJ6+8g/VlHw7o4kgNrrY+CtZjTF2eN/a5J8qRgjYyMoHBW7JcjL/lOlPUPjOwIFVxaFmzZFrrR9CoZEEbe48lTNisTivZqXAzY8xRu5v/SnbYAelsdtp0beY/BKmIXagD4iVqwwc3kr7564zZ3Iz42XiuYJn2/8XF+YrBV8bnXlvltz9tduUQ6cCdnsabAOCjWNL86mfxId/VyQ2reAIxre3J+vqbYwhGlDrmlWfMHZaFM3CN6gwpVeJFRf2SvzV+phh923tORNQ7xVUEWR0MBMtKdaL6OWctOvOk1BSV2zZJKYB1875uvdEIWXvXIBLNNDl7eXvk/ACRC0OnCChPCIMCkzl7neHUQMaUI4bRFuVHDOI8N/CvaDxE2PRU5w7+o8uFI7CNIpJhQ61CoTv2kpKOvYROpGh5GwMBrHITYxhyGe545ShfDOWmjilb+7e2gja0ILxvZODDj2NPz8shbdmNiM/de0K5TKyWELT0RRkHyhwu4Rz6PLvWtsY5mB9xADvajW8HBL+RPuZQpRU71V8QU+aCpC8OQV22o+vaOoR8cNMmAik09WYrC97eOIK/sMdo8+to0Ju3yPDjaQP/mViE8vRq5Pwh2eTcaTmwUU5+Xq9oOMZaVcE6m9PkKo3eNxPeBYX5JvVITv7QPHxr22Y9Ohcj2HBGLJ8Ov2l SNtcnfcT FedyxfGsMgZbhuUJ/MYZ/luIQBAcuydWesv8zR2F7KiFvzlHp4idJUBtcEq2Wdc7gWtJG4ByKZNUusrxKXAkpg3WW9zrzl3q2REz04sASoV3QVYLKGv2vYgJU314FH8yz2GOHHZgsDZ6EcEBTXUXpAOIjK5/F+TAnLohqiUO9RHK4Av9taFxdYhCeEymE9EO2jyU05Gaio4MA+eYIfrNpWehr0RSy22JqhDv3MlQrbwL4dNffSqDSDDEVIPoH+E3OHomltNJQzeOZUyeHX6Zh7hNxvc1bWYB19jrXH6qmaD9nhnLBZpbEEDxIlEz8sMAB+P492CzEtKnPdhEepdiaqgtgjBxNwKYWreDGGIgYtYNvSl+0Upx2Nthc8yHq2848wElBkVYKo63bvoAHt5qrEotyL0gGCPWe4Sihb3oD994e/lDrZIDYwEXkkuhEV8S6r77611D1DExlXL3hsjmENNye42EWQrc7zJAt5awVG7/MCXu4gWiB5HR0Xc949OAjp4wG5ny92QDVYGjeda2rtANxQg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000109, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Fixes: 22d407b164ff ("lib: add allocation tagging support for memory allocation profiling") Signed-off-by: Yu Zhao Acked-by: Suren Baghdasaryan Acked-by: Muchun Song --- include/linux/alloc_tag.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h index 8c61ccd161ba..896491d9ebe8 100644 --- a/include/linux/alloc_tag.h +++ b/include/linux/alloc_tag.h @@ -70,7 +70,7 @@ static inline struct alloc_tag *ct_to_alloc_tag(struct codetag *ct) /* * When percpu variables are required to be defined as weak, static percpu * variables can't be used inside a function (see comments for DECLARE_PER_CPU_SECTION). - * Instead we will accound all module allocations to a single counter. + * Instead we will account all module allocations to a single counter. */ DECLARE_PER_CPU(struct alloc_tag_counters, _shared_alloc_tag); From patchwork Tue Sep 3 21:36:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13789329 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8680CD3440 for ; Tue, 3 Sep 2024 21:37:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0CC718D016E; Tue, 3 Sep 2024 17:37:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 07B0C8D01E4; Tue, 3 Sep 2024 17:37:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0C238D016E; Tue, 3 Sep 2024 17:37:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B48168D01E4 for ; Tue, 3 Sep 2024 17:37:00 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 5E6CB80955 for ; Tue, 3 Sep 2024 21:37:00 +0000 (UTC) X-FDA: 82524737400.29.D4711F3 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf23.hostedemail.com (Postfix) with ESMTP id 8326D14000B for ; Tue, 3 Sep 2024 21:36:58 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=b7C+dYRz; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf23.hostedemail.com: domain of 3eYHXZgYKCBgMIN5yC4CC492.0CA96BIL-AA8Jy08.CF4@flex--yuzhao.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3eYHXZgYKCBgMIN5yC4CC492.0CA96BIL-AA8Jy08.CF4@flex--yuzhao.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725399348; a=rsa-sha256; cv=none; b=A0c3lsrzuNTbbGMr4i6cFqu0+LoSNl+mOnkNLKJu1/yYh7j/noKPNmFIIFrMR8zNmJv4Cs HGRhtdSb6VlQzdcS/aayKg3SCh71AnBeWAlC23z33K8EhRnbwF/RqOb5aXNcxR5XGQwddK fZLhgmyKpnOznTJbG1WX9cmUfLRl5bA= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=b7C+dYRz; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf23.hostedemail.com: domain of 3eYHXZgYKCBgMIN5yC4CC492.0CA96BIL-AA8Jy08.CF4@flex--yuzhao.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3eYHXZgYKCBgMIN5yC4CC492.0CA96BIL-AA8Jy08.CF4@flex--yuzhao.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725399348; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Z/7NuEBRfCNoI3kJSS18ZDURR7hz7WK5gt1vcDxkEbA=; b=rdzL7g3HA/+xJW6JWDYnIQRrA24pwVVKAnFR174RoNQDoGUHxRIGcPT5V7SZ2nRV95uRGa X5uM0JFQJrdESI6DdIsN4EZN/QeVxaFZmZM3JteDxCOmIDDXFpkvu5lm6jp30RCaWVaNI9 K69LrzNCUyIMafy7/lQrizddMjjonfg= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6d54ab222fcso2929967b3.1 for ; Tue, 03 Sep 2024 14:36:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725399417; x=1726004217; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Z/7NuEBRfCNoI3kJSS18ZDURR7hz7WK5gt1vcDxkEbA=; b=b7C+dYRzNJYHIGrGFlipjry5g0CSBWgeo1hvUTTilmpVgyI1sceMW4ilaBbEz6mYEU /PurP7aHN4IF6/xw1J/K13jYU12SHUn8y+Ki8nnQIt0mCKmvEs7dTOcK7HlO0D2CTav9 yIVlycA0Ty07AXKppsoRrmEecRlJwf1sVhDH8H/XuXrBbzCf14PUaaQ01FROgZQVhQgE c8pNVL7v4owY+veWXKAVYjrneuSgtrEmGIhXi7635/vmoZeoRyCFBAYrQWrCbFEYHJk/ yOjWhdEVMLtCRr1fnvu+DnIkOQio8IeLCX+0i8Td/GRgyK15HlrOc698J0KpFkHBDBL+ E5Gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725399417; x=1726004217; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Z/7NuEBRfCNoI3kJSS18ZDURR7hz7WK5gt1vcDxkEbA=; b=Cmmg0XvfnrauxheZwatwK++Ao3GM9k3cr2Rlpy61TS2YaOcjZkqZGloJrU5++T+VMt 3u5gsdIZmtEoecBI9uULndGnfv4M5174ijLXWQs7BtJErC8g+x0kdH3v2Md2VG3+Uudc PFmaMi3n9MFK4EocNiZ5it3ZHNtCvVP+b6TXKvu6YdX6FkSjTG779s7+HeoYP/FRM0yP wnTkQVRFQrqBrf8tI1FAvAbfRKK5Wcn24/ZTtJuvat6p1DTZ/WDBRWRu4cVm+tAh6MLa v+muI83K5DRZam9VK2svBZMStVhJ8FxmXLv+Vy1MBvFBb7/D0RooY/coCU67PRvm3jf2 uFyA== X-Forwarded-Encrypted: i=1; AJvYcCWyWSTPLaF6NyAZc7WpaOgtvQgF5bgArLLIemmyfnKhNAtY8BIUPXh/YxigwaHIxLT1OKUmqJNRBQ==@kvack.org X-Gm-Message-State: AOJu0YzuszWz09SLWVgji1+tCkMAexo4d9QY5VdB9LkcLEP3FJuRVix3 iEchnA2H0xS8B0/DfV/lqdBbu2ObVpQ85cRAUDNdpH9K8SOK5qz9O+6IAdi9hSn5Zcb/Gb8IqPY iHQ== X-Google-Smtp-Source: AGHT+IGGSeNgfu3VxAiHe7OfPaoKscE8qEOTQ+EoyIisWRyArXESs2P4m407AuhgVZ//yoQWwR8kUho664U= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:bb58:3d85:3f9c:c6c6]) (user=yuzhao job=sendgmr) by 2002:a25:8592:0:b0:e0e:c9bc:3206 with SMTP id 3f1490d57ef6-e1d0e78ad12mr315276.5.1725399417227; Tue, 03 Sep 2024 14:36:57 -0700 (PDT) Date: Tue, 3 Sep 2024 15:36:48 -0600 In-Reply-To: <20240903213649.3566695-1-yuzhao@google.com> Mime-Version: 1.0 References: <20240903213649.3566695-1-yuzhao@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240903213649.3566695-2-yuzhao@google.com> Subject: [PATCH mm-unstable v1 2/3] mm/codetag: fix pgalloc_tag_split() From: Yu Zhao To: Andrew Morton , Kent Overstreet , Suren Baghdasaryan Cc: Muchun Song , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao X-Rspamd-Queue-Id: 8326D14000B X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: d96cugr17ww6mrs3do8mx7s8frk8hjkh X-HE-Tag: 1725399418-778737 X-HE-Meta: U2FsdGVkX18n9B9ZpQHfAFWjahz32c9a1CFLzpm+KY3he1q+H58jlLJNeL3NY8Nyiv0bcWm2ivPQQyPtiIuF2Ac2kCanuk/j9r5ibZpXuL6RlBq41AOwzGTf734bXZGudumcl8+veCZTVSpDGMh+L94+VbRtQj5qvHpjgdxYu1PlGpt33M5iNOwJeoNU0rm15Bx7TngFtu4tLqrhBiOR5jPw1HRurItLi1FAo9dGF4S1S4dO72gavi+ssK0WAeZBPNYxADT/PCFmVrZoKXxBsD69iBxYAdEo2VJ/loxQSQo24+8RT6UvbnKY1D9656x+82unVd50XQFK/+QbqATthBhx1HtmEIZ/sctAW9WGh1tI+PZ19mbLCBKosWfgaq0DdRHzj9n+Hfm1lgqGnmU5ZMUZPFMydPkJkqRtb0Ek80CLVOqojX8UNhGtbWvn9OA2j0wWgbEENUHpcU9012UoIZ2jN1UDwSwQvK4Ut6gG/HaRvaPbEWCNPS7GCKnW0rgmR/bPVWo1jXR5Vw3g8FacptwibHhkaoxHLWyeCKyuq1rMsnW38cndKOxF3WDRSKpUaWVnZvNqrAsGEREYUnVcZ8O6nthjc2fmUYA9CqoG8eswh2Ks6XmVGbtRGOYRZrLZH66rjTvwChnbWSPgJfATtO+s+aJy5eOw+UNkHj5AUISqWHYWSs8GFHQLsVra6K1nSaFnnZVrOJihRpG0As8Y+rf0MMponL8wPsX8fW1qL3IkOXuJ8l86epQz9VTgVlQ4H+1WeEDAf/DvlFHV8B7aCj6Bwz9UmyFju2VS0Mba13wX+sHxWpRxCwtDTGerjbbs4jXVVbsgyWp9wt2MgNstR6x0PN74+KSeNYC/CmoAxWxzHkNMQt00quOVoNsMZ3U/mnZz19sw1Dz40N4gAfznbtHyDPhGM2u6SXj1jzNwG1HiAgpoiyCAPjdpG1giOHl0/fUPO6L+GQL1fBKvb4K aCda+AIG DbeGqi2QIh7n7ibbNRGTHvK6CQ1bCtDFSsMDr4mKlqTCpbp66zbaAkVzrVZCoU5f1lahH8GSrjJybjlf8b8Xxyogr2fAJGMkbHW5Xz974Gaa8B5cfA+c8LkhaqSClaqrh4b50Csy5GgmninZjC/zFCtnANsbteMzZChLhBJHm0/YIqcZBIMtvYXRzZr84O6J0ukPy5LXcVh2v1hiQug9wl40QFhgcLfLPAw5ePwfrLi7qNZxWTwo1hKLNlj/zODCbIXVqnrouQp/KlVFyHTafErhJ6pI3fihCK7dRmaBEk89p55bVYjhUsjgCSSNt2U/Vr6oGL5kkCe/dzpABHzc69mWt16P0sSRPxXcLXMcvegCH/r4R48jrH2WwY+/x/kntiAtafAv4vlOQphkyEZnVDQwV3DGCEBopyfYQgwRYci81c5Cicwl4bYmAYOPy89Pxfr7oZPrDWmUTe3Dv1p7+54W/ko+KQuYAzr+BKBUzIPGEQub6k1qTKS6RYO/FJQATJ7evZTtCvsrVRPHa6D7mN8zM+LbMjedHh4BfmWccojD/eBIb5ooWHarJuvtdnDDt6wtty98hethZiiirEFPFR+MjRQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Only tag the new head pages when splitting one large folio to multiple ones of a lower order. Tagging tail pages can cause imbalanced "calls" counters, since only head pages are untagged by pgalloc_tag_sub() and reference counts on tail pages are leaked, e.g., # echo 2048kB >/sys/kernel/mm/hugepages/hugepages-1048576kB/demote_size # echo 700 >/sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages # time echo 700 >/sys/kernel/mm/hugepages/hugepages-1048576kB/demote # grep alloc_gigantic_folio /proc/allocinfo Before this patch: 0 549427200 mm/hugetlb.c:1549 func:alloc_gigantic_folio real 0m2.057s user 0m0.000s sys 0m2.051s After this patch: 0 0 mm/hugetlb.c:1549 func:alloc_gigantic_folio real 0m1.711s user 0m0.000s sys 0m1.704s Not tagging tail pages also improves the splitting time, e.g., by about 15% when demoting 1GB hugeTLB folios to 2MB ones, as shown above. Fixes: be25d1d4e822 ("mm: create new codetag references during page splitting") Signed-off-by: Yu Zhao Acked-by: Suren Baghdasaryan --- include/linux/mm.h | 30 ++++++++++++++++++++++++++++++ include/linux/pgalloc_tag.h | 31 ------------------------------- mm/huge_memory.c | 2 +- mm/hugetlb.c | 2 +- mm/page_alloc.c | 4 ++-- 5 files changed, 34 insertions(+), 35 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index b31d4bdd65ad..a07e93adb8ad 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4137,4 +4137,34 @@ void vma_pgtable_walk_end(struct vm_area_struct *vma); int reserve_mem_find_by_name(const char *name, phys_addr_t *start, phys_addr_t *size); +#ifdef CONFIG_MEM_ALLOC_PROFILING +static inline void pgalloc_tag_split(struct folio *folio, int old_order, int new_order) +{ + int i; + struct alloc_tag *tag; + unsigned int nr_pages = 1 << new_order; + + if (!mem_alloc_profiling_enabled()) + return; + + tag = pgalloc_tag_get(&folio->page); + if (!tag) + return; + + for (i = nr_pages; i < (1 << old_order); i += nr_pages) { + union codetag_ref *ref = get_page_tag_ref(folio_page(folio, i)); + + if (ref) { + /* Set new reference to point to the original tag */ + alloc_tag_ref_set(ref, tag); + put_page_tag_ref(ref); + } + } +} +#else /* !CONFIG_MEM_ALLOC_PROFILING */ +static inline void pgalloc_tag_split(struct folio *folio, int old_order, int new_order) +{ +} +#endif /* CONFIG_MEM_ALLOC_PROFILING */ + #endif /* _LINUX_MM_H */ diff --git a/include/linux/pgalloc_tag.h b/include/linux/pgalloc_tag.h index 207f0c83c8e9..59a3deb792a8 100644 --- a/include/linux/pgalloc_tag.h +++ b/include/linux/pgalloc_tag.h @@ -80,36 +80,6 @@ static inline void pgalloc_tag_sub(struct page *page, unsigned int nr) } } -static inline void pgalloc_tag_split(struct page *page, unsigned int nr) -{ - int i; - struct page_ext *first_page_ext; - struct page_ext *page_ext; - union codetag_ref *ref; - struct alloc_tag *tag; - - if (!mem_alloc_profiling_enabled()) - return; - - first_page_ext = page_ext = page_ext_get(page); - if (unlikely(!page_ext)) - return; - - ref = codetag_ref_from_page_ext(page_ext); - if (!ref->ct) - goto out; - - tag = ct_to_alloc_tag(ref->ct); - page_ext = page_ext_next(page_ext); - for (i = 1; i < nr; i++) { - /* Set new reference to point to the original tag */ - alloc_tag_ref_set(codetag_ref_from_page_ext(page_ext), tag); - page_ext = page_ext_next(page_ext); - } -out: - page_ext_put(first_page_ext); -} - static inline struct alloc_tag *pgalloc_tag_get(struct page *page) { struct alloc_tag *tag = NULL; @@ -142,7 +112,6 @@ static inline void clear_page_tag_ref(struct page *page) {} static inline void pgalloc_tag_add(struct page *page, struct task_struct *task, unsigned int nr) {} static inline void pgalloc_tag_sub(struct page *page, unsigned int nr) {} -static inline void pgalloc_tag_split(struct page *page, unsigned int nr) {} static inline struct alloc_tag *pgalloc_tag_get(struct page *page) { return NULL; } static inline void pgalloc_tag_sub_pages(struct alloc_tag *tag, unsigned int nr) {} diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 0993dfe9ae94..aa8a4c938ba9 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3244,7 +3244,7 @@ static void __split_huge_page(struct page *page, struct list_head *list, /* Caller disabled irqs, so they are still disabled here */ split_page_owner(head, order, new_order); - pgalloc_tag_split(head, 1 << order); + pgalloc_tag_split(folio, order, new_order); /* See comment in __split_huge_page_tail() */ if (folio_test_anon(folio)) { diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 3faf5aad142d..a8624c07d8bf 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3778,7 +3778,7 @@ static long demote_free_hugetlb_folios(struct hstate *src, struct hstate *dst, list_del(&folio->lru); split_page_owner(&folio->page, huge_page_order(src), huge_page_order(dst)); - pgalloc_tag_split(&folio->page, 1 << huge_page_order(src)); + pgalloc_tag_split(folio, huge_page_order(src), huge_page_order(dst)); for (i = 0; i < pages_per_huge_page(src); i += pages_per_huge_page(dst)) { struct page *page = folio_page(folio, i); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index c242d61fc4fd..13ce8e8899ed 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2822,7 +2822,7 @@ void split_page(struct page *page, unsigned int order) for (i = 1; i < (1 << order); i++) set_page_refcounted(page + i); split_page_owner(page, order, 0); - pgalloc_tag_split(page, 1 << order); + pgalloc_tag_split(page_folio(page), order, 0); split_page_memcg(page, order, 0); } EXPORT_SYMBOL_GPL(split_page); @@ -5020,7 +5020,7 @@ static void *make_alloc_exact(unsigned long addr, unsigned int order, struct page *last = page + nr; split_page_owner(page, order, 0); - pgalloc_tag_split(page, 1 << order); + pgalloc_tag_split(page_folio(page), order, 0); split_page_memcg(page, order, 0); while (page < --last) set_page_refcounted(last); From patchwork Tue Sep 3 21:36:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13789330 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C80BCD3440 for ; Tue, 3 Sep 2024 21:37:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B29368D01E5; Tue, 3 Sep 2024 17:37:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AD9F08D01E4; Tue, 3 Sep 2024 17:37:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92AC28D01E5; Tue, 3 Sep 2024 17:37:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6EE498D01E4 for ; Tue, 3 Sep 2024 17:37:03 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2756C80719 for ; Tue, 3 Sep 2024 21:37:03 +0000 (UTC) X-FDA: 82524737526.03.7B8077A Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf08.hostedemail.com (Postfix) with ESMTP id 57DBB16000D for ; Tue, 3 Sep 2024 21:37:01 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=bF6jsTcK; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf08.hostedemail.com: domain of 3fIHXZgYKCBsPLQ81F7FF7C5.3FDC9ELO-DDBM13B.FI7@flex--yuzhao.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3fIHXZgYKCBsPLQ81F7FF7C5.3FDC9ELO-DDBM13B.FI7@flex--yuzhao.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725399315; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7lRbWVNLaQ1IZX12itRVMEELUogajjWtfSLYwZb6BSc=; b=BWfyl+XiVZB5MDsGfn8ZqL42Cclr6tFdPX76mj3X1odm6aj+zkiSd3AvZ30+Z1BlUCz73H YVHVYJtST8CmUCw+iISlDlSaDMlAzLG8IAky7FE35FLjayPMNlhiyUKuNUxTIYFQFPkJXz IAzJp5aVXZIOhJiFN9QCP4kFq+Gdkss= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725399315; a=rsa-sha256; cv=none; b=nc7flOMC2PudIMIrO79A1LjZfulDTIIkhx8mhq64e5ugb1UUWy5nnZsO0RZSeySKYLlrjf N4zap5MqlA/CQQgGRmb57MREcRKscfC0+oxv++oLTtYY785G3SZJ/zwyc/TNv//tX8oq/y h8iLSnBnTL4Wxv3hYKnf41ltaB5AiOA= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=bF6jsTcK; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf08.hostedemail.com: domain of 3fIHXZgYKCBsPLQ81F7FF7C5.3FDC9ELO-DDBM13B.FI7@flex--yuzhao.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3fIHXZgYKCBsPLQ81F7FF7C5.3FDC9ELO-DDBM13B.FI7@flex--yuzhao.bounces.google.com Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e116c697904so11715490276.1 for ; Tue, 03 Sep 2024 14:37:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725399420; x=1726004220; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=7lRbWVNLaQ1IZX12itRVMEELUogajjWtfSLYwZb6BSc=; b=bF6jsTcKdBKjA+fNgi4VPdFAVateGaQCdpkUzVC2eiLzyjgRPVLjlzXwbQAZgww79Q lDLagmpSGcCvJR+m4PG7vHIHxerPTekWSbP9x9TvCpSdjZGte1SSoPEAOJ+QGcMUQ3kA 1TusYN5/l7+yTavHh1Tmcp53+jg8/Fh5kNcq4loYtQl8FekBWLjI/Ugv62+KElO7sepo BawCvCJwQvJQOEDgGftONH7Fdc6X/B0VE5CJjcwkytQgurlCWAdoLDsowWR5RWPjATka ZWDd3CSO7BUjgLi1dwemrSPNEfoVnbIgDboMCBMTxrVAsMNqqNlEsIgJAh522z0uDcY3 1/NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725399420; x=1726004220; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7lRbWVNLaQ1IZX12itRVMEELUogajjWtfSLYwZb6BSc=; b=kF+yJUP7FZUTtP6L7s0fLYmr01PDNW+B8/jsGrUZNrThjdPxY+ldu6Db//4QzsiR8y H2ZT/guukY19vVbUZ8IvCNG+sezAmakYD0Wk9+MeLj8Zixa57w2CYGJmc64NX0de/QkB 7O3oLAFbq5i5pCZYnfM83wUbrUEaCMhgtkQeBvUqqFZoEPYmzLBxOjF8/EWKdC409wtN oMgQvba847INpcJLRNw4NIM73kNpJyOjvVNoQcEr13hnqk4g/SHAHBaRF63WbfSqvEbU qmww1FfU3U4ATUmgT/ynlwtVvXP55fWWNhEkw8x1a7qzJdaPNQDHxdnLA6k3K+EwYnOe GGoQ== X-Forwarded-Encrypted: i=1; AJvYcCVeQy9mFFabPNND1fARvCHSXBojnP+hrOnNGZRoZdiD7RoJW2I8GwX5QILRKVv2AJBCVFDUtpgQCA==@kvack.org X-Gm-Message-State: AOJu0YwGgLXTu8bpftlYue9IgsvonsSnajAit8FzAZoLdm1BdS+5Rqjc pxdFX0oNXsQsuhAV3QJouWs3p+9CJZ+GxjUR+yiKD3NEJUimd7EJDXREPgMZt+oWNB6vE4yGKLt NAA== X-Google-Smtp-Source: AGHT+IFKMuOj0a8gCYr8Reqa7ik1LwLwagBRCBwi0mhu448Z4IqlPF9etRzCqLqKmBASiXsj6Jc8sLxDbM4= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:bb58:3d85:3f9c:c6c6]) (user=yuzhao job=sendgmr) by 2002:a25:264d:0:b0:e11:7246:963b with SMTP id 3f1490d57ef6-e1a79fb492dmr639799276.1.1725399420440; Tue, 03 Sep 2024 14:37:00 -0700 (PDT) Date: Tue, 3 Sep 2024 15:36:49 -0600 In-Reply-To: <20240903213649.3566695-1-yuzhao@google.com> Mime-Version: 1.0 References: <20240903213649.3566695-1-yuzhao@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240903213649.3566695-3-yuzhao@google.com> Subject: [PATCH mm-unstable v1 3/3] mm/codetag: add pgalloc_tag_copy() From: Yu Zhao To: Andrew Morton , Kent Overstreet , Suren Baghdasaryan Cc: Muchun Song , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 57DBB16000D X-Stat-Signature: rmr5hqokpawpsdfe5twz3i6kgizk9tsk X-Rspam-User: X-HE-Tag: 1725399421-144028 X-HE-Meta: U2FsdGVkX18oH4liRwZn4m/tAdKhIM1tWHCBKAhA/8FOIdQhz4+77aMSJ/eDHd+GeiVqzAQjbjbmaE+sgvQU+ZgSkfIj8DtlqEceNjbMsk7nA5AiELLjd7wUuoySZaOp1OBwbAqeVIR+NMqEPUgPDv/mxdz7mjBVUG9RlCivTXrWC3tpBdTYKrqU9Pb2rUFX729Sb9nT0BeRjB/PUAOkGu+ePhpYac8iG7xmz/pkEa+LI3JhSsazAypbJ7iamHib2tlh1NdmakKoQPrfjx6ppiMDFdi+NKY/l++3jhYLP14GTzm6W4j2HgG5Hj/2Qnb1hBQqWgHTA9OhHRNsxEEAxJiwWrVMhovNlnd/mlcaKIp5gN3HVmWbyEfWNN/kiFNIiPwoaEDvY2ownfEJy8HopXgAhXz2r/F4N8+KtkDUB+LfREco/yH0hH2WjtEgwYPYHIAMR+U/gu0utwkmJBJG9dI14Z2Xe4QB8KM/fXpBpewscrmB7oWNlaEixwz050L9nHLa/D6WhfK9ftV6ug3eSBtZPC43d9A5SLidi9c5Yd9EKtPN7aRSXPA/1YAMwpe57SbRNGs19cpr5X5AJvGWCfaYUqJ/ygvEYO/8qzyNDD9z/GUJI0i/iF0XbrK4o2UfFIjcUrnMYiEUrhY/cpJfobSpMSJNyuU1B9nZRDq+lCuB3eXI0mfwnU2rddqdzVJ5AWT6QvDaxTdcRGxjqFq0wNNGm9qcdx5kRjxCtoNcJG+2zbG2GKih3axQZvmrAzQMT9/OSLIwKhnX7RUtNMuj8Ma7PWYCTUmy51I9GM0Sgx9/ny1XoKNdQWo6dPq1HZJZLlQVrriq+Lxjdicu3n7sbeIim9XzEOsZ1zAP5w5ybo68VDOLD8uBMIsqtey+YH7f/ruRTWBfBb5nSkSAoL6dk6wFnHqAe/8TNwaPAWzqknI5yyrnAnt66enMK4xAo7viZ089TYfGNkgC0Uwb/lo Clo4cCm7 8oKfUv4x4PKPdh+2pJrEXiaSvJxtpf5CfnNvqlVtMRb70g99zZIRSfA4ADlJu7RK7hAwr8GbYwrBGigJR8clRFiDtC8XSnu/VEZA4Z6dtmYDZT6Stmm2/6pulsckzPccIok9GYfjCe7lAeENAAY36Oj3y5K1BFT3wwXbnCu888I7rLTaGOAHkFztGCzri+O6UFmp6HLeB3QSi9WqOah1mXpD2AfRr2oF4KsGj74trhutpK4v57CmRP733bJbJRoNb1RDztg3R4tMLdMVZOYbTW29IJzanPhbCdMRD6CS7TOdCIHfwmiYASQG3arsF00+1xxDGdlham0Cmwl0rMdLnvk2+G0H3ARaOxArRutBy8u4+TomIhT+ZwhpfsJbdURCYWuSF3phC+eSEFOdlnAwUU9IHevwtryjxI2rWZgPNBFlFFCs7ECO85JjYBDpwH+ITCf0UlR7bClzxnP2JzmJ+P0f7q2zaBKQPGwoP0/U/3CiK7HLYhOV67P2KHSqsrQOTsROovATYBA+SRIyfYGOytCpPBEPxUHx9wzzPaSCOSzgsE+o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add pgalloc_tag_copy() to transfer the codetag from the old folio to the new one during migration. This makes original allocation sites persist cross migration rather than lump into compaction_alloc, e.g., # echo 1 >/proc/sys/vm/compact_memory # grep compaction_alloc /proc/allocinfo Before this patch: 132968448 32463 mm/compaction.c:1880 func:compaction_alloc After this patch: 0 0 mm/compaction.c:1880 func:compaction_alloc Signed-off-by: Yu Zhao Acked-by: Suren Baghdasaryan --- include/linux/alloc_tag.h | 24 ++++++++++-------------- include/linux/mm.h | 25 +++++++++++++++++++++++++ mm/migrate.c | 1 + 3 files changed, 36 insertions(+), 14 deletions(-) diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h index 896491d9ebe8..1f0a9ff23a2c 100644 --- a/include/linux/alloc_tag.h +++ b/include/linux/alloc_tag.h @@ -137,7 +137,16 @@ static inline void alloc_tag_sub_check(union codetag_ref *ref) {} /* Caller should verify both ref and tag to be valid */ static inline void __alloc_tag_ref_set(union codetag_ref *ref, struct alloc_tag *tag) { + alloc_tag_add_check(ref, tag); + if (!ref || !tag) + return; + ref->ct = &tag->ct; +} + +static inline void alloc_tag_ref_set(union codetag_ref *ref, struct alloc_tag *tag) +{ + __alloc_tag_ref_set(ref, tag); /* * We need in increment the call counter every time we have a new * allocation or when we split a large allocation into smaller ones. @@ -147,22 +156,9 @@ static inline void __alloc_tag_ref_set(union codetag_ref *ref, struct alloc_tag this_cpu_inc(tag->counters->calls); } -static inline void alloc_tag_ref_set(union codetag_ref *ref, struct alloc_tag *tag) -{ - alloc_tag_add_check(ref, tag); - if (!ref || !tag) - return; - - __alloc_tag_ref_set(ref, tag); -} - static inline void alloc_tag_add(union codetag_ref *ref, struct alloc_tag *tag, size_t bytes) { - alloc_tag_add_check(ref, tag); - if (!ref || !tag) - return; - - __alloc_tag_ref_set(ref, tag); + alloc_tag_ref_set(ref, tag); this_cpu_add(tag->counters->bytes, bytes); } diff --git a/include/linux/mm.h b/include/linux/mm.h index a07e93adb8ad..1b98d843a5e9 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4161,10 +4161,35 @@ static inline void pgalloc_tag_split(struct folio *folio, int old_order, int new } } } + +static inline void pgalloc_tag_copy(struct folio *new, struct folio *old) +{ + struct alloc_tag *tag; + union codetag_ref *ref; + + tag = pgalloc_tag_get(&old->page); + if (!tag) + return; + + ref = get_page_tag_ref(&new->page); + if (!ref) + return; + + /* Clear the old ref to the original allocation site. */ + clear_page_tag_ref(&old->page); + /* Decrement the counters of the tag on get_new_folio. */ + alloc_tag_sub(ref, folio_nr_pages(new)); + __alloc_tag_ref_set(ref, tag); + put_page_tag_ref(ref); +} #else /* !CONFIG_MEM_ALLOC_PROFILING */ static inline void pgalloc_tag_split(struct folio *folio, int old_order, int new_order) { } + +static inline void pgalloc_tag_copy(struct folio *new, struct folio *old) +{ +} #endif /* CONFIG_MEM_ALLOC_PROFILING */ #endif /* _LINUX_MM_H */ diff --git a/mm/migrate.c b/mm/migrate.c index 35cc9d35064b..0b24021d5fee 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -750,6 +750,7 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio) folio_set_readahead(newfolio); folio_copy_owner(newfolio, folio); + pgalloc_tag_copy(newfolio, folio); mem_cgroup_migrate(folio, newfolio); }