From patchwork Mon Sep 2 04:41:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13786703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE7A8C54FC6 for ; Mon, 2 Sep 2024 04:41:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B0A28D0070; Mon, 2 Sep 2024 00:41:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23D2F8D0065; Mon, 2 Sep 2024 00:41:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 067028D0070; Mon, 2 Sep 2024 00:41:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id DA8178D0065 for ; Mon, 2 Sep 2024 00:41:41 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 59121141661 for ; Mon, 2 Sep 2024 04:41:41 +0000 (UTC) X-FDA: 82518550002.25.1CE71FF Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf11.hostedemail.com (Postfix) with ESMTP id 8A85640014 for ; Mon, 2 Sep 2024 04:41:39 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=MLPguSra; spf=pass (imf11.hostedemail.com: domain of 3AkLVZgYKCBkHJG3C05DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--surenb.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3AkLVZgYKCBkHJG3C05DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725252076; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GtHW3QbMIbL8VkRhZ9UoruDWIR+5Ejb5CjRZSzRSfG0=; b=xABWTpMET1qJTzonvpfus8PrJsX4LbHxWIgoqMbWcvoTmCcFVYnQAfSez6RGeVzv82DLzd 71z5kcmXCAnBu9cVyKwi2kpNGcIk7VufRBzW++Vv5aJy++ytK/rTuZysxnHK7WiYNBSRHM yT+Zi4Mb29VKsqrP+h2Qa91qt355oW4= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=MLPguSra; spf=pass (imf11.hostedemail.com: domain of 3AkLVZgYKCBkHJG3C05DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--surenb.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3AkLVZgYKCBkHJG3C05DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725252076; a=rsa-sha256; cv=none; b=6CZxEb9wiQuv5sKQLS0ZF58idtYa3VrXcW8AlXD67yqt0ZhQD/L24GGuXeAga3RiVMqqc8 roaH52ioOifV/b4xA4Ofubj6xojQHKKjMO7oT5e8y8Nn75FyBvhnop+0M0paFSs0YQmGXt q6a+s+wqGt30VhgJyspedHDwrgQ+0n4= Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e1a7aedfff1so5015706276.3 for ; Sun, 01 Sep 2024 21:41:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725252099; x=1725856899; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=GtHW3QbMIbL8VkRhZ9UoruDWIR+5Ejb5CjRZSzRSfG0=; b=MLPguSrarxhYBXDPZXbCtavzdrTCz5Uf3AawaveyfWxmXDdpZ7kBh4QH+alLIZISHJ rIE/yyK3ShqIdw/ASVU7y/ispUO1PRihdbd04FRHm6thuW7zR8VJiN6g9MXS2brcEfsz uKKPaT0O3z4Xu9xYzuBmZM2cDqfRUk/2gO4KShQMvk+kODWCUMCe2bdSf9D2wEAojtUT lJTvIMg5LLvWqVObzV3HtB5OLPJRMoUTgIVw/3ARO1q2+hOMIsQTqkLMrkV9DTWOAPq1 Z2AnrvHQPU+Q51/p2hicaMRImI5LbErug5amFA3G8N7y/wZfr67fWdLteK4kV0a/ZHYm ZysA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725252099; x=1725856899; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GtHW3QbMIbL8VkRhZ9UoruDWIR+5Ejb5CjRZSzRSfG0=; b=YHpZyOcUIMLG0bFsxIpRpXK1PW8mKRDa2kfCzO7ubQda0qVaMzt1kaQC7jH0NY9M37 9wbWdqJAsau1v8N0b/tnGxi8zJ4xFccYLMq3nbFt0E8ZrJry3inZzB+pWDCa3XtFXdlZ QN9ErOta4OUuFvzbXHfS0LgH3XV/fwrBYf6QtdpXaF75wNEJjLNp8oBpQSUyB7mEs26h Uz6uU27fqIB+FF3u0B55L7zC8A6nmn7QPw3GE+MyQElBL8EwOIQkdhMA4rwosk+4i2qR 7L6AeXCNUvkazepVxwJZjQbwxOgGuRLL+jb1qnP8a2r+NvwccSMqRXcaUsScbvt+lIkh 3khA== X-Forwarded-Encrypted: i=1; AJvYcCUYwlnhBrxb2GZvvx7BA7nB7DrFPf89cSxB4QBddSfBdZiDyTsAqyNdaySH8LlRMlkmvD0W3m4yWg==@kvack.org X-Gm-Message-State: AOJu0YyguWxZTnoCjD0fATFpzxVsynAcyXoGlRT8ipXkFPPH3EfZOt46 pDRzrvPpGR6e/GZKzTphf+zGbBOkg/vriS+q17g0+4oQcGrncb7wQFE1zkBamPzqq5drnIrg650 FzA== X-Google-Smtp-Source: AGHT+IGHUG44Vos/pxlFWbZNjhZtZ0OaAmv4KSGmEpEw821HAQXG3t5pT+l+pURZYG9PiSEp/j/9qgeoB6s= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:7343:ecd2:aed:5b8f]) (user=surenb job=sendgmr) by 2002:a25:7e46:0:b0:e0b:ab63:b9c8 with SMTP id 3f1490d57ef6-e1a7a3dbab3mr14893276.11.1725252098507; Sun, 01 Sep 2024 21:41:38 -0700 (PDT) Date: Sun, 1 Sep 2024 21:41:25 -0700 In-Reply-To: <20240902044128.664075-1-surenb@google.com> Mime-Version: 1.0 References: <20240902044128.664075-1-surenb@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240902044128.664075-4-surenb@google.com> Subject: [PATCH v2 3/6] alloc_tag: eliminate alloc_tag_ref_set From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: kent.overstreet@linux.dev, corbet@lwn.net, arnd@arndb.de, mcgrof@kernel.org, rppt@kernel.org, paulmck@kernel.org, thuth@redhat.com, tglx@linutronix.de, bp@alien8.de, xiongwei.song@windriver.com, ardb@kernel.org, david@redhat.com, vbabka@suse.cz, mhocko@suse.com, hannes@cmpxchg.org, roman.gushchin@linux.dev, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, pasha.tatashin@soleen.com, souravpanda@google.com, keescook@chromium.org, dennis@kernel.org, jhubbard@nvidia.com, yuzhao@google.com, vvvvvv@google.com, rostedt@goodmis.org, iamjoonsoo.kim@lge.com, rientjes@google.com, minchan@google.com, kaleshsingh@google.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, kernel-team@android.com, surenb@google.com X-Rspam-User: X-Stat-Signature: pw8jip4diactz1y1wghgknksit8swfis X-Rspamd-Queue-Id: 8A85640014 X-Rspamd-Server: rspam11 X-HE-Tag: 1725252099-519610 X-HE-Meta: U2FsdGVkX19NtDoYpnF22CB5VnimINf2UfRSm00CUrDwsmHssYLvXWUj9zLCycvIHknomND18mTQXrE7xB7VeVZrTSbiWWnpR0Zwe6pCocaW+8mJO7ggoWT73ZlpGko1y14qHi6jfOcgFfQ4Z2y4tizaH+izOefTzc9OrlvR8W5ffo10rRYAf7NY5ORefpvuBDRzhQsQM6IOFtzBEecnR6j8H7NXwCWSBaZTXtKmTEQfgNnlP4aC2RdOE0rlDLq4pAH0bpQJILBPCE3fknts5gIs2urb9SC5bhBBaIRV05nndO4pJTT2u5b+xC/+8LeHr1Q3skmuBPbduJJA9bp5yAts7/icrOZnOKnlYjN9uNaBh5p0z+SfeDyAcdglpmIebhRx3JkEOTj3/FmZUjTIdaLEoUdBBTsLZWGAS2hO0btx+fGtqayo4gghR/V4sv46d0Q+TwSR0QXTWP8knPWKjW4cMOST0DhFZ3+wooVSBES+CEacrBTMpv9JYFgekGqQweiLO7NyHXX+3dx78wB4Ek10gjb/9dWqH6rQ6didYhwgUPODk7niHPQfbOUnwM6JhhyMLCGifjV0WeIpmWLe6BERn6PujRUpiaB/bGMol5YvgPV2h7JYAAJQFDbqoX0v4sW4bqpiyJH1y9eoNLN+t4P/Xh8Zz9DcVazp9fDI4nEGkGrD7aGvmm07bLrX832USjUcqBgIckALyL7SGBGc+iZJnATn/O17Ni+R0ZRSqgS54Ij/tbeBpS+U0NiQ8uRsxpwQrP6s0/Ciq1cKvUsQevPuLhDqmd+ehfxKyNM7AMk5PdBGLScG9o//d/8BKonaIZItzFX5N7OHQbXSTnnNw0ukgixtjI8DC9A/PX/cbqWDcR3gRbH2zp2dkCwG4RnddAKoVEHJB+pCrQJRFsLaMml78QMV0yp1CHpq56y28XtY09aIqlsKx/gDu20M5V60cz7iHp1TOg0KSCqhAHG J/gYT6v5 omngpqW0n6HJPOZwt2hwv6l0evmLw8itgp2LlTbeFg7WqnOjEC0itlj3nlQHU7nqgLI9sVVmm+SFUCJO1xtebqTQqOun9Rv+LWBGzQU8cEj06VrxHkXEcH401PsxjjjNOqDWXk71VLFvQuFYdx69yanxgt95ycWNUl2pWehb0asVzlL48qeVKm6STBcLmv9uwowEOVj13oKkQptgFn2FxU6/9RZa/seFPLswdESuW7pbxKvj6RlomDwJXa9OasUCkRZ6E/bkgbdUhzreaQZWa9jyGvyzyDTUcEGmzlVRJ9T5pdEr4kTuSRCNl2F3DztBut9QryVzXzfqoS+Zu9y1yNXsryoi5iBrq7/54P0GWt4sfVAcDoc1a+IAOT4W5VWegFZkWwlXuRWCAlfxwd+/iKb0vbKZFSOajb4oQXXYYSPEP/Y19AqhrroHAt14sXrrr7l2i0Or+kjkcaiX0+cSLme70URpOuvo/fMeqnA7jRiFubG+qmnYYfpxlYHNzBOWXAgMa6L00EZ0SsxOv/MkNdQcN7bIw964uoqnWCE4wBA1uQZ0XWVWPonOnZnvPUVEqYXYrj9dskEj183ObseZ+YLG7eg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000052, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To simplify further refactoring, open-code the only two callers of alloc_tag_ref_set(). Signed-off-by: Suren Baghdasaryan --- include/linux/alloc_tag.h | 25 ++----------------------- include/linux/pgalloc_tag.h | 12 +++++++++++- 2 files changed, 13 insertions(+), 24 deletions(-) diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h index 99cbc7f086ad..21e3098220e3 100644 --- a/include/linux/alloc_tag.h +++ b/include/linux/alloc_tag.h @@ -143,36 +143,15 @@ static inline void alloc_tag_add_check(union codetag_ref *ref, struct alloc_tag static inline void alloc_tag_sub_check(union codetag_ref *ref) {} #endif -/* Caller should verify both ref and tag to be valid */ -static inline void __alloc_tag_ref_set(union codetag_ref *ref, struct alloc_tag *tag) -{ - ref->ct = &tag->ct; - /* - * We need in increment the call counter every time we have a new - * allocation or when we split a large allocation into smaller ones. - * Each new reference for every sub-allocation needs to increment call - * counter because when we free each part the counter will be decremented. - */ - this_cpu_inc(tag->counters->calls); -} - -static inline void alloc_tag_ref_set(union codetag_ref *ref, struct alloc_tag *tag) -{ - alloc_tag_add_check(ref, tag); - if (!ref || !tag) - return; - - __alloc_tag_ref_set(ref, tag); -} - static inline void alloc_tag_add(union codetag_ref *ref, struct alloc_tag *tag, size_t bytes) { alloc_tag_add_check(ref, tag); if (!ref || !tag) return; - __alloc_tag_ref_set(ref, tag); + ref->ct = &tag->ct; this_cpu_add(tag->counters->bytes, bytes); + this_cpu_inc(tag->counters->calls); } static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) diff --git a/include/linux/pgalloc_tag.h b/include/linux/pgalloc_tag.h index 207f0c83c8e9..244a328dff62 100644 --- a/include/linux/pgalloc_tag.h +++ b/include/linux/pgalloc_tag.h @@ -103,7 +103,17 @@ static inline void pgalloc_tag_split(struct page *page, unsigned int nr) page_ext = page_ext_next(page_ext); for (i = 1; i < nr; i++) { /* Set new reference to point to the original tag */ - alloc_tag_ref_set(codetag_ref_from_page_ext(page_ext), tag); + ref = codetag_ref_from_page_ext(page_ext); + alloc_tag_add_check(ref, tag); + if (ref) { + ref->ct = &tag->ct; + /* + * We need in increment the call counter every time we split a + * large allocation into smaller ones because when we free each + * part the counter will be decremented. + */ + this_cpu_inc(tag->counters->calls); + } page_ext = page_ext_next(page_ext); } out: