From patchwork Mon Sep 4 15:08:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uros Bizjak X-Patchwork-Id: 13374087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DED3CC83F2C for ; Mon, 4 Sep 2023 15:09:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 27CE38D0009; Mon, 4 Sep 2023 11:09:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 206438D0001; Mon, 4 Sep 2023 11:09:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 07FF78D0009; Mon, 4 Sep 2023 11:09:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E90088D0001 for ; Mon, 4 Sep 2023 11:09:30 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B7474140736 for ; Mon, 4 Sep 2023 15:09:30 +0000 (UTC) X-FDA: 81199248900.30.B99788B Received: from mail-lj1-f173.google.com (mail-lj1-f173.google.com [209.85.208.173]) by imf12.hostedemail.com (Postfix) with ESMTP id E5BBB4001C for ; Mon, 4 Sep 2023 15:09:28 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=Yi7+amQX; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf12.hostedemail.com: domain of ubizjak@gmail.com designates 209.85.208.173 as permitted sender) smtp.mailfrom=ubizjak@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693840169; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=9pqa6X94cnx/SA5Kco+qqSw4Yn4MifOppIUbtHTsLH8=; b=7yUkOPxU0hVIv17lJl8iA42wUT7tG0K7tW6pk4sHEUSRW8iWAg8EJJN2eZUAB+1WdBF25K to/tIAQGKg50LrM0yJQdJ1rlf41IW4rzuCfRho4JRlR0NiIAq2gsThkkL+0YG0qOtFHlNE fLORD//sHK51OAhN0rvI/8pdJJ9vee0= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=Yi7+amQX; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf12.hostedemail.com: domain of ubizjak@gmail.com designates 209.85.208.173 as permitted sender) smtp.mailfrom=ubizjak@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1693840169; a=rsa-sha256; cv=none; b=IIoodZc4jMRPqMJZBOosu8E2VflDFyt9ANyTN+R5pOF7UngD5PBonlXPJvhiHDfDtAlOqb ekbAzd2VFaumfA53ODZerGl8ia66XsqX72SqTxzwH1MPnvjt5MiNsqzFJ+eQlQDgZNyo+h UBufpU4QNbW3ZAs52P6bG6NY2p0xd0U= Received: by mail-lj1-f173.google.com with SMTP id 38308e7fff4ca-2bcfdadd149so25960741fa.0 for ; Mon, 04 Sep 2023 08:09:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693840167; x=1694444967; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=9pqa6X94cnx/SA5Kco+qqSw4Yn4MifOppIUbtHTsLH8=; b=Yi7+amQXEJorEQQtO0e9FlCMMHdwCwd7JZ08AviP1uumjtxuIMfbk+5gEJHFmW4HSf 43hEQas6A2RWT/HAJxrqHWdFcv5AgNMgvGmwpnWzPRyzHVjfTn0T0PFQK9MMfhAkaYP3 /6uN+pO1TOvK3V2bJfFwie/5CBV60P6lRsmXRXHZeFM3E0Yl2BRStMgD+5GcL7a4UT3M L/skqBTh+wZbotxfoxp13aTXvrFyqAkscBul3ysZxiQbsAX0OsJa6+Qkz6+NuJMoMr9+ s7mlmnYtC+h3yv7ST0OaTRqFjGzgCfS5uf9qgzuYQt9k3d4RCCLg0E90TpqiWhEbPAb5 rhSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693840167; x=1694444967; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9pqa6X94cnx/SA5Kco+qqSw4Yn4MifOppIUbtHTsLH8=; b=WfnSws+qztkaSYZaQ6qLGYzctnWOgUwWJo217sBw9Yap41xlMeE2Ce2Sn9a12o+w/W Gcf56OxvPVjVz89KEnFTiyroKZjnqhOmbXQ/ULH4CZZqAwn0vsUvBcPIWQFzFCNu6INL Y85IjdXDa5fHhZ3txIerjbUe5hYv/y/da3Sh7fOGTZPNOp4abMDD2LMFQv/OU0du3Gvu vvxbXfxQo1T8jj6aKhxkU+k/q+hftNYaIDOmBSO7s5M1Vm4UY9zn6KCDXvpLrpzS2YoL TZkiFHV510WNz6PquSBqz5b1F2bVSPON9ZWm90J6qJ5lTHPeA8IJfI8vObwNLW/c04VQ ENcw== X-Gm-Message-State: AOJu0YzMkBTg4QcnpCVaQfOJSVy4i1oRHFmuBID7fJCNPRFDx/Ar0kGx bQkfhh/L9ugBEdmKw05x8ytA26OYfIEE+A== X-Google-Smtp-Source: AGHT+IFSefigFg6EPsF+g1BaJ26YH6ra4bHnK+bpoPkThpcpOU1PW7RYGE71sdcXjwKweO9K0gD5tg== X-Received: by 2002:a2e:96d2:0:b0:2bd:16e6:e34e with SMTP id d18-20020a2e96d2000000b002bd16e6e34emr7701626ljj.1.1693840166553; Mon, 04 Sep 2023 08:09:26 -0700 (PDT) Received: from localhost.localdomain ([46.248.82.114]) by smtp.gmail.com with ESMTPSA id fx4-20020a170906b74400b009887f4e0291sm6251775ejb.27.2023.09.04.08.09.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Sep 2023 08:09:26 -0700 (PDT) From: Uros Bizjak To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Uros Bizjak , Andrew Morton Subject: [PATCH] mm/vmstat: Use this_cpu_try_cmpxchg in mod_{zone,node}_state Date: Mon, 4 Sep 2023 17:08:49 +0200 Message-ID: <20230904150917.8318-1-ubizjak@gmail.com> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-Rspamd-Queue-Id: E5BBB4001C X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: k864o79gj4t57rfsfnf8yqbkjiwzysiz X-HE-Tag: 1693840168-274943 X-HE-Meta: U2FsdGVkX1/Kvfm5uFAsrh79XCr1+meOBowqeKmnVHHor99hAZWhRZHMF9skT7ItVSUYdd7IvWgEhMpDnNS3bEfAjHPrFq5v2+UyY5KRG+cLfftx6oipnReYoEInIa2t5oZwsPf9Ihqavbjcmmn4L3qdQ3cYQ9pN84Y8wOD3yGAY91M++uCW1VU9HYJhm2mYq2tNDvR5pjTRt3WKC9mTk8t9AydB01qToH/APYqd63nTv7zRjMRXhlLjR7Ptt+Ra628e2kZas64jqHQH2tkNMc6hTsBjWIUKkJm+og5eKIIw3stRAoh1P98lTUVA3lgwB1lNnG8G1MtnJgdBd80jz54igeEVvIV58NZBsX7Jjgdle1wV9rWoegGT/noBYsI+i/a9RfpKLlx1OdxQB2Nol+Z83SF8AXPVZc+dhCG24uLc+DGWNoz12fg+Q6VIMrehW3PgWJVmIuzKkjFo0CaLXYQ7BtoD/WWdGrEVxc58al1j6vJFhVBKMXrGZFUQKFeFN713DvL5LqrjO5Vcoezk6J+OqLx+MzZM4TMyFbYf59lxSq1aPLP680uDfYgqjHZbF2Se25slVHhPNnlHBrCqc9Tq6wGx7sJqTsjITKSUzC8K3f8koA99Q8DFXA5wc6rIuF+2iDr1xIuhbJcokKmw+tcoJx8rNc8pxGq7LsvBhl/f1Mt4kQN1eBuoauTD4FJH3qbLG6smjl9dCyZKaBf2spZJycOuQ1R5NqNCo0gMLrHHvq81EtW9PmvVXhsJRXAVwx6w0eejL7VRrnyu5QWmUGVkPXXL7iEXgTfyW21wz375SIuUjvBSiw6+mWUJ2bjzREaaJC/s6pBoWPpnLiTD+eHyIDGfdwr4BSlK/NgZTnazTxJIARezfPmhKQ/ftN7QV2+crlYFBDTa4/S3j0YLX+ONYM6G/AOIPwCH/OdOWd9+LPDZ3nqAfFh5xbaJE86NYgNY93xUlpnmB8qE2Vb +j5tOEEo R32cwD43uLmQaDxLkssq8q7Maca1p+ZZM2fe+rgd55K5XF0MgeGNkMy8GknqNXSV3OGHyxTceVeMtIVefbwcm3pbX+bk1G4ZaRJByqBYkSiZzh7oqxkGS/sUg48pmSswFFj7WQQB09BMjE3vAw20xQOQPTsxSWhpg4QeyiBooducvRYf29fSvgGi/2oeNcbAwTQauliMuQ8E59LWNFSVac8aGjZcqvDFUM8vBeByCQ/T7zJc6QAcVuQ0lLv1BGlkuXWltAF8ZSG1/IYlnYUeTJShDU4NhIK1fKpfXHA69ElXBoJCLJ1LuD2diuanF2kYqyxYPx2Z6aS8XSD1ph82XVljRobhq8eyJOMkXR9x/ZVrN77OAoQC4L2PGGu7Viae4jnTXwhaANesons0BRsS48J67wx26ENVQFpAa+uGUElWAYGoa+EoyYgI1D19pFx6Ax+WO4LoxYd6CInOjizRhTSMLnGC+ygwNjNV6u4lbCdbnHywnXVppG63aXcFhL0JX4aRwrbIZb2WpvX7zowYB89NmHBsB+ONvbVIF6BraaSfz5/U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use this_cpu_try_cmpxchg instead of this_cpu_cmpxchg (*ptr, old, new) == old in mod_zone_state and mod_node_state. x86 CMPXCHG instruction returns success in ZF flag, so this change saves a compare after cmpxchg (and related move instruction in front of cmpxchg). Also, try_cmpxchg implicitly assigns old *ptr value to "old" when cmpxchg fails. There is no need to re-read the value in the loop. No functional change intended. Cc: Andrew Morton Signed-off-by: Uros Bizjak --- mm/vmstat.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/mm/vmstat.c b/mm/vmstat.c index 00e81e99c6ee..894e4c88d241 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -559,8 +559,10 @@ static inline void mod_zone_state(struct zone *zone, { struct per_cpu_zonestat __percpu *pcp = zone->per_cpu_zonestats; s8 __percpu *p = pcp->vm_stat_diff + item; - long o, n, t, z; + long n, t, z; + s8 o; + o = this_cpu_read(*p); do { z = 0; /* overflow to zone counters */ @@ -576,8 +578,7 @@ static inline void mod_zone_state(struct zone *zone, */ t = this_cpu_read(pcp->stat_threshold); - o = this_cpu_read(*p); - n = delta + o; + n = delta + (long)o; if (abs(n) > t) { int os = overstep_mode * (t >> 1) ; @@ -586,7 +587,7 @@ static inline void mod_zone_state(struct zone *zone, z = n + os; n = -os; } - } while (this_cpu_cmpxchg(*p, o, n) != o); + } while (!this_cpu_try_cmpxchg(*p, &o, n)); if (z) zone_page_state_add(z, zone, item); @@ -616,7 +617,8 @@ static inline void mod_node_state(struct pglist_data *pgdat, { struct per_cpu_nodestat __percpu *pcp = pgdat->per_cpu_nodestats; s8 __percpu *p = pcp->vm_node_stat_diff + item; - long o, n, t, z; + long n, t, z; + s8 o; if (vmstat_item_in_bytes(item)) { /* @@ -629,6 +631,7 @@ static inline void mod_node_state(struct pglist_data *pgdat, delta >>= PAGE_SHIFT; } + o = this_cpu_read(*p); do { z = 0; /* overflow to node counters */ @@ -644,8 +647,7 @@ static inline void mod_node_state(struct pglist_data *pgdat, */ t = this_cpu_read(pcp->stat_threshold); - o = this_cpu_read(*p); - n = delta + o; + n = delta + (long)o; if (abs(n) > t) { int os = overstep_mode * (t >> 1) ; @@ -654,7 +656,7 @@ static inline void mod_node_state(struct pglist_data *pgdat, z = n + os; n = -os; } - } while (this_cpu_cmpxchg(*p, o, n) != o); + } while (!this_cpu_try_cmpxchg(*p, &o, n)); if (z) node_page_state_add(z, pgdat, item);