From patchwork Tue Sep 17 14:34:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boqun Feng X-Patchwork-Id: 13806222 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A23BEE6426 for ; Tue, 17 Sep 2024 14:35:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B71876B008A; Tue, 17 Sep 2024 10:35:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B230A6B0089; Tue, 17 Sep 2024 10:35:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B89F6B008C; Tue, 17 Sep 2024 10:35:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5545F6B0085 for ; Tue, 17 Sep 2024 10:35:01 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id CCDB8120544 for ; Tue, 17 Sep 2024 14:35:00 +0000 (UTC) X-FDA: 82574477160.28.5A19702 Received: from mail-qk1-f175.google.com (mail-qk1-f175.google.com [209.85.222.175]) by imf06.hostedemail.com (Postfix) with ESMTP id 99A2418000B for ; Tue, 17 Sep 2024 14:34:58 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OWcqMGpB; spf=pass (imf06.hostedemail.com: domain of boqun.feng@gmail.com designates 209.85.222.175 as permitted sender) smtp.mailfrom=boqun.feng@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726583587; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IDKpmr/a+FSU5QkWBrZE0xc7m3+8vAVQA9drcgUYXjs=; b=n77VDYDo4Oi8wGPIo6VVU/GShngLbZXzKPPkl1uIFcUrUzTyvyXSal1bH0/MBFS70I4yzj Qc8lUJz3bE/y1VO71Y2WZW3G+18m+eD++pKZnLUNyvNRamKGTO6viYl9a++/a0bt9NPahi Zl76Q4wKVzQLSe+LDlegGqZSitvtxpE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726583587; a=rsa-sha256; cv=none; b=BmAQ+hCIWrpuizjOe6CD0WQNtbLYN9r8VhQsdEQcESCu8XBinbYbFKsvFwncKDK2HCStgU WtELHF9vIrGCUuqAxZ6jSxPcN+cVsgXH6gzs0YQKkz/ufgx+ySoP8+jbzbMTQmZL3/MWdE 5Gr6xMrAIC+VW15R+IqH/dEnwXMX0c4= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OWcqMGpB; spf=pass (imf06.hostedemail.com: domain of boqun.feng@gmail.com designates 209.85.222.175 as permitted sender) smtp.mailfrom=boqun.feng@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-qk1-f175.google.com with SMTP id af79cd13be357-7a9ab721058so587741685a.1 for ; Tue, 17 Sep 2024 07:34:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726583698; x=1727188498; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:feedback-id:from:to:cc:subject :date:message-id:reply-to; bh=IDKpmr/a+FSU5QkWBrZE0xc7m3+8vAVQA9drcgUYXjs=; b=OWcqMGpBvTVZlDUkiGZfrfRI/R/JIPEU5IFipX0VCVADN5e6ZulhjW8Jyy4z6MM0/x ams8qMkaITty6mmgX8FYbsiYlHpHXu5laCzzzu/hppeouDQP/v12RA81ROgtVz/X/3EF vVHceq8u1CvcJzUB0Q12JVhgGtXSt7q8zGu1afGjugcU8XaKjxco/dX4ziXDhO9WXQgc +0C7grgdLTVf4ObuKUcnSVLHloDtXAoEq2z3qGdu5oSjyyPsCw1ZAseNPYEpdmRSJ+Ry hHq2uKmxobeZ2ehNPGRo3/Ei9C3D+4spHGTZm3Fs2ZHChE+nGhcPH4+38sZGtZV/+S35 VNAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726583698; x=1727188498; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:feedback-id:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=IDKpmr/a+FSU5QkWBrZE0xc7m3+8vAVQA9drcgUYXjs=; b=TaqOkZBV9C8NJg+OvuNuhCHwLCQkfSiszzYdvVRqBF+gjLWX3BdnARkb2POL4rOEzZ 78DobzKQMU786TGirvv4cPIRbA7UjXbYNStQDGc8Ah4Hy5driouS6hrzK+rDn+/jgsQ3 TaHW41Cq3iZGYxfUm4Kdiddr9DpKpmDN4XxhBUXE3LlP5CYVZ1dcuxyd2BbBLYIzYO/s gSZctZjus+PP8hT9DJv1qVFwimEs0rvNt6RXbXVkiMUfOoAgZ2vqzYfiVNBZ1y1lxXvl 2qyt3eqbDpR/XSYFil7WSyMWylH/KrLOq3pMfSr1bPumPhYraSS6rU8UQxoW8LbeB2vV XbGw== X-Forwarded-Encrypted: i=1; AJvYcCWXSVu9AxLbda36H2CiGxxnNbvM6tDa7e+CbJYZCTDRnMGNT2tY9gBIhZkEG1QRBmPEnzipPzSM9A==@kvack.org X-Gm-Message-State: AOJu0YzA8+kYVZuH2IkV80xWnj/2LrT4jVVoL2QGuobXCTKKnSKlMg1x hx7tRRy1OluEpqZwSaFltqd7R0ELrZFxnAWJc7Yw7wgK2LFFjgoq X-Google-Smtp-Source: AGHT+IG2fokLCq1TmoMpmwBtIu0E6iGiVOGurVZcFpBIdhqlPDe2hAN6K3EzM3w6osDlDfCf58pZgw== X-Received: by 2002:a05:620a:450c:b0:7a4:d56a:a928 with SMTP id af79cd13be357-7a9bf9abe50mr4404654785a.21.1726583697681; Tue, 17 Sep 2024 07:34:57 -0700 (PDT) Received: from fauth1-smtp.messagingengine.com (fauth1-smtp.messagingengine.com. [103.168.172.200]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7ab3e9b39cesm363636485a.65.2024.09.17.07.34.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2024 07:34:57 -0700 (PDT) Received: from phl-compute-05.internal (phl-compute-05.phl.internal [10.202.2.45]) by mailfauth.phl.internal (Postfix) with ESMTP id B5DBB1200070; Tue, 17 Sep 2024 10:34:56 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-05.internal (MEProxy); Tue, 17 Sep 2024 10:34:56 -0400 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrudekjedgjeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhephffvvefufffkofgjfhgggfestdekredtredttden ucfhrhhomhepuehoqhhunhcuhfgvnhhguceosghoqhhunhdrfhgvnhhgsehgmhgrihhlrd gtohhmqeenucggtffrrghtthgvrhhnpeegleejiedthedvheeggfejveefjeejkefgveff ieeujefhueeigfegueehgeeggfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmh epmhgrihhlfhhrohhmpegsohhquhhnodhmvghsmhhtphgruhhthhhpvghrshhonhgrlhhi thihqdeiledvgeehtdeigedqudejjeekheehhedvqdgsohhquhhnrdhfvghngheppehgmh grihhlrdgtohhmsehfihigmhgvrdhnrghmvgdpnhgspghrtghpthhtohepvdehpdhmohgu vgepshhmthhpohhuthdprhgtphhtthhopehlihhnuhigqdhkvghrnhgvlhesvhhgvghrrd hkvghrnhgvlhdrohhrghdprhgtphhtthhopehrtghusehvghgvrhdrkhgvrhhnvghlrdho rhhgpdhrtghpthhtoheplhhinhhugidqmhhmsehkvhgrtghkrdhorhhgpdhrtghpthhtoh eplhhkmhhmsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepphgruhhlmhgt kheskhgvrhhnvghlrdhorhhgpdhrtghpthhtohepfhhrvgguvghrihgtsehkvghrnhgvlh drohhrghdprhgtphhtthhopehnvggvrhgrjhdruhhprgguhhihrgihsehkvghrnhgvlhdr ohhrghdprhgtphhtthhopehjohgvlhesjhhovghlfhgvrhhnrghnuggvshdrohhrghdprh gtphhtthhopehjohhshhesjhhoshhhthhrihhplhgvthhtrdhorhhg X-ME-Proxy: Feedback-ID: iad51458e:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 17 Sep 2024 10:34:56 -0400 (EDT) From: Boqun Feng To: linux-kernel@vger.kernel.org, rcu@vger.kernel.org, linux-mm@kvack.org, lkmm@vger.kernel.org Cc: "Paul E. McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long , Mark Rutland , Thomas Gleixner , Kent Overstreet , Linus Torvalds , Vlastimil Babka , maged.michael@gmail.com Subject: [RFC PATCH 2/4] refscale: Add benchmarks for hazptr Date: Tue, 17 Sep 2024 07:34:00 -0700 Message-ID: <20240917143402.930114-3-boqun.feng@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240917143402.930114-1-boqun.feng@gmail.com> References: <20240917143402.930114-1-boqun.feng@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 99A2418000B X-Stat-Signature: 1fjsyemwg5rmpq1sazcfq1gbmbp6tzgo X-HE-Tag: 1726583698-464548 X-HE-Meta: U2FsdGVkX1+z9pGTyST/9+xadNuif1FI+GINscd20Q0EtIT1Zd8YDagGrEkOxsIH2nebrT0KEf1wI+UOSZqLOdmnCG4Vlj0/uEhj9Cb7QNvchSozQiqmRtiQkt0s44py4EVDyDaU+SETktrIQGKY1p54oQxf5yS2lR+7qJwG4vaju8Loi0jusU4AawRvQIX1/eM/IaLkqANSddRSuWH1A4SZ47Vtu5oBYyjmIoKurcnROzYV3ImCSTq1KPTjUuqCeqnhTBArgT7f5Jyz3i3I3Sb4oBWQGfdMaz6bVutOCCiuOkeGncAWb92jial73n6W1ZwrdeNwC+1HYbE5guTqbWF0a+cQDNVmX9hHilZztdUI2QejpgXQtukPWvV8Lnm9w+z/OmtBKScXvDxo1i2ONzUF8J5AY9YZSxItHr/3t6Q42IaLRcRoMr4EiA/9kRImCdfekOS/3ZpZXPwtonpz4qyce3tM8Rk8lKqAomAQ/MHgJjBoWv1wRMyKmaqIv/1evnnSK50+wvWO1AjD7rHCtOHJRsUCtxOtkNwbdkxExMPZYdQSrVBaDdpE4Xm9tcf0vJywKhfKLiuRLqep2LiD7GCm8GqwdHCs6ubmFgeZycceA9UyCi3+rBIFhiSmO4V+l8rupal8gXIAbsJr1qs5ORaxIVUwmeWukgyBX6K5VRt2o72WV568An/I+hnjOt5daznUQtbJJuhmPaktr+ankOzpknBy13/JsPrebwr+5fNI4r76OXFn9UTEoWpaYjXgsbyjJEt0G/+p7Tc9gSTC42rT7GXrus+Y+e4ckH5pfC9P0MOEsXyDcXATEfjrPkjmUEsRpam3uEjLK1SP7AiER3BXGVU4Cz36RRP98OyO5MDJ25UY0lnpbPXcY3fkDc0kOHd8C3lDXOY1z6rIPFVX8/c27dpSmwfyHPtVDgrMkuU/Xn3fAX3OyDaafuWfyJicEdf27OTYcZIwkoA9Bk1 C8OuheUb kPaBzVnWGnAaAKLY4A1aGFo6twf2ROrD6IvzDKbVqWTlF7GzIcDhqVQDv66MA/NYYT2HPb4KAwwjI79CCM0MA8OAhYtSTwd4SGwTrcggHOH0qAhktjDIHPVKmlaylsKCeb5kI4vetxAAipjeYWZ6UT7KjzEcJyw8vugvPC34A8/+WvV9LpgGglsMXo8yIEtsRVUQ3M6HCUnNdBcu+lt49PjuHq9R5ITiFjet1Kh+vDodNLEpVhAE1+70mAogtl4h0YZwyYa6HdS/NMoLE+EA4SAa2JKiPtdUh25Lxo2YyA9s3BzuXWBgniNqYv4ypExkoDhYMbrf6AQlS4pQCxm1KlWcc9UrqM0p/AldR50jIz3pqIMMAgX6CX+9Z/OKcsigbY648K6kqFzXi2BSd63mh/lRTPgqsKmozAhmg11wrod67Epj5PUyw5SkfV3NOqJ7LVEYCCQjGHws3lkyrWFMkWOh5CBJcivCfMRWGF1+CatHBZR11VwMlLr4FnA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Benchmarks for hazptr are added to evaluate the reader side performance between hazptr and other refcounting mechanisms. Signed-off-by: Boqun Feng --- kernel/rcu/refscale.c | 79 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 78 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/refscale.c b/kernel/rcu/refscale.c index f4ea5b1ec068..7e76ae5159e6 100644 --- a/kernel/rcu/refscale.c +++ b/kernel/rcu/refscale.c @@ -35,6 +35,7 @@ #include #include #include +#include #include "rcu.h" @@ -316,6 +317,82 @@ static struct ref_scale_ops refcnt_ops = { .name = "refcnt" }; +struct hazptr_data { + struct callback_head head; + int i; +}; + +static struct hazptr_data *hazptr_data; + +static bool hazptr_scale_init(void) +{ + hazptr_data = kmalloc(sizeof(*hazptr_data), GFP_KERNEL); + + return !!hazptr_data; +} + +static void free_hazptr_data(struct callback_head *head) +{ + struct hazptr_data *tofree = container_of(head, struct hazptr_data, head); + + kfree(tofree); +} + +static void hazptr_scale_cleanup(void) +{ + if (hazptr_data) { + struct hazptr_data *tmp = hazptr_data; + WRITE_ONCE(hazptr_data, NULL); + + call_hazptr(&tmp->head, free_hazptr_data); + } +} + +static void hazptr_section(const int nloops) +{ + int i; + struct hazptr_context ctx; + hazptr_t *hzptr; + + init_hazptr_context(&ctx); + hzptr = hazptr_alloc(&ctx); + + for (i = nloops; i >= 0; i--) { + BUG_ON(!hazptr_protect(hzptr, hazptr_data, head)); + hazptr_clear(hzptr); + } + + hazptr_free(&ctx, hzptr); + cleanup_hazptr_context(&ctx); +} + +static void hazptr_delay_section(const int nloops, const int udl, const int ndl) +{ + int i; + struct hazptr_context ctx; + hazptr_t *hzptr; + + init_hazptr_context(&ctx); + hzptr = hazptr_alloc(&ctx); + + for (i = nloops; i >= 0; i--) { + BUG_ON(!hazptr_protect(hzptr, hazptr_data, head)); + un_delay(udl, ndl); + hazptr_clear(hzptr); + } + + hazptr_free(&ctx, hzptr); + cleanup_hazptr_context(&ctx); +} + +static struct ref_scale_ops hazptr_ops = { + .init = hazptr_scale_init, + .cleanup = hazptr_scale_cleanup, + .readsection = hazptr_section, + .delaysection = hazptr_delay_section, + .name = "hazptr" +}; + // Definitions for rwlock static rwlock_t test_rwlock; @@ -1081,7 +1158,7 @@ ref_scale_init(void) static struct ref_scale_ops *scale_ops[] = { &rcu_ops, &srcu_ops, RCU_TRACE_OPS RCU_TASKS_OPS &refcnt_ops, &rwlock_ops, &rwsem_ops, &lock_ops, &lock_irq_ops, &acqrel_ops, &clock_ops, &jiffies_ops, - &typesafe_ref_ops, &typesafe_lock_ops, &typesafe_seqlock_ops, + &typesafe_ref_ops, &typesafe_lock_ops, &typesafe_seqlock_ops, &hazptr_ops, }; if (!torture_init_begin(scale_type, verbose))