From patchwork Wed Jan 17 16:35:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 13522027 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A543122F02 for ; Wed, 17 Jan 2024 16:36:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509382; cv=none; b=Ctyb2v8+IIlb5MrhJRqexqO1ThEoEWEESyAC2CsozQUMdt27slijSBHfFb11wcx5zV4gXxN1lpf4JiXl1eE/gysBpEciugykVT7i5dkPKb1HUxuw2d1c4MieUCdADWY4SA0ayMCfiF4zuZD/YKIX/kxqLdGg1ZuKJ+S+Bk6B+Uc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509382; c=relaxed/simple; bh=RIu0wCoHEEwmwrWrI4fk2QwIf3Z4+DNKewqmZMfVEmc=; h=DKIM-Signature:Received:X-MC-Unique:Received:Received:From:To:Cc: Subject:Date:Message-Id:In-Reply-To:References:MIME-Version: Content-Transfer-Encoding:X-Scanned-By; b=AYxxmfUF3aOnS5cInHFHiW9xgocQtXh/1aaXVD8AVhJtWf+3NSbuFPlGaxgCzoDaGFIuiianfPF6EGiJlqBr83GPXaZmMnys6I0i4fcUviq439ijeq4MC53kX3DPpTSkrffDRbP0Ue7nnWiTmSPx4yLpk8prkVLXr8eI4U7dIVA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=g4gAgs2D; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="g4gAgs2D" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705509379; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pwiZxk2/LIns0OuqqkiNPgLv6/0v49jXf5KKeW3bftU=; b=g4gAgs2DoxwZfL6Zk3WmV/pM9jbxg2ojQCo2vUnhFBFAQs22hR5+imOEAd7hS2rAff7elV l/Axz+ekS10WCzhBqDItyqKhGPtagi82bmaNO/Q6YvwvjYz9w/XJ6qE8VSZec+Ewj9rRe9 tjP5p3hF+RPuKGd9pztvnbf00bV+dVQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-306-Ry1xxOizP6KjrjiwrnqQ_w-1; Wed, 17 Jan 2024 11:36:16 -0500 X-MC-Unique: Ry1xxOizP6KjrjiwrnqQ_w-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BEFDB81DA88; Wed, 17 Jan 2024 16:36:14 +0000 (UTC) Received: from llong.com (unknown [10.22.16.147]) by smtp.corp.redhat.com (Postfix) with ESMTP id 272321121306; Wed, 17 Jan 2024 16:36:13 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Frederic Weisbecker , Jonathan Corbet , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Davidlohr Bueso , Shuah Khan Cc: cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org, Mrunal Patel , Ryan Phillips , Brent Rowsell , Peter Hunt , Cestmir Kalina , Nicolas Saenz Julienne , Alex Gladkov , Marcelo Tosatti , Phil Auld , Paul Gortmaker , Daniel Bristot de Oliveira , Juri Lelli , Peter Zijlstra , Costa Shulyupin , Waiman Long Subject: [RFC PATCH 6/8] cgroup/cpuset: Enable dynamic rcu_nocb mode on isolated CPUs Date: Wed, 17 Jan 2024 11:35:09 -0500 Message-Id: <20240117163511.88173-7-longman@redhat.com> In-Reply-To: <20240117163511.88173-1-longman@redhat.com> References: <20240117163511.88173-1-longman@redhat.com> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.3 The patch adds RCU no-callback isolation mode dynamically to isolated CPUs within isolated partitions when the full CPU isolation mode is enabled. This isolation feature will only be available for use by cpuset if the "rcu_nocb" boot command line option is specified in the kernel command line with or without the optional CPU list argument. Signed-off-by: Waiman Long --- kernel/cgroup/cpuset.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index d1d4ce213979..40bbb0a9cb84 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -218,6 +218,11 @@ enum isolated_cpus_modifiers { */ static cpumask_var_t isolated_cpus; +/* + * rcu_nocb_mask set up at boot time. + */ +static cpumask_var_t rcu_nocb_mask_preset; + /* * Enable full CPU isolation in isolated partitions, if set. */ @@ -229,15 +234,26 @@ static bool isolation_full; static int isolation_flags; enum cpuset_isolation_types { + ISOL_TYPE_RCU, /* RCU no-callback CPU mode */ ISOL_TYPE_MAX, }; +enum cpuset_isolation_flags { + ISOL_FLAG_RCU = BIT(ISOL_TYPE_RCU), +}; + static const char * const isolation_type_names[ISOL_TYPE_MAX] = { + [ISOL_TYPE_RCU] = "rcu_nocbs", }; /* Detect the cpuset isolation modes that can be enabled */ static __init int set_isolation_flags(void) { + if (rcu_nocb_enabled(NULL)) { + BUG_ON(!zalloc_cpumask_var(&rcu_nocb_mask_preset, GFP_KERNEL)); + (void)rcu_nocb_enabled(rcu_nocb_mask_preset); + isolation_flags |= ISOL_FLAG_RCU; + } return 0; } late_initcall(set_isolation_flags); @@ -1554,6 +1570,7 @@ static int partition_xcpus_del(int old_prs, struct cpuset *parent, static int update_isolation_cpumasks(struct cpumask *mask, int modifier) { int err; + bool enable = (modifier == ISOL_CPUS_ADD); lockdep_assert_cpus_held(); @@ -1569,6 +1586,25 @@ static int update_isolation_cpumasks(struct cpumask *mask, int modifier) if (WARN_ON_ONCE(cpumask_empty(mask))) return -EINVAL; + err = 0; + if (isolation_flags & ISOL_FLAG_RCU) { + /* + * When disabling rcu_nocb, make sure that we don't touch any + * CPUs that have already been set in rcu_nocb_mask_preset. + */ + if (!enable && cpumask_intersects(mask, rcu_nocb_mask_preset)) { + cpumask_var_t tmp_mask; + + if (WARN_ON_ONCE(!alloc_cpumask_var(&tmp_mask, GFP_KERNEL))) + return -ENOMEM; + if (cpumask_andnot(tmp_mask, mask, rcu_nocb_mask_preset)) + err = rcu_nocb_cpumask_update(tmp_mask, enable); + free_cpumask_var(tmp_mask); + } else { + err = rcu_nocb_cpumask_update(mask, enable); + } + } + WARN_ON_ONCE(err); return err; }