From patchwork Fri Jul 15 11:33:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brian Foster X-Patchwork-Id: 12919118 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D00CC43334 for ; Fri, 15 Jul 2022 11:33:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B0CBF9401E2; Fri, 15 Jul 2022 07:33:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ABD3C9401A5; Fri, 15 Jul 2022 07:33:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 986D49401E2; Fri, 15 Jul 2022 07:33:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 859289401A5 for ; Fri, 15 Jul 2022 07:33:53 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5A1CE215FB for ; Fri, 15 Jul 2022 11:33:53 +0000 (UTC) X-FDA: 79689124746.26.9B9783E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf03.hostedemail.com (Postfix) with ESMTP id 075D02004D for ; Fri, 15 Jul 2022 11:33:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657884831; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yxgyt0qVaY9WwgzpJ9tasNw5y5ALrf5uKQLs4e0miO8=; b=KsG+gkOMgCfpsfi3xo+k0Imed0klO+56Uj4Ij2quioaoPqxnfDTSAVX2Yl8Ob10UV6Uab2 LWFY3FO0dlnwFgDzrB/p21rN5a53e6VjW1kUXeRos7/lOcT0gpwIltFKu8uz1NrMjyrdpK qkxk5tPi9C8vpHO9RUTxRVySKn/kPH8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-329-6T1jPdT4OGOhsQrctocbmA-1; Fri, 15 Jul 2022 07:33:50 -0400 X-MC-Unique: 6T1jPdT4OGOhsQrctocbmA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0ED738041BE; Fri, 15 Jul 2022 11:33:50 +0000 (UTC) Received: from bfoster.redhat.com (unknown [10.22.32.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id CD1692026D64; Fri, 15 Jul 2022 11:33:49 +0000 (UTC) From: Brian Foster To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Matthew Wilcox , ikent@redhat.com, oleg@redhat.com Subject: [PATCH 1/3] pid: replace pidmap_lock with xarray lock Date: Fri, 15 Jul 2022 07:33:47 -0400 Message-Id: <20220715113349.831370-2-bfoster@redhat.com> In-Reply-To: <20220715113349.831370-1-bfoster@redhat.com> References: <20220715113349.831370-1-bfoster@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KsG+gkOM; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf03.hostedemail.com: domain of bfoster@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=bfoster@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657884832; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yxgyt0qVaY9WwgzpJ9tasNw5y5ALrf5uKQLs4e0miO8=; b=2BGYlh4yqZtT2me+1uXfUOm4z5ZELRwAlnWCkjl2ezUSeSlkK4hxXhCMXGhvnHoMb07M/Z q4MLsSkBMYkD9zPzep+HC7p+roo0jZAkq1Yi7QmSFX3C/QqOIiWbXZ8xXURHHWheNvc5rv XBf0aW4GNyjFMduqHnvIUn4+8zge5bw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657884832; a=rsa-sha256; cv=none; b=XQ3rjJpcdlbRwYACtrRVv0UJzoW2RiZBMi7QMTJhY7y0QdIs+EuBt1lyawMEhw3rlpCpFD jrXart7hJkTg2rKRpgH/OeW77R1xKdDU8thvVRGYFmvTwE6YsVdLGioSnNmBWqApPixybL sxUV2BWUZfDX8QYeUiEsvBBYzD05ZCU= X-Rspam-User: X-Rspamd-Queue-Id: 075D02004D X-Rspamd-Server: rspam02 X-Stat-Signature: n51jui56j8x1cuo4rnoybbbti4qwnnz6 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KsG+gkOM; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf03.hostedemail.com: domain of bfoster@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=bfoster@redhat.com X-HE-Tag: 1657884831-242898 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: As a first step to changing the struct pid tracking code from the idr over to the xarray, replace the custom pidmap_lock spinlock with the internal lock associated with the underlying xarray. This is effectively equivalent to using idr_lock() and friends, but since the goal is to disentangle from the idr, move directly to the underlying xarray api. Signed-off-by: Matthew Wilcox Signed-off-by: Brian Foster --- kernel/pid.c | 79 ++++++++++++++++++++++++++-------------------------- 1 file changed, 40 insertions(+), 39 deletions(-) diff --git a/kernel/pid.c b/kernel/pid.c index 2fc0a16ec77b..72a6e9d0db81 100644 --- a/kernel/pid.c +++ b/kernel/pid.c @@ -86,22 +86,6 @@ struct pid_namespace init_pid_ns = { }; EXPORT_SYMBOL_GPL(init_pid_ns); -/* - * Note: disable interrupts while the pidmap_lock is held as an - * interrupt might come in and do read_lock(&tasklist_lock). - * - * If we don't disable interrupts there is a nasty deadlock between - * detach_pid()->free_pid() and another cpu that does - * spin_lock(&pidmap_lock) followed by an interrupt routine that does - * read_lock(&tasklist_lock); - * - * After we clean up the tasklist_lock and know there are no - * irq handlers that take it we can leave the interrupts enabled. - * For now it is easier to be safe than to prove it can't happen. - */ - -static __cacheline_aligned_in_smp DEFINE_SPINLOCK(pidmap_lock); - void put_pid(struct pid *pid) { struct pid_namespace *ns; @@ -129,10 +113,11 @@ void free_pid(struct pid *pid) int i; unsigned long flags; - spin_lock_irqsave(&pidmap_lock, flags); for (i = 0; i <= pid->level; i++) { struct upid *upid = pid->numbers + i; struct pid_namespace *ns = upid->ns; + + xa_lock_irqsave(&ns->idr.idr_rt, flags); switch (--ns->pid_allocated) { case 2: case 1: @@ -150,8 +135,8 @@ void free_pid(struct pid *pid) } idr_remove(&ns->idr, upid->nr); + xa_unlock_irqrestore(&ns->idr.idr_rt, flags); } - spin_unlock_irqrestore(&pidmap_lock, flags); call_rcu(&pid->rcu, delayed_put_pid); } @@ -206,7 +191,7 @@ struct pid *alloc_pid(struct pid_namespace *ns, pid_t *set_tid, } idr_preload(GFP_KERNEL); - spin_lock_irq(&pidmap_lock); + xa_lock_irq(&tmp->idr.idr_rt); if (tid) { nr = idr_alloc(&tmp->idr, NULL, tid, @@ -233,7 +218,7 @@ struct pid *alloc_pid(struct pid_namespace *ns, pid_t *set_tid, nr = idr_alloc_cyclic(&tmp->idr, NULL, pid_min, pid_max, GFP_ATOMIC); } - spin_unlock_irq(&pidmap_lock); + xa_unlock_irq(&tmp->idr.idr_rt); idr_preload_end(); if (nr < 0) { @@ -266,34 +251,38 @@ struct pid *alloc_pid(struct pid_namespace *ns, pid_t *set_tid, INIT_HLIST_HEAD(&pid->inodes); upid = pid->numbers + ns->level; - spin_lock_irq(&pidmap_lock); - if (!(ns->pid_allocated & PIDNS_ADDING)) - goto out_unlock; for ( ; upid >= pid->numbers; --upid) { + tmp = upid->ns; + + xa_lock_irq(&tmp->idr.idr_rt); + if (tmp == ns && !(tmp->pid_allocated & PIDNS_ADDING)) { + xa_unlock_irq(&tmp->idr.idr_rt); + put_pid_ns(ns); + goto out_free; + } + /* Make the PID visible to find_pid_ns. */ - idr_replace(&upid->ns->idr, pid, upid->nr); - upid->ns->pid_allocated++; + idr_replace(&tmp->idr, pid, upid->nr); + tmp->pid_allocated++; + xa_unlock_irq(&tmp->idr.idr_rt); } - spin_unlock_irq(&pidmap_lock); return pid; -out_unlock: - spin_unlock_irq(&pidmap_lock); - put_pid_ns(ns); - out_free: - spin_lock_irq(&pidmap_lock); while (++i <= ns->level) { upid = pid->numbers + i; - idr_remove(&upid->ns->idr, upid->nr); - } + tmp = upid->ns; - /* On failure to allocate the first pid, reset the state */ - if (ns->pid_allocated == PIDNS_ADDING) - idr_set_cursor(&ns->idr, 0); + xa_lock_irq(&tmp->idr.idr_rt); - spin_unlock_irq(&pidmap_lock); + /* On failure to allocate the first pid, reset the state */ + if (tmp == ns && tmp->pid_allocated == PIDNS_ADDING) + idr_set_cursor(&ns->idr, 0); + + idr_remove(&tmp->idr, upid->nr); + xa_unlock_irq(&tmp->idr.idr_rt); + } kmem_cache_free(ns->pid_cachep, pid); return ERR_PTR(retval); @@ -301,9 +290,9 @@ struct pid *alloc_pid(struct pid_namespace *ns, pid_t *set_tid, void disable_pid_allocation(struct pid_namespace *ns) { - spin_lock_irq(&pidmap_lock); + xa_lock_irq(&ns->idr.idr_rt); ns->pid_allocated &= ~PIDNS_ADDING; - spin_unlock_irq(&pidmap_lock); + xa_unlock_irq(&ns->idr.idr_rt); } struct pid *find_pid_ns(int nr, struct pid_namespace *ns) @@ -646,6 +635,18 @@ SYSCALL_DEFINE2(pidfd_open, pid_t, pid, unsigned int, flags) return fd; } +/* + * Note: disable interrupts while the xarray lock is held as an interrupt might + * come in and do read_lock(&tasklist_lock). + * + * If we don't disable interrupts there is a nasty deadlock between + * detach_pid()->free_pid() and another cpu that does xa_lock() followed by an + * interrupt routine that does read_lock(&tasklist_lock); + * + * After we clean up the tasklist_lock and know there are no irq handlers that + * take it we can leave the interrupts enabled. For now it is easier to be safe + * than to prove it can't happen. + */ void __init pid_idr_init(void) { /* Verify no one has done anything silly: */