From patchwork Wed May 1 15:31:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13650945 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EAF1C4345F for ; Wed, 1 May 2024 15:31:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DBF126B0085; Wed, 1 May 2024 11:31:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D6E456B0087; Wed, 1 May 2024 11:31:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5D7F6B0088; Wed, 1 May 2024 11:31:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A85676B0085 for ; Wed, 1 May 2024 11:31:32 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 580D71A0BCA for ; Wed, 1 May 2024 15:31:32 +0000 (UTC) X-FDA: 82070216424.07.40CCFEF Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf19.hostedemail.com (Postfix) with ESMTP id 3868C1A0007 for ; Wed, 1 May 2024 15:31:29 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=EezvSwcv; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714577490; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=XZIMUVzeCJFd/i9fs7YI0OvweKFJssoNWijoVhKZP4M=; b=sPl9pi+kuzk2py7lRBJ56kS6OqvXQdGzuSSL9OqzaGP/ntKPTJ/Jy67HRNyH8lqk5MC2N5 ZDrvjhxZHVIR+1y/HMjlO6d0grs6ulpOEVPbN+MWGu4bWjsHoEiCgiXkZcxWVPAKu+pKLp lBQW2MZPPsTuK/dRqBS8ku9Oc9ESAxk= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=EezvSwcv; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714577490; a=rsa-sha256; cv=none; b=sIT4gLSPllHePrERNSCkchuMcsRhbAFQYIZVG3hJIWpGPMAzrt+F65LgbNdsCfCXdmOOZf xsbncVzcd4cvNo0VY3v3DajEl05L21GhCf16Q9LAKQfTd5mKsv60nlKHgzEd7ClShOZR0o AtSsUqvlpXq/9VqScsh5yGx5nCi/PR8= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:In-Reply-To:References; bh=XZIMUVzeCJFd/i9fs7YI0OvweKFJssoNWijoVhKZP4M=; b=EezvSwcvlmxK90Q6NuYmO18x6I M0B1IRc9HRN3oK2TGfh0Md5hYoTKFRBM3xcBk74Va8UJKNF+gx3dZetJZ2xQMXNe3d3AcvZo32tBR 1jD5TSVDVzjpl/NldRB0XBrwoRZP/R3BjrErpQrviNyZyOiqNuycFP4LtCrJPXWUqh1jGmBoDw7rR +jFQVDbRRwj88faJp8DrxFPGW/f5IF9IqiGHbZeITDRBHomzbmFY+QH++9iAnht5qbFRJJL3tjpGY SaJueaYP4iRWbjJIHoFDeC4H5Wp8chmDzsIOFAfe+CZmhDAezY8VRpn5vBcCEKrQ+s7omugfTuft+ 4XmqnjDw==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1s2BvV-0000000HBB0-2Wrz; Wed, 01 May 2024 15:31:21 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , Zi Yan , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Luis Chamberlain Subject: [PATCH] XArray: Set the marks correctly when splitting an entry Date: Wed, 1 May 2024 16:31:18 +0100 Message-ID: <20240501153120.4094530-1-willy@infradead.org> X-Mailer: git-send-email 2.44.0 MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 3868C1A0007 X-Stat-Signature: fjnje5cex8sra5utdkikhmsgsgokb1pn X-Rspam-User: X-HE-Tag: 1714577489-538671 X-HE-Meta: U2FsdGVkX1/WsOhzXnk+vSiMHh0ma9ve4LkE8exzcr51rn+hzQJcbLxBiQFLIzgIzSY86gQRsdyhghJXFDngcQ9eZgydLh4fOtjVhDs0pBIPs/ctNNNiWefUDfi9ldaTmrClBSqObaIh/ejJ1rhe55QYuCZQo+YT+FG1vciKygHkzL6+5PfwZf5qvIflQPnXwz9e6DIo0X7zScNYxAmMrCWvyka8PfbXPMpmNaQfbcNWtkXB8ZGCd8SQg6PZzhkapozixNB8/CmPJdIbAuxDWdx9Fko9wKQcjRMLltlhYdM82Fxb1G33sBBZAk6bAkpWCf8mKJqO/e0/gNNcMIFGSJw+HKftrJDUkz37Oy5No1D40TFxOcNsx0IK5j4B6EPx6+dgSpAditA3xNtsWZl62FhAwZiycH9bCeLjc9wRlsdsOXdgXwP4D0gDrAhtMBMEm0brYk/0NJXBf9FdWXnIPDwanfsmcVWLNTkU/EAKzaslFpgmHRfzJBBDlMocJPUtEz5usUwYwZz2bkZy1+LafHEoQ6OIjgAZ658H+xEycPgusQhcrmxZkarJfMqaIB40ZHkqbXWWkxYqoZsN5+QkxNp6tdqZ1zNzw1TeqTTNnl/cbCGI6l1mGeFeqO+vFCFS/9x3BY9PQBr8lomfvcn/Y99/1dz1wDajDVewuesY+PQ4OUpQD3k5t/ClKiDt/5P7DfPL1A8EcoS4dqy3Il9Z7q1FvUhNVtrIPYC6BFtUBR8/lj3mC0y6tCIpj6UW8cl9QVl9xtRy21qC5Sm6paQjqvU/kf2fQhjqZ+Iu8N8Dkpe3i1MjIDASsnAc6qexxC8jCNSjwZxhu5Y9JEiLlxOVUkbMhBokWvLwK604FQf77JRwoKN/J5M9/w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If we created a new node to replace an entry which had search marks set, we were setting the search mark on every entry in that node. That works fine when we're splitting to order 0, but when splitting to a larger order, we must not set the search marks on the sibling entries. Reported-by: Luis Chamberlain Link: https://lore.kernel.org/r/ZjFGCOYk3FK_zVy3@bombadil.infradead.org Fixes: c010d47f107f ("mm: thp: split huge page to any lower order pages") Signed-off-by: Matthew Wilcox (Oracle) Tested-by: Luis Chamberlain --- lib/test_xarray.c | 14 +++++++++++++- lib/xarray.c | 23 +++++++++++++++++++---- 2 files changed, 32 insertions(+), 5 deletions(-) diff --git a/lib/test_xarray.c b/lib/test_xarray.c index 0aea6a85099d..ab9cc42a0d74 100644 --- a/lib/test_xarray.c +++ b/lib/test_xarray.c @@ -1788,9 +1788,11 @@ static void check_split_1(struct xarray *xa, unsigned long index, unsigned int order, unsigned int new_order) { XA_STATE_ORDER(xas, xa, index, new_order); - unsigned int i; + unsigned int i, found; + void *entry; xa_store_order(xa, index, order, xa, GFP_KERNEL); + xa_set_mark(xa, index, XA_MARK_1); xas_split_alloc(&xas, xa, order, GFP_KERNEL); xas_lock(&xas); @@ -1807,6 +1809,16 @@ static void check_split_1(struct xarray *xa, unsigned long index, xa_set_mark(xa, index, XA_MARK_0); XA_BUG_ON(xa, !xa_get_mark(xa, index, XA_MARK_0)); + xas_set_order(&xas, index, 0); + found = 0; + rcu_read_lock(); + xas_for_each_marked(&xas, entry, ULONG_MAX, XA_MARK_1) { + found++; + XA_BUG_ON(xa, xa_is_internal(entry)); + } + rcu_read_unlock(); + XA_BUG_ON(xa, found != 1 << (order - new_order)); + xa_destroy(xa); } diff --git a/lib/xarray.c b/lib/xarray.c index 1c87d871cacf..32d4bac8c94c 100644 --- a/lib/xarray.c +++ b/lib/xarray.c @@ -970,8 +970,22 @@ static unsigned int node_get_marks(struct xa_node *node, unsigned int offset) return marks; } +static inline void node_mark_slots(struct xa_node *node, unsigned int sibs, + xa_mark_t mark) +{ + int i; + + if (sibs == 0) + node_mark_all(node, mark); + else { + for (i = 0; i < XA_CHUNK_SIZE; i += sibs + 1) + node_set_mark(node, i, mark); + } +} + static void node_set_marks(struct xa_node *node, unsigned int offset, - struct xa_node *child, unsigned int marks) + struct xa_node *child, unsigned int sibs, + unsigned int marks) { xa_mark_t mark = XA_MARK_0; @@ -979,7 +993,7 @@ static void node_set_marks(struct xa_node *node, unsigned int offset, if (marks & (1 << (__force unsigned int)mark)) { node_set_mark(node, offset, mark); if (child) - node_mark_all(child, mark); + node_mark_slots(child, sibs, mark); } if (mark == XA_MARK_MAX) break; @@ -1078,7 +1092,8 @@ void xas_split(struct xa_state *xas, void *entry, unsigned int order) child->nr_values = xa_is_value(entry) ? XA_CHUNK_SIZE : 0; RCU_INIT_POINTER(child->parent, node); - node_set_marks(node, offset, child, marks); + node_set_marks(node, offset, child, xas->xa_sibs, + marks); rcu_assign_pointer(node->slots[offset], xa_mk_node(child)); if (xa_is_value(curr)) @@ -1087,7 +1102,7 @@ void xas_split(struct xa_state *xas, void *entry, unsigned int order) } else { unsigned int canon = offset - xas->xa_sibs; - node_set_marks(node, canon, NULL, marks); + node_set_marks(node, canon, NULL, 0, marks); rcu_assign_pointer(node->slots[canon], entry); while (offset > canon) rcu_assign_pointer(node->slots[offset--],