From patchwork Mon Dec 28 13:08:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jann Horn X-Patchwork-Id: 11991213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0957C433DB for ; Mon, 28 Dec 2020 13:10:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4BD9F22582 for ; Mon, 28 Dec 2020 13:10:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4BD9F22582 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8B74A8D0007; Mon, 28 Dec 2020 08:10:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 867318D0001; Mon, 28 Dec 2020 08:10:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 709218D0007; Mon, 28 Dec 2020 08:10:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0064.hostedemail.com [216.40.44.64]) by kanga.kvack.org (Postfix) with ESMTP id 57A2C8D0001 for ; Mon, 28 Dec 2020 08:10:39 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 11153181AEF00 for ; Mon, 28 Dec 2020 13:10:39 +0000 (UTC) X-FDA: 77642725398.10.mouth78_5f1418d27494 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id D849416A4A9 for ; Mon, 28 Dec 2020 13:10:38 +0000 (UTC) X-HE-Tag: mouth78_5f1418d27494 X-Filterd-Recvd-Size: 4243 Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) by imf13.hostedemail.com (Postfix) with ESMTP for ; Mon, 28 Dec 2020 13:10:38 +0000 (UTC) Received: by mail-wm1-f41.google.com with SMTP id 190so10323180wmz.0 for ; Mon, 28 Dec 2020 05:10:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=zftq0G0O7QGXq9h4glxWYJpeeRtUvNit+ou+JTWbY3g=; b=OELhWxcuL53FNsdC6+tcvBdGBwCtiwHpUcARzk9hyAswemPyZAGkZ7Q7hDnvwzUu/K fuVSkNQpP7wFpf/0EqMLIG5qPRBqhOBoEEiNPetdKbdZtDwuPAyoTD72Sgg8Y/FI0M36 bpzCBCcnrk0d+JltV1DQ90m+9470gutUpGDJn/iO4wN8lFvJm5L1vgyxFNZczeiqTBfL 9DUhQELJ7+06CvPU1y8oJj1KUoOPxKU6sNXTwDxXI6C1TLG6wYsLNVpjzLIN704RkZWk Sob8LihCUI8GAk2T+zVem5lFC4Qinixa1CHOQvg6P44gwtPAEm0Inb6lH+UeLSBzhE3w 89Tg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=zftq0G0O7QGXq9h4glxWYJpeeRtUvNit+ou+JTWbY3g=; b=fhRZg8s1C4vMtQxZGXxnEsfRIi8O+yQsSoEk7bT2jECnV+CfYXNv3pW4+bJnVhRP7V QXXNU7d1LRZ+P5SNUk4LqhFvXsU3FQyNg1Gpuj1alxU6opJBNzXzWWRWchelDUfFrx/W YP06jnD7iEE4xxvkYTdgkdV4diFd78fFaDhPx11sL5D0AspdZ/WMC6C8zQ65b6prEc96 z2Cfrhx83ro4X4iNhBZSR9seCrNvAvV7BItWXk63P255tUP0p8MCj3a6OMFPPVwtojjX ZP+KJJwBM2WSGssmosbMsJihHHxVzZCNCJioTsx0rJoTmx99tzktnAfeC4qx/hjC+EkA 781Q== X-Gm-Message-State: AOAM5320GeCaBJGHtjig/lrvJmrvPg5EKLZPhzyJJCI55Zz6d+4VHV1a ktFnXlqYU5K8qbgT6F3zbzYAiw== X-Google-Smtp-Source: ABdhPJzpQFEgHuV56rvoENm0o2TMz62ZmYZzx2qPmRPQyU4imjso2CB9zz8eCjqqBcmuNiFqO+/3kA== X-Received: by 2002:a1c:186:: with SMTP id 128mr20370509wmb.176.1609161037060; Mon, 28 Dec 2020 05:10:37 -0800 (PST) Received: from localhost ([2a02:168:96c5:1:55ed:514f:6ad7:5bcc]) by smtp.gmail.com with ESMTPSA id a62sm21401461wmh.40.2020.12.28.05.10.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Dec 2020 05:10:36 -0800 (PST) From: Jann Horn To: Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH] mm, slub: Consider rest of partial list if acquire_slab() fails Date: Mon, 28 Dec 2020 14:08:53 +0100 Message-Id: <20201228130853.1871516-1-jannh@google.com> X-Mailer: git-send-email 2.29.2.729.g45daf8777d-goog MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: acquire_slab() fails if there is contention on the freelist of the page (probably because some other CPU is concurrently freeing an object from the page). In that case, it might make sense to look for a different page (since there might be more remote frees to the page from other CPUs, and we don't want contention on struct page). However, the current code accidentally stops looking at the partial list completely in that case. Especially on kernels without CONFIG_NUMA set, this means that get_partial() fails and new_slab_objects() falls back to new_slab(), allocating new pages. This could lead to an unnecessary increase in memory fragmentation. Fixes: 7ced37197196 ("slub: Acquire_slab() avoid loop") Signed-off-by: Jann Horn Acked-by: David Rientjes Acked-by: Joonsoo Kim --- mm/slub.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) base-commit: 5c8fe583cce542aa0b84adc939ce85293de36e5e diff --git a/mm/slub.c b/mm/slub.c index 0c8b43a5b3b0..b1777ba06735 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1974,7 +1974,7 @@ static void *get_partial_node(struct kmem_cache *s, struct kmem_cache_node *n, t = acquire_slab(s, n, page, object == NULL, &objects); if (!t) - break; + continue; /* cmpxchg raced */ available += objects; if (!object) {