From patchwork Tue Apr 18 19:13:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13216101 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FA21C77B78 for ; Tue, 18 Apr 2023 19:13:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8C127280004; Tue, 18 Apr 2023 15:13:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7FBF1280001; Tue, 18 Apr 2023 15:13:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 64D3E280004; Tue, 18 Apr 2023 15:13:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 52E07280001 for ; Tue, 18 Apr 2023 15:13:37 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 362438042A for ; Tue, 18 Apr 2023 19:13:37 +0000 (UTC) X-FDA: 80695460874.20.3FFF794 Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by imf28.hostedemail.com (Postfix) with ESMTP id 5DF18C0015 for ; Tue, 18 Apr 2023 19:13:35 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=YCxaymFj; spf=pass (imf28.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.176 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681845215; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fvrO/1+4yglcd2Jvs04tq1hDUyEyh2m1Sh3LN7l5OG8=; b=O+DITbFtruUSwcUQXDjrC5+pOPTyS7Rx8QEtfjQKuDJw0fg4A/JKPr0uZx0Bt7CRHK6Vqj Tg5XXdJmslFdFgG9z5nL9vUndHpBaEfD0Lt54SbW4ZJ21p3qHbE3+btmrt1AlCRDRtZXDU gyKO1BGr+nRROqZSryOvSJAPyEGZzXU= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=YCxaymFj; spf=pass (imf28.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.176 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681845215; a=rsa-sha256; cv=none; b=t+0gOIAOI+q/Fy78g0c5awd4m9inquWmsDIu+TbEh0BP2n0bAXkF19DI8a1NTkPzwv44ID ZVENugsP6Z9WOVUru+5Afe8utqKSyA8GgVwUQFvLg0zAFhtE4wAapTaJTBsdLO8P9nT3e0 rGCjeGsENV2pDnYt9yo0/OZSRTew36c= Received: by mail-qt1-f176.google.com with SMTP id l11so30662106qtj.4 for ; Tue, 18 Apr 2023 12:13:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20221208.gappssmtp.com; s=20221208; t=1681845214; x=1684437214; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fvrO/1+4yglcd2Jvs04tq1hDUyEyh2m1Sh3LN7l5OG8=; b=YCxaymFjTOB/xEVxOKn/iRYET7rdfjWPu3AZDYJ5KtaGsEjeOO2z8QOjdpBzKiI+fh zq/KjeYnjCaxkKVGYNcD9mghxgZwPN5syoeQUj2HsktOmS7F69bhZq+UdMOlukwdy6mr zhoMS/Tx7lhyB4yQTexZo6hq18nMIH/j4LR8HxWWTAFstqWcBJC/+aaBnM/4R2GA77/e eaZxJoD9eYoQfHGUVsharCxWu8P9awFx8F/6/oFYKGnqiIiwRsnchL6XPeWp9ufx15vN it9IVwB25jyUUFrrcgBYhumIpZbTgM/FEDmcZyOcdKC+L06MuZYxfBEyLL+xGpVRPQMZ aekw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681845214; x=1684437214; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fvrO/1+4yglcd2Jvs04tq1hDUyEyh2m1Sh3LN7l5OG8=; b=VVYij0k3XvPpTa+XsWzzlJkXJ8WN+3NqxT2RK191I2ZbjgkfhJKooQSfCmzQdC4mw1 05C2wverSQl5G+Bx51v8GVd75rdxx3tinGsqXu0Wm/ckKP/zbuIJclP5o5xA0srp8kq2 13mAIJFu3+S2OlFYFjMpAHVMaoLYmMPY54JJrLkD41mJYQLc6Ru1IvEPKQrayZYJ027H OpQALpyUt66Uf0Oa5VNW8efABZK/VUjI9UHi6voowO3eDGXN2DJSGKLPqar2LZLm0u2T QNX5TQBLR4jxfhQVxhqEVC3IvKjnSZk8HKRZgrpOHfpqF1HeOZOFvB0d+XdxnTfd7TSe 4QZg== X-Gm-Message-State: AAQBX9eKGMn8SlVJd9/rvHnZPXIXSpwzEoL/MjSK2s6Xkwlea2keE5ca +qPdHQTvvEVEVa1h3vJ3iHc8XPDHLc/rloZiJo0= X-Google-Smtp-Source: AKy350b8I9apZCXgd3Fc463SUtxO97BghAjCQl+N88Qns+QuGdCqUDlXOuuktIWM1UP9lMqTVC8ihw== X-Received: by 2002:ac8:4e93:0:b0:3ef:3880:9db6 with SMTP id 19-20020ac84e93000000b003ef38809db6mr1674175qtp.6.1681845214206; Tue, 18 Apr 2023 12:13:34 -0700 (PDT) Received: from localhost ([2620:10d:c091:400::5:e646]) by smtp.gmail.com with ESMTPSA id f17-20020a05622a1a1100b003ef415f0184sm39541qtb.69.2023.04.18.12.13.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 12:13:33 -0700 (PDT) From: Johannes Weiner To: linux-mm@kvack.org Cc: Kaiyang Zhao , Mel Gorman , Vlastimil Babka , David Rientjes , linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [RFC PATCH 13/26] mm: compaction: remove compaction result helpers Date: Tue, 18 Apr 2023 15:13:00 -0400 Message-Id: <20230418191313.268131-14-hannes@cmpxchg.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230418191313.268131-1-hannes@cmpxchg.org> References: <20230418191313.268131-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5DF18C0015 X-Stat-Signature: 5fjiqh4adrmwaypyayhfc3tfpzowqzn7 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1681845215-866691 X-HE-Meta: U2FsdGVkX1+WDvtveXlruWPSHE/0XLcHvxCT4uil0qKc/bMsOfx5OVIR1sQp/b/puMZN2l1kCYXabdec5p2BPS0hPhm2aEU5sGLNsG9GjsUDyKFse2nKqbyKBeIFD/u7GUbQlD/ncViOkElgA9O19xjE65A8Ncmxx2JCxtcYbbrwWqUarbPuP68x6kJBLfsrNeyjRMrnIcd+Lt/2g2dJhLsfCElBFnyx8sXUY96OJOHEdzfbx+ZikzJ/Vc9Z8aommGuCmXtrsZCC6O3pYFE7p6KMq0lbD1si/qeJlkQQkSaIDvraUcJCASzSmdEyZAXDTE//ZFgIYLNA90ZYyKKrqKwHLC96eDHDfqCo1UmnlzGYpIqsLrXok18TyFa3INo9wmbkEIXpSVMXtGmxlGeCD7Qst2UynOtaU9OUeTkwL4aJyhMojuUO1uKdMbHMUAC1Ahw8fMmMhol443dg17ZkTNdLf+4zfWOvtagxca7tSz3hSSADubIJZLiGaeVh4cHSdWVn7yWCVeLX2bo7A+o6QRBU/N5rk5R/M3jv6Nin83pA0Af95ePNXxEGWL/YBDR30aQGLDKKzvtQkBwIRx2Wcal3ktQ/OXhRFT3poQeQTx+FIpcN8iVEQZByPVpnN4aqTfPYVKWzEOX7iFqgSivZaq6p3n8S7r1teGMzMhvjFCOlW4g31vWdim4n+igqyIwsFgdOEoQtSd4JZv4EpgHy11KeGZEhSsrK8r076spqmEZQ704LeTvGoS+S/XvA+DPZeRdLzA6z4315d+r8IYoP9XojXr19BYaFn7XKpzosoKcZbo2/p2U5sodOGUuIV9bp0MzJibANO1ItK3WANOzobFPNgjpl4x/z6r2Sx4dXaNsBY02lrPjxvtx9vMNvhQZYH8WaDBNhLvwLKPgfH7JhxmWBv0PsB2OmGVsrWzZi6+g8L5DWpKjaQ1w+IhB1IuuQNJtjYS1xzXOSU0Uc0Us A3GSN7ob iVTaieZCCkPY6Ho3CGCE48sWOwUT6S4JHSkok7jmJ9JCzLAPKEXDFObGNVL3sK3Oif61t5oNDMzIFfucTZ0bOk8Zs8viB8pD+GMqXC/xdUAQ6ZCKHsy8SdgVendTjMKPglMOUmk8W4Ce/Mk6h0NK5EqCCg3kB8sjEOhSwkq/p0o1VROMjmcQNZhA8XwfmNG0vltp/OoFWxH20OSWHmLaiIcgCfOCw3flcbkeOvE5hb/Kq5N0oDN/4J8enQcBj4vaS7fST7qgULAUUPazqzDb+rhVZDXeX48rYG+JEJhEMUD1t3zt5V3+OtjAD471LFlnwqdNZWcIM6u44vniXgsta9oZWN3lo/KmCDJBTUiNewfa5coXRCGaP3UC6UsaGcY8XX2OgR/x5/cAcgOOgv6eOEPIzjpGNo6uWHvu08yuUDwHLm7ca44asv4YAHV/yf4FY9uN+fG0H6ftB7oHWSxg/mYUgSYA1RMZIT8/VlLiTaR49Mqc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: I found myself repeatedly looking up the implementation of these helpers while working on the code, which suggests they are not a helpful abstraction. Inline them. Signed-off-by: Johannes Weiner --- include/linux/compaction.h | 92 ---------------------------------- include/trace/events/mmflags.h | 4 +- mm/page_alloc.c | 30 ++++++----- 3 files changed, 19 insertions(+), 107 deletions(-) diff --git a/include/linux/compaction.h b/include/linux/compaction.h index 06eeb2e25833..7635e220215a 100644 --- a/include/linux/compaction.h +++ b/include/linux/compaction.h @@ -103,78 +103,6 @@ extern enum compact_result compaction_suitable(struct zone *zone, int order, extern void compaction_defer_reset(struct zone *zone, int order, bool alloc_success); -/* Compaction has made some progress and retrying makes sense */ -static inline bool compaction_made_progress(enum compact_result result) -{ - /* - * Even though this might sound confusing this in fact tells us - * that the compaction successfully isolated and migrated some - * pageblocks. - */ - if (result == COMPACT_SUCCESS) - return true; - - return false; -} - -/* Compaction has failed and it doesn't make much sense to keep retrying. */ -static inline bool compaction_failed(enum compact_result result) -{ - /* All zones were scanned completely and still not result. */ - if (result == COMPACT_COMPLETE) - return true; - - return false; -} - -/* Compaction needs reclaim to be performed first, so it can continue. */ -static inline bool compaction_needs_reclaim(enum compact_result result) -{ - /* - * Compaction backed off due to watermark checks for order-0 - * so the regular reclaim has to try harder and reclaim something. - */ - if (result == COMPACT_SKIPPED) - return true; - - return false; -} - -/* - * Compaction has backed off for some reason after doing some work or none - * at all. It might be throttling or lock contention. Retrying might be still - * worthwhile, but with a higher priority if allowed. - */ -static inline bool compaction_withdrawn(enum compact_result result) -{ - /* - * If compaction is deferred for high-order allocations, it is - * because sync compaction recently failed. If this is the case - * and the caller requested a THP allocation, we do not want - * to heavily disrupt the system, so we fail the allocation - * instead of entering direct reclaim. - */ - if (result == COMPACT_DEFERRED) - return true; - - /* - * If compaction in async mode encounters contention or blocks higher - * priority task we back off early rather than cause stalls. - */ - if (result == COMPACT_CONTENDED) - return true; - - /* - * Page scanners have met but we haven't scanned full zones so this - * is a back off in fact. - */ - if (result == COMPACT_PARTIAL_SKIPPED) - return true; - - return false; -} - - bool compaction_zonelist_suitable(struct alloc_context *ac, int order, int alloc_flags); @@ -193,26 +121,6 @@ static inline enum compact_result compaction_suitable(struct zone *zone, int ord return COMPACT_SKIPPED; } -static inline bool compaction_made_progress(enum compact_result result) -{ - return false; -} - -static inline bool compaction_failed(enum compact_result result) -{ - return false; -} - -static inline bool compaction_needs_reclaim(enum compact_result result) -{ - return false; -} - -static inline bool compaction_withdrawn(enum compact_result result) -{ - return true; -} - static inline void kcompactd_run(int nid) { } diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index 412b5a46374c..47bfeca4cf02 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -222,8 +222,8 @@ IF_HAVE_VM_SOFTDIRTY(VM_SOFTDIRTY, "softdirty" ) \ #define compact_result_to_feedback(result) \ ({ \ enum compact_result __result = result; \ - (compaction_failed(__result)) ? COMPACTION_FAILED : \ - (compaction_withdrawn(__result)) ? COMPACTION_WITHDRAWN : COMPACTION_PROGRESS; \ + (__result == COMPACT_COMPLETE) ? COMPACTION_FAILED : \ + (__result == COMPACT_SUCCESS) ? COMPACTION_PROGRESS : COMPACTION_WITHDRAWN; \ }) #define COMPACTION_FEEDBACK \ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 5f2a0037bed1..c3b7dc479936 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -4620,35 +4620,39 @@ should_compact_retry(struct alloc_context *ac, int order, int alloc_flags, if (fatal_signal_pending(current)) return false; - if (compaction_made_progress(compact_result)) + /* + * Compaction managed to coalesce some page blocks, but the + * allocation failed presumably due to a race. Retry some. + */ + if (compact_result == COMPACT_SUCCESS) (*compaction_retries)++; /* - * compaction considers all the zone as desperately out of memory - * so it doesn't really make much sense to retry except when the + * All zones were scanned completely and still no result. It + * doesn't really make much sense to retry except when the * failure could be caused by insufficient priority */ - if (compaction_failed(compact_result)) + if (compact_result == COMPACT_COMPLETE) goto check_priority; /* - * compaction was skipped because there are not enough order-0 pages - * to work with, so we retry only if it looks like reclaim can help. + * Compaction was skipped due to a lack of free order-0 + * migration targets. Continue if reclaim can help. */ - if (compaction_needs_reclaim(compact_result)) { + if (compact_result == COMPACT_SKIPPED) { ret = compaction_zonelist_suitable(ac, order, alloc_flags); goto out; } /* - * make sure the compaction wasn't deferred or didn't bail out early - * due to locks contention before we declare that we should give up. - * But the next retry should use a higher priority if allowed, so - * we don't just keep bailing out endlessly. + * If compaction backed due to being deferred, due to + * contended locks in async mode, or due to scanners meeting + * after a partial scan, retry with increased priority. */ - if (compaction_withdrawn(compact_result)) { + if (compact_result == COMPACT_DEFERRED || + compact_result == COMPACT_CONTENDED || + compact_result == COMPACT_PARTIAL_SKIPPED) goto check_priority; - } /* * !costly requests are much more important than __GFP_RETRY_MAYFAIL