From patchwork Fri May 19 11:13:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13248237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5283C77B7A for ; Fri, 19 May 2023 11:14:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3939D900005; Fri, 19 May 2023 07:14:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 343F8900003; Fri, 19 May 2023 07:14:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20AD8900005; Fri, 19 May 2023 07:14:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 119C8900003 for ; Fri, 19 May 2023 07:14:04 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id D1879ADEBE for ; Fri, 19 May 2023 11:14:03 +0000 (UTC) X-FDA: 80806745166.12.D4ADD35 Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) by imf26.hostedemail.com (Postfix) with ESMTP id C64B114001C for ; Fri, 19 May 2023 11:14:01 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=yHVqeGy7; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf26.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.208.45 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684494842; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=4+L9HR/cxdd2EoQqIApS/F4mIL7RlSJb5MUrnsXMoGw=; b=yKn4Hf5EptRlnYjTFMj42CIw88pEReMpJbgxsODzOFfPgNJU/u25AsS4ALNlwvrCr9TWDA 9REe0Pto7ru+XVrmWLpN1cHOC4+lf91njZ/fokkabWT31aTJPDtiWuA1rrgTB5D/J+weVo QgvqDFrlMQ5EuLK5GiIobS0dFvgBLmc= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=yHVqeGy7; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf26.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.208.45 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684494842; a=rsa-sha256; cv=none; b=wvQeiN7uylZ2ATTHAGOwAoLwGqhylkSinf9+GbRm0ytZNGwPwq9c2sKypUhYTRhyC8ZS5M ojAn/ZZrSSE1NiDWQdfIDROV8cuKtS7amvdQN1sZAkoGlvzZahmgnbe070myUVYlZ/LFRm 96GJmVWVjAw2NNOC3D0kqth8VOMYEEI= Received: by mail-ed1-f45.google.com with SMTP id 4fb4d7f45d1cf-510d8ead2f1so4068875a12.3 for ; Fri, 19 May 2023 04:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20221208.gappssmtp.com; s=20221208; t=1684494840; x=1687086840; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=4+L9HR/cxdd2EoQqIApS/F4mIL7RlSJb5MUrnsXMoGw=; b=yHVqeGy7uesoSL052/4GNJ7wDGNjsh5p/SyHiocWBSC4ig0qJmsUMSFxZjUXQpG4II sx7bvOG1WgdahaajQFDCdQNo5lq+Cl3GzPX3bimadgE1GtrCtvkllj7lvNHXlEf11+dO FjHor6izgaxO2yKOeW8DZoHHbUaNHgnlV7vka9hwVyN0OidXMy0dIA0MB+2FOrflG1dv JgictY+692VK0L9C38hnFv0IPYcg1Zov2Le+udVdTPa4+XcrJYI5cQBn1BxNoCKLO+cO GsUWwvGgM4XQXNA4uxQ0MvYUisPfsvHohd3QnI/xMwqqccJQwg0gmpTlarm9RCHajqz1 rqsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684494840; x=1687086840; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=4+L9HR/cxdd2EoQqIApS/F4mIL7RlSJb5MUrnsXMoGw=; b=jTVOrfSjUla/XCe2IQSZLyQMF8DoA+J8wGyPZ5K0LUeU1vxlJoaVgl0+vdhG6DU2ok Na/Yv1g8zP1/eB9jcILNIHA/W4m893OCorCAn+7dreg2UfaT6TpbcwCVT8UM6rmtoU2Q Rq6e5CdT2XE/Sp3QZBxf3AX9XMcraelEemk5lcYSYYzvwragj66foPMdFfeQgRP03WOc 1gnkSWf9ZZsE5NRogp6s+VR/ZPkZIxrfbmueBVcPZH2qwYBuUf4r+wWacfq2c042Wa/K GhnSwB1D8IOQr9KPySMQLXzLZG4cuNzo4Wl6ia5MUnUHeKXd6VPnvFG4Aou1LtiKhIX0 Qp3Q== X-Gm-Message-State: AC+VfDxhfcMK6LFgKaI/255Uu69lL+VJ/VbSi6xZACu7w5ecjErg0GhB GLob3yleGBmrnrlNLwmmHUkZ0w== X-Google-Smtp-Source: ACHHUZ4FmEnNGPERWDF/ykElYe7/UICJF+x9kwyPXx762MMcpfOmOZUjIwoeNwbPHZr/+1tTVZw0hA== X-Received: by 2002:aa7:ce02:0:b0:510:d197:e873 with SMTP id d2-20020aa7ce02000000b00510d197e873mr1298299edv.31.1684494839920; Fri, 19 May 2023 04:13:59 -0700 (PDT) Received: from localhost ([2a02:8070:6389:7d40:e266:3092:9afb:a7b1]) by smtp.gmail.com with ESMTPSA id x23-20020aa7dad7000000b005067d089aafsm1536743eds.11.2023.05.19.04.13.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 May 2023 04:13:59 -0700 (PDT) From: Johannes Weiner To: Andrew Morton Cc: Mel Gorman , Vlastimil Babka , Michal Hocko , linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [PATCH] mm: compaction: avoid GFP_NOFS ABBA deadlock Date: Fri, 19 May 2023 13:13:59 +0200 Message-Id: <20230519111359.40475-1-hannes@cmpxchg.org> X-Mailer: git-send-email 2.40.0 MIME-Version: 1.0 X-Rspamd-Queue-Id: C64B114001C X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 4r4d6h1eha35yi54arkr4ea89j8n3kwm X-HE-Tag: 1684494841-100633 X-HE-Meta: U2FsdGVkX1+K2HWqK0NjHkzBFkPnCV3ByfIFn7XQtMJsH3oPfWc4tjEADihLxfSF0Jod+qv06tJ1QZeZoIas3X+bfS1i9jHwG3F7vmEuL0LCg3+yOEsHTmKdbG1Ngv5Mo+oZbo2gr1c4vpLXmztWRfhhSTXQmW0OL10Qy57AS++3v8BZ3GtRZ1yxTPn7p1Dj9Az6/I6r+UgFDsajrmmzNdFFq0mEPwNFEmYujPwCg75OAqB+aVUzw7j67W1Ky5TNGqgUb7nRgHN2daHAgeLDBYlahbQId2s0VEMb8CCr18/2puEjnyTY7/8vH3y1Ol0LowZw5j3uS0OWIOkFRdbkwbHDhkcgWZka/GzMSgJANhvkjk4zUcjP/cFjXTguYwd8lNOgkqKpEob0v9CTu4RKQbEzThKWa6zKMv2w51a57c4FWrFEU7Mf4PqlILzPGAGKDXv3/mwMvKskaJxTMevHXT7axyYHlWuOccuTs7w/g727Dl1J/b5bqO+iWhof/hJML4iXhoBTh/GlOOL2+AS163BVQGeW9jNdpeghBxYBKzMb57dXYoTJcv4nYz+tWU95uPRHNd4J15m9woBHgOVl4YvfKXfYxs7ShFujKkaMVEad5S3f/A1zhKWUN/IlCtwbPZEi4+SslV4S1Ct4OEkaxyejv8QC1WBG61vhz7Ws41ryHsiT96vokqvFD7PWpvUoXx18q4Arf7MQLraGfvPtqGubeMxpSJSCcjSNExOkjgX9495jST+OccQ1ms4KTgrdXLTDUFyitEl4PSchXxsfcYrabxX1hzhbnTpPwFRg5SxiX2kkMpWF5+SufVMOjatMn87OMtwGk6TIAO/Rjvo+hYQboAAO2/AKbWMb6k+P5XPeltb2MeD1ThSzFPgtIAwQzbHBb0gp3oQqO80ylxuYq7MKImew1Wu927SlVBdZQK3wjLbptS9eC0nZsj4tJyJG23oq7oRaNb04P5bZzyT Te3lSxf9 PpDdtTeJiQgT1kskswo11IZxezO0qYxvjwmOhCB2F5GDAXr+9PdSFEwBrHMHRL12Xm5V/SFPtuYNBPCK6evSNWQHmSuMRrmQlMVqf7HHejLK7FbMqsl/AAnbdmwmFZb3mxuwkVn4UsO1G+DYvSABiFLFSe0+ScXs89XgO62g2ndqilecndHgNV9+oSvIMlAOtNHTmYjzegMEirU/dXD8PSC2faNHm1MVJfDMqpbOx75rI5bq/jFAoU7wcs7pdWX/u3l7pu9GqXvBubGVu2pmawCyFbGmpkdE3UJkqjDkbz+BKUmuKuR43+FP5BkKZ6ybDqS/oFgZLw9QLrMRiEBCwRRWeboMrnUJIJMzvf8QprG2j0NhO8GbgD4yp6mqoIP6NCRmExCWiXzrQOD43JoqAaN9gDJDsXBjpgKqpKnF9fmw+cTPaqSLV0hKJJcOY4Zm9i+RObyvRFZDXbhXbjtKFpGhIog44d3mIyfzlxlqYQBLHk0XwuSkpbAiERp4JhUclnnQ5NrMs1dPOWDE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: During stress testing with higher-order allocations, a deadlock scenario was observed in compaction: One GFP_NOFS allocation was sleeping on mm/compaction.c::too_many_isolated(), while all CPUs in the system were busy with compactors spinning on buffer locks held by the sleeping GFP_NOFS allocation. Reclaim is susceptible to this same deadlock; we fixed it by granting GFP_NOFS allocations additional LRU isolation headroom, to ensure it makes forward progress while holding fs locks that other reclaimers might acquire. Do the same here. This code has been like this since compaction was initially merged, and I only managed to trigger this with out-of-tree patches that dramatically increase the contexts that do GFP_NOFS compaction. While the issue is real, it seems theoretical in nature given existing allocation sites. Worth fixing now, but no Fixes tag or stable CC. Signed-off-by: Johannes Weiner Acked-by: Vlastimil Babka Acked-by: Mel Gorman --- mm/compaction.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) v2: - clarify too_many_isolated() comment (Mel) - split isolation deadlock from no-contiguous-anon lockups as that's a different scenario and deserves its own patch diff --git a/mm/compaction.c b/mm/compaction.c index c8bcdea15f5f..c9a4b6dffcf2 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -745,8 +745,9 @@ isolate_freepages_range(struct compact_control *cc, } /* Similar to reclaim, but different enough that they don't share logic */ -static bool too_many_isolated(pg_data_t *pgdat) +static bool too_many_isolated(struct compact_control *cc) { + pg_data_t *pgdat = cc->zone->zone_pgdat; bool too_many; unsigned long active, inactive, isolated; @@ -758,6 +759,17 @@ static bool too_many_isolated(pg_data_t *pgdat) isolated = node_page_state(pgdat, NR_ISOLATED_FILE) + node_page_state(pgdat, NR_ISOLATED_ANON); + /* + * Allow GFP_NOFS to isolate past the limit set for regular + * compaction runs. This prevents an ABBA deadlock when other + * compactors have already isolated to the limit, but are + * blocked on filesystem locks held by the GFP_NOFS thread. + */ + if (cc->gfp_mask & __GFP_FS) { + inactive >>= 3; + active >>= 3; + } + too_many = isolated > (inactive + active) / 2; if (!too_many) wake_throttle_isolated(pgdat); @@ -806,7 +818,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, * list by either parallel reclaimers or compaction. If there are, * delay for some time until fewer pages are isolated */ - while (unlikely(too_many_isolated(pgdat))) { + while (unlikely(too_many_isolated(cc))) { /* stop isolation if there are still pages not migrated */ if (cc->nr_migratepages) return -EAGAIN;