From patchwork Sat Feb 19 17:49:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 12752344 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C087C433F5 for ; Sat, 19 Feb 2022 17:49:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 58CCC6B0071; Sat, 19 Feb 2022 12:49:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 53C5C6B0072; Sat, 19 Feb 2022 12:49:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 405496B0074; Sat, 19 Feb 2022 12:49:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0001.hostedemail.com [216.40.44.1]) by kanga.kvack.org (Postfix) with ESMTP id 3357C6B0071 for ; Sat, 19 Feb 2022 12:49:46 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id E665B95B13 for ; Sat, 19 Feb 2022 17:49:45 +0000 (UTC) X-FDA: 79160267130.15.C03C4EC Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf02.hostedemail.com (Postfix) with ESMTP id 7958180003 for ; Sat, 19 Feb 2022 17:49:45 +0000 (UTC) Received: by mail-yb1-f202.google.com with SMTP id a19-20020a25ca13000000b0061db44646b3so17016192ybg.2 for ; Sat, 19 Feb 2022 09:49:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=g6NUY8tWVaq9hcc+7rvde17O72ywerB7Kw8Oqcv6Cb8=; b=O7hSka9m0Losb8V4/MYsnc1jyzFMyVcjtIBwf5aOuBQhnKI5PJ5SUJZLQDdthEGXkf oKvbQRTnBPz85wCj9iCvNyZAchcUULrl9UbaFw38VP4rek0jLNzck2VSqE5MzSBEAPvi BcURtgl4WRLsAhlnSTGJenIqDNWF5A7Gm7JxV/jj2nfv3Uox4GSokTaqUmW+Aq+vseEb 5Eb26fIhmLlqmRkqLFG3uUq4l9NHZz9KphHZD3NqexhGND7e2Hdrwjp8TrvZfIJyHSKC jtRSzKAd/wJq96JuQXDifhX/Kphnt6w5DUdeE8JawqjgxYAxwfKUw9we7FA/26OqIRRy Lx3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=g6NUY8tWVaq9hcc+7rvde17O72ywerB7Kw8Oqcv6Cb8=; b=5GKDIPwahp015/7E9b8vtEmh3AgoazflN8dpMz7Ur5gUbz7L3c0gnOyi4xewCiqlnG xEVqIjSUPxilP5osh/kyQIIAg56I68ssNdUdCoMjLOKUyOLcx9GfwEVpZl3cInlVts+z pPmmSvZ10lGrJ49oINxm5z0z2GNxCb/Fol1OUcZ4pzz2rLyxqcbodc80/fwDvexmzqOy Z7FOM1sjrlGI8iq316dtk+bLZeMmY6bKUjQY5/l/F+NBvvSs2nj8dEWSNrSuGDW0zOX8 6yIPkkP8mh6MJGbfXBQozAKTUNm65JRZ5k2Ehbj8dkbAzvL6zSkknu27rxnisLZLyXU3 /OOg== X-Gm-Message-State: AOAM533sBW/tO8/YOwMl4qjAKj3QsR1VGKQfqOiY2J+yfB+mNFpouqfX ZLW87d/XCAMRV9EnOxsxfz5fU6oBPDA= X-Google-Smtp-Source: ABdhPJwf14QjNvOBlEKfJtniVeJfLtC2EP2lzgT+pRF8F00YZc/vptpdnJGB4C1slL1oxsIeafyrTGtmROs= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:200:cbd0:4467:228e:5eea]) (user=surenb job=sendgmr) by 2002:a25:bdc9:0:b0:61d:dfc5:a755 with SMTP id g9-20020a25bdc9000000b0061ddfc5a755mr11864217ybk.357.1645292984665; Sat, 19 Feb 2022 09:49:44 -0800 (PST) Date: Sat, 19 Feb 2022 09:49:40 -0800 Message-Id: <20220219174940.2570901-1-surenb@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.35.1.473.g83b2b277ed-goog Subject: [PATCH 1/1] mm: count time in drain_all_pages during direct reclaim as memory pressure From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, mhocko@suse.com, peterz@infradead.org, guro@fb.com, shakeelb@google.com, minchan@kernel.org, timmurray@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com, surenb@google.com X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7958180003 X-Stat-Signature: rmmw6cd4ojwiwk8jojgjjtpf6hangi3d Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=O7hSka9m; spf=pass (imf02.hostedemail.com: domain of 3uC0RYgYKCG8fheRaOTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--surenb.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3uC0RYgYKCG8fheRaOTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-HE-Tag: 1645292985-622476 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When page allocation in direct reclaim path fails, the system will make one attempt to shrink per-cpu page lists and free pages from high alloc reserves. Draining per-cpu pages into buddy allocator can be a very slow operation because it's done using workqueues and the task in direct reclaim waits for all of them to finish before proceeding. Currently this time is not accounted as psi memory stall. While testing mobile devices under extreme memory pressure, when allocations are failing during direct reclaim, we notices that psi events which would be expected in such conditions were not triggered. After profiling these cases it was determined that the reason for missing psi events was that a big chunk of time spent in direct reclaim is not accounted as memory stall, therefore psi would not reach the levels at which an event is generated. Further investigation revealed that the bulk of that unaccounted time was spent inside drain_all_pages call. Annotate drain_all_pages and unreserve_highatomic_pageblock during page allocation failure in the direct reclaim path so that delays caused by these calls are accounted as memory stall. Reported-by: Tim Murray Signed-off-by: Suren Baghdasaryan Acked-by: Johannes Weiner --- mm/page_alloc.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3589febc6d31..7fd0d392b39b 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -4639,8 +4639,12 @@ __alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order, * Shrink them and try again */ if (!page && !drained) { + unsigned long pflags; + + psi_memstall_enter(&pflags); unreserve_highatomic_pageblock(ac, false); drain_all_pages(NULL); + psi_memstall_leave(&pflags); drained = true; goto retry; }