From patchwork Mon Feb 15 16:13:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12088845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2B38C43331 for ; Mon, 15 Feb 2021 16:14:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 78B6664E34 for ; Mon, 15 Feb 2021 16:14:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 78B6664E34 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5429C8D0126; Mon, 15 Feb 2021 11:14:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 51DF18D00FD; Mon, 15 Feb 2021 11:14:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E6CD8D0126; Mon, 15 Feb 2021 11:14:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0050.hostedemail.com [216.40.44.50]) by kanga.kvack.org (Postfix) with ESMTP id EE7F28D00FD for ; Mon, 15 Feb 2021 11:14:02 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id B5CCF81F3 for ; Mon, 15 Feb 2021 16:14:02 +0000 (UTC) X-FDA: 77820998724.21.wax68_1507b262763c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id 8E7C018023840 for ; Mon, 15 Feb 2021 16:14:02 +0000 (UTC) X-HE-Tag: wax68_1507b262763c X-Filterd-Recvd-Size: 5762 Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Mon, 15 Feb 2021 16:14:01 +0000 (UTC) Received: by mail-qk1-f181.google.com with SMTP id t63so6849434qkc.1 for ; Mon, 15 Feb 2021 08:14:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=p3LY61RMvEkOPbLRzp9wTQQd7Y9CKIynM/s58sr6Xh8=; b=BNcGqGP+5P/yEVtn1w00Xz2Dq3FRjNmwC/PCJblRo5T/nPxHmGtCPYHviMTXfgudEu IvEopPQNUYzy7y77LTOW2htxPKl3cFcDMV9sbJ4hw1wD+0tm5cs5UHu7LIktPJVYx49t 5ovwtmG43xfEipv14T2sORMGyYsEfiP6sUSgclCiyKdttMwcByulGWoZ9yZmDE9aKPtS Z9WEmwLUdSzVpZDc3KFy/bN67hDjS+KCTH9cbZ4tqmEVsF3yjJVA/0DhlXMbKsfC8un7 FFnSsvgRERv849iVWpple0i6I7NsL41pyJ0eGcfH0eAB3wLrb906ovsgMPzNBZ15bzAC 0UBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=p3LY61RMvEkOPbLRzp9wTQQd7Y9CKIynM/s58sr6Xh8=; b=oDG4STPya+5DjZyXhHxQu/4t2P1fBUQWNdL4l8Cn4ZxazmBjOyg/rKxs50ioIEWBkD WfG8MZyvLU5GeM7WTVEbdA/lV5PuUg5QBxujDfpOyHGt6qRTzvovCVZOz5eLeGwfyZb5 n6EzQZUcMhRfKej7b0mRbQw6a6oSYCuRTnPXDfuCJ/NSp9UyJBy61jmHvVPwMRatX1pO iA/chzkYdXdl7XBUqUNHPwsZt2t2CqGPR6xNukEvrOsWnYUx7FmCYrs8VAT6O5yniK0g CXzjq+b/sXqU1ZVzxXIm3Z/nNKO9WendPslAjFlr9e4t2vgGjOoz3SmDIRutGYGDrDAs +bBw== X-Gm-Message-State: AOAM530sZW7srLDFxIA8wrg1MKaybvsY3DIEHB0ZSUwq3yLoe1lmcL2h zFYyVSmLQxMs6IxR36nGin7Opw== X-Google-Smtp-Source: ABdhPJzk2DXr5/nGv//pmWIhQ1tujT4HU+iuHKQUbcp2PsdKdtw7Upgx021BNudhTqEeWym5AAV38A== X-Received: by 2002:a05:620a:908:: with SMTP id v8mr16047359qkv.201.1613405640836; Mon, 15 Feb 2021 08:14:00 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id u7sm10909213qta.75.2021.02.15.08.13.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Feb 2021 08:14:00 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, vbabka@suse.cz, mhocko@suse.com, david@redhat.com, osalvador@suse.de, dan.j.williams@intel.com, sashal@kernel.org, tyhicks@linux.microsoft.com, iamjoonsoo.kim@lge.com, mike.kravetz@oracle.com, rostedt@goodmis.org, mingo@redhat.com, jgg@ziepe.ca, peterz@infradead.org, mgorman@suse.de, willy@infradead.org, rientjes@google.com, jhubbard@nvidia.com, linux-doc@vger.kernel.org, ira.weiny@intel.com, linux-kselftest@vger.kernel.org, jmorris@namei.org Subject: [PATCH v11 06/14] mm: apply per-task gfp constraints in fast path Date: Mon, 15 Feb 2021 11:13:41 -0500 Message-Id: <20210215161349.246722-7-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210215161349.246722-1-pasha.tatashin@soleen.com> References: <20210215161349.246722-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Function current_gfp_context() is called after fast path. However, soon we will add more constraints which will also limit zones based on context. Move this call into fast path, and apply the correct constraints for all allocations. Also update .reclaim_idx based on value returned by current_gfp_context() because it soon will modify the allowed zones. Note: With this patch we will do one extra current->flags load during fast path, but we already load current->flags in fast-path: __alloc_pages_nodemask() prepare_alloc_pages() current_alloc_flags(gfp_mask, *alloc_flags); Later, when we add the zone constrain logic to current_gfp_context() we will be able to remove current->flags load from current_alloc_flags, and therefore return fast-path to the current performance level. Suggested-by: Michal Hocko Signed-off-by: Pavel Tatashin Acked-by: Michal Hocko --- mm/page_alloc.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e4b1eda87827..f6058e8f3105 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -4981,6 +4981,13 @@ __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, int preferred_nid, } gfp_mask &= gfp_allowed_mask; + /* + * Apply scoped allocation constraints. This is mainly about GFP_NOFS + * resp. GFP_NOIO which has to be inherited for all allocation requests + * from a particular context which has been marked by + * memalloc_no{fs,io}_{save,restore}. + */ + gfp_mask = current_gfp_context(gfp_mask); alloc_mask = gfp_mask; if (!prepare_alloc_pages(gfp_mask, order, preferred_nid, nodemask, &ac, &alloc_mask, &alloc_flags)) return NULL; @@ -4996,13 +5003,7 @@ __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, int preferred_nid, if (likely(page)) goto out; - /* - * Apply scoped allocation constraints. This is mainly about GFP_NOFS - * resp. GFP_NOIO which has to be inherited for all allocation requests - * from a particular context which has been marked by - * memalloc_no{fs,io}_{save,restore}. - */ - alloc_mask = current_gfp_context(gfp_mask); + alloc_mask = gfp_mask; ac.spread_dirty_pages = false; /*