From patchwork Mon Feb 15 16:13:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12088843 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79DDDC433DB for ; Mon, 15 Feb 2021 16:14:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0F87664DF2 for ; Mon, 15 Feb 2021 16:14:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0F87664DF2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AEB548D0125; Mon, 15 Feb 2021 11:14:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A9ACD8D00FD; Mon, 15 Feb 2021 11:14:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9AF8C8D0125; Mon, 15 Feb 2021 11:14:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0151.hostedemail.com [216.40.44.151]) by kanga.kvack.org (Postfix) with ESMTP id 862F18D00FD for ; Mon, 15 Feb 2021 11:14:00 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5112218020B2A for ; Mon, 15 Feb 2021 16:14:00 +0000 (UTC) X-FDA: 77820998640.28.A44CB9D Received: from mail-qk1-f182.google.com (mail-qk1-f182.google.com [209.85.222.182]) by imf30.hostedemail.com (Postfix) with ESMTP id A1BE3E0011C3 for ; Mon, 15 Feb 2021 16:13:55 +0000 (UTC) Received: by mail-qk1-f182.google.com with SMTP id z190so892776qka.9 for ; Mon, 15 Feb 2021 08:13:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+qKWgotgEdvGjys3/bE4u2jFoUM8emF2bpCTDvCGsH0=; b=R+eYrK+Q4M6RkrL1ulUsWY73UAPh+P5N6PGssDIfBpBYLtUATYaGEWy8uXiYym8kDA xk2KDpPBj1vD9bJi2xVKkQ37G2VGgOq6vXKv9he/G1i/6Y6HKCSwVsmxTs+ygAn3Nd3G NaMUtjrDZ8fS8mzmJNIsZy26QqOe0hHNyNkxX31JxxfR5oqiDiVb3FifgBzVDLpaFTFC cLr/SqRiNC8D6UdL8eCgXc/+YtTQyBNFElYqgoKgb1hsQUt+jZ0ZSKd2AUuScw4N/UsI MCeqbViLOfpAhvCncGC4Uam9wPr4Zk6U3zD+pupK/IstJepwbBYDgbtjYJPHhdZZffrQ jJew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+qKWgotgEdvGjys3/bE4u2jFoUM8emF2bpCTDvCGsH0=; b=SEuBFoYlJt+J1wzBdhMZ6qBhpjOR2GX+6d4eEMWphs18y9ioYT/g3grFRAOrEMLCs2 FQxYf45yVhDJlNOsKSOErXqgbzoHcejtMOFZezrXD+zV0R5ogPGQCSz0GH7bPURj1lqV o3KasLwCJ1W3dR1RJxTYVNSavMplOxJ+QpGew/khZ1gz1XRi8yaS0WlYHXqHSZe5OGV0 /lbsGKWJmqQt6XYGkHvtcgUwnpPaNgf01Sl2fX45YaDJ5Ruso/AqGcx8IAgCkPpGDsgj izgWD3q8MLEs7pSb+XJU7aOg8JBty0BFRl5sdpPa8aIn5VxSbKUhLFZQcdXhkxrw52c9 UPKQ== X-Gm-Message-State: AOAM530Ooi00eXQN/6S74jskTH9DQvIdW4HX89MCC/fJhMnbe4d713Db Amb4ruuu3vKhwTAB+aoutBdhmg== X-Google-Smtp-Source: ABdhPJwvXmPNyrexesKj606DgnJ/YOQ8iGFbuk+vK73ZOWobC75IMwhqBFGpQKXK1oJJxqostKAEXQ== X-Received: by 2002:a05:620a:985:: with SMTP id x5mr13605459qkx.403.1613405639233; Mon, 15 Feb 2021 08:13:59 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id u7sm10909213qta.75.2021.02.15.08.13.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Feb 2021 08:13:58 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, vbabka@suse.cz, mhocko@suse.com, david@redhat.com, osalvador@suse.de, dan.j.williams@intel.com, sashal@kernel.org, tyhicks@linux.microsoft.com, iamjoonsoo.kim@lge.com, mike.kravetz@oracle.com, rostedt@goodmis.org, mingo@redhat.com, jgg@ziepe.ca, peterz@infradead.org, mgorman@suse.de, willy@infradead.org, rientjes@google.com, jhubbard@nvidia.com, linux-doc@vger.kernel.org, ira.weiny@intel.com, linux-kselftest@vger.kernel.org, jmorris@namei.org Subject: [PATCH v11 05/14] mm cma: rename PF_MEMALLOC_NOCMA to PF_MEMALLOC_PIN Date: Mon, 15 Feb 2021 11:13:40 -0500 Message-Id: <20210215161349.246722-6-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210215161349.246722-1-pasha.tatashin@soleen.com> References: <20210215161349.246722-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A1BE3E0011C3 X-Stat-Signature: eet1uwi4pnux3zd1uisc1qe5wyinz7eg Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf30; identity=mailfrom; envelope-from=""; helo=mail-qk1-f182.google.com; client-ip=209.85.222.182 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1613405635-497789 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: PF_MEMALLOC_NOCMA is used ot guarantee that the allocator will not return pages that might belong to CMA region. This is currently used for long term gup to make sure that such pins are not going to be done on any CMA pages. When PF_MEMALLOC_NOCMA has been introduced we haven't realized that it is focusing on CMA pages too much and that there is larger class of pages that need the same treatment. MOVABLE zone cannot contain any long term pins as well so it makes sense to reuse and redefine this flag for that usecase as well. Rename the flag to PF_MEMALLOC_PIN which defines an allocation context which can only get pages suitable for long-term pins. Also re-name: memalloc_nocma_save()/memalloc_nocma_restore to memalloc_pin_save()/memalloc_pin_restore() and make the new functions common. Signed-off-by: Pavel Tatashin Reviewed-by: John Hubbard Acked-by: Michal Hocko --- include/linux/sched.h | 2 +- include/linux/sched/mm.h | 21 +++++---------------- mm/gup.c | 4 ++-- mm/hugetlb.c | 4 ++-- mm/page_alloc.c | 4 ++-- 5 files changed, 12 insertions(+), 23 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 6e3a5eeec509..0fbb03bb776e 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1568,7 +1568,7 @@ extern struct pid *cad_pid; #define PF_SWAPWRITE 0x00800000 /* Allowed to write to swap */ #define PF_NO_SETAFFINITY 0x04000000 /* Userland is not allowed to meddle with cpus_mask */ #define PF_MCE_EARLY 0x08000000 /* Early kill for mce process policy */ -#define PF_MEMALLOC_NOCMA 0x10000000 /* All allocation request will have _GFP_MOVABLE cleared */ +#define PF_MEMALLOC_PIN 0x10000000 /* Allocation context constrained to zones which allow long term pinning. */ #define PF_FREEZER_SKIP 0x40000000 /* Freezer should not count it as freezable */ #define PF_SUSPEND_TASK 0x80000000 /* This thread called freeze_processes() and should not be frozen */ diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h index 1ae08b8462a4..5f4dd3274734 100644 --- a/include/linux/sched/mm.h +++ b/include/linux/sched/mm.h @@ -270,29 +270,18 @@ static inline void memalloc_noreclaim_restore(unsigned int flags) current->flags = (current->flags & ~PF_MEMALLOC) | flags; } -#ifdef CONFIG_CMA -static inline unsigned int memalloc_nocma_save(void) +static inline unsigned int memalloc_pin_save(void) { - unsigned int flags = current->flags & PF_MEMALLOC_NOCMA; + unsigned int flags = current->flags & PF_MEMALLOC_PIN; - current->flags |= PF_MEMALLOC_NOCMA; + current->flags |= PF_MEMALLOC_PIN; return flags; } -static inline void memalloc_nocma_restore(unsigned int flags) +static inline void memalloc_pin_restore(unsigned int flags) { - current->flags = (current->flags & ~PF_MEMALLOC_NOCMA) | flags; + current->flags = (current->flags & ~PF_MEMALLOC_PIN) | flags; } -#else -static inline unsigned int memalloc_nocma_save(void) -{ - return 0; -} - -static inline void memalloc_nocma_restore(unsigned int flags) -{ -} -#endif #ifdef CONFIG_MEMCG DECLARE_PER_CPU(struct mem_cgroup *, int_active_memcg); diff --git a/mm/gup.c b/mm/gup.c index be57836ba90f..9af6faf1b2b3 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1662,7 +1662,7 @@ static long __gup_longterm_locked(struct mm_struct *mm, long rc; if (gup_flags & FOLL_LONGTERM) - flags = memalloc_nocma_save(); + flags = memalloc_pin_save(); rc = __get_user_pages_locked(mm, start, nr_pages, pages, vmas, NULL, gup_flags); @@ -1671,7 +1671,7 @@ static long __gup_longterm_locked(struct mm_struct *mm, if (rc > 0) rc = check_and_migrate_cma_pages(mm, start, rc, pages, vmas, gup_flags); - memalloc_nocma_restore(flags); + memalloc_pin_restore(flags); } return rc; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 4bdb58ab14cb..861de87daf07 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1049,10 +1049,10 @@ static void enqueue_huge_page(struct hstate *h, struct page *page) static struct page *dequeue_huge_page_node_exact(struct hstate *h, int nid) { struct page *page; - bool nocma = !!(current->flags & PF_MEMALLOC_NOCMA); + bool pin = !!(current->flags & PF_MEMALLOC_PIN); list_for_each_entry(page, &h->hugepage_freelists[nid], lru) { - if (nocma && is_migrate_cma_page(page)) + if (pin && is_migrate_cma_page(page)) continue; if (PageHWPoison(page)) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 519a60d5b6f7..e4b1eda87827 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3813,8 +3813,8 @@ static inline unsigned int current_alloc_flags(gfp_t gfp_mask, #ifdef CONFIG_CMA unsigned int pflags = current->flags; - if (!(pflags & PF_MEMALLOC_NOCMA) && - gfp_migratetype(gfp_mask) == MIGRATE_MOVABLE) + if (!(pflags & PF_MEMALLOC_PIN) && + gfp_migratetype(gfp_mask) == MIGRATE_MOVABLE) alloc_flags |= ALLOC_CMA; #endif