From patchwork Mon May 8 23:40:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 13235215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01C1DC77B75 for ; Mon, 8 May 2023 23:41:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 600656B009B; Mon, 8 May 2023 19:41:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5B0C6280001; Mon, 8 May 2023 19:41:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 478016B009D; Mon, 8 May 2023 19:41:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 36B9C6B009B for ; Mon, 8 May 2023 19:41:06 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id CDC914010C for ; Mon, 8 May 2023 23:41:05 +0000 (UTC) X-FDA: 80768710890.03.1C84FE1 Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) by imf03.hostedemail.com (Postfix) with ESMTP id 2DDE520005 for ; Mon, 8 May 2023 23:41:03 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=NTno1hBV; spf=pass (imf03.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.222.181 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1683589264; a=rsa-sha256; cv=none; b=2f170nvHFa6+Fyjb0qj23ITTYEQYn5KJVUUvnAhiKAYbEvORrdz7fSWR4yP/F4QGXUrzng hRjvxsWzko4yXiVVx+CRp3mlsCAzvUIb44KDCdDTZhOA2Dhu/EEyVZ7JsscnL+Pm1rN+za 9ulVEe5wA6wlRIyQpnYYwbuywDq0HKU= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=NTno1hBV; spf=pass (imf03.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.222.181 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1683589264; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=gXyotAzP+VPxO7VVOPWj4wSwSUrdQsmucc2QXoXVneg=; b=cXdhOV+vrNaI0GmFnkP8fh/xKxlk8J22xmoChrjteLops5w2QeV5se1nY3pFNn78XdhFYS mRG3rx1F/+s5zS61/OwNDygVhlQikJ+WgoD7Z0x/OL3lyUYwXaD9Z8BwrB/m//N/xCxDQV EPHc21QhKxEBDlh0WKmQMqoWo+Oj44g= Received: by mail-qk1-f181.google.com with SMTP id af79cd13be357-757731a32ecso114738685a.0 for ; Mon, 08 May 2023 16:41:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; t=1683589263; x=1686181263; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=gXyotAzP+VPxO7VVOPWj4wSwSUrdQsmucc2QXoXVneg=; b=NTno1hBVsyYnrek/PMyx+m0tZ4K6xLpWDC5gWFZAj5BshzMxsP27OcOq9NTagyo2CV hGa+Qmu7sVB7GX5230dpYckispNBjEyXREQoEFQfu7B0uTZR9iDtw5oX/hbAYSci3Sz0 cqncSiXX0P+NmbipJgflFGAOCQ618y+daoGrNS5F2/b+IX1QZkmxqQjFcn8pG1+kv7uT gi0kJ7wnJlyDCkutOEAcs3/BgT2JHla/snfnbtSVw5EVkSbtucoLHsNhhsKNvjKCQIl9 MDAshjNqjW6Zp3oeQIqS+Jji2ZAQ7Skk7zk1AcMEa2Egpv3ts99audqtXnm/RJHqbaqW lKPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683589263; x=1686181263; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=gXyotAzP+VPxO7VVOPWj4wSwSUrdQsmucc2QXoXVneg=; b=RFj0lUyH0NB7bSKgqo6jl/9rQ3uioRY2pXdHqLlAFE61GyZ9Xzot1ghT4qpfj+JJsH ozMn7ZmYFJ+3+BIlg0w8sbt/SAmKDdEM3M04Nx+MUlki1S37f5ngOFHC3RTNgjK2SsFb 3vyfsEkwPsP5QHfDb3dwNr3iCYzH+RIxQeqO6T1zGUnTOStYt0Zr59Up3IM3T/yAv8kF 8igBny7GKycgWjuZITKpN+dnbSpDYiK4PUbkI4EE4iFxRJWH1J2IdRlfa0px+iCB8iu7 FR/Gs1cojh3UgskdCGRTQsq5ANriPy/0hODo83RBihkGNAMcqENBWqNy44VU4/TRawtc 6j5w== X-Gm-Message-State: AC+VfDylRIMTOJAfMowLRK0kDDBkxNb2e1Qttx0O9QGpoEV3Elvs9vLN 635l5SeX4MYHvI/QuYBlejOt5Q== X-Google-Smtp-Source: ACHHUZ5K/IMkHlHxC6ISgy6AEyScv8HjSVfJbJ1Lzy2TT/TOP0WrdHo7hSGOMDmQcFxJTwRpeksFKg== X-Received: by 2002:a05:622a:1708:b0:3ef:3f51:dad5 with SMTP id h8-20020a05622a170800b003ef3f51dad5mr18505310qtk.20.1683589263162; Mon, 08 May 2023 16:41:03 -0700 (PDT) Received: from soleen.c.googlers.com.com (193.132.150.34.bc.googleusercontent.com. [34.150.132.193]) by smtp.gmail.com with ESMTPSA id j29-20020a05620a001d00b0074411b03972sm1357415qki.51.2023.05.08.16.41.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 May 2023 16:41:02 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, mike.kravetz@oracle.com, mhocko@suse.com, muchun.song@linux.dev, rientjes@google.com, souravpanda@google.com Subject: [PATCH v3] mm: hugetlb_vmemmap: provide stronger vmemmap allocation guarantees Date: Mon, 8 May 2023 23:40:59 +0000 Message-ID: <20230508234059.2529638-1-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.40.1.521.gf1e218fcd8-goog MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 2DDE520005 X-Stat-Signature: y4jb33x6jo77tg6ns17gazthk1cxq3jm X-HE-Tag: 1683589263-326514 X-HE-Meta: U2FsdGVkX18JjZEcyiO8ettgxpae+fCtyRPJksF6NwYMBxOPpsQJ7i8c5Pq6AcWR0/5DRkZic4V40s1pdympgOnwD81gNlpbA7fhABa3ALMyVr1HLqZaLFaFWojvIYc1Hfrz8/uZ8eDPeVL02b/wTsw4xVx7/KL6cf1e1eOVqxw28Ag8BYy7A0aaOxvQs5X9RPl8mj0MLdlzpkV0KeE3XaGD4SG2EgiWx8spe6A2VnGq8TI9xLcAkKxfyoRhF3Li6MC4C6x4x1eKj/aarxwNbJ4ehkyGdOi0KqQ83KzR2YBHbyvbyH1Dm1ahsVrIxMpCPznq20STU8/2LKgvFDAHFvS+OrTQDJZWYI82GOayBkh9KC+f+ryE17vuenvysatdpulE4uQMPFsEmkh3eoGVTWbuQV5/P1PQGWFkWPRUJK9VE0kwiQ/350Q7qRnPwfwtfbiJQtrATobbbJefW+S3j1aNc2hBJvqoIAI2sYUwrj9FERirCg8nNebos/7dJJU3JrpN9nMTEimrA9ZhbrGwb+VJRcRECZQ8ceZLwgxQocbKyGCqcnGQpSsUUVVQJR8BwjfBtsVdfiCkd7XzWca3K7Zn3IZLJtQtqZd0i0yW6/+sHgF1k/cidHKdgADKUirV0QPWrdgR09Grspu34qBkFHdqSL//azPgbEcWAG9IswWvnltmAVEqk0iaQHl6ncBHEnpT/18KBCULZpoEaJB2blVFVxZN1nh/vJIgeegbC4MABfurwBNCpUhITf4ZtI9ukCYOQMnRE8czZ0fZ4sdyast6smuubo1Ao61CC9HlrvD1SgzP797PBbfgfMV9qT02Sp0HY40c82Pe1/J9MErFGC/XZmCuw2ZKLEdCkQNGNaO0B0GZfuMWw2rRyMzkSn0jwN82v+MC7a7+yCFyOiFUCDjE/MPFERcDJQJX9ZDGDcVT8qnKzp3/VnhSjxVAD40xH2fhgER/GJu7MWuA4st x8UjGIxD R7AhIRr+SYVbtAEctahOfIVX+Vkz02b1ul46/GLaK8stPfxRX45+P8h+29TyxqV7Liray7dS5ZaqlcJ/9a62mgzipNOokut06sSWrOr9dnTLBPfTxC8KHFeOSk0ZEzBcAENjTT5KfcWS2z8PMeuc9HrRxAP0R/ifdEnSqQZhFj4RogdrwLEIWNjdqaGc8co5hwz8BJZM/Yjg8C4Y6P4WKP7c/cx3tcTTh5UQeZxOzs5WQ4k6S9CQGjsswlTN1QqXGnpDhaQzQmZDp+32cIfaAsnRd16uHtXublPJhnSUERFrzUZMP9w+RYTQpUhTqNNEj2gaTKnSwWCdJki/MyzHjs5gpxmfjFfAX4wQOe8uIQK9tUEAxO9EajEmtw8b2dd1rxFYyz39G4CfiiQmAMhmSJFN4ZU7Z+JJJsHtnR2UH46R1t1INakDBQdpQS6PgJSXI0Nr4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: HugeTLB pages have a struct page optimizations where struct pages for tail pages are freed. However, when HugeTLB pages are destroyed, the memory for struct pages (vmemmap) need to be allocated again. Currently, __GFP_NORETRY flag is used to allocate the memory for vmemmap, but given that this flag makes very little effort to actually reclaim memory the returning of huge pages back to the system can be problem. Lets use __GFP_RETRY_MAYFAIL instead. This flag is also performs graceful reclaim without causing ooms, but at least it may perform a few retries, and will fail only when there is genuinely little amount of unused memory in the system. Freeing a 1G page requires 16M of free memory. A machine might need to be reconfigured from one task to another, and release a large number of 1G pages back to the system if allocating 16M fails, the release won't work. Signed-off-by: Pasha Tatashin Suggested-by: David Rientjes Reviewed-by: Mike Kravetz --- mm/hugetlb_vmemmap.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) Changelog: v3 - updated patch log to include details about when the failure can happen. v2 - removed gfp_mask argument from alloc_vmemmap_page_list as suggested by David Rientjes. - Fixed spelling in the patch title. diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 27f001e0f0a2..f42079b73f82 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -384,8 +384,9 @@ static int vmemmap_remap_free(unsigned long start, unsigned long end, } static int alloc_vmemmap_page_list(unsigned long start, unsigned long end, - gfp_t gfp_mask, struct list_head *list) + struct list_head *list) { + gfp_t gfp_mask = GFP_KERNEL | __GFP_RETRY_MAYFAIL | __GFP_THISNODE; unsigned long nr_pages = (end - start) >> PAGE_SHIFT; int nid = page_to_nid((struct page *)start); struct page *page, *next; @@ -413,12 +414,11 @@ static int alloc_vmemmap_page_list(unsigned long start, unsigned long end, * @end: end address of the vmemmap virtual address range that we want to * remap. * @reuse: reuse address. - * @gfp_mask: GFP flag for allocating vmemmap pages. * * Return: %0 on success, negative error code otherwise. */ static int vmemmap_remap_alloc(unsigned long start, unsigned long end, - unsigned long reuse, gfp_t gfp_mask) + unsigned long reuse) { LIST_HEAD(vmemmap_pages); struct vmemmap_remap_walk walk = { @@ -430,7 +430,7 @@ static int vmemmap_remap_alloc(unsigned long start, unsigned long end, /* See the comment in the vmemmap_remap_free(). */ BUG_ON(start - reuse != PAGE_SIZE); - if (alloc_vmemmap_page_list(start, end, gfp_mask, &vmemmap_pages)) + if (alloc_vmemmap_page_list(start, end, &vmemmap_pages)) return -ENOMEM; mmap_read_lock(&init_mm); @@ -476,8 +476,7 @@ int hugetlb_vmemmap_restore(const struct hstate *h, struct page *head) * When a HugeTLB page is freed to the buddy allocator, previously * discarded vmemmap pages must be allocated and remapping. */ - ret = vmemmap_remap_alloc(vmemmap_start, vmemmap_end, vmemmap_reuse, - GFP_KERNEL | __GFP_NORETRY | __GFP_THISNODE); + ret = vmemmap_remap_alloc(vmemmap_start, vmemmap_end, vmemmap_reuse); if (!ret) { ClearHPageVmemmapOptimized(head); static_branch_dec(&hugetlb_optimize_vmemmap_key);