From patchwork Tue Sep 10 23:43:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ackerley Tng X-Patchwork-Id: 13799478 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85F99EE01F1 for ; Tue, 10 Sep 2024 23:44:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 692378D00D0; Tue, 10 Sep 2024 19:44:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 618E28D0002; Tue, 10 Sep 2024 19:44:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 46BA38D00D0; Tue, 10 Sep 2024 19:44:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 2622B8D0002 for ; Tue, 10 Sep 2024 19:44:45 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B1671140D89 for ; Tue, 10 Sep 2024 23:44:44 +0000 (UTC) X-FDA: 82550460888.12.2FE5596 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) by imf14.hostedemail.com (Postfix) with ESMTP id E563410000D for ; Tue, 10 Sep 2024 23:44:42 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=fCwOAY2L; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of 36dngZgsKCF468GANHAUPJCCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--ackerleytng.bounces.google.com designates 209.85.210.202 as permitted sender) smtp.mailfrom=36dngZgsKCF468GANHAUPJCCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--ackerleytng.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726011806; a=rsa-sha256; cv=none; b=AlR/cPMrFghxl2k4DlluP1EAcponXPKD5tCozuPY12PCQ6QVpvYshRvRw/QA1hun39FIk/ 3CoVnnJs3wW92tuubpGEwnu5a2OcW8Rp7oV/6GYUsbCSQHfzf7sabnXSvqtWZeM9AEoQTJ 5W3pShm9j6TUpURrgozfoCotiq5MxWM= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=fCwOAY2L; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of 36dngZgsKCF468GANHAUPJCCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--ackerleytng.bounces.google.com designates 209.85.210.202 as permitted sender) smtp.mailfrom=36dngZgsKCF468GANHAUPJCCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--ackerleytng.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726011806; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=P0FEJ4ZKBglbUpSmnWaSSUOycb+KTZhx2f+dpCTvB7Q=; b=32ckbhdP8JCOoEjlp0OjHH78hakaEeIZSUkmcuXYBfFth48uUY4E9hqD3EyeunZ+KlRPmY WFiTBjjYFAhfzD3WxqZaGeY7xpBFFiV8YT3IDXbJ8yF/EfvVhbf9LIekj/TGDTr2dqims3 W6XTQEOTQj5gjRSOEJgpHAIgjKMOr0A= Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-718d6ad6105so6971347b3a.1 for ; Tue, 10 Sep 2024 16:44:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1726011881; x=1726616681; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=P0FEJ4ZKBglbUpSmnWaSSUOycb+KTZhx2f+dpCTvB7Q=; b=fCwOAY2LGkI9fFyAHnv9tf1wUa+0rE5IPmcPkvyYU4FCf6MIhJHwZ48HIaYsiXIKIN Gijddo9ysM4fZ9fn6feQn0TFKMsLR2e4UM86nK05MxMPMqvp9q2WQhQw/Z/fGEWUOocB rbLMGDn4ztWjfwZys35XC98CNT7rOKfKMUkgbDqwgMWdjxsQ+/cblq+6R6BggxI3sDNh 6IjR7y9ObDSrXZ2u1RBvOkI5/YenAEDdpoA5+kj/rIlts5tFjIGeF9U6fuiJWtl+pAbt fm10p75AkvxxUwZe01hMD8iVxacZ0DWoSS34Inlq0+AKlx1LNG6zet/EaxlUB1vYIpNQ piyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726011881; x=1726616681; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=P0FEJ4ZKBglbUpSmnWaSSUOycb+KTZhx2f+dpCTvB7Q=; b=hdoYoJZCpkSdfnooEys3MUg046FOFYw29d7tPGbKKRd7BletfKjxispEQHiyj3O5+P BQqhBQyYefsu7+sWupLNMfskEanv5DKFNrfIb0HfF2NAgAX4lJdB010aiqhqXKAur7pk 3Q8iwiXZD6j16l7OKrhoTU4l+ImyDDj7UKDR9Mga86FChIfCyINgzfI1CBnkW9RKxa6V glQ5pkpkC6Fb6Lic5f5InEI43D8wYOOhwqsvQwRfKmaG2hjK5ZB+ad1OMoVpfZyUB7X1 q0eEJrxqSIqW878HQhpYzYfdbXYmYZfj7x+xOOtoAx8u8JXOfzS30aYOENFVsAXvonz4 hE2g== X-Forwarded-Encrypted: i=1; AJvYcCXrZunkTNTR87ZTA92T1kf8kMoasby8ma3DHJHhaP1JV9vkQT01VK/VtP5kLQUaVTHyot/sGbLDKA==@kvack.org X-Gm-Message-State: AOJu0YwbtXok7dX9JSDSFStqES49RBSIV1rLXv4ixlie6Fkdl2h7p5PN EO5j/eSS0keGAIH8EnyxeDjdUdvfbr8lnNtaGBnNH1aVnPQvrdd9h3cZ0gtTMR04Nmnw0/Ndg0z v6JvjwPbuZSWVONSMTvgMDA== X-Google-Smtp-Source: AGHT+IHx4+TfNxnFVDcFxBFG2SogMutruOf/RcFp6adNqekz63WzrINlsJVnsUCpFS0h+PHdTGU8HSxvhRc6JvhHRw== X-Received: from ackerleytng-ctop.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:13f8]) (user=ackerleytng job=sendgmr) by 2002:a62:ab02:0:b0:710:9d5e:4b9a with SMTP id d2e1a72fcca58-718d5e04dadmr37747b3a.2.1726011881266; Tue, 10 Sep 2024 16:44:41 -0700 (PDT) Date: Tue, 10 Sep 2024 23:43:36 +0000 In-Reply-To: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.46.0.598.g6f2099f65c-goog Message-ID: <1778a7324a1242fa907981576ebd69716a94d778.1726009989.git.ackerleytng@google.com> Subject: [RFC PATCH 05/39] mm: hugetlb: Refactor alloc_buddy_hugetlb_folio_with_mpol() to interpret mempolicy instead of vma From: Ackerley Tng To: tabba@google.com, quic_eberman@quicinc.com, roypat@amazon.co.uk, jgg@nvidia.com, peterx@redhat.com, david@redhat.com, rientjes@google.com, fvdl@google.com, jthoughton@google.com, seanjc@google.com, pbonzini@redhat.com, zhiquan1.li@intel.com, fan.du@intel.com, jun.miao@intel.com, isaku.yamahata@intel.com, muchun.song@linux.dev, mike.kravetz@oracle.com Cc: erdemaktas@google.com, vannapurve@google.com, ackerleytng@google.com, qperret@google.com, jhubbard@nvidia.com, willy@infradead.org, shuah@kernel.org, brauner@kernel.org, bfoster@redhat.com, kent.overstreet@linux.dev, pvorel@suse.cz, rppt@kernel.org, richard.weiyang@gmail.com, anup@brainfault.org, haibo1.xu@intel.com, ajones@ventanamicro.com, vkuznets@redhat.com, maciej.wieczor-retman@intel.com, pgonda@google.com, oliver.upton@linux.dev, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kvm@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-fsdevel@kvack.org X-Rspamd-Queue-Id: E563410000D X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 9qkizn17cwzyjq9dbynbu134934fkobg X-HE-Tag: 1726011882-418113 X-HE-Meta: U2FsdGVkX1/ATPCHL+9HEPPaOTGKcSZgxlEgH38aJzRgVoiqKRpItzeTB63NcZ/0tkZbBrI6v6G5s/Nx41oDP49Ju4tHLY+Ipor5UbCIG4aRozrd9FbMl2DtTF9zAvq3JPmrQYetH48YmbgrQsJB4i1kz7Jcfql6kqusPLE5FgQdGskRIZHOyFJQ6awqouVruL9wxvWCeckzR0EkYUKRHOKIf4xmaYZ5mjp/hSBFjzcpyXLdPeqvBGoldIUUHKzSHrC54H28h11eOTImViZsNgyQaE7+h3MP1Sfhdb2/g7bU/IwniG6lQdjjus2kDuKUG8sI03k6KTa/robJzzCJEThE4zEM+xae2GbqP8OHM/jSvirQBhrLmkCTwBGqaDRzHK3S9ZqYpUPM8q6kFLcwXJZ2jvDzlcIk/nqh7z+Lwh3FiSz9HybQRIz/aLSi3Z61wlTVCkoqAa9z0+apoOHcPloE30h1X4GjdZo0VAwpg+90TXN5AV//jVQRzUkk1SQuAmtfOhH6ONn9Bp1E9iqW2aObZUEfMdm3zkpGXxMuZw+eYECmGD9/gsQadTZps3N3Y/VpUyCPrF1ogHHZ6zyPFF6YFCLyPhbzJWCAVZ7p15HoXzkM0CwRT+mSxig2U39m37BwFV8zH1XoKY1cCeoAq8PN9KDREJtm97B3HmmQIW5EJlg0LtNjcWGFrKi0S0w3FkYIqXrGOa7u1QqO4QZ7jINqIKWAmOzPr7gnIix5bUwOxv4/C2C1jN2NfjB8o5TaIH9wv5cy+1QJL14ni7gA/e8Bd8vIUem50k0y8ZZ+IFAKwO5S6TmJsD5vBJ8dz1F9BpyVtHkQp+Fusx96/+arHiTh8ltPRI9BqUFXU16jd68CnobTUVcOujpCLds30C8kNPAj/hrIfQkXsfuYLiskdFaDBEvsBTCwaPgpNMsTPk7DR6gc7GbGy1yaaEtrRHp53W80E/4pCwsW++XPEaO W1U0jVGM ywTcMvJD8CIe4/H1RetLuyptkuf4hmTXrdnGVSfCmDxLEuZjACMU+ZnxntozEICGs7XZoj/to8BrkIjIqqWIeAeic5Y40IuWuMl98g3vkdK0q4mAKD4fNGid6Q0OA4WPyzKv8HjfwdEdB2v499ZqqCRy3mmmwyWRFC8t2QU9lSf10MqL4Cv6zNwlye8eTZtgc3kJV/gDZmIEPXLltrRnTKV4Hf/uC8nytxjrTtTi+/20fVf0rUICtoi4srwkdMlfHNQZ/2IcuAhWCHFnO1xINuH2erPt8LaiYgeronzKVvRJDZxsYFuDfrMRcPsFIKcD2/osJIPuw7TVn0zXUFck35x7zbfZKKRaSaJ8FrmhPFH/T2tE5UMQj1c819C/fjW65t24KEiZnFXZExKbqNMbK1KDVjx58cwDHTNKHsicHTqkfDUv4KM5EjhMgi+jZ4lyGvg/W3mA4c5klheFr/KyxXtdTJ7npUG7zdjJ78Mh2nkSet6vRSySe5jZkYXzS58fhUdMIGWxRDlx8Piy1kkx7fz00Kn29LXObbMI5NSxg3xt8ntjHo8eGcRaIY8MCBTtPMGTcI/jfZUFmegU+VpR3bktmQDGfxpmGXpLS+f0LgMfDQe8ofp4g4XPnLlFHuBWR8E3aLJ62TFIwByJhuKdEIBCySG3Xsm3IzQdsMKXLSXJFDZBZLT8gbLWFfCQTO4P1yGWO1j6uxvd1V7Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Reducing dependence on vma avoids the hugetlb-specific assumption of where the mempolicy is stored. This will open up other ways of using hugetlb. Signed-off-by: Ackerley Tng --- mm/hugetlb.c | 37 +++++++++++++++++++++++-------------- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 5cf7fb117e9d..2f2bd2444ae2 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2536,32 +2536,31 @@ static struct folio *alloc_migrate_hugetlb_folio(struct hstate *h, gfp_t gfp_mas } /* - * Use the VMA's mpolicy to allocate a huge page from the buddy. + * Allocate a huge page from the buddy allocator, given memory policy, node id + * and nodemask. */ -static -struct folio *alloc_buddy_hugetlb_folio_with_mpol(struct hstate *h, - struct vm_area_struct *vma, unsigned long addr) +static struct folio *alloc_buddy_hugetlb_folio_from_node(struct hstate *h, + struct mempolicy *mpol, + int nid, + nodemask_t *nodemask) { - struct folio *folio = NULL; - struct mempolicy *mpol; gfp_t gfp_mask = htlb_alloc_mask(h); - int nid; - nodemask_t *nodemask; + struct folio *folio = NULL; - nid = huge_node(vma, addr, gfp_mask, &mpol, &nodemask); if (mpol_is_preferred_many(mpol)) { gfp_t gfp = gfp_mask | __GFP_NOWARN; gfp &= ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL); folio = alloc_surplus_hugetlb_folio(h, gfp, nid, nodemask); + } - /* Fallback to all nodes if page==NULL */ + if (!folio) { + /* Fallback to all nodes if earlier allocation failed */ nodemask = NULL; - } - if (!folio) folio = alloc_surplus_hugetlb_folio(h, gfp_mask, nid, nodemask); - mpol_cond_put(mpol); + } + return folio; } @@ -3187,8 +3186,18 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, spin_lock_irq(&hugetlb_lock); folio = dequeue_hugetlb_folio_vma(h, vma, addr, use_hstate_resv); if (!folio) { + struct mempolicy *mpol; + nodemask_t *nodemask; + pgoff_t ilx; + int nid; + spin_unlock_irq(&hugetlb_lock); - folio = alloc_buddy_hugetlb_folio_with_mpol(h, vma, addr); + + mpol = get_vma_policy(vma, addr, hstate_vma(vma)->order, &ilx); + nid = policy_node_nodemask(mpol, htlb_alloc_mask(h), ilx, &nodemask); + folio = alloc_buddy_hugetlb_folio_from_node(h, mpol, nid, nodemask); + mpol_cond_put(mpol); + if (!folio) goto out_uncharge_cgroup; spin_lock_irq(&hugetlb_lock);