From patchwork Mon Oct 26 14:51:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11857481 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 947066A2 for ; Mon, 26 Oct 2020 14:54:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5168720773 for ; Mon, 26 Oct 2020 14:54:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=bytedance-com.20150623.gappssmtp.com header.i=@bytedance-com.20150623.gappssmtp.com header.b="wHxCia5N" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5168720773 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7B4326B0083; Mon, 26 Oct 2020 10:54:58 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 763856B0085; Mon, 26 Oct 2020 10:54:58 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5DD5C6B0087; Mon, 26 Oct 2020 10:54:58 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0201.hostedemail.com [216.40.44.201]) by kanga.kvack.org (Postfix) with ESMTP id 2D4736B0083 for ; Mon, 26 Oct 2020 10:54:58 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id AC882824999B for ; Mon, 26 Oct 2020 14:54:57 +0000 (UTC) X-FDA: 77414373834.21.nest81_0f0f26e27274 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id 83E33180442C3 for ; Mon, 26 Oct 2020 14:54:57 +0000 (UTC) X-Spam-Summary: 1,0,0,09901e513620a440,d41d8cd98f00b204,songmuchun@bytedance.com,,RULES_HIT:41:355:379:541:800:960:966:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1542:1711:1730:1747:1777:1792:2196:2199:2393:2553:2559:2562:3138:3139:3140:3141:3142:3353:3865:3867:3871:3872:3874:4321:4385:5007:6119:6261:6653:6737:6738:7903:8957:10004:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12895:13894:14096:14181:14394:14721:21080:21094:21323:21444:21451:21627:21990:30054:30090,0,RBL:209.85.216.66:@bytedance.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201;04y8nzm78m6zt1x1irfh7grf3g6fqopsmwb74xhuchodr4raxqbdy1hff54yizb.th5qk9k9qu8bbtmnfitdykeqh8jinxacy178zw5hdbtyah3wfuqbw8phx5oftgw.k-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: nest81_0f0f26e27274 X-Filterd-Recvd-Size: 5622 Received: from mail-pj1-f66.google.com (mail-pj1-f66.google.com [209.85.216.66]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Mon, 26 Oct 2020 14:54:56 +0000 (UTC) Received: by mail-pj1-f66.google.com with SMTP id m3so3221987pjf.4 for ; Mon, 26 Oct 2020 07:54:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9KzGJW7+3gJH1QbMETcNFJ5S+jm03xO8soaD+n5t7bs=; b=wHxCia5NROho+R2WxNTOTYMXm7HB4+q/L8kpv/2zF0uom6mMQb/JnSQ3A2X1ln3ktA xVQhSWUcsBaUKAGC3K2QjY8+r5RkQZ3KAik14xVwXpDfHeUy6myMUbEvXopYOF7lxoCh HgyxJamFeCA2OSW7wtC9Dnek5iyFF61WptjLnNfUZjc+wfTpq+9aQaGs8smf8OIGNogt RJq+6xUfkCV/lqMtubBnmhWWu0ivZ5TXV8TxC7hMhCvNl4EdwyOB2n2bYGSnonutHQ5K 5/vOMylWZVIsiSma26xpFjtykZXV+KK8Lfpw7Vp77ug/TUgHf2C5ZG0Bv/2Gp5DvKJpn Aq2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9KzGJW7+3gJH1QbMETcNFJ5S+jm03xO8soaD+n5t7bs=; b=G8liZHKaY5ZidvCd2UlowP5UsPgisnIGITtiDLfR3T5SzHwWuVPiDjaoER2Hoe7ET/ UT3WBMWpaMqS0i5QE+27MS7ukuFIS/UV2c4QWJHdraSOqg6Jm85+dV5bg5ziI7YqdQVF Y7oCHyQpto6UgJJv0UBM6ornOHP83x3irMvzeMw7/UFNnIJ/uXEwwtMNRT2vc4zBJ6xS 3MNsd4AuV433VunTU6P57NrNYGQ2jYewt4Vu6+49MSsh6uRYZR0rUp5tvwqZ+8T8cXD+ aDKs5k/Xf9x3aZw7nbL5X91iQ2wgA6E7CyToInzPA4zUxG8fveJat37+ZCgagHl3Gtw9 fdbQ== X-Gm-Message-State: AOAM533rzThr19g3MADmgDUWvjw950qrlr+z83u+LA5FgOPcb4M3IGa9 tZwnEdHyxDfWgM+HEjnG21qtvA== X-Google-Smtp-Source: ABdhPJx65sojl1+93c7ngV5cWzTPUiQrP6xuE+Jh3erKuVbh2qQqYY/M8O+c06KujOkvN6WJMDlfVw== X-Received: by 2002:a17:90b:300a:: with SMTP id hg10mr21002506pjb.72.1603724096151; Mon, 26 Oct 2020 07:54:56 -0700 (PDT) Received: from localhost.localdomain ([103.136.220.89]) by smtp.gmail.com with ESMTPSA id x123sm12042726pfb.212.2020.10.26.07.54.46 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Oct 2020 07:54:55 -0700 (PDT) From: Muchun Song To: corbet@lwn.net, mike.kravetz@oracle.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, viro@zeniv.linux.org.uk, akpm@linux-foundation.org, paulmck@kernel.org, mchehab+huawei@kernel.org, pawan.kumar.gupta@linux.intel.com, rdunlap@infradead.org, oneukum@suse.com, anshuman.khandual@arm.com, jroedel@suse.de, almasrymina@google.com, rientjes@google.com, willy@infradead.org Cc: duanxiongchun@bytedance.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Muchun Song Subject: [PATCH v2 11/19] mm/hugetlb: Use PG_slab to indicate split pmd Date: Mon, 26 Oct 2020 22:51:06 +0800 Message-Id: <20201026145114.59424-12-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201026145114.59424-1-songmuchun@bytedance.com> References: <20201026145114.59424-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When we allocate hugetlb page from buddy, we may need split huge pmd to pte. When we free the hugetlb page, we can merge pte to pmd. So we need to distinguish whether the previous pmd has been split. The page table is not allocated from slab. So we can reuse the PG_slab to indicate that the pmd has been split. Signed-off-by: Muchun Song --- mm/hugetlb.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index bd0c4e7fd994..f75b93fb4c07 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1588,6 +1588,25 @@ static void split_vmemmap_huge_page(struct page *head, pmd_t *pmd) flush_tlb_kernel_range(start, addr); } +static inline bool pmd_split(pmd_t *pmd) +{ + return PageSlab(pmd_page(*pmd)); +} + +static inline void set_pmd_split(pmd_t *pmd) +{ + /* + * We should not use slab for page table allocation. So we can set + * PG_slab to indicate that the pmd has been split. + */ + __SetPageSlab(pmd_page(*pmd)); +} + +static inline void clear_pmd_split(pmd_t *pmd) +{ + __ClearPageSlab(pmd_page(*pmd)); +} + static void free_huge_page_vmemmap(struct hstate *h, struct page *head) { pmd_t *pmd; @@ -1604,6 +1623,7 @@ static void free_huge_page_vmemmap(struct hstate *h, struct page *head) if (vmemmap_pmd_huge(pmd)) { VM_BUG_ON(!nr_pgtable(h)); split_vmemmap_huge_page(head, pmd); + set_pmd_split(pmd); } remap_huge_page_pmd_vmemmap(h, pmd, (unsigned long)head, &free_pages, @@ -1677,11 +1697,12 @@ static void alloc_huge_page_vmemmap(struct hstate *h, struct page *head) spin_lock(ptl); remap_huge_page_pmd_vmemmap(h, pmd, (unsigned long)head, &remap_pages, __remap_huge_page_pte_vmemmap); - if (!freed_vmemmap_hpage_dec(pmd_page(*pmd))) { + if (!freed_vmemmap_hpage_dec(pmd_page(*pmd)) && pmd_split(pmd)) { /* * Todo: * Merge pte to huge pmd if it has ever been split. */ + clear_pmd_split(pmd); } spin_unlock(ptl); }