From patchwork Fri Jun 22 03:51:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 10481185 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 763E660380 for ; Fri, 22 Jun 2018 03:56:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 68C0E28F91 for ; Fri, 22 Jun 2018 03:56:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5D68F28F98; Fri, 22 Jun 2018 03:56:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BFF3228F91 for ; Fri, 22 Jun 2018 03:56:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C46BF6B0288; Thu, 21 Jun 2018 23:56:07 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BFC556B028A; Thu, 21 Jun 2018 23:56:07 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8EBD46B028B; Thu, 21 Jun 2018 23:56:07 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl0-f72.google.com (mail-pl0-f72.google.com [209.85.160.72]) by kanga.kvack.org (Postfix) with ESMTP id 3CA656B0288 for ; Thu, 21 Jun 2018 23:56:07 -0400 (EDT) Received: by mail-pl0-f72.google.com with SMTP id w1-v6so2928420plq.8 for ; Thu, 21 Jun 2018 20:56:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=hvTVvXPPBPzeGECAxcfKz0ja6MvVJEk0sVtAQLI16EM=; b=p4gVoxihQrZNA1mElIRNhNOoVHLT6CE1R+G4HLwrq/fL7dBYzPpqHrUMU/B9v+XmDz F3kYO+Zmip7waaaVNsLCxzH55bcywxJyQNyvROx9XiyoknUKRzvuz/Brrhkdq8IqtnWU 2TOuuTiotj6nKl9tSjRg6h8gSuL+7oKF6qc9gArJYcJi64YWHjLKsZS5MtFuf/O6w+pn loi/yk1TwX2fAwlb7WwqvNc81P8SbDFNIvhXu5k+HxZ4LbKWXwpFw5DfWQWyUqvQl9AL qYFkDaMAB5kAjwNeTWPNJqnWY+8WewH/LFWnzsAOkk0EIXKykHr6T1h+PbdUgA8DfgtK uHSA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APt69E1Hloe6QO5Sv+bkMcYVgX2+5U5C2TrOek4xttrZPRifDwKhE1qf LSCPGxTjGakgGAkrbGGXpGwDaC7D/S5e+koBjpIW6BEeu2o+4wserh5KX1Z8C991WUTrjOXF0LZ FRZ7zcFxn+nJ5n6l/SZTV6WSCe0NqAaiRRIZTxLGZHN35hmjbtIF5Wow1zEu00rsNpw== X-Received: by 2002:a62:3dca:: with SMTP id x71-v6mr30084104pfj.134.1529639766905; Thu, 21 Jun 2018 20:56:06 -0700 (PDT) X-Google-Smtp-Source: ADUXVKIQCYRuQhe2hWDN8YG0m9ZjaukpaQyfm/IoY3LMGtnrL9tafKE3sT+kXVepGvea67RuXaU6 X-Received: by 2002:a62:3dca:: with SMTP id x71-v6mr30084061pfj.134.1529639765935; Thu, 21 Jun 2018 20:56:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1529639765; cv=none; d=google.com; s=arc-20160816; b=M5WNVeC95RU5M98MQHXyupaJjmo9jAQgxZaWLDIJ38SPZsJ06S3bpp+3xxByWmQjTX mjxQeV9ol92CU7bxSaA2Xk1eruvtJ2istpFM0sQiaPUeGxwEHYLXIXYtdDhHLsRJ1cxk 26tmg4GbqEtQwiF7jHDKj1Oe0sQppp3DbsX0K4JYvbtnAefP/kN80zJXQCGbWMXdkOTZ RdPY9CIf/ZIOtt5LWG2SSXjmQcDKULBYYXQRKa83OQP0eT3SUfccdEUPvKq7CKoZNdAI 0T68ReraZ82TXgeFMCe8BeIdwaPXqx/HlRWp4956epgsizTAUlXj5C0bM0yN4UrcEVYU 4/AQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=hvTVvXPPBPzeGECAxcfKz0ja6MvVJEk0sVtAQLI16EM=; b=PHTmF1fMM/6u7qo4zxtr357jaiLA1rbHuUNjxyAbMfaS26hRBEPA4UcEdpQX3WFqrJ j4zp1P602yrcxYHU6CqzBaAKBVC2vKPqjuuN0op1SuFSd+BytaTUfPi9bmsKDs3ZdEcD DFwsNsRcWtYqlbuW3EOk4EhlZpU6NJpKadgZFD0bX633dqo/MhVNKkMMiKAig1fr+jcQ +mJ3ryx6XTFtnEUkBdDx7e0pVavUSngBgtkIayZEdpJk9znbIeT4vHOhwJjNA1rR9X0Y 5pPsP5zkvdmDzrK+SJyV+uC2AdnJ03AoaRXMCCqD8LtthxAXuA0gyu5BfVPaDd63a4ad 5N5w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga09.intel.com (mga09.intel.com. [134.134.136.24]) by mx.google.com with ESMTPS id k74-v6si5146207pgc.304.2018.06.21.20.56.05 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Jun 2018 20:56:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ying.huang@intel.com designates 134.134.136.24 as permitted sender) client-ip=134.134.136.24; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Jun 2018 20:56:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,255,1526367600"; d="scan'208";a="65335240" Received: from wanpingl-mobl.ccr.corp.intel.com (HELO yhuang6-ux31a.ccr.corp.intel.com) ([10.254.212.200]) by fmsmga004.fm.intel.com with ESMTP; 21 Jun 2018 20:56:03 -0700 From: "Huang, Ying" To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , "Kirill A. Shutemov" , Andrea Arcangeli , Michal Hocko , Johannes Weiner , Shaohua Li , Hugh Dickins , Minchan Kim , Rik van Riel , Dave Hansen , Naoya Horiguchi , Zi Yan , Daniel Jordan Subject: [PATCH -mm -v4 21/21] mm, THP: Avoid to split THP when reclaim MADV_FREE THP Date: Fri, 22 Jun 2018 11:51:51 +0800 Message-Id: <20180622035151.6676-22-ying.huang@intel.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20180622035151.6676-1-ying.huang@intel.com> References: <20180622035151.6676-1-ying.huang@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Huang Ying Previously, to reclaim MADV_FREE THP, the THP will be split firstly, then reclaim each sub-pages. This wastes cycles to split THP and unmap and free each sub-pages, and split THP even if it has been written since MADV_FREE. We have to do this because MADV_FREE THP reclaiming shares same try_to_unmap() calling with swap, while swap needs to split the PMD page mapping at that time. Now swap can process PMD mapping, this makes it easy to avoid to split THP when MADV_FREE THP is reclaimed. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel Jordan --- mm/huge_memory.c | 41 ++++++++++++++++++++++++++++++++--------- mm/vmscan.c | 3 ++- 2 files changed, 34 insertions(+), 10 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 195f24040b41..7c6edd897f15 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1671,6 +1671,15 @@ int do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd) return 0; } +static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) +{ + pgtable_t pgtable; + + pgtable = pgtable_trans_huge_withdraw(mm, pmd); + pte_free(mm, pgtable); + mm_dec_nr_ptes(mm); +} + #ifdef CONFIG_THP_SWAP void __split_huge_swap_pmd(struct vm_area_struct *vma, unsigned long haddr, @@ -1885,6 +1894,28 @@ bool set_pmd_swap_entry(struct page_vma_mapped_walk *pvmw, struct page *page, pmd_t swp_pmd; swp_entry_t entry = { .val = page_private(page) }; + if (unlikely(PageSwapBacked(page) != PageSwapCache(page))) { + WARN_ON_ONCE(1); + return false; + } + + /* MADV_FREE page check */ + if (!PageSwapBacked(page)) { + if (!PageDirty(page)) { + zap_deposited_table(mm, pvmw->pmd); + add_mm_counter(mm, MM_ANONPAGES, -HPAGE_PMD_NR); + goto out_remove_rmap; + } + + /* + * If the page was redirtied, it cannot be + * discarded. Remap the page to page table. + */ + set_pmd_at(mm, address, pvmw->pmd, pmdval); + SetPageSwapBacked(page); + return false; + } + if (swap_duplicate(&entry, true) < 0) { set_pmd_at(mm, address, pvmw->pmd, pmdval); return false; @@ -1902,21 +1933,13 @@ bool set_pmd_swap_entry(struct page_vma_mapped_walk *pvmw, struct page *page, swp_pmd = pmd_swp_mksoft_dirty(swp_pmd); set_pmd_at(mm, address, pvmw->pmd, swp_pmd); +out_remove_rmap: page_remove_rmap(page, true); put_page(page); return true; } #endif -static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) -{ - pgtable_t pgtable; - - pgtable = pgtable_trans_huge_withdraw(mm, pmd); - pte_free(mm, pgtable); - mm_dec_nr_ptes(mm); -} - /* * Return true if we do MADV_FREE successfully on entire pmd page. * Otherwise, return false. diff --git a/mm/vmscan.c b/mm/vmscan.c index 891d3c7b8f21..80d0150dd028 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1137,7 +1137,8 @@ static unsigned long shrink_page_list(struct list_head *page_list, /* Adding to swap updated mapping */ mapping = page_mapping(page); } - } else if (unlikely(PageTransHuge(page))) { + } else if (unlikely(PageTransHuge(page)) && + (!thp_swap_supported() || !PageAnon(page))) { /* Split file THP */ if (split_huge_page_to_list(page, page_list)) goto keep_locked;