From patchwork Tue Nov 20 08:54:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 10690007 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 195ED13AD for ; Tue, 20 Nov 2018 08:55:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0A0B42991D for ; Tue, 20 Nov 2018 08:55:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F246B29A1B; Tue, 20 Nov 2018 08:55:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 63C3B2991D for ; Tue, 20 Nov 2018 08:55:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9BFE96B1F43; Tue, 20 Nov 2018 03:55:34 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 96E786B1F44; Tue, 20 Nov 2018 03:55:34 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 886BB6B1F45; Tue, 20 Nov 2018 03:55:34 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id 46B696B1F43 for ; Tue, 20 Nov 2018 03:55:34 -0500 (EST) Received: by mail-pf1-f198.google.com with SMTP id s14so1063099pfk.16 for ; Tue, 20 Nov 2018 00:55:34 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=TFAFLyMtcJjXCOTjjkL82qcfCtqqFM3BCy/8n1um5n0=; b=h2e+rCPqJqPIaZri3JU5X1AcIprEZXQ6UdHswhgcYoitLTDJPRwj2OO8u5Fo0yHONM yckZqB7YWdSl5fNtbOIotGFptquqEPo/mdEEVnrLVVLtg6Vl94VNO78e5J1mMYHzbCmw 9n6nXoYhYOGaYxjZUqh0zYcN2AosCjYyMqtfgNOMlASLYe5nlIf9o69zOyW4r9VKY3DG 0Woccq4np82BGIsNW/LJr8C6MBuDQ7RdnYE4o2R+wCiensSjjK6iUM8clc1C8VVBn88T 5jCMb2mheQIk4RGpGja7m0sAL7JggIpbgb35ZgkbStCVCeMXVbYJEESFJJY2g6m7kG1l LzRg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AA+aEWYIYx6SNI27LFVOZ9PQ8N/fCYxIMnzTUSMU8mX0PvsGCebo0o5z NLY5YgSvzmTDl8oyrkAXajWrwYmQ1VR9vOpmgkypAUExXuKz6UKx36i2QyXdn4xY9l60IPYtwu8 FvxhSNVUojSn4mLurCi8+E5C1aK2TKa3l6+fynMzH5WBHbTvmn2fWGOzprJODv0RqFw== X-Received: by 2002:a63:f412:: with SMTP id g18mr1160453pgi.262.1542704133943; Tue, 20 Nov 2018 00:55:33 -0800 (PST) X-Google-Smtp-Source: AFSGD/VC60S+YtvYNy+ABkyD0RF18nWTaVHXlgsOHbSnWVBw/hX97LZl+nA8qQXF60N1zFgyQwAD X-Received: by 2002:a63:f412:: with SMTP id g18mr1160408pgi.262.1542704132766; Tue, 20 Nov 2018 00:55:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542704132; cv=none; d=google.com; s=arc-20160816; b=Nwlaj8O1voDCJXqd67QcFV3LmpDnZFvcxL0+nlra2pKoG82fhPImvKEV45eHbDGQhI 013rtaIZwMMZL0YL9/qfoTDi+i3S0U9FZh8XBtobpv/vX+BU6JbkrWcG6k8x0rajBzr2 O5vtKqyI0ixZ9cP4rlmvvuPbAutMRS+hVdhkeN+mQgLujDUmUZm5wlpuy8sLi0yabvmr PTe4sdEdWsRBIkhBwVL7U1OB6tR8DODcz7BbzM3wFcu+L/o5I4/rLMy7nOIcfFMA54PC +IeT7vouAm4p9cfrs0MgHdybqarOqkwuZDNM0i/ksW1URAQ/0xkpfaWfBVr37po/pSjy XWcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=TFAFLyMtcJjXCOTjjkL82qcfCtqqFM3BCy/8n1um5n0=; b=jnTh8mRHMJQJmOQW7d0FwaBatnDEtXOYlaGFoQ4FK2TNi2KYXZM0Hms0ZLT2I5JIyK bXZQXQTqVjPv7nZhyK6YqqE9eTCyOw6GlNWKRWBXdNxpea5nxUsW7NctW5P0FjOIlK++ EIOYSRq9LQC7D6AjH832SSOMGR6R0Iz6mJ8tkbHAWil8sr73qml9bj+W4PLldQgJ0G4C B+fWJ9SUtAPZkUI233KnZFtDSXIf9AbKKytMKT1NNmywZe6cKauaaUVRME+btcS+NvWC ifkViGBU62/NeA247zbWYeSdCWUfCW0G1ydbAFMvj5WcxX3GsMJH2dCjE7thlFd1OBss Fi/g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga01.intel.com (mga01.intel.com. [192.55.52.88]) by mx.google.com with ESMTPS id b15si24149550plm.431.2018.11.20.00.55.32 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 00:55:32 -0800 (PST) Received-SPF: pass (google.com: domain of ying.huang@intel.com designates 192.55.52.88 as permitted sender) client-ip=192.55.52.88; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 20 Nov 2018 00:55:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,256,1539673200"; d="scan'208";a="106105867" Received: from yhuang-mobile.sh.intel.com ([10.239.197.245]) by fmsmga002.fm.intel.com with ESMTP; 20 Nov 2018 00:55:29 -0800 From: Huang Ying To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , "Kirill A. Shutemov" , Andrea Arcangeli , Michal Hocko , Johannes Weiner , Shaohua Li , Hugh Dickins , Minchan Kim , Rik van Riel , Dave Hansen , Naoya Horiguchi , Zi Yan , Daniel Jordan Subject: [PATCH -V7 RESEND 13/21] swap: Support PMD swap mapping in madvise_free() Date: Tue, 20 Nov 2018 16:54:41 +0800 Message-Id: <20181120085449.5542-14-ying.huang@intel.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20181120085449.5542-1-ying.huang@intel.com> References: <20181120085449.5542-1-ying.huang@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When madvise_free() found a PMD swap mapping, if only part of the huge swap cluster is operated on, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Otherwise, if all huge swap cluster is operated on, free_swap_and_cache() will be called to decrease the PMD swap mapping count and probably free the swap space and the THP in swap cache too. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel Jordan --- mm/huge_memory.c | 52 ++++++++++++++++++++++++++++++++++-------------- mm/madvise.c | 2 +- 2 files changed, 38 insertions(+), 16 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 079592b9f4a5..89aa93d586ec 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1900,6 +1900,15 @@ int do_huge_pmd_swap_page(struct vm_fault *vmf, pmd_t orig_pmd) } #endif +static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) +{ + pgtable_t pgtable; + + pgtable = pgtable_trans_huge_withdraw(mm, pmd); + pte_free(mm, pgtable); + mm_dec_nr_ptes(mm); +} + /* * Return true if we do MADV_FREE successfully on entire pmd page. * Otherwise, return false. @@ -1920,15 +1929,37 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, goto out_unlocked; orig_pmd = *pmd; - if (is_huge_zero_pmd(orig_pmd)) - goto out; - if (unlikely(!pmd_present(orig_pmd))) { - VM_BUG_ON(thp_migration_supported() && - !is_pmd_migration_entry(orig_pmd)); - goto out; + swp_entry_t entry = pmd_to_swp_entry(orig_pmd); + + if (is_migration_entry(entry)) { + VM_BUG_ON(!thp_migration_supported()); + goto out; + } else if (IS_ENABLED(CONFIG_THP_SWAP) && + !non_swap_entry(entry)) { + /* + * If part of THP is discarded, split the PMD + * swap mapping and operate on the PTEs + */ + if (next - addr != HPAGE_PMD_SIZE) { + __split_huge_swap_pmd(vma, addr, pmd); + goto out; + } + free_swap_and_cache(entry, HPAGE_PMD_NR); + pmd_clear(pmd); + zap_deposited_table(mm, pmd); + if (current->mm == mm) + sync_mm_rss(mm); + add_mm_counter(mm, MM_SWAPENTS, -HPAGE_PMD_NR); + ret = true; + goto out; + } else + VM_BUG_ON(1); } + if (is_huge_zero_pmd(orig_pmd)) + goto out; + page = pmd_page(orig_pmd); /* * If other processes are mapping this page, we couldn't discard @@ -1974,15 +2005,6 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, return ret; } -static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) -{ - pgtable_t pgtable; - - pgtable = pgtable_trans_huge_withdraw(mm, pmd); - pte_free(mm, pgtable); - mm_dec_nr_ptes(mm); -} - int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr) { diff --git a/mm/madvise.c b/mm/madvise.c index cbb3d7e38e51..0c1f96c605f8 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -321,7 +321,7 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, unsigned long next; next = pmd_addr_end(addr, end); - if (pmd_trans_huge(*pmd)) + if (pmd_trans_huge(*pmd) || is_swap_pmd(*pmd)) if (madvise_free_huge_pmd(tlb, vma, pmd, addr, next)) goto next;