From patchwork Fri Jun 22 03:51:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 10481161 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1D44360532 for ; Fri, 22 Jun 2018 03:56:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 10BB628F91 for ; Fri, 22 Jun 2018 03:56:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 049B428F98; Fri, 22 Jun 2018 03:56:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 81DAC28F94 for ; Fri, 22 Jun 2018 03:56:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3B9646B0274; Thu, 21 Jun 2018 23:55:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 369986B0276; Thu, 21 Jun 2018 23:55:46 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 16F056B0277; Thu, 21 Jun 2018 23:55:46 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf0-f200.google.com (mail-pf0-f200.google.com [209.85.192.200]) by kanga.kvack.org (Postfix) with ESMTP id BDB4F6B0274 for ; Thu, 21 Jun 2018 23:55:45 -0400 (EDT) Received: by mail-pf0-f200.google.com with SMTP id a13-v6so2542745pfo.22 for ; Thu, 21 Jun 2018 20:55:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=3CIL/VFGgRtEZ8AmjeiN8j+oHlGhmj5XQFWTKYoA2Lc=; b=sTLzrLaqf3J0q/c7yOcIBxLW8NXIfr53+a5cbQ5xwjl1MnFTHHoudM3LmK53wy4/Yp vBIGBMGXseu2ABa+1InAcKA9GpwP+7oEpVVb4pm8bZfwS+4q+DascSV4+lNai/US5GJy 0ae/YcL3aZD/LD2UNnAJCla7bAd3dUrIWfExAG1KLJAh84Tj8WRimxM134mMj5lBl2Kg z90eZyRg6tH0JTBOQw3b1Qe6ddStUW0Sentk8ZLGIZrUJxtU6PJYYuny3nmKcFy5AlpG ddLJwSICsCSsVv6ya0QzDvi7xtLgt8ffNOL9fgpf/n/5mrQQpfVHAU+69v6xSlAqw51V Pl8A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APt69E2UCcoKErBxXXHu/0n6OFAoaavNavqCB0/7V9HJoApqyuVUYfJE Cbhv3xAidd0sFuuDOl2YQ2k3UrBGGV9uchCObxRxG6WhffIEcY9Bf2Fp1EK4apnPu3VvFZH+hnQ 78zf/DP8nN2Xs1xhkqcGQXpBJr16tGy8aveXrLt1zhN53SzUS970lANoTMn4M5Z3vRg== X-Received: by 2002:a65:524d:: with SMTP id q13-v6mr24964381pgp.244.1529639745451; Thu, 21 Jun 2018 20:55:45 -0700 (PDT) X-Google-Smtp-Source: ADUXVKKuc5pyL0MD/Ejq3UHVVf5Ny5hgRSSZ2fgsGtstIDlG3rKNRMh5BzFwCDHLzUFx9iWVMcJt X-Received: by 2002:a65:524d:: with SMTP id q13-v6mr24964349pgp.244.1529639744663; Thu, 21 Jun 2018 20:55:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1529639744; cv=none; d=google.com; s=arc-20160816; b=aHqfE0+hT1BILdfThPRr6Xs1Fg13y9lSreQoJmuU4yxa7v2ISH85LFMGh5JZbz1iPZ oHGDRmZAIZqd5i7uTOw1rBWUvjbPw5iUZzGz3L1agrTnUHYjiVq6v1EOMZaoHiVEHLXZ giae2artL6lasfo0wP55uBJCQuBRJinqaHz8MpkqAT0kzVUidT9cQrx0ZDcPhYmMZwAc l/SPWO/vDy0DATUfdY9l9kQ1irpUqDj2Q/h9C/8MFErYCbbWqBFHmZxO8hezLZuImzhO QPAe5CsUwKRsNqtlGTF06h6dVx+P93yoDtscsk50c+8WtmfonwewswXT7X+hyizqML5R NeVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=3CIL/VFGgRtEZ8AmjeiN8j+oHlGhmj5XQFWTKYoA2Lc=; b=xPL/IGRTuO4fYxoRbOGV3LQirjR4IPITKf4SLaJgxI91rikcqUYYi9JcZK8kSCF4/c GH32wV5wuX2BrvJGX5Jo1YaBfRMETewiXsH2Y6FfP6nP0diU9KD1/Vscw4ZsBDBcrxg5 M0omo7/wfXGs2Vk1WOluow3adQ+McGJmDhUmdZLVFQ+IMZ2vfFCMxStNtCXX6Ko9bUBZ cTnR98LNBf6OG+V1fE9kZXWp9PCwfPmnSY/ld58UcamcNJE3jI+3wbK5xxM2wdEMmWfE bvczhfmQYMwxARpuLqVDGTwo/qb59hBOEs+s+5UgunGkji4IVza0gHGohMKiZ0cLHdGo Tc1g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga01.intel.com (mga01.intel.com. [192.55.52.88]) by mx.google.com with ESMTPS id n61-v6si6160066plb.256.2018.06.21.20.55.44 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Jun 2018 20:55:44 -0700 (PDT) Received-SPF: pass (google.com: domain of ying.huang@intel.com designates 192.55.52.88 as permitted sender) client-ip=192.55.52.88; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Jun 2018 20:55:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,255,1526367600"; d="scan'208";a="65335160" Received: from wanpingl-mobl.ccr.corp.intel.com (HELO yhuang6-ux31a.ccr.corp.intel.com) ([10.254.212.200]) by fmsmga004.fm.intel.com with ESMTP; 21 Jun 2018 20:55:41 -0700 From: "Huang, Ying" To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , "Kirill A. Shutemov" , Andrea Arcangeli , Michal Hocko , Johannes Weiner , Shaohua Li , Hugh Dickins , Minchan Kim , Rik van Riel , Dave Hansen , Naoya Horiguchi , Zi Yan , Daniel Jordan Subject: [PATCH -mm -v4 13/21] mm, THP, swap: Support PMD swap mapping in madvise_free() Date: Fri, 22 Jun 2018 11:51:43 +0800 Message-Id: <20180622035151.6676-14-ying.huang@intel.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20180622035151.6676-1-ying.huang@intel.com> References: <20180622035151.6676-1-ying.huang@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Huang Ying When madvise_free() found a PMD swap mapping, if only part of the huge swap cluster is operated on, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Otherwise, if all huge swap cluster is operated on, free_swap_and_cache() will be called to decrease the PMD swap mapping count and probably free the swap space and the THP in swap cache too. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel Jordan --- mm/huge_memory.c | 50 +++++++++++++++++++++++++++++++++++--------------- mm/madvise.c | 2 +- 2 files changed, 36 insertions(+), 16 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 73fc77633642..fe946ca09b28 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1842,6 +1842,15 @@ static inline void __split_huge_swap_pmd(struct vm_area_struct *vma, } #endif +static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) +{ + pgtable_t pgtable; + + pgtable = pgtable_trans_huge_withdraw(mm, pmd); + pte_free(mm, pgtable); + mm_dec_nr_ptes(mm); +} + /* * Return true if we do MADV_FREE successfully on entire pmd page. * Otherwise, return false. @@ -1862,15 +1871,35 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, goto out_unlocked; orig_pmd = *pmd; - if (is_huge_zero_pmd(orig_pmd)) - goto out; - if (unlikely(!pmd_present(orig_pmd))) { - VM_BUG_ON(thp_migration_supported() && - !is_pmd_migration_entry(orig_pmd)); - goto out; + swp_entry_t entry = pmd_to_swp_entry(orig_pmd); + + if (is_migration_entry(entry)) { + VM_BUG_ON(!thp_migration_supported()); + goto out; + } else if (thp_swap_supported() && !non_swap_entry(entry)) { + /* If part of THP is discarded */ + if (next - addr != HPAGE_PMD_SIZE) { + unsigned long haddr = addr & HPAGE_PMD_MASK; + + __split_huge_swap_pmd(vma, haddr, pmd); + goto out; + } + free_swap_and_cache(entry, true); + pmd_clear(pmd); + zap_deposited_table(mm, pmd); + if (current->mm == mm) + sync_mm_rss(mm); + add_mm_counter(mm, MM_SWAPENTS, -HPAGE_PMD_NR); + ret = true; + goto out; + } else + VM_BUG_ON(1); } + if (is_huge_zero_pmd(orig_pmd)) + goto out; + page = pmd_page(orig_pmd); /* * If other processes are mapping this page, we couldn't discard @@ -1916,15 +1945,6 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, return ret; } -static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) -{ - pgtable_t pgtable; - - pgtable = pgtable_trans_huge_withdraw(mm, pmd); - pte_free(mm, pgtable); - mm_dec_nr_ptes(mm); -} - int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr) { diff --git a/mm/madvise.c b/mm/madvise.c index a96abb915e22..fc423d96d4d6 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -321,7 +321,7 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, unsigned long next; next = pmd_addr_end(addr, end); - if (pmd_trans_huge(*pmd)) + if (pmd_trans_huge(*pmd) || is_swap_pmd(*pmd)) if (madvise_free_huge_pmd(tlb, vma, pmd, addr, next)) goto next;