From patchwork Wed May 23 08:26:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 10420609 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 744BA60224 for ; Wed, 23 May 2018 08:27:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 66AB528E3F for ; Wed, 23 May 2018 08:27:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5B5AF28E54; Wed, 23 May 2018 08:27:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AECFD28E3F for ; Wed, 23 May 2018 08:27:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E4C606B026D; Wed, 23 May 2018 04:27:07 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DD1476B026E; Wed, 23 May 2018 04:27:07 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CEBA36B026F; Wed, 23 May 2018 04:27:07 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg0-f70.google.com (mail-pg0-f70.google.com [74.125.83.70]) by kanga.kvack.org (Postfix) with ESMTP id 8A72C6B026D for ; Wed, 23 May 2018 04:27:07 -0400 (EDT) Received: by mail-pg0-f70.google.com with SMTP id k1-v6so6444119pgq.20 for ; Wed, 23 May 2018 01:27:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=gMQ3TplL1eJvLqKUiYQQxMUbTIBG/VrcVoTakXERoFQ=; b=Y4dm1iMwAECUuPXFF7NIgydgO+EXZjSf1tCB7gZKsBmDNBNwQBxH0xGhcGNmYEO1q1 +Utqf6cCqeE4WO/9Yj0UL0pVMVPmxvK0NTwEWY5T7Lu66MrisLDekzCJwOK3Dqm5NYum //ctwYyrKAIRcLYHEfcP7gJBU7tel/gbQQaQeYZOpy6FzPX+HqgoxVncN1XTu3fMASkU NCDTvvtjZJ3/wMXEASBMIl3cqWUcUYUQOAyXp8m/NvDZkgltIMEBkXikChA5lmsS3YJs E8GqIXcCyyq97Bs/nOt9HIEkQtn+OzBoR7unbQU85h5G5cCvK2+mrsvVjwu/bOkwegi6 kbzg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: ALKqPwcgg9FT6qCZW7mhQVSS/+iqUH8aUK8RTMM8XTnk+9iIi/I27BDe 6kcixLD7amtUlEr+Q/VggbkdPB7vaTnKQJRoqWi89s9cPFTQJ+ibB88LwAwU6MUuPrvA9zSBzsP y7mPyYwIws/Dpg9EvGpjXPm4vcgkj15EqB1iKVQFxY2dRGuXTQWWKnWvyS/U6O3i7jQ== X-Received: by 2002:a63:a312:: with SMTP id s18-v6mr1530580pge.187.1527064027230; Wed, 23 May 2018 01:27:07 -0700 (PDT) X-Google-Smtp-Source: AB8JxZqhLD4Vd8lPvo6eCq1Ylv5pE95zK+n1Qaanp5nKz06O4GtUdXoXdYsbsFT/fai94gwBrl8N X-Received: by 2002:a63:a312:: with SMTP id s18-v6mr1530546pge.187.1527064026517; Wed, 23 May 2018 01:27:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527064026; cv=none; d=google.com; s=arc-20160816; b=cDTxMGPetaFzsgAzBHByhfRxyhHY3jmyJLlpW861kn4TpIAhwnPnunwFAFo2hcsceJ /IAl6rZrFxx82qgK6UWMQQFaGaJ6vUzmShBftLO+ysfNMad+ERFhTdIZcCjiBo+Rojc2 ymL/IGG5ikMpoHhqhrMDYqA6n8Yhw9TyRW8J9X+QRTE3+sZsUWNmTOFlbHYj6oPzQt0b /flscocVBn+Pd+f73ZUnvVk5/LZvEtbs/ktNA6IBQ5FJ5eb+yOUl6C3gBpvRaVXwMGAg N5r1LSqiScGJiPwmEVNDVSYjX91U9lDdo8wp2YP8ggApVXIAA79JHRWIgtvKTfEY0QCH tHqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=gMQ3TplL1eJvLqKUiYQQxMUbTIBG/VrcVoTakXERoFQ=; b=ToXKmLPV2azBnwIq79Yu/pDjlsPZpFWRPprAHVTJYA3qEa5NcGdgbDYJjCk7s7C1Vx YFU/7cvW979C+R0hmeSwcgpTzdQ/pqoh1aAS16GE4wXDL9BfHnQGPgF392FUA2Wd/ZL+ b4weHz74yBjE9vYmuDBIDCUH1ypn7gxCw8WiLibpUO07wslHSwl6++gzxYLcJc6Xy6DB JJEOIYETf00vWU/Ydg6wX0LLQ3ri400VPb/lAgpkqCTbe64RX6w2m1n6VgZ6K8eJuesa ZALevDF11y6AJjbtzWhVrBSq9WHWdMnA4HIyNlvraubgubKdLUrRfHCPTgAkKGFcQj0z 1cWA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga17.intel.com (mga17.intel.com. [192.55.52.151]) by mx.google.com with ESMTPS id y16-v6si17687140pfm.140.2018.05.23.01.27.06 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 May 2018 01:27:06 -0700 (PDT) Received-SPF: pass (google.com: domain of ying.huang@intel.com designates 192.55.52.151 as permitted sender) client-ip=192.55.52.151; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 May 2018 01:27:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,432,1520924400"; d="scan'208";a="57726139" Received: from yhuang6-ux31a.sh.intel.com ([10.239.197.97]) by fmsmga001.fm.intel.com with ESMTP; 23 May 2018 01:27:03 -0700 From: "Huang, Ying" To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , "Kirill A. Shutemov" , Andrea Arcangeli , Michal Hocko , Johannes Weiner , Shaohua Li , Hugh Dickins , Minchan Kim , Rik van Riel , Dave Hansen , Naoya Horiguchi , Zi Yan Subject: [PATCH -mm -V3 13/21] mm, THP, swap: Support PMD swap mapping in madvise_free() Date: Wed, 23 May 2018 16:26:17 +0800 Message-Id: <20180523082625.6897-14-ying.huang@intel.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180523082625.6897-1-ying.huang@intel.com> References: <20180523082625.6897-1-ying.huang@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Huang Ying When madvise_free() found a PMD swap mapping, if only part of the huge swap cluster is operated on, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Otherwise, if all huge swap cluster is operated on, free_swap_and_cache() will be called to decrease the PMD swap mapping count and probably free the swap space and the THP in swap cache too. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan --- mm/huge_memory.c | 50 +++++++++++++++++++++++++++++++++++--------------- mm/madvise.c | 2 +- 2 files changed, 36 insertions(+), 16 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 668d77cec14d..a8af2ddc578a 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1842,6 +1842,15 @@ static inline void __split_huge_swap_pmd(struct vm_area_struct *vma, } #endif +static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) +{ + pgtable_t pgtable; + + pgtable = pgtable_trans_huge_withdraw(mm, pmd); + pte_free(mm, pgtable); + mm_dec_nr_ptes(mm); +} + /* * Return true if we do MADV_FREE successfully on entire pmd page. * Otherwise, return false. @@ -1862,15 +1871,35 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, goto out_unlocked; orig_pmd = *pmd; - if (is_huge_zero_pmd(orig_pmd)) - goto out; - if (unlikely(!pmd_present(orig_pmd))) { - VM_BUG_ON(thp_migration_supported() && - !is_pmd_migration_entry(orig_pmd)); - goto out; + swp_entry_t entry = pmd_to_swp_entry(orig_pmd); + + if (is_migration_entry(entry)) { + VM_BUG_ON(!thp_migration_supported()); + goto out; + } else if (thp_swap_supported() && !non_swap_entry(entry)) { + /* If part of THP is discarded */ + if (next - addr != HPAGE_PMD_SIZE) { + unsigned long haddr = addr & HPAGE_PMD_MASK; + + __split_huge_swap_pmd(vma, haddr, pmd); + goto out; + } + free_swap_and_cache(entry, true); + pmd_clear(pmd); + zap_deposited_table(mm, pmd); + if (current->mm == mm) + sync_mm_rss(mm); + add_mm_counter(mm, MM_SWAPENTS, -HPAGE_PMD_NR); + ret = true; + goto out; + } else + VM_BUG_ON(1); } + if (is_huge_zero_pmd(orig_pmd)) + goto out; + page = pmd_page(orig_pmd); /* * If other processes are mapping this page, we couldn't discard @@ -1916,15 +1945,6 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, return ret; } -static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) -{ - pgtable_t pgtable; - - pgtable = pgtable_trans_huge_withdraw(mm, pmd); - pte_free(mm, pgtable); - mm_dec_nr_ptes(mm); -} - int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr) { diff --git a/mm/madvise.c b/mm/madvise.c index d180000c626b..e03e85a20fb4 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -321,7 +321,7 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, unsigned long next; next = pmd_addr_end(addr, end); - if (pmd_trans_huge(*pmd)) + if (pmd_trans_huge(*pmd) || is_swap_pmd(*pmd)) if (madvise_free_huge_pmd(tlb, vma, pmd, addr, next)) goto next;