From patchwork Mon Sep 3 07:22:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 10585575 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3CBC414BD for ; Mon, 3 Sep 2018 07:23:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2747E2876E for ; Mon, 3 Sep 2018 07:23:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1B3652879D; Mon, 3 Sep 2018 07:23:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7C3342876E for ; Mon, 3 Sep 2018 07:23:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E1356B66B8; Mon, 3 Sep 2018 03:22:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 168486B66BA; Mon, 3 Sep 2018 03:22:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 057546B66BB; Mon, 3 Sep 2018 03:22:54 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) by kanga.kvack.org (Postfix) with ESMTP id B892F6B66B8 for ; Mon, 3 Sep 2018 03:22:53 -0400 (EDT) Received: by mail-pg1-f199.google.com with SMTP id s11-v6so2248130pgv.9 for ; Mon, 03 Sep 2018 00:22:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=dB+ELHZrrtMkZ296hswR6Zmfp6auy0mx1dozSVvZnos=; b=ttZQWi3nmwxfqqeOf9aUp03XtiCExqH+SENL8XQQBpAIWtd0g8uerIdIAD6rxKhHHR k9bSprUILRwFjUtqQCaizABUy8+cJntd3UZsgOTHaYx3q9EFSbDMjpQAqNdxbPzAvEn+ FqaupDHiYO6zVrztrpGegZKCcS8VYB06my5RAtunZ3NmnJDehPN+ovA1kbUBx7J1YbT8 i/jZOoYat3IEQhk2o7oG0dZPP6Gd28iKBo+eZ8APOQUoXEiPopJ2s0MqYjhXeMo9Fxwq 6PqISgzlb6NFtZFjiTrZ46IqDyAzwLq9d7fLZatlu6QfxHAfP6aJ4/u2srNf8cp82bMP gQ+g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.100 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APzg51Dcyek0pAkqwAoT7V16bM+DFZxwZHyscb9nlXpVd7ZJ2wAVwxlj jh40SVAyTpK9/0d1yMeqxtvwCnyLNkxh37eO8XDleu9YZQjZ3Vix6tfdORa5MaM+YFLTKd/Ankc 8xJSkqEsyxoOrOqDIeCz8agwnXHWocTg2WQMz5tPrd7TBe8iuJcrpxYpRIdq+pmyvfQ== X-Received: by 2002:a17:902:ab94:: with SMTP id f20-v6mr27388308plr.231.1535959373433; Mon, 03 Sep 2018 00:22:53 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbWGFGUY2Khcq13nHGfg4mS1nv38oxXIvSEDSUtAaH9JvkRzZkU13cbG1l1JTmNUKM4b5ex X-Received: by 2002:a17:902:ab94:: with SMTP id f20-v6mr27388276plr.231.1535959372723; Mon, 03 Sep 2018 00:22:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535959372; cv=none; d=google.com; s=arc-20160816; b=EBgt2U1wQ64HbVYD/0eYyKU9X8i7jVXELP4yq5JMHds+dqeMC3sOped4WtfVAifQlK SxbseTWQuWYtDR0aTia5ISL3y49cHH6+0+cgEN6jt8MFvB89/5rdFaPTpvzV4KH9te7o P24Chv45IiRzSZfkyzwvbjDBe7ABvZxpaGcUJ66KoJv5tD+2DGrxNdim8crZ/pw44qY2 gE8+M2yBai4JwIODgYpgeB7j6+PPKM5jfuKMP4p2iuVvMwmGycLrmYqJj9YXFlHHXP8M 2InkuQTOoA4fIesOLOVr7ZKUnHPuaNh88TNrsw7Sw4yjysVDNDQ2vek8bjcFBCsYG+09 gp5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=dB+ELHZrrtMkZ296hswR6Zmfp6auy0mx1dozSVvZnos=; b=0kG5hJHHI4OuLYa2HFvUDI+IWqpPdv6aABTY4830qdSkkkkWfya87OInhXdzM/IaeY uBDk5m/v3NwVJ0oDSE86grqdm2eT5TDBzJESUvmBBuyub2iF7SQfCHAznYqKbumuL5ow O6U0jIcEu1vslwWNUyK+NeOdCP8OF14iR6nye4myC9LXECnm5T3AcNMRSgxdssJXY8pb d9t4dCidB6E1SWWkb4MMbO/GRaOyp9688w5QooBY4yIFUUrqaX2MRgvYFUa2xzlJL5XK 1tQF1sRWJR9A3R+Z+vihuTZ2VfJSIeTNRXXaJyVBKE0FCfeR8EekR3id5DK0JD+BWcgE d0ew== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.100 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga07.intel.com (mga07.intel.com. [134.134.136.100]) by mx.google.com with ESMTPS id az1-v6si9327648plb.513.2018.09.03.00.22.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 03 Sep 2018 00:22:52 -0700 (PDT) Received-SPF: pass (google.com: domain of ying.huang@intel.com designates 134.134.136.100 as permitted sender) client-ip=134.134.136.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.100 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Sep 2018 00:22:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,324,1531810800"; d="scan'208";a="70146924" Received: from yhuang-mobile.sh.intel.com ([10.239.196.86]) by orsmga008.jf.intel.com with ESMTP; 03 Sep 2018 00:22:49 -0700 From: Huang Ying To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , "Kirill A. Shutemov" , Andrea Arcangeli , Michal Hocko , Johannes Weiner , Shaohua Li , Hugh Dickins , Minchan Kim , Rik van Riel , Dave Hansen , Naoya Horiguchi , Zi Yan , Daniel Jordan Subject: [PATCH -V5 13/21] swap: Support PMD swap mapping in madvise_free() Date: Mon, 3 Sep 2018 15:22:06 +0800 Message-Id: <20180903072214.24602-14-ying.huang@intel.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20180903072214.24602-1-ying.huang@intel.com> References: <20180903072214.24602-1-ying.huang@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When madvise_free() found a PMD swap mapping, if only part of the huge swap cluster is operated on, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Otherwise, if all huge swap cluster is operated on, free_swap_and_cache() will be called to decrease the PMD swap mapping count and probably free the swap space and the THP in swap cache too. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel Jordan --- mm/huge_memory.c | 54 +++++++++++++++++++++++++++++++++++++++--------------- mm/madvise.c | 2 +- 2 files changed, 40 insertions(+), 16 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 4b7ee510a9fe..656e760d19e2 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1844,6 +1844,15 @@ int do_huge_pmd_swap_page(struct vm_fault *vmf, pmd_t orig_pmd) } #endif +static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) +{ + pgtable_t pgtable; + + pgtable = pgtable_trans_huge_withdraw(mm, pmd); + pte_free(mm, pgtable); + mm_dec_nr_ptes(mm); +} + /* * Return true if we do MADV_FREE successfully on entire pmd page. * Otherwise, return false. @@ -1864,15 +1873,39 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, goto out_unlocked; orig_pmd = *pmd; - if (is_huge_zero_pmd(orig_pmd)) - goto out; - if (unlikely(!pmd_present(orig_pmd))) { - VM_BUG_ON(thp_migration_supported() && - !is_pmd_migration_entry(orig_pmd)); - goto out; + swp_entry_t entry = pmd_to_swp_entry(orig_pmd); + + if (is_migration_entry(entry)) { + VM_BUG_ON(!thp_migration_supported()); + goto out; + } else if (IS_ENABLED(CONFIG_THP_SWAP) && + !non_swap_entry(entry)) { + /* + * If part of THP is discarded, split the PMD + * swap mapping and operate on the PTEs + */ + if (next - addr != HPAGE_PMD_SIZE) { + unsigned long haddr = addr & HPAGE_PMD_MASK; + + __split_huge_swap_pmd(vma, haddr, pmd); + goto out; + } + free_swap_and_cache(entry, HPAGE_PMD_NR); + pmd_clear(pmd); + zap_deposited_table(mm, pmd); + if (current->mm == mm) + sync_mm_rss(mm); + add_mm_counter(mm, MM_SWAPENTS, -HPAGE_PMD_NR); + ret = true; + goto out; + } else + VM_BUG_ON(1); } + if (is_huge_zero_pmd(orig_pmd)) + goto out; + page = pmd_page(orig_pmd); /* * If other processes are mapping this page, we couldn't discard @@ -1918,15 +1951,6 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, return ret; } -static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) -{ - pgtable_t pgtable; - - pgtable = pgtable_trans_huge_withdraw(mm, pmd); - pte_free(mm, pgtable); - mm_dec_nr_ptes(mm); -} - int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr) { diff --git a/mm/madvise.c b/mm/madvise.c index 50282ba862e2..20101ff125d0 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -321,7 +321,7 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, unsigned long next; next = pmd_addr_end(addr, end); - if (pmd_trans_huge(*pmd)) + if (pmd_trans_huge(*pmd) || is_swap_pmd(*pmd)) if (madvise_free_huge_pmd(tlb, vma, pmd, addr, next)) goto next;