From patchwork Wed May 23 08:26:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 10420583 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1321E60224 for ; Wed, 23 May 2018 08:26:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0482E28E3F for ; Wed, 23 May 2018 08:26:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ED40528E54; Wed, 23 May 2018 08:26:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 75DD228E3F for ; Wed, 23 May 2018 08:26:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE5F76B000C; Wed, 23 May 2018 04:26:44 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E96FE6B000D; Wed, 23 May 2018 04:26:44 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D39016B000E; Wed, 23 May 2018 04:26:44 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf0-f197.google.com (mail-pf0-f197.google.com [209.85.192.197]) by kanga.kvack.org (Postfix) with ESMTP id 907FE6B000C for ; Wed, 23 May 2018 04:26:44 -0400 (EDT) Received: by mail-pf0-f197.google.com with SMTP id q15-v6so12672689pff.17 for ; Wed, 23 May 2018 01:26:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=nT+5o2EvIPvhfoV22RAhP5mCQF6EKt3UikIVOVFClCA=; b=gOHl9z3naLJY3/dPoIfgyuGvAAnCC3HL6jnAYHxssvW7cepEDgWx+rrkBhsnaj8JLu DeD9NW4oLdM2teNRVbPs063TX3CIdNCha0AXHt6QmU7gEUDNJS4Lqudtxn/D5U0uYZmZ P6raDEwk5XqMIHZjsHLNdtzBgUYpD5eV/dc0YgJ/XXEFwfF5nPyuui0En8ZdH5SzLsle p/DypZmlWYB3SVVL5DUuV6Js9eicGP1zxElWeucuOmbzOkdg+F9CWO/NyNGCWpEeIouo NhcMpd/358d8FfgVZOGkcAN39FkL56dPd72kA1ysg9/sF9tr1CdCAR1M4wg7G19m/vWE 0woA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: ALKqPwfhNkmYT1SMhlzKFmhxZbOYP8yzkmMlCpgDLp+C8cBlPYjZctlO TmtpRZJ+bGPqZ1lXM726hyjt9n/Q130afk3mLZrPZevXf/M1rapbsIpCP5CY2i1pS+O/dQwrR6N //FSMNGOt2vJTeGvZwki5wJ2inoP5nQ1tGh+EuDB8rpeQ4HGdW9d7yi5bnblZcRbCig== X-Received: by 2002:a17:902:164:: with SMTP id 91-v6mr2010609plb.134.1527064004289; Wed, 23 May 2018 01:26:44 -0700 (PDT) X-Google-Smtp-Source: AB8JxZrzLrcqZZ1jvbojzmbyj2Td/ULa+R6PgKDu20Jt851foiNYT9xrOFHXaVdVXbPvGAmN720p X-Received: by 2002:a17:902:164:: with SMTP id 91-v6mr2010575plb.134.1527064003623; Wed, 23 May 2018 01:26:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527064003; cv=none; d=google.com; s=arc-20160816; b=gMw7yiPSShbhXTU+1wACQb9QgGmuKS85Jbv7FUAucz0zEwHp0FZsmFnnt4+uPDqBiY 2ow2gvDyKtAe0b7j6FSEXrXWCDqmV5CGMgwucEdSfYQZvcBXtu2BvOpQRomSAy8+i9U0 6vv+KuErMA+Efk/AvHOTT1HtCcArt8aB20JSqwchfCMYUBztoRcJnF7jqLyIg0++t0kT DDYKNsUbjbldrD1kJzO7VQyptb2zgNUfdRs+uf6WZ2onqdb1eaa4sdjeg0eQ1tLMelWY CeBdhwH9FmuJY17sUXVkiNShqZgf7o+wRyT5vwhIy/z5nSPBG7Qkd6OGxBsCwUdY3TNY SuFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=nT+5o2EvIPvhfoV22RAhP5mCQF6EKt3UikIVOVFClCA=; b=XzJ4DaiWl+d3UENIlmTZXqyTtKIeP773l4wBy4dB6RDwJmNS6xoJ+4Q3IRQRhCnRAP 1+9m6p+lLIpVf1RYVijblOI5jXiw8bRZut0S5R5ZIuomW4b7SlI3YVKBWEHhIK0hJnGw gFfF3fEcPByRLtwu64EIhwM9idzwLZ9IjqCQl1+08cFtE5OiBVW2u4tW9QeIpGQcB/no PrL3BF0QPmYRedYculyh3zatDRGEBxzEnGtuY4tf9JxpFMn4Opfeecj7PyTHh9olq+TT okaqFwEgQOxH5WOmGo7NHaPH4xXHwuBFfN6zPE9Vaxy5jgqD3U0IJKgbT/r4awuM5wzU TjZA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga17.intel.com (mga17.intel.com. [192.55.52.151]) by mx.google.com with ESMTPS id y16-v6si17687140pfm.140.2018.05.23.01.26.43 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 May 2018 01:26:43 -0700 (PDT) Received-SPF: pass (google.com: domain of ying.huang@intel.com designates 192.55.52.151 as permitted sender) client-ip=192.55.52.151; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 May 2018 01:26:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,432,1520924400"; d="scan'208";a="57726023" Received: from yhuang6-ux31a.sh.intel.com ([10.239.197.97]) by fmsmga001.fm.intel.com with ESMTP; 23 May 2018 01:26:40 -0700 From: "Huang, Ying" To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , "Kirill A. Shutemov" , Andrea Arcangeli , Michal Hocko , Johannes Weiner , Shaohua Li , Hugh Dickins , Minchan Kim , Rik van Riel , Dave Hansen , Naoya Horiguchi , Zi Yan Subject: [PATCH -mm -V3 04/21] mm, THP, swap: Support PMD swap mapping in swapcache_free_cluster() Date: Wed, 23 May 2018 16:26:08 +0800 Message-Id: <20180523082625.6897-5-ying.huang@intel.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180523082625.6897-1-ying.huang@intel.com> References: <20180523082625.6897-1-ying.huang@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Huang Ying Previously, during swapout, all PMD page mapping will be split and replaced with PTE swap mapping. And when clearing the SWAP_HAS_CACHE flag for the huge swap cluster in swapcache_free_cluster(), the huge swap cluster will be split. Now, during swapout, the PMD page mapping will be changed to PMD swap mapping. So when clearing the SWAP_HAS_CACHE flag, the huge swap cluster will only be split if the PMD swap mapping count is 0. Otherwise, we will keep it as the huge swap cluster. So that we can swapin a THP as a whole later. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan --- mm/swapfile.c | 41 ++++++++++++++++++++++++++++++----------- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index 075048032383..8dbc0f9b2f90 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -514,6 +514,18 @@ static void dec_cluster_info_page(struct swap_info_struct *p, free_cluster(p, idx); } +#ifdef CONFIG_THP_SWAP +static inline int cluster_swapcount(struct swap_cluster_info *ci) +{ + if (!ci || !cluster_is_huge(ci)) + return 0; + + return cluster_count(ci) - SWAPFILE_CLUSTER; +} +#else +#define cluster_swapcount(ci) 0 +#endif + /* * It's possible scan_swap_map() uses a free cluster in the middle of free * cluster list. Avoiding such abuse to avoid list corruption. @@ -905,6 +917,7 @@ static void swap_free_cluster(struct swap_info_struct *si, unsigned long idx) struct swap_cluster_info *ci; ci = lock_cluster(si, offset); + memset(si->swap_map + offset, 0, SWAPFILE_CLUSTER); cluster_set_count_flag(ci, 0, 0); free_cluster(si, idx); unlock_cluster(ci); @@ -1288,24 +1301,30 @@ static void swapcache_free_cluster(swp_entry_t entry) ci = lock_cluster(si, offset); VM_BUG_ON(!cluster_is_huge(ci)); + VM_BUG_ON(!is_cluster_offset(offset)); + VM_BUG_ON(cluster_count(ci) < SWAPFILE_CLUSTER); map = si->swap_map + offset; - for (i = 0; i < SWAPFILE_CLUSTER; i++) { - val = map[i]; - VM_BUG_ON(!(val & SWAP_HAS_CACHE)); - if (val == SWAP_HAS_CACHE) - free_entries++; + if (!cluster_swapcount(ci)) { + for (i = 0; i < SWAPFILE_CLUSTER; i++) { + val = map[i]; + VM_BUG_ON(!(val & SWAP_HAS_CACHE)); + if (val == SWAP_HAS_CACHE) + free_entries++; + } + if (free_entries != SWAPFILE_CLUSTER) + cluster_clear_huge(ci); } if (!free_entries) { - for (i = 0; i < SWAPFILE_CLUSTER; i++) - map[i] &= ~SWAP_HAS_CACHE; + for (i = 0; i < SWAPFILE_CLUSTER; i++) { + val = map[i]; + VM_BUG_ON(!(val & SWAP_HAS_CACHE) || + val == SWAP_HAS_CACHE); + map[i] = val & ~SWAP_HAS_CACHE; + } } - cluster_clear_huge(ci); unlock_cluster(ci); if (free_entries == SWAPFILE_CLUSTER) { spin_lock(&si->lock); - ci = lock_cluster(si, offset); - memset(map, 0, SWAPFILE_CLUSTER); - unlock_cluster(ci); mem_cgroup_uncharge_swap(entry, SWAPFILE_CLUSTER); swap_free_cluster(si, idx); spin_unlock(&si->lock);