From patchwork Fri Jun 22 03:51:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 10481131 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A28F460532 for ; Fri, 22 Jun 2018 03:55:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 967E728F94 for ; Fri, 22 Jun 2018 03:55:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8B09328F91; Fri, 22 Jun 2018 03:55:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1514828F94 for ; Fri, 22 Jun 2018 03:55:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DCF156B000A; Thu, 21 Jun 2018 23:55:19 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D54E56B000C; Thu, 21 Jun 2018 23:55:19 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BCF126B000D; Thu, 21 Jun 2018 23:55:19 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl0-f69.google.com (mail-pl0-f69.google.com [209.85.160.69]) by kanga.kvack.org (Postfix) with ESMTP id 7559A6B000A for ; Thu, 21 Jun 2018 23:55:19 -0400 (EDT) Received: by mail-pl0-f69.google.com with SMTP id j22-v6so2101315pll.7 for ; Thu, 21 Jun 2018 20:55:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=+yUdAcP6Y1/g+YIH0tAV6VHH0Luix62nLrjYRbiglY0=; b=A6FUCXO+OLfUqu0CVxjAif7Y8avPUGEF32kjM6wPhaBgiM9vSeGRFDq4Jm9XE3+DB/ c5+o6dcC5AwJrLu66nUqpb0gqB6hZKGZy+aEe2JzaaaFfWW2FOieB8UkeqVaoJHgCtvQ jwvc2797rHr3ZbNI0f1vu+gXvr2Je4XZ27JlwC7YZuFNfJ1ZX7i6RDawWAISnZ+taMol G5eJjXZvT/AlxmDkmzjotnN5b+YYVHCBtKr2zGG8pIp3l+FeIF+XfTkRPAAfHMrrBs3q 2ThBimgjl8cr47+FjmEI7+Oe4yeBJGxHAVhC79xUZewypTBcRls+X4PgqG+bgBHZNn4e IiEg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APt69E00tSOzcl7KEAM4D5X7xG+rCDQTkS7xCPeDtMupawo2CkNLbi5p puE2hGNjpz1Gca9Qhnyu7xATaPKANL3jDMhLVUaFHFThSEgp84IqnfXD9WAhr9AMTMBeXion2kU dWzTqipqVjIP4RgSim86lpVEaulDLqlnqqZmDWEf1jvzV1Ky2U9QoZDjAhoHg9y1W+w== X-Received: by 2002:aa7:8491:: with SMTP id u17-v6mr30178312pfn.48.1529639719156; Thu, 21 Jun 2018 20:55:19 -0700 (PDT) X-Google-Smtp-Source: ADUXVKIiOpub/+eG3ubvvWUuhZpby8uqjOltsADuPg5JIAhfqRJHsO0TfDDoU53JOVDhC5eD/Pf3 X-Received: by 2002:aa7:8491:: with SMTP id u17-v6mr30178275pfn.48.1529639717870; Thu, 21 Jun 2018 20:55:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1529639717; cv=none; d=google.com; s=arc-20160816; b=Hkf6i65GyMn7hUXFqTt0bh7EFDqJZaW/B+Wk6kmALuQeFz05Tw+IMAfgcBdimpCx5u pKYZauxyhWDa6em8hKGbka91wcbGkTLHScX9HSRhOlDqi2P+UiAlnXpEhgLVUNeN9dUU jCBa4zcRT710KcFxtv7LYJXghuyhK0gaX+5pOh44Md4/JSVD6/AjOKGuVP6lMEurnHZG xQQ2hTnBVmwZwoYWKWF1szv6JX6pCA5qV/RTqNObStlu/JgepCXyZ3VV+QQBtqINy596 uc5eLtgEhl67m61xSADpLxNW85Lpp2UOoTkd1GQnl+kycRKp6Is0yuLBuzXCzlkdld3z WCcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=+yUdAcP6Y1/g+YIH0tAV6VHH0Luix62nLrjYRbiglY0=; b=0OXKGpq4hh2BS60CIi85m5i+HhUJDtd9olc+hiWKcUtXqGPC4sucvdZGgWTp5xphDC uyiRm3fbG0HKruaJPKXIGBxRNCotlIAskg3S/YC9xznaamG36eZbdWdkAb2YznrfFH/Y fBpfnQVqJFaBrUM+PPF+33Nr/B0TqktrYshHqHTN/82rMaNFlbS79tY6u0WnrJwjrT6A fuf2mSX5ju+34ETUrng9Es8b4IarsMh9H1/CHwaZKT0sjQzFxEP5uXs+0pGxKN8Y/eRm 4L4+BVI+jtCDXtI7xhXZ3amZivDptaFl9e2hVbjFW/PQG0AEl7F3w6CuhKdaLdgkEqeM fT7A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga05.intel.com (mga05.intel.com. [192.55.52.43]) by mx.google.com with ESMTPS id q13-v6si6671342plr.220.2018.06.21.20.55.17 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Jun 2018 20:55:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ying.huang@intel.com designates 192.55.52.43 as permitted sender) client-ip=192.55.52.43; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Jun 2018 20:55:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,255,1526367600"; d="scan'208";a="65335082" Received: from wanpingl-mobl.ccr.corp.intel.com (HELO yhuang6-ux31a.ccr.corp.intel.com) ([10.254.212.200]) by fmsmga004.fm.intel.com with ESMTP; 21 Jun 2018 20:55:15 -0700 From: "Huang, Ying" To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , "Kirill A. Shutemov" , Andrea Arcangeli , Michal Hocko , Johannes Weiner , Shaohua Li , Hugh Dickins , Minchan Kim , Rik van Riel , Dave Hansen , Naoya Horiguchi , Zi Yan , Daniel Jordan Subject: [PATCH -mm -v4 04/21] mm, THP, swap: Support PMD swap mapping in swapcache_free_cluster() Date: Fri, 22 Jun 2018 11:51:34 +0800 Message-Id: <20180622035151.6676-5-ying.huang@intel.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20180622035151.6676-1-ying.huang@intel.com> References: <20180622035151.6676-1-ying.huang@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Huang Ying Previously, during swapout, all PMD page mapping will be split and replaced with PTE swap mapping. And when clearing the SWAP_HAS_CACHE flag for the huge swap cluster in swapcache_free_cluster(), the huge swap cluster will be split. Now, during swapout, the PMD page mapping will be changed to PMD swap mapping. So when clearing the SWAP_HAS_CACHE flag, the huge swap cluster will only be split if the PMD swap mapping count is 0. Otherwise, we will keep it as the huge swap cluster. So that we can swapin a THP as a whole later. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel Jordan --- mm/swapfile.c | 41 ++++++++++++++++++++++++++++++----------- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index 48e2c54385ee..36f4b6451360 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -514,6 +514,18 @@ static void dec_cluster_info_page(struct swap_info_struct *p, free_cluster(p, idx); } +#ifdef CONFIG_THP_SWAP +static inline int cluster_swapcount(struct swap_cluster_info *ci) +{ + if (!ci || !cluster_is_huge(ci)) + return 0; + + return cluster_count(ci) - SWAPFILE_CLUSTER; +} +#else +#define cluster_swapcount(ci) 0 +#endif + /* * It's possible scan_swap_map() uses a free cluster in the middle of free * cluster list. Avoiding such abuse to avoid list corruption. @@ -905,6 +917,7 @@ static void swap_free_cluster(struct swap_info_struct *si, unsigned long idx) struct swap_cluster_info *ci; ci = lock_cluster(si, offset); + memset(si->swap_map + offset, 0, SWAPFILE_CLUSTER); cluster_set_count_flag(ci, 0, 0); free_cluster(si, idx); unlock_cluster(ci); @@ -1288,24 +1301,30 @@ static void swapcache_free_cluster(swp_entry_t entry) ci = lock_cluster(si, offset); VM_BUG_ON(!cluster_is_huge(ci)); + VM_BUG_ON(!is_cluster_offset(offset)); + VM_BUG_ON(cluster_count(ci) < SWAPFILE_CLUSTER); map = si->swap_map + offset; - for (i = 0; i < SWAPFILE_CLUSTER; i++) { - val = map[i]; - VM_BUG_ON(!(val & SWAP_HAS_CACHE)); - if (val == SWAP_HAS_CACHE) - free_entries++; + if (!cluster_swapcount(ci)) { + for (i = 0; i < SWAPFILE_CLUSTER; i++) { + val = map[i]; + VM_BUG_ON(!(val & SWAP_HAS_CACHE)); + if (val == SWAP_HAS_CACHE) + free_entries++; + } + if (free_entries != SWAPFILE_CLUSTER) + cluster_clear_huge(ci); } if (!free_entries) { - for (i = 0; i < SWAPFILE_CLUSTER; i++) - map[i] &= ~SWAP_HAS_CACHE; + for (i = 0; i < SWAPFILE_CLUSTER; i++) { + val = map[i]; + VM_BUG_ON(!(val & SWAP_HAS_CACHE) || + val == SWAP_HAS_CACHE); + map[i] = val & ~SWAP_HAS_CACHE; + } } - cluster_clear_huge(ci); unlock_cluster(ci); if (free_entries == SWAPFILE_CLUSTER) { spin_lock(&si->lock); - ci = lock_cluster(si, offset); - memset(map, 0, SWAPFILE_CLUSTER); - unlock_cluster(ci); mem_cgroup_uncharge_swap(entry, SWAPFILE_CLUSTER); swap_free_cluster(si, idx); spin_unlock(&si->lock);