From patchwork Thu Jul 12 23:36:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 10522417 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id CE9846032C for ; Thu, 12 Jul 2018 23:37:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BFDB627D16 for ; Thu, 12 Jul 2018 23:37:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B3B38298A6; Thu, 12 Jul 2018 23:37:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3312A27D16 for ; Thu, 12 Jul 2018 23:37:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 704F36B026C; Thu, 12 Jul 2018 19:37:07 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5C6126B0266; Thu, 12 Jul 2018 19:37:07 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F4A16B026B; Thu, 12 Jul 2018 19:37:07 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf0-f199.google.com (mail-pf0-f199.google.com [209.85.192.199]) by kanga.kvack.org (Postfix) with ESMTP id C2A206B0010 for ; Thu, 12 Jul 2018 19:37:06 -0400 (EDT) Received: by mail-pf0-f199.google.com with SMTP id f9-v6so18316633pfn.22 for ; Thu, 12 Jul 2018 16:37:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=VFfjJ5gJlQUtVxIzFw++gmL3WRLChAgQUmpKqn6i/6I=; b=sZFrId5FW5z/sX6yM4SnmN7bEM7HiBNKIqgSgGnyk3sWDV3EoRW7HWgTbS/AhDl9FL YD8s0nmkSarMTzdaw1Lylz5xVj6SJziaxsB7pPEkG1iERJKxDGIR8jUI0MrICPg0V6Al A/LERRPJwLEUTLfj1PgukVj1bQBIrXXbd0Fq3yBG0aDKvXQGD6n8sDTPPzRO3/mRQ7/B RtosBD7CiOq4QNlJrhzb/Y3PdA1OfKFBwRV/JTYp/eNeEofYsK+I4N9cU9cxbs4b+2ci PNe65yLZBEBUIA5qVmOZr1Yw1dRpOON66DoRfB1wOe131Bd0k+912cZyOqcjib+4+2F5 IhiA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AOUpUlHxUcfGqgDMM+hfP1DwzbmIDSN5PfE/q01+Vj+erxXrrNqPrOHM VVm2BOucg4womO3wj913ZKX9jCznJnM4RGynTtfI4MKJMrdce+c+4AM1ivPQjqT7g5Lwv9oEFuq Eo13OjHB1ELxuLNmCvwkyRxtQRkOZ2iMvc/lMcmikIBdjDT+NWF529uw0gWW6n+Rlkw== X-Received: by 2002:a62:3d86:: with SMTP id x6-v6mr4480804pfj.192.1531438626477; Thu, 12 Jul 2018 16:37:06 -0700 (PDT) X-Google-Smtp-Source: AAOMgpdLy9nTp8NwCquse8p5wU+Luiusi3mwCpJrEC6flDccwlmB1XyowgFMgi4TQ/0p8ZnJhoR6 X-Received: by 2002:a62:3d86:: with SMTP id x6-v6mr4480763pfj.192.1531438625722; Thu, 12 Jul 2018 16:37:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1531438625; cv=none; d=google.com; s=arc-20160816; b=NfZ7obghNDuELZxMRzvvziUBHleLCINctPNkGawdM/sOEwHYbqyjxYISKppFpSd9+I 19aXGNikffqfpw7LnpTWPhOp+DleQFLV78zPphVq9pPiprzKrLipqc3rfxsan2SyDxMu 61G6MbTuQsGxVOfDlcXcRHf8VgQsJ1O9vJK7cNUQvBnQK3ebrwGbmgLDtCjmn6AqbRKI 1/v8e+IkZSDjnK5InkfdelTmAhNOtdKl734cMYIvau0S/FcXuyv7LAM4VE9CE69MKOxc ZDmdhSukBgZIjXj6ulPjSv3RfDHF2gHtW41Ls9hYqb76gUzLSsJrdmFH93nkyhFNtsd6 dIug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=VFfjJ5gJlQUtVxIzFw++gmL3WRLChAgQUmpKqn6i/6I=; b=cGSq7lXZW3m5RvWPMotWmwCa3Cs0w/ZrEINr+BO2kKHMlhjmHxbiio8X2rU44WAosL Zf2X+l8Ft8/JXWhp2s6qH1e0WW8zE8PSlxzT9pzPhhjSmgaZzVnaHwdV6E9QXLY4T12X 0yAw0Aa7pchowStJJ4SSk302bcO9b4ArfzENTr+9ttpbp/8C6DEeweSk962L0pIH/Bha mFJ/u2kZYW763zE2oUwIAu0hxEi7munZbtfc5JVFdHn+grDTppXEZkCbUVGsor5+Rdv4 pKZApAsg1gNpJ1bIb53YHeqWxWFdFIafI4WVsIfn2c5Jm80/jIFC+rcHiMg2//6A4p5O cwXQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga17.intel.com (mga17.intel.com. [192.55.52.151]) by mx.google.com with ESMTPS id o184-v6si21060983pga.520.2018.07.12.16.37.05 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Jul 2018 16:37:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ying.huang@intel.com designates 192.55.52.151 as permitted sender) client-ip=192.55.52.151; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Jul 2018 16:37:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,345,1526367600"; d="scan'208";a="57272256" Received: from yongjuns-mobl2.ccr.corp.intel.com (HELO yhuang6-ux31a.ccr.corp.intel.com) ([10.254.213.67]) by orsmga006.jf.intel.com with ESMTP; 12 Jul 2018 16:36:52 -0700 From: "Huang, Ying" To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , Michal Hocko , Johannes Weiner , Shaohua Li , Hugh Dickins , Minchan Kim , Rik van Riel , Daniel Jordan , Dan Williams , Dave Hansen Subject: [PATCH 4/6] swap: Unify normal/huge code path in put_swap_page() Date: Fri, 13 Jul 2018 07:36:34 +0800 Message-Id: <20180712233636.20629-5-ying.huang@intel.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20180712233636.20629-1-ying.huang@intel.com> References: <20180712233636.20629-1-ying.huang@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Huang Ying In this patch, the normal/huge code path in put_swap_page() and several helper functions are unified to avoid duplicated code, bugs, etc. and make it easier to review the code. The removed lines are more than added lines. And the binary size is kept exactly same when CONFIG_TRANSPARENT_HUGEPAGE=n. Signed-off-by: "Huang, Ying" Suggested-by: Dave Hansen Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Daniel Jordan Cc: Dan Williams --- mm/swapfile.c | 83 ++++++++++++++++++++++++++--------------------------------- 1 file changed, 37 insertions(+), 46 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index 160f78072667..e0df8d22ac92 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -205,8 +205,16 @@ static void discard_swap_cluster(struct swap_info_struct *si, #ifdef CONFIG_THP_SWAP #define SWAPFILE_CLUSTER HPAGE_PMD_NR + +#define nr_swap_entries(nr) (nr) #else #define SWAPFILE_CLUSTER 256 + +/* + * Define nr_swap_entries() as constant to let compiler to optimize + * out some code if !CONFIG_THP_SWAP + */ +#define nr_swap_entries(nr) 1 #endif #define LATENCY_LIMIT 256 @@ -1249,18 +1257,7 @@ void swap_free(swp_entry_t entry) /* * Called after dropping swapcache to decrease refcnt to swap entries. */ -static void swapcache_free(swp_entry_t entry) -{ - struct swap_info_struct *p; - - p = _swap_info_get(entry); - if (p) { - if (!__swap_entry_free(p, entry, SWAP_HAS_CACHE)) - free_swap_slot(entry); - } -} - -static void swapcache_free_cluster(swp_entry_t entry) +void put_swap_page(struct page *page, swp_entry_t entry) { unsigned long offset = swp_offset(entry); unsigned long idx = offset / SWAPFILE_CLUSTER; @@ -1269,39 +1266,41 @@ static void swapcache_free_cluster(swp_entry_t entry) unsigned char *map; unsigned int i, free_entries = 0; unsigned char val; - - if (!IS_ENABLED(CONFIG_THP_SWAP)) - return; + int nr = nr_swap_entries(hpage_nr_pages(page)); si = _swap_info_get(entry); if (!si) return; - ci = lock_cluster(si, offset); - VM_BUG_ON(!cluster_is_huge(ci)); - map = si->swap_map + offset; - for (i = 0; i < SWAPFILE_CLUSTER; i++) { - val = map[i]; - VM_BUG_ON(!(val & SWAP_HAS_CACHE)); - if (val == SWAP_HAS_CACHE) - free_entries++; - } - if (!free_entries) { - for (i = 0; i < SWAPFILE_CLUSTER; i++) - map[i] &= ~SWAP_HAS_CACHE; - } - cluster_clear_huge(ci); - unlock_cluster(ci); - if (free_entries == SWAPFILE_CLUSTER) { - spin_lock(&si->lock); + if (nr == SWAPFILE_CLUSTER) { ci = lock_cluster(si, offset); - memset(map, 0, SWAPFILE_CLUSTER); + VM_BUG_ON(!cluster_is_huge(ci)); + map = si->swap_map + offset; + for (i = 0; i < SWAPFILE_CLUSTER; i++) { + val = map[i]; + VM_BUG_ON(!(val & SWAP_HAS_CACHE)); + if (val == SWAP_HAS_CACHE) + free_entries++; + } + if (!free_entries) { + for (i = 0; i < SWAPFILE_CLUSTER; i++) + map[i] &= ~SWAP_HAS_CACHE; + } + cluster_clear_huge(ci); unlock_cluster(ci); - mem_cgroup_uncharge_swap(entry, SWAPFILE_CLUSTER); - swap_free_cluster(si, idx); - spin_unlock(&si->lock); - } else if (free_entries) { - for (i = 0; i < SWAPFILE_CLUSTER; i++, entry.val++) { + if (free_entries == SWAPFILE_CLUSTER) { + spin_lock(&si->lock); + ci = lock_cluster(si, offset); + memset(map, 0, SWAPFILE_CLUSTER); + unlock_cluster(ci); + mem_cgroup_uncharge_swap(entry, SWAPFILE_CLUSTER); + swap_free_cluster(si, idx); + spin_unlock(&si->lock); + return; + } + } + if (nr == 1 || free_entries) { + for (i = 0; i < nr; i++, entry.val++) { if (!__swap_entry_free(si, entry, SWAP_HAS_CACHE)) free_swap_slot(entry); } @@ -1325,14 +1324,6 @@ int split_swap_cluster(swp_entry_t entry) } #endif -void put_swap_page(struct page *page, swp_entry_t entry) -{ - if (!PageTransHuge(page)) - swapcache_free(entry); - else - swapcache_free_cluster(entry); -} - static int swp_entry_cmp(const void *ent1, const void *ent2) { const swp_entry_t *e1 = ent1, *e2 = ent2;