From patchwork Sat Apr 29 08:27:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yin Fengwei X-Patchwork-Id: 13226928 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C65DAC77B7E for ; Sat, 29 Apr 2023 08:28:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B1019900002; Sat, 29 Apr 2023 04:27:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A98426B0078; Sat, 29 Apr 2023 04:27:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 938EF900002; Sat, 29 Apr 2023 04:27:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 811E96B0075 for ; Sat, 29 Apr 2023 04:27:59 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 03B501403F7 for ; Sat, 29 Apr 2023 08:27:58 +0000 (UTC) X-FDA: 80733750678.04.B3EE6EF Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by imf19.hostedemail.com (Postfix) with ESMTP id 4CC8F1A0015 for ; Sat, 29 Apr 2023 08:27:56 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=DnHKoEoR; spf=pass (imf19.hostedemail.com: domain of fengwei.yin@intel.com designates 134.134.136.100 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682756877; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=zKQU3h0mOgLQRf8uhsUpipR50Y7q4Nl87sKkM9QlYAU=; b=xckyhL3tuee3FHG5pEObuTTKk6HeaJL5pWXqVbCF8uOoRFGUYGGWcC/WtRyHbwR1kdrBjv QlBYEFLY7zDiCd9y+Spgq+hBwt2H88+SHpWyAInFUSsAsN4YvTyefKAPjLkJymddWqnVrP VjWwiH2KtglMe3SOOuKvD/faHk4akC0= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=DnHKoEoR; spf=pass (imf19.hostedemail.com: domain of fengwei.yin@intel.com designates 134.134.136.100 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682756877; a=rsa-sha256; cv=none; b=a7SJwnOiplXd+T1HUP/Me5XkmUklICubADHaLdw0uaktJTYVDoa76MzW9Z46NVRviTMZWd ID7VzysNDKngUAnWHVIaTa/TonJ3pB5iO9ayU3o5QG91Y9IKov/quope0mGgvDMVz8mZdz b8Hf4tu7hU6tDH1BgNJaHeZPuHDXWwU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682756875; x=1714292875; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=/GxFtTECdCkQQhYUpcyrIqPHTrQk5UNWslrGe54YY90=; b=DnHKoEoRa4cLOtZKGFCAJ3pgpHDJiIn+3UkPBPWEyDRL1rNBqCCnFPm6 xDd0kdTWHknluYP4US/BIAgolJwQcvMyuv3ZoZr6/Fa1OuMIhUsZ8ZPR3 CscKgddM6Fn+cc+DJQoiJ7WffD4iybWg4H90ZkaTyZjYrviIu1A3NIBgo zwFNUrlQCfmeB2ca7rCKEg/0rKrUTk6CGPx7WYe29G3DmCyPe6BMeuqMJ oX/pIv8JwkW+VhNTSx8371/SaoHEc44ZeL6mvkUUIksGJoOKCIWUXLiRF v3U+el3NsSOThW5I+IRh8vvvBOQrWNeD/vDbYKgz4gyRLqDaPJLVc5Lzl Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10694"; a="413326016" X-IronPort-AV: E=Sophos;i="5.99,236,1677571200"; d="scan'208";a="413326016" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Apr 2023 01:27:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10694"; a="784580207" X-IronPort-AV: E=Sophos;i="5.99,236,1677571200"; d="scan'208";a="784580207" Received: from fyin-dev.sh.intel.com ([10.239.159.32]) by FMSMGA003.fm.intel.com with ESMTP; 29 Apr 2023 01:27:51 -0700 From: Yin Fengwei To: linux-mm@kvack.org, akpm@linux-foundation.org, willy@infradead.org, kirill@shutemov.name, yuzhao@google.com, ryan.roberts@arm.com, ying.huang@intel.com Cc: fengwei.yin@intel.com Subject: [PATCH v3 0/2] Reduce lock contention related with large folio Date: Sat, 29 Apr 2023 16:27:57 +0800 Message-Id: <20230429082759.1600796-1-fengwei.yin@intel.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: w8zik9kibtxxioezoq5kxemykzonodzg X-Rspamd-Queue-Id: 4CC8F1A0015 X-HE-Tag: 1682756876-61341 X-HE-Meta: U2FsdGVkX1+uyQv6SZEwtPjyESsiH+3vRDLBhYZebU/WaIiAlYR0/tk2B6HqNB6HK9MW2a6Gg8rdeuD187vliJo613qWTcsfMXKNDeNjMvJSIPqtRlB2+TlR4qGwEFIIJbdXApjhZHfufVso5O8EMXnP1Pym+IgkFC0FCLtbdTk+1r3iFI+D6o3wgzmKPq1uZOuFNE222e+wDrZaVyDPQ0brBxOqegEt7kx/QrFcjO6w/EV+cd8ZCmFTWXB1cPQSdoxmLRIKnVhgFiDsWE6dzr6jn9Sv5zJjE+W0YGNpHEgqSlcJtAxKyjshWORNCkv/8KPDdhpJ7WwwRMRzgDr+KdWX87SSfXGidDOCak8T2gRIsV6cNttFA5OybDmlTA2I75mEQ8oJpND2FzF0CYfEDHK0PFTFKqkJdnm9nglNETZKy+LcZ6doWPJo6VviI538k/21Sq3BpDQn/Mkpx2bdoFxDy+BqJSdsORm7/AGGPcEbk6Uc50txi4Usc5atSWzO72pap6ng8DutvBhFc1ylUUsHefxq5fFFAqiRGUQONiPMgbcoysDREMMscNaU5+Z1qbQemAd39/5E7r5UOk8MjYnH7FRFM0YDzyg/AvHHUtx9EFisTTdeHafxOCoYtcz6rhaRyE35cIBO1OMcR9J03g8bAjm3vf7qvY/mZhRQ4/UeAs+EQ4gCfm8gMQOAqkqdj3/cyAkeDoJooNBTDkM9JrrL11ThCRtpRvAoCD7OK45arscblj5hMF21nTBl8Hkr0STB0ep1Mi/qZAKQ3lMPhlR45qtyeRL6CQaA24XQY1I1V7jxCXie6LEjJlLJ4/aazRp9opm1fh3ThqrFM3A4yys0vGavlJaR3rfuMGTKyW5F0eWEHz0HiInWRKw5SGlKeqOZ+UylELjFxM7EC2TrzqybWzGIRYxXXOCbu3s1QJx5QkqONtmJEggWE2aZ/CMrFDtYsJ8faV90bAW3PVk ZyAwbm+l m5RiupJejDfjV+a7dHvcOQuhmHUSk1861jmXxaDDJIywIakTOTrDAHVfYalWM2F526rnp0IpHHnkSNuNnRrbPFkrqTxUbd4lMDNBqpfwS7FeJgCmuT1sXwpHO4/GuuydpFP80po8k4HauVn83gGC1KvdjyNU0u8VFrPD98whCNM5euWNnHpCZ/D0CL3D4m86siz87LObQEZ/m4tL47bTOEVpY/ZpkCnvoGQSxRL+n1JmfX/p7wKlqCUpOvqV76MFpBu7Y X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: yan tried to enable the large folio for anonymous mapping [1]. Unlike large folio for page cache which doesn't trigger frequent page allocation/free, large folio for anonymous mapping is allocated/freeed more frequently. So large folio for anonymous mapping exposes some lock contention. Ryan mentioned the deferred queue lock in [1]. We also met other two lock contention: lru lock and zone lock. This series tries to mitigate the deferred queue lock and reduce lru lock in some level. The patch1 tries to reduce deferred queue lock by not acquiring queue lock when check whether the folio is in deferred list or not. Test page fault1 of will-it-scale showed 60% deferred queue lock contention reduction. The patch2 tries to reduce lru lock by allowing batched add large folio to lru list. Test page fault1 of will-it-scale showed 20% lru lock contention reduction. The zone lock contention happens on large folio free path and related with commit f26b3fa04611 "mm/page_alloc: limit number of high-order pages on PCP during bulk free" and will not be address by this series. [1] https://lore.kernel.org/linux-mm/20230414130303.2345383-1-ryan.roberts@arm.com/ Changelog from v2: - Rebased to v6.3-rc7 - Removed Tested-by: Ryan Roberts as patches got some updated after Ryan tested them. - Updated the perf data change for deferred queue lock and lru lock with v3. - recheck whether folio is in deferred_list or not after take the deferred queue lock as Kirill suggested. Changelog from v1: For patch2: - Add Reported-by from Huang Ying which was missed by my mistake. - Fix kernel panic issue. The folio_batch_add() can have folio which doesn't reference folio directly: - For mlock usage, add new interface with extra parameter nr_pages. And callee pass nr_pages by direct reference folio. - For swap, shawdow and dax entries as parameter folio, treat the nr_pages as 1. With the fix, the stress testing can run 12 hours without any issue while hit kernel panic in around 3 minutes. - Update the lock contention info in commit message. - Change field name from pages_nr to nr_pages as Ying's suggestion. For this version, still use PAGEVEC_SIZE as max nr_pages in fbatch. We can revise it after we make decision about the page order for anonymous large folio. Yin Fengwei (2): THP: avoid lock when check whether THP is in deferred list lru: allow large batched add large folio to lru list include/linux/pagevec.h | 46 ++++++++++++++++++++++++++++++++++++++--- mm/huge_memory.c | 17 ++++++++++----- mm/mlock.c | 7 +++---- mm/swap.c | 3 +-- 4 files changed, 59 insertions(+), 14 deletions(-)