From patchwork Tue Apr 25 08:46:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yin Fengwei X-Patchwork-Id: 13223081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0043C6FD18 for ; Tue, 25 Apr 2023 08:46:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1C3D66B0071; Tue, 25 Apr 2023 04:46:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 175516B0074; Tue, 25 Apr 2023 04:46:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 03A266B0075; Tue, 25 Apr 2023 04:46:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E56FB6B0071 for ; Tue, 25 Apr 2023 04:46:32 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id ACE5C1C6662 for ; Tue, 25 Apr 2023 08:46:32 +0000 (UTC) X-FDA: 80719282224.28.52C392C Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by imf18.hostedemail.com (Postfix) with ESMTP id 01E391C0006 for ; Tue, 25 Apr 2023 08:46:28 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Gxnul6s7; spf=pass (imf18.hostedemail.com: domain of fengwei.yin@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682412389; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=i8JDXy0itk9Fhc/Jfs91cnIFOiLTb74xcttCy3RYkqM=; b=ciMlgZXW8Y2NsC1eGYoySGmh7g2h1QD9ymrCW5evZ8873WVmD6MH+A0ziECU2wvR+KdRLN laiLWrNPgVcus0Mf2sw41jipPhumkcDC5p7nHGZcmuwiGRYy/4mZYWfa1iVgKtj+Om0XDo YYTHr7NGgSTryMK8x+JyDnuFw/FSBRQ= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Gxnul6s7; spf=pass (imf18.hostedemail.com: domain of fengwei.yin@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682412389; a=rsa-sha256; cv=none; b=cTHLgsyyC+gg+tSy3JDiQBH835X0ad5NitGdsQTfx8dMULGIOTtjUEWSuweIozpSmowQrQ wHOkDw8UAOHT2etJud67JScGjwrUPO35OaApvj9p+xKe8ylaOOsrHafRvPZbAlwYeK7Nq1 g4Mp+0cBCm61jj8c+dwiuSye5gIn06I= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682412389; x=1713948389; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=0+TcKXrGOoCrFYqKG38zct6+3lDWfW6itU95Hr7LgUA=; b=Gxnul6s7sDBox4MmE+X5LCxvPBsno79VEMbWguH5tmInbfCLd6NgJKls lzDdZIibBDuy9BvVkRFUqJ2puKApVHUjU1JHAtWjozgWFdf//e6HgGE9h sR6ZiYV/0zTVn6tX+4FyvJrye7KU8iiszl142ptqcN8UOI8JxfUgOg91M R3D8N/MEivKf0W4jwEnzDgCtIoI45OjTjQFjRjbBgfUNpP1uHfTI8eKzJ aCNqtu7AvHUkUad1In+VMcLK7Qhs6TFUi2KizgmWxmVLj+TphVOCQvjtZ MJbLJ1Xc1Hl3q7MVqNl75H8thHUn+VV5b4MnduqYgciaZDatxRWlZ0goY Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10690"; a="345454668" X-IronPort-AV: E=Sophos;i="5.99,225,1677571200"; d="scan'208";a="345454668" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Apr 2023 01:46:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10690"; a="696097997" X-IronPort-AV: E=Sophos;i="5.99,225,1677571200"; d="scan'208";a="696097997" Received: from fyin-dev.sh.intel.com ([10.239.159.32]) by fmsmga007.fm.intel.com with ESMTP; 25 Apr 2023 01:46:22 -0700 From: Yin Fengwei To: linux-mm@kvack.org, akpm@linux-foundation.org, willy@infradead.org, yuzhao@google.com, ryan.roberts@arm.com, ying.huang@intel.com Cc: fengwei.yin@intel.com Subject: [PATCH v2 0/2] Reduce lock contention related with large folio Date: Tue, 25 Apr 2023 16:46:25 +0800 Message-Id: <20230425084627.3573866-1-fengwei.yin@intel.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-Rspamd-Queue-Id: 01E391C0006 X-Stat-Signature: hudymdy6cewikyur4m1kziwjfgt8ehj7 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1682412388-393599 X-HE-Meta: U2FsdGVkX18Fcet6zk+WA8sD2CRD421LY8TDHzAoIXl6byuqt6zTTTga1BpCxvFK9/mrPxkGl5xuqscJjMwlkjqW5a4mFEvYFz2ya3LG33FhJcTrCQZG8vr4smZyhuzrzhR74CMQPm2xFNUBHUJp2qsiI8FNaZm67IzSll0NIlfb4GXe+BZa45u4oiY2CWWjKtVI59/2xmWX4kdF6eZRrQCDuvFJe+V8s4xjjavu3GNczm1qqLV/5CNaP5ebKXnjaj8TmTkoxKZXlfUmRGxguAM/P6N4QUvji7FqH/80to6FbwTa/sNDsYj3bvhCsxKz+UWcV76ErrnRsah5r57IHyzpcUah5J9u1RQi3EctAD2b05S+X/BxNoJf65XftIfwS1qvY3hncy2yRYZBGFK8OjKRodX7X1ZXNsBAQquQNAFOXweJDCzuZPHAcn9KiHAiCnOdHQCpmbUFARQNgTcXm3mOU56SvAubj7T3CiqCBzRqWrjCRL8xcr67uIEJjm3bc8Dpgy+xWtwUl6LGsVhUfztU8Em+f5UZ1+cG8bdl7M5bQiTK2ZQpGVA2iPixSjsnBNAQ9LduMLC232mN70EnRxS9SvtVCWnz5Xvb+omoRab98au1x/2dnb6SVmp9/5GKpD36/wA1CfCWYRyKBkSOc/ZSLEEcFxiKB4A3ejGurLDaU0iynnKrcQ/RpVrecle854miqrcOO5KrqLk0dALDhZBjFLRoNK2ulFCaYG5Y2/EJcaPH3GnLpBrME472wY64KiTe0MrEDasxwdQsP8aRYbHBGGMMIpl02EvYpSfPv5ecx0XI+EjK6fvumi+WmiIN5K8dnAD0rVqGgJYaYA1Z+ILH7413LyB5ogE3/U/Pq7N1qJ0MtE7fQmjFvAUpaZFE6uNIHwzU4ICBi9LQmtA2Pszf0MAb+06S/gXuo0a7OWjm3caED/iEt5v/ElMMvWg+bmlLs7CHUQuPtNXSHEt +dyxdczP In9d0xNIJx1P43DppewIe7ishbVtNC3fNRafTAHd5Y4xM52XWVK2hvarB95UnNJdydATFbom2hfUOpbWaEleGLiuY8sqC5+ra0ItzpL9r4pxDU/0orWsMF/bzLbW3Ow8XAM+rnRmW96I4TpLp0fEK6LmaIrfuoy+3wB2a/gmczMpWvMumi4cnrBX9sWOs8nYimCK4VSrNmJKeKu8sE+qj//dtgwFl7BmiFp44qn9PXC5EIH2bdXoTLxI6jg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Ryan tried to enable the large folio for anonymous mapping [1]. Unlike large folio for page cache which doesn't trigger frequent page allocation/free, large folio for anonymous mapping is allocated/freeed more frequently. So large folio for anonymous mapping exposes some lock contention. Ryan mentioned the deferred queue lock in [1]. We also met other two lock contention: lru lock and zone lock. This series tries to mitigate the deferred queue lock and reduce lru lock in some level. The patch1 tries to reduce deferred queue lock by not acquiring queue lock when check whether the folio is in deferred list or not. Test page fault1 of will-it-scale showed 60% deferred queue lock contention reduction. The patch2 tries to reduce lru lock by allowing batched add large folio to lru list. Test page fault1 of will-it-scale showed 20% lru lock contention reduction. The zone lock contention happens on large folio free path and related with commit f26b3fa04611 "mm/page_alloc: limit number of high-order pages on PCP during bulk free" and will not be address by this series. [1] https://lore.kernel.org/linux-mm/20230414130303.2345383-1-ryan.roberts@arm.com/ Changelog from v1: For patch2: - Add Reported-by from Huang Ying which was missed by my mistake. - Fix kernel panic issue. The folio_batch_add() can have folio which doesn't reference folio directly: - For mlock usage, add new interface with extra parameter nr_pages. And callee pass nr_pages by direct reference folio. - For swap, shawdow and dax entries as parameter folio, treat the nr_pages as 1. With the fix, the stress testing can run 12 hours without any issue while hit kernel panic in around 3 minutes. - Update the lock contention info in commit message. - Change field name from pages_nr to nr_pages as Ying's suggestion. For this version, still use PAGEVEC_SIZE as max nr_pages in fbatch. We can revise it after we make decision about the page order for anonymous large folio. Yin Fengwei (2): THP: avoid lock when check whether THP is in deferred list lru: allow large batched add large folio to lru list include/linux/pagevec.h | 46 ++++++++++++++++++++++++++++++++++++++--- mm/huge_memory.c | 19 ++++++++++++++--- mm/mlock.c | 7 +++---- mm/swap.c | 3 +-- 4 files changed, 63 insertions(+), 12 deletions(-)