From patchwork Mon Mar 6 09:22:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yin Fengwei X-Patchwork-Id: 13160756 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26BC7C61DA4 for ; Mon, 6 Mar 2023 09:21:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3C8B86B0074; Mon, 6 Mar 2023 04:21:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 378B36B0075; Mon, 6 Mar 2023 04:21:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 23F9C6B0078; Mon, 6 Mar 2023 04:21:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 0F23A6B0074 for ; Mon, 6 Mar 2023 04:21:46 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id DCFD5A0B7E for ; Mon, 6 Mar 2023 09:21:45 +0000 (UTC) X-FDA: 80537930970.07.E8C1FCB Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by imf03.hostedemail.com (Postfix) with ESMTP id 9B50F20014 for ; Mon, 6 Mar 2023 09:21:43 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=fhnssq2e; spf=pass (imf03.hostedemail.com: domain of fengwei.yin@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678094503; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=MaFyiBL1iW5guQiNoSKb8dFjNAoye+fgesKPyu+a+J4=; b=qG1J80VxUwpwI8s4Kk0D2gAnyIRs3F9AbFijbZdt0nZHd2zrcUA1kM7q4Wi/DRUz4Q7WKP cu7nXGS5d711YrQKqesCX4Q7He8XJmysW9bqe4LWDgks5AWACA6kpsZn+CRK8qu5vj/Pmz Fhr8WZyqWah8oJ/fo30KMJedfBzUTJY= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=fhnssq2e; spf=pass (imf03.hostedemail.com: domain of fengwei.yin@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678094503; a=rsa-sha256; cv=none; b=NOvnnMysvofK7QCdRyQqN1FPlmF7MAV5xm2zhpuQTgGhmRvB410ix4AXItdqggBhIjzH80 oPx5ymkxTmx0S+OC1fsHW2H5HgKwUoaDSEizgKK4FqPcX9fa6Ey9wbhMoB4uwg1rKtjWkh g8Q/nY3+pvkbgm5JqnqA+DLi9zcRZnI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678094503; x=1709630503; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=e5gEVutZ9wsXjR6fAcqDfS329gf3sKhe6B/nYAtlFxU=; b=fhnssq2eCI/zDxwd8YzvyrZSKIQsXEe7Vvj3BPJrslNHgr3OTVwuhGL0 VUKsTUcEyOnlY7PCkbBsLIw/Tz21NshxMaOLQuEQPHHEpcohTXS7yYdE3 QLR/rhmyq7FW1AmQTN1xrLm5eoCag8FcSElxmwvOpd+F5pHS9jJ3ktRei NNvs3ZU7qlPTAJQztiyLavieRtLpIOPu4F5kfe9iLvWIzntduwWu+YFnY ITAAn4PacG+qUpkO+KOyVlAmu8ihbwVeMJdZIWLTUQiPaR+HQNKL4/Twc C9I88GJtLCd4HHbH8Ih7Dqr4xdL3ctiMYWTfXerUYJE33SxS/ulm4VDWn g==; X-IronPort-AV: E=McAfee;i="6500,9779,10640"; a="334225561" X-IronPort-AV: E=Sophos;i="5.98,236,1673942400"; d="scan'208";a="334225561" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Mar 2023 01:21:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10640"; a="676099376" X-IronPort-AV: E=Sophos;i="5.98,236,1673942400"; d="scan'208";a="676099376" Received: from fyin-dev.sh.intel.com ([10.239.159.32]) by orsmga002.jf.intel.com with ESMTP; 06 Mar 2023 01:21:39 -0800 From: Yin Fengwei To: linux-mm@kvack.org, akpm@linux-foundation.org, willy@infradead.org, mike.kravetz@oracle.com, sidhartha.kumar@oracle.com, naoya.horiguchi@nec.com, jane.chu@oracle.com, david@redhat.com Cc: fengwei.yin@intel.com Subject: [PATCH v3 0/5] batched remove rmap in try_to_unmap_one() Date: Mon, 6 Mar 2023 17:22:54 +0800 Message-Id: <20230306092259.3507807-1-fengwei.yin@intel.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-Rspamd-Queue-Id: 9B50F20014 X-Stat-Signature: cr5jwhh9kgb18785gsqd3m7wmqi3pocx X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1678094503-961413 X-HE-Meta: U2FsdGVkX1/qirYQSAfEqcH8CumY0LMnTD8TL5Wz/OSr0ZT140QO0aH9Jh5v6NHl44lVD7TBzHF6w2Yjbm/qOcOrOngNsL50gu5wu6YM+WMfRP5Z5LgSOMMkvDhSO+MW8IZ3pQLWEixny8JgnQszuOhJPh/Lw5KNsNq/1uHD3YreUFxy6Bn5foiYWXmAz58dYcuZUmzc7gsH+gc22xq8NUxV/W4trSBJqmkJYvqgwWPzhgvQ+XgB2FeCSbU/0B5MrAbVHxz8wezrQtEwWYko4stDIGNAVYvupg7g9RgnPFQA9BfR3Mim11lUUQMHkxaw5uyArNPSLCm7kSuHlbzJfZ6b6vP2LckeNZ48NDTPs+Yg4tEUt8clhpUtpXRoUlp+G2+vpUkLXihybPB6YtsVXbA2fGHrtDwosmIwtmhAxzLoQ4anv6l/kPWdFmg0Sdr+mIQWqleA44MFsuF6XuCvW1iTWTw5atiIVheDtySp9332QwN3poRCTf1A0xkKvKoZiQJbWGLKq0/p1aPW35bYrbhiNtKPw8AtnYKnUu16A12UBzR6wpQaHxsp59WsshsC9IKhVimFbhNa6e+y7vEs7WxEYPJYsAtswshOSKThDd6vkz0jsvEj9FD5T+bO+joRBrCN/Q/a/O7yVl7kItLjHk6EXerABKXeQWUJ2zH9otZBU1NMPajcUPY81tu5HKFyajz4oKKx2ZBL0ioRuuokrTtu0CEbEalaseQdJR1gR3GhMnDYVbu5T6wdrudbro2NgumK5uABveisIscVz+//Kdh10EoMXFxp/L62KxymC9A4l94G46yGvxn0T4DmhIS2jlzEws6Komn3o/z8w0mSZnhOGUrM4ulerwLlIdOTIM2W7JSedeIvHMckPiTFxnuc/ACWKbiTH9uOkUYi/ijrDEztvu1kOMw7bIP7nPdaesJIXEIotSZYBMEYaCnPzqGTE9APid5CvIsiA3/3sNk c7Pi8ieP 8LzoP4BViSQ/yK377cs1ReonEXOEipQq+HHQ78P6Fy3dGF4KXFLz27nbzpyDcd9nfSDs85NF0hQq2FR8A28p8LCRX1psvdV4FlTnLjOJ6ctEuQqgBCmtvM49EjgvVJ00o7xBPWXEQFztlkLLERhR99+ku2Fw2MC31gsNaRGHWhhCC9IqijYtTmUJAJCyPLJDYimKo X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This series is trying to bring the batched rmap removing to try_to_unmap_one(). It's expected that the batched rmap removing bring performance gain than remove rmap per page. This series reconstruct the try_to_unmap_one() from: loop: clear and update PTE unmap one page goto loop to: loop: clear and update PTE goto loop unmap the range of folio in one call It is one step to always map/unmap the entire folio in one call. Which can simplify the folio mapcount handling by avoid dealing with each page map/unmap. The changes are organized as: Patch1/2 move the hugetlb and normal page unmap to dedicated functions to make try_to_unmap_one() logic clearer and easy to add batched rmap removing. To make code review easier, no function change. Patch3 cleanup the try_to_unmap_one_page(). Try to removed some duplicated function calls. Patch4 adds folio_remove_rmap_range() which batched remove rmap. Patch5 make try_to_unmap_one() to batched remove rmap. Testing done with the V3 patchset in a qemu guest with 4G mem: - kernel mm selftest to trigger vmscan() and final hit try_to_unmap_one(). - Inject hwpoison to hugetlb page to trigger try_to_unmap_one() call against hugetlb. - 8 hours stress testing: Firefox + kernel mm selftest + kernel build. This series is based on next-20230303. Changes from v2: - General - Rebase the patch to next-20230303 - Update cover letter about the preparation to unmap the entire folio in one call - No code change comparing to V2. But fix the patch applying conflict because of wrong patch order in V2. Changes from v1: - General - Rebase the patch to next-20230228 - Patch1 - Removed the if (PageHWPoison(page) && !(flags & TTU_HWPOISON) as suggestion from Mike Kravetz and HORIGUCHI NAOYA - Removed the mlock_drain_local() as suggestion from Mike Kravetz _ Removed the comments about the mm counter change as suggestion from Mike Kravetz Yin Fengwei (5): rmap: move hugetlb try_to_unmap to dedicated function rmap: move page unmap operation to dedicated function rmap: cleanup exit path of try_to_unmap_one_page() rmap:addd folio_remove_rmap_range() try_to_unmap_one: batched remove rmap, update folio refcount include/linux/rmap.h | 5 + mm/page_vma_mapped.c | 30 +++ mm/rmap.c | 623 +++++++++++++++++++++++++------------------ 3 files changed, 398 insertions(+), 260 deletions(-)