From patchwork Wed Jul 19 13:54:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13319019 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 738F6EB64DA for ; Wed, 19 Jul 2023 13:55:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BD04F28005D; Wed, 19 Jul 2023 09:55:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B805B28004C; Wed, 19 Jul 2023 09:55:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A702928005D; Wed, 19 Jul 2023 09:55:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 99E5628004C for ; Wed, 19 Jul 2023 09:55:07 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 565AAB0EB4 for ; Wed, 19 Jul 2023 13:55:07 +0000 (UTC) X-FDA: 81028507854.26.3C62649 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf05.hostedemail.com (Postfix) with ESMTP id 8B27D100008 for ; Wed, 19 Jul 2023 13:55:05 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689774905; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references; bh=ZtvQGP1mXU/+/oKvo0Ief14HguIdmvwePfK5XLB/jmI=; b=8kn5eBgJRiZ7pV0bkP8YB9Q+AIj0FjgI/EbaXfgqC5IFlhf4R35gfupo1hlFMGJOY9fvuy Eiw/pixaes6b1uuhoPamVBGiYiqG+uTx9i47a2xzIyBYAnku6nZS8zN32FXaRJe6Im7T7r iBr20BT+oVSdw5EOeDutxX4YA4rOMhE= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689774905; a=rsa-sha256; cv=none; b=lxxwrVoGAXqiIRMqhwKVPDmOSVrBnrsZ/DIk0bPF12cWNFCBm433od4UetZ/gR5DXS1W3N ceNtDATWUFc2zLK/7+Tl614SmrRjpbSvNLqBCm16ZWtVkjRY5VmEmRyX9Mf5uNBtxn/2rj VaWiSmHha9cscdawx+fSB8aFuvl0iDA= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 95FFC2F4; Wed, 19 Jul 2023 06:55:47 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B9A253F6C4; Wed, 19 Jul 2023 06:55:02 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Matthew Wilcox , Yin Fengwei , David Hildenbrand , Yu Zhao , Yang Shi , "Huang, Ying" , Zi Yan Cc: Ryan Roberts , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 0/3] Optimize large folio interaction with deferred split Date: Wed, 19 Jul 2023 14:54:47 +0100 Message-Id: <20230719135450.545227-1-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: y5fgoswbfi1mw7f81ht758ky3k8kzi3d X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 8B27D100008 X-HE-Tag: 1689774905-999378 X-HE-Meta: U2FsdGVkX1/Kt+nrz/ToLgX/mmgNF4fa+oa35ZL8/eCdMmDK+F9BoE66tubpnw66ndMIyp1zBZthT6OziwBbTTO5Ag8e81IH9gu5jTtJhcLBsbrDLwPepNaUatBFR3WnPg/XtEbp4x74goHH7ZlR9qm22x0GoldYdLVCBeUjp/NZLF7ShBni6UqljPuG8inzdubUOTfd8zLTTepMDznS5O7Kq9fYrA+cB+S4wChMgwtNHpzDmO45iK4jR2V3qQeVmiam3nrS8VqwBw+3Ap/LBs2QFx/r+zlLXCTw0Tuj9iMCM2P1bdhf0NM+BP4taHLOuvhqaRH8i8cyOtKMs1hXluJsu+ZXQ6UqCbS5dxHWnHXu+aMT+u/brbsTUH9OOLbH1DA4meDsz0F658y+BYqq6/IomsEW7A02xuzlozaAPmWM7b6m+js95CpSHY9dnOQtv5+d3j8Cfh/8BtB751y9zvhCVKYhvo5ChPvNf0A05P5yDh5/rjCmv4Q36BHw6Ji6+TAhmjm+lJLSMSlh+h8jnPM0QnqnMfkUkfasKCKL+Os/3DaFD+zeLDZP36EfN5Yke96kBdJvImEDymuYg8Q7cOr4iD1rBFarDqs05UlBWNpD7dX/+ohMOY6yw+MCYUMpmRFlfXx4o/usof0yho6x+9+OmCqfG5fLVOwqhHhDsi171j6BIuAHfqxw04/viKQrvqNajJaUjwE/VevfvvK0Llp/F4NAIWsb+GOxclH9aMHFzXngpbjt2j/6PoBBQpposm4ZRi7iHXx+BYFsNTeBGT2FKw+zHjxpZWxkbjIgMgiBuOarbLb3WvCFkZUo0Mb3v2Hx0DkCVwWiNTMtf7tczWalqoSZ0yCTYrdeMvQcaEPVRbW1rk30pZc2eNV/cByr4Ec6ON05d4L3uKWYux8Hf7ppfGdayz3RqSYG2y0G/uXe45kktQ3lw6l9Zs5FCSzjNjV6Vs7s1dZe2QkpZ3n CmuNaBt+ 7BSIDB7PFW7j8fuYNcXzyiSfdddWQzfoub3L2cMvb9HLfLPQeXS97tJUdEAO13aFhEohneUSgRzieRQqLpZBNXgMEulk+/jj7qiqp51zmeHdTupHavBxe6NOWq2cWUukeVdfOHM6m1DO9zxsjBMGtNbxqgQWA1JmHzmsSL3ivb4JzPl9bv2OpGIP7lQCNbOnUZl7u+007k10syPuXGdm81HiA2Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi All, This is v2 of a small series in support of my work to enable the use of large folios for anonymous memory (known as "FLEXIBLE_THP" or "LARGE_ANON_FOLIO") [1]. It first makes it possible to add large, non-pmd-mappable folios to the deferred split queue. Then it modifies zap_pte_range() to batch-remove spans of physically contiguous pages from the rmap, which means that in the common case, we elide the need to ever put the folio on the deferred split queue, thus reducing lock contention and improving performance. This becomes more visible once we have lots of large anonymous folios in the system, and Huang Ying has suggested solving this needs to be a prerequisit for merging the main FLEXIBLE_THP/LARGE_ANON_FOLIO work. The series applies on top of v6.5-rc2 and a branch is available at [2]. I don't have a full test run with the latest versions of all the patches on top of the latest baseline, so not posting results formally. I can get these if people feel they are neccessary though. But anecdotally, for the kernel compilation workload, this series reduces kernel time by ~4% and reduces real-time by ~0.4%, compared with [1]. Changes since v1 [3] -------------------- - patch 2: Modified doc comment for folio_remove_rmap_range() - patch 2: Hoisted _nr_pages_mapped manipulation out of page loop so its now modified once per folio_remove_rmap_range() call. - patch 2: Added check that page range is fully contained by folio in folio_remove_rmap_range() - patch 2: Fixed some nits raised by Huang, Ying for folio_remove_rmap_range() - patch 3: Support batch-zap of all anon pages, not just those in anon vmas - patch 3: Renamed various functions to make their use clear - patch 3: Various minor refactoring/cleanups - Added Reviewed-By tags - thanks! [1] https://lore.kernel.org/linux-mm/20230714160407.4142030-1-ryan.roberts@arm.com/ [2] https://gitlab.arm.com/linux-arm/linux-rr/-/tree/features/granule_perf/deferredsplit-lkml_v2 [3] https://lore.kernel.org/linux-mm/20230717143110.260162-1-ryan.roberts@arm.com/ Thanks, Ryan Ryan Roberts (3): mm: Allow deferred splitting of arbitrary large anon folios mm: Implement folio_remove_rmap_range() mm: Batch-zap large anonymous folio PTE mappings include/linux/rmap.h | 2 + mm/memory.c | 120 +++++++++++++++++++++++++++++++++++++++++++ mm/rmap.c | 76 ++++++++++++++++++++++++++- 3 files changed, 196 insertions(+), 2 deletions(-) --- 2.25.1