From patchwork Fri May 31 10:48:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13681509 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3F5EC25B75 for ; Fri, 31 May 2024 10:48:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 473BA6B007B; Fri, 31 May 2024 06:48:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3F9C46B0088; Fri, 31 May 2024 06:48:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 29A5D6B0082; Fri, 31 May 2024 06:48:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 0A01C6B0088 for ; Fri, 31 May 2024 06:48:48 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id AF6F31C05B6 for ; Fri, 31 May 2024 10:48:47 +0000 (UTC) X-FDA: 82178367894.15.C31BBEF Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf04.hostedemail.com (Postfix) with ESMTP id E6C4340003 for ; Fri, 31 May 2024 10:48:45 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=klJ+nf4M; spf=pass (imf04.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717152526; a=rsa-sha256; cv=none; b=q+OOaGzt3HTHTNcarA1R/FfNvKMHHkrOt8KI/osYwb5elPxNKTlr+VXWbkATf4kN289dXL Heb1j6RrTgVJQa1Jrjk34Klj95cyJemHixs0b77W5RH3UjU3apXqRH/6GUdAvizq5BA2xC aLKS5j4gi3eK67jtrzK1SDrS4iCmgUI= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=klJ+nf4M; spf=pass (imf04.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717152526; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=gmIlqVaF+gTSDu6ZMeuIa1Q/iwiJtQnVlmLiKz7OyH0=; b=SRlQ1sx5HV1Fnc9lxEkjP/mJwqnn0IffBZDic0TmwuoTOknhJYGB+ybpiClB//B8SSUBA7 h5av53ZYF+iyEDJPoNeCsUeIAzyWiaEAYKYIusJkykKWYawyxvjy4B+FqEeoeB40mta7YR PcseGWeYxbD1lJWbW/Ll4NgZs58md24= Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-1f44b42e9a6so15017975ad.0 for ; Fri, 31 May 2024 03:48:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717152525; x=1717757325; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=gmIlqVaF+gTSDu6ZMeuIa1Q/iwiJtQnVlmLiKz7OyH0=; b=klJ+nf4MWBekQBS5xMPE2PpbHLhvqJsVqUltgC82IyohHGARl0GsX0g2W3qPUv8p3o YIdpmfh39EEZdlF+wr+SHOc+qfp8SO78FJkCu0To00KCJXGf3NRHywGSZA2X3nd2oO0b 3JbKOm/RL/+Fw67EwLbHv0qgnbxw8yBA02e9TBAXouLwaeII55JWPxurYiH+zkJZ5+hE V6pkmGJ8AGpBgFe0dZhhlM9HJdJKK/or/3okC2kzOvFH7RYmwEvyvPsbVuSi6bctrbA1 56d+YA4+8rN3DGueTBihXLNYk+f+/4ZePb8cct5HjAEnGs0DOaQTYYqqZSuTUMw7KcCP qNZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717152525; x=1717757325; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=gmIlqVaF+gTSDu6ZMeuIa1Q/iwiJtQnVlmLiKz7OyH0=; b=h59IcZrGf1vqH890876xERFmLPekqaN0BevcCRi14LKWYk7nLFoExLRjnMRtSFLn49 rN2aUYqGCuLI4XJudSWLrfeWTxXTpHWHigZCawQNLhvrjFpTErXFXjO2DwKzs9LFrHBJ BTsjctFlt12CLPWTUehzvIxg9c0048qWFB0ffNFv3uhoH1mw6sOo7c6AazGMmFM1DtQL nTYz4U53fhxdzpiVAcEzHFrNKFsji6FoPLVqSAVLQSuAHr0yF8heSifl8kbNFEStIMaM eKDS5wt8pVqlH4GsmcNggzvmpdiLXOEUgYKi6JleZNO8DfinOtRLkZXgwny5Ykjo0WRb 97UA== X-Forwarded-Encrypted: i=1; AJvYcCXlWaQXSWL0RCtWEGELkJckgC6elJLtggNR3D8XwXU6cYNonRWFSyuJyew56iQSCV+C4nECFyiOvT398K8DPAXdvCE= X-Gm-Message-State: AOJu0YwPTSUPTFpOhCwnUwruFkWSSiDZRabMGpZjr+R+4MPa+87KJNVr 8HRZW76IRBY+B+c7nK0r7d+89p6b6sM56RhiIFCFpJQOSIbvwz9A X-Google-Smtp-Source: AGHT+IHzA+YBy8vACfHvhDgW1gXqu48sXgE81Qo3VpWgAtv6/Ia4Lvv8nfSpPCCbe44D/soe649FvA== X-Received: by 2002:a17:902:da85:b0:1f4:bb25:b8c8 with SMTP id d9443c01a7336-1f6370179f6mr15110175ad.27.1717152524557; Fri, 31 May 2024 03:48:44 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f632355e0asm13533215ad.72.2024.05.31.03.48.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 May 2024 03:48:44 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: david@redhat.com, chrisl@kernel.org, surenb@google.com, kasong@tencent.com, minchan@kernel.org, willy@infradead.org, ryan.roberts@arm.com, linux-kernel@vger.kernel.org, Barry Song Subject: [RFC PATCH] mm: swap: reuse exclusive folio directly instead of wp page faults Date: Fri, 31 May 2024 22:48:19 +1200 Message-Id: <20240531104819.140218-1-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: E6C4340003 X-Stat-Signature: wizar6qffqyj4nigccsb1347wjgj57iy X-HE-Tag: 1717152525-843757 X-HE-Meta: U2FsdGVkX1+ugtJkr654OHaUZtbywOFOqsySGlOrcdj0A8Wp/eAWVNvuXV6lB2nWOdy+AYuqJdXr+r49SDmH/HXk60DLvVzeFYDv9mjH2aPqzaOqPSRjtg3ZcOoGnXSQTc/5OA+ITQOKdz8EI6UT4VkYvsPdtRzhqF6GGRC+7ODL5yyfWB/pzBAPKNYJoqgGyXcXBw+6IPk/Ui3D5bipM6r3Vd2EOSFA1lofpuQH2crT/kYCkrPQ9nhkB9BF88RLIjXL+q6KfCjKYwOINnEU7n1uQb5vxuIEK7zVtZzoJvKJjxw4PbjYr3Oylw4pfOssgEqGoaYdsDYvX/wxio/ULvhhl5SwFiVJa7IuNiIyiBF7XTpNa21XFjooezSqUpYqdnzND+O11Vd3elmKaQ9rSkOpd061xOMJLnqcZ5j+W3b5DwBTKZH6d4SNULYHX7bavk6eGcEBhsO7RXt6lPGVtfBv9UC00QHUINamkLFqvxs0wg9smNreOTJCExp7sx3IVjeJnI4ECrhJX5BbvJIrdOBCLpZ40hKy1wDlcysQbj0mRF3vQn9NpXtygeOdsmqGoKgDEDnoXslKPtAooY5WM/o0k7ulTEZscaQDiHmhEP8ZKAf1XJt5/v/zXQRJaiCrZ2KiKR89aTLH/CbwvLb8QaVJS5lDTQ9fLbV7I3ESTq2qsgD+ZRIu4SMiWpgbVYh8e86HQrVByYmX/ZccUUzHNIY4dZJATA7giX2Oz0pE2e1LrM4apAnbUmkIdQtADYXPEIZr2AULmsjUdLMx1qMo0uX3Dbgy+iOGG/I4LnCLPcAeCNxCH37ljUHwQC1safp9uCjab/lEyKZlBOJoFm6kjSJgirCQ8abYxaFRMdVtC3+33VeQmTFdei4DsecaAO4jt/N6XAnuMcabme5rj14LyymI0Bso+YNU3YUjQZOvI1yccrxDs1ZwU6cA+bQ/5zOb+masU0EWNFG3jAU3tZl DxtUHpz8 xBsGgz4rJRydDHePaM76FPkDFcswzXXgSPa///1RNr0jRaaqZQx0Q0Ln2oo61SulB+BS4HQ/hN2fyn8Q0YdS5FQeEhBqDEyMk40u/N5pinzzJtB/eUR3aJPhkJndkR6Z22cOXLGuS2+ZO+M/BFeLyEtq2KSif7SLZOpV9YqVL2ckwH3pUcMDAx0XjFBq1P5j7fLATxJTUnZJe+gCGKkxVi7lQLbWE/zQrrGBO+Uqm4Wt5q2oOAMEVff9eYAoCSd5oOZMCmwoUHVhd/BMLEs8GNYN7tUsr4x9Q4haFWb809NLfOfnEp7H81tv6XBCck4D3v+trx9ICHwMELOodgP6qmFFU1jz4+Ef9LTLpBNYgGeTVMGDwn5jAk0FZupXGRK9lKcsLDgq97ojYG7K3h7ycJL86buA6ngYpoAjuBr1z2AXf47Xfs/X2Vd9SSOOFSnMf/xBxIVaTaYnqGpFHkJ3WaoejMcR0V/L+e474+oMh9k2jOSKjFZixKucJNeITeuhp/mRyU/IbR4kwfHQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song After swapping out, we perform a swap-in operation. If we first read and then write, we encounter a major fault in do_swap_page for reading, along with additional minor faults in do_wp_page for writing. However, the latter appears to be unnecessary and inefficient. Instead, we can directly reuse in do_swap_page and completely eliminate the need for do_wp_page. This patch achieves that optimization specifically for exclusive folios. The following microbenchmark demonstrates the significant reduction in minor faults. #define DATA_SIZE (2UL * 1024 * 1024) #define PAGE_SIZE (4UL * 1024) static void *read_write_data(char *addr) { char tmp; for (int i = 0; i < DATA_SIZE; i += PAGE_SIZE) { tmp = *(volatile char *)(addr + i); *(volatile char *)(addr + i) = tmp; } } int main(int argc, char **argv) { struct rusage ru; char *addr = mmap(NULL, DATA_SIZE, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0); memset(addr, 0x11, DATA_SIZE); do { long old_ru_minflt, old_ru_majflt; long new_ru_minflt, new_ru_majflt; madvise(addr, DATA_SIZE, MADV_PAGEOUT); getrusage(RUSAGE_SELF, &ru); old_ru_minflt = ru.ru_minflt; old_ru_majflt = ru.ru_majflt; read_write_data(addr); getrusage(RUSAGE_SELF, &ru); new_ru_minflt = ru.ru_minflt; new_ru_majflt = ru.ru_majflt; printf("minor faults:%ld major faults:%ld\n", new_ru_minflt - old_ru_minflt, new_ru_majflt - old_ru_majflt); } while(0); return 0; } w/o patch, / # ~/a.out minor faults:512 major faults:512 w/ patch, / # ~/a.out minor faults:0 major faults:512 Minor faults decrease to 0! Signed-off-by: Barry Song --- mm/memory.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index eef4e482c0c2..e1d2e339958e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4325,9 +4325,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) */ if (!folio_test_ksm(folio) && (exclusive || folio_ref_count(folio) == 1)) { - if (vmf->flags & FAULT_FLAG_WRITE) { - pte = maybe_mkwrite(pte_mkdirty(pte), vma); - vmf->flags &= ~FAULT_FLAG_WRITE; + if (vma->vm_flags & VM_WRITE) { + pte = pte_mkwrite(pte_mkdirty(pte), vma); + if (vmf->flags & FAULT_FLAG_WRITE) + vmf->flags &= ~FAULT_FLAG_WRITE; } rmap_flags |= RMAP_EXCLUSIVE; }