From patchwork Wed Feb 19 11:25:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13981959 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D555FC021AA for ; Wed, 19 Feb 2025 11:25:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5E78A280229; Wed, 19 Feb 2025 06:25:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 595CF280226; Wed, 19 Feb 2025 06:25:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 40FF1280229; Wed, 19 Feb 2025 06:25:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 244C9280226 for ; Wed, 19 Feb 2025 06:25:37 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 9571B51096 for ; Wed, 19 Feb 2025 11:25:36 +0000 (UTC) X-FDA: 83136463872.16.44EB5F8 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf20.hostedemail.com (Postfix) with ESMTP id CB3CB1C001B for ; Wed, 19 Feb 2025 11:25:34 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=AvTv6Rhu; spf=pass (imf20.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739964334; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=Eb3qjAdIKb0cGpN4NTEEcHe/MgWIebClO9nvYHKBt2c=; b=u1f2L1U3uevedXM53IgLNNb0GJzkC0kd3JQQ/RGaIO30XC/QJj1wJlBx2K1DLalNNleTUI dSqCHB5EhBl8OhAxyfgvmQK7i5WcC2os6GiSlAsD3XjbppqWXttelet1RRMwhtsWAE2gy1 T8ucqm+lXuAtNfuHEO2iyxJd90HDACg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739964334; a=rsa-sha256; cv=none; b=4TJJVcv66urm3wi0opMtovbjZc59mOV1AQ+Ww2tr997NtnOeHU4naPttVDdYyRUePregI6 OpwsQIpsAdev/exAwNnrgrhBj3LtEgZCrdGqXPBqevMbNoARsySJVNnARTSUWdXFWd2eIX TTFUZ2jGp894e34D20mGsTS1hqtjKN0= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=AvTv6Rhu; spf=pass (imf20.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-220c92c857aso12463835ad.0 for ; Wed, 19 Feb 2025 03:25:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739964333; x=1740569133; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Eb3qjAdIKb0cGpN4NTEEcHe/MgWIebClO9nvYHKBt2c=; b=AvTv6RhuVIxZA874YLbOUSKTKBdV8lSlCoqkmHu1iucYJOEmtKKoao2+gJFL8ALIXv krSPZGhGW+pKKHxSZk7SXqLfTmc9wWe6PhsRoYnR1co/RCUw9QT9I6uit+xPICgX7jhm SQFKJfz+xDBXIv3y9aOfj8G/NsnLoSYUU3e0ozY239EOjgIb5Kt3H0K3aJkM/KK5GkQu 8E/zqmuXGVWwRx+vgHTJ5lnT20boRFD7mMURnZp29IpacRTjeZo/ytVl3p4y+MsHqhv3 8746RIlIDRQ6a95QBRsAaoDCS68+ZNAlDchpmRi8KKhqkD6R/1GJBvcIEJ3/b65q8xd+ y3KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739964333; x=1740569133; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Eb3qjAdIKb0cGpN4NTEEcHe/MgWIebClO9nvYHKBt2c=; b=bZnhEbzH9DEygMY5EPU6ng5ODjn2xlSd7rvrMuTSTKuw+mtYPuCwdl5qSk4dUwjCbr xKo077SAuWovvmzHmXZfCVYaSAkQrYbXka5MrsCg4TkdwglotQwtHErPQM+EXwlEUTeo 9bNmLBQCQVe/8dMShCmol2r0Gq2MYM5wbCpTTkfEbUrm7nGu3KoXAudBebYOrTTQI8lq kRnbxZ7eGH033FuFhqdpIhej0U1i4ZR9AyGV847TPy6bvZaGiuaV5/2k7EaYKs9HV441 2MbM2beFnqN72gr3A45YC16sJxkx2ByfbXW0jP94ELkSrTVjsiAaFIgiBUWn4st2X7KY FNPA== X-Gm-Message-State: AOJu0YyFf/aKoDL/WWWywMJ2pOB+8WxQMAr9jSlNBqXE+t9WpfqOAW7J 0p/1PWr5GPrrEgQKrNMezBGun97cPMkrQos5fAgDucy/rI/C7VlAkax6zOCd X-Gm-Gg: ASbGncuwfCIGdNoXJUl8K3ZuB1DfBOFaOXR86uLSPQeArknQUisAcAUo2ZLlqIWviCS z9Okv5geoRfy8PyIro3ds2vBxrzlbWpw55+NZRwqzDkTnjKCAF5FwKZhcAgzXIQiVitU6kpXzec FFpXIbd2RAcZWPuNgVzcsy1sB8T9siW9m0ZnxhK2xgSlkB/XOi/GOeHwPsXHjj9NM1jzvahpqWz 5Q6JNuccpIuPtOjk5CagRdbUgAtGxfbIC1wB91eR3v82RffYvO7jRIEhc2Eyc7NC09ZK3DzjQgh yWCIVMpLFjWlDcWpKlg4Gt6wD+kNc+zH X-Google-Smtp-Source: AGHT+IF8kNT2MetyoLuxF9ILx52K0ftVCLidA67ewR5lWC8LU67l4Cc0GQJorlauNh1IkgSBpGIeuw== X-Received: by 2002:a17:902:f612:b0:216:5db1:5dc1 with SMTP id d9443c01a7336-2216ee83a79mr58769555ad.1.1739964333330; Wed, 19 Feb 2025 03:25:33 -0800 (PST) Received: from Barrys-MBP.hub ([118.92.30.135]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d53491ecsm102728565ad.9.2025.02.19.03.25.24 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Feb 2025 03:25:32 -0800 (PST) From: Barry Song <21cnbao@gmail.com> To: linux-mm@kvack.org, akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, zhengtangquan@oppo.com, Barry Song , Andrea Arcangeli , Suren Baghdasaryan , Al Viro , Axel Rasmussen , Brian Geffon , Christian Brauner , David Hildenbrand , Hugh Dickins , Jann Horn , Kalesh Singh , "Liam R . Howlett" , Lokesh Gidra , Matthew Wilcox , Michal Hocko , Mike Rapoport , Nicolas Geoffray , Peter Xu , Ryan Roberts , Shuah Khan , ZhangPeng Subject: [PATCH RFC] mm: Fix kernel BUG when userfaultfd_move encounters swapcache Date: Thu, 20 Feb 2025 00:25:19 +1300 Message-Id: <20250219112519.92853-1-21cnbao@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: CB3CB1C001B X-Rspamd-Server: rspam07 X-Stat-Signature: upr6eat41pzmaeydn4ot8zgdwbzh1zee X-HE-Tag: 1739964334-313616 X-HE-Meta: U2FsdGVkX1+dF1gjhlrDRVGztG5WnmwRXj+X35YPHVBCLLooiYUyV6P8+hJoTPTbnIXow7cyBORPtU6EftKNS513R8UqWmqQ8TlfHv74LFCyo1s7F7cGl1WQ5WlZkJ0ULaRnyVLIW6XSMH8aqpqQt27WZzFLF3qrMlrPh0yt5SkjvxVNYQ0szHlvgwtgB7MtnVTgajy+EbMcUKWVLqtlri5UQFoN76a/LCdMWoPpULiYooJpUfDgYE3V7Jwo/LLf8ShEChtZ3V/ZmS2PuI/nrQ7YrUarHtiMZ16sJvWd4G+FOfpSQdEmwcB128xy9BAX8chz/wLXyG56Qb3VY5xIEwnwBcTxKPyLTva3u7fxL18dxLBNLv4m2Ne3xpcIF5JaLzRTp12JOwZBTSp9nd82cEwDbY+Yf5HCfULU3mT2fwes7vrABWp4466kmYpfc66i9oMpvhNxoZn9sZPE850QHJS4YHxjikVaYNR6oiIjLH4774a+92icaIAnUTQG647nJHE2d8xwDEH5GGuMFNimvmfkTLX99V8VoSVWEcGLhhMIdbnkjn/YO2xB9DbXby7nr+9An3dAdNvJdbVxBgcpTgFZiuUnQg0Qsg8l7sGuwGvGu6CsMMIt8s8TB5OKpkI9wkRN5G9Djbe/tmk0Yh3hvWNQ8fh2LJ771LyxTXWgz9FQ8eiv94IjsdeIZnuQkLiedhLU/BmDFow66PXsjeX38rD1EZPA0PDfksMks1ViT5ipeH/UsclrTZcDaekeeBs3gHZRvYToDgh1GGNCSljFHAf6g6W4LHgwehp38m8Cysil1jnH9rt8Kgj1dNVKrT2rGYtd7LKCioWeyLf0eOCF/FoN1hyvm2XMkxV+0UP+GQnpZlcaJaa7ezzVBP/xsyRlOUK+L7EnwcZfeV0MJQ/oBW/C6DO6aij9yXahGXlz5VxmJl7QCg9Ls9csxafO1OKjDsbWYTD5WozJjcb6AP+ lHTq9AMj DH4LtU5BcVwSuWDtfQYl/gkPhoml+3RJO2Yq/JjYQ9Vp3im96KK9LH9rP5nGCNopwK6aPcL7PKlu5Z2Zzt62HyzeXv0FtcdCJNqq9z51wN6bKJ6UWu2XjqbYKf3UvMS7TU6FkmA5MGZFHkTpxmF0OV4JneEJiFYM0pyrUltFejAEhEUKi0RbEXv3zsbhxcWPq7fExpgExI0zwrpReOYjwf98rHGZXPtK7EmVdgeKw36UFeK3LBbFJPhAPUPFz20pdtWRRG6xcgxNcLhzr6MD0jg5JDE8vaTZ+CMbW42sds9oXz5Z+o1j8aQ8FAN+5njox/kIVHl2FS/4KTiah5U9H0ANcjrRsHP7i4Ikr3mwThHZBU13XXkEIlG5HGQiTPnxgSHOGIeFmBAYjPvkyma/PcinHdBmiAoEKGKyMFT9HtMhnh9O9/nDF7lESCNVRd6nOt2mgdN4ff0G6fqNG2np8r7mVy+i67al14v5nVPLD0/ffUbsV6IYuc2bZ6sWrRHkDAiugGf4Wg2ZxrLstovm6QLpU6a7ut9PG4k1uLypGr1D2R4jWBaP6E9LXl+9pYr18LFYg7Eg4KbsH/SlKBsB2xYAI/ZMLY8aa+47zrAiQGsRsXZXiJwtPHUam0XEev4+wEoNvys8Oa1dfv3SRVzaZPye+aL/21+g7YVZToSxT9Ub4eDqLshbavqcqjVoXCTZLMJrUmlaZjWbJXkf/7By10JA3jJlXP8vj8NLeKMMalY2YZb9Sl1edXU8ooDjuya1FclqWvqre+McEvgEBoR5SUE/AlQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song userfaultfd_move() checks whether the PTE entry is present or a swap entry. - If the PTE entry is present, move_present_pte() handles folio migration by setting: src_folio->index = linear_page_index(dst_vma, dst_addr); - If the PTE entry is a swap entry, move_swap_pte() simply copies the PTE to the new dst_addr. This approach is incorrect because even if the PTE is a swap entry, it can still reference a folio that remains in the swap cache. If do_swap_page() is triggered, it may locate the folio in the swap cache. However, during add_rmap operations, a kernel panic can occur due to: page_pgoff(folio, page) != linear_page_index(vma, address) $./a.out > /dev/null [ 13.336953] page: refcount:6 mapcount:1 mapping:00000000f43db19c index:0xffffaf150 pfn:0x4667c [ 13.337520] head: order:2 mapcount:1 entire_mapcount:0 nr_pages_mapped:1 pincount:0 [ 13.337716] memcg:ffff00000405f000 [ 13.337849] anon flags: 0x3fffc0000020459(locked|uptodate|dirty|owner_priv_1|head|swapbacked|node=0|zone=0|lastcpupid=0xffff) [ 13.338630] raw: 03fffc0000020459 ffff80008507b538 ffff80008507b538 ffff000006260361 [ 13.338831] raw: 0000000ffffaf150 0000000000004000 0000000600000000 ffff00000405f000 [ 13.339031] head: 03fffc0000020459 ffff80008507b538 ffff80008507b538 ffff000006260361 [ 13.339204] head: 0000000ffffaf150 0000000000004000 0000000600000000 ffff00000405f000 [ 13.339375] head: 03fffc0000000202 fffffdffc0199f01 ffffffff00000000 0000000000000001 [ 13.339546] head: 0000000000000004 0000000000000000 00000000ffffffff 0000000000000000 [ 13.339736] page dumped because: VM_BUG_ON_PAGE(page_pgoff(folio, page) != linear_page_index(vma, address)) [ 13.340190] ------------[ cut here ]------------ [ 13.340316] kernel BUG at mm/rmap.c:1380! [ 13.340683] Internal error: Oops - BUG: 00000000f2000800 [#1] PREEMPT SMP [ 13.340969] Modules linked in: [ 13.341257] CPU: 1 UID: 0 PID: 107 Comm: a.out Not tainted 6.14.0-rc3-gcf42737e247a-dirty #299 [ 13.341470] Hardware name: linux,dummy-virt (DT) [ 13.341671] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 13.341815] pc : __page_check_anon_rmap+0xa0/0xb0 [ 13.341920] lr : __page_check_anon_rmap+0xa0/0xb0 [ 13.342018] sp : ffff80008752bb20 [ 13.342093] x29: ffff80008752bb20 x28: fffffdffc0199f00 x27: 0000000000000001 [ 13.342404] x26: 0000000000000000 x25: 0000000000000001 x24: 0000000000000001 [ 13.342575] x23: 0000ffffaf0d0000 x22: 0000ffffaf0d0000 x21: fffffdffc0199f00 [ 13.342731] x20: fffffdffc0199f00 x19: ffff000006210700 x18: 00000000ffffffff [ 13.342881] x17: 6c203d2120296567 x16: 6170202c6f696c6f x15: 662866666f67705f [ 13.343033] x14: 6567617028454741 x13: 2929737365726464 x12: ffff800083728ab0 [ 13.343183] x11: ffff800082996bf8 x10: 0000000000000fd7 x9 : ffff80008011bc40 [ 13.343351] x8 : 0000000000017fe8 x7 : 00000000fffff000 x6 : ffff8000829eebf8 [ 13.343498] x5 : c0000000fffff000 x4 : 0000000000000000 x3 : 0000000000000000 [ 13.343645] x2 : 0000000000000000 x1 : ffff0000062db980 x0 : 000000000000005f [ 13.343876] Call trace: [ 13.344045] __page_check_anon_rmap+0xa0/0xb0 (P) [ 13.344234] folio_add_anon_rmap_ptes+0x22c/0x320 [ 13.344333] do_swap_page+0x1060/0x1400 [ 13.344417] __handle_mm_fault+0x61c/0xbc8 [ 13.344504] handle_mm_fault+0xd8/0x2e8 [ 13.344586] do_page_fault+0x20c/0x770 [ 13.344673] do_translation_fault+0xb4/0xf0 [ 13.344759] do_mem_abort+0x48/0xa0 [ 13.344842] el0_da+0x58/0x130 [ 13.344914] el0t_64_sync_handler+0xc4/0x138 [ 13.345002] el0t_64_sync+0x1ac/0x1b0 [ 13.345208] Code: aa1503e0 f000f801 910f6021 97ff5779 (d4210000) [ 13.345504] ---[ end trace 0000000000000000 ]--- [ 13.345715] note: a.out[107] exited with irqs disabled [ 13.345954] note: a.out[107] exited with preempt_count 2 Fully fixing it would be quite complex, requiring similar handling of folios as done in move_present_pte. For now, a quick solution is to return -EBUSY. I'd like to see others' opinions on whether a full fix is worth pursuing. For anyone interested in reproducing it, the a.out test program is as below, #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include #define PAGE_SIZE 4096 #define REGION_SIZE (512 * 1024) #ifndef UFFDIO_MOVE struct uffdio_move { __u64 dst; __u64 src; __u64 len; #define UFFDIO_MOVE_MODE_DONTWAKE ((__u64)1<<0) #define UFFDIO_MOVE_MODE_ALLOW_SRC_HOLES ((__u64)1<<1) __u64 mode; __s64 move; }; #define _UFFDIO_MOVE (0x05) #define UFFDIO_MOVE _IOWR(UFFDIO, _UFFDIO_MOVE, struct uffdio_move) #endif void *src, *dst; int uffd; void *madvise_thread(void *arg) { if (madvise(src, REGION_SIZE, MADV_PAGEOUT) == -1) { perror("madvise MADV_PAGEOUT"); } return NULL; } void *fault_handler_thread(void *arg) { struct uffd_msg msg; struct uffdio_move move; struct pollfd pollfd = { .fd = uffd, .events = POLLIN }; pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL); pthread_setcanceltype(PTHREAD_CANCEL_DEFERRED, NULL); while (1) { if (poll(&pollfd, 1, -1) == -1) { perror("poll"); exit(EXIT_FAILURE); } if (read(uffd, &msg, sizeof(msg)) <= 0) { perror("read"); exit(EXIT_FAILURE); } if (msg.event != UFFD_EVENT_PAGEFAULT) { fprintf(stderr, "Unexpected event\n"); exit(EXIT_FAILURE); } move.src = (unsigned long)src + (msg.arg.pagefault.address - (unsigned long)dst); move.dst = msg.arg.pagefault.address & ~(PAGE_SIZE - 1); move.len = PAGE_SIZE; move.mode = 0; if (ioctl(uffd, UFFDIO_MOVE, &move) == -1) { perror("UFFDIO_MOVE"); exit(EXIT_FAILURE); } } return NULL; } int main() { again: pthread_t thr, madv_thr; struct uffdio_api uffdio_api = { .api = UFFD_API, .features = 0 }; struct uffdio_register uffdio_register; src = mmap(NULL, REGION_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (src == MAP_FAILED) { perror("mmap src"); exit(EXIT_FAILURE); } memset(src, 1, REGION_SIZE); dst = mmap(NULL, REGION_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (dst == MAP_FAILED) { perror("mmap dst"); exit(EXIT_FAILURE); } uffd = syscall(SYS_userfaultfd, O_CLOEXEC | O_NONBLOCK); if (uffd == -1) { perror("userfaultfd"); exit(EXIT_FAILURE); } if (ioctl(uffd, UFFDIO_API, &uffdio_api) == -1) { perror("UFFDIO_API"); exit(EXIT_FAILURE); } uffdio_register.range.start = (unsigned long)dst; uffdio_register.range.len = REGION_SIZE; uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING; if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) == -1) { perror("UFFDIO_REGISTER"); exit(EXIT_FAILURE); } if (pthread_create(&madv_thr, NULL, madvise_thread, NULL) != 0) { perror("pthread_create madvise_thread"); exit(EXIT_FAILURE); } if (pthread_create(&thr, NULL, fault_handler_thread, NULL) != 0) { perror("pthread_create fault_handler_thread"); exit(EXIT_FAILURE); } for (size_t i = 0; i < REGION_SIZE; i += PAGE_SIZE) { char val = ((char *)dst)[i]; printf("Accessing dst at offset %zu, value: %d\n", i, val); } pthread_join(madv_thr, NULL); pthread_cancel(thr); pthread_join(thr, NULL); munmap(src, REGION_SIZE); munmap(dst, REGION_SIZE); close(uffd); goto again; return 0; } As long as you enable mTHP (which likely increases the residency time of swapcache), you can reproduce the issue within a few seconds. But I guess the same race condition also exists with small folios. Fixes: adef440691bab ("userfaultfd: UFFDIO_MOVE uABI") Cc: Andrea Arcangeli Cc: Suren Baghdasaryan Cc: Al Viro Cc: Axel Rasmussen Cc: Brian Geffon Cc: Christian Brauner Cc: David Hildenbrand Cc: Hugh Dickins Cc: Jann Horn Cc: Kalesh Singh Cc: Liam R. Howlett Cc: Lokesh Gidra Cc: Matthew Wilcox (Oracle) Cc: Michal Hocko Cc: Mike Rapoport (IBM) Cc: Nicolas Geoffray Cc: Peter Xu Cc: Ryan Roberts Cc: Shuah Khan Cc: ZhangPeng Signed-off-by: Barry Song Signed-off-by: Barry Song --- mm/userfaultfd.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 867898c4e30b..34cf1c8c725d 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -18,6 +18,7 @@ #include #include #include "internal.h" +#include "swap.h" static __always_inline bool validate_dst_vma(struct vm_area_struct *dst_vma, unsigned long dst_end) @@ -1079,9 +1080,19 @@ static int move_swap_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t dst_pmdval, spinlock_t *dst_ptl, spinlock_t *src_ptl) { + struct folio *folio; + swp_entry_t entry; + if (!pte_swp_exclusive(orig_src_pte)) return -EBUSY; + entry = pte_to_swp_entry(orig_src_pte); + folio = filemap_get_folio(swap_address_space(entry), swap_cache_index(entry)); + if (!IS_ERR(folio)) { + folio_put(folio); + return -EBUSY; + } + double_pt_lock(dst_ptl, src_ptl); if (!is_pte_pages_stable(dst_pte, src_pte, orig_dst_pte, orig_src_pte,