From patchwork Tue Nov 29 15:47:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jann Horn X-Patchwork-Id: 13058732 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CC52C4167B for ; Tue, 29 Nov 2022 15:47:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E3FF56B0074; Tue, 29 Nov 2022 10:47:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DF5C16B0075; Tue, 29 Nov 2022 10:47:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB3FB6B0078; Tue, 29 Nov 2022 10:47:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id BDF846B0074 for ; Tue, 29 Nov 2022 10:47:49 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 9782C1405AC for ; Tue, 29 Nov 2022 15:47:49 +0000 (UTC) X-FDA: 80186910258.25.DBBB901 Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) by imf18.hostedemail.com (Postfix) with ESMTP id 1ABB41C000D for ; Tue, 29 Nov 2022 15:47:48 +0000 (UTC) Received: by mail-wr1-f51.google.com with SMTP id bs21so22760954wrb.4 for ; Tue, 29 Nov 2022 07:47:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YSdw+42zFeQjk2DGkc/5JCpwWvHgOHoXpertIATsPMg=; b=kyY6Gy0JKC+0AUgiybPvuNjn+SU6DV8iudSjb8b55zB8kB4CqhVidPvZfk6mww3B8n 6tPPi7aTD28MO7gKzrHO+ZFZTkMmX5YCdYvsqIG/OXyXs9ykyLPP5wRNXHSRX0t6RVsh IZX4qH8FwqDWynlcIRmrySVkQJImf/gX52fCnR1vlGWe2x9tFDFP0yt7ZfZIHsBm6j36 kYSmD90/AkP2Qg+2MXUuwV1NFk3p2dA+7dZpdiWsjPuLXQCQpLBU6xTlqxUMgxjOTIMH /nQkV9fl8zI6rpYGdiluQ5g7ciMaMfjc9nt2qlBPLrhiHOj/rqFidysvQv8pXgIoMM3b SNCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YSdw+42zFeQjk2DGkc/5JCpwWvHgOHoXpertIATsPMg=; b=lT3ftdc4aeqqtKdJvLNC2MXDB6A/ey/YXPaZADKYwTMknPnhtsULkq08Ntm7zx5G8V DB57ge6goUySXkSGknANgQoVojS3M44yFePBQqkkSLeGhs2yvfP5ZxIME6R5+IFgYBJG PwBkXHsLpmM+on9s4hjS9QDIBbCjPFEiWFcxMxsQh6Z9VrESAuJPzt6b1O0LDTSFt3D4 hOb9pXiJFM4J2wePqheRmexnGP84Q9HHM5G8yjz8J+RrujBa3+L7MF7QC8DDwY+yJMO+ QG9uVn1Ety1zluGFpHjgNs9NbMlu9CrnbycBdpLDIk6r9EiRmrqyEDbfpPlCY0gurlhM UHMg== X-Gm-Message-State: ANoB5pn5SMpvC+lBUS7V7e9k9P8YjBW3p+IqMYu92mi/kPV3GtEI8bwD 6c+FYs2J99Ved0W3ZU5Eb7M0iA== X-Google-Smtp-Source: AA0mqf4wfVRsEWf9AFOdhAoE+E+wRnIO6UPQlwSQr5h7WZkHCgo4KnQbZUBu1EEsJEC13nvpyrb1bg== X-Received: by 2002:a5d:5709:0:b0:241:d71c:5dde with SMTP id a9-20020a5d5709000000b00241d71c5ddemr27080262wrv.678.1669736867789; Tue, 29 Nov 2022 07:47:47 -0800 (PST) Received: from localhost ([2a00:79e0:9d:4:5011:adcc:fddd:accf]) by smtp.gmail.com with ESMTPSA id x9-20020a5d6b49000000b002366e3f1497sm14090560wrw.6.2022.11.29.07.47.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Nov 2022 07:47:47 -0800 (PST) From: Jann Horn To: security@kernel.org, Andrew Morton Cc: Yang Shi , David Hildenbrand , Peter Xu , John Hubbard , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v5 3/3] mm/khugepaged: Invoke MMU notifiers in shmem/file collapse paths Date: Tue, 29 Nov 2022 16:47:30 +0100 Message-Id: <20221129154730.2274278-3-jannh@google.com> X-Mailer: git-send-email 2.38.1.584.g0f3c55d4c2-goog In-Reply-To: <20221129154730.2274278-1-jannh@google.com> References: <20221129154730.2274278-1-jannh@google.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=kyY6Gy0J; spf=pass (imf18.hostedemail.com: domain of jannh@google.com designates 209.85.221.51 as permitted sender) smtp.mailfrom=jannh@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669736869; a=rsa-sha256; cv=none; b=Jpe7kse/fyPyCV1Xviyo4Ut4wehl8lpGu/WcFTNzk6aeT36nVoJH9sSi64ke1dbzeOQA9z Akj04xtYYVBSzytxA2D+YP6Pg0iOg/bcoE2kEvOWHVLAWbE/L1VUYCArhJpVmqVU/bqs1T IQPEpDE634dvgf0/fvCLmYt+btlEkmU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669736869; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YSdw+42zFeQjk2DGkc/5JCpwWvHgOHoXpertIATsPMg=; b=U8XKnIQ0nbXn3/DQlAzn0W159Gqe0UDiSvLekcIHTtmzWVP1UjA+5MdWQ1aeV7FEdoq+HR ic7epJJiLBREctB+T0YOJH8AaUhwQ8pOV+8Iff3hatZMRKQV6PTsjb8o9n0XHFzsmhlyXX pTCet3MX7ckKBkfs3uAwl8AwFkKpLpM= Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=kyY6Gy0J; spf=pass (imf18.hostedemail.com: domain of jannh@google.com designates 209.85.221.51 as permitted sender) smtp.mailfrom=jannh@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam01 X-Stat-Signature: 6fcmktzqih64kyp63kfb8jsb8559nofn X-Rspamd-Queue-Id: 1ABB41C000D X-Rspam-User: X-HE-Tag: 1669736868-47529 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Any codepath that zaps page table entries must invoke MMU notifiers to ensure that secondary MMUs (like KVM) don't keep accessing pages which aren't mapped anymore. Secondary MMUs don't hold their own references to pages that are mirrored over, so failing to notify them can lead to page use-after-free. I'm marking this as addressing an issue introduced in commit f3f0e1d2150b ("khugepaged: add support of collapse for tmpfs/shmem pages"), but most of the security impact of this only came in commit 27e1f8273113 ("khugepaged: enable collapse pmd for pte-mapped THP"), which actually omitted flushes for the removal of present PTEs, not just for the removal of empty page tables. Cc: stable@kernel.org Fixes: f3f0e1d2150b ("khugepaged: add support of collapse for tmpfs/shmem pages") Acked-by: David Hildenbrand Reviewed-by: Yang Shi Signed-off-by: Jann Horn --- Notes: v4: no changes v5: - added ack and reviewed-by mm/khugepaged.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index c3d3ce596bff7..49eb4b4981d88 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1404,6 +1404,7 @@ static void collapse_and_free_pmd(struct mm_struct *mm, struct vm_area_struct *v unsigned long addr, pmd_t *pmdp) { pmd_t pmd; + struct mmu_notifier_range range; mmap_assert_write_locked(mm); if (vma->vm_file) @@ -1415,8 +1416,12 @@ static void collapse_and_free_pmd(struct mm_struct *mm, struct vm_area_struct *v if (vma->anon_vma) lockdep_assert_held_write(&vma->anon_vma->root->rwsem); + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, NULL, mm, addr, + addr + HPAGE_PMD_SIZE); + mmu_notifier_invalidate_range_start(&range); pmd = pmdp_collapse_flush(vma, addr, pmdp); tlb_remove_table_sync_one(); + mmu_notifier_invalidate_range_end(&range); mm_dec_nr_ptes(mm); page_table_check_pte_clear_range(mm, addr, pmd); pte_free(mm, pmd_pgtable(pmd));