From patchwork Mon Nov 28 18:02:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jann Horn X-Patchwork-Id: 13057846 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FCCBC433FE for ; Mon, 28 Nov 2022 18:03:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2D1FD6B0074; Mon, 28 Nov 2022 13:03:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2820D6B0075; Mon, 28 Nov 2022 13:03:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1717F6B0078; Mon, 28 Nov 2022 13:03:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 08AD26B0074 for ; Mon, 28 Nov 2022 13:03:14 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C494E4015A for ; Mon, 28 Nov 2022 18:03:13 +0000 (UTC) X-FDA: 80183622666.28.E6BC103 Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by imf15.hostedemail.com (Postfix) with ESMTP id 45345A005B for ; Mon, 28 Nov 2022 18:03:10 +0000 (UTC) Received: by mail-wm1-f45.google.com with SMTP id t25-20020a1c7719000000b003cfa34ea516so9867131wmi.1 for ; Mon, 28 Nov 2022 10:03:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=l98v59BwmjV3ozOfAPw93XrEkUrol6K2zteQO/pxMQA=; b=e/bGIA0HwGB+g6h3LY4eVczqCQQA7q3lac86AA8ayrG5RmwDmE42BKfG03HoYIQKsz fMIN5YuLApIg7+jeOJkwev91+mwmAodAoBnqKNmf+ZyiCiDkOu2GYqHqzKzhbJWp/ecI vWsg6LX5RDFYedCD530UHz52LBU5F7BC2+HnuTEORVeZrSQlOPbK4fD7O+EJmC+ChSqT 4/QrQNGQYxmjFVHe4CjMFUxzDdcOEnJOVqPakZboLMtfqHUntjEYD8GQnRWIeBPl84lH 7h9OmgR2dU+cq7dQy/wrZmcmqRNiPwgNox6e/naAqPQyFZei0ecGmQ4QHiMSLPBUWKVm BCUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=l98v59BwmjV3ozOfAPw93XrEkUrol6K2zteQO/pxMQA=; b=t3g7N72LkH7hY3vuans0xGrorVwm+08tAjMCqHHeI5TtsIhbbpzWHgZDW3/2lIKo9s VdVRHi+ejd3nz1b63ejZW+Kmn9BvOMexFk5eo7w5aTiya0yZsbacaNNbVE1GNe8bmlcz m6j7ACwloue7DYe9NpiprQnEa3TiVdECG4Hw9hFaYX19oOGpWoMM5LIodYGxy2mSE++L XCkUNZm0X3yFawJkYX43N64NE17r8o1rcUOgwW4NLsJ+GroVzwj6yek90w5NHRYtZHcR F3EKsEmJuEchFJvQl33Iu8pTSCJvI2Ag4+p2EtTf6s9/OYTSMeESoMkwNO4c5qef8VX4 AJPA== X-Gm-Message-State: ANoB5pm5iqrtLOR716tMfENe+GYJtJ5msdyh1FmLyoAgNG9ogrRemonk 8yIThp1LnN5u5Yef0QxVA3f7NA== X-Google-Smtp-Source: AA0mqf5bPnlo2nRCz9Eyg7DCrG+Pr9DR0sbrCKNNE0RnhBm/HPey1JfsWRdDN/GOXPZxYrMjEe4RKA== X-Received: by 2002:a05:600c:4fd0:b0:3cf:70a0:f689 with SMTP id o16-20020a05600c4fd000b003cf70a0f689mr43238540wmq.161.1669658589915; Mon, 28 Nov 2022 10:03:09 -0800 (PST) Received: from localhost ([2a00:79e0:9d:4:f4d1:b340:8675:e840]) by smtp.gmail.com with ESMTPSA id d8-20020a5d6dc8000000b002415dd45320sm11359859wrz.112.2022.11.28.10.03.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Nov 2022 10:03:09 -0800 (PST) From: Jann Horn To: security@kernel.org, Andrew Morton Cc: Yang Shi , David Hildenbrand , Peter Xu , John Hubbard , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v4 3/3] mm/khugepaged: Invoke MMU notifiers in shmem/file collapse paths Date: Mon, 28 Nov 2022 19:02:52 +0100 Message-Id: <20221128180252.1684965-3-jannh@google.com> X-Mailer: git-send-email 2.38.1.584.g0f3c55d4c2-goog In-Reply-To: <20221128180252.1684965-1-jannh@google.com> References: <20221128180252.1684965-1-jannh@google.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="e/bGIA0H"; spf=pass (imf15.hostedemail.com: domain of jannh@google.com designates 209.85.128.45 as permitted sender) smtp.mailfrom=jannh@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669658591; a=rsa-sha256; cv=none; b=bO43uiPArS8OVbrAwYAx6u4NavuxTmMgp67pKKkkuvofrxrcKHeyrVHyL/9lG/xO96/D7I 2YLBcHjtr8RsmiUXmmYHCJwnR2kV3EXTRkVQNUSNWi3/2Ec00FNjCsx65/zCdR7IaLvi5b aRFKPLvQ1ZY44hxLnvbuQxWf1kPPt4g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669658591; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=l98v59BwmjV3ozOfAPw93XrEkUrol6K2zteQO/pxMQA=; b=gdncPyx9+96anE1qgQfc/qdJ8lm5IjwUqhQHMXg8saOxnmfPUcHChHbTUVuuy2YNN/VEea pKCmLC4HTtrSYogMJsDrFnPdGDqniLYEFBUw6/gjpuFKbN4OWgSuU+AbURLFggFsCddRjy gGDl3Vcw8/j28CrTyhnrQ8MPjLLqiDc= Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="e/bGIA0H"; spf=pass (imf15.hostedemail.com: domain of jannh@google.com designates 209.85.128.45 as permitted sender) smtp.mailfrom=jannh@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam01 X-Stat-Signature: puddfy84ndgioakscbioujwywte8zbqe X-Rspamd-Queue-Id: 45345A005B X-Rspam-User: X-HE-Tag: 1669658590-701524 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Any codepath that zaps page table entries must invoke MMU notifiers to ensure that secondary MMUs (like KVM) don't keep accessing pages which aren't mapped anymore. Secondary MMUs don't hold their own references to pages that are mirrored over, so failing to notify them can lead to page use-after-free. I'm marking this as addressing an issue introduced in commit f3f0e1d2150b ("khugepaged: add support of collapse for tmpfs/shmem pages"), but most of the security impact of this only came in commit 27e1f8273113 ("khugepaged: enable collapse pmd for pte-mapped THP"), which actually omitted flushes for the removal of present PTEs, not just for the removal of empty page tables. Cc: stable@kernel.org Fixes: f3f0e1d2150b ("khugepaged: add support of collapse for tmpfs/shmem pages") Signed-off-by: Jann Horn Acked-by: David Hildenbrand Reviewed-by: Yang Shi --- v4: no changes mm/khugepaged.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index c3d3ce596bff7..49eb4b4981d88 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1404,6 +1404,7 @@ static void collapse_and_free_pmd(struct mm_struct *mm, struct vm_area_struct *v unsigned long addr, pmd_t *pmdp) { pmd_t pmd; + struct mmu_notifier_range range; mmap_assert_write_locked(mm); if (vma->vm_file) @@ -1415,8 +1416,12 @@ static void collapse_and_free_pmd(struct mm_struct *mm, struct vm_area_struct *v if (vma->anon_vma) lockdep_assert_held_write(&vma->anon_vma->root->rwsem); + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, NULL, mm, addr, + addr + HPAGE_PMD_SIZE); + mmu_notifier_invalidate_range_start(&range); pmd = pmdp_collapse_flush(vma, addr, pmdp); tlb_remove_table_sync_one(); + mmu_notifier_invalidate_range_end(&range); mm_dec_nr_ptes(mm); page_table_check_pte_clear_range(mm, addr, pmd); pte_free(mm, pmd_pgtable(pmd));