From patchwork Thu Dec 22 20:41:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 13080316 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6384AC4332F for ; Thu, 22 Dec 2022 20:42:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EC527900003; Thu, 22 Dec 2022 15:42:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E7571900002; Thu, 22 Dec 2022 15:42:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3CB1900003; Thu, 22 Dec 2022 15:42:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C4A4F900002 for ; Thu, 22 Dec 2022 15:42:14 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 926CF120A56 for ; Thu, 22 Dec 2022 20:42:14 +0000 (UTC) X-FDA: 80271114588.10.D6A899E Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by imf02.hostedemail.com (Postfix) with ESMTP id E9A3280008 for ; Thu, 22 Dec 2022 20:42:11 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=TzW24rCa; spf=pass (imf02.hostedemail.com: domain of hughd@google.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671741732; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=tZV/0sDZ7NRfwGKBq6mHwBasInBGHVDFUdPPeXHm9Rg=; b=13fKt18cBwnRtZXBwD6SDFS+dokXTjoXB49tkEAEPNqNwZSNR44T+4JNOUojO615p6/jZO fP7BCyNjtQyuGnHA0dZhOEOJpBdOBMFIs1/hbuWDwLdS+bMOiIjkZI4zjEFON9qZuTWWLE /RD0BcZKmJIo4YfbCAzogHpBhl/ppis= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=TzW24rCa; spf=pass (imf02.hostedemail.com: domain of hughd@google.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671741732; a=rsa-sha256; cv=none; b=7VZTchQ8MCPCgwjMva6AaG4kVc2i0/rpzNX28kX0/ENikzhzcHND9+BrIKouAOi2/o3hYj uZkmlIQ+Q0CllsakJsqLC5aaY5XYWwMSRybmn2EoDUhyiqXLDjeiD15pAGgL7UAmJQWD/T lYDY0jNczMwZ7abTSfexz8SRdtY+tCk= Received: by mail-qt1-f176.google.com with SMTP id j16so2306621qtv.4 for ; Thu, 22 Dec 2022 12:42:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:message-id:subject:cc:to:from:date:from:to:cc:subject :date:message-id:reply-to; bh=tZV/0sDZ7NRfwGKBq6mHwBasInBGHVDFUdPPeXHm9Rg=; b=TzW24rCa4e8C15tFOOWZ7pmpUCdmpbw+64G2xzNN4/Sq/+QOE6J+K7juRWwFtIvBVw L/8k/vuK7JN25MWgSxDvbDhi2Fkvf1liQbDAaO3Se4uFVLbtTgG+3ADI40c/x3fw5QHz Lk1LMUyeWj4RoEzCK36scnKywO63MjDfM8bt/qQbvGEtT41hME90BUGcBd8j2P3QBMbp 9EKK7t4XIOxzUz4DuoIwJ1Vs6k0QpcrzyFMUIxPdv6CKOSxS7FUj7DRSyOG0vhI+yW89 L7GH3/1PzRPDLHcM/tNndVSmvYSpJPPedNvq5AbaIO/S67kmoat9zl+3EvgVZ22tqEsy R4TQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=tZV/0sDZ7NRfwGKBq6mHwBasInBGHVDFUdPPeXHm9Rg=; b=b5PU1rtBZpF6SsUXY3H1RatLR5pIDIHSLaNV3zEtXQ7uNwGLyecGGhPtOY565s+t0S V55vOVz5e+oRoD7vy8RbXmrkcr5Lz/MI1F8oXgSnBEemm+xS0VCcgbxpAe+E66qkzJfU psEB8RWGTQojy+uBmdizHNS6Z5FijXh+9zBNZ7tFp3ABkDEMuEdS8i2nCWDPgoqEVEBK m4qyVyK8tc8bEpqoCWqPIyqMAr0pPCoOpwyuwD7Wl5e9kHDtSral4e1zOBj4qsl7JCWE mogyaUyjZdemEq76cMFNaM1mfnz+gdXC5Srsv7WAaUTJyIFVRo+Ns1m19Naq4LIsCdfc BSGg== X-Gm-Message-State: AFqh2kp1HKLSf6rt+d1cVdBS+6YLJQkPnXZwtiWQEvK7fsLpM59n4Wzh EZAzJtdFO5B1dae1MxU9YHpxoQ== X-Google-Smtp-Source: AMrXdXsS4pfVufbB8p+zEzKjunFe2SUSzdllt/cCSVI75tW1M6UEfHywDzvLi3JRZ3B+n4ldFj2Mlg== X-Received: by 2002:ac8:1085:0:b0:3a9:6b48:a130 with SMTP id a5-20020ac81085000000b003a96b48a130mr10119937qtj.34.1671741730924; Thu, 22 Dec 2022 12:42:10 -0800 (PST) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id h19-20020a05620a245300b006f9e103260dsm936918qkn.91.2022.12.22.12.42.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Dec 2022 12:42:10 -0800 (PST) Date: Thu, 22 Dec 2022 12:41:50 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Andrew Morton cc: Jann Horn , Yang Shi , David Hildenbrand , Zach O'Keefe , Song Liu , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH] mm/khugepaged: fix collapse_pte_mapped_thp() to allow anon_vma Message-ID: MIME-Version: 1.0 X-Rspamd-Queue-Id: E9A3280008 X-Stat-Signature: cr9p7tgqtod3ka11uhsche5oa1gsjttn X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1671741731-206620 X-HE-Meta: U2FsdGVkX19+p3lgPceSddjcgVTshtoqU+n5pviG9yIDzQrIEKwahB7hT+pTWezWlOVMyFr6ThLdJ6h7vvpDcym7f1G9MLGXlup08J5CMFjdxWDG5+G90isubeoXvi6CBixDrtKSj2VO+YCc6EFz5OBwSI6SMxBVEUVHK9SSnNHCju1lM2EdSVMbHAYt5vjnWTpkWo3a3Jy/ZH07di2lM93xZ8zX1Ow/fC7fxRpoKzm+OUqgaG+RPdn+w2sK4NB7KJ4Y9NiiqEXbkKPmPIN/79nYaTZAYLJpW69ESTjHp2WKDiZfttUb5bGJhNRQoH6USSCIWsfTt9b5gGj5jRM+y2RhbGQWtHSiCh9aOFae6Q/IjgCX38YekydTnffIh4up2wRh1dBEe8xNjYz04peYioJGTaW2MlunTdRLndZKudKBL2FH20XiL7SZgzpQtJ4EZTeTt5kEr94payNqbPvUQUJ5+anLLnGBWNUXhPG5vO3EPYiRh0ZcedegUUYu2WRY9NsLCekU/eMWMsdpMHV6FJq6CdLkx0Z4GGy8YhZjL5yAbehXsr9NqsznIOuoFIjRo+cqBpbCAqldjU4/Px5k8edSXqc63DGm4spzzbUtzZ7+gekdfyX6deoek92hGCmc+kYVbVFejhEhv+VAeFPSOTkZOvM86zxwcAs4m3szlasRuPsLZqMg7kqrMPylIEr0tmFJArH6dy+3kuIU2RSVvvkJSNyGDbEvxCmngvgD+1GLi8+6vL8JABIa/VD7yCv567EiFVnwjBkfypZ1r/eVGPXiF1PGYyLs5k2uQkHzbkD/cmfcesoFlQmPprfBwreWdYW2szMdYwdLB8o483Nxhg8BDdSXCdL6/hNK5FwTuUgwUXNQEoyGeubSTeBZ8QJVGv7Nbj/5wTWNfn+Zv7SHs3XWqyihaJeknFMxl4QmzYNPNiMnw+JtlJ0u/yOTkNu7Sp7GgKKopE8KDQ2azqx mKOiH628 QGT6qGN/3i5v5lyikFuoLhimC68hI70IZTcrdX30Q3WjHtKHHn7VxH887UqNhCr2lyD5/NwaI5sn/LK+jqoa/hM37QSfbkyajbaxZV9wkqqAXEGRoIu2lAO3NpvQAhhzIyHUqga4Gr3DSB8g= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: uprobe_write_opcode() uses collapse_pte_mapped_thp() to restore huge pmd, when removing a breakpoint from hugepage text: vma->anon_vma is always set in that case, so undo the prohibition. And MADV_COLLAPSE ought to be able to collapse some page tables in a vma which happens to have anon_vma set from CoWing elsewhere. Is anon_vma lock required? Almost not: if any page other than expected subpage of the non-anon huge page is found in the page table, collapse is aborted without making any change. However, it is possible that an anon page was CoWed from this extent in another mm or vma, in which case a concurrent lookup might look here: so keep it away while clearing pmd (but perhaps we shall go back to using pmd_lock() there in future). Note that collapse_pte_mapped_thp() is exceptional in freeing a page table without having cleared its ptes: I'm uneasy about that, and had thought pte_clear()ing appropriate; but exclusive i_mmap lock does fix the problem, and we would have to move the mmu_notification if clearing those ptes. Fixes: 8d3c106e19e8 ("mm/khugepaged: take the right locks for page table retraction") Signed-off-by: Hugh Dickins Cc: Jann Horn Cc: Yang Shi Cc: David Hildenbrand Cc: Zach O'Keefe Cc: Song Liu Cc: [5.4+] Acked-by: David Hildenbrand --- What this fixes is not a dangerous instability! But I suggest Cc stable because uprobes "healing" has regressed in that way, so this should follow 8d3c106e19e8 into those stable releases where it was backported (and may want adjustment there - I'll supply backports as needed). mm/khugepaged.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) --- 6.2-rc/mm/khugepaged.c +++ linux/mm/khugepaged.c @@ -1460,14 +1460,6 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, if (!hugepage_vma_check(vma, vma->vm_flags, false, false, false)) return SCAN_VMA_CHECK; - /* - * Symmetry with retract_page_tables(): Exclude MAP_PRIVATE mappings - * that got written to. Without this, we'd have to also lock the - * anon_vma if one exists. - */ - if (vma->anon_vma) - return SCAN_VMA_CHECK; - /* Keep pmd pgtable for uffd-wp; see comment in retract_page_tables() */ if (userfaultfd_wp(vma)) return SCAN_PTE_UFFD_WP; @@ -1567,8 +1559,14 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, } /* step 4: remove pte entries */ + /* we make no change to anon, but protect concurrent anon page lookup */ + if (vma->anon_vma) + anon_vma_lock_write(vma->anon_vma); + collapse_and_free_pmd(mm, vma, haddr, pmd); + if (vma->anon_vma) + anon_vma_unlock_write(vma->anon_vma); i_mmap_unlock_write(vma->vm_file->f_mapping); maybe_install_pmd: