From patchwork Mon Jan 31 20:32:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12731114 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE34BC433F5 for ; Mon, 31 Jan 2022 20:32:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B8116B00EF; Mon, 31 Jan 2022 15:32:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 219DB6B00F4; Mon, 31 Jan 2022 15:32:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F38B56B00F5; Mon, 31 Jan 2022 15:32:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0069.hostedemail.com [216.40.44.69]) by kanga.kvack.org (Postfix) with ESMTP id D76236B00EF for ; Mon, 31 Jan 2022 15:32:55 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 8F975181C5C32 for ; Mon, 31 Jan 2022 20:32:55 +0000 (UTC) X-FDA: 79091731110.28.117B3D8 Received: from mail-qk1-f177.google.com (mail-qk1-f177.google.com [209.85.222.177]) by imf17.hostedemail.com (Postfix) with ESMTP id 1DEB140003 for ; Mon, 31 Jan 2022 20:32:54 +0000 (UTC) Received: by mail-qk1-f177.google.com with SMTP id m25so12281262qka.9 for ; Mon, 31 Jan 2022 12:32:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+b2VDnizluGVh9+la/y+t/dAo3W53LzEqz2Kl5SWq3M=; b=YngBIjjb/nHQSRa+qiaFZjTH9m/x8VBAIf2kGQ32pwNNBHyLxbRoo3ZxdB+I+6Oroe 8As6fNzzko3D/+CepuYqaHnseC9dE4lJZHAKbOeGxwlM3VV2+NEqYRGq7EZKMx/KFChK FWgvvuxmkldI3pOZosmJ7ktVltjev2UMC+qOgyGSEj/dvYjxGHoenQrLZTuXY+MqxtQH G6VqMbiuCYFjJS9Xi9jOPxvLuRc1K4fTP7k8ursBzujYyJxREZFkqCIw5YzvxtROhWa+ AMQF60zVFOrek/a1GrwkxGd6/H7mpiu0H/vGg0l1ymArsRf8Oa0L8c5Ja5h7C2lnZ8RV I4Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+b2VDnizluGVh9+la/y+t/dAo3W53LzEqz2Kl5SWq3M=; b=XQQ9vY3RtqEGBJT5P8AQcv5H7FY986li/6ObeEudjB6LwZMyAKdchwMHqOlSMA+zO4 tfhpWN9NZVkNJlKghrGkE0ageO0NbSpKeJorYZqLSVcTw/K4FLY6xMDjaJFuJ8842mng W8uIlpUsuQwulvY3vvqQXfDhsnh0sueHtxQL1WtJZlb4FLMhRhwZxJCc+KKUZpuu34pl DdS83t6TzV0X9NurUgjBoW9itAm34OTz484LKUJnqXCQVki+WNRMRzSvOABuLgllh5VT 1tm0NJPthjwTRXw8KSHacAhPrj3JF97p2D5T+Tw9BV1wm1xW/wbXotB+aWG0OAWNicEb EhOg== X-Gm-Message-State: AOAM531QdgcBg4nH5BXLcMHyqejz0IbCuhDDHdLi/2aG4I+m+Y3Xqj2r sR7Z0V2u7KD2felljGQWv/uOxw== X-Google-Smtp-Source: ABdhPJzfANMgqiVqJGB+ggntyJhpzn27PcIgePLyPDDStslF+pnhqRXRQJMP/Ar96PUQKuG6r8ktmQ== X-Received: by 2002:a37:aa08:: with SMTP id t8mr14647255qke.773.1643661174408; Mon, 31 Jan 2022 12:32:54 -0800 (PST) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id j14sm5349661qko.10.2022.01.31.12.32.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Jan 2022 12:32:53 -0800 (PST) From: Pasha Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, rientjes@google.com, pjt@google.com, weixugc@google.com, gthelen@google.com, mingo@redhat.com, will@kernel.org, rppt@kernel.org, dave.hansen@linux.intel.com, hpa@zytor.com, aneesh.kumar@linux.ibm.com, jirislaby@kernel.org, songmuchun@bytedance.com, qydwhotmail@gmail.com, hughd@google.com, ziy@nvidia.com, anshuman.khandual@arm.com Subject: [PATCH v5 1/4] mm/debug_vm_pgtable: remove pte entry from the page table Date: Mon, 31 Jan 2022 20:32:46 +0000 Message-Id: <20220131203249.2832273-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.35.0.rc2.247.g8bbb082509-goog In-Reply-To: <20220131203249.2832273-1-pasha.tatashin@soleen.com> References: <20220131203249.2832273-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspam-User: nil X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 1DEB140003 X-Stat-Signature: qbyp7k8yy8s5hk7nj94ikcakcrpxjwhe Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=YngBIjjb; dmarc=none; spf=pass (imf17.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.222.177 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com X-HE-Tag: 1643661174-64604 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The pte entry that is used in pte_advanced_tests() is never removed from the page table at the end of the test. The issue is detected by page_table_check, to repro compile kernel with the following configs: CONFIG_DEBUG_VM_PGTABLE=y CONFIG_PAGE_TABLE_CHECK=y CONFIG_PAGE_TABLE_CHECK_ENFORCED=y During the boot the following BUG is printed: [ 2.262821] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers [ 2.276826] ------------[ cut here ]------------ [ 2.280426] kernel BUG at mm/page_table_check.c:162! [ 2.284118] invalid opcode: 0000 [#1] PREEMPT SMP PTI [ 2.287787] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.16.0-11413-g2c271fe77d52 #3 [ 2.293226] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.15.0-0-g2dd4b9b3f840-prebuilt.qemu.org 04/01/2014 ... The entry should be properly removed from the page table before the page is released to the free list. Fixes: a5c3b9ffb0f4 ("mm/debug_vm_pgtable: add tests validating advanced arch page table helpers") Cc: stable@vger.kernel.org # 5.9+ Signed-off-by: Pasha Tatashin Reviewed-by: Zi Yan Tested-by: Zi Yan Acked-by: David Rientjes Reviewed-by: Anshuman Khandual --- mm/debug_vm_pgtable.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index a7ac97c76762..db2abd9e415b 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -171,6 +171,8 @@ static void __init pte_advanced_tests(struct pgtable_debug_args *args) ptep_test_and_clear_young(args->vma, args->vaddr, args->ptep); pte = ptep_get(args->ptep); WARN_ON(pte_young(pte)); + + ptep_get_and_clear_full(args->mm, args->vaddr, args->ptep, 1); } static void __init pte_savedwrite_tests(struct pgtable_debug_args *args) From patchwork Mon Jan 31 20:32:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12731115 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F00DC4332F for ; Mon, 31 Jan 2022 20:32:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C7C7B6B00F4; Mon, 31 Jan 2022 15:32:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B90466B00F6; Mon, 31 Jan 2022 15:32:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A9696B00F7; Mon, 31 Jan 2022 15:32:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0170.hostedemail.com [216.40.44.170]) by kanga.kvack.org (Postfix) with ESMTP id 71AA66B00F4 for ; Mon, 31 Jan 2022 15:32:57 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 2878E181FA8CB for ; Mon, 31 Jan 2022 20:32:57 +0000 (UTC) X-FDA: 79091731194.25.729FB94 Received: from mail-qk1-f178.google.com (mail-qk1-f178.google.com [209.85.222.178]) by imf08.hostedemail.com (Postfix) with ESMTP id CA40A16000A for ; Mon, 31 Jan 2022 20:32:56 +0000 (UTC) Received: by mail-qk1-f178.google.com with SMTP id b22so13183793qkk.12 for ; Mon, 31 Jan 2022 12:32:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ztMzOXuGTev/L+fnIZswc+dCNyphqzRLyPQWRmdFkGQ=; b=Md6GZ12Jl8JVEpV/uGyYEs0IP3JBRv0zFtP+THkOMK7/Sr8KxQ7O9qiU+8DW/TzpNC Akc5nWbdd35Z3AqErZVBsGtXj+3F+SrZ9IIn+7RS6XqDfLQdq3Dv3BQlUtwGX2qweZSb /bWB+7PT+wjthvCUUbSwK8+lmPKQXhcFXy0Kp1t0Bcjpw0EXmzL1cGfVIMBR0NfbdUUq mIE4DJeLUC7E4uC4wB+mSQbmS5A7bgKJYfLjZd+gYMmrZH8qw3K5rW/ES4bQfgIA3l50 xTxC9ATLUYebxRDdso0G9dbU2k2bni9mX7X3AV6CzbxsVGhBEI2r4qtyLn0+0ALREg9m DU0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ztMzOXuGTev/L+fnIZswc+dCNyphqzRLyPQWRmdFkGQ=; b=R24tDgbKZiGrs54/pY8d9elpMApKADfjxq/gVe/datQzXapP9KrAl2hWBRxqTLBI0O LOl1mCIdy0OPW96BhBJ8Ax3RHMSg6O0JikQHuzUqKzAH/nQtADe+a2M/xdB31vmOK6Sa AiU+zvwNGJ0gy99qGwvEP6PHafKc11ceCtxJm1yMUBaF1xrMLRe96ImU40pdXbx6IywS mBpwLU2148CqqvYa/YSl7Pwszs9rM87tjOict2rw54g7IrYRd2+nXgdpYM7thnM3U+10 cBIS8WrGGrabjl8XZEC59PQt2P3FBj0Wm8FHOc9sli/ehVLKxU67yXh/vkYN805hQrLe kN7A== X-Gm-Message-State: AOAM533kXAExKCx32TKYFGyiPA0M2BKzd3rJJRu3tsLTDC0wwFCw6UJT elSNQvYkckgU8nuxE+YUY7iNrA== X-Google-Smtp-Source: ABdhPJy7TkmRAP3kG1fRwG+5u3nYXG/B+PDJTVHIT7qNMvRw2rYa1WyDbrqsKB6mPHKWvTMPZpXt/g== X-Received: by 2002:a05:620a:4151:: with SMTP id k17mr14312024qko.630.1643661176090; Mon, 31 Jan 2022 12:32:56 -0800 (PST) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id j14sm5349661qko.10.2022.01.31.12.32.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Jan 2022 12:32:55 -0800 (PST) From: Pasha Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, rientjes@google.com, pjt@google.com, weixugc@google.com, gthelen@google.com, mingo@redhat.com, will@kernel.org, rppt@kernel.org, dave.hansen@linux.intel.com, hpa@zytor.com, aneesh.kumar@linux.ibm.com, jirislaby@kernel.org, songmuchun@bytedance.com, qydwhotmail@gmail.com, hughd@google.com, ziy@nvidia.com, anshuman.khandual@arm.com Subject: [PATCH v5 2/4] mm/page_table_check: use unsigned long for page counters and cleanup Date: Mon, 31 Jan 2022 20:32:47 +0000 Message-Id: <20220131203249.2832273-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.35.0.rc2.247.g8bbb082509-goog In-Reply-To: <20220131203249.2832273-1-pasha.tatashin@soleen.com> References: <20220131203249.2832273-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: CA40A16000A X-Stat-Signature: tr8mpah3p5txmsrhh6gfqc5hsdzhpzz5 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=Md6GZ12J; dmarc=none; spf=pass (imf08.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.222.178 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com X-Rspam-User: nil X-HE-Tag: 1643661176-406669 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For the consistency, use "unsigned long" for all page counters. Also, reduce code duplication by calling __page_table_check_*_clear() from __page_table_check_*_set() functions. Signed-off-by: Pasha Tatashin Reviewed-by: Wei Xu Acked-by: David Rientjes --- mm/page_table_check.c | 35 +++++++---------------------------- 1 file changed, 7 insertions(+), 28 deletions(-) diff --git a/mm/page_table_check.c b/mm/page_table_check.c index 7504e7caa2a1..c61d7ebe13b1 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -86,8 +86,8 @@ static void page_table_check_clear(struct mm_struct *mm, unsigned long addr, { struct page_ext *page_ext; struct page *page; + unsigned long i; bool anon; - int i; if (!pfn_valid(pfn)) return; @@ -121,8 +121,8 @@ static void page_table_check_set(struct mm_struct *mm, unsigned long addr, { struct page_ext *page_ext; struct page *page; + unsigned long i; bool anon; - int i; if (!pfn_valid(pfn)) return; @@ -152,10 +152,10 @@ static void page_table_check_set(struct mm_struct *mm, unsigned long addr, void __page_table_check_zero(struct page *page, unsigned int order) { struct page_ext *page_ext = lookup_page_ext(page); - int i; + unsigned long i; BUG_ON(!page_ext); - for (i = 0; i < (1 << order); i++) { + for (i = 0; i < (1ul << order); i++) { struct page_table_check *ptc = get_page_table_check(page_ext); BUG_ON(atomic_read(&ptc->anon_map_count)); @@ -206,17 +206,10 @@ EXPORT_SYMBOL(__page_table_check_pud_clear); void __page_table_check_pte_set(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte) { - pte_t old_pte; - if (&init_mm == mm) return; - old_pte = *ptep; - if (pte_user_accessible_page(old_pte)) { - page_table_check_clear(mm, addr, pte_pfn(old_pte), - PAGE_SIZE >> PAGE_SHIFT); - } - + __page_table_check_pte_clear(mm, addr, *ptep); if (pte_user_accessible_page(pte)) { page_table_check_set(mm, addr, pte_pfn(pte), PAGE_SIZE >> PAGE_SHIFT, @@ -228,17 +221,10 @@ EXPORT_SYMBOL(__page_table_check_pte_set); void __page_table_check_pmd_set(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp, pmd_t pmd) { - pmd_t old_pmd; - if (&init_mm == mm) return; - old_pmd = *pmdp; - if (pmd_user_accessible_page(old_pmd)) { - page_table_check_clear(mm, addr, pmd_pfn(old_pmd), - PMD_PAGE_SIZE >> PAGE_SHIFT); - } - + __page_table_check_pmd_clear(mm, addr, *pmdp); if (pmd_user_accessible_page(pmd)) { page_table_check_set(mm, addr, pmd_pfn(pmd), PMD_PAGE_SIZE >> PAGE_SHIFT, @@ -250,17 +236,10 @@ EXPORT_SYMBOL(__page_table_check_pmd_set); void __page_table_check_pud_set(struct mm_struct *mm, unsigned long addr, pud_t *pudp, pud_t pud) { - pud_t old_pud; - if (&init_mm == mm) return; - old_pud = *pudp; - if (pud_user_accessible_page(old_pud)) { - page_table_check_clear(mm, addr, pud_pfn(old_pud), - PUD_PAGE_SIZE >> PAGE_SHIFT); - } - + __page_table_check_pud_clear(mm, addr, *pudp); if (pud_user_accessible_page(pud)) { page_table_check_set(mm, addr, pud_pfn(pud), PUD_PAGE_SIZE >> PAGE_SHIFT, From patchwork Mon Jan 31 20:32:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12731116 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 603AEC433FE for ; Mon, 31 Jan 2022 20:33:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE8116B00F6; Mon, 31 Jan 2022 15:33:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D636B6B00F8; Mon, 31 Jan 2022 15:33:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BB83B6B00F9; Mon, 31 Jan 2022 15:33:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0070.hostedemail.com [216.40.44.70]) by kanga.kvack.org (Postfix) with ESMTP id A4C9F6B00F6 for ; Mon, 31 Jan 2022 15:33:00 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 6597B181F0FE6 for ; Mon, 31 Jan 2022 20:33:00 +0000 (UTC) X-FDA: 79091731320.21.86B5CF8 Received: from mail-qv1-f44.google.com (mail-qv1-f44.google.com [209.85.219.44]) by imf15.hostedemail.com (Postfix) with ESMTP id 036A4A0006 for ; Mon, 31 Jan 2022 20:32:58 +0000 (UTC) Received: by mail-qv1-f44.google.com with SMTP id b4so9130141qvf.0 for ; Mon, 31 Jan 2022 12:32:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fF5QFjKZhDTZd5B31TRW94GY69vdDJby6v3gBBayCwY=; b=HMakM+xY1YLPhqv6PfRXrYF6lEKAz8ldAQlsUVG77GLVRPT4XgfUdXYBbDqjbHerrt xlAlPDccJTu54jM1IX3IhH9eY7bHfcLScHm8cpZdvEAtBUEe0OwdWTmm3OpsgrHQOxKc p9fJbv6S+USh2jyF9NwYK2PuMR4blz+hZQUSoTJxMiyTUL+opiBq+wcMSSz/d0GmBIHw F8lNPaRz+Uaj2G743xUJPbnHO5FCBhIIRR5PK9Mt7vGZv7uHbvALBbVPi2mD/ESWMX4Q 9JSwCrXHiq/NqgjmRJ945HfsESAs/4CAd03ax9ZYClBPRTsNg5k7euUEIDikB6zCMR8T Za3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fF5QFjKZhDTZd5B31TRW94GY69vdDJby6v3gBBayCwY=; b=XGtjj2iYgQttlzpzJbMDVfi1a6a69x1zjRnVSdOip5sKTQ+FdHRr7kkRJ7STt+AHje LzaV+Uz1OVTQU73kbBWiGpEd9XWNn19eRx61q42SGed4OjVo5zWkFAixcesriUTaD/Kw xw0a8xodMaHyr6QxGeIdX9q0J0LIsNacaP6qGtQclguvhU75lIbiDMF5fiKFbetTfog4 ln6wXC7W6ODhhMdusUhIpwhfDRxPux+m7MShrLrgG/J61Yw5oIL5Sd9ZL5kfjc18hBZb seTWsqS3sJvM3UO6BOvwQysS/vYzw4Jke7Z3Njb/BHFXZxBnjoFLV7ms8SupkvTma0yF 7LVw== X-Gm-Message-State: AOAM533ndduNL9iYTHWRMauWm75fmcbljA/2fkAd5vcL/A3J5YyrP3R0 D//00jKYuYiwbw7rRcOURR6hiQ== X-Google-Smtp-Source: ABdhPJx9Y6qc+SG5iROabTkXGy2K2xhjBhgrJadrpqmMWIheHQtwTINxgKfqF3vSMeC0FLwOvq+rfw== X-Received: by 2002:a05:6214:5014:: with SMTP id jo20mr19417458qvb.28.1643661178443; Mon, 31 Jan 2022 12:32:58 -0800 (PST) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id j14sm5349661qko.10.2022.01.31.12.32.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Jan 2022 12:32:57 -0800 (PST) From: Pasha Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, rientjes@google.com, pjt@google.com, weixugc@google.com, gthelen@google.com, mingo@redhat.com, will@kernel.org, rppt@kernel.org, dave.hansen@linux.intel.com, hpa@zytor.com, aneesh.kumar@linux.ibm.com, jirislaby@kernel.org, songmuchun@bytedance.com, qydwhotmail@gmail.com, hughd@google.com, ziy@nvidia.com, anshuman.khandual@arm.com Subject: [PATCH v5 3/4] mm/khugepaged: unify collapse pmd clear, flush and free Date: Mon, 31 Jan 2022 20:32:48 +0000 Message-Id: <20220131203249.2832273-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.35.0.rc2.247.g8bbb082509-goog In-Reply-To: <20220131203249.2832273-1-pasha.tatashin@soleen.com> References: <20220131203249.2832273-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 036A4A0006 X-Stat-Signature: bn1kofg9wmqu39apc5wrf9cb5eq3odoj Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=HMakM+xY; dmarc=none; spf=pass (imf15.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.219.44 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com X-Rspam-User: nil X-HE-Tag: 1643661178-925497 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Unify the code that flushes, clears pmd entry, and frees the PTE table level into a new function collapse_and_free_pmd(). This clean-up is useful as in the next patch we will add another call to this function to iterate through PTE prior to freeing the level for page table check. Signed-off-by: Pasha Tatashin Acked-by: David Rientjes --- mm/khugepaged.c | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 35f14d0a00a6..30e59e4af272 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1416,6 +1416,19 @@ static int khugepaged_add_pte_mapped_thp(struct mm_struct *mm, return 0; } +static void collapse_and_free_pmd(struct mm_struct *mm, struct vm_area_struct *vma, + unsigned long addr, pmd_t *pmdp) +{ + spinlock_t *ptl; + pmd_t pmd; + + ptl = pmd_lock(vma->vm_mm, pmdp); + pmd = pmdp_collapse_flush(vma, addr, pmdp); + spin_unlock(ptl); + mm_dec_nr_ptes(mm); + pte_free(mm, pmd_pgtable(pmd)); +} + /** * collapse_pte_mapped_thp - Try to collapse a pte-mapped THP for mm at * address haddr. @@ -1433,7 +1446,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr) struct vm_area_struct *vma = find_vma(mm, haddr); struct page *hpage; pte_t *start_pte, *pte; - pmd_t *pmd, _pmd; + pmd_t *pmd; spinlock_t *ptl; int count = 0; int i; @@ -1509,12 +1522,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr) } /* step 4: collapse pmd */ - ptl = pmd_lock(vma->vm_mm, pmd); - _pmd = pmdp_collapse_flush(vma, haddr, pmd); - spin_unlock(ptl); - mm_dec_nr_ptes(mm); - pte_free(mm, pmd_pgtable(_pmd)); - + collapse_and_free_pmd(mm, vma, haddr, pmd); drop_hpage: unlock_page(hpage); put_page(hpage); @@ -1552,7 +1560,7 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) struct vm_area_struct *vma; struct mm_struct *mm; unsigned long addr; - pmd_t *pmd, _pmd; + pmd_t *pmd; i_mmap_lock_write(mapping); vma_interval_tree_foreach(vma, &mapping->i_mmap, pgoff, pgoff) { @@ -1591,14 +1599,8 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) * reverse order. Trylock is a way to avoid deadlock. */ if (mmap_write_trylock(mm)) { - if (!khugepaged_test_exit(mm)) { - spinlock_t *ptl = pmd_lock(mm, pmd); - /* assume page table is clear */ - _pmd = pmdp_collapse_flush(vma, addr, pmd); - spin_unlock(ptl); - mm_dec_nr_ptes(mm); - pte_free(mm, pmd_pgtable(_pmd)); - } + if (!khugepaged_test_exit(mm)) + collapse_and_free_pmd(mm, vma, addr, pmd); mmap_write_unlock(mm); } else { /* Try again later */ From patchwork Mon Jan 31 20:32:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12731117 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E866C433FE for ; Mon, 31 Jan 2022 20:33:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E28B96B00F8; Mon, 31 Jan 2022 15:33:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DB1F16B00FA; Mon, 31 Jan 2022 15:33:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C057F6B00FB; Mon, 31 Jan 2022 15:33:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay030.a.hostedemail.com [64.99.140.30]) by kanga.kvack.org (Postfix) with ESMTP id AB3EB6B00F8 for ; Mon, 31 Jan 2022 15:33:03 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 74606120BF3 for ; Mon, 31 Jan 2022 20:33:03 +0000 (UTC) X-FDA: 79091731446.02.41CB867 Received: from mail-qv1-f52.google.com (mail-qv1-f52.google.com [209.85.219.52]) by imf04.hostedemail.com (Postfix) with ESMTP id 85E1740007 for ; Mon, 31 Jan 2022 20:33:00 +0000 (UTC) Received: by mail-qv1-f52.google.com with SMTP id o9so13930939qvy.13 for ; Mon, 31 Jan 2022 12:33:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=AM3L4Ev7NB1R87m9UVMqXlPBBRGhukRV4JpXV46C9WE=; b=VgxBIxJ0hiHLQyNThvhA4DR5VIF7DaW5tnlYercL6lG4BdPNRcJ4feaFDr49XnPJgj OAuGN9kkbL/k0WS/ESNGQTOxhfQoFghohuY+0yCEAMeQxbUzMydi6tHXfZgMXfbs7Kf4 N4lYqp3WrX/Jf6/domHnPz0jX13A73MaOvIY9txUUBt1/NS77Jes+qhBP4UYT69Y7R6m 1Ki68agcRKkXDE4MXkSN63RF1O9OkCSqwxFJTwX4vxULibefaqLTgi4AStPTCktfFvxT n/sJ3qOOkYjLtNjHR0GU8V8Fn7ozCOvOBMxfmU7OdEpAreN8LFLqdCGdKAYzrMUWNefn 9i5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AM3L4Ev7NB1R87m9UVMqXlPBBRGhukRV4JpXV46C9WE=; b=QQwYL6NHXWFdkCeXnbZVJ3bzANrrRdYb8IMy499y8XHDrD9P27PlXae7Q4/8gQpQx7 KTMZ7RKZvupBmRm+AH1BGO5LsSty6RR4XWRiwCHrLgRQgU2Jc9fWNIr1vmiBGBXZTLkI aKCZiBqoLXL95QQAiIvnqmNWrLy1A/gvJwU8TNtrhByNI7wkJzTg0ijZYgtw2pTrZkoO bYblRfIVE5JWKGFOjdioRZ9rUnuGLNCDI3zpZmPcR+XRUWiTRk2CoAw17tgOk/v5ECjN TSSqiHclskvDNqWz6NjMSBhJ8WoA3eeSvdD18bcGTDxmW7YZDzEV6lY7VRHP+sGx2JJO 7lSw== X-Gm-Message-State: AOAM533WPmhbqGCLQ1cM9I0rDT+IQcHnM+bPLqHkZIwU3q/keykcD13r BjnjrZy509RrsJHB4FX/oyGC8Q== X-Google-Smtp-Source: ABdhPJzoXyE3v1wQg9qr6hkLjh4LSwViEgR3/H8/DbVGMEMW+KOaQDeNNotNwxteh9FE9xUG/VOW+Q== X-Received: by 2002:a05:6214:29ce:: with SMTP id gh14mr18628233qvb.81.1643661180014; Mon, 31 Jan 2022 12:33:00 -0800 (PST) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id j14sm5349661qko.10.2022.01.31.12.32.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Jan 2022 12:32:59 -0800 (PST) From: Pasha Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, rientjes@google.com, pjt@google.com, weixugc@google.com, gthelen@google.com, mingo@redhat.com, will@kernel.org, rppt@kernel.org, dave.hansen@linux.intel.com, hpa@zytor.com, aneesh.kumar@linux.ibm.com, jirislaby@kernel.org, songmuchun@bytedance.com, qydwhotmail@gmail.com, hughd@google.com, ziy@nvidia.com, anshuman.khandual@arm.com Subject: [PATCH v5 4/4] mm/page_table_check: check entries at pmd levels Date: Mon, 31 Jan 2022 20:32:49 +0000 Message-Id: <20220131203249.2832273-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.35.0.rc2.247.g8bbb082509-goog In-Reply-To: <20220131203249.2832273-1-pasha.tatashin@soleen.com> References: <20220131203249.2832273-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 85E1740007 X-Stat-Signature: joqyywscswymzxi5ajspewspoqjr3ktt X-Rspam-User: nil Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=VgxBIxJ0; spf=pass (imf04.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.219.52 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none X-HE-Tag: 1643661180-33831 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: syzbot detected a case where the page table counters were not properly updated. syzkaller login: ------------[ cut here ]------------ kernel BUG at mm/page_table_check.c:162! invalid opcode: 0000 [#1] PREEMPT SMP KASAN CPU: 0 PID: 3099 Comm: pasha Not tainted 5.16.0+ #48 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIO4 RIP: 0010:__page_table_check_zero+0x159/0x1a0 Code: 7d 3a b2 ff 45 39 f5 74 2a e8 43 38 b2 ff 4d 85 e4 01 RSP: 0018:ffff888010667418 EFLAGS: 00010293 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000 RDX: ffff88800cea8680 RSI: ffffffff81becaf9 RDI: 0000000003 RBP: ffff888010667450 R08: 0000000000000001 R09: 0000000000 R10: ffffffff81becaab R11: 0000000000000001 R12: ffff888008 R13: 0000000000000001 R14: 0000000000000200 R15: dffffc0000 FS: 0000000000000000(0000) GS:ffff888035e00000(0000) knlG0 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007ffd875cad00 CR3: 00000000094ce000 CR4: 0000000000 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000 Call Trace: free_pcp_prepare+0x3be/0xaa0 free_unref_page+0x1c/0x650 ? trace_hardirqs_on+0x6a/0x1d0 free_compound_page+0xec/0x130 free_transhuge_page+0x1be/0x260 __put_compound_page+0x90/0xd0 release_pages+0x54c/0x1060 ? filemap_remove_folio+0x161/0x210 ? lock_downgrade+0x720/0x720 ? __put_page+0x150/0x150 ? filemap_free_folio+0x164/0x350 __pagevec_release+0x7c/0x110 shmem_undo_range+0x85e/0x1250 ... The repro involved having a huge page that is split due to uprobe event temporarily replacing one of the pages in the huge page. Later the huge page was combined again, but the counters were off, as the PTE level was not properly updated. Make sure that when PMD is cleared and prior to freeing the level the PTEs are updated. Fixes: df4e817b7108 ("mm: page table check") Signed-off-by: Pasha Tatashin Acked-by: David Rientjes Reviewed-by: Wei Xu --- include/linux/page_table_check.h | 19 +++++++++++++++++++ mm/khugepaged.c | 3 +++ mm/page_table_check.c | 20 ++++++++++++++++++++ 3 files changed, 42 insertions(+) diff --git a/include/linux/page_table_check.h b/include/linux/page_table_check.h index 38cace1da7b6..01e16c7696ec 100644 --- a/include/linux/page_table_check.h +++ b/include/linux/page_table_check.h @@ -26,6 +26,9 @@ void __page_table_check_pmd_set(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp, pmd_t pmd); void __page_table_check_pud_set(struct mm_struct *mm, unsigned long addr, pud_t *pudp, pud_t pud); +void __page_table_check_pte_clear_range(struct mm_struct *mm, + unsigned long addr, + pmd_t pmd); static inline void page_table_check_alloc(struct page *page, unsigned int order) { @@ -100,6 +103,16 @@ static inline void page_table_check_pud_set(struct mm_struct *mm, __page_table_check_pud_set(mm, addr, pudp, pud); } +static inline void page_table_check_pte_clear_range(struct mm_struct *mm, + unsigned long addr, + pmd_t pmd) +{ + if (static_branch_likely(&page_table_check_disabled)) + return; + + __page_table_check_pte_clear_range(mm, addr, pmd); +} + #else static inline void page_table_check_alloc(struct page *page, unsigned int order) @@ -143,5 +156,11 @@ static inline void page_table_check_pud_set(struct mm_struct *mm, { } +static inline void page_table_check_pte_clear_range(struct mm_struct *mm, + unsigned long addr, + pmd_t pmd) +{ +} + #endif /* CONFIG_PAGE_TABLE_CHECK */ #endif /* __LINUX_PAGE_TABLE_CHECK_H */ diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 30e59e4af272..131492fd1148 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include @@ -1422,10 +1423,12 @@ static void collapse_and_free_pmd(struct mm_struct *mm, struct vm_area_struct *v spinlock_t *ptl; pmd_t pmd; + mmap_assert_write_locked(mm); ptl = pmd_lock(vma->vm_mm, pmdp); pmd = pmdp_collapse_flush(vma, addr, pmdp); spin_unlock(ptl); mm_dec_nr_ptes(mm); + page_table_check_pte_clear_range(mm, addr, pmd); pte_free(mm, pmd_pgtable(pmd)); } diff --git a/mm/page_table_check.c b/mm/page_table_check.c index c61d7ebe13b1..3763bd077861 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -247,3 +247,23 @@ void __page_table_check_pud_set(struct mm_struct *mm, unsigned long addr, } } EXPORT_SYMBOL(__page_table_check_pud_set); + +void __page_table_check_pte_clear_range(struct mm_struct *mm, + unsigned long addr, + pmd_t pmd) +{ + if (&init_mm == mm) + return; + + if (!pmd_bad(pmd) && !pmd_leaf(pmd)) { + pte_t *ptep = pte_offset_map(&pmd, addr); + unsigned long i; + + pte_unmap(ptep); + for (i = 0; i < PTRS_PER_PTE; i++) { + __page_table_check_pte_clear(mm, addr, *ptep); + addr += PAGE_SIZE; + ptep++; + } + } +}