From patchwork Wed Mar 9 08:37:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peng Liu X-Patchwork-Id: 12774830 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22F5AC433FE for ; Wed, 9 Mar 2022 08:19:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7BABA8D0019; Wed, 9 Mar 2022 03:19:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D4C18D0001; Wed, 9 Mar 2022 03:19:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4F6F38D0019; Wed, 9 Mar 2022 03:19:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0201.hostedemail.com [216.40.44.201]) by kanga.kvack.org (Postfix) with ESMTP id 425998D0001 for ; Wed, 9 Mar 2022 03:19:42 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id E6C6DA0F85 for ; Wed, 9 Mar 2022 08:19:41 +0000 (UTC) X-FDA: 79224148962.17.4EB4265 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf23.hostedemail.com (Postfix) with ESMTP id F4091140002 for ; Wed, 9 Mar 2022 08:19:40 +0000 (UTC) Received: from kwepemi500009.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4KD4nL3RngzBrhW; Wed, 9 Mar 2022 16:17:42 +0800 (CST) Received: from kwepemm600017.china.huawei.com (7.193.23.234) by kwepemi500009.china.huawei.com (7.221.188.199) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Wed, 9 Mar 2022 16:19:37 +0800 Received: from localhost.localdomain (10.175.112.125) by kwepemm600017.china.huawei.com (7.193.23.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Wed, 9 Mar 2022 16:19:36 +0800 From: Peng Liu To: , , , , , , , , , CC: , Subject: [PATCH v2 1/3] kunit: fix UAF when run kfence test case test_gfpzero Date: Wed, 9 Mar 2022 08:37:51 +0000 Message-ID: <20220309083753.1561921-2-liupeng256@huawei.com> X-Mailer: git-send-email 2.18.0.huawei.25 In-Reply-To: <20220309083753.1561921-1-liupeng256@huawei.com> References: <20220309083753.1561921-1-liupeng256@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To kwepemm600017.china.huawei.com (7.193.23.234) X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: F4091140002 X-Stat-Signature: rox4egjg35iokka6dre4dgrwmjs3h9aa Authentication-Results: imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of liupeng256@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=liupeng256@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com X-HE-Tag: 1646813980-725004 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Kunit will create a new thread to run an actual test case, and the main process will wait for the completion of the actual test thread until overtime. The variable "struct kunit test" has local property in function kunit_try_catch_run, and will be used in the test case thread. Task kunit_try_catch_run will free "struct kunit test" when kunit runs overtime, but the actual test case is still run and an UAF bug will be triggered. The above problem has been both observed in a physical machine and qemu platform when running kfence kunit tests. The problem can be triggered when setting CONFIG_KFENCE_NUM_OBJECTS = 65535. Under this setting, the test case test_gfpzero will cost hours and kunit will run to overtime. The follows show the panic log. BUG: unable to handle page fault for address: ffffffff82d882e9 Call Trace: kunit_log_append+0x58/0xd0 ... test_alloc.constprop.0.cold+0x6b/0x8a [kfence_test] test_gfpzero.cold+0x61/0x8ab [kfence_test] kunit_try_run_case+0x4c/0x70 kunit_generic_run_threadfn_adapter+0x11/0x20 kthread+0x166/0x190 ret_from_fork+0x22/0x30 Kernel panic - not syncing: Fatal exception Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014 To solve this problem, the test case thread should be stopped when the kunit frame runs overtime. The stop signal will send in function kunit_try_catch_run, and test_gfpzero will handle it. Signed-off-by: Peng Liu Reviewed-by: Marco Elver Reviewed-by: Brendan Higgins --- lib/kunit/try-catch.c | 1 + mm/kfence/kfence_test.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/kunit/try-catch.c b/lib/kunit/try-catch.c index be38a2c5ecc2..6b3d4db94077 100644 --- a/lib/kunit/try-catch.c +++ b/lib/kunit/try-catch.c @@ -78,6 +78,7 @@ void kunit_try_catch_run(struct kunit_try_catch *try_catch, void *context) if (time_remaining == 0) { kunit_err(test, "try timed out\n"); try_catch->try_result = -ETIMEDOUT; + kthread_stop(task_struct); } exit_code = try_catch->try_result; diff --git a/mm/kfence/kfence_test.c b/mm/kfence/kfence_test.c index 50dbb815a2a8..caed6b4eba94 100644 --- a/mm/kfence/kfence_test.c +++ b/mm/kfence/kfence_test.c @@ -623,7 +623,7 @@ static void test_gfpzero(struct kunit *test) break; test_free(buf2); - if (i == CONFIG_KFENCE_NUM_OBJECTS) { + if (kthread_should_stop() || (i == CONFIG_KFENCE_NUM_OBJECTS)) { kunit_warn(test, "giving up ... cannot get same object back\n"); return; } From patchwork Wed Mar 9 08:37:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peng Liu X-Patchwork-Id: 12774831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 871DBC433F5 for ; Wed, 9 Mar 2022 08:19:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 19E208D001A; Wed, 9 Mar 2022 03:19:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0AFE68D0001; Wed, 9 Mar 2022 03:19:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E91B68D001A; Wed, 9 Mar 2022 03:19:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0238.hostedemail.com [216.40.44.238]) by kanga.kvack.org (Postfix) with ESMTP id DB84B8D0001 for ; Wed, 9 Mar 2022 03:19:42 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 9D0C4A0F85 for ; Wed, 9 Mar 2022 08:19:42 +0000 (UTC) X-FDA: 79224149004.29.9BDB324 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by imf31.hostedemail.com (Postfix) with ESMTP id B20EF2000D for ; Wed, 9 Mar 2022 08:19:41 +0000 (UTC) Received: from kwepemi500010.china.huawei.com (unknown [172.30.72.56]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4KD4lK3VNGz9sSn; Wed, 9 Mar 2022 16:15:57 +0800 (CST) Received: from kwepemm600017.china.huawei.com (7.193.23.234) by kwepemi500010.china.huawei.com (7.221.188.191) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Wed, 9 Mar 2022 16:19:38 +0800 Received: from localhost.localdomain (10.175.112.125) by kwepemm600017.china.huawei.com (7.193.23.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Wed, 9 Mar 2022 16:19:37 +0800 From: Peng Liu To: , , , , , , , , , CC: , Subject: [PATCH v2 2/3] kunit: make kunit_test_timeout compatible with comment Date: Wed, 9 Mar 2022 08:37:52 +0000 Message-ID: <20220309083753.1561921-3-liupeng256@huawei.com> X-Mailer: git-send-email 2.18.0.huawei.25 In-Reply-To: <20220309083753.1561921-1-liupeng256@huawei.com> References: <20220309083753.1561921-1-liupeng256@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To kwepemm600017.china.huawei.com (7.193.23.234) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: B20EF2000D X-Stat-Signature: qrg1di4dgysezfn6dxxzr6k77r7ichfz X-Rspam-User: Authentication-Results: imf31.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf31.hostedemail.com: domain of liupeng256@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=liupeng256@huawei.com X-Rspamd-Server: rspam03 X-HE-Tag: 1646813981-748166 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In function kunit_test_timeout, it is declared "300 * MSEC_PER_SEC" represent 5min. However, it is wrong when dealing with arm64 whose default HZ = 250, or some other situations. Use msecs_to_jiffies to fix this, and kunit_test_timeout will work as desired. Fixes: 5f3e06208920 ("kunit: test: add support for test abort") Signed-off-by: Peng Liu Reviewed-by: Marco Elver Reviewed-by: Daniel Latypov Reviewed-by: Brendan Higgins --- lib/kunit/try-catch.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/kunit/try-catch.c b/lib/kunit/try-catch.c index 6b3d4db94077..f7825991d576 100644 --- a/lib/kunit/try-catch.c +++ b/lib/kunit/try-catch.c @@ -52,7 +52,7 @@ static unsigned long kunit_test_timeout(void) * If tests timeout due to exceeding sysctl_hung_task_timeout_secs, * the task will be killed and an oops generated. */ - return 300 * MSEC_PER_SEC; /* 5 min */ + return 300 * msecs_to_jiffies(MSEC_PER_SEC); /* 5 min */ } void kunit_try_catch_run(struct kunit_try_catch *try_catch, void *context) From patchwork Wed Mar 9 08:37:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peng Liu X-Patchwork-Id: 12774832 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F248EC433EF for ; Wed, 9 Mar 2022 08:19:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D340C8D001B; Wed, 9 Mar 2022 03:19:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CE6338D0001; Wed, 9 Mar 2022 03:19:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B39D38D001B; Wed, 9 Mar 2022 03:19:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0104.hostedemail.com [216.40.44.104]) by kanga.kvack.org (Postfix) with ESMTP id A16A88D0001 for ; Wed, 9 Mar 2022 03:19:43 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 580DA181BECDF for ; Wed, 9 Mar 2022 08:19:43 +0000 (UTC) X-FDA: 79224149046.18.1BF9129 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf09.hostedemail.com (Postfix) with ESMTP id 83104140002 for ; Wed, 9 Mar 2022 08:19:42 +0000 (UTC) Received: from kwepemi500005.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4KD4k241ZNzbc0G; Wed, 9 Mar 2022 16:14:50 +0800 (CST) Received: from kwepemm600017.china.huawei.com (7.193.23.234) by kwepemi500005.china.huawei.com (7.221.188.179) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Wed, 9 Mar 2022 16:19:39 +0800 Received: from localhost.localdomain (10.175.112.125) by kwepemm600017.china.huawei.com (7.193.23.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Wed, 9 Mar 2022 16:19:38 +0800 From: Peng Liu To: , , , , , , , , , CC: , Subject: [PATCH v2 3/3] kfence: test: try to avoid test_gfpzero trigger rcu_stall Date: Wed, 9 Mar 2022 08:37:53 +0000 Message-ID: <20220309083753.1561921-4-liupeng256@huawei.com> X-Mailer: git-send-email 2.18.0.huawei.25 In-Reply-To: <20220309083753.1561921-1-liupeng256@huawei.com> References: <20220309083753.1561921-1-liupeng256@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To kwepemm600017.china.huawei.com (7.193.23.234) X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 83104140002 X-Stat-Signature: qtm5up639hnigppn4ytf56ek37oaqgng Authentication-Results: imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of liupeng256@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=liupeng256@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com X-HE-Tag: 1646813982-837967 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When CONFIG_KFENCE_NUM_OBJECTS is set to a big number, kfence kunit-test-case test_gfpzero will eat up nearly all the CPU's resources and rcu_stall is reported as the following log which is cut from a physical server. rcu: INFO: rcu_sched self-detected stall on CPU rcu: 68-....: (14422 ticks this GP) idle=6ce/1/0x4000000000000002 softirq=592/592 fqs=7500 (t=15004 jiffies g=10677 q=20019) Task dump for CPU 68: task:kunit_try_catch state:R running task stack: 0 pid: 9728 ppid: 2 flags:0x0000020a Call trace: dump_backtrace+0x0/0x1e4 show_stack+0x20/0x2c sched_show_task+0x148/0x170 ... rcu_sched_clock_irq+0x70/0x180 update_process_times+0x68/0xb0 tick_sched_handle+0x38/0x74 ... gic_handle_irq+0x78/0x2c0 el1_irq+0xb8/0x140 kfree+0xd8/0x53c test_alloc+0x264/0x310 [kfence_test] test_gfpzero+0xf4/0x840 [kfence_test] kunit_try_run_case+0x48/0x20c kunit_generic_run_threadfn_adapter+0x28/0x34 kthread+0x108/0x13c ret_from_fork+0x10/0x18 To avoid rcu_stall and unacceptable latency, a schedule point is added to test_gfpzero. Signed-off-by: Peng Liu Reviewed-by: Marco Elver --- mm/kfence/kfence_test.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/kfence/kfence_test.c b/mm/kfence/kfence_test.c index caed6b4eba94..1b50f70a4c0f 100644 --- a/mm/kfence/kfence_test.c +++ b/mm/kfence/kfence_test.c @@ -627,6 +627,7 @@ static void test_gfpzero(struct kunit *test) kunit_warn(test, "giving up ... cannot get same object back\n"); return; } + cond_resched(); } for (i = 0; i < size; i++)