From patchwork Mon Sep 26 21:31:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jason A. Donenfeld" X-Patchwork-Id: 12989430 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D232C6FA82 for ; Mon, 26 Sep 2022 21:31:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 691378E0087; Mon, 26 Sep 2022 17:31:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 641EE8E0066; Mon, 26 Sep 2022 17:31:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 509558E0087; Mon, 26 Sep 2022 17:31:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3FC678E0066 for ; Mon, 26 Sep 2022 17:31:46 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 118491A042A for ; Mon, 26 Sep 2022 21:31:46 +0000 (UTC) X-FDA: 79955533812.06.BDF35AF Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf14.hostedemail.com (Postfix) with ESMTP id 6D9AC100009 for ; Mon, 26 Sep 2022 21:31:44 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 5F04BB811CF; Mon, 26 Sep 2022 21:31:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 69695C433D7; Mon, 26 Sep 2022 21:31:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zx2c4.com; s=20210105; t=1664227898; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=wSpLFm27xoetrBI5uTPMQazc/5C4ahgd2M/Py/ahUW8=; b=JSF3Q0f9P1Ptcv2SDsu2vVnXTq/MynGFbI0Xj6deMj7J85Bh384Pc7HnMM5Aw+bubiQXuA aSWS4ImKueBMQ48QaUpQNP4sRFYF3ixwVsa9rRGqeseqIOjJDOpQ0Tuth211T+MGHykb6A JK6PQUGFCiCc64t/FYtyiXgDGib6zIY= Received: by mail.zx2c4.com (ZX2C4 Mail Server) with ESMTPSA id 01a7bf88 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 26 Sep 2022 21:31:37 +0000 (UTC) From: "Jason A. Donenfeld" To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com Cc: "Jason A. Donenfeld" , Kees Cook , Andrew Morton , stable@vger.kernel.org Subject: [PATCH v2 1/2] random: split initialization into early step and later step Date: Mon, 26 Sep 2022 23:31:29 +0200 Message-Id: <20220926213130.1508261-1-Jason@zx2c4.com> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664227904; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=wSpLFm27xoetrBI5uTPMQazc/5C4ahgd2M/Py/ahUW8=; b=m6Yu24xN2MiPmRycscZW9QmnEHPNwrr3kgsTGnfHu92NkZih+VV/E0feIVwao3FeyNHI31 1uEkxScbJlfD7gHKIbQ2ObCGD+U5xvcvSP/6F8jUZb5971UOhjs4nLQg0JwqOCnvzORk1R LcBdOGcrR+g5tX8PJM/U7VgxeK8dW+o= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=zx2c4.com header.s=20210105 header.b=JSF3Q0f9; spf=pass (imf14.hostedemail.com: domain of "SRS0=WzR3=Z5=zx2c4.com=Jason@kernel.org" designates 145.40.68.75 as permitted sender) smtp.mailfrom="SRS0=WzR3=Z5=zx2c4.com=Jason@kernel.org"; dmarc=pass (policy=quarantine) header.from=zx2c4.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664227904; a=rsa-sha256; cv=none; b=i3VaqosvSy4lnjbliBa/O1M0cMOE4PYPCcfwrCM+VvwIRpA4O3ylUy8rv69/Nl8Ozyi18e yDc7DPjmpYFOnO0keViBI60nVxnOGmftLwMlZL42/XJdPlre6KoWRDYVnpqCvGdhF/IlSx d8HlhfLIwipZ3mZi5HjPXX1CQw/gArY= X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 6D9AC100009 Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=zx2c4.com header.s=20210105 header.b=JSF3Q0f9; spf=pass (imf14.hostedemail.com: domain of "SRS0=WzR3=Z5=zx2c4.com=Jason@kernel.org" designates 145.40.68.75 as permitted sender) smtp.mailfrom="SRS0=WzR3=Z5=zx2c4.com=Jason@kernel.org"; dmarc=pass (policy=quarantine) header.from=zx2c4.com X-Stat-Signature: 837jhihtqtbdwhm3q1r5jjsdtjwyrfmn X-Rspam-User: X-HE-Tag: 1664227904-543335 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The full RNG initialization relies on some timestamps, made possible with general functions like time_init() and timekeeping_init(). However, these are only available rather late in initialization. Meanwhile, other things, such as memory allocator functions, make use of the RNG much earlier. So split RNG initialization into two phases. We can give arch randomness very early on, and then later, after timekeeping and such are available, initialize the rest. This ensures that, for example, slabs are properly randomized if RDRAND is available. Without this, CONFIG_SLAB_FREELIST_RANDOM=y loses a degree of its security, because its random seed is potentially deterministic, since it hasn't yet incorporated RDRAND. It also makes it possible to use a better seed in kfence, which currently relies on only the cycle counter. Another positive consequence is that on systems with RDRAND, running with CONFIG_WARN_ALL_UNSEEDED_RANDOM=y results in no warnings at all. Cc: Kees Cook Cc: Andrew Morton Cc: stable@vger.kernel.org Signed-off-by: Jason A. Donenfeld --- drivers/char/random.c | 47 ++++++++++++++++++++++++------------------ include/linux/random.h | 3 ++- init/main.c | 17 +++++++-------- 3 files changed, 37 insertions(+), 30 deletions(-) diff --git a/drivers/char/random.c b/drivers/char/random.c index a90d96f4b3bb..1cb53495e8f7 100644 --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -772,18 +772,13 @@ static int random_pm_notification(struct notifier_block *nb, unsigned long actio static struct notifier_block pm_notifier = { .notifier_call = random_pm_notification }; /* - * The first collection of entropy occurs at system boot while interrupts - * are still turned off. Here we push in latent entropy, RDSEED, a timestamp, - * utsname(), and the command line. Depending on the above configuration knob, - * RDSEED may be considered sufficient for initialization. Note that much - * earlier setup may already have pushed entropy into the input pool by the - * time we get here. + * This is called extremely early, before time keeping functionality is + * available, but arch randomness is. Interrupts are not yet enabled. */ -int __init random_init(const char *command_line) +void __init random_init_early(const char *command_line) { - ktime_t now = ktime_get_real(); - size_t i, longs, arch_bits; unsigned long entropy[BLAKE2S_BLOCK_SIZE / sizeof(long)]; + size_t i, longs, arch_bits; #if defined(LATENT_ENTROPY_PLUGIN) static const u8 compiletime_seed[BLAKE2S_BLOCK_SIZE] __initconst __latent_entropy; @@ -803,34 +798,46 @@ int __init random_init(const char *command_line) i += longs; continue; } - entropy[0] = random_get_entropy(); - _mix_pool_bytes(entropy, sizeof(*entropy)); arch_bits -= sizeof(*entropy) * 8; ++i; } - _mix_pool_bytes(&now, sizeof(now)); - _mix_pool_bytes(utsname(), sizeof(*(utsname()))); + _mix_pool_bytes(command_line, strlen(command_line)); + + if (trust_cpu) + credit_init_bits(arch_bits); +} + +/* + * This is called a little bit after the prior function, and now there is + * access to timestamps counters. Interrupts are not yet enabled. + */ +void __init random_init(void) +{ + unsigned long entropy = random_get_entropy(); + ktime_t now = ktime_get_real(); + + _mix_pool_bytes(utsname(), sizeof(*(utsname()))); + _mix_pool_bytes(&now, sizeof(now)); + _mix_pool_bytes(&entropy, sizeof(entropy)); add_latent_entropy(); /* - * If we were initialized by the bootloader before jump labels are - * initialized, then we should enable the static branch here, where + * If we were initialized by the cpu or bootloader before jump labels + * are initialized, then we should enable the static branch here, where * it's guaranteed that jump labels have been initialized. */ if (!static_branch_likely(&crng_is_ready) && crng_init >= CRNG_READY) crng_set_ready(NULL); + /* Reseed if already seeded by earlier phases. */ if (crng_ready()) crng_reseed(); - else if (trust_cpu) - _credit_init_bits(arch_bits); WARN_ON(register_pm_notifier(&pm_notifier)); - WARN(!random_get_entropy(), "Missing cycle counter and fallback timer; RNG " - "entropy collection will consequently suffer."); - return 0; + WARN(!entropy, "Missing cycle counter and fallback timer; RNG " + "entropy collection will consequently suffer."); } /* diff --git a/include/linux/random.h b/include/linux/random.h index 3fec206487f6..a9e6e16f9774 100644 --- a/include/linux/random.h +++ b/include/linux/random.h @@ -72,7 +72,8 @@ static inline unsigned long get_random_canary(void) return get_random_long() & CANARY_MASK; } -int __init random_init(const char *command_line); +void __init random_init_early(const char *command_line); +void __init random_init(void); bool rng_is_initialized(void); int wait_for_random_bytes(void); diff --git a/init/main.c b/init/main.c index 1fe7942f5d4a..0866e5d0d467 100644 --- a/init/main.c +++ b/init/main.c @@ -976,6 +976,9 @@ asmlinkage __visible void __init __no_sanitize_address start_kernel(void) parse_args("Setting extra init args", extra_init_args, NULL, 0, -1, -1, NULL, set_init_arg); + /* Architectural and non-timekeeping rng init, before allocator init */ + random_init_early(command_line); + /* * These use large bootmem allocations and must precede * kmem_cache_init() @@ -1035,17 +1038,13 @@ asmlinkage __visible void __init __no_sanitize_address start_kernel(void) hrtimers_init(); softirq_init(); timekeeping_init(); - kfence_init(); time_init(); - /* - * For best initial stack canary entropy, prepare it after: - * - setup_arch() for any UEFI RNG entropy and boot cmdline access - * - timekeeping_init() for ktime entropy used in random_init() - * - time_init() for making random_get_entropy() work on some platforms - * - random_init() to initialize the RNG from from early entropy sources - */ - random_init(command_line); + /* This must be after timekeeping is initialized */ + random_init(); + + /* These make use of the fully initialized rng */ + kfence_init(); boot_init_stack_canary(); perf_event_init(); From patchwork Mon Sep 26 21:31:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jason A. Donenfeld" X-Patchwork-Id: 12989431 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D04F0C07E9D for ; Mon, 26 Sep 2022 21:31:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4215F8E0088; Mon, 26 Sep 2022 17:31:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D0B38E0066; Mon, 26 Sep 2022 17:31:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2713D8E0088; Mon, 26 Sep 2022 17:31:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 17FFE8E0066 for ; Mon, 26 Sep 2022 17:31:49 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id E2BD5160386 for ; Mon, 26 Sep 2022 21:31:48 +0000 (UTC) X-FDA: 79955533896.23.6B8FE91 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf24.hostedemail.com (Postfix) with ESMTP id 90B79180007 for ; Mon, 26 Sep 2022 21:31:48 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CE2606145A; Mon, 26 Sep 2022 21:31:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 75A9EC433C1; Mon, 26 Sep 2022 21:31:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zx2c4.com; s=20210105; t=1664227905; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JLbf48vjMlZ56ruPBS9/QkT83yj7vuavaEds8C6XntY=; b=ZejhL1fXQ/v8Qg/iK+oOm7uXWm2BzU4H34s3VE+vjpIghtIo61Co1OU/kvkIwXpZREMKNi w0qUJE8VoDmqFEd+4H/uXVHWsfgPOwG54ULNfvRuZ0U3LI/7o/szg3J9bALMa3Z/z6j9gs 80hQNP19vXnYYd6MKvG+WW8jNxloAuY= Received: by mail.zx2c4.com (ZX2C4 Mail Server) with ESMTPSA id 0b02cf89 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 26 Sep 2022 21:31:44 +0000 (UTC) From: "Jason A. Donenfeld" To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com Cc: "Jason A. Donenfeld" , Alexander Potapenko , Dmitry Vyukov , Andrew Morton , Marco Elver Subject: [PATCH v2 2/2] kfence: use better stack hash seed Date: Mon, 26 Sep 2022 23:31:30 +0200 Message-Id: <20220926213130.1508261-2-Jason@zx2c4.com> In-Reply-To: <20220926213130.1508261-1-Jason@zx2c4.com> References: <20220926213130.1508261-1-Jason@zx2c4.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664227908; a=rsa-sha256; cv=none; b=HmYEXyUyIR4b9zX2XmKwBbKqfBbtgtuo0hKuy1s9BVd79a+Mm+pfnGHiZ8mXB/6BW6c0p1 h/qrBO4m/oTpk8Fn+jXkwZTi+R5aA7yVk0fjeGCERgF+6evhfwOscmWuK3L03d4dQxV50k CNJ2M/J48KeSwt1MdBWDqqX3cn4cUls= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=zx2c4.com header.s=20210105 header.b=ZejhL1fX; dmarc=pass (policy=quarantine) header.from=zx2c4.com; spf=pass (imf24.hostedemail.com: domain of "SRS0=WzR3=Z5=zx2c4.com=Jason@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=WzR3=Z5=zx2c4.com=Jason@kernel.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664227908; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JLbf48vjMlZ56ruPBS9/QkT83yj7vuavaEds8C6XntY=; b=NeGESzuI+DN3bm3P2dMB+78PD/M19RJAx2mttMYU+nyMrUUO+4+kWiR0er8lx/yTPnblXV UO5xqoVEvksmorIv1Kn3j1DTZso3t9NFITeJD2pdw2e/Gdvx53zQD+vD0ktQnXrI7CpGRC m4v1tsS1n2s9aLowa+ZLr+ep+Xy3K+c= X-Stat-Signature: fh8kmi6r6qzyd9b36tq45pzq1nz5n4wf X-Rspam-User: X-Rspamd-Server: rspam04 Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=zx2c4.com header.s=20210105 header.b=ZejhL1fX; dmarc=pass (policy=quarantine) header.from=zx2c4.com; spf=pass (imf24.hostedemail.com: domain of "SRS0=WzR3=Z5=zx2c4.com=Jason@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=WzR3=Z5=zx2c4.com=Jason@kernel.org" X-Rspamd-Queue-Id: 90B79180007 X-HE-Tag: 1664227908-906733 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: As of the prior commit, the RNG will have incorporated both a cycle counter value and RDRAND, in addition to various other environmental noise. Therefore, using get_random_u32() will supply a stronger seed than simply using random_get_entropy(). N.B.: random_get_entropy() should be considered an internal API of random.c and not generally consumed. Cc: Alexander Potapenko Cc: Dmitry Vyukov Cc: Andrew Morton Reviewed-by: Marco Elver Signed-off-by: Jason A. Donenfeld --- mm/kfence/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/kfence/core.c b/mm/kfence/core.c index c252081b11df..239b1b4b094f 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -864,7 +864,7 @@ static void kfence_init_enable(void) void __init kfence_init(void) { - stack_hash_seed = (u32)random_get_entropy(); + stack_hash_seed = get_random_u32(); /* Setting kfence_sample_interval to 0 on boot disables KFENCE. */ if (!kfence_sample_interval)