From patchwork Fri Apr 26 00:11:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nadav Amit X-Patchwork-Id: 10918557 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 78CA892A for ; Fri, 26 Apr 2019 07:32:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6640628DA3 for ; Fri, 26 Apr 2019 07:32:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5ABEA28DA6; Fri, 26 Apr 2019 07:32:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.4 required=2.0 tests=BAYES_00,DATE_IN_PAST_06_12, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=no version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B82F128DA3 for ; Fri, 26 Apr 2019 07:32:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D84F6B0010; Fri, 26 Apr 2019 03:31:49 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 548EC6B0266; Fri, 26 Apr 2019 03:31:49 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F10606B0007; Fri, 26 Apr 2019 03:31:48 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by kanga.kvack.org (Postfix) with ESMTP id 5DF306B0266 for ; Fri, 26 Apr 2019 03:31:48 -0400 (EDT) Received: by mail-pl1-f199.google.com with SMTP id gn10so1419648plb.23 for ; Fri, 26 Apr 2019 00:31:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version; bh=5xl4Z+yngPU6NzI5z0NBRpJ19Qz003jNFX+DruJR6U8=; b=kx/B0zzjDsndlf3LhYFY4NAULp3WkhtqF5xb3vd6W8iReBcDAHmVlnjmQv9i0HX5Fs OzEHiLheYacVLSjB4ENyRRyFwTlQivGyOxpomKFBUwR/K2DuUL1dR8PN91PIJZaZuyX0 +BkE0j54T363t5j5QwdIZEPcuzVjGFaDdW5BK6H7NLoFJRSmlyTHKXoQ5xNVQix8MY4U 3jmiIXZlSlLoO8qyi6Be9DLB2Lz2mKRGx2mhBZhcz+eup+pZe8NEa2xbAeTHGyxF5IIN DQ6/wtTamFm7Yo/+7tej8k1WLGxObgm4k6s5NWLEfDxe2Nq48yHxvRmhmnUO38drCOHD pGTQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of namit@vmware.com designates 208.91.0.190 as permitted sender) smtp.mailfrom=namit@vmware.com; dmarc=pass (p=QUARANTINE sp=NONE dis=NONE) header.from=vmware.com X-Gm-Message-State: APjAAAXo7zNQGAgfI1HMYyoPGSdlmw+Znu9s0JcH2U3wg1oDA6h8aKa6 WrhApG27aChojrM62UTcsXSirdeCMg1w4n+895QGa1MGEhCUGUs3drwOgObIxonZ9aNL8YAmdyv knM4DAExk5YLsxRfokWLako0JxMW2AZ29bAOaxz6408GSL3qfZGstAbGsTwpAZJa4Bw== X-Received: by 2002:a63:165f:: with SMTP id 31mr42513483pgw.321.1556263908018; Fri, 26 Apr 2019 00:31:48 -0700 (PDT) X-Google-Smtp-Source: APXvYqyVOtrK1i0pipILPqMhFEQmCjpa+TbvBAnimyaEyuVMkLFQTqWJ/sOwiN6NR4rQs4T4X9tn X-Received: by 2002:a63:165f:: with SMTP id 31mr42513406pgw.321.1556263906949; Fri, 26 Apr 2019 00:31:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1556263906; cv=none; d=google.com; s=arc-20160816; b=gKQ4KMd60FJx/6DkYaiW08NUqYyvcdv7tDpo16TADzJNv5Lgaw1jzyjlmZgUzWgn+P 9tDFj5IEqbApACWS1PhZ1djzgKyzJNTIpa2NMPgaIN/P77orJVhf7MaQHZ5T0qrp+zB0 g73G/bPlyHNwCag5Rw88ggF8VrWcaDEzc3k4T7N4LS+rmuKIu8ION6MFpmljdkuV1O2r GO2j5+GgjbvVuxZQRBb+9d59CjBedLaoCzawESU2xTXpje12I5ovh9nlzKyN1zRzkRUe 3p3egSbzMheoW3a+Idnkz2wZHaWGVe9ak8XAdFM5PIk6oAl+02nkZxpcQqc8RACUG38s OHoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from; bh=5xl4Z+yngPU6NzI5z0NBRpJ19Qz003jNFX+DruJR6U8=; b=XY0GAgxs/3ce+7GnFy/NnxC2zn9QE9udZzbYviqIsIEIOF2FMBHkiwK7L/0ESrxUI9 XszRATxRSCyfA5MF8djTlIkhxFce3w0F0WTw4prJnXHWNeFf3O8BBxMY4bZ5g1mqaUf8 EGhilQeZtsymo3nWMfLHfRu1h7//VW6rBUOJPBUUfBcORJQTI0Z2HyW0ozecaATO+TZm GcGW36/u3b27OloBbbcf5Qctd7hVzMPJ9ZEQG+3ufOK+1AGju1wtTg61ldTEnUiMHngV myx/sQHPH8MgE1s50MNB6DIbiMvS8TsWNfijVfqTXmnusaNt5MvPnBUX44+5/lNyXpuB 6JoQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of namit@vmware.com designates 208.91.0.190 as permitted sender) smtp.mailfrom=namit@vmware.com; dmarc=pass (p=QUARANTINE sp=NONE dis=NONE) header.from=vmware.com Received: from EX13-EDG-OU-002.vmware.com (ex13-edg-ou-002.vmware.com. [208.91.0.190]) by mx.google.com with ESMTPS id f9si22844507pgq.347.2019.04.26.00.31.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 26 Apr 2019 00:31:46 -0700 (PDT) Received-SPF: pass (google.com: domain of namit@vmware.com designates 208.91.0.190 as permitted sender) client-ip=208.91.0.190; Authentication-Results: mx.google.com; spf=pass (google.com: domain of namit@vmware.com designates 208.91.0.190 as permitted sender) smtp.mailfrom=namit@vmware.com; dmarc=pass (p=QUARANTINE sp=NONE dis=NONE) header.from=vmware.com Received: from sc9-mailhost3.vmware.com (10.113.161.73) by EX13-EDG-OU-002.vmware.com (10.113.208.156) with Microsoft SMTP Server id 15.0.1156.6; Fri, 26 Apr 2019 00:31:44 -0700 Received: from sc2-haas01-esx0118.eng.vmware.com (sc2-haas01-esx0118.eng.vmware.com [10.172.44.118]) by sc9-mailhost3.vmware.com (Postfix) with ESMTP id 9016541299; Fri, 26 Apr 2019 00:31:45 -0700 (PDT) From: Nadav Amit To: Peter Zijlstra , Borislav Petkov , Andy Lutomirski , Ingo Molnar CC: , , , Thomas Gleixner , Nadav Amit , Dave Hansen , , , , , , , , , , , Rick Edgecombe , Nadav Amit , Kees Cook , Dave Hansen Subject: [PATCH v5 06/23] x86/alternative: Initialize temporary mm for patching Date: Thu, 25 Apr 2019 17:11:26 -0700 Message-ID: <20190426001143.4983-7-namit@vmware.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190426001143.4983-1-namit@vmware.com> References: <20190426001143.4983-1-namit@vmware.com> MIME-Version: 1.0 Received-SPF: None (EX13-EDG-OU-002.vmware.com: namit@vmware.com does not designate permitted sender hosts) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP To prevent improper use of the PTEs that are used for text patching, the next patches will use a temporary mm struct. Initailize it by copying the init mm. The address that will be used for patching is taken from the lower area that is usually used for the task memory. Doing so prevents the need to frequently synchronize the temporary-mm (e.g., when BPF programs are installed), since different PGDs are used for the task memory. Finally, randomize the address of the PTEs to harden against exploits that use these PTEs. Cc: Kees Cook Cc: Dave Hansen Acked-by: Peter Zijlstra (Intel) Reviewed-by: Masami Hiramatsu Tested-by: Masami Hiramatsu Suggested-by: Andy Lutomirski Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/pgtable.h | 3 +++ arch/x86/include/asm/text-patching.h | 2 ++ arch/x86/kernel/alternative.c | 3 +++ arch/x86/mm/init_64.c | 36 ++++++++++++++++++++++++++++ init/main.c | 3 +++ 5 files changed, 47 insertions(+) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 5cfbbb6d458d..6b6bfdfe83aa 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1038,6 +1038,9 @@ static inline void __meminit init_trampoline_default(void) /* Default trampoline pgd value */ trampoline_pgd_entry = init_top_pgt[pgd_index(__PAGE_OFFSET)]; } + +void __init poking_init(void); + # ifdef CONFIG_RANDOMIZE_MEMORY void __meminit init_trampoline(void); # else diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/text-patching.h index f8fc8e86cf01..a75eed841eed 100644 --- a/arch/x86/include/asm/text-patching.h +++ b/arch/x86/include/asm/text-patching.h @@ -39,5 +39,7 @@ extern void *text_poke_kgdb(void *addr, const void *opcode, size_t len); extern int poke_int3_handler(struct pt_regs *regs); extern void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler); extern int after_bootmem; +extern __ro_after_init struct mm_struct *poking_mm; +extern __ro_after_init unsigned long poking_addr; #endif /* _ASM_X86_TEXT_PATCHING_H */ diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 0a814d73547a..11d5c710a94f 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -679,6 +679,9 @@ void *__init_or_module text_poke_early(void *addr, const void *opcode, return addr; } +__ro_after_init struct mm_struct *poking_mm; +__ro_after_init unsigned long poking_addr; + static void *__text_poke(void *addr, const void *opcode, size_t len) { unsigned long flags; diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index bccff68e3267..125c8c48aa24 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -53,6 +53,7 @@ #include #include #include +#include #include "mm_internal.h" @@ -1383,6 +1384,41 @@ unsigned long memory_block_size_bytes(void) return memory_block_size_probed; } +/* + * Initialize an mm_struct to be used during poking and a pointer to be used + * during patching. + */ +void __init poking_init(void) +{ + spinlock_t *ptl; + pte_t *ptep; + + poking_mm = copy_init_mm(); + BUG_ON(!poking_mm); + + /* + * Randomize the poking address, but make sure that the following page + * will be mapped at the same PMD. We need 2 pages, so find space for 3, + * and adjust the address if the PMD ends after the first one. + */ + poking_addr = TASK_UNMAPPED_BASE; + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) + poking_addr += (kaslr_get_random_long("Poking") & PAGE_MASK) % + (TASK_SIZE - TASK_UNMAPPED_BASE - 3 * PAGE_SIZE); + + if (((poking_addr + PAGE_SIZE) & ~PMD_MASK) == 0) + poking_addr += PAGE_SIZE; + + /* + * We need to trigger the allocation of the page-tables that will be + * needed for poking now. Later, poking may be performed in an atomic + * section, which might cause allocation to fail. + */ + ptep = get_locked_pte(poking_mm, poking_addr, &ptl); + BUG_ON(!ptep); + pte_unmap_unlock(ptep, ptl); +} + #ifdef CONFIG_SPARSEMEM_VMEMMAP /* * Initialise the sparsemem vmemmap using huge-pages at the PMD level. diff --git a/init/main.c b/init/main.c index 598e278b46f7..949eed8015ec 100644 --- a/init/main.c +++ b/init/main.c @@ -504,6 +504,8 @@ void __init __weak thread_stack_cache_init(void) void __init __weak mem_encrypt_init(void) { } +void __init __weak poking_init(void) { } + bool initcall_debug; core_param(initcall_debug, initcall_debug, bool, 0644); @@ -737,6 +739,7 @@ asmlinkage __visible void __init start_kernel(void) taskstats_init_early(); delayacct_init(); + poking_init(); check_bugs(); acpi_subsystem_init();