From patchwork Thu Jan 17 00:32:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10767247 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1D78F14E5 for ; Thu, 17 Jan 2019 00:34:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BAD0F2DDDC for ; Thu, 17 Jan 2019 00:33:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AB3FE2DE41; Thu, 17 Jan 2019 00:33:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F0D8A2DDDC for ; Thu, 17 Jan 2019 00:33:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ABC0C8E0009; Wed, 16 Jan 2019 19:33:40 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A329E8E000E; Wed, 16 Jan 2019 19:33:40 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4D9648E0009; Wed, 16 Jan 2019 19:33:40 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by kanga.kvack.org (Postfix) with ESMTP id C40788E0009 for ; Wed, 16 Jan 2019 19:33:39 -0500 (EST) Received: by mail-pl1-f199.google.com with SMTP id l9so4939369plt.7 for ; Wed, 16 Jan 2019 16:33:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=TldYzqiExwf3rqiOIlhUUDyqJwbGTn6e1zIomGztBUw=; b=Fk8BO0FfpllBA1sTs5Gh8A/jq4Cmd2mJta/GRVxrZA2JV5rY8lIhqoJAGpcX0yPcbj 5/BmGOhjikO/bDqfhbCTsBHVx+EhBLKe5Cyt6b2PusQWXfHkEOBoB5zx5L089HbpEK4R 9Tm2nXtuPTM5iYSfVijK9HZcijCkC5Xno32bt9EpDZDsJEdw6lHupFXI6VUz4l5wtlq6 HfooqiNKMvJ5L+evH/zHUKDRfKtKF2VlE++MYyvu91L9yK2wZ1tn7K1YbQRC2iiK22DX bE7RRpg2PGbCJTSOyKE5wgZ+ZV1q5pRDGnf0Wtwt53nSxVpiaXBiOeyGYVscn9pymjmI vqSg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukcfDoXgaihJJmISPu2pSl/gV+xflU33ryLRWe6jIe+aUCYmBRlF aDkGtThNTArU7ZvBVrQEMNh9r6P5K8nUB3UZ3h8LFxxQEvWY+usPyk4C6IWOwXGhzh23Ad9viaM jqYMt8C36l0g3Wx9tPaCWAkuWKiLswIbzIi5JgcvmTKx9xuYoF7QgdRIK7CBEkne69w== X-Received: by 2002:a17:902:6bc9:: with SMTP id m9mr12616635plt.173.1547685219433; Wed, 16 Jan 2019 16:33:39 -0800 (PST) X-Google-Smtp-Source: ALg8bN7axEiX/+vD1RpMbvCmgfQqLa5KmKsKfNDpFDl/P0WAs9HqMYOm9z65fEUb6ypIST/l3+iZ X-Received: by 2002:a17:902:6bc9:: with SMTP id m9mr12616547plt.173.1547685218369; Wed, 16 Jan 2019 16:33:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547685218; cv=none; d=google.com; s=arc-20160816; b=TievufwodFzaytaZV+FItuyfHLQuywFjMF+BU3qxRmRNbw05czw9kkBEEK02WmYoSf pwmjsY0c+aLtLyzI9wPliIbh/HPDYGVgKqlkiBK6puEwfW3qSeoIONtuLK97rLo+Bn1x sstoG19JvDPfrSQgd6iadi2gj2AIrWDY8A40SmvvCgp6itjfmYDaqrRVvgFKJlz9jc2b /JqCb9CeD2T+qjTN4Lpn/krwgD9RjyyPz+PoqakmDEUxQVM8lmlRbE0qjHwLwoQiAkaS zg9uhm8O1sxw8n0Xs9wpkBDhq87/XON6X9D4laEaoazgaBjj4W6pLLRTkC2PdB8eRk0r ddbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=TldYzqiExwf3rqiOIlhUUDyqJwbGTn6e1zIomGztBUw=; b=0YuQNeoJsIJwvcNfJviZreYL50QSwGwojDx5EURNceI79mokZwmXoLlkzrnAg5pZRF sywJNtL2gjq9H8sWjOkLUPUhmygS2tlXQYJubN0adML7AZ65YsOInWfPlVjdzVYsU+Zd E2CHWPAyLoY7QX+zCiEUlZ0JmcCxFVg2JhJvGd6IykG6LuyJHatS8knNw8VDA2PDmloX nl4D8XMNh5HSbSvU0CHqLFleI1C6Ka+B2obp0iS34laYuU3j8kXQA3NxYQkYAbSxyWnQ NAzWjgvtBIN8Dm6bRY5R9rtmJwd0lBZj041Wkh5fQUoYIcJfgY/mIlz05Qg0xzkHGJ/J dfSg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga04.intel.com (mga04.intel.com. [192.55.52.120]) by mx.google.com with ESMTPS id y10si7582915plt.406.2019.01.16.16.33.37 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 16:33:38 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) client-ip=192.55.52.120; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Jan 2019 16:33:36 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,488,1539673200"; d="scan'208";a="292166022" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga005.jf.intel.com with ESMTP; 16 Jan 2019 16:33:35 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Rick Edgecombe Subject: [PATCH 05/17] x86/alternative: initializing temporary mm for patching Date: Wed, 16 Jan 2019 16:32:47 -0800 Message-Id: <20190117003259.23141-6-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190117003259.23141-1-rick.p.edgecombe@intel.com> References: <20190117003259.23141-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit To prevent improper use of the PTEs that are used for text patching, we want to use a temporary mm struct. We initailize it by copying the init mm. The address that will be used for patching is taken from the lower area that is usually used for the task memory. Doing so prevents the need to frequently synchronize the temporary-mm (e.g., when BPF programs are installed), since different PGDs are used for the task memory. Finally, we randomize the address of the PTEs to harden against exploits that use these PTEs. Cc: Kees Cook Cc: Peter Zijlstra Cc: Dave Hansen Reviewed-by: Masami Hiramatsu Tested-by: Masami Hiramatsu Suggested-by: Andy Lutomirski Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/pgtable.h | 3 +++ arch/x86/include/asm/text-patching.h | 2 ++ arch/x86/kernel/alternative.c | 3 +++ arch/x86/mm/init_64.c | 36 ++++++++++++++++++++++++++++ init/main.c | 3 +++ 5 files changed, 47 insertions(+) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 40616e805292..e8f630d9a2ed 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1021,6 +1021,9 @@ static inline void __meminit init_trampoline_default(void) /* Default trampoline pgd value */ trampoline_pgd_entry = init_top_pgt[pgd_index(__PAGE_OFFSET)]; } + +void __init poking_init(void); + # ifdef CONFIG_RANDOMIZE_MEMORY void __meminit init_trampoline(void); # else diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/text-patching.h index f8fc8e86cf01..a75eed841eed 100644 --- a/arch/x86/include/asm/text-patching.h +++ b/arch/x86/include/asm/text-patching.h @@ -39,5 +39,7 @@ extern void *text_poke_kgdb(void *addr, const void *opcode, size_t len); extern int poke_int3_handler(struct pt_regs *regs); extern void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler); extern int after_bootmem; +extern __ro_after_init struct mm_struct *poking_mm; +extern __ro_after_init unsigned long poking_addr; #endif /* _ASM_X86_TEXT_PATCHING_H */ diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index c6a3a10a2fd5..57fdde308bb6 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -678,6 +678,9 @@ void *__init_or_module text_poke_early(void *addr, const void *opcode, return addr; } +__ro_after_init struct mm_struct *poking_mm; +__ro_after_init unsigned long poking_addr; + static void *__text_poke(void *addr, const void *opcode, size_t len) { unsigned long flags; diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index bccff68e3267..125c8c48aa24 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -53,6 +53,7 @@ #include #include #include +#include #include "mm_internal.h" @@ -1383,6 +1384,41 @@ unsigned long memory_block_size_bytes(void) return memory_block_size_probed; } +/* + * Initialize an mm_struct to be used during poking and a pointer to be used + * during patching. + */ +void __init poking_init(void) +{ + spinlock_t *ptl; + pte_t *ptep; + + poking_mm = copy_init_mm(); + BUG_ON(!poking_mm); + + /* + * Randomize the poking address, but make sure that the following page + * will be mapped at the same PMD. We need 2 pages, so find space for 3, + * and adjust the address if the PMD ends after the first one. + */ + poking_addr = TASK_UNMAPPED_BASE; + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) + poking_addr += (kaslr_get_random_long("Poking") & PAGE_MASK) % + (TASK_SIZE - TASK_UNMAPPED_BASE - 3 * PAGE_SIZE); + + if (((poking_addr + PAGE_SIZE) & ~PMD_MASK) == 0) + poking_addr += PAGE_SIZE; + + /* + * We need to trigger the allocation of the page-tables that will be + * needed for poking now. Later, poking may be performed in an atomic + * section, which might cause allocation to fail. + */ + ptep = get_locked_pte(poking_mm, poking_addr, &ptl); + BUG_ON(!ptep); + pte_unmap_unlock(ptep, ptl); +} + #ifdef CONFIG_SPARSEMEM_VMEMMAP /* * Initialise the sparsemem vmemmap using huge-pages at the PMD level. diff --git a/init/main.c b/init/main.c index e2e80ca3165a..f5947ba53bb4 100644 --- a/init/main.c +++ b/init/main.c @@ -496,6 +496,8 @@ void __init __weak thread_stack_cache_init(void) void __init __weak mem_encrypt_init(void) { } +void __init __weak poking_init(void) { } + bool initcall_debug; core_param(initcall_debug, initcall_debug, bool, 0644); @@ -730,6 +732,7 @@ asmlinkage __visible void __init start_kernel(void) taskstats_init_early(); delayacct_init(); + poking_init(); check_bugs(); acpi_subsystem_init();