From patchwork Tue Jan 29 00:34:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785113 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5251C6C2 for ; Tue, 29 Jan 2019 00:39:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E1112AE85 for ; Tue, 29 Jan 2019 00:39:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 316282AEA6; Tue, 29 Jan 2019 00:39:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 753F82AEA1 for ; Tue, 29 Jan 2019 00:39:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 500B88E0004; Mon, 28 Jan 2019 19:39:14 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 491888E0009; Mon, 28 Jan 2019 19:39:14 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 26E908E0006; Mon, 28 Jan 2019 19:39:14 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f198.google.com (mail-pl1-f198.google.com [209.85.214.198]) by kanga.kvack.org (Postfix) with ESMTP id CB3768E0003 for ; Mon, 28 Jan 2019 19:39:13 -0500 (EST) Received: by mail-pl1-f198.google.com with SMTP id p3so13023025plk.9 for ; Mon, 28 Jan 2019 16:39:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=YsGRUJOea66o8yQ6mNkBDAZI1aC2dZ1ziRGeicCYzT8=; b=NDBDCBLVTGFAu94sY0luW+5ijP4E95t9I04kXlGkMFKLELmp++Bf+eQtOsP6HLIahm F5IpN5aXUXQr7ClZdY84BJ5z8yCiKEosJzBUu5nguetZ+AxOQmDFgGJCuETMWW7069Yl nsTsaDqMC9SX+CRk7nlU+4AeO99Mmq7vPPdt56e5RVRasydhn89rrgHk/rYiCvGlj4mx pIVmmSKq3KllrDx48b5aJM5XY4MVroDtMZTFzYQ3orhGpuSqQdXUjhF1mis7HktOyWq3 M9VceJwRWzb/d7CcgNRWJIo50kmNZoVWuF2SvpaI8fWUbqU4N2x5qw3UG16w47xmc+Oo SIlQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukczE0gwddW6faiF9gsyVoyba5qP97VLzcEHiFmYYZmDqAyiOrNB NIXOsu1cMhkrN8arg2pDL8vp7p7K4b7JhJJFcTDmq0B1q38bT1jTj0rZdB6vzzgWt3z7S+qNZ+X I/K1+oId7+QfkGWKOARGE5c1FZKQtL3piZlXilfhyl6NUbFONpbGWUKYGJf0nTuot2Q== X-Received: by 2002:a63:24c2:: with SMTP id k185mr21083104pgk.406.1548722353449; Mon, 28 Jan 2019 16:39:13 -0800 (PST) X-Google-Smtp-Source: ALg8bN4hv2M8oWcfpCTcid7Tf0mUgFP65LhyIZNWoEjaOFPaEyiY985PINGorpsBKnVYPcjfFF2D X-Received: by 2002:a63:24c2:: with SMTP id k185mr21083056pgk.406.1548722352459; Mon, 28 Jan 2019 16:39:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722352; cv=none; d=google.com; s=arc-20160816; b=BAuDLbGogrFA1vdIGkMmG4ltwl/xILVR9/WEXGacImCmcwEnQdvmi9KsNKSN1iWwYM cJvp64HwMMm81nFBvc5UzaRURpnatQHv4UsgglcwQJq85UhClWfbxr4LvFkf7ME/d66p 2JuJNCcklIE/CLiNTLROoO0fMtcnVnha2R1ZmlOh0QTq5elYOjeAElB9IW8IKvKyYqvt VxSZaS9Gzn6PSBqcDlTUQD6SIiidkWxbfMRJsiwUdEnJwEBB/GQPSXCPjYvsBSQJEc5D YV/XaP8cxaK+fJxpJE/vWRsXj4e3hwc8E8DyyJ1IP0y5NrchF3frQQrzpvWdU4ph6kA7 9tGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=YsGRUJOea66o8yQ6mNkBDAZI1aC2dZ1ziRGeicCYzT8=; b=VNPfkVGlhooGL3Fdp+BV2f8kJP/LepaUXrQmcdCtXJTKeyr30WGCVtLJ+TDdWaH++5 bmUJJj5+42SYRwuS+KvII6BwXh9WV0A4xw+ErJv37a2a2M5nsloy4ZSHUdxjZEMC4riS AONHi+bMlX7ciH34QNh4Z0TmibZeVZqLoolUKR9ec1DAyQZhuQD5H3xQmGEYJQXVy/P2 HK2HvcIQjMQ9WJmwyYTijk8s97TREUQwcv3Kvll5LzlG+JtqcgdDH0wo5ZLG3eAbSMrs 83NjHDFO3UFoynY5IbVPGjAMhXtISwAp3c4mxImZwO/xaz1YCyNRfNnyKl2Bcj4F2r3K 7siw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga04.intel.com (mga04.intel.com. [192.55.52.120]) by mx.google.com with ESMTPS id l7si33052569pfg.245.2019.01.28.16.39.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:12 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) client-ip=192.55.52.120; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921886" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:11 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Masami Hiramatsu , Rick Edgecombe Subject: [PATCH v2 01/20] Fix "x86/alternatives: Lockdep-enforce text_mutex in text_poke*()" Date: Mon, 28 Jan 2019 16:34:03 -0800 Message-Id: <20190129003422.9328-2-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit text_mutex is currently expected to be held before text_poke() is called, but we kgdb does not take the mutex, and instead *supposedly* ensures the lock is not taken and will not be acquired by any other core while text_poke() is running. The reason for the "supposedly" comment is that it is not entirely clear that this would be the case if kgdb_do_roundup is zero. Create two wrapper functions, text_poke() and text_poke_kgdb() which do or do not run the lockdep assertion respectively. While we are at it, change the return code of text_poke() to something meaningful. One day, callers might actually respect it and the existing BUG_ON() when patching fails could be removed. For kgdb, the return value can actually be used. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Cc: Masami Hiramatsu Fixes: 9222f606506c ("x86/alternatives: Lockdep-enforce text_mutex in text_poke*()") Suggested-by: Peter Zijlstra Acked-by: Jiri Kosina Acked-by: Peter Zijlstra (Intel) Reviewed-by: Masami Hiramatsu Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/text-patching.h | 1 + arch/x86/kernel/alternative.c | 52 ++++++++++++++++++++-------- arch/x86/kernel/kgdb.c | 11 +++--- 3 files changed, 45 insertions(+), 19 deletions(-) diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/text-patching.h index e85ff65c43c3..f8fc8e86cf01 100644 --- a/arch/x86/include/asm/text-patching.h +++ b/arch/x86/include/asm/text-patching.h @@ -35,6 +35,7 @@ extern void *text_poke_early(void *addr, const void *opcode, size_t len); * inconsistent instruction while you patch. */ extern void *text_poke(void *addr, const void *opcode, size_t len); +extern void *text_poke_kgdb(void *addr, const void *opcode, size_t len); extern int poke_int3_handler(struct pt_regs *regs); extern void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler); extern int after_bootmem; diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index d458c7973c56..12fddbc8c55b 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -678,18 +678,7 @@ void *__init_or_module text_poke_early(void *addr, const void *opcode, return addr; } -/** - * text_poke - Update instructions on a live kernel - * @addr: address to modify - * @opcode: source of the copy - * @len: length to copy - * - * Only atomic text poke/set should be allowed when not doing early patching. - * It means the size must be writable atomically and the address must be aligned - * in a way that permits an atomic write. It also makes sure we fit on a single - * page. - */ -void *text_poke(void *addr, const void *opcode, size_t len) +static void *__text_poke(void *addr, const void *opcode, size_t len) { unsigned long flags; char *vaddr; @@ -702,8 +691,6 @@ void *text_poke(void *addr, const void *opcode, size_t len) */ BUG_ON(!after_bootmem); - lockdep_assert_held(&text_mutex); - if (!core_kernel_text((unsigned long)addr)) { pages[0] = vmalloc_to_page(addr); pages[1] = vmalloc_to_page(addr + PAGE_SIZE); @@ -732,6 +719,43 @@ void *text_poke(void *addr, const void *opcode, size_t len) return addr; } +/** + * text_poke - Update instructions on a live kernel + * @addr: address to modify + * @opcode: source of the copy + * @len: length to copy + * + * Only atomic text poke/set should be allowed when not doing early patching. + * It means the size must be writable atomically and the address must be aligned + * in a way that permits an atomic write. It also makes sure we fit on a single + * page. + */ +void *text_poke(void *addr, const void *opcode, size_t len) +{ + lockdep_assert_held(&text_mutex); + + return __text_poke(addr, opcode, len); +} + +/** + * text_poke_kgdb - Update instructions on a live kernel by kgdb + * @addr: address to modify + * @opcode: source of the copy + * @len: length to copy + * + * Only atomic text poke/set should be allowed when not doing early patching. + * It means the size must be writable atomically and the address must be aligned + * in a way that permits an atomic write. It also makes sure we fit on a single + * page. + * + * Context: should only be used by kgdb, which ensures no other core is running, + * despite the fact it does not hold the text_mutex. + */ +void *text_poke_kgdb(void *addr, const void *opcode, size_t len) +{ + return __text_poke(addr, opcode, len); +} + static void do_sync_core(void *info) { sync_core(); diff --git a/arch/x86/kernel/kgdb.c b/arch/x86/kernel/kgdb.c index 5db08425063e..1461544cba8b 100644 --- a/arch/x86/kernel/kgdb.c +++ b/arch/x86/kernel/kgdb.c @@ -758,13 +758,13 @@ int kgdb_arch_set_breakpoint(struct kgdb_bkpt *bpt) if (!err) return err; /* - * It is safe to call text_poke() because normal kernel execution + * It is safe to call text_poke_kgdb() because normal kernel execution * is stopped on all cores, so long as the text_mutex is not locked. */ if (mutex_is_locked(&text_mutex)) return -EBUSY; - text_poke((void *)bpt->bpt_addr, arch_kgdb_ops.gdb_bpt_instr, - BREAK_INSTR_SIZE); + text_poke_kgdb((void *)bpt->bpt_addr, arch_kgdb_ops.gdb_bpt_instr, + BREAK_INSTR_SIZE); err = probe_kernel_read(opc, (char *)bpt->bpt_addr, BREAK_INSTR_SIZE); if (err) return err; @@ -783,12 +783,13 @@ int kgdb_arch_remove_breakpoint(struct kgdb_bkpt *bpt) if (bpt->type != BP_POKE_BREAKPOINT) goto knl_write; /* - * It is safe to call text_poke() because normal kernel execution + * It is safe to call text_poke_kgdb() because normal kernel execution * is stopped on all cores, so long as the text_mutex is not locked. */ if (mutex_is_locked(&text_mutex)) goto knl_write; - text_poke((void *)bpt->bpt_addr, bpt->saved_instr, BREAK_INSTR_SIZE); + text_poke_kgdb((void *)bpt->bpt_addr, bpt->saved_instr, + BREAK_INSTR_SIZE); err = probe_kernel_read(opc, (char *)bpt->bpt_addr, BREAK_INSTR_SIZE); if (err || memcmp(opc, bpt->saved_instr, BREAK_INSTR_SIZE)) goto knl_write; From patchwork Tue Jan 29 00:34:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785111 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 068306C2 for ; Tue, 29 Jan 2019 00:39:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E6C242AE87 for ; Tue, 29 Jan 2019 00:39:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D98252AEA6; Tue, 29 Jan 2019 00:39:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5F7752AE87 for ; Tue, 29 Jan 2019 00:39:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 282628E0008; Mon, 28 Jan 2019 19:39:14 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1F9318E0007; Mon, 28 Jan 2019 19:39:14 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 112428E0006; Mon, 28 Jan 2019 19:39:14 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by kanga.kvack.org (Postfix) with ESMTP id C0CAF8E0001 for ; Mon, 28 Jan 2019 19:39:13 -0500 (EST) Received: by mail-pf1-f197.google.com with SMTP id p15so15383073pfk.7 for ; Mon, 28 Jan 2019 16:39:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=MIEM7tFANWCvfO0km255hENS9lmP/VKfFRdymqsmEG4=; b=LBHmsfyv/LCx6Jt9Ckok6Im5Lv1xPZ8Y+utaYlbt3nRXBrCIm6lOvYeAas2J+ci5Fo JOo+zdw0mAfwl1Pj6xSrM5tVsEsOeTOGL5V5kN0/5yVp0neIDgLUdzLsE8YsPwtTaHCe SyCjqKSXmeIXGA71dXnUkCszIg1e1CYxigXfOMqeeGzqTFoJezSzyV1QLrIychvZF2n0 z64hXnuzAsi/judViyW2A30tTziQDCLQ5LPd5vyf4R8ZJ5DeOOGC4gXG91SeKv6k68zL lPkfyaUsSCvkhaUOdmINrMtdAJlVDf4xSrmqKk7p0KQyVvi1xAlTMcPEL+dU0UpSRccQ TNJg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukflHXBBWw73UDPhr4TA/r1JmHI5wl32r3A10Tu7e+91BUWze3LJ OWe8RzO4SVaxJ1tS/M1I1YxddLMoE33n6peyM0PMYnQEVes9dxf7XEKBC2/75+WpmKyP6rXgRUI sCkGJuqZVs+DcJ3MHcNv+tz0Z+tQtmUAQBFURXliSYHpOBgHUJMkAnGHQtrauKY0JBQ== X-Received: by 2002:a17:902:c05:: with SMTP id 5mr24106496pls.155.1548722353450; Mon, 28 Jan 2019 16:39:13 -0800 (PST) X-Google-Smtp-Source: ALg8bN51RWJbF4K+0gtIldLEJvGY0Ez67nKfgyHYuP4+WVi6SnGB7nc76EDOzCyKdxFGumAggyl8 X-Received: by 2002:a17:902:c05:: with SMTP id 5mr24106466pls.155.1548722352785; Mon, 28 Jan 2019 16:39:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722352; cv=none; d=google.com; s=arc-20160816; b=pyzKWu4aOoGqq4iivSy6Xiw42iWnMnY8eaxl6KSIGklR+YprG9uLgKpQp6ZS9dzS7n +5cD+3ZZgx29Q+d2TkyDCX3lYj77+YKt2G1tgAAAsSAPA0GowqrHUgwgJbtjPXlBLwVl SubsEF1ZGRNuUUxE7iVyrSNmIZzfZnXemECkYWeHheSX34SaEJ15b2mDmMZ1OGpUwODI 8r51IPAhnbDBm/CKFzkFypFSFC1UGXqsKZVOhQrO4HecnEpU3jmtbrN7t4/aJ7qnpIpz LSeQXKMPrQnrG5CMdTtJgae+A+WCdyUwt6++Krjux2c+8bnutSPxCwYAdbFN3lUrrICR RVlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=MIEM7tFANWCvfO0km255hENS9lmP/VKfFRdymqsmEG4=; b=fEn2iEQ+WRFtV2jwvTfGx4PqL416jrHGfzTlsUMQweAvMgMiy3zo4ctprgwIDbBHSY gH8DzkxWsLs82yixPul6UhChBIrPFri8fBamf0v2KLtSB5TJnmssebjG5LF2/ciJDZOd 4FaAKRvfwu6YrAzwViSXnJl+hWNwNcYR5oItxZhXDiIb6pN/rtW1TmlYqNKnvCHcnFWR /Oeu0g6u3YDLs9Dvk0mcgPFMiMXpFZ6FJGxvyYLPMMpWsiI8xRksPwFEMpTuTqmXI1/r 7573d+fAiX2EHduNkY3faGbqZY7RrHPIiWrUQkEqAyMfiJr1is5zuC86qDokP0V5QJM+ ddnQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga04.intel.com (mga04.intel.com. [192.55.52.120]) by mx.google.com with ESMTPS id l7si33052569pfg.245.2019.01.28.16.39.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:12 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) client-ip=192.55.52.120; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921888" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:11 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Masami Hiramatsu , Rick Edgecombe Subject: [PATCH v2 02/20] x86/jump_label: Use text_poke_early() during early init Date: Mon, 28 Jan 2019 16:34:04 -0800 Message-Id: <20190129003422.9328-3-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit There is no apparent reason not to use text_poke_early() while we are during early-init and we do not patch code that might be on the stack (i.e., we'll return to the middle of the patched code). This appears to be the case of jump-labels, so do so. This is required for the next patches that would set a temporary mm for patching, which is initialized after some static-keys are enabled/disabled. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Cc: Masami Hiramatsu Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/jump_label.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/jump_label.c b/arch/x86/kernel/jump_label.c index f99bd26bd3f1..e36cfec0f35e 100644 --- a/arch/x86/kernel/jump_label.c +++ b/arch/x86/kernel/jump_label.c @@ -50,7 +50,12 @@ static void __ref __jump_label_transform(struct jump_entry *entry, jmp.offset = jump_entry_target(entry) - (jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE); - if (early_boot_irqs_disabled) + /* + * As long as we're UP and not yet marked RO, we can use + * text_poke_early; SYSTEM_BOOTING guarantees both, as we switch to + * SYSTEM_SCHEDULING before going either. + */ + if (system_state == SYSTEM_BOOTING) poker = text_poke_early; if (type == JUMP_LABEL_JMP) { From patchwork Tue Jan 29 00:34:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785127 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 74E6B6C2 for ; Tue, 29 Jan 2019 00:39:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 639E32AEB9 for ; Tue, 29 Jan 2019 00:39:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 55F3B2B4C4; Tue, 29 Jan 2019 00:39:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D87382AEB9 for ; Tue, 29 Jan 2019 00:39:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 137A48E000D; Mon, 28 Jan 2019 19:39:15 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0C0D98E0003; Mon, 28 Jan 2019 19:39:14 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D00308E000C; Mon, 28 Jan 2019 19:39:14 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) by kanga.kvack.org (Postfix) with ESMTP id 62D2D8E0006 for ; Mon, 28 Jan 2019 19:39:14 -0500 (EST) Received: by mail-pg1-f200.google.com with SMTP id s27so12711592pgm.4 for ; Mon, 28 Jan 2019 16:39:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=6v9W4vLv93QaEUNRE1Az/Zy+/uNslS9lkuxp7pzJKCg=; b=ovnIdDrjo6+K0mIlqU2mcJYyI+7KqpbcgoXGDZxFWKPQHSXIYPi5gileADA9gLrZ0f 7dKownvHtxk5k/t5x9uf6leWi73CyF0pEUX8JG6fWj4npdk2ElzbVHF+q7+TLmUJbI0R 18CRATEPMwDHbwMWBShkYSzAzKOKTzRZ9Pwqdl4+JQUwYx3OndK0GQ6PyuTOBXki9UiR mE5gmTPSvwP1yyEcUVBB9Vptm2w3EYnSm1j4xgbJ3VfuMZs592FqQrs5OBVL80F2M2Ht QMgjJIxKh4KoXoK34NUyU9u9LlAgsOysfUMV4xTD3bcut36fkeeTZycNPAi8PXIP9OlP U7ug== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukegwAbj6xb2ZjcVsBiBDnsDEK45EgamzhmfFKLGukGKrTZsLiqH 94o1Pyzo9FOmiviKnyMJeQ8Bez3+rSNwZjPQ8PpOGZ2I92NCJPBUWUxMcIWnlFAYUbyvyRby85X Jl2IjH8cDvSfbvJI9dSya22E5tdtaALkAw+upKK2LLTIlX6xy6gUTjd0xoGiA+pv9Tg== X-Received: by 2002:a63:c0f:: with SMTP id b15mr21950916pgl.314.1548722354007; Mon, 28 Jan 2019 16:39:14 -0800 (PST) X-Google-Smtp-Source: ALg8bN4apOgYjfWYHgbpvnUEFbBtpaR8Rkk56rzDPgIFRZwExYHPg9fXDQnIv8QlHUwsDVDCz5L1 X-Received: by 2002:a63:c0f:: with SMTP id b15mr21950879pgl.314.1548722353032; Mon, 28 Jan 2019 16:39:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722352; cv=none; d=google.com; s=arc-20160816; b=SRv4fC24zhofdOT5VFmehIGILdIR+Oax8SdyxhS8SSPknA7FgogQAQLsVWZXbQV8e6 pQlQi3NldT7gIN/lEUzZuEv9TUQvfjkYfjBc35kKYkqnbij90YEQHiXLvmipXLg9WgEC WJ4SgDF5pjZk69WgQ4/uAP5NUVaBMzZ9aBPTNkYpItzJ4X9YOdU4ZTO7mkLyC6fsthNv orl6G5sxbbI69iefzGjWwoW1soOHIAopv/cfkD68esja0dnFTCB36JXSOcJRbm01XVrl UTUmtIseL2IE67N+Y7aXjFX2FQGBaQRVjwITvmiXoqKEU0h/U82Uxh7TK8MKcRTCFLLv 8PjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=6v9W4vLv93QaEUNRE1Az/Zy+/uNslS9lkuxp7pzJKCg=; b=v0oLH6N1acbC5hpZXA2IQi1BwciC6YVBlYubrKF9jeFZUPnZqkg9dn06sGtP+QTcsL PcHXSSPDXKl3Ex9lcLV60qGqbJ77EF+dYZhmNZNC9Ahl7RInlcFvOZLN7cBag9H0rjQv 1+mxljm4BbLu9QiY6R2Ho28Ic+4hSvz82MCIGRs8R5nsJyCtj8oKUM9g+2/s7NG/SvvM jSXNRK+xsGJOs1obT9Owsab+ZgsyUB5Rvqy1qEqbT2EGa/UKIEFcb61KHTKKt9tqFJjR NosbMlcv63JrPA+EgnohvUdWtz4Endz2Uw/pZsMYXlpouoFo8IPoET3P1En7jxlqiiUe 1alg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga04.intel.com (mga04.intel.com. [192.55.52.120]) by mx.google.com with ESMTPS id l7si33052569pfg.245.2019.01.28.16.39.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:12 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) client-ip=192.55.52.120; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921891" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:11 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Kees Cook , Dave Hansen , Nadav Amit , Rick Edgecombe Subject: [PATCH v2 03/20] x86/mm: temporary mm struct Date: Mon, 28 Jan 2019 16:34:05 -0800 Message-Id: <20190129003422.9328-4-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Andy Lutomirski Sometimes we want to set a temporary page-table entries (PTEs) in one of the cores, without allowing other cores to use - even speculatively - these mappings. There are two benefits for doing so: (1) Security: if sensitive PTEs are set, temporary mm prevents their use in other cores. This hardens the security as it prevents exploding a dangling pointer to overwrite sensitive data using the sensitive PTE. (2) Avoiding TLB shootdowns: the PTEs do not need to be flushed in remote page-tables. To do so a temporary mm_struct can be used. Mappings which are private for this mm can be set in the userspace part of the address-space. During the whole time in which the temporary mm is loaded, interrupts must be disabled. The first use-case for temporary PTEs, which will follow, is for poking the kernel text. [ Commit message was written by Nadav ] Cc: Kees Cook Cc: Dave Hansen Acked-by: Peter Zijlstra (Intel) Reviewed-by: Masami Hiramatsu Tested-by: Masami Hiramatsu Signed-off-by: Andy Lutomirski Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/mmu_context.h | 32 ++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h index 19d18fae6ec6..cd0c29e494a6 100644 --- a/arch/x86/include/asm/mmu_context.h +++ b/arch/x86/include/asm/mmu_context.h @@ -356,4 +356,36 @@ static inline unsigned long __get_current_cr3_fast(void) return cr3; } +typedef struct { + struct mm_struct *prev; +} temporary_mm_state_t; + +/* + * Using a temporary mm allows to set temporary mappings that are not accessible + * by other cores. Such mappings are needed to perform sensitive memory writes + * that override the kernel memory protections (e.g., W^X), without exposing the + * temporary page-table mappings that are required for these write operations to + * other cores. + * + * Context: The temporary mm needs to be used exclusively by a single core. To + * harden security IRQs must be disabled while the temporary mm is + * loaded, thereby preventing interrupt handler bugs from override the + * kernel memory protection. + */ +static inline temporary_mm_state_t use_temporary_mm(struct mm_struct *mm) +{ + temporary_mm_state_t state; + + lockdep_assert_irqs_disabled(); + state.prev = this_cpu_read(cpu_tlbstate.loaded_mm); + switch_mm_irqs_off(NULL, mm, current); + return state; +} + +static inline void unuse_temporary_mm(temporary_mm_state_t prev) +{ + lockdep_assert_irqs_disabled(); + switch_mm_irqs_off(NULL, prev.prev, current); +} + #endif /* _ASM_X86_MMU_CONTEXT_H */ From patchwork Tue Jan 29 00:34:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785147 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4B7796C2 for ; Tue, 29 Jan 2019 00:39:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 37E562B4C4 for ; Tue, 29 Jan 2019 00:39:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2B9DB2B4D9; Tue, 29 Jan 2019 00:39:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AA5BA2B4C4 for ; Tue, 29 Jan 2019 00:39:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 07B528E0011; Mon, 28 Jan 2019 19:39:24 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EF8A28E0010; Mon, 28 Jan 2019 19:39:23 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CF5AA8E0011; Mon, 28 Jan 2019 19:39:23 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) by kanga.kvack.org (Postfix) with ESMTP id 89B428E0010 for ; Mon, 28 Jan 2019 19:39:23 -0500 (EST) Received: by mail-pl1-f197.google.com with SMTP id v2so13025616plg.6 for ; Mon, 28 Jan 2019 16:39:23 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=1TyrV2qExacMWNjbbAQV/870MI2JQFH4ha6NlSEuZPU=; b=SJtQgoroL2gHm9oWiybKtX0n+4j2BGkRD4kOjov4RBDrytaTxLWfZ0jB2Qm9Zm/gVO gi9YfGdIRVIhUDw1ioN3W5Mqt8vFSSg4v/yvbpkJAM91S6Fv/dATlE3EdbRm5wP4ciDE a1gndRa12bvNyekUSItGQjsL5DQYjGWdXh7PrHqQoKRFn/00jnNKYRM5wXfRyEkTaAe4 dFROFqKoXOxLeFcx/DsduuPhoHKWzq6ipUT2WRiQp3vPhbu+A8oRzNxDJ/7O84X2ymK3 leyOywvRGjIeYlpfGxdmaxS2i9bx2Fa1hLWhmurLLmvL5dz3tyyt/LQg1KW7dY8i07Xk Yocg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukeImhszIJpdvfH9pg7U+yV+OnhBFOCYq6ShzqNrTwI53NLMhIzK JyQUGTBPna4R3gZ1IhHGmzFScmThFc/oxXIveBdWp0xVQxB9ZD2I0n5+64hd6uyGnRi2x00/1mv XGdXpDoFPxU1SnRBarA+XK6OMJlvEISwXKxgoExg+fVEYCPFA8S40MmAcFWLaP7eaew== X-Received: by 2002:a65:43c5:: with SMTP id n5mr22312206pgp.250.1548722363218; Mon, 28 Jan 2019 16:39:23 -0800 (PST) X-Google-Smtp-Source: ALg8bN5dM0MY7oBuKHcZBgcAxAEhe0HdRadyhEigPoYX71rwZu7lMtHwezvKxkhflIWZQac2imZ6 X-Received: by 2002:a65:43c5:: with SMTP id n5mr22311692pgp.250.1548722353162; Mon, 28 Jan 2019 16:39:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722353; cv=none; d=google.com; s=arc-20160816; b=hWWdjYR9u2gCnjhxtffgYXMcPEaldfJEI/SuyoTVJD/DTRxPLOpO25bDCpAQ4oIvT/ e3M7J0IyNIF8Z+K4LXnB52uhs2U6HTcgoiBB1eOrt6pOjWlh5VCxIGcGQ84pVsx75oH1 PPsfDT0bgfvQim1nXzt/s9k+EDvq6HTIhFvnveao7A0wKjAvGlwkjLFMeoeCco0rsS6T qE0LsOWT88qmuLlB5R5T7B45tJurV3Jt0bw6EHs+2qknjWMBLF5pOOh7ZNBBlyg4BhQn hmho119aTmePa9SPDfsXUMBfvr24/RTipKUUjeK24UPQw+7QSD4RsgCam2K/CxlSX0DH Zl4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=1TyrV2qExacMWNjbbAQV/870MI2JQFH4ha6NlSEuZPU=; b=0ru26so2BDQCi245Xjjuic099V5hlhW7vKIrtu/YlkKzcGvp5IAlGiDxxz0KD/3ZFM HT4JhR0Kpvlmea8scDtbi1k4XDpDbIGlfVO8wn527o79bSjdgHdRxxCDwWqRfABHFo/A yOqSl8kjl3ZS7kdMhn5d9C+xMKXLhLfht5190Ks4K9zPwFr81SU3XTtCh+EmKkHNPisO 5KAwyDgBQA/1IzsiF/eGCW8IsOuoxpg1HG6NBSYi1ZhlZ1Di1hC5dkhL+mGHjEkkvcBh /GYferwhf3OQW89H7iWZCtXlfX8PQq4zPH9t2IwB577omJVzQlspZQ48/4JQ1p54JwLC GlOA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga04.intel.com (mga04.intel.com. [192.55.52.120]) by mx.google.com with ESMTPS id s17si4514712pgi.513.2019.01.28.16.39.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:13 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) client-ip=192.55.52.120; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921894" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:11 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Rick Edgecombe Subject: [PATCH v2 04/20] fork: provide a function for copying init_mm Date: Mon, 28 Jan 2019 16:34:06 -0800 Message-Id: <20190129003422.9328-5-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit Provide a function for copying init_mm. This function will be later used for setting a temporary mm. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Acked-by: Peter Zijlstra (Intel) Reviewed-by: Masami Hiramatsu Tested-by: Masami Hiramatsu Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- include/linux/sched/task.h | 1 + kernel/fork.c | 24 ++++++++++++++++++------ 2 files changed, 19 insertions(+), 6 deletions(-) diff --git a/include/linux/sched/task.h b/include/linux/sched/task.h index 44c6f15800ff..c5a00a7b3beb 100644 --- a/include/linux/sched/task.h +++ b/include/linux/sched/task.h @@ -76,6 +76,7 @@ extern void exit_itimers(struct signal_struct *); extern long _do_fork(unsigned long, unsigned long, unsigned long, int __user *, int __user *, unsigned long); extern long do_fork(unsigned long, unsigned long, unsigned long, int __user *, int __user *); struct task_struct *fork_idle(int); +struct mm_struct *copy_init_mm(void); extern pid_t kernel_thread(int (*fn)(void *), void *arg, unsigned long flags); extern long kernel_wait4(pid_t, int __user *, int, struct rusage *); diff --git a/kernel/fork.c b/kernel/fork.c index b69248e6f0e0..d7b156c49f29 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -1299,13 +1299,20 @@ void mm_release(struct task_struct *tsk, struct mm_struct *mm) complete_vfork_done(tsk); } -/* - * Allocate a new mm structure and copy contents from the - * mm structure of the passed in task structure. +/** + * dup_mm() - duplicates an existing mm structure + * @tsk: the task_struct with which the new mm will be associated. + * @oldmm: the mm to duplicate. + * + * Allocates a new mm structure and copy contents from the provided + * @oldmm structure. + * + * Return: the duplicated mm or NULL on failure. */ -static struct mm_struct *dup_mm(struct task_struct *tsk) +static struct mm_struct *dup_mm(struct task_struct *tsk, + struct mm_struct *oldmm) { - struct mm_struct *mm, *oldmm = current->mm; + struct mm_struct *mm; int err; mm = allocate_mm(); @@ -1372,7 +1379,7 @@ static int copy_mm(unsigned long clone_flags, struct task_struct *tsk) } retval = -ENOMEM; - mm = dup_mm(tsk); + mm = dup_mm(tsk, current->mm); if (!mm) goto fail_nomem; @@ -2187,6 +2194,11 @@ struct task_struct *fork_idle(int cpu) return task; } +struct mm_struct *copy_init_mm(void) +{ + return dup_mm(NULL, &init_mm); +} + /* * Ok, this is the main fork-routine. * From patchwork Tue Jan 29 00:34:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785131 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 165C214E1 for ; Tue, 29 Jan 2019 00:39:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F2AB62B4C4 for ; Tue, 29 Jan 2019 00:39:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E64F72B4D9; Tue, 29 Jan 2019 00:39:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3709F2B4C4 for ; Tue, 29 Jan 2019 00:39:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 373AF8E0003; Mon, 28 Jan 2019 19:39:15 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 30E358E000A; Mon, 28 Jan 2019 19:39:15 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09F538E0009; Mon, 28 Jan 2019 19:39:14 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) by kanga.kvack.org (Postfix) with ESMTP id 8462F8E0003 for ; Mon, 28 Jan 2019 19:39:14 -0500 (EST) Received: by mail-pg1-f198.google.com with SMTP id u17so12647481pgn.17 for ; Mon, 28 Jan 2019 16:39:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=+wi88hx6YH8dIFlmUwZIgPkFWlSQwDsLhghYbt+YCm0=; b=VDYt+W5COjnG3YpCAr9fxT8qkCGcQevCStthLg72NUrgMftyGa+p7jmQRp5jW7nZbQ CrbshApQHXRICbzdO6ujMFCk5e+4gxlqt7c1EfA3PXs4d4zrmrXfkl1VXmi4EqU24UUh ifVYEVkNJ4XyDyYMj7h7+qoC55u1F4Xpo036hGh/yfl7QiHy1toqFObusotRIJbt5F2d 2TGqM69VRm61cdqIYBJczsImhxz5EznOb9rgnF6wblXbdPhJj9InOOEyRBdtIdyva8rb lH9ZnT1nOUA2TFbMpH+B5bLrbEivxE+gi5jfUi3f/VqA5TSthfSRU+lWtS5//9Kdo02Q 1YAA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukeQAlt2dt4LYmuLgajs6GonLI1eD+XCfKeWG26h7hrtn8HVyDo7 nz377jkS3SaiIjADKDaL8wCG5UfFaaWk6t2BUrxZSawmWHExuPjQ2grb6C6wh7gN9bYCTEQRQhU O2XHntjHBmxSLRwotXoZCPz+naLalWe/eGBQUIKEt6VD/zTQ3lWdoBuJ27DeykKQCGA== X-Received: by 2002:a17:902:6113:: with SMTP id t19mr23186416plj.248.1548722354190; Mon, 28 Jan 2019 16:39:14 -0800 (PST) X-Google-Smtp-Source: ALg8bN63uyFqWqF/hsM1JhZLnlvXme4b3OvpUuM/rK1fNLZN54IjYvcxYREgQR4j/ua5cN4cmFda X-Received: by 2002:a17:902:6113:: with SMTP id t19mr23186367plj.248.1548722353303; Mon, 28 Jan 2019 16:39:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722353; cv=none; d=google.com; s=arc-20160816; b=GtJyvAzsrl2FWTQrhpyx+EFYXXsnVhjvEGge8ujp0QANr/OYeO9xFCAGhG9zAqeg7p dfDLxWWY7aQgT1WdScp9bAF1LiKz3UWEuF6waM/lArd3oB6L3jox0mbw8ZOP/lFOf4ay 97Vx21kiN6OeNuRZFXrmVwoZb6chRR9szjTjZ+p6zQ+7jXCsUg1lttlRAedcCW7KjA/P 7JONj6IuPHKE2b9WlmaZAnBxnsYPS5uFZSyNlb2w5z2JLtDS9lIIMtldqqVBzxS/k3rt VfgYFk4S6JuO2F9cETBJscXdDSWFfUI2RFX9yWBO4g1GRpCSkFWs+VHq/QkKKubmE6If BtUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=+wi88hx6YH8dIFlmUwZIgPkFWlSQwDsLhghYbt+YCm0=; b=IqqQzoQyxTbTE7aRQaig0RJNpioTI63sAhcoS6crO4dUV9W6gEHGmJhvIWcETh83/2 KZfK3SRvxVjLCF8rCGRA3ZbjyWrDohSpqAE4Xwxq6PpGIFRTO9Z/Uyl/XwFaRevWX8SN qU3nC7SHsmJnBgRxQnWSFxTbhlJMgxCn/+oConAW70/aH+ml5/RxWETMtm3JxYFauws8 yC//PA3DjR3MKVjp0buMzYPVI9pn8Iz/MdYXJDv4rLdmQkZvSYAGlF2TYDxQNlBc6beq 1XWSkJoD5ZBxrTLG+c7A5fhOxjuboe2n79k+H+uMl+dEIc1ltMJdLwAPRYN0CGBXKVMn sglA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga04.intel.com (mga04.intel.com. [192.55.52.120]) by mx.google.com with ESMTPS id l7si33052569pfg.245.2019.01.28.16.39.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:13 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) client-ip=192.55.52.120; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921898" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:11 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Rick Edgecombe Subject: [PATCH v2 05/20] x86/alternative: initializing temporary mm for patching Date: Mon, 28 Jan 2019 16:34:07 -0800 Message-Id: <20190129003422.9328-6-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit To prevent improper use of the PTEs that are used for text patching, we want to use a temporary mm struct. We initailize it by copying the init mm. The address that will be used for patching is taken from the lower area that is usually used for the task memory. Doing so prevents the need to frequently synchronize the temporary-mm (e.g., when BPF programs are installed), since different PGDs are used for the task memory. Finally, we randomize the address of the PTEs to harden against exploits that use these PTEs. Cc: Kees Cook Cc: Dave Hansen Acked-by: Peter Zijlstra (Intel) Reviewed-by: Masami Hiramatsu Tested-by: Masami Hiramatsu Suggested-by: Andy Lutomirski Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe Signed-off-by: Nadav Amit Signed-off-by: Nadav Amit --- arch/x86/include/asm/pgtable.h | 3 +++ arch/x86/include/asm/text-patching.h | 2 ++ arch/x86/kernel/alternative.c | 3 +++ arch/x86/mm/init_64.c | 36 ++++++++++++++++++++++++++++ init/main.c | 3 +++ 5 files changed, 47 insertions(+) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 40616e805292..e8f630d9a2ed 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1021,6 +1021,9 @@ static inline void __meminit init_trampoline_default(void) /* Default trampoline pgd value */ trampoline_pgd_entry = init_top_pgt[pgd_index(__PAGE_OFFSET)]; } + +void __init poking_init(void); + # ifdef CONFIG_RANDOMIZE_MEMORY void __meminit init_trampoline(void); # else diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/text-patching.h index f8fc8e86cf01..a75eed841eed 100644 --- a/arch/x86/include/asm/text-patching.h +++ b/arch/x86/include/asm/text-patching.h @@ -39,5 +39,7 @@ extern void *text_poke_kgdb(void *addr, const void *opcode, size_t len); extern int poke_int3_handler(struct pt_regs *regs); extern void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler); extern int after_bootmem; +extern __ro_after_init struct mm_struct *poking_mm; +extern __ro_after_init unsigned long poking_addr; #endif /* _ASM_X86_TEXT_PATCHING_H */ diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 12fddbc8c55b..ae05fbb50171 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -678,6 +678,9 @@ void *__init_or_module text_poke_early(void *addr, const void *opcode, return addr; } +__ro_after_init struct mm_struct *poking_mm; +__ro_after_init unsigned long poking_addr; + static void *__text_poke(void *addr, const void *opcode, size_t len) { unsigned long flags; diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index bccff68e3267..125c8c48aa24 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -53,6 +53,7 @@ #include #include #include +#include #include "mm_internal.h" @@ -1383,6 +1384,41 @@ unsigned long memory_block_size_bytes(void) return memory_block_size_probed; } +/* + * Initialize an mm_struct to be used during poking and a pointer to be used + * during patching. + */ +void __init poking_init(void) +{ + spinlock_t *ptl; + pte_t *ptep; + + poking_mm = copy_init_mm(); + BUG_ON(!poking_mm); + + /* + * Randomize the poking address, but make sure that the following page + * will be mapped at the same PMD. We need 2 pages, so find space for 3, + * and adjust the address if the PMD ends after the first one. + */ + poking_addr = TASK_UNMAPPED_BASE; + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) + poking_addr += (kaslr_get_random_long("Poking") & PAGE_MASK) % + (TASK_SIZE - TASK_UNMAPPED_BASE - 3 * PAGE_SIZE); + + if (((poking_addr + PAGE_SIZE) & ~PMD_MASK) == 0) + poking_addr += PAGE_SIZE; + + /* + * We need to trigger the allocation of the page-tables that will be + * needed for poking now. Later, poking may be performed in an atomic + * section, which might cause allocation to fail. + */ + ptep = get_locked_pte(poking_mm, poking_addr, &ptl); + BUG_ON(!ptep); + pte_unmap_unlock(ptep, ptl); +} + #ifdef CONFIG_SPARSEMEM_VMEMMAP /* * Initialise the sparsemem vmemmap using huge-pages at the PMD level. diff --git a/init/main.c b/init/main.c index e2e80ca3165a..f5947ba53bb4 100644 --- a/init/main.c +++ b/init/main.c @@ -496,6 +496,8 @@ void __init __weak thread_stack_cache_init(void) void __init __weak mem_encrypt_init(void) { } +void __init __weak poking_init(void) { } + bool initcall_debug; core_param(initcall_debug, initcall_debug, bool, 0644); @@ -730,6 +732,7 @@ asmlinkage __visible void __init start_kernel(void) taskstats_init_early(); delayacct_init(); + poking_init(); check_bugs(); acpi_subsystem_init(); From patchwork Tue Jan 29 00:34:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785135 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ACAF914E1 for ; Tue, 29 Jan 2019 00:39:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 97E002B4C4 for ; Tue, 29 Jan 2019 00:39:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8BEF22B4D9; Tue, 29 Jan 2019 00:39:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CD9102B4C4 for ; Tue, 29 Jan 2019 00:39:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 89D6B8E0006; Mon, 28 Jan 2019 19:39:15 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 68D5F8E000C; Mon, 28 Jan 2019 19:39:15 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 331E88E000E; Mon, 28 Jan 2019 19:39:15 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f198.google.com (mail-pl1-f198.google.com [209.85.214.198]) by kanga.kvack.org (Postfix) with ESMTP id B50E38E000B for ; Mon, 28 Jan 2019 19:39:14 -0500 (EST) Received: by mail-pl1-f198.google.com with SMTP id p3so13023055plk.9 for ; Mon, 28 Jan 2019 16:39:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=EyhXd8rZ4nSo/EtAgiNf2g+ELch4cYLylR2UlgOhqZY=; b=qWSDWqg7mH6wycwxkuf44jCzNMYiX/VO5ZCm1OS0YEYh9hf8Hr832Nqr/6nsllaDWH F5y/jR2RjNRv1IYL8doguN7vQyV5kn6JluMUAppUjYzUMGYrWgJHto6z+EfzdnPFUm3c ddP2Udy5MRCpQm/JVav2IrFFEpSqC5F2aqvSzdTu9CnEIg7X6SGQ4IaRST+lbB4d9dA5 uWF8KKcM9Bxyw+h9HRsXA4Mz1CMP6FNLM8iClQqKN/eB0WGRlfxm6OkDaSPToZm9bEln vtf3pq+puBIngFaJJ7sRxZPhTxrgwpB3bTD0VIONAFHoZB/ukQVPI1BlvYrG3FEkLqCn JnOg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukfJeALEC87uatVpbJ+yahE5syqbjx4tNOtkI1J61ISWRB2Ie/Rz odM9EJW3JSwv3YgEB6DkjYS9WG0WQIOw4pk1jiWEpxvbz7994ZxNqsZqEupzybOlD7F7Lk4sOHF peik9Ga67UMA9bDNpe7VdCoOA6dvjO8lxQ74nydG93puKyVagwAgQ3cpcSfgvDvXWkA== X-Received: by 2002:a62:6dc7:: with SMTP id i190mr23997750pfc.166.1548722354367; Mon, 28 Jan 2019 16:39:14 -0800 (PST) X-Google-Smtp-Source: ALg8bN4xM2gDU0aK6MEqglVw0ed5ebDxS8spWJExZX/iiFoW9aTktx7O65qJWFCIu42zI5CeKQcC X-Received: by 2002:a62:6dc7:: with SMTP id i190mr23997692pfc.166.1548722353470; Mon, 28 Jan 2019 16:39:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722353; cv=none; d=google.com; s=arc-20160816; b=V5QJow5rlBMC1hgpFYlFqD/mNvKhKAQhGbaXFGZU8Vao+KDFObrJV3uCrl/Pei92ja yWZJm91FPf+ah1p+xvX6ZWSWK13z0pcpF2NjEqL0OWfti/DWBnxXAypEtQWxFJ1xpvfh HPyGQ0GDH9Qfoj+bRyOwnMDZ4fNfZAGAtBx3tX9nLCpi2rmWUdDGuNrVAEvp5zyOZTJj UYsNpUdHpbgvPN10mpcFjflL9kvAT95yhCeaL6uHvyx8KR7tN4xXcnC5q8qrpncEqelV 1SQ727kJ7l4AbKOBwkXWhTMYPxNAl9d4mETvnIix5x3ox9ivrpaKvKxl/pJ4U6V92N2I 9WBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=EyhXd8rZ4nSo/EtAgiNf2g+ELch4cYLylR2UlgOhqZY=; b=EG+BMZlu9R/IHA2FsTwW7PLLCANRRx054/h6XHtvZiqfuEtTFIcJUhl0G2qq7Ac7HZ H7qZY7ZHknUr33YUemJ9jQm/sjgNPp9BA5PctYTA6ova89nXqewPsGl6rQgFvKtV9L3Z DjHLToPbwjnskReX58UFuC1vBtC/NhTtWXwUrvDinhb9L4r9g10qqmEhmwQ06YG9wafs LiwRwDycUowNZkK/bEiERzxOBHuqJPDaJ9vn0EwS5f5t6QEMw1120gFgJcBEZZ/jm1NR rMWR6Jf8JvN1ITJJ7UvMpGTZu2sM+gzbOOoufR6LOEEwdpFLfiRvgkxphGpDDg2BKSun 9cow== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga04.intel.com (mga04.intel.com. [192.55.52.120]) by mx.google.com with ESMTPS id s17si4514712pgi.513.2019.01.28.16.39.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:13 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) client-ip=192.55.52.120; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921901" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:11 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Masami Hiramatsu , Rick Edgecombe Subject: [PATCH v2 06/20] x86/alternative: use temporary mm for text poking Date: Mon, 28 Jan 2019 16:34:08 -0800 Message-Id: <20190129003422.9328-7-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit text_poke() can potentially compromise the security as it sets temporary PTEs in the fixmap. These PTEs might be used to rewrite the kernel code from other cores accidentally or maliciously, if an attacker gains the ability to write onto kernel memory. Moreover, since remote TLBs are not flushed after the temporary PTEs are removed, the time-window in which the code is writable is not limited if the fixmap PTEs - maliciously or accidentally - are cached in the TLB. To address these potential security hazards, we use a temporary mm for patching the code. Finally, text_poke() is also not conservative enough when mapping pages, as it always tries to map 2 pages, even when a single one is sufficient. So try to be more conservative, and do not map more than needed. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Cc: Masami Hiramatsu Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/fixmap.h | 2 - arch/x86/kernel/alternative.c | 106 +++++++++++++++++++++++++++------- arch/x86/xen/mmu_pv.c | 2 - 3 files changed, 84 insertions(+), 26 deletions(-) diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h index 50ba74a34a37..9da8cccdf3fb 100644 --- a/arch/x86/include/asm/fixmap.h +++ b/arch/x86/include/asm/fixmap.h @@ -103,8 +103,6 @@ enum fixed_addresses { #ifdef CONFIG_PARAVIRT FIX_PARAVIRT_BOOTMAP, #endif - FIX_TEXT_POKE1, /* reserve 2 pages for text_poke() */ - FIX_TEXT_POKE0, /* first page is last, because allocation is backward */ #ifdef CONFIG_X86_INTEL_MID FIX_LNW_VRTC, #endif diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index ae05fbb50171..76d482a2b716 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -683,41 +684,102 @@ __ro_after_init unsigned long poking_addr; static void *__text_poke(void *addr, const void *opcode, size_t len) { + bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE; + temporary_mm_state_t prev; + struct page *pages[2] = {NULL}; unsigned long flags; - char *vaddr; - struct page *pages[2]; - int i; + pte_t pte, *ptep; + spinlock_t *ptl; + pgprot_t prot; /* - * While boot memory allocator is runnig we cannot use struct - * pages as they are not yet initialized. + * While boot memory allocator is running we cannot use struct pages as + * they are not yet initialized. */ BUG_ON(!after_bootmem); if (!core_kernel_text((unsigned long)addr)) { pages[0] = vmalloc_to_page(addr); - pages[1] = vmalloc_to_page(addr + PAGE_SIZE); + if (cross_page_boundary) + pages[1] = vmalloc_to_page(addr + PAGE_SIZE); } else { pages[0] = virt_to_page(addr); WARN_ON(!PageReserved(pages[0])); - pages[1] = virt_to_page(addr + PAGE_SIZE); + if (cross_page_boundary) + pages[1] = virt_to_page(addr + PAGE_SIZE); } - BUG_ON(!pages[0]); + BUG_ON(!pages[0] || (cross_page_boundary && !pages[1])); + local_irq_save(flags); - set_fixmap(FIX_TEXT_POKE0, page_to_phys(pages[0])); - if (pages[1]) - set_fixmap(FIX_TEXT_POKE1, page_to_phys(pages[1])); - vaddr = (char *)fix_to_virt(FIX_TEXT_POKE0); - memcpy(&vaddr[(unsigned long)addr & ~PAGE_MASK], opcode, len); - clear_fixmap(FIX_TEXT_POKE0); - if (pages[1]) - clear_fixmap(FIX_TEXT_POKE1); - local_flush_tlb(); - sync_core(); - /* Could also do a CLFLUSH here to speed up CPU recovery; but - that causes hangs on some VIA CPUs. */ - for (i = 0; i < len; i++) - BUG_ON(((char *)addr)[i] != ((char *)opcode)[i]); + + /* + * The lock is not really needed, but this allows to avoid open-coding. + */ + ptep = get_locked_pte(poking_mm, poking_addr, &ptl); + + /* + * This must not fail; preallocated in poking_init(). + */ + VM_BUG_ON(!ptep); + + /* + * flush_tlb_mm_range() would be called when the poking_mm is not + * loaded. When PCID is in use, the flush would be deferred to the time + * the poking_mm is loaded again. Set the PTE as non-global to prevent + * it from being used when we are done. + */ + prot = __pgprot(pgprot_val(PAGE_KERNEL) & ~_PAGE_GLOBAL); + + pte = mk_pte(pages[0], prot); + set_pte_at(poking_mm, poking_addr, ptep, pte); + + if (cross_page_boundary) { + pte = mk_pte(pages[1], prot); + set_pte_at(poking_mm, poking_addr + PAGE_SIZE, ptep + 1, pte); + } + + /* + * Loading the temporary mm behaves as a compiler barrier, which + * guarantees that the PTE will be set at the time memcpy() is done. + */ + prev = use_temporary_mm(poking_mm); + + kasan_disable_current(); + memcpy((u8 *)poking_addr + offset_in_page(addr), opcode, len); + kasan_enable_current(); + + /* + * Ensure that the PTE is only cleared after the instructions of memcpy + * were issued by using a compiler barrier. + */ + barrier(); + + pte_clear(poking_mm, poking_addr, ptep); + if (cross_page_boundary) + pte_clear(poking_mm, poking_addr + PAGE_SIZE, ptep + 1); + + /* + * Loading the previous page-table hierarchy requires a serializing + * instruction that already allows the core to see the updated version. + * Xen-PV is assumed to serialize execution in a similar manner. + */ + unuse_temporary_mm(prev); + + /* + * Flushing the TLB might involve IPIs, which would require enabled + * IRQs, but not if the mm is not used, as it is in this point. + */ + flush_tlb_mm_range(poking_mm, poking_addr, poking_addr + + (cross_page_boundary ? 2 : 1) * PAGE_SIZE, + PAGE_SHIFT, false); + + pte_unmap_unlock(ptep, ptl); + /* + * If the text doesn't match what we just wrote; something is + * fundamentally screwy, there's nothing we can really do about that. + */ + BUG_ON(memcmp(addr, opcode, len)); + local_irq_restore(flags); return addr; } diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c index 0f4fe206dcc2..82b181fcefe5 100644 --- a/arch/x86/xen/mmu_pv.c +++ b/arch/x86/xen/mmu_pv.c @@ -2319,8 +2319,6 @@ static void xen_set_fixmap(unsigned idx, phys_addr_t phys, pgprot_t prot) #elif defined(CONFIG_X86_VSYSCALL_EMULATION) case VSYSCALL_PAGE: #endif - case FIX_TEXT_POKE0: - case FIX_TEXT_POKE1: /* All local page mappings */ pte = pfn_pte(phys, prot); break; From patchwork Tue Jan 29 00:34:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785237 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 953AE6C2 for ; Tue, 29 Jan 2019 00:41:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 857FF2B4DC for ; Tue, 29 Jan 2019 00:41:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 790902B4E2; Tue, 29 Jan 2019 00:41:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 027BD2B882 for ; Tue, 29 Jan 2019 00:41:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 02BEF8E0001; Mon, 28 Jan 2019 19:41:02 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id ECAA58E0008; Mon, 28 Jan 2019 19:41:01 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CCBD28E0012; Mon, 28 Jan 2019 19:41:01 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f198.google.com (mail-pl1-f198.google.com [209.85.214.198]) by kanga.kvack.org (Postfix) with ESMTP id 88E308E0008 for ; Mon, 28 Jan 2019 19:41:01 -0500 (EST) Received: by mail-pl1-f198.google.com with SMTP id v11so13049944ply.4 for ; Mon, 28 Jan 2019 16:41:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=H4uqmF5HLlBo9r9rzOTTevq+cJIiPJBHTeYeqiaw1PU=; b=PDutqypkMKekHFhUwPse1zt4sP9bINqKsmFkrZaYpwv0GlAUJ8JGKV5hw4pRhB0zZU e+kcVr+J0+6yF1y1sqqO6qZ8iUzeyxoDK775e4TustumcZVKgPidU0ovcXwwLnnZuVvO q0pKEtZvkHuLrnTmOVUOG4Zdml8nqal33QOVgGAuJGT2qylz7gBMFRTdyTZLf6jboAk5 XSOv8N5raUWyQOaDMBEOg1Ks5GVIRCKWohn/lzK8bXbCmattRQNaX3UrmpqFknWk6chk aL6V5/DZOTUOIsHN6yNk66DTG9+bguaCIftCnIGNqRuAe1nFsSCrYU2Jq8cYz5NrMN/u 97OQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukfM0zY+ntu/94PuTVCbZVjm1W0WS1XgGSovnm88uph9ktEaQ5LB P5t0gvT5dLr2/7CTRcWf70oaiHFrmbbQ+mnYOvGYjSRW+tRXTPbgYYLfoGxySu+FhyE9OAfcDs9 F7FME4hPfLvETa+7HbwTiHyorqsZWutpeTfJMqaFXHgjVU5wjeXVtgf9lat2edYuyhg== X-Received: by 2002:a62:670f:: with SMTP id b15mr23853617pfc.212.1548722461218; Mon, 28 Jan 2019 16:41:01 -0800 (PST) X-Google-Smtp-Source: ALg8bN5BOnTv4W/+n3CwgomL+oUr+6PCwswleo4+aAFSXsDXZW7jLo0YBwjs6Sbo5/vIII5G0LDA X-Received: by 2002:a62:670f:: with SMTP id b15mr23848064pfc.212.1548722353576; Mon, 28 Jan 2019 16:39:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722353; cv=none; d=google.com; s=arc-20160816; b=wBtHy558MKBl8yi448b8Ad37AzsVZAAyowEitWiQdSgoFbNs9a2D6ldXWEN6Qbnqqq Bx6EuDXdSB2jtkWkxrOlAbQH720pOcQek3/2H+/vQkWXLrmCZLaERYOMn6o0kb3iL30g TdM4ARLRUwGhJSuztBBLINHWmuQ5l/ylDTPAsJNMsF6n8mpxsZi08Q7AGUGRUCg19faO ICxEr7Icft6hrAqCXO7s2aIHalQOspjgEJ/9isgp0wSYrck3Sk5R2esuuDYcUNRZLSmu pkf92HIPL4JDxeFu29CQSIZoIy1P4cwzhKIr/COABXJezhOmcBJYJkbiLk2EM++uJzJW WFTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=H4uqmF5HLlBo9r9rzOTTevq+cJIiPJBHTeYeqiaw1PU=; b=vGE1jyCMgkDrMG1g6wN2fH35ysdVD55lOcaL18ApnSHDanB3Gqdd5gUX+4Wadag+i0 zDrn0g7Spo6P4tZRzopEGd/09NQn5d28pVU1W3fxTbwU8mXaAUBTQG1XYiLdMb/diZHP dB66qjUvivjLaQDHnvW7wUFcJrC54MXJg9BizuBbNceCmmI/SgOqxbW7uDp9UFw01AkN 42wAgus0J0ejysZFOA465bEFk8N6ZppxP18Ewq3zdyz3n3lIQVk2GHo4MYeqO644OIC3 nJq8KbQ7E0sLE/AMp70h71IqYV4b4/BKBcBQ6QAFN5mmxhcN6M5NKUN5Bo0jQzD0YRKw 4d4Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga04.intel.com (mga04.intel.com. [192.55.52.120]) by mx.google.com with ESMTPS id l7si33052569pfg.245.2019.01.28.16.39.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:13 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) client-ip=192.55.52.120; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921904" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:11 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Rick Edgecombe Subject: [PATCH v2 07/20] x86/kgdb: avoid redundant comparison of patched code Date: Mon, 28 Jan 2019 16:34:09 -0800 Message-Id: <20190129003422.9328-8-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit text_poke() already ensures that the written value is the correct one and fails if that is not the case. There is no need for an additional comparison. Remove it. Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/kgdb.c | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-) diff --git a/arch/x86/kernel/kgdb.c b/arch/x86/kernel/kgdb.c index 1461544cba8b..057af9187a04 100644 --- a/arch/x86/kernel/kgdb.c +++ b/arch/x86/kernel/kgdb.c @@ -746,7 +746,6 @@ void kgdb_arch_set_pc(struct pt_regs *regs, unsigned long ip) int kgdb_arch_set_breakpoint(struct kgdb_bkpt *bpt) { int err; - char opc[BREAK_INSTR_SIZE]; bpt->type = BP_BREAKPOINT; err = probe_kernel_read(bpt->saved_instr, (char *)bpt->bpt_addr, @@ -765,11 +764,6 @@ int kgdb_arch_set_breakpoint(struct kgdb_bkpt *bpt) return -EBUSY; text_poke_kgdb((void *)bpt->bpt_addr, arch_kgdb_ops.gdb_bpt_instr, BREAK_INSTR_SIZE); - err = probe_kernel_read(opc, (char *)bpt->bpt_addr, BREAK_INSTR_SIZE); - if (err) - return err; - if (memcmp(opc, arch_kgdb_ops.gdb_bpt_instr, BREAK_INSTR_SIZE)) - return -EINVAL; bpt->type = BP_POKE_BREAKPOINT; return err; @@ -777,9 +771,6 @@ int kgdb_arch_set_breakpoint(struct kgdb_bkpt *bpt) int kgdb_arch_remove_breakpoint(struct kgdb_bkpt *bpt) { - int err; - char opc[BREAK_INSTR_SIZE]; - if (bpt->type != BP_POKE_BREAKPOINT) goto knl_write; /* @@ -790,10 +781,7 @@ int kgdb_arch_remove_breakpoint(struct kgdb_bkpt *bpt) goto knl_write; text_poke_kgdb((void *)bpt->bpt_addr, bpt->saved_instr, BREAK_INSTR_SIZE); - err = probe_kernel_read(opc, (char *)bpt->bpt_addr, BREAK_INSTR_SIZE); - if (err || memcmp(opc, bpt->saved_instr, BREAK_INSTR_SIZE)) - goto knl_write; - return err; + return 0; knl_write: return probe_kernel_write((char *)bpt->bpt_addr, From patchwork Tue Jan 29 00:34:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785137 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD6D614E1 for ; Tue, 29 Jan 2019 00:39:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC95B2B4D3 for ; Tue, 29 Jan 2019 00:39:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C092F2B4DC; Tue, 29 Jan 2019 00:39:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 69AA02B4D3 for ; Tue, 29 Jan 2019 00:39:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B32A48E0009; Mon, 28 Jan 2019 19:39:15 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 818208E000E; Mon, 28 Jan 2019 19:39:15 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A37C8E0006; Mon, 28 Jan 2019 19:39:15 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) by kanga.kvack.org (Postfix) with ESMTP id E14EC8E0006 for ; Mon, 28 Jan 2019 19:39:14 -0500 (EST) Received: by mail-pg1-f199.google.com with SMTP id r13so12656765pgb.7 for ; Mon, 28 Jan 2019 16:39:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=Vw4Tc9/qTce/ps387hTBEGuO1Y4CjbTaT+4DjGZJWvU=; b=T5boXAyUMQEKpbIXOvfNnyOC884jzm7P9k8qm/vwSIriGX82anwDDkI556hviD7z3w GnfzLE1bxdEhBxzoBfB4JENPl/B84ouUKbGWMN3yCaq0pbX6QyzI1PxVdG53DUO1MQsB r8pNYAiAOF2kOZHo5Qf6YT/lMk82lQx8Za4+CuYe+GfMQRYoU6D4KNau9O9v9znpqQow GSbpL9T1A7u81rbzBN0NBj2ohht30Ph/VxSPDD6f+VqCtyMxGqm54+REEvRQGDIi9f/t 8RrhBBu7N1vEVR+yQ+XeTVSQWNitMdUF3NakzNSfrXukyEpcOmZpvjFARyfFmve4K9JH 0L+w== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUuke9q+5Pgx+QDTXT21kE+iW5XxzUuXCCVF6ESRpLzU8SrqChaI4y yw1cIu7JXQ0zWLbhKaGIJ4EPdxagCQVDlMuTwWKWph0pIFE6+WrUElk3dfEyn9VVZbqfOz7IwQs K+jtYMyq4l5DtaJLCmLbLm2YXYmoDHK2VSmHnF2Um+HCaoEgfqV7BpnVezkcTeD1TLQ== X-Received: by 2002:a17:902:bd46:: with SMTP id b6mr23794256plx.231.1548722354605; Mon, 28 Jan 2019 16:39:14 -0800 (PST) X-Google-Smtp-Source: ALg8bN7HcAIhcAgWkYG36PqXjWXi9LhFTeeznhI4+kjWdpIyeohcwpZTzf0EUwlxh27kmjdDeOYV X-Received: by 2002:a17:902:bd46:: with SMTP id b6mr23794223plx.231.1548722353858; Mon, 28 Jan 2019 16:39:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722353; cv=none; d=google.com; s=arc-20160816; b=eG5MIpsKDRmUbEb4wVEEhZE0EoG5HF7w4Odc6WZuwqJD5JPi5hw49HL/DrHJT5bI7p HbOThegSC+wVRlsQpVUw1KJoTo0gmHyIasvxfZdISk51PVG9AQDqiYliahuoEDCgM2NO cbkcDaJTqyOPkWwRzrWsa1+z0k0rZVUrxVvS060Z/Zj6xuu/OM1kPdPTtcWL1FJknwfI jMyMIDQv3h+hHh6qTOALL4DCfNQO0woewfMHroyYX7KrxD4UrU9Rx29lDjPQXV/Glx6V b6l5bu+7hzlvfiecULUs1PXXnQ617n4gHPOUk4/n+zrONwBG6AyL23UWWPQV6b94YnzX ZcrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=Vw4Tc9/qTce/ps387hTBEGuO1Y4CjbTaT+4DjGZJWvU=; b=dCJv8KXd4wEBvk0l1qdtnFVO5DCtmbWz/VqbH+m4HDGEUZcTSrSU2PTt5M6F2IQIFq LqysW+gvU1mdq47OIj+2yZaiDNOzc7l79Sp3CvoDm8iGsjPxrQkiLFCbzMg2p/9CcXT/ trlH0IH/q5pmmb9NRAdjM9TCaACX0TWdFJlPIu9kAqQSGji2cYSFul/wmBsYhYfZNvdF 7XGm1qyxBWPBNMOIxG/SGD2YfD07UHnAcOwth7e8uc5iAdd3/8rjntyivg4hS7jOAIho sBvEL9VPkVFO7iBinN6I56bc6df86wze9OTmBfbwBv6Od+5e6yDvN77ePwXNEpivB07G M20Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga04.intel.com (mga04.intel.com. [192.55.52.120]) by mx.google.com with ESMTPS id s17si4514712pgi.513.2019.01.28.16.39.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:13 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) client-ip=192.55.52.120; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921906" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:11 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Steven Rostedt , Rick Edgecombe Subject: [PATCH v2 08/20] x86/ftrace: set trampoline pages as executable Date: Mon, 28 Jan 2019 16:34:10 -0800 Message-Id: <20190129003422.9328-9-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit Since alloc_module() will not set the pages as executable soon, we need to do so for ftrace trampoline pages after they are allocated. For the time being, we do not change ftrace to use the text_poke() interface. As a result, ftrace breaks still breaks W^X. Cc: Steven Rostedt Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/ftrace.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index 8257a59704ae..13c8249b197f 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -742,6 +742,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) unsigned long end_offset; unsigned long op_offset; unsigned long offset; + unsigned long npages; unsigned long size; unsigned long retq; unsigned long *ptr; @@ -774,6 +775,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) return 0; *tramp_size = size + RET_SIZE + sizeof(void *); + npages = DIV_ROUND_UP(*tramp_size, PAGE_SIZE); /* Copy ftrace_caller onto the trampoline memory */ ret = probe_kernel_read(trampoline, (void *)start_offset, size); @@ -818,6 +820,12 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) /* ALLOC_TRAMP flags lets us know we created it */ ops->flags |= FTRACE_OPS_FL_ALLOC_TRAMP; + /* + * Module allocation needs to be completed by making the page + * executable. The page is still writable, which is a security hazard, + * but anyhow ftrace breaks W^X completely. + */ + set_memory_x((unsigned long)trampoline, npages); return (unsigned long)trampoline; fail: tramp_free(trampoline, *tramp_size); From patchwork Tue Jan 29 00:34:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785121 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9497E1874 for ; Tue, 29 Jan 2019 00:39:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 80E432AEB9 for ; Tue, 29 Jan 2019 00:39:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7414E2AFA9; Tue, 29 Jan 2019 00:39:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F22242B4D3 for ; Tue, 29 Jan 2019 00:39:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 825398E0001; Mon, 28 Jan 2019 19:39:14 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6DB408E0007; Mon, 28 Jan 2019 19:39:14 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 44B218E0003; Mon, 28 Jan 2019 19:39:14 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) by kanga.kvack.org (Postfix) with ESMTP id EC1378E0004 for ; Mon, 28 Jan 2019 19:39:13 -0500 (EST) Received: by mail-pg1-f197.google.com with SMTP id d71so12729829pgc.1 for ; Mon, 28 Jan 2019 16:39:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=OWv/u9lyuaHGR/jx5Ggc4M68NwrNRo7DfonKYg1/6LY=; b=GHT1SOnCwAtYwH25RqxuxjimhCDvSDnjUF/zPhcsfvtIuFLghmPG51SIQ4hVk4Y3cp 5Gt9lj8omFIdPPtPi8rN5yq8RR8pz5u0IYHZMQuCLzrCTzgElwpvgImGcQK6WfpR2hVD oRfieGf7U7SyCZFim0I0so3zpCN2FSEJTiRkLe+xhsMub6fc5chhOf2MHYMamVBNAESY SSodLMyxfMMVDVWADXoJA+a70/1S16bB1US9w0VtkfsuPMLRdlPinhjlwipmHDEPfJBP lg0bk7Ulc3E9Bhs6C37VKRZNkeIvVODZ+o2jgJ2GpS1DeCc2TXHE1+AsIxDzoHAbsgPV MqDg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukfkgn0vvH+WwOtcC4TtU/eMyVC0bmy0Ep6XYkxQGbmFvf8TBRdq jb3c/ZSCSiHmR80ynbT3WKsuBDlfk8qf/6qWr7B5vVQrPT0EwElxnKgolIVoCxNiRHEYLHnaQW5 940UygOOW1eC5SM3ycyvqKwCxo9H6Isa0CeagkyrUl+qyqEMjk5WM5t85J6zQIKp7bQ== X-Received: by 2002:a63:4f5e:: with SMTP id p30mr21728829pgl.71.1548722353605; Mon, 28 Jan 2019 16:39:13 -0800 (PST) X-Google-Smtp-Source: ALg8bN6lko/xhaZY9OYbdcZKFE56eITyhdxBCGKpnEIem9qX3uRZKonKTj2L5wECPwV2PQsBrczc X-Received: by 2002:a63:4f5e:: with SMTP id p30mr21728799pgl.71.1548722352789; Mon, 28 Jan 2019 16:39:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722352; cv=none; d=google.com; s=arc-20160816; b=NWHSSNxcDk42X3BOPc29GC9eluHP1qaZft108aGod7hmO7C+/XdNNYS3y0dj31jbWB PByd1Nxc1mW+NNp9uS04HlT90sjZBjiYjd+h6s2MekRH5RegbEzf0N6o2Z/UaZ/V+K5e AJZ3a+ibfEsPY64PKN3SEdqW/yJfOf/pDINlEOhGl6eEs3xWs+uMdljlhfuU+UA13Pz8 ww4LzQ34THfzN5BvWnWdCpvS+mM1QV48uOa6x+Em+aR8nNcw0cfqWe6B1PkysGwfMHPq Z1bPg2clv4vJv2MQz+GhemFnUGEsUDPMYZ1r8Ek9rvU0kXKssABa55ri2HtLPfHYShAQ 1REg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=OWv/u9lyuaHGR/jx5Ggc4M68NwrNRo7DfonKYg1/6LY=; b=kMiMYlyF4OoU2H2er+COlUxTzkH2G6fAQoOHCwuoBlTB0mTHWTIrv16pyolDto0j1j YariuUGvs1c9yP4yXgphuMwukel9CSLkZkM7V05pvTGdYxc3dDrhg/YxpNNuppNVVqRs zBzMd2u7SwjCYhmGvLSOQ+WIng4m9N7YdpvObtKqk+RVt6OAjZPAX7ITEHrrrE5T2UVd YNO0a2G3lFQzPvhUYRnTaBTEU9iCmAYa26BfQ+cNJ/lrwNb9If88VoYmQ/fCbTX0cLrA a3xE2OM81/fJvA43KPndWFMmecgOP5hF+iBI6TXINJ/CXqXgtqRu3YKleJkYO6QJDZ3m OHJw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga01.intel.com (mga01.intel.com. [192.55.52.88]) by mx.google.com with ESMTPS id i9si7660357plb.35.2019.01.28.16.39.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:12 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) client-ip=192.55.52.88; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921909" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:11 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Rick Edgecombe Subject: [PATCH v2 09/20] x86/kprobes: instruction pages initialization enhancements Date: Mon, 28 Jan 2019 16:34:11 -0800 Message-Id: <20190129003422.9328-10-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit Make kprobes instruction pages read-only (and executable) after they are set to prevent them from mistaken or malicious modifications. This is a preparatory patch for a following patch that makes module allocated pages non-executable and sets the page as executable after allocation. While at it, do some small cleanup of what appears to be unnecessary masking. Acked-by: Masami Hiramatsu Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/kprobes/core.c | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index 4ba75afba527..fac692e36833 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -431,8 +431,20 @@ void *alloc_insn_page(void) void *page; page = module_alloc(PAGE_SIZE); - if (page) - set_memory_ro((unsigned long)page & PAGE_MASK, 1); + if (page == NULL) + return NULL; + + /* + * First make the page read-only, and then only then make it executable + * to prevent it from being W+X in between. + */ + set_memory_ro((unsigned long)page, 1); + + /* + * TODO: Once additional kernel code protection mechanisms are set, ensure + * that the page was not maliciously altered and it is still zeroed. + */ + set_memory_x((unsigned long)page, 1); return page; } @@ -440,8 +452,12 @@ void *alloc_insn_page(void) /* Recover page to RW mode before releasing it */ void free_insn_page(void *page) { - set_memory_nx((unsigned long)page & PAGE_MASK, 1); - set_memory_rw((unsigned long)page & PAGE_MASK, 1); + /* + * First make the page non-executable, and then only then make it + * writable to prevent it from being W+X in between. + */ + set_memory_nx((unsigned long)page, 1); + set_memory_rw((unsigned long)page, 1); module_memfree(page); } From patchwork Tue Jan 29 00:34:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785143 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 41381159A for ; Tue, 29 Jan 2019 00:39:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2D2B32B4C4 for ; Tue, 29 Jan 2019 00:39:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 205B72B4D9; Tue, 29 Jan 2019 00:39:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7AE062B4C4 for ; Tue, 29 Jan 2019 00:39:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 42FBD8E000B; Mon, 28 Jan 2019 19:39:16 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1887C8E000C; Mon, 28 Jan 2019 19:39:15 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D1EF38E000C; Mon, 28 Jan 2019 19:39:15 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) by kanga.kvack.org (Postfix) with ESMTP id 4E1768E000F for ; Mon, 28 Jan 2019 19:39:15 -0500 (EST) Received: by mail-pf1-f199.google.com with SMTP id p15so15383154pfk.7 for ; Mon, 28 Jan 2019 16:39:15 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=RlvGVO5EXL5pU3ciPdRuGoj9s4OpHijn3MolXrSDLC0=; b=jGRFN9A6fnlKZJDpuWS/euEr83US6Zv3ThvepVthXqY6lg1PgLmsNHHGELfPZrEMh/ +ezbtrXVpXGBOtfqtXiq/TkjFbX+vf1tQLE1I9Fklg6vqIPAOJ6TLBC88LwoBQSWfPlN HK+h1h8at+ERBkfM9tEvAzR0fxJyA8rrIyehcHTqYbsVz4kxXqmV1IclV7QqO6s3hZ2u WLNNOlGkivDRbQICfCSpb//iyGybgNCmpIKS+N7cUAN/FPhtnxjHgmSPAadNpyQxZiOv twVOz/Yi3W6yo3KvAGEpd9Ld2z5Gs887+5/dDByIxr9Wfw95IU2EftjfxV0gOivK6scI TPQw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukely0L598Nq6WvXbzriruWq86/lSzp6LLIKwBPffZu4okOSJiPj iwgVXb4xP1qbbxge8Sp/CVtgog/KHUIRAxkMXc3zHr4eP2s840KMVPgQHkihIyWWjxhPhJoy6nJ 5lfm9WBD2KROLVo9EfAW46fHnMb1qZuaxTMRy/Uo5gu8gPfIBib2ciEqdUaOu4uW0Pw== X-Received: by 2002:a17:902:1005:: with SMTP id b5mr24022662pla.310.1548722354786; Mon, 28 Jan 2019 16:39:14 -0800 (PST) X-Google-Smtp-Source: ALg8bN4Y252LgC9cm2BhBxHTi6arM9vVyvvF17+8bXcE2bwJI3HNRkcO8xkQFWloU++hDvkkpoqP X-Received: by 2002:a17:902:1005:: with SMTP id b5mr24022617pla.310.1548722353875; Mon, 28 Jan 2019 16:39:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722353; cv=none; d=google.com; s=arc-20160816; b=r+8uPHtci6Lsr5JyCpXsUuj6Jj65uCTQ/A+7zW077b10bzJGu+Opmgts3Tptv6bErM IVnIUwh3EvuTTt+vsERc4iM6G462zkmL0TBejtl0ukouHX9EvGljQT3nfXO9HF4mZztG GmscEV9AVrzU5+noj9spJwemPyfF2pqlQAd0of83vGdUg1KwGeVtWK6fCscbiZcn2248 wCkAvzk6/O8QyqFIr1hmtUqkKl7TjUjJIxdiAIe+7KJgn3ThxOGWjs+s8w/zLv0GnJYr 76CuQNqjSIUxZd8K2vNolLsCGJcitUk35rATLwduPDL+dSWALlJBDg+1OMufCpDYWDFI Q0oQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=RlvGVO5EXL5pU3ciPdRuGoj9s4OpHijn3MolXrSDLC0=; b=xtze8N9Oqnb5ySjn0lzaPMJz1L2RGFHPHxbfFc63hUaGjqlHFEiZctw02ZKUg7moeN FtoIkj3TkqCJQMA2oUQ2VjyM/+b+qjOUNqcYG+E09+X8PvFLhk6LstnUTveFDVFdaLee y/O/wHchCC57aGShzWrnYTi6aFgvwnOQlU5Wi1AHTcy7eCv+JqaW+1fsAFdPKLE/LGq5 bwLJIefojI+yQZ1g64Me8lUC5f9mkXmlLJCN+Mz3Etu9Z/5OsymJt+O8Hv7ywqMMoAbp 9nEHAQ7jxZ4lmhNyj9+2E48yR1lD1fSmB4sxA1Ai0qCufLEcD3BVCiK5CATitRbwB478 R7bA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga04.intel.com (mga04.intel.com. [192.55.52.120]) by mx.google.com with ESMTPS id l7si33052569pfg.245.2019.01.28.16.39.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:13 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) client-ip=192.55.52.120; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921913" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:12 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Masami Hiramatsu , Rick Edgecombe Subject: [PATCH v2 10/20] x86: avoid W^X being broken during modules loading Date: Mon, 28 Jan 2019 16:34:12 -0800 Message-Id: <20190129003422.9328-11-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit When modules and BPF filters are loaded, there is a time window in which some memory is both writable and executable. An attacker that has already found another vulnerability (e.g., a dangling pointer) might be able to exploit this behavior to overwrite kernel code. Prevent having writable executable PTEs in this stage. In addition, avoiding having W+X mappings can also slightly simplify the patching of modules code on initialization (e.g., by alternatives and static-key), as would be done in the next patch. To avoid having W+X mappings, set them initially as RW (NX) and after they are set as RO set them as X as well. Setting them as executable is done as a separate step to avoid one core in which the old PTE is cached (hence writable), and another which sees the updated PTE (executable), which would break the W^X protection. Cc: Kees Cook Cc: Peter Zijlstra Cc: Dave Hansen Cc: Masami Hiramatsu Suggested-by: Thomas Gleixner Suggested-by: Andy Lutomirski Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/alternative.c | 28 +++++++++++++++++++++------- arch/x86/kernel/module.c | 2 +- include/linux/filter.h | 2 +- kernel/module.c | 5 +++++ 4 files changed, 28 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 76d482a2b716..69f3e650ada8 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -667,15 +667,29 @@ void __init alternative_instructions(void) * handlers seeing an inconsistent instruction while you patch. */ void *__init_or_module text_poke_early(void *addr, const void *opcode, - size_t len) + size_t len) { unsigned long flags; - local_irq_save(flags); - memcpy(addr, opcode, len); - local_irq_restore(flags); - sync_core(); - /* Could also do a CLFLUSH here to speed up CPU recovery; but - that causes hangs on some VIA CPUs. */ + + if (static_cpu_has(X86_FEATURE_NX) && + is_module_text_address((unsigned long)addr)) { + /* + * Modules text is marked initially as non-executable, so the + * code cannot be running and speculative code-fetches are + * prevented. We can just change the code. + */ + memcpy(addr, opcode, len); + } else { + local_irq_save(flags); + memcpy(addr, opcode, len); + local_irq_restore(flags); + sync_core(); + + /* + * Could also do a CLFLUSH here to speed up CPU recovery; but + * that causes hangs on some VIA CPUs. + */ + } return addr; } diff --git a/arch/x86/kernel/module.c b/arch/x86/kernel/module.c index b052e883dd8c..cfa3106faee4 100644 --- a/arch/x86/kernel/module.c +++ b/arch/x86/kernel/module.c @@ -87,7 +87,7 @@ void *module_alloc(unsigned long size) p = __vmalloc_node_range(size, MODULE_ALIGN, MODULES_VADDR + get_module_load_offset(), MODULES_END, GFP_KERNEL, - PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE, + PAGE_KERNEL, 0, NUMA_NO_NODE, __builtin_return_address(0)); if (p && (kasan_module_alloc(p, size) < 0)) { vfree(p); diff --git a/include/linux/filter.h b/include/linux/filter.h index d531d4250bff..9cdfab7f383c 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -681,7 +681,6 @@ bpf_ctx_narrow_access_ok(u32 off, u32 size, u32 size_default) static inline void bpf_prog_lock_ro(struct bpf_prog *fp) { - fp->undo_set_mem = 1; set_memory_ro((unsigned long)fp, fp->pages); } @@ -694,6 +693,7 @@ static inline void bpf_prog_unlock_ro(struct bpf_prog *fp) static inline void bpf_jit_binary_lock_ro(struct bpf_binary_header *hdr) { set_memory_ro((unsigned long)hdr, hdr->pages); + set_memory_x((unsigned long)hdr, hdr->pages); } static inline void bpf_jit_binary_unlock_ro(struct bpf_binary_header *hdr) diff --git a/kernel/module.c b/kernel/module.c index 2ad1b5239910..ae1b77da6a20 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -1950,8 +1950,13 @@ void module_enable_ro(const struct module *mod, bool after_init) return; frob_text(&mod->core_layout, set_memory_ro); + frob_text(&mod->core_layout, set_memory_x); + frob_rodata(&mod->core_layout, set_memory_ro); + frob_text(&mod->init_layout, set_memory_ro); + frob_text(&mod->init_layout, set_memory_x); + frob_rodata(&mod->init_layout, set_memory_ro); if (after_init) From patchwork Tue Jan 29 00:34:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785133 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 49A4A6C2 for ; Tue, 29 Jan 2019 00:39:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 34BB82B4C4 for ; Tue, 29 Jan 2019 00:39:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 26FA52B4D9; Tue, 29 Jan 2019 00:39:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9F9812B4C4 for ; Tue, 29 Jan 2019 00:39:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5A6A28E000A; Mon, 28 Jan 2019 19:39:15 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4BC848E000B; Mon, 28 Jan 2019 19:39:15 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 135998E000C; Mon, 28 Jan 2019 19:39:15 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) by kanga.kvack.org (Postfix) with ESMTP id AC13C8E000A for ; Mon, 28 Jan 2019 19:39:14 -0500 (EST) Received: by mail-pg1-f199.google.com with SMTP id i124so12724307pgc.2 for ; Mon, 28 Jan 2019 16:39:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=zkYOBKDkCPssvs3YCV5eJaPkZC0ctM/7YsEXK+Vqdbc=; b=CCb1qXMxyTyl6zBqkpEEmjiYo4iYtW3QOvFEGdqmaM0kk3dYnN1wa+USGK/3B2PXY8 CNlvSJQufTgvoNvpED3AO8kW4ph1KvCXqcurvlnDfDboTRS+ZRTlpIC9Etrr63xDW7fF yHZMUZIsnpvjbcJQB0Gzixi74tLcvt/U+TAbSP2oXyxqxyplkQmmcaGgjM7jprrlQaVv j2R+/OYROQavrVqniwG7MHPTQa2YUBeQGxov6h+puBTfIRQW6g3JKQSyqgfbruqWFdQH 8aa1+u9HJ7Tcr5BulGQ7OKyEjQDOMTVDJ7sleDhAsLd2Xz5NRThrQ38+d5obv5IOB1Bf Qynw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukc7c7Cm5nVt9dFyQh7af792xauOvKYOHmkmEoVXcb+pd8WDZgS/ HuBuxKvxZCcMO4KVSa1SJs6CZnlsMuXC4VQSeYO8uBYJ/5znaQxL/pasQuPzZAn31yHhSxhd2CX JEHPyBKN9gdeCTXLpm01vVUDQOjfRaavrEg5qQ7IUKU6rmf0RS3v+633KzKEy6LiWkw== X-Received: by 2002:a17:902:6948:: with SMTP id k8mr23500667plt.2.1548722354367; Mon, 28 Jan 2019 16:39:14 -0800 (PST) X-Google-Smtp-Source: ALg8bN7OjLVAb4RNNRV/Wd68Y59voUQ/HDAkgVEm3CFxZHJJrvTnITDAaCqt2Ep40+Lk4orFhTUF X-Received: by 2002:a17:902:6948:: with SMTP id k8mr23500597plt.2.1548722353037; Mon, 28 Jan 2019 16:39:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722353; cv=none; d=google.com; s=arc-20160816; b=ZsTky7MfSxqt8yGie4VcTem4atDhtGkdgAyKexUFFD5SFmSOZZdBnNSUtuAV877oZD Mq8Xe5w2Zm1u6hvrsNixRGy0tpxxw2/nF/EmunTqc6MNA1/RthS12FsefJWj5CmiVVzo gqXasINC3bObE2+1n+8DyESq4Bbzun2rpjsbxLf7G3opa5W17cQzDIewpqrj58UPnmUk SpsyCOeVABLSRcegc10mdcOOoSahwXve2J4FfQ2SkQ2bKC6VhW9amBoh/PHkni1p+X8q Lctdg9AQ54LCXdpmkCizf+a+KWC8ylJuO60/Tn75VgRh84Gtlo6cbbdTze9zP3Gx+6Gd 463g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=zkYOBKDkCPssvs3YCV5eJaPkZC0ctM/7YsEXK+Vqdbc=; b=PUt/uX4p462OW8X/eAEg3iMFxWxR7fcObclJ3XVGqWKchpff8sQBy00yCi1WP4BbKr bkiajBuZlvhvWx9FoWBWAIFegco6UQxEUPiIcNADXylQXk/J3kjDuWw3rAdH5mZQ3nZW FlwUQeyNYC2iIIIwkmoevKNkz1unkzN9bDoDirNg9BK3FGmGXFTayXG23r+td5qYcRTx uzo9nq0EU4tXq8d4w5M+BIjUjvZvgBzFy7aUd0eWcvCIN/Rag5gRcfb4a+naaHPO8CSc 3f7/lRrkXIzIFE8hLHD9nIDHq7uL04O9cB21kJhVgGq4yskI4AuVzKe5RUzRsgrCUCZ4 whNw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga01.intel.com (mga01.intel.com. [192.55.52.88]) by mx.google.com with ESMTPS id i9si7660357plb.35.2019.01.28.16.39.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:13 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) client-ip=192.55.52.88; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921915" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:12 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Masami Hiramatsu , Rick Edgecombe Subject: [PATCH v2 11/20] x86/jump-label: remove support for custom poker Date: Mon, 28 Jan 2019 16:34:13 -0800 Message-Id: <20190129003422.9328-12-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit There are only two types of poking: early and breakpoint based. The use of a function pointer to perform poking complicates the code and is probably inefficient due to the use of indirect branches. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Cc: Masami Hiramatsu Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/jump_label.c | 24 ++++++++---------------- 1 file changed, 8 insertions(+), 16 deletions(-) diff --git a/arch/x86/kernel/jump_label.c b/arch/x86/kernel/jump_label.c index e36cfec0f35e..427facef8aff 100644 --- a/arch/x86/kernel/jump_label.c +++ b/arch/x86/kernel/jump_label.c @@ -37,7 +37,6 @@ static void bug_at(unsigned char *ip, int line) static void __ref __jump_label_transform(struct jump_entry *entry, enum jump_label_type type, - void *(*poker)(void *, const void *, size_t), int init) { union jump_code_union jmp; @@ -50,14 +49,6 @@ static void __ref __jump_label_transform(struct jump_entry *entry, jmp.offset = jump_entry_target(entry) - (jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE); - /* - * As long as we're UP and not yet marked RO, we can use - * text_poke_early; SYSTEM_BOOTING guarantees both, as we switch to - * SYSTEM_SCHEDULING before going either. - */ - if (system_state == SYSTEM_BOOTING) - poker = text_poke_early; - if (type == JUMP_LABEL_JMP) { if (init) { expect = default_nop; line = __LINE__; @@ -80,16 +71,17 @@ static void __ref __jump_label_transform(struct jump_entry *entry, bug_at((void *)jump_entry_code(entry), line); /* - * Make text_poke_bp() a default fallback poker. + * As long as we're UP and not yet marked RO, we can use + * text_poke_early; SYSTEM_BOOTING guarantees both, as we switch to + * SYSTEM_SCHEDULING before going either. * * At the time the change is being done, just ignore whether we * are doing nop -> jump or jump -> nop transition, and assume * always nop being the 'currently valid' instruction - * */ - if (poker) { - (*poker)((void *)jump_entry_code(entry), code, - JUMP_LABEL_NOP_SIZE); + if (init || system_state == SYSTEM_BOOTING) { + text_poke_early((void *)jump_entry_code(entry), code, + JUMP_LABEL_NOP_SIZE); return; } @@ -101,7 +93,7 @@ void arch_jump_label_transform(struct jump_entry *entry, enum jump_label_type type) { mutex_lock(&text_mutex); - __jump_label_transform(entry, type, NULL, 0); + __jump_label_transform(entry, type, 0); mutex_unlock(&text_mutex); } @@ -131,5 +123,5 @@ __init_or_module void arch_jump_label_transform_static(struct jump_entry *entry, jlstate = JL_STATE_NO_UPDATE; } if (jlstate == JL_STATE_UPDATE) - __jump_label_transform(entry, type, text_poke_early, 1); + __jump_label_transform(entry, type, 1); } From patchwork Tue Jan 29 00:34:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785223 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CEE8D6C2 for ; Tue, 29 Jan 2019 00:40:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BB7862B80B for ; Tue, 29 Jan 2019 00:40:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B92D42B855; Tue, 29 Jan 2019 00:40:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4AEBC2B83C for ; Tue, 29 Jan 2019 00:40:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 556068E0002; Mon, 28 Jan 2019 19:40:46 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 504D78E0001; Mon, 28 Jan 2019 19:40:46 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41B9E8E0002; Mon, 28 Jan 2019 19:40:46 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) by kanga.kvack.org (Postfix) with ESMTP id F20178E0001 for ; Mon, 28 Jan 2019 19:40:45 -0500 (EST) Received: by mail-pg1-f199.google.com with SMTP id d3so12640062pgv.23 for ; Mon, 28 Jan 2019 16:40:45 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=9618xtgXUZ8Ek1wm3UNb4fJDvCUikNkBQc6KAG864pg=; b=PqUchcSkoHXgLa2F6rSIUQLXQoKo+P/ycu5z24LeowLrsel6rUwn06Y0R48oT3Dh1T ELqr1eO/0zPKbUMaD5soBBYQfJkOrlLLIGVHI8v787EuFvvxlg3VOd7hjJXzxlTursJa BNY2POKduL7Fj33mbvjyEZNAxjUt5iAhrjz9wRsrG32iX8Ws2P1nE8M5H9DB0vhNIVGb noz0ymBVxd+ux/avvT4qIsJmtO9UKzG2mPs7Li0kU3FPupr6ti/ljXf+mT+t7sYSSKXa IjzFjJg9Q7ntHenbdgL364TNMYxHjRbRj3DiAAdoB+ycCh3dXcZlHZ1BRQKt/yi4mvN5 1BVA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukfj5IfG8v2xdW96BS3Sp/y4WHdRsmyP8xU8W4f+gWFbOhwINo5W pWxJQQCDaGJJMuxBk3/FWigDORvEhAbrM/c6+yb2RifhLAPTEzviEPAqeGQRrjov/3OuPzsccsx Hkuwz2UTNIeqwlLcBz19fOSpeRN+pQdIrOYVpNUumAf1WPeFBhGHV1CWhDG3WK9V77A== X-Received: by 2002:a62:9f1b:: with SMTP id g27mr23311351pfe.87.1548722445642; Mon, 28 Jan 2019 16:40:45 -0800 (PST) X-Google-Smtp-Source: ALg8bN5iI6amSwTGwJbLBEOH1oe08f84a01Z1ups8g49tB4v8QSP79plwyyao2jCgURN5W1eDvWm X-Received: by 2002:a62:9f1b:: with SMTP id g27mr23306610pfe.87.1548722353321; Mon, 28 Jan 2019 16:39:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722353; cv=none; d=google.com; s=arc-20160816; b=cRvVn2QOHrVfduEpwC8o4zvTF3fGigPAfWFQBTdsfq56l3hp91QFerkop9qnEKh0c2 ZqJYS1nVvtIyyKhflZWYWI6V/fOMM7imh1y+XwGyelKv8BJvLKbBd78+Tni0H6xDleNR fGvqJYhWrgXqQNjW3CkyAvy/Jd003IPmSkRTixVLXC+7MQKnpmrwY76nMFsGGn8DUJ7S dMbG2AiByP2IYIgCldkQePpx/pR8ieSQgullWvvM2wpkJPoI+lB9ea6nwHKbU0mI9COk xPDHOSvlAdvzJcbfOxYI0Mcxf9A+5Pdvngs2gmts40D/V0eeGbIl+quqzQCpXnluhkEN qh9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=9618xtgXUZ8Ek1wm3UNb4fJDvCUikNkBQc6KAG864pg=; b=xTaZmmL4A/gOybMIKTDWf4fxl2MrjNyrSIYEFYqeAdEjtalI/jZBgCCvvD3kmTCS7x EzRzTAhJ+8dCF+3rjqlbEFWDTVeMyanwNGOj8rQe9dwtl/Pn6jBE2rpTzxWk0aFhf8UE 2BEaAKHa54SneqxRkCmXywo1UeOVfLMz/pZYmOb8cCU5RRYEafni+QbyuIYxJzpHX/0/ w50OqRtk2Qvi1FcmOTZ5TtS65i3scTMZ5g7n1E81Dch2s3J4UnCz3FN7ouJ0J2AiEne6 VCdUMA0fIs9Yz8qTb5mdHyi9pn+WcTcZ3L9PxJC96QNEMklqRephbOfeQ6F5RCQHEl4W Anng== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga01.intel.com (mga01.intel.com. [192.55.52.88]) by mx.google.com with ESMTPS id i9si7660357plb.35.2019.01.28.16.39.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:13 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) client-ip=192.55.52.88; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921919" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:12 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Masami Hiramatsu , Rick Edgecombe Subject: [PATCH v2 12/20] x86/alternative: Remove the return value of text_poke_*() Date: Mon, 28 Jan 2019 16:34:14 -0800 Message-Id: <20190129003422.9328-13-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit The return value of text_poke_early() and text_poke_bp() is useless. Remove it. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Cc: Masami Hiramatsu Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/text-patching.h | 4 ++-- arch/x86/kernel/alternative.c | 11 ++++------- 2 files changed, 6 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/text-patching.h index a75eed841eed..c90678fd391a 100644 --- a/arch/x86/include/asm/text-patching.h +++ b/arch/x86/include/asm/text-patching.h @@ -18,7 +18,7 @@ static inline void apply_paravirt(struct paravirt_patch_site *start, #define __parainstructions_end NULL #endif -extern void *text_poke_early(void *addr, const void *opcode, size_t len); +extern void text_poke_early(void *addr, const void *opcode, size_t len); /* * Clear and restore the kernel write-protection flag on the local CPU. @@ -37,7 +37,7 @@ extern void *text_poke_early(void *addr, const void *opcode, size_t len); extern void *text_poke(void *addr, const void *opcode, size_t len); extern void *text_poke_kgdb(void *addr, const void *opcode, size_t len); extern int poke_int3_handler(struct pt_regs *regs); -extern void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler); +extern void text_poke_bp(void *addr, const void *opcode, size_t len, void *handler); extern int after_bootmem; extern __ro_after_init struct mm_struct *poking_mm; extern __ro_after_init unsigned long poking_addr; diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 69f3e650ada8..81876e3ef3fd 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -264,7 +264,7 @@ static void __init_or_module add_nops(void *insns, unsigned int len) extern struct alt_instr __alt_instructions[], __alt_instructions_end[]; extern s32 __smp_locks[], __smp_locks_end[]; -void *text_poke_early(void *addr, const void *opcode, size_t len); +void text_poke_early(void *addr, const void *opcode, size_t len); /* * Are we looking at a near JMP with a 1 or 4-byte displacement. @@ -666,8 +666,8 @@ void __init alternative_instructions(void) * instructions. And on the local CPU you need to be protected again NMI or MCE * handlers seeing an inconsistent instruction while you patch. */ -void *__init_or_module text_poke_early(void *addr, const void *opcode, - size_t len) +void __init_or_module text_poke_early(void *addr, const void *opcode, + size_t len) { unsigned long flags; @@ -690,7 +690,6 @@ void *__init_or_module text_poke_early(void *addr, const void *opcode, * that causes hangs on some VIA CPUs. */ } - return addr; } __ro_after_init struct mm_struct *poking_mm; @@ -890,7 +889,7 @@ int poke_int3_handler(struct pt_regs *regs) * replacing opcode * - sync cores */ -void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler) +void text_poke_bp(void *addr, const void *opcode, size_t len, void *handler) { unsigned char int3 = 0xcc; @@ -932,7 +931,5 @@ void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler) * the writing of the new instruction. */ bp_patching_in_progress = false; - - return addr; } From patchwork Tue Jan 29 00:34:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785225 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9BD736C2 for ; Tue, 29 Jan 2019 00:40:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8C34F2B870 for ; Tue, 29 Jan 2019 00:40:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 803582B819; Tue, 29 Jan 2019 00:40:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E903D2B872 for ; Tue, 29 Jan 2019 00:40:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BE9698E0004; Mon, 28 Jan 2019 19:40:48 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B6E778E0001; Mon, 28 Jan 2019 19:40:48 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A86318E0004; Mon, 28 Jan 2019 19:40:48 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) by kanga.kvack.org (Postfix) with ESMTP id 659508E0001 for ; Mon, 28 Jan 2019 19:40:48 -0500 (EST) Received: by mail-pg1-f197.google.com with SMTP id p4so12659084pgj.21 for ; Mon, 28 Jan 2019 16:40:48 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=Y027RJiR9dYgsRTHYJcUS1hjFZKi3qjEOVY152/wOMs=; b=rMvBrZke5ALQI4QQFV8ThUfwAm2mZ18cPVJX2ybXKdwLaFLqftaYZWt9AOoco4vGKC JQQWRYeg7z8IvFOnquYyD1USnuAvoIw+g3B7FycingnvGm2B/ZFuZfAeLo6Lmaxwv4x/ K7qyd395J8hdcrbbZUSfJD1KsTCco889u8MfhibpzNlGsm2VAydzov07w7zRgBsv0qUR eQq4h7KOageaI2HAj1GdKJcHRF7I9ePB4S+myDh1mHW+XOTGFwXp7HEo/ESyiqfyPqwJ JLXEXwU08ankGA3fpn11fVSIV3jjRvx8SkAXwCbOpIbrkynCFtgSGc9UZM4IId2HbRFn k33Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukfSy0h3Tys8UQ5u9kJQFaX1Nj8xFJKxZwapl0koAYrMwuE3FAco k058IeKoUQeqBypRCdnfDjD6xcEq77/6r8Tlz43BQhRV9QbMu4OyTmYSnAAb0qZwDSaokL5hclW 3GNCjS4Z9pa7LQRpI2qgFJalARZ1DIMvMkqaDdb8gdzmBiuH2pBiNU227jQq1x0jcCw== X-Received: by 2002:a63:304:: with SMTP id 4mr19893574pgd.99.1548722448059; Mon, 28 Jan 2019 16:40:48 -0800 (PST) X-Google-Smtp-Source: ALg8bN48o1HRcT06+m4qjR4g8R7v72g9wmAQ1oE/wRNA67RCPxIc0M7bV2Vh5vVHmZkJlDxUEudv X-Received: by 2002:a63:304:: with SMTP id 4mr19889583pgd.99.1548722353604; Mon, 28 Jan 2019 16:39:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722353; cv=none; d=google.com; s=arc-20160816; b=xdiZl/wv9/0dun/O7MhDBAwEC7V7SEtZB9N9ZQD29Afz8RB20pNI2+LtxGXsC+QlSy 9hzCzBSNtE4IRXR02ujxjq9a4W/jBY10jWoY5JeGdSkn2t3VsGJCTwJpKmjobINVK8ej ab4KaInPf0xbeuOxh1Ycns9vHAslNOXTnz+WZYxobd0u1wGzi9rD15jMi7JODXfSV6eI CqjXkAHlF6EGX0sO8HZqKdKc1O/PGFCI1AbnPBS67yoacUV2IGe8N0dB0bvcszCgfYJK fK3XhYrIOzlXYWdv8lRUIPqQeOqra+JBEVjQLAiJ0Wxv+6iEekLeZUBnXSf4yyG93r/o A8iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=Y027RJiR9dYgsRTHYJcUS1hjFZKi3qjEOVY152/wOMs=; b=hzP/K5vbjWDi3Ab9z1/Jy4f/ztilNQknAvbaCFIs4b9E1rY4LUx2XiOuq7iXkag1fI pxQd7N4b9caZGlqzJfvnH1EDxlukSIWI89bj8x+h+3wHc0619wnKZtC5ABWebjoCml1g HxjkSQmHo737OKemTKOS8/mZCxr9Hl2qDn1CTZ23nqt0gcyRqUQQkgYMF12ZOb+WU+Lt LBgUEAPKC9xiZSzGuuAYKmbC9lD8B5VbwiDsmrlK2grGaM7UAWu0inJXuQQre+qcAlQx fsy0IS3ivHLnZMsCsdXyaRVaLKLyFDxm7DsBVco3Jr5+ORXNGUeIaHQf6pgxFKaRiwAP 5c3Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga01.intel.com (mga01.intel.com. [192.55.52.88]) by mx.google.com with ESMTPS id i9si7660357plb.35.2019.01.28.16.39.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:13 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) client-ip=192.55.52.88; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921922" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:12 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Rick Edgecombe Subject: [PATCH v2 13/20] Add set_alias_ function and x86 implementation Date: Mon, 28 Jan 2019 16:34:15 -0800 Message-Id: <20190129003422.9328-14-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This adds two new functions set_alias_default_noflush and set_alias_nv_noflush for setting the alias mapping for the page to its default valid permissions and to an invalid state that cannot be cached in a TLB, respectively. These functions to not flush the TLB. Note, __kernel_map_pages does something similar but flushes the TLB and doesn't reset the permission bits to default on all architectures. There is also an ARCH config ARCH_HAS_SET_ALIAS for specifying whether these have an actual implementation or a default empty one. Cc: Dave Hansen Cc: Andy Lutomirski Cc: Peter Zijlstra Signed-off-by: Rick Edgecombe --- arch/Kconfig | 4 ++++ arch/x86/Kconfig | 1 + arch/x86/include/asm/set_memory.h | 3 +++ arch/x86/mm/pageattr.c | 14 +++++++++++--- include/linux/set_memory.h | 10 ++++++++++ 5 files changed, 29 insertions(+), 3 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 4cfb6de48f79..4ef9db190f2d 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -249,6 +249,10 @@ config ARCH_HAS_FORTIFY_SOURCE config ARCH_HAS_SET_MEMORY bool +# Select if arch has all set_alias_nv/default() functions +config ARCH_HAS_SET_ALIAS + bool + # Select if arch init_task must go in the __init_task_data section config ARCH_TASK_STRUCT_ON_STACK bool diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 26387c7bf305..42bb1df4ea94 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -66,6 +66,7 @@ config X86 select ARCH_HAS_UACCESS_FLUSHCACHE if X86_64 select ARCH_HAS_UACCESS_MCSAFE if X86_64 && X86_MCE select ARCH_HAS_SET_MEMORY + select ARCH_HAS_SET_ALIAS select ARCH_HAS_STRICT_KERNEL_RWX select ARCH_HAS_STRICT_MODULE_RWX select ARCH_HAS_SYNC_CORE_BEFORE_USERMODE diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_memory.h index 07a25753e85c..2ef4e4222df1 100644 --- a/arch/x86/include/asm/set_memory.h +++ b/arch/x86/include/asm/set_memory.h @@ -85,6 +85,9 @@ int set_pages_nx(struct page *page, int numpages); int set_pages_ro(struct page *page, int numpages); int set_pages_rw(struct page *page, int numpages); +int set_alias_nv_noflush(struct page *page); +int set_alias_default_noflush(struct page *page); + extern int kernel_set_to_readonly; void set_kernel_text_rw(void); void set_kernel_text_ro(void); diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 4f8972311a77..3a51915a1410 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -2209,8 +2209,6 @@ int set_pages_rw(struct page *page, int numpages) return set_memory_rw(addr, numpages); } -#ifdef CONFIG_DEBUG_PAGEALLOC - static int __set_pages_p(struct page *page, int numpages) { unsigned long tempaddr = (unsigned long) page_address(page); @@ -2249,6 +2247,17 @@ static int __set_pages_np(struct page *page, int numpages) return __change_page_attr_set_clr(&cpa, 0); } +int set_alias_nv_noflush(struct page *page) +{ + return __set_pages_np(page, 1); +} + +int set_alias_default_noflush(struct page *page) +{ + return __set_pages_p(page, 1); +} + +#ifdef CONFIG_DEBUG_PAGEALLOC void __kernel_map_pages(struct page *page, int numpages, int enable) { if (PageHighMem(page)) @@ -2282,7 +2291,6 @@ void __kernel_map_pages(struct page *page, int numpages, int enable) } #ifdef CONFIG_HIBERNATION - bool kernel_page_present(struct page *page) { unsigned int level; diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h index 2a986d282a97..d19481ac6a8f 100644 --- a/include/linux/set_memory.h +++ b/include/linux/set_memory.h @@ -10,6 +10,16 @@ #ifdef CONFIG_ARCH_HAS_SET_MEMORY #include +#ifndef CONFIG_ARCH_HAS_SET_ALIAS +static inline int set_alias_nv_noflush(struct page *page) +{ + return 0; +} +static inline int set_alias_default_noflush(struct page *page) +{ + return 0; +} +#endif #else static inline int set_memory_ro(unsigned long addr, int numpages) { return 0; } static inline int set_memory_rw(unsigned long addr, int numpages) { return 0; } From patchwork Tue Jan 29 00:34:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785139 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD18E6C2 for ; Tue, 29 Jan 2019 00:39:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CA5892B4D3 for ; Tue, 29 Jan 2019 00:39:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BE4432B4E2; Tue, 29 Jan 2019 00:39:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CE83F2B4D3 for ; Tue, 29 Jan 2019 00:39:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DA0168E000E; Mon, 28 Jan 2019 19:39:15 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D45648E0012; Mon, 28 Jan 2019 19:39:15 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A88AF8E0011; Mon, 28 Jan 2019 19:39:15 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by kanga.kvack.org (Postfix) with ESMTP id 463238E0009 for ; Mon, 28 Jan 2019 19:39:15 -0500 (EST) Received: by mail-pf1-f197.google.com with SMTP id p9so15440437pfj.3 for ; Mon, 28 Jan 2019 16:39:15 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=Yncdr+ZUNpa2Uv2D6DxE8u2f4VXkTONRNt4fbXj4dXE=; b=tUYV7V6euKCofYFdFpkXbbRP1l6qpUHdW/VJ4pusvTBhQcSmMafquL7FSUIaHYgnyo +5gduKz+prQMA83bY2y55UkVCfDJtOuXKgUkUrePooaRrAHgW72uTMtMAO9fADsdpJMa wGgR8q0+jbB88hEhylZSph69sz9sID4dx6XeLnQ8gP8K9MFZ2idOh+A8ypTB2FQOkqVa b6gpAKLeSQcTEP7WJT+XHJMA8+e4C5uqnkD+MvibFkVsI0Rh18tioXSXVZZL66wToZr3 tD9wGHQXVBWYBdzrW58wBJCSkxrpXGUJHvVgXi4MWZTVW7K6fD3DfpCQ2fkCbXvymG5D aumg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukfXQP+RZCvpfqn/34E2jXqTu97rYbHLV1Ygxii5yyMUPH4EXHoN 68HQ54rXaUYLvuBrLEApGW7qEvg+mQ7np8ySNNsWKly1yBPyP/yGzkC7gXkzOSYxyJiBc3/bAaN eNoOuz0uw07f7dzM0QWSFvQYp3maWaU1LSJzMlK4vZixY3hkshcBPrR/b7pbf7F1M6w== X-Received: by 2002:a63:4e41:: with SMTP id o1mr22288089pgl.282.1548722354914; Mon, 28 Jan 2019 16:39:14 -0800 (PST) X-Google-Smtp-Source: ALg8bN6ffbGCGAXsR7TwLxUOYY4bVKEWl+2olHuNXdgnkLMv2ln8/1zCOJn/ZOBMFG3SeT/52LiY X-Received: by 2002:a63:4e41:: with SMTP id o1mr22288054pgl.282.1548722354139; Mon, 28 Jan 2019 16:39:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722354; cv=none; d=google.com; s=arc-20160816; b=On4b4Pubelru3gz7XftceKtmR2qn7bP4TZ1uuAPnBWsnQ5pAMJT99VnaWfD5shdAZ5 4V95B1AbU8ThmOjZplAVCBwBQE2nhO0aUX/3dGLPN8MEx2W5vClukeatWAK+ew0dagjN VMEwSVHvXmU6dGpaLNXiql/Vu8sqLG0cpUJdmmNVMqZLbkce3ZvXjTeTDBjx7TOV00MK DBMf+cMUDARVh+xFKJcfyCW+B5Qk6bYeAVhwJncA49TtbyVx9oFY97RRlmOjjoZd6183 xoOPv+PO8H9vyX+RbIFSOgjFDC8U4O1BTQP46REsqLnBneZb6WWYkiD8da51YBkGIWTp vdkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=Yncdr+ZUNpa2Uv2D6DxE8u2f4VXkTONRNt4fbXj4dXE=; b=aH6nckAEizc3aI4gpnrikSLRw6k1s91bpEMAem9i4N/OTU9dAICxW86DXJbsj+j8r8 jvWRgiNgbp5BzkAgqL66KmOYAVz8Xo0fk10KqdPFjzgh+T9AfGPJtjHCGH+jisyjaw5+ NdOjWuaXfA6xUqQyVWrIX8BPcIGlSkfNDVvG0kpznnd9cNRLl1SwacqqIulUWGz9MxRW 1kTy5w6BMUCvzP3g8roqV7jDIN66DMcYu8cE71PCF4wpxh8u1Ks9x6hgCdbSIHPa8uGq gMA+s+ICBgjwnDH0LdIYiYJuP0fenLaHB8FmFue1UAHHMADKPi8XQB23ignb1hNwT0Iw h9mg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga04.intel.com (mga04.intel.com. [192.55.52.120]) by mx.google.com with ESMTPS id s17si4514712pgi.513.2019.01.28.16.39.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:14 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) client-ip=192.55.52.120; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921926" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:12 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Rick Edgecombe , "Rafael J. Wysocki" , Pavel Machek Subject: [PATCH v2 14/20] mm: Make hibernate handle unmapped pages Date: Mon, 28 Jan 2019 16:34:16 -0800 Message-Id: <20190129003422.9328-15-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP For architectures with CONFIG_ARCH_HAS_SET_ALIAS, pages can be unmapped briefly on the directmap, even when CONFIG_DEBUG_PAGEALLOC is not configured. So this changes kernel_map_pages and kernel_page_present to be defined when CONFIG_ARCH_HAS_SET_ALIAS is defined as well. It also changes places (page_alloc.c) where those functions are assumed to only be implemented when CONFIG_DEBUG_PAGEALLOC is defined. So now when CONFIG_ARCH_HAS_SET_ALIAS=y, hibernate will handle not present page when saving. Previously this was already done when CONFIG_DEBUG_PAGEALLOC was configured. It does not appear to have a big hibernating performance impact. Before: [ 4.670938] PM: Wrote 171996 kbytes in 0.21 seconds (819.02 MB/s) After: [ 4.504714] PM: Wrote 178932 kbytes in 0.22 seconds (813.32 MB/s) Cc: Dave Hansen Cc: Andy Lutomirski Cc: Peter Zijlstra Cc: "Rafael J. Wysocki" Cc: Pavel Machek Acked-by: Pavel Machek Signed-off-by: Rick Edgecombe --- arch/x86/mm/pageattr.c | 4 ---- include/linux/mm.h | 18 ++++++------------ mm/page_alloc.c | 7 +++++-- 3 files changed, 11 insertions(+), 18 deletions(-) diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 3a51915a1410..717bdc188aab 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -2257,7 +2257,6 @@ int set_alias_default_noflush(struct page *page) return __set_pages_p(page, 1); } -#ifdef CONFIG_DEBUG_PAGEALLOC void __kernel_map_pages(struct page *page, int numpages, int enable) { if (PageHighMem(page)) @@ -2302,11 +2301,8 @@ bool kernel_page_present(struct page *page) pte = lookup_address((unsigned long)page_address(page), &level); return (pte_val(*pte) & _PAGE_PRESENT); } - #endif /* CONFIG_HIBERNATION */ -#endif /* CONFIG_DEBUG_PAGEALLOC */ - int __init kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address, unsigned numpages, unsigned long page_flags) { diff --git a/include/linux/mm.h b/include/linux/mm.h index 80bb6408fe73..b362a280a919 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2642,37 +2642,31 @@ static inline void kernel_poison_pages(struct page *page, int numpages, int enable) { } #endif -#ifdef CONFIG_DEBUG_PAGEALLOC extern bool _debug_pagealloc_enabled; -extern void __kernel_map_pages(struct page *page, int numpages, int enable); static inline bool debug_pagealloc_enabled(void) { - return _debug_pagealloc_enabled; + return IS_ENABLED(CONFIG_DEBUG_PAGEALLOC) && _debug_pagealloc_enabled; } +#if defined(CONFIG_DEBUG_PAGEALLOC) || defined(CONFIG_ARCH_HAS_SET_ALIAS) +extern void __kernel_map_pages(struct page *page, int numpages, int enable); + static inline void kernel_map_pages(struct page *page, int numpages, int enable) { - if (!debug_pagealloc_enabled()) - return; - __kernel_map_pages(page, numpages, enable); } #ifdef CONFIG_HIBERNATION extern bool kernel_page_present(struct page *page); #endif /* CONFIG_HIBERNATION */ -#else /* CONFIG_DEBUG_PAGEALLOC */ +#else /* CONFIG_DEBUG_PAGEALLOC || CONFIG_ARCH_HAS_SET_ALIAS */ static inline void kernel_map_pages(struct page *page, int numpages, int enable) {} #ifdef CONFIG_HIBERNATION static inline bool kernel_page_present(struct page *page) { return true; } #endif /* CONFIG_HIBERNATION */ -static inline bool debug_pagealloc_enabled(void) -{ - return false; -} -#endif /* CONFIG_DEBUG_PAGEALLOC */ +#endif /* CONFIG_DEBUG_PAGEALLOC || CONFIG_ARCH_HAS_SET_ALIAS */ #ifdef __HAVE_ARCH_GATE_AREA extern struct vm_area_struct *get_gate_vma(struct mm_struct *mm); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index d295c9bc01a8..92d0a0934274 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1074,7 +1074,9 @@ static __always_inline bool free_pages_prepare(struct page *page, } arch_free_page(page, order); kernel_poison_pages(page, 1 << order, 0); - kernel_map_pages(page, 1 << order, 0); + if (debug_pagealloc_enabled()) + kernel_map_pages(page, 1 << order, 0); + kasan_free_nondeferred_pages(page, order); return true; @@ -1944,7 +1946,8 @@ inline void post_alloc_hook(struct page *page, unsigned int order, set_page_refcounted(page); arch_alloc_page(page, order); - kernel_map_pages(page, 1 << order, 1); + if (debug_pagealloc_enabled()) + kernel_map_pages(page, 1 << order, 1); kernel_poison_pages(page, 1 << order, 1); kasan_alloc_pages(page, order); set_page_owner(page, order, gfp_flags); From patchwork Tue Jan 29 00:34:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785245 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7F03A6C2 for ; Tue, 29 Jan 2019 00:41:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6F7F32B879 for ; Tue, 29 Jan 2019 00:41:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 632BD2B889; Tue, 29 Jan 2019 00:41:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3B5CA2B5A2 for ; Tue, 29 Jan 2019 00:41:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 025878E0012; Mon, 28 Jan 2019 19:41:07 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F1A9E8E0008; Mon, 28 Jan 2019 19:41:06 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E061B8E0012; Mon, 28 Jan 2019 19:41:06 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id A29B88E0008 for ; Mon, 28 Jan 2019 19:41:06 -0500 (EST) Received: by mail-pf1-f198.google.com with SMTP id p15so15387112pfk.7 for ; Mon, 28 Jan 2019 16:41:06 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=5ojdOxUYVtZ95xBX5GOCrCT5kCgaS2LvmI/53rS/NiY=; b=hTYxLJu1Z2ps/0xuKlsM5CzjLQd3qJtO2WeyMjn/Nr+Pub4OfQTkW9b8Da2xZVlsFL eYQIMFEQpCN9Qf4B8cexauHtJpUuJSwm4SgPpsWOAA5tP85+IDlhcVvwHQo7QwdWogqn 7f72DGK2YHSQE+Sa0vbtXgDCCDwlW+0616aVcVK3VAH864jxXXKWw6v2f2GSPLaASD03 o6xz5zx5kLCDtstHg4Qi062sbCZhsDxTG9x1+BRbSJfUu3Lxl0hTnZAXvrqZmG/YoZVj LCoC9jvDbqOKhrXGk+scHTJxA3GD1hFcRFstKQtVekS47XoIXfQqLzhquiyDuPJniOpf JEkw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukdY2/QMfpSTvnsiL7YVmD7t5QIn9SZNOBticuY3BAPu127u4QNQ 2f417m2dq2/u4l5lqAXDCwApnE4HuE/Duk0n8jnQvEJXz+QZGh718oS6z/ZW+GcLfukus80c5Ur H6WR8aGpAzfpdTdrKJLMZ5b+UH8zF3IG74fmOFzSJNZqcBRYIuzNSSZIfPReszWaN4w== X-Received: by 2002:a62:3a04:: with SMTP id h4mr23852584pfa.119.1548722466300; Mon, 28 Jan 2019 16:41:06 -0800 (PST) X-Google-Smtp-Source: ALg8bN5/I3RVZXCSBx7qWx8Ok2zwGOIRFLCIgUGw4w+yVxKHm/8SSSoQ6stQhdz5bXxC8GgveTN8 X-Received: by 2002:a62:3a04:: with SMTP id h4mr23846787pfa.119.1548722353880; Mon, 28 Jan 2019 16:39:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722353; cv=none; d=google.com; s=arc-20160816; b=y9vDBT7ZN7MOXMEy2okpIxvz5Ctp6cW+F72q6e/qP/SungcEE28AcPBoTRQ8VwK4fV /H9KXeaNZVa18blLY1csdEa6gDkRDmFMawYj8Ywr2rFhAdY0GkPz1azcoOEH0qfZ9dsu jUYgX4o6OyOcUUXRy3vv3+LZDddp312rwbM2vINDCMSF3TzNLUPZtW3SEyJ8+/80otJK QJ6qBvQbWzXzNiQZiKc+SExOCvWGJUuYbTjPmREgbmjdzrjvb8gqLLVrHRdn/4VK1R7P J2W2JICzhQ+0UFqAhw0wdqd3yfW1fUH2BR5zlOAxlLB3bvqQdyGc4uSFZPxTRjv6b8nd Dfnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=5ojdOxUYVtZ95xBX5GOCrCT5kCgaS2LvmI/53rS/NiY=; b=BeWdUTKLfD5QQ7+xX1hmGafqN1fKWt4oMInI/rNnncEyzxUm1LluvsJOXdSH/rF6Yc KBlT3hSd5fxMY7BQlD7GqXrZfX18RogjO52H5U8ZocAFFZH0hJi/Q79d1PPWXB0Ddp+S v4JVys25WvqkDIsmXJ7sAHOiSzKt8aaTjWQiedmioCe6Lg/rxUIBb1rRMzu1pdU7oZr6 8upN52+c7RIa/D68c+YeWNFnqt/ZOwHJj7GXCJmw0AMpWa61csrGLAGDnNduBztntqRE Fv6q3aR5Z6ccsf+iCQioLEAVlmGwRlUcinuMeqQIGrI+NnrlRAZX6Sskettwz5SH/AVQ afAA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga01.intel.com (mga01.intel.com. [192.55.52.88]) by mx.google.com with ESMTPS id i9si7660357plb.35.2019.01.28.16.39.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:13 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) client-ip=192.55.52.88; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921929" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:12 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Rick Edgecombe Subject: [PATCH v2 15/20] vmalloc: New flags for safe vfree on special perms Date: Mon, 28 Jan 2019 16:34:17 -0800 Message-Id: <20190129003422.9328-16-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This adds a new flags VM_HAS_SPECIAL_PERMS, for enabling vfree operations to immediately clear executable TLB entries to freed pages, and handle freeing memory with special permissions. It also takes care of resetting the direct map permissions for the pages being unmapped. So this flag is useful for any kind of memory with elevated permissions, or where there can be related permissions changes on the directmap. Today this is RO+X and RO memory. Although this enables directly vfreeing RO memory now, RO memory cannot be freed in an interrupt because the allocation itself is used as a node on deferred free list. So when RO memory needs to be freed in an interrupt the code doing the vfree needs to have its own work queue, as was the case before the deferred vfree list handling was added. Today there is only one case where this happens. For architectures with set_alias_ implementations this whole operation can be done with one TLB flush when centralized like this. For others with directmap permissions, currently only arm64, a backup method using set_memory functions is used to reset the directmap. When arm64 adds set_alias_ functions, this backup can be removed. When the TLB is flushed to both remove TLB entries for the vmalloc range mapping and the direct map permissions, the lazy purge operation could be done to try to save a TLB flush later. However today vm_unmap_aliases could flush a TLB range that does not include the directmap. So a helper is added with extra parameters that can allow both the vmalloc address and the direct mapping to be flushed during this operation. The behavior of the normal vm_unmap_aliases function is unchanged. Suggested-by: Dave Hansen Suggested-by: Andy Lutomirski Suggested-by: Will Deacon Signed-off-by: Rick Edgecombe --- include/linux/vmalloc.h | 13 +++++ mm/vmalloc.c | 122 +++++++++++++++++++++++++++++++++------- 2 files changed, 116 insertions(+), 19 deletions(-) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 398e9c95cd61..9f643f917360 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -21,6 +21,11 @@ struct notifier_block; /* in notifier.h */ #define VM_UNINITIALIZED 0x00000020 /* vm_struct is not fully initialized */ #define VM_NO_GUARD 0x00000040 /* don't add guard page */ #define VM_KASAN 0x00000080 /* has allocated kasan shadow memory */ +/* + * Memory with VM_HAS_SPECIAL_PERMS cannot be freed in an interrupt or with + * vfree_atomic. + */ +#define VM_HAS_SPECIAL_PERMS 0x00000200 /* Reset directmap and flush TLB on unmap */ /* bits [20..32] reserved for arch specific ioremap internals */ /* @@ -135,6 +140,14 @@ extern struct vm_struct *__get_vm_area_caller(unsigned long size, extern struct vm_struct *remove_vm_area(const void *addr); extern struct vm_struct *find_vm_area(const void *addr); +static inline void set_vm_special(void *addr) +{ + struct vm_struct *vm = find_vm_area(addr); + + if (vm) + vm->flags |= VM_HAS_SPECIAL_PERMS; +} + extern int map_vm_area(struct vm_struct *area, pgprot_t prot, struct page **pages); #ifdef CONFIG_MMU diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 871e41c55e23..d459b5b9649b 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -1055,24 +1056,11 @@ static void vb_free(const void *addr, unsigned long size) spin_unlock(&vb->lock); } -/** - * vm_unmap_aliases - unmap outstanding lazy aliases in the vmap layer - * - * The vmap/vmalloc layer lazily flushes kernel virtual mappings primarily - * to amortize TLB flushing overheads. What this means is that any page you - * have now, may, in a former life, have been mapped into kernel virtual - * address by the vmap layer and so there might be some CPUs with TLB entries - * still referencing that page (additional to the regular 1:1 kernel mapping). - * - * vm_unmap_aliases flushes all such lazy mappings. After it returns, we can - * be sure that none of the pages we have control over will have any aliases - * from the vmap layer. - */ -void vm_unmap_aliases(void) +static void _vm_unmap_aliases(unsigned long start, unsigned long end, + int must_flush) { - unsigned long start = ULONG_MAX, end = 0; int cpu; - int flush = 0; + int flush = must_flush; if (unlikely(!vmap_initialized)) return; @@ -1109,6 +1097,27 @@ void vm_unmap_aliases(void) flush_tlb_kernel_range(start, end); mutex_unlock(&vmap_purge_lock); } + +/** + * vm_unmap_aliases - unmap outstanding lazy aliases in the vmap layer + * + * The vmap/vmalloc layer lazily flushes kernel virtual mappings primarily + * to amortize TLB flushing overheads. What this means is that any page you + * have now, may, in a former life, have been mapped into kernel virtual + * address by the vmap layer and so there might be some CPUs with TLB entries + * still referencing that page (additional to the regular 1:1 kernel mapping). + * + * vm_unmap_aliases flushes all such lazy mappings. After it returns, we can + * be sure that none of the pages we have control over will have any aliases + * from the vmap layer. + */ +void vm_unmap_aliases(void) +{ + unsigned long start = ULONG_MAX, end = 0; + int must_flush = 0; + + _vm_unmap_aliases(start, end, must_flush); +} EXPORT_SYMBOL_GPL(vm_unmap_aliases); /** @@ -1494,6 +1503,79 @@ struct vm_struct *remove_vm_area(const void *addr) return NULL; } +static inline void set_area_alias(const struct vm_struct *area, + int (*set_alias)(struct page *page)) +{ + int i; + + for (i = 0; i < area->nr_pages; i++) { + unsigned long addr = + (unsigned long)page_address(area->pages[i]); + + if (addr) + set_alias(area->pages[i]); + } +} + +/* This handles removing and resetting vm mappings related to the vm_struct. */ +static void vm_remove_mappings(struct vm_struct *area, int deallocate_pages) +{ + unsigned long addr = (unsigned long)area->addr; + unsigned long start = ULONG_MAX, end = 0; + int special = area->flags & VM_HAS_SPECIAL_PERMS; + int i; + + /* + * The below block can be removed when all architectures that have + * direct map permissions also have set_alias_ implementations. This is + * to do resetting on the directmap for any special permissions (today + * only X), without leaving a RW+X window. + */ + if (special && !IS_ENABLED(CONFIG_ARCH_HAS_SET_ALIAS)) { + set_memory_nx(addr, area->nr_pages); + set_memory_rw(addr, area->nr_pages); + } + + remove_vm_area(area->addr); + + /* If this is not special memory, we can skip the below. */ + if (!special) + return; + + /* + * If we are not deallocating pages, we can just do the flush of the VM + * area and return. + */ + if (!deallocate_pages) { + vm_unmap_aliases(); + return; + } + + /* + * If we are here, we need to flush the vm mapping and reset the direct + * map. + * First find the start and end range of the direct mappings to make + * sure the vm_unmap_aliases flush includes the direct map. + */ + for (i = 0; i < area->nr_pages; i++) { + unsigned long addr = + (unsigned long)page_address(area->pages[i]); + if (addr) { + start = min(addr, start); + end = max(addr, end); + } + } + + /* + * First we set direct map to something not valid so that it won't be + * cached if there are any accesses after the TLB flush, then we flush + * the TLB, and reset the directmap permissions to the default. + */ + set_area_alias(area, set_alias_nv_noflush); + _vm_unmap_aliases(start, end, 1); + set_area_alias(area, set_alias_default_noflush); +} + static void __vunmap(const void *addr, int deallocate_pages) { struct vm_struct *area; @@ -1515,7 +1597,8 @@ static void __vunmap(const void *addr, int deallocate_pages) debug_check_no_locks_freed(area->addr, get_vm_area_size(area)); debug_check_no_obj_freed(area->addr, get_vm_area_size(area)); - remove_vm_area(addr); + vm_remove_mappings(area, deallocate_pages); + if (deallocate_pages) { int i; @@ -1925,8 +2008,9 @@ EXPORT_SYMBOL(vzalloc_node); void *vmalloc_exec(unsigned long size) { - return __vmalloc_node(size, 1, GFP_KERNEL, PAGE_KERNEL_EXEC, - NUMA_NO_NODE, __builtin_return_address(0)); + return __vmalloc_node_range(size, 1, VMALLOC_START, VMALLOC_END, + GFP_KERNEL, PAGE_KERNEL_EXEC, VM_HAS_SPECIAL_PERMS, + NUMA_NO_NODE, __builtin_return_address(0)); } #if defined(CONFIG_64BIT) && defined(CONFIG_ZONE_DMA32) From patchwork Tue Jan 29 00:34:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785141 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DA08F6C2 for ; Tue, 29 Jan 2019 00:39:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C87162B4C4 for ; Tue, 29 Jan 2019 00:39:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BCAD02B4D9; Tue, 29 Jan 2019 00:39:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A3D8A2B4C4 for ; Tue, 29 Jan 2019 00:39:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E2768E000F; Mon, 28 Jan 2019 19:39:16 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1AD7D8E0010; Mon, 28 Jan 2019 19:39:16 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E7FB08E000F; Mon, 28 Jan 2019 19:39:15 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) by kanga.kvack.org (Postfix) with ESMTP id 819CC8E000B for ; Mon, 28 Jan 2019 19:39:15 -0500 (EST) Received: by mail-pl1-f200.google.com with SMTP id e68so13039765plb.3 for ; Mon, 28 Jan 2019 16:39:15 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=DKNSNx9TC+YmyW1Pv02PcHIIet+m2H/do1xAqKXpM3I=; b=hwhuzeNjDzA0mLZGkfUB5+ar6e11EqzfepV+eteQ+GT+TollgvS6PI6mVTBE1TDaA3 kGFvGd4vydL0ee08XjNLg5NdrHfDIrvLy35OXXTZYmbLQlMNrph523DfnzlINEBEoSBF G9P7tM4Wvjy9bZ4hpCQwk45hpk3pPdtZ9KhT7pqDxsEXB1wjJSsrz2AqwmKap78ffPdf rIkGs0XSfK5dJYuisv1OcB381JPLeiaihMKGLAyA7txWgEfr/pTwdxR2FjxL6zshR6kP SEhPSsHTDLVYIr+qGPROwhbpOB0UUnhoVYLGXqMFBhuQV1o7cNSP3RcmY1dTm+t5h049 9RaQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukeOS6JmM91Ccu1HucKv8KoRVE0t0g0cf/Tnc4gpaXym4MIABD77 iLfv9N01aOhFCugSFrmPXxPHGQSV5L8wZ6oLWX7znBfC4NlRbMXKfu/QJ+sCEy+xSY2SAKI8W+4 RZGEJ5dcqzo1G93XmQvrQuPgN0iymuLkT7+ZfM1df5Liyxlm4f/u4AAoItXmtvLBvow== X-Received: by 2002:a62:68c5:: with SMTP id d188mr24764865pfc.194.1548722355164; Mon, 28 Jan 2019 16:39:15 -0800 (PST) X-Google-Smtp-Source: ALg8bN5hGNv20LQstWtQzGXUbbO5yRTygBqrv6Kw0GdeU+a5QazuWDqhWNKsUj6laK70XUulqtZ6 X-Received: by 2002:a62:68c5:: with SMTP id d188mr24764807pfc.194.1548722354167; Mon, 28 Jan 2019 16:39:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722354; cv=none; d=google.com; s=arc-20160816; b=xZMSFOPUMN+Is7swVHXpjdsdvuA6JsG/p2MfIPppGHftd9t5r/UMUumF/rnDExVQdc 5dz7OkNW6Jyib2YwDhSHTviq/sJ0NGdO0YIMR5p9vNaehA4jXL98hwQ44jnR62gYXdQz 9f2Jw6ve6xjLmNiAQi+2stKyQ8tvjBosjWAOaVy7gr2PxzNkcahGTL1p3rsLHBut1Lxr SYneeaHN3kk7/bigmguuwAMn4L2eMG7J+f6bwsOX54db3R6kSLuX8n6Jdtn6S9WilaI2 3Y1KXrm8zgoaGZnPyMWzNu7vLGe78JwbZ8merXSeKJPHvHnZSOQIEuh4Ac2OGUWd1aWe KBsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=DKNSNx9TC+YmyW1Pv02PcHIIet+m2H/do1xAqKXpM3I=; b=oOVx8+OOEbeGa3XYDt6KdCzhqy/qm4eQshgfrGMO8uB62x2BUzjcLPQOdqDG45+YEN 9QfFZpliFrJ/AIE2A6mAnQtcagFn1RlLUgXcLCWGlLFDwqmzk4RBQf2LOWZBYiK4XRbW uipEjNh13fdNMKRTLPEt2YfTKcnP96NAIImARKLsxzLnjrUawlTCqBtY1TttidIcFcji wYTMjv73ZasJs4/fEiAR8Va8b/9vDqnvfGZqpoO6uD6nKnI8ah0OXpmrbT+9hkXSaORK kD3y03DnibzKMYSq0yCT4eoFGtD0q+/ALoH0gppu+lQI3YywBusZD8svYopA8o9w8iKD dHWA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga04.intel.com (mga04.intel.com. [192.55.52.120]) by mx.google.com with ESMTPS id l7si33052569pfg.245.2019.01.28.16.39.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:14 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) client-ip=192.55.52.120; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921934" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:12 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Rick Edgecombe , Jessica Yu , Steven Rostedt Subject: [PATCH v2 16/20] modules: Use vmalloc special flag Date: Mon, 28 Jan 2019 16:34:18 -0800 Message-Id: <20190129003422.9328-17-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Use new flag for handling freeing of special permissioned memory in vmalloc and remove places where memory was set RW before freeing which is no longer needed. Since vfreeing of VM_HAS_SPECIAL_PERMS memory is not supported in an interrupt by vmalloc, the freeing of init sections is moved to a work queue. Instead of call_rcu it now uses synchronize_rcu() in the work queue. Lastly, there is now a WARN_ON in module_memfree since it should not be called in an interrupt with special memory as is required for VM_HAS_SPECIAL_PERMS. Cc: Jessica Yu Cc: Steven Rostedt Signed-off-by: Rick Edgecombe --- kernel/module.c | 77 +++++++++++++++++++++++++------------------------ 1 file changed, 39 insertions(+), 38 deletions(-) diff --git a/kernel/module.c b/kernel/module.c index ae1b77da6a20..1af5c8e19086 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -98,6 +98,10 @@ DEFINE_MUTEX(module_mutex); EXPORT_SYMBOL_GPL(module_mutex); static LIST_HEAD(modules); +/* Work queue for freeing init sections in success case */ +static struct work_struct init_free_wq; +static struct llist_head init_free_list; + #ifdef CONFIG_MODULES_TREE_LOOKUP /* @@ -1949,6 +1953,8 @@ void module_enable_ro(const struct module *mod, bool after_init) if (!rodata_enabled) return; + set_vm_special(mod->core_layout.base); + set_vm_special(mod->init_layout.base); frob_text(&mod->core_layout, set_memory_ro); frob_text(&mod->core_layout, set_memory_x); @@ -1972,15 +1978,6 @@ static void module_enable_nx(const struct module *mod) frob_writable_data(&mod->init_layout, set_memory_nx); } -static void module_disable_nx(const struct module *mod) -{ - frob_rodata(&mod->core_layout, set_memory_x); - frob_ro_after_init(&mod->core_layout, set_memory_x); - frob_writable_data(&mod->core_layout, set_memory_x); - frob_rodata(&mod->init_layout, set_memory_x); - frob_writable_data(&mod->init_layout, set_memory_x); -} - /* Iterate through all modules and set each module's text as RW */ void set_all_modules_text_rw(void) { @@ -2024,23 +2021,8 @@ void set_all_modules_text_ro(void) } mutex_unlock(&module_mutex); } - -static void disable_ro_nx(const struct module_layout *layout) -{ - if (rodata_enabled) { - frob_text(layout, set_memory_rw); - frob_rodata(layout, set_memory_rw); - frob_ro_after_init(layout, set_memory_rw); - } - frob_rodata(layout, set_memory_x); - frob_ro_after_init(layout, set_memory_x); - frob_writable_data(layout, set_memory_x); -} - #else -static void disable_ro_nx(const struct module_layout *layout) { } static void module_enable_nx(const struct module *mod) { } -static void module_disable_nx(const struct module *mod) { } #endif #ifdef CONFIG_LIVEPATCH @@ -2120,6 +2102,11 @@ static void free_module_elf(struct module *mod) void __weak module_memfree(void *module_region) { + /* + * This memory may be RO, and freeing RO memory in an interrupt is not + * supported by vmalloc. + */ + WARN_ON(in_interrupt()); vfree(module_region); } @@ -2171,7 +2158,6 @@ static void free_module(struct module *mod) mutex_unlock(&module_mutex); /* This may be empty, but that's OK */ - disable_ro_nx(&mod->init_layout); module_arch_freeing_init(mod); module_memfree(mod->init_layout.base); kfree(mod->args); @@ -2181,7 +2167,6 @@ static void free_module(struct module *mod) lockdep_free_key_range(mod->core_layout.base, mod->core_layout.size); /* Finally, free the core (containing the module structure) */ - disable_ro_nx(&mod->core_layout); module_memfree(mod->core_layout.base); } @@ -3424,17 +3409,34 @@ static void do_mod_ctors(struct module *mod) /* For freeing module_init on success, in case kallsyms traversing */ struct mod_initfree { - struct rcu_head rcu; + struct llist_node node; void *module_init; }; -static void do_free_init(struct rcu_head *head) +static void do_free_init(struct work_struct *w) { - struct mod_initfree *m = container_of(head, struct mod_initfree, rcu); - module_memfree(m->module_init); - kfree(m); + struct llist_node *pos, *n, *list; + struct mod_initfree *initfree; + + list = llist_del_all(&init_free_list); + + synchronize_rcu(); + + llist_for_each_safe(pos, n, list) { + initfree = container_of(pos, struct mod_initfree, node); + module_memfree(initfree->module_init); + kfree(initfree); + } } +static int __init modules_wq_init(void) +{ + INIT_WORK(&init_free_wq, do_free_init); + init_llist_head(&init_free_list); + return 0; +} +module_init(modules_wq_init); + /* * This is where the real work happens. * @@ -3511,7 +3513,6 @@ static noinline int do_init_module(struct module *mod) #endif module_enable_ro(mod, true); mod_tree_remove_init(mod); - disable_ro_nx(&mod->init_layout); module_arch_freeing_init(mod); mod->init_layout.base = NULL; mod->init_layout.size = 0; @@ -3522,14 +3523,18 @@ static noinline int do_init_module(struct module *mod) * We want to free module_init, but be aware that kallsyms may be * walking this with preempt disabled. In all the failure paths, we * call synchronize_rcu(), but we don't want to slow down the success - * path, so use actual RCU here. + * path. We can't do module_memfree in an interrupt, so we do the work + * and call synchronize_rcu() in a work queue. + * * Note that module_alloc() on most architectures creates W+X page * mappings which won't be cleaned up until do_free_init() runs. Any * code such as mark_rodata_ro() which depends on those mappings to * be cleaned up needs to sync with the queued work - ie * rcu_barrier() */ - call_rcu(&freeinit->rcu, do_free_init); + if (llist_add(&freeinit->node, &init_free_list)) + schedule_work(&init_free_wq); + mutex_unlock(&module_mutex); wake_up_all(&module_wq); @@ -3826,10 +3831,6 @@ static int load_module(struct load_info *info, const char __user *uargs, module_bug_cleanup(mod); mutex_unlock(&module_mutex); - /* we can't deallocate the module until we clear memory protection */ - module_disable_ro(mod); - module_disable_nx(mod); - ddebug_cleanup: ftrace_release_mod(mod); dynamic_debug_remove(mod, info->debug); From patchwork Tue Jan 29 00:34:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785229 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0118914E1 for ; Tue, 29 Jan 2019 00:41:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E4BFE2B517 for ; Tue, 29 Jan 2019 00:41:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D8BB32B896; Tue, 29 Jan 2019 00:41:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C21C12B517 for ; Tue, 29 Jan 2019 00:41:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C5ABC8E0010; Mon, 28 Jan 2019 19:41:01 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C303F8E0001; Mon, 28 Jan 2019 19:41:01 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF7EA8E0010; Mon, 28 Jan 2019 19:41:01 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) by kanga.kvack.org (Postfix) with ESMTP id 6E6618E0001 for ; Mon, 28 Jan 2019 19:41:01 -0500 (EST) Received: by mail-pg1-f197.google.com with SMTP id o17so12695835pgi.14 for ; Mon, 28 Jan 2019 16:41:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=f3R2YUO4FGOPRke7XU/+0tFiGuW/e9uBw+X2/zRPj2A=; b=m0zTJmHx8y3OsXpJbDZF9s9gz7g0+AtOJyTvui5sad5uHdEn4qOFZijLJPVgPqvtf3 JQPXtXpsilxjxXnO97pf9k8cHWy1vKfTAIPl0s8Rj4gUMyJEGJBhGBSbm3MvvwFMxGQ6 GhXznCRFHsHZljDraJHO1xPNBHiuYh420mGHNTstPPT7dnj0WBJOefJUdMDHE0JMq4h+ KkZf54LIXOiZMskWdsY5s57E0a+i/bhPK8x0+b0Lwq36FHU1Lfg7n1WI6souGFwyGrCf zj6tV26iq1GLy+8W4zQDc9Wchn5yB5VD/Bsw0I6e86ehXB2GHzl2nNb2RZRoZsmzH6TH miiw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukchaLCYnUPaNgGgJe+TcCF6rb/nmWGH68gikcGhtXkfE4d6uU1a rIegVuCZnbqPovu3I352SEMYiMvhgpyy+gxm9/+8HhLxATHlhww0w12VQZo3OOj6E0lvv5u8eF5 pnp/AI+z20O1iSs9G0BFZI092qMLMW21p7S57WPtAXwN3nGkjsQouLJYM96d8emfsxw== X-Received: by 2002:a17:902:9692:: with SMTP id n18mr24332659plp.333.1548722461110; Mon, 28 Jan 2019 16:41:01 -0800 (PST) X-Google-Smtp-Source: ALg8bN42Tcnf6ywYNI2ugd7jiRTiSfMM9B6W2pAuXX4Gy/KBe/Ihyog53B0R/NQbWI6qlmElxjxb X-Received: by 2002:a17:902:9692:: with SMTP id n18mr24327214plp.333.1548722354058; Mon, 28 Jan 2019 16:39:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722354; cv=none; d=google.com; s=arc-20160816; b=rzSRxtyoOFDH8rLlP1Nkxf+z2A4cyN/rEcfrui/jGIwJGKt/LDO4d50xSZEShr2DAf YAa7PJibY+vS/SjWDCAAzZ4Ipo55ORTkjNa/RQZvvNF6FVwCiK0lOM/1j9G3CnqR11R2 P2Ze2ydlMHulK53pZdNoVPeP8ImxBnGgYMis3lLUfX4o59ezY9kSeumFXhgHr8t/a3iy TkF6jYvrl1ECQS4TUKv2OY79+wqAIca85F6eTzeKRUmMhQzzhvAYtCdmHrYjH2EGVAKd /7sQI5skc2nuZZL3B+gv4zER55i/Mjbie/FHCcLwEoHksTTeOLIBwaDVVBxUVfkqWwX5 lRpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=f3R2YUO4FGOPRke7XU/+0tFiGuW/e9uBw+X2/zRPj2A=; b=j54U/20tT8eDMMI2jDCZCB/C3UGGG8ia/dZmQpyDfj/2iLaf/kJ2Cxet1PJnsLkh0U WmG/UOtoV3NgM72deL585j1UcA2JL6+kjIWW7MX06AiatOtRXRduyLSIxBPFeb3MIEZl QOTtwPUkhLArKfhxMSKSpRbbrs70lyOJXhDRxFvAwahQ64gIQkHdsZD25WjVa0uwSNeM 7hWjRToIVMfpQCL7uaakiGQr1GmB8QYaHHP2Lg+2XhOpeTmntygvfjHajqNQQJNKFesA c7Sf6yyOIdY7grDD6KvkmMVjGF3j36j5SoE5Vsl2z6gN5lclKrMMcQLXNvZOw8Uq/Gko 3AqQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga01.intel.com (mga01.intel.com. [192.55.52.88]) by mx.google.com with ESMTPS id 33si8810596plt.228.2019.01.28.16.39.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:14 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) client-ip=192.55.52.88; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921937" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:12 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Rick Edgecombe , Daniel Borkmann , Alexei Starovoitov Subject: [PATCH v2 17/20] bpf: Use vmalloc special flag Date: Mon, 28 Jan 2019 16:34:19 -0800 Message-Id: <20190129003422.9328-18-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Use new flag VM_HAS_SPECIAL_PERMS for handling freeing of special permissioned memory in vmalloc and remove places where memory was set RW before freeing which is no longer needed. Also we no longer need a bit to track if the memory is RO because it is tracked in vmalloc. Cc: Daniel Borkmann Cc: Alexei Starovoitov Signed-off-by: Rick Edgecombe --- include/linux/filter.h | 16 +++------------- kernel/bpf/core.c | 1 - 2 files changed, 3 insertions(+), 14 deletions(-) diff --git a/include/linux/filter.h b/include/linux/filter.h index 9cdfab7f383c..cc9581dd9c58 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -20,6 +20,7 @@ #include #include #include +#include #include @@ -483,7 +484,6 @@ struct bpf_prog { u16 pages; /* Number of allocated pages */ u16 jited:1, /* Is our filter JIT'ed? */ jit_requested:1,/* archs need to JIT the prog */ - undo_set_mem:1, /* Passed set_memory_ro() checkpoint */ gpl_compatible:1, /* Is filter GPL compatible? */ cb_access:1, /* Is control block accessed? */ dst_needed:1, /* Do we need dst entry? */ @@ -681,26 +681,17 @@ bpf_ctx_narrow_access_ok(u32 off, u32 size, u32 size_default) static inline void bpf_prog_lock_ro(struct bpf_prog *fp) { + set_vm_special(fp); set_memory_ro((unsigned long)fp, fp->pages); } -static inline void bpf_prog_unlock_ro(struct bpf_prog *fp) -{ - if (fp->undo_set_mem) - set_memory_rw((unsigned long)fp, fp->pages); -} - static inline void bpf_jit_binary_lock_ro(struct bpf_binary_header *hdr) { + set_vm_special(hdr); set_memory_ro((unsigned long)hdr, hdr->pages); set_memory_x((unsigned long)hdr, hdr->pages); } -static inline void bpf_jit_binary_unlock_ro(struct bpf_binary_header *hdr) -{ - set_memory_rw((unsigned long)hdr, hdr->pages); -} - static inline struct bpf_binary_header * bpf_jit_binary_hdr(const struct bpf_prog *fp) { @@ -735,7 +726,6 @@ void __bpf_prog_free(struct bpf_prog *fp); static inline void bpf_prog_unlock_free(struct bpf_prog *fp) { - bpf_prog_unlock_ro(fp); __bpf_prog_free(fp); } diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 19c49313c709..465c1c3623e8 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -804,7 +804,6 @@ void __weak bpf_jit_free(struct bpf_prog *fp) if (fp->jited) { struct bpf_binary_header *hdr = bpf_jit_binary_hdr(fp); - bpf_jit_binary_unlock_ro(hdr); bpf_jit_binary_free(hdr); WARN_ON_ONCE(!bpf_prog_kallsyms_verify_off(fp)); From patchwork Tue Jan 29 00:34:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785249 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B8A7C14E1 for ; Tue, 29 Jan 2019 00:41:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A85CF2B875 for ; Tue, 29 Jan 2019 00:41:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9C7992B868; Tue, 29 Jan 2019 00:41:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4D47B2B896 for ; Tue, 29 Jan 2019 00:41:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 35AFC8E0013; Mon, 28 Jan 2019 19:41:12 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3084A8E0008; Mon, 28 Jan 2019 19:41:12 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1D2A88E0013; Mon, 28 Jan 2019 19:41:12 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) by kanga.kvack.org (Postfix) with ESMTP id 6FD388E0008 for ; Mon, 28 Jan 2019 19:41:11 -0500 (EST) Received: by mail-pg1-f197.google.com with SMTP id v72so12685172pgb.10 for ; Mon, 28 Jan 2019 16:41:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=yHQwJpfGDvOx+xHKw3fwkxaiviK2x3hbH3nzRfUqk4U=; b=IHLwIgNQs9LwEuDdk6Bj74q98VR3ScVhaE5tMNMUbn2EXuYuAKxsniqquTOryAeQF2 l2HqArS3k3SpJQnF2b+ucE8LEd6MRTskEuokU3C1gT8w9J0DPbMvbZvq65Wq49pD9Dvj tCgeYGFSEA8KlYKi8mneYDFSygFujv/3jAbavX4d2iFa8W+N1ZFvotKwpnOSBNc/PrcO CWMR5llrm4i6Uba5TyYmEq9gY7iwdQ5ErvfK2xzRPJR4IbsQz62vlz8EVn4oqqathqgb 3CnT3BZM4w48yRsFqDwW4O35gWvD1nqPI8SdkrOXoJBE6qsKz+1t/ckNsypPR1Ie4F4i B3pg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukc4QBMBEqJC8rKpUmcuvP96Yzmcg+aNoi2FYWGPE3HBvZLwDW0w omq/OmCx2Uq5wDjY9vYudWZwk1Xi5bzA6j0CPbZQQ8hrut3IpV5NggmFgcGt6eIkcQjVUvYYEZd +sOZ8lmjeL3u5C2ls9KEPJ3bzc9k8MX4Knp/o0Ff7q3sd+3KgfLPdhQan/EvnGkNNpg== X-Received: by 2002:a62:f247:: with SMTP id y7mr24061684pfl.25.1548722471116; Mon, 28 Jan 2019 16:41:11 -0800 (PST) X-Google-Smtp-Source: ALg8bN77UwLKDiFu+6adYjd1pkd5TqJEGuXvxhtMei3OGsZ07CN5hdwS+aqMypwqY81/p6Taz/c1 X-Received: by 2002:a62:f247:: with SMTP id y7mr24055495pfl.25.1548722354305; Mon, 28 Jan 2019 16:39:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722354; cv=none; d=google.com; s=arc-20160816; b=g0KneIySxUH0hIaPgGtS4kHa5NT9PSCNwkgrWTIUjZxbUXCbD3+Mvy+2fZLZ2I2AOH 3nc9GIrPc1GvJqRz4BxCiBafIZFp5gUiOtm+Rz9+ze7VlM7u7TzihRNIP7q9iIqaP8BP e/WHt7Ac0quGGU4+mcMmRV3YWyBFeb8gQd0DeTJRJW1YdXKOXVi4J97jkqf060W59O5j WRaj4ss69oDn+o+xWweRqk8Cr4pxfFAUDkNoV1Va5oenvs96+N6c6p4K5xQy6zbEyYVh m0BGOC/rGMPocz9NHuTgB1EehM6Hvw9nVTJ1pSm9fbWNJAKdw+/IFDofSi7brSarlpFa JFhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=yHQwJpfGDvOx+xHKw3fwkxaiviK2x3hbH3nzRfUqk4U=; b=haqxzf8Khbg7g4DUg2K3AXcjDzXf9rxDUugvzTd1WFQwa/kIWrSq2B3uR7kOlhoCnT j2mQIahDG1tlY7AKrG17Z3M2NT4EiXo4MVkQaEsH3lxx6DvVVQn/6xXpmvRIkQO0O4Mo bjMP09uXPmcyZoRIpbOmC33T12BBUYXslgiyFEYcC1Ug1NjOnisqDiOh91GtJ1AQ4nm8 bvcVBVt+nA+X2uEenWmBnQc044HGICUn45uU6ULBUpq+nuAXOBtKymUrXbz4JueWgEsn 1LYu3p8BdagxO1NwAYOs7TuwOxvawjzYK4s0NAB5sfiGXLbh11ebLUsUinV4snNP0eMv pSgA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga04.intel.com (mga04.intel.com. [192.55.52.120]) by mx.google.com with ESMTPS id u184si34858171pgd.262.2019.01.28.16.39.14 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:14 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) client-ip=192.55.52.120; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921940" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:12 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Rick Edgecombe , Steven Rostedt Subject: [PATCH v2 18/20] x86/ftrace: Use vmalloc special flag Date: Mon, 28 Jan 2019 16:34:20 -0800 Message-Id: <20190129003422.9328-19-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Use new flag VM_HAS_SPECIAL_PERMS for handling freeing of special permissioned memory in vmalloc and remove places where memory was set NX and RW before freeing which is no longer needed. Cc: Steven Rostedt Signed-off-by: Rick Edgecombe --- arch/x86/kernel/ftrace.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index 13c8249b197f..cf30594a2032 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -692,10 +692,6 @@ static inline void *alloc_tramp(unsigned long size) } static inline void tramp_free(void *tramp, int size) { - int npages = PAGE_ALIGN(size) >> PAGE_SHIFT; - - set_memory_nx((unsigned long)tramp, npages); - set_memory_rw((unsigned long)tramp, npages); module_memfree(tramp); } #else @@ -820,6 +816,8 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) /* ALLOC_TRAMP flags lets us know we created it */ ops->flags |= FTRACE_OPS_FL_ALLOC_TRAMP; + set_vm_special(trampoline); + /* * Module allocation needs to be completed by making the page * executable. The page is still writable, which is a security hazard, From patchwork Tue Jan 29 00:34:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785253 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3F3DB6C2 for ; Tue, 29 Jan 2019 00:41:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2FC542B88D for ; Tue, 29 Jan 2019 00:41:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 23EDF2B890; Tue, 29 Jan 2019 00:41:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C9A202B875 for ; Tue, 29 Jan 2019 00:41:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B47D48E0014; Mon, 28 Jan 2019 19:41:17 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AF8978E0008; Mon, 28 Jan 2019 19:41:17 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A0ED68E0014; Mon, 28 Jan 2019 19:41:17 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) by kanga.kvack.org (Postfix) with ESMTP id 5D01A8E0008 for ; Mon, 28 Jan 2019 19:41:17 -0500 (EST) Received: by mail-pg1-f200.google.com with SMTP id v72so12685358pgb.10 for ; Mon, 28 Jan 2019 16:41:17 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=f0ndo1Xal1a6v/4PjrGdAuv0FcoU8GzPJ1j2wQZwxYY=; b=ULQvnqsZ/VE+Lv5cA2/hWFeykoyJAsR+gQydP1Gy19IqVUed9lC0VrRlIMJ4zLQGJ3 7/g7F4QEocfPeaMH5uFqry3jCwGJ4Ynzg3GzQW6JU+o7CdzWyN306GEhg0go1rOSSu1r uIpXLK0UkXkWzZI758W+zPyC/FhJ+kXxhALbfFUc9E+s+jjRmzmt2xAJhsUR1pSQyJuw qqN9S9FPeW3OwxDGFCf5xZFgGFdIEc+DQReCWdCNOME6nBaYa8eAwEBt4E3U6wZLClwe KMC1WucMQ1wSICRNAnb+31pPaemwvZHLrkUBWEVfSnnlhVEzUjNROORTGDxAsw73TRlf wYWA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukfgKQcPpmQtqBX8bdPoFT6VIa5e5F/XpscfNRqFDDaDyqZvMa+d QMWGKGtyt+FD4G6DQqsFYhecnd0hjauF9qbG71cJU+Dx7psKBxF5Q7kyXjFAYp5smL5bPykkLRo zO2flukhJBs1BVI2deBwRmc/s9gXI0uEgLFPBySv6j4KBt1vGWZPyjFkwhJ+z7g6ipg== X-Received: by 2002:a62:8dd9:: with SMTP id p86mr23709382pfk.143.1548722477018; Mon, 28 Jan 2019 16:41:17 -0800 (PST) X-Google-Smtp-Source: ALg8bN6m/eC68OJKOYsg+BCOQHBSL4u/pp7zw+nFgBbt0xobQ+qneQIoiIFOkCDj8ozCSDeH+Rf+ X-Received: by 2002:a62:8dd9:: with SMTP id p86mr23703056pfk.143.1548722354158; Mon, 28 Jan 2019 16:39:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722354; cv=none; d=google.com; s=arc-20160816; b=BHfe1bm2/4/MDhZt6y0bKiP58pZhZM+lfaOzT4ntLfP4Rs036rut2o/FlCo+PEu1XZ pgSknASDvPWbrpBLRDQG501StTK0Ovw8sBC4B1EhqwHqpcROCkameTl7fAFfBWRyQFJt gt/mpNTFqcib14v/+5Rv6bAXzj4DBF7nXrz9GnV0zXjDzmrR+RSG7JcEGGslyGFG/HBw sQPW4VYgUEqNmmHZpoE0/fLkZTBr0GJozxSon3oQMgQqH/GSFISFrkGKeRkfy8r9qU/S 9TCpV1yMW/G2TGi/lg2NZbKtVPZFzTQEoDjRslHKsEgbW8mjapdC4aTYlAmOiCR2CJ84 XWjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=f0ndo1Xal1a6v/4PjrGdAuv0FcoU8GzPJ1j2wQZwxYY=; b=UspSNeE+oGmgHBl61SDi7HtUFSwfJcMvlSn5nR5/dLUERs6SAegno3FcTxq7KS/l3j iYkCKFWcTQ/oG7M/gwVjjV1uTTn9O7owxHTtaNt+rx/CO1cVhTbS3rQ0y8Gx4eIkNYxG qaY13IvQbMZzkSxYtFKNfew7kG/lJc2kpWECd90TkTy06NEQmLGIZuM5yE4QpS/MW4bB vabIv9oUZst5ak/tf5SUGsKHWWAo25dXib4ofS7p1Z+Nk2OtLpFTq68cD6u4EgUzcz7D rxsZIpal2v+896Eoyi4TU6djegG7ANPdCz3dNqhMLNEwuCTkfuva7gGPkbOIOrStvu6/ 95Fw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga01.intel.com (mga01.intel.com. [192.55.52.88]) by mx.google.com with ESMTPS id i9si7660357plb.35.2019.01.28.16.39.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:14 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) client-ip=192.55.52.88; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921942" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:12 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Rick Edgecombe , Masami Hiramatsu Subject: [PATCH v2 19/20] x86/kprobes: Use vmalloc special flag Date: Mon, 28 Jan 2019 16:34:21 -0800 Message-Id: <20190129003422.9328-20-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Use new flag VM_HAS_SPECIAL_PERMS for handling freeing of special permissioned memory in vmalloc and remove places where memory was set NX and RW before freeing which is no longer needed. Cc: Masami Hiramatsu Signed-off-by: Rick Edgecombe --- arch/x86/kernel/kprobes/core.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index fac692e36833..f2fab35bcb82 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -434,6 +434,7 @@ void *alloc_insn_page(void) if (page == NULL) return NULL; + set_vm_special(page); /* * First make the page read-only, and then only then make it executable * to prevent it from being W+X in between. @@ -452,12 +453,6 @@ void *alloc_insn_page(void) /* Recover page to RW mode before releasing it */ void free_insn_page(void *page) { - /* - * First make the page non-executable, and then only then make it - * writable to prevent it from being W+X in between. - */ - set_memory_nx((unsigned long)page, 1); - set_memory_rw((unsigned long)page, 1); module_memfree(page); } From patchwork Tue Jan 29 00:34:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10785145 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9742F14E1 for ; Tue, 29 Jan 2019 00:39:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 858AA2B4C4 for ; Tue, 29 Jan 2019 00:39:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 78E3E2B4D9; Tue, 29 Jan 2019 00:39:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 111812B4C4 for ; Tue, 29 Jan 2019 00:39:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 678C98E000C; Mon, 28 Jan 2019 19:39:16 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3D1B18E0011; Mon, 28 Jan 2019 19:39:16 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 04B7C8E000B; Mon, 28 Jan 2019 19:39:15 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) by kanga.kvack.org (Postfix) with ESMTP id 95D708E0010 for ; Mon, 28 Jan 2019 19:39:15 -0500 (EST) Received: by mail-pg1-f197.google.com with SMTP id r13so12656792pgb.7 for ; Mon, 28 Jan 2019 16:39:15 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=3s8AOXI4GMnsmXm4fbD4VoH1iGqOMdZAYRD6Y7Z/6yo=; b=AVkKcQJtsSFQ7jIjjPXBPnv4cEqldmSdvazsu+ZbkgGj76IwCxQEXdGymKBLgLncaG CIpP86JUkiv3t0xKNre6po/vCiiA6+e4XJIrLwZQ6cOhrrKIE1JHasGzsz8DMx4Ewifd JzTelswnK2dkefP5YnYneEb1T7F93kQ2538nWM6mwfVveR4csQbwfSEiTSx6QenBHoGP QzbZA/ZrEa1WmEA7G9/I+TSDsrtt0S78BGF+WL1v3ejj12FVHdu9r5AzQGhqbv7yKVvb lRqAK1cdeC39Z1XZkJ+9jBeVpIzanZVHOLSOzkZz5MIHaq1MEOnGtvqUwbqksrChFo/J xoiA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukelQ5VQ37voQh3oQAzWEYGPuLN5tPgYcWjoWzetLpTKFjjXhkv/ ailB9PX7lDQPANubg23UqdJMiGneVfk36XYp3ssTqCa+pHpUpqOX5piFLNL5iuFzSSHeHEjgOQc fbW3JhIbn4NiyfwL/wTRvU2+1tDf1y/bxWrw53CFCaVNjxWr81H7BEpbWzSQh5v5K/Q== X-Received: by 2002:a17:902:7044:: with SMTP id h4mr23639545plt.35.1548722355296; Mon, 28 Jan 2019 16:39:15 -0800 (PST) X-Google-Smtp-Source: ALg8bN4VQaVfarOk3iHxe8ebQTkQ1mYFWSENUZBDAJQV341v7VDKecjOSfJYZKZ/1RtvMsiNF0DA X-Received: by 2002:a17:902:7044:: with SMTP id h4mr23639495plt.35.1548722354417; Mon, 28 Jan 2019 16:39:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548722354; cv=none; d=google.com; s=arc-20160816; b=ETeYuZaKBY1oy9xPlG/MrZTQXy+J1tY+EyoBdu3AJ9PbizHVrj3SsmjPjX7CY3QuUd gQz9VmDMIarwqQUPmIRtKwmU1ppVsqe90+cvPmhiwk4iImP46kp755dL+bXpmiMoUmlr ZfLOD+OF+B+JHTvgkOK42Ilek8Uw65oAAwevDSV0KtxRAX/wSJ5yRBYOh5i+JhlOLHyt zp6iE+5tTe3u+TcCpzmLgYHlL6SvilUnG34PyMlCnWgy4g52XnNiweMA8ySOSl7XKf+X Zk3Ukc4Jz4FC4qUtCytf0t/7z2lP/Sf6pD7eN8hAQql5RhX3DzrFnT5l2KxiyCYqNZwf fdxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=3s8AOXI4GMnsmXm4fbD4VoH1iGqOMdZAYRD6Y7Z/6yo=; b=HSSP4jbcHK4pkISG3L/hCR9zL+wT3Z/5uxFwOqPk5ZnDNJcV7d79kbxruHI+elxS8t +oVMgL+hlP7XEFOvyMtyKz9M8/mUcQO3UqSHKFZbMULEwzW5WjJN+cjiir8F1d4oOoS4 RZGQmN+8gTIBOOLQ76WOaNL2wpAoi9p2HK5VXpUDKd6ojtejptijcz93av8xfsR4LCbt bzMyw8eaYDAdk9N/lrTeU7xMlWZASWtY9sCFkuUz6gtxOwZKOD/0J6fuyKPTMWf5O9xx 9bLfTNbHU06pVpNkxnChnLob7sr2QTEPnuPT9FP3QHwfrT5VPTERNDCCWgWQz+/oabBH 1AzQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga04.intel.com (mga04.intel.com. [192.55.52.120]) by mx.google.com with ESMTPS id s17si4514712pgi.513.2019.01.28.16.39.14 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 16:39:14 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) client-ip=192.55.52.120; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 16:39:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,535,1539673200"; d="scan'208";a="133921945" Received: from rpedgeco-desk5.jf.intel.com ([10.54.75.79]) by orsmga001.jf.intel.com with ESMTP; 28 Jan 2019 16:39:12 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Masami Hiramatsu , Rick Edgecombe Subject: [PATCH v2 20/20] x86/alternative: comment about module removal races Date: Mon, 28 Jan 2019 16:34:22 -0800 Message-Id: <20190129003422.9328-21-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190129003422.9328-1-rick.p.edgecombe@intel.com> References: <20190129003422.9328-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit Add a comment to clarify that users of text_poke() must ensure that no races with module removal take place. Cc: Masami Hiramatsu Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/alternative.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 81876e3ef3fd..cc3b6222857a 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -807,6 +807,11 @@ static void *__text_poke(void *addr, const void *opcode, size_t len) * It means the size must be writable atomically and the address must be aligned * in a way that permits an atomic write. It also makes sure we fit on a single * page. + * + * Note that the caller must ensure that if the modified code is part of a + * module, the module would not be removed during poking. This can be achieved + * by registering a module notifier, and ordering module removal and patching + * trough a mutex. */ void *text_poke(void *addr, const void *opcode, size_t len) {