From patchwork Fri Mar 26 00:02:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Luck X-Patchwork-Id: 12165201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C418BC433C1 for ; Fri, 26 Mar 2021 00:02:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 51B24619F8 for ; Fri, 26 Mar 2021 00:02:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 51B24619F8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6780B6B0074; Thu, 25 Mar 2021 20:02:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5F29F6B007D; Thu, 25 Mar 2021 20:02:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 37B1D6B0075; Thu, 25 Mar 2021 20:02:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0027.hostedemail.com [216.40.44.27]) by kanga.kvack.org (Postfix) with ESMTP id 123FA6B0075 for ; Thu, 25 Mar 2021 20:02:51 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id D01C3180ACF7F for ; Fri, 26 Mar 2021 00:02:50 +0000 (UTC) X-FDA: 77960074500.07.954FE7D Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf04.hostedemail.com (Postfix) with ESMTP id 8E320132 for ; Fri, 26 Mar 2021 00:02:48 +0000 (UTC) IronPort-SDR: Kx844VQeasu6yDrWZ+RDeHAPrUtgDAgvD7PKTuK/gIrbiBEt6DpGPKZUC6suUp6XMU8XRrv5FR rboZZxpa3Zjg== X-IronPort-AV: E=McAfee;i="6000,8403,9934"; a="276171510" X-IronPort-AV: E=Sophos;i="5.81,278,1610438400"; d="scan'208";a="276171510" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Mar 2021 17:02:43 -0700 IronPort-SDR: jsHsDEqu/weqLP3OJ95iv8Mg/NMIUrDTLzjsPL6wiIDNiqHEwgIydXYrMNenNt/r/BKjP90xK8 9TqF+unw3HcQ== X-IronPort-AV: E=Sophos;i="5.81,278,1610438400"; d="scan'208";a="416265834" Received: from agluck-desk2.sc.intel.com ([10.3.52.146]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Mar 2021 17:02:43 -0700 From: Tony Luck To: Borislav Petkov Cc: Tony Luck , x86@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andy Lutomirski , Aili Yao , =?utf-8?b?SE9SSUdVQ0hJIE5BT1lBKCDloIDlj6PjgIDnm7TkuZ8p?= Subject: [PATCH 1/4] x86/mce: Fix copyin code to return -EFAULT on machine check. Date: Thu, 25 Mar 2021 17:02:32 -0700 Message-Id: <20210326000235.370514-2-tony.luck@intel.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210326000235.370514-1-tony.luck@intel.com> References: <20210326000235.370514-1-tony.luck@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 8E320132 X-Stat-Signature: tcgmwtnd6af7r9ydz3bkxp97jtk59w5f Received-SPF: none (intel.com>: No applicable sender policy available) receiver=imf04; identity=mailfrom; envelope-from=""; helo=mga05.intel.com; client-ip=192.55.52.43 X-HE-DKIM-Result: none/none X-HE-Tag: 1616716968-550013 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When copy from user fails due to a machine check on poison reading user data it should return an error code. --- Separate patch just now, but likely needs to be combined with patches to iteration code for bisection safety. --- arch/x86/lib/copy_user_64.S | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/arch/x86/lib/copy_user_64.S b/arch/x86/lib/copy_user_64.S index 77b9b2a3b5c8..2987118c541a 100644 --- a/arch/x86/lib/copy_user_64.S +++ b/arch/x86/lib/copy_user_64.S @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -237,18 +238,21 @@ SYM_CODE_START_LOCAL(.Lcopy_user_handle_tail) cmp $X86_TRAP_MC,%eax /* check if X86_TRAP_MC */ je 3f 1: rep movsb -2: mov %ecx,%eax + mov %ecx,%eax + ASM_CLAC + ret + +2: + cmp $X86_TRAP_MC,%eax + je 3f + mov %ecx,%eax ASM_CLAC ret /* - * Return zero to pretend that this copy succeeded. This - * is counter-intuitive, but needed to prevent the code - * in lib/iov_iter.c from retrying and running back into - * the poison cache line again. The machine check handler - * will ensure that a SIGBUS is sent to the task. + * Return -EFAULT for the machine check cases */ -3: xorl %eax,%eax +3: movl $-EFAULT,%eax ASM_CLAC ret From patchwork Fri Mar 26 00:02:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Luck X-Patchwork-Id: 12165199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACD85C433E1 for ; Fri, 26 Mar 2021 00:02:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3E5D561A0A for ; Fri, 26 Mar 2021 00:02:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3E5D561A0A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 41D7D6B0078; Thu, 25 Mar 2021 20:02:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3F6686B0074; Thu, 25 Mar 2021 20:02:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 221056B007B; Thu, 25 Mar 2021 20:02:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0126.hostedemail.com [216.40.44.126]) by kanga.kvack.org (Postfix) with ESMTP id 0187C6B0074 for ; Thu, 25 Mar 2021 20:02:50 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C0F656129 for ; Fri, 26 Mar 2021 00:02:50 +0000 (UTC) X-FDA: 77960074500.17.4410628 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf15.hostedemail.com (Postfix) with ESMTP id 2CC2BA000251 for ; Fri, 26 Mar 2021 00:02:47 +0000 (UTC) IronPort-SDR: krEatQuSqb6So0GktLBnlokCdXKP8aK7PPvE4mia7EXxOl4QYpNcgi4AaC8fUUzieGF8hro+Lb BwFzD+ZbCRQA== X-IronPort-AV: E=McAfee;i="6000,8403,9934"; a="276171512" X-IronPort-AV: E=Sophos;i="5.81,278,1610438400"; d="scan'208";a="276171512" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Mar 2021 17:02:44 -0700 IronPort-SDR: wN2owymOr8LuNbROqRY5grqOI0GqflWynmkq92DqloOFlO9DkdfPS4k+ss2TfBJymhpfMtG3Gn hBOppYjcZfwg== X-IronPort-AV: E=Sophos;i="5.81,278,1610438400"; d="scan'208";a="416265839" Received: from agluck-desk2.sc.intel.com ([10.3.52.146]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Mar 2021 17:02:43 -0700 From: Tony Luck To: Borislav Petkov Cc: Tony Luck , x86@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andy Lutomirski , Aili Yao , =?utf-8?b?SE9SSUdVQ0hJIE5BT1lBKCDloIDlj6PjgIDnm7TkuZ8p?= Subject: [PATCH 2/4] mce/iter: Check for copyin failure & return error up stack Date: Thu, 25 Mar 2021 17:02:33 -0700 Message-Id: <20210326000235.370514-3-tony.luck@intel.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210326000235.370514-1-tony.luck@intel.com> References: <20210326000235.370514-1-tony.luck@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 2CC2BA000251 X-Stat-Signature: x3oeotttp9m1eufwftdhrfy8xtjqq6wu Received-SPF: none (intel.com>: No applicable sender policy available) receiver=imf15; identity=mailfrom; envelope-from=""; helo=mga05.intel.com; client-ip=192.55.52.43 X-HE-DKIM-Result: none/none X-HE-Tag: 1616716967-443083 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Check for failure from low level copy from user code. Doing this requires some type changes from the unsigned "size_t" so some signed type (so that "if (xxx < 0)" works!). I picked "loff_t" but there may be some other more appropriate type. Very likely more places need to be changed. These changes based on a machine check copying user data for a write(2) system call to an xfs file system where the stack trace looks like: [ 149.445163] ? copyin+0x2d/0x40 [ 149.448687] ? iov_iter_copy_from_user_atomic+0xd6/0x3a0 [ 149.454625] ? iomap_write_actor+0xc7/0x190 [ 149.459319] ? iomap_apply+0x12a/0x440 [ 149.463508] ? iomap_write_begin+0x530/0x530 [ 149.468276] ? iomap_write_begin+0x530/0x530 [ 149.473041] ? iomap_file_buffered_write+0x62/0x90 [ 149.478390] ? iomap_write_begin+0x530/0x530 [ 149.483157] ? xfs_file_buffered_write+0xe0/0x340 [xfs] [ 149.489393] ? new_sync_write+0x122/0x1b0 [ 149.493879] ? vfs_write+0x20a/0x350 [ 149.497888] ? ksys_write+0x5f/0xe0 [ 149.501784] ? do_syscall_64+0x33/0x40 --- Needs to be bundled with other patches for bisection safety --- fs/iomap/buffered-io.c | 8 +++++++- include/linux/uio.h | 2 +- lib/iov_iter.c | 15 +++++++++++++-- 3 files changed, 21 insertions(+), 4 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 414769a6ad11..6cc28e3f32ff 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -757,7 +757,7 @@ iomap_write_actor(struct inode *inode, loff_t pos, loff_t length, void *data, struct page *page; unsigned long offset; /* Offset into pagecache page */ unsigned long bytes; /* Bytes to write to page */ - size_t copied; /* Bytes copied from user */ + loff_t copied; /* Bytes copied from user */ offset = offset_in_page(pos); bytes = min_t(unsigned long, PAGE_SIZE - offset, @@ -791,6 +791,12 @@ iomap_write_actor(struct inode *inode, loff_t pos, loff_t length, void *data, copied = iov_iter_copy_from_user_atomic(page, i, offset, bytes); + if (copied < 0) { + unlock_page(page); + put_page(page); + return copied; + } + copied = iomap_write_end(inode, pos, bytes, copied, page, iomap, srcmap); diff --git a/include/linux/uio.h b/include/linux/uio.h index 27ff8eb786dc..c2d82e8e309c 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -111,7 +111,7 @@ static inline struct iovec iov_iter_iovec(const struct iov_iter *iter) }; } -size_t iov_iter_copy_from_user_atomic(struct page *page, +loff_t iov_iter_copy_from_user_atomic(struct page *page, struct iov_iter *i, unsigned long offset, size_t bytes); void iov_iter_advance(struct iov_iter *i, size_t bytes); void iov_iter_revert(struct iov_iter *i, size_t bytes); diff --git a/lib/iov_iter.c b/lib/iov_iter.c index f66c62aa7154..57e2f81c51ee 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -16,13 +16,18 @@ #define PIPE_PARANOIA /* for now */ #define iterate_iovec(i, n, __v, __p, skip, STEP) { \ - size_t left; \ + loff_t left; \ size_t wanted = n; \ __p = i->iov; \ __v.iov_len = min(n, __p->iov_len - skip); \ if (likely(__v.iov_len)) { \ __v.iov_base = __p->iov_base + skip; \ left = (STEP); \ + if (left < 0) { \ + wanted = left; \ + n = 0; \ + goto err; \ + } \ __v.iov_len -= left; \ skip += __v.iov_len; \ n -= __v.iov_len; \ @@ -36,10 +41,16 @@ continue; \ __v.iov_base = __p->iov_base; \ left = (STEP); \ + if (left < 0) { \ + wanted = left; \ + n = 0; \ + break; \ + } \ __v.iov_len -= left; \ skip = __v.iov_len; \ n -= __v.iov_len; \ } \ +err: \ n = wanted - n; \ } @@ -975,7 +986,7 @@ size_t iov_iter_zero(size_t bytes, struct iov_iter *i) } EXPORT_SYMBOL(iov_iter_zero); -size_t iov_iter_copy_from_user_atomic(struct page *page, +loff_t iov_iter_copy_from_user_atomic(struct page *page, struct iov_iter *i, unsigned long offset, size_t bytes) { char *kaddr = kmap_atomic(page), *p = kaddr + offset; From patchwork Fri Mar 26 00:02:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Luck X-Patchwork-Id: 12165203 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB6ADC433E0 for ; Fri, 26 Mar 2021 00:02:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 432FE619FE for ; Fri, 26 Mar 2021 00:02:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 432FE619FE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 444D46B0075; Thu, 25 Mar 2021 20:02:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 41EBB6B007B; Thu, 25 Mar 2021 20:02:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24AC16B007D; Thu, 25 Mar 2021 20:02:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0192.hostedemail.com [216.40.44.192]) by kanga.kvack.org (Postfix) with ESMTP id E7D836B0075 for ; Thu, 25 Mar 2021 20:02:51 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id ACBA21DE5 for ; Fri, 26 Mar 2021 00:02:51 +0000 (UTC) X-FDA: 77960074542.16.42FACB0 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf11.hostedemail.com (Postfix) with ESMTP id 1FA0720001D6 for ; Fri, 26 Mar 2021 00:02:47 +0000 (UTC) IronPort-SDR: Z3RCaCGYnDILnEHiaSN6ky5VCILcU5+AA/K1I77EDLytP1ldpcnra47bIl8mRZUwt6Gp53x/x6 PV2g8IuHT0wg== X-IronPort-AV: E=McAfee;i="6000,8403,9934"; a="276171513" X-IronPort-AV: E=Sophos;i="5.81,278,1610438400"; d="scan'208";a="276171513" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Mar 2021 17:02:44 -0700 IronPort-SDR: 6M0k84g+CKYIoUlN3FSrybSxkhZDcKiZliPuopz+OPLLsK/nBPB6BnJGO/1bIf3CYOarYuVJEk r1lQxPJkzKXA== X-IronPort-AV: E=Sophos;i="5.81,278,1610438400"; d="scan'208";a="416265843" Received: from agluck-desk2.sc.intel.com ([10.3.52.146]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Mar 2021 17:02:43 -0700 From: Tony Luck To: Borislav Petkov Cc: Tony Luck , x86@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andy Lutomirski , Aili Yao , =?utf-8?b?SE9SSUdVQ0hJIE5BT1lBKCDloIDlj6PjgIDnm7TkuZ8p?= Subject: [PATCH 3/4] mce/copyin: fix to not SIGBUS when copying from user hits poison Date: Thu, 25 Mar 2021 17:02:34 -0700 Message-Id: <20210326000235.370514-4-tony.luck@intel.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210326000235.370514-1-tony.luck@intel.com> References: <20210326000235.370514-1-tony.luck@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 1FA0720001D6 X-Stat-Signature: 8xb5ykbeaxgtp3prsyk4jyskp5ddn6ea Received-SPF: none (intel.com>: No applicable sender policy available) receiver=imf11; identity=mailfrom; envelope-from=""; helo=mga05.intel.com; client-ip=192.55.52.43 X-HE-DKIM-Result: none/none X-HE-Tag: 1616716967-953018 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Andy Lutomirski pointed out that sending SIGBUS to tasks that hit poison in the kernel copying syscall parameters from user address space is not the right semantic. So stop doing that. Add a new kill_me_never() call back that simply unmaps and offlines the poison page. current-mce_vaddr is no longer used, so drop this field --- Needs to be combined with other patches for bisectability --- arch/x86/kernel/cpu/mce/core.c | 35 ++++++++++++++++-------------- arch/x86/kernel/cpu/mce/severity.c | 2 -- include/linux/sched.h | 1 - 3 files changed, 19 insertions(+), 19 deletions(-) diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c index 7962355436da..1570310cadab 100644 --- a/arch/x86/kernel/cpu/mce/core.c +++ b/arch/x86/kernel/cpu/mce/core.c @@ -1263,32 +1263,32 @@ static void kill_me_maybe(struct callback_head *cb) if (!p->mce_ripv) flags |= MF_MUST_KILL; - if (!memory_failure(p->mce_addr >> PAGE_SHIFT, flags) && - !(p->mce_kflags & MCE_IN_KERNEL_COPYIN)) { + if (!memory_failure(p->mce_addr >> PAGE_SHIFT, flags)) { set_mce_nospec(p->mce_addr >> PAGE_SHIFT, p->mce_whole_page); sync_core(); return; } - if (p->mce_vaddr != (void __user *)-1l) { - force_sig_mceerr(BUS_MCEERR_AR, p->mce_vaddr, PAGE_SHIFT); - } else { - pr_err("Memory error not recovered"); - kill_me_now(cb); - } + pr_err("Memory error not recovered"); + kill_me_now(cb); } -static void queue_task_work(struct mce *m, int kill_current_task) +static void kill_me_never(struct callback_head *cb) +{ + struct task_struct *p = container_of(cb, struct task_struct, mce_kill_me); + + pr_err("Kernel accessed poison in user space at %llx\n", p->mce_addr); + if (!memory_failure(p->mce_addr >> PAGE_SHIFT, 0)) + set_mce_nospec(p->mce_addr >> PAGE_SHIFT, p->mce_whole_page); +} + +static void queue_task_work(struct mce *m, void (*func)(struct callback_head *)) { current->mce_addr = m->addr; current->mce_kflags = m->kflags; current->mce_ripv = !!(m->mcgstatus & MCG_STATUS_RIPV); current->mce_whole_page = whole_page(m); - - if (kill_current_task) - current->mce_kill_me.func = kill_me_now; - else - current->mce_kill_me.func = kill_me_maybe; + current->mce_kill_me.func = func; task_work_add(current, ¤t->mce_kill_me, TWA_RESUME); } @@ -1426,7 +1426,10 @@ noinstr void do_machine_check(struct pt_regs *regs) /* If this triggers there is no way to recover. Die hard. */ BUG_ON(!on_thread_stack() || !user_mode(regs)); - queue_task_work(&m, kill_current_task); + if (kill_current_task) + queue_task_work(&m, kill_me_now); + else + queue_task_work(&m, kill_me_maybe); } else { /* @@ -1444,7 +1447,7 @@ noinstr void do_machine_check(struct pt_regs *regs) } if (m.kflags & MCE_IN_KERNEL_COPYIN) - queue_task_work(&m, kill_current_task); + queue_task_work(&m, kill_me_never); } out: mce_wrmsrl(MSR_IA32_MCG_STATUS, 0); diff --git a/arch/x86/kernel/cpu/mce/severity.c b/arch/x86/kernel/cpu/mce/severity.c index 83df991314c5..47810d12f040 100644 --- a/arch/x86/kernel/cpu/mce/severity.c +++ b/arch/x86/kernel/cpu/mce/severity.c @@ -251,8 +251,6 @@ static bool is_copy_from_user(struct pt_regs *regs) if (fault_in_kernel_space(addr)) return false; - current->mce_vaddr = (void __user *)addr; - return true; } diff --git a/include/linux/sched.h b/include/linux/sched.h index ef00bb22164c..2d213b52730c 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1358,7 +1358,6 @@ struct task_struct { #endif #ifdef CONFIG_X86_MCE - void __user *mce_vaddr; __u64 mce_kflags; u64 mce_addr; __u64 mce_ripv : 1, From patchwork Fri Mar 26 00:02:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Luck X-Patchwork-Id: 12165195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4C8FC433DB for ; Fri, 26 Mar 2021 00:02:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5EE07619F8 for ; Fri, 26 Mar 2021 00:02:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5EE07619F8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D92626B0072; Thu, 25 Mar 2021 20:02:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D69A16B0073; Thu, 25 Mar 2021 20:02:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE3DD6B0074; Thu, 25 Mar 2021 20:02:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0005.hostedemail.com [216.40.44.5]) by kanga.kvack.org (Postfix) with ESMTP id A57426B0072 for ; Thu, 25 Mar 2021 20:02:49 -0400 (EDT) Received: from smtpin33.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 6A67D180ACF7F for ; Fri, 26 Mar 2021 00:02:49 +0000 (UTC) X-FDA: 77960074458.33.D8FDDE8 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf15.hostedemail.com (Postfix) with ESMTP id CACFAA00024B for ; Fri, 26 Mar 2021 00:02:45 +0000 (UTC) IronPort-SDR: 9dAJy4Lb2h4v07MxKTmeZGBDuxf3orvekGomU5LlUJuPNo0ISjTj7j3zJAByufJs94G/ea/ka1 eu4G4cXl+UIA== X-IronPort-AV: E=McAfee;i="6000,8403,9934"; a="276171514" X-IronPort-AV: E=Sophos;i="5.81,278,1610438400"; d="scan'208";a="276171514" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Mar 2021 17:02:44 -0700 IronPort-SDR: 381eQXWHSCu6mbpZPx3KOQD5wTsa+RnwhGNbuhw/jc5ma2bOZKWInhprjdOs1buwQMYVeAObVq ymmK1r5NISPA== X-IronPort-AV: E=Sophos;i="5.81,278,1610438400"; d="scan'208";a="416265846" Received: from agluck-desk2.sc.intel.com ([10.3.52.146]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Mar 2021 17:02:43 -0700 From: Tony Luck To: Borislav Petkov Cc: Tony Luck , x86@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andy Lutomirski , Aili Yao , =?utf-8?b?SE9SSUdVQ0hJIE5BT1lBKCDloIDlj6PjgIDnm7TkuZ8p?= Subject: [PATCH 4/4] x86/mce: Avoid infinite loop for copy from user recovery Date: Thu, 25 Mar 2021 17:02:35 -0700 Message-Id: <20210326000235.370514-5-tony.luck@intel.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210326000235.370514-1-tony.luck@intel.com> References: <20210326000235.370514-1-tony.luck@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: CACFAA00024B X-Stat-Signature: 17kh8ku6bnc4tqsbo7yfm1ctgtsm3i7n Received-SPF: none (intel.com>: No applicable sender policy available) receiver=imf15; identity=mailfrom; envelope-from=""; helo=mga05.intel.com; client-ip=192.55.52.43 X-HE-DKIM-Result: none/none X-HE-Tag: 1616716965-506718 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Recovery action when get_user() triggers a machine check uses the fixup path to make get_user() return -EFAULT. Also queue_task_work() sets up so that kill_me_maybe() will be called on return to user mode to send a SIGBUS to the current process. But there are places in the kernel where the code assumes that this EFAULT return was simply because of a page fault. The code takes some action to fix that, and then retries the access. This results in a second machine check. While processing this second machine check queue_task_work() is called again. But since this uses the same callback_head structure that was used in the first call, the net result is an entry on the current->task_works list that points to itself. When task_work_run() is called it loops forever in this code: do { next = work->next; work->func(work); work = next; cond_resched(); } while (work); Add a counter (current->mce_count) to keep track of repeated machine checks before task_work() is called. First machine check saves the address information and calls task_work_add(). Subsequent machine checks before that task_work call back is executed check that the address is in the same page as the first machine check (since the callback will offline exactly one page). Expected worst case is two machine checks before moving on (e.g. one user access with page faults disabled, then a repeat to the same addrsss with page faults enabled). Just in case there is some code that loops forever enforce a limit of 10. Signed-off-by: Tony Luck --- arch/x86/kernel/cpu/mce/core.c | 40 ++++++++++++++++++++++++++-------- include/linux/sched.h | 1 + 2 files changed, 32 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c index 1570310cadab..999fd7f0330b 100644 --- a/arch/x86/kernel/cpu/mce/core.c +++ b/arch/x86/kernel/cpu/mce/core.c @@ -1250,6 +1250,9 @@ static void __mc_scan_banks(struct mce *m, struct pt_regs *regs, struct mce *fin static void kill_me_now(struct callback_head *ch) { + struct task_struct *p = container_of(ch, struct task_struct, mce_kill_me); + + p->mce_count = 0; force_sig(SIGBUS); } @@ -1258,6 +1261,7 @@ static void kill_me_maybe(struct callback_head *cb) struct task_struct *p = container_of(cb, struct task_struct, mce_kill_me); int flags = MF_ACTION_REQUIRED; + p->mce_count = 0; pr_err("Uncorrected hardware memory error in user-access at %llx", p->mce_addr); if (!p->mce_ripv) @@ -1277,18 +1281,36 @@ static void kill_me_never(struct callback_head *cb) { struct task_struct *p = container_of(cb, struct task_struct, mce_kill_me); + p->mce_count = 0; pr_err("Kernel accessed poison in user space at %llx\n", p->mce_addr); if (!memory_failure(p->mce_addr >> PAGE_SHIFT, 0)) set_mce_nospec(p->mce_addr >> PAGE_SHIFT, p->mce_whole_page); } -static void queue_task_work(struct mce *m, void (*func)(struct callback_head *)) +static void queue_task_work(struct mce *m, char *msg, void (*func)(struct callback_head *)) { - current->mce_addr = m->addr; - current->mce_kflags = m->kflags; - current->mce_ripv = !!(m->mcgstatus & MCG_STATUS_RIPV); - current->mce_whole_page = whole_page(m); - current->mce_kill_me.func = func; + int count = ++current->mce_count; + + /* First call, save all the details */ + if (count == 1) { + current->mce_addr = m->addr; + current->mce_kflags = m->kflags; + current->mce_ripv = !!(m->mcgstatus & MCG_STATUS_RIPV); + current->mce_whole_page = whole_page(m); + current->mce_kill_me.func = func; + } + + /* Ten is likley overkill. Don't expect more than two faults before task_work() */ + if (count > 10) + mce_panic("Too many machine checks while accessing user data", m, msg); + + /* Second or later call, make sure page address matches the one from first call */ + if (count > 1 && (current->mce_addr >> PAGE_SHIFT) != (m->addr >> PAGE_SHIFT)) + mce_panic("Machine checks to different user pages", m, msg); + + /* Do not call task_work_add() more than once */ + if (count > 1) + return; task_work_add(current, ¤t->mce_kill_me, TWA_RESUME); } @@ -1427,9 +1449,9 @@ noinstr void do_machine_check(struct pt_regs *regs) BUG_ON(!on_thread_stack() || !user_mode(regs)); if (kill_current_task) - queue_task_work(&m, kill_me_now); + queue_task_work(&m, msg, kill_me_now); else - queue_task_work(&m, kill_me_maybe); + queue_task_work(&m, msg, kill_me_maybe); } else { /* @@ -1447,7 +1469,7 @@ noinstr void do_machine_check(struct pt_regs *regs) } if (m.kflags & MCE_IN_KERNEL_COPYIN) - queue_task_work(&m, kill_me_never); + queue_task_work(&m, msg, kill_me_never); } out: mce_wrmsrl(MSR_IA32_MCG_STATUS, 0); diff --git a/include/linux/sched.h b/include/linux/sched.h index 2d213b52730c..8f9dc91498cf 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1364,6 +1364,7 @@ struct task_struct { mce_whole_page : 1, __mce_reserved : 62; struct callback_head mce_kill_me; + int mce_count; #endif #ifdef CONFIG_KRETPROBES