From patchwork Wed Dec 21 14:23:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jason A. Donenfeld" X-Patchwork-Id: 13078828 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A53CBC4167B for ; Wed, 21 Dec 2022 14:23:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DD8CC8E0003; Wed, 21 Dec 2022 09:23:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D62EF8E0001; Wed, 21 Dec 2022 09:23:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BDB598E0003; Wed, 21 Dec 2022 09:23:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AFA5F8E0001 for ; Wed, 21 Dec 2022 09:23:49 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 842A9401F0 for ; Wed, 21 Dec 2022 14:23:49 +0000 (UTC) X-FDA: 80266532178.16.1761AA1 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf27.hostedemail.com (Postfix) with ESMTP id F094140012 for ; Wed, 21 Dec 2022 14:23:47 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=zx2c4.com header.s=20210105 header.b=Ao0tnENC; spf=pass (imf27.hostedemail.com: domain of "SRS0=kBoL=4T=zx2c4.com=Jason@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=kBoL=4T=zx2c4.com=Jason@kernel.org"; dmarc=pass (policy=quarantine) header.from=zx2c4.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671632628; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UiIAUwifV2YyWZQAy2FXEyRLZlyxjnNXA+vsOwxLvB8=; b=1l4itNPrkqvl54w1pTzbRHMDlkwSSWVJUjtxhHF4GBme72FWJoHhjBQwAfoN7itSccTwV6 0M8AIYgb3QPsR2OTES2Oeegl5xTxvsdsixzDoaxMqP7PXcc7Erw/PoYkZ6dYFCBcdM3c9j +MesepjQTpKovD+lMiEBaEjTVuWGYtU= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=zx2c4.com header.s=20210105 header.b=Ao0tnENC; spf=pass (imf27.hostedemail.com: domain of "SRS0=kBoL=4T=zx2c4.com=Jason@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=kBoL=4T=zx2c4.com=Jason@kernel.org"; dmarc=pass (policy=quarantine) header.from=zx2c4.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671632628; a=rsa-sha256; cv=none; b=3XNAk1vdWaZxAikSq4hsZCj6TEERiZ+M8pn4KTYrCW7SpwyI3rbEM+SQ03uxNgLn4J6ox/ +75SUimr8MH24V/iyOUbIfeWiOOi3dVp6YF8J8tH4vApr5dJc5zGMmtIZXrO7PFfMVeIec usLcILUrvoBudncwjKkpWhozJFCnhV8= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 09257617CE; Wed, 21 Dec 2022 14:23:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D176DC433F0; Wed, 21 Dec 2022 14:23:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zx2c4.com; s=20210105; t=1671632624; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UiIAUwifV2YyWZQAy2FXEyRLZlyxjnNXA+vsOwxLvB8=; b=Ao0tnENCjdlEmgrrLimO/1H4vX88RsqTMlKIUi2l1rMCPgr9NYG5elruck2BP1UO9bdMzV BGA7V1OsByS0G3q0cvPyX8oRJLctZdHRiP6emiwWXNtpYOZimRM20eEmRRNLu3W81i9RHD Lnf394xVEEdaSlqZvaVlfJZiEzIE2f8= Received: by mail.zx2c4.com (ZX2C4 Mail Server) with ESMTPSA id 95b72b9d (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Wed, 21 Dec 2022 14:23:44 +0000 (UTC) From: "Jason A. Donenfeld" To: linux-kernel@vger.kernel.org, patches@lists.linux.dev, tglx@linutronix.de Cc: "Jason A. Donenfeld" , linux-crypto@vger.kernel.org, linux-api@vger.kernel.org, x86@kernel.org, Greg Kroah-Hartman , Adhemerval Zanella Netto , Carlos O'Donell , Florian Weimer , Arnd Bergmann , Jann Horn , Christian Brauner , linux-mm@kvack.org Subject: [PATCH v13 3/7] x86: mm: Skip faulting instruction for VM_DROPPABLE faults Date: Wed, 21 Dec 2022 15:23:23 +0100 Message-Id: <20221221142327.126451-4-Jason@zx2c4.com> In-Reply-To: <20221221142327.126451-1-Jason@zx2c4.com> References: <20221221142327.126451-1-Jason@zx2c4.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: F094140012 X-Stat-Signature: bpaei97ju5wsgbouj37jahsp5efmer1w X-HE-Tag: 1671632627-221856 X-HE-Meta: U2FsdGVkX1+DvuS1cbtnOI/lFG2e6ap1TEjLt9fxZmMrnCXRBsalL8Ayj0B163GvvYRXHylMP5RuSN1N6SQhDelseW2LObbETfSCVJFkxlkhGXn1Qmu6kRJQXnmOQJ9/W1KcX5rBwUOICfFfzd/KwVC/v3Fa6STVI9dDrElxRbWtD/GglhM862KRCp1hF6CLVBVUm/D++Eu3CbFC5oVS+G9w0MKUvUmL9TENPbpgT5mV3mKUX5f57RET58T3h3X/L6ibVOjs7vCqu4NUOiZgaxJ+O+CE5IVmfj51LwcHmubcTqPgWTg6z1QwqSGJDE1iHw/URjiRNN5GzYemakSSJKZXl7t3npcpTIlIRvceOoqXVda6x+A4hgEOXt/IJ4QQ6LVQ8LA9StR6oD8N4xJajy76WgbvtD8u0bhHgQpssrrKANKCApb5iqLUE8aOVr55iXWfKh9g+wYMZ4AodmyNiqBdO6rj/HHQCTxesTLVIN5QUmxIfxMv0Mi3dahdS7vyxkTPya1Biu2jBZSUni1aVkpZ05BYgEQL5+26gvDh+ml1tOYKztv1s6VNjbK43oMvfYv/vseV15djEa3LbBDscuRK9uCt5VWzMuM5ORfQL6dNBSWXnCZLnVl5A5jyOUAq5QCzbtdKkSZhLYAI7bGtGyrVJzoeK60v8Rv6ToJW/V+X/3mtyVXm3gtzWDYLYFvjBZ0oGeqisGQuPDD4auwCPcD/quo2BvZJw+DfMvFLsFtlWmL/T8KwQ5Wt3csA5dLBWyUd75+GJVLY2eZLW43wmE8GR0SOWxC54nU2abtU+Sb6TmgrPaeSXPa+DKVxpihxgqH9WCEJf/4dE2bZOf9CgIBNMpUyU0o8kJ5OB7pSERcuJSYGt65lxww6N8mvHmNbGEST/ztCPgBg4Im2gqRv1pbFqnwsIxruUr1t0A08VpxeOEvYotHqzck40VQvMZTgSlDnnSH+YJ7yc8KuoGY ZNOFo9jQ h6L0+Sq3qf93L/z6zSsTzMatiL2PrkitYvtrONBOS7bMQgrFUSsgK1rlZtWyXd+JCEsTGd2Q1YxNUzAFveExpSDeDT8Ftvvf9e7NKauVyBdDTKi3NPAx7z2Ws/UAKKlxmh6wnPWjwjHYMBq8rbG6ZOSYpoXxCJhyvcQHwbxY8ttLztBOnJGtGdU4aNMV/+MDA9NBzalLHI0jYPthj3KPZnjN7LpxIx4+36tD0aC0qG47hicoOB/vpUuSJC4wFTvZyyMIrTYzKca8kLvoxJrN/S/QRxQWO4c68wbBNOku8jHPRWFo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The prior commit introduced VM_DROPPABLE, but in a limited form where the faulting instruction was retried instead of skipped. Finish that up with the platform-specific aspect of skipping the actual instruction. This works by copying userspace's %rip to a stack buffer of size MAX_INSN_SIZE, decoding it, and then adding the length of the decoded instruction to userspace's %rip. In the event any of these fail, just fallback to not advancing %rip and trying again. Cc: linux-mm@kvack.org Signed-off-by: Jason A. Donenfeld --- arch/x86/mm/fault.c | 19 +++++++++++++++++++ include/linux/mm_types.h | 5 ++++- mm/memory.c | 4 +++- 3 files changed, 26 insertions(+), 2 deletions(-) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 7b0d4ab894c8..76ca99ab6eb7 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -33,6 +33,8 @@ #include /* kvm_handle_async_pf */ #include /* fixup_vdso_exception() */ #include +#include /* struct insn */ +#include /* insn_fetch_from_user(), ... */ #define CREATE_TRACE_POINTS #include @@ -1454,6 +1456,23 @@ void do_user_addr_fault(struct pt_regs *regs, } mmap_read_unlock(mm); + + if (fault & VM_FAULT_SKIP_INSN) { + u8 buf[MAX_INSN_SIZE]; + struct insn insn; + int nr_copied; + + nr_copied = insn_fetch_from_user(regs, buf); + if (nr_copied <= 0) + return; + + if (!insn_decode_from_regs(&insn, regs, buf, nr_copied)) + return; + + regs->ip += insn.length; + return; + } + if (likely(!(fault & VM_FAULT_ERROR))) return; diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 3b8475007734..e76ab9ad555c 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -945,6 +945,7 @@ typedef __bitwise unsigned int vm_fault_t; * fsync() to complete (for synchronous page faults * in DAX) * @VM_FAULT_COMPLETED: ->fault completed, meanwhile mmap lock released + * @VM_FAULT_SKIP_INSN: ->handle the fault by skipping faulting instruction * @VM_FAULT_HINDEX_MASK: mask HINDEX value * */ @@ -962,6 +963,7 @@ enum vm_fault_reason { VM_FAULT_DONE_COW = (__force vm_fault_t)0x001000, VM_FAULT_NEEDDSYNC = (__force vm_fault_t)0x002000, VM_FAULT_COMPLETED = (__force vm_fault_t)0x004000, + VM_FAULT_SKIP_INSN = (__force vm_fault_t)0x008000, VM_FAULT_HINDEX_MASK = (__force vm_fault_t)0x0f0000, }; @@ -985,7 +987,8 @@ enum vm_fault_reason { { VM_FAULT_RETRY, "RETRY" }, \ { VM_FAULT_FALLBACK, "FALLBACK" }, \ { VM_FAULT_DONE_COW, "DONE_COW" }, \ - { VM_FAULT_NEEDDSYNC, "NEEDDSYNC" } + { VM_FAULT_NEEDDSYNC, "NEEDDSYNC" }, \ + { VM_FAULT_SKIP_INSN, "SKIP_INSN" } struct vm_special_mapping { const char *name; /* The name, e.g. "[vdso]". */ diff --git a/mm/memory.c b/mm/memory.c index 1ade407ccbf9..62ba9b7b713e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5221,8 +5221,10 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, lru_gen_exit_fault(); /* If the mapping is droppable, then errors due to OOM aren't fatal. */ - if (vma->vm_flags & VM_DROPPABLE) + if ((ret & VM_FAULT_OOM) && (vma->vm_flags & VM_DROPPABLE)) { ret &= ~VM_FAULT_OOM; + ret |= VM_FAULT_SKIP_INSN; + } if (flags & FAULT_FLAG_USER) { mem_cgroup_exit_user_fault();