From patchwork Sun Jan 1 16:29:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jason A. Donenfeld" X-Patchwork-Id: 13086391 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0067DC3DA7D for ; Sun, 1 Jan 2023 16:29:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 538628E0003; Sun, 1 Jan 2023 11:29:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E74E8E0001; Sun, 1 Jan 2023 11:29:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3B0DD8E0003; Sun, 1 Jan 2023 11:29:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2E0BC8E0001 for ; Sun, 1 Jan 2023 11:29:31 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id E6E21C0477 for ; Sun, 1 Jan 2023 16:29:30 +0000 (UTC) X-FDA: 80306765700.07.55B9B77 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf08.hostedemail.com (Postfix) with ESMTP id 72643160006 for ; Sun, 1 Jan 2023 16:29:29 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=zx2c4.com header.s=20210105 header.b=om7QQQFB; spf=pass (imf08.hostedemail.com: domain of "SRS0=CKF8=46=zx2c4.com=Jason@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=CKF8=46=zx2c4.com=Jason@kernel.org"; dmarc=pass (policy=quarantine) header.from=zx2c4.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1672590569; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UiIAUwifV2YyWZQAy2FXEyRLZlyxjnNXA+vsOwxLvB8=; b=HSyLGUdhrmbp04v2bCIy8cq1zLIHXGTYz68NWF5B/v/CzwBj6AzCZCPLfz0nWlj1LU4Dk1 4gSodH7Tj8OuKtletMEiINtIBVp5EvUPjPgicViafjZo2R6+xQz1cAs3ZW3D2Q6T9Fgt4/ snyTOdwNp381mcJVdHFJZ0IufbLVKkM= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=zx2c4.com header.s=20210105 header.b=om7QQQFB; spf=pass (imf08.hostedemail.com: domain of "SRS0=CKF8=46=zx2c4.com=Jason@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=CKF8=46=zx2c4.com=Jason@kernel.org"; dmarc=pass (policy=quarantine) header.from=zx2c4.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1672590569; a=rsa-sha256; cv=none; b=7RHPOLaUQdEuCRHzA0aEvQboAHBZ7dCkazTzN7C9EAG8lo2P4g7sbyMeyuzikib9R8ZbQb P+JJ/HT+pcs6d6O3fwP2JpdOU3twM9ap08UtkDPhbXoGqqciXT1wb1kB/geCTFZj1npMwu wwmKq99JrHADSpzcYnqRcxPwgoP5YYM= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8A14B60DE3; Sun, 1 Jan 2023 16:29:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 76B7DC433EF; Sun, 1 Jan 2023 16:29:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zx2c4.com; s=20210105; t=1672590565; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UiIAUwifV2YyWZQAy2FXEyRLZlyxjnNXA+vsOwxLvB8=; b=om7QQQFBT0KIy48f0KIKF7IKAx6VXrDKB5ghc0Ty4WM4REmdc2tCRbBqwkb+bXoj+j4qHI lrqqFhk7hKI2iApw3EOUuGvBFiIosMuCQP2UgTgetshNqU3hf9nBvObellB6TbG5HHNd9C eWP4eAwsjO4gzu7PZAR6aJ0dvW47BGg= Received: by mail.zx2c4.com (ZX2C4 Mail Server) with ESMTPSA id 7c051842 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sun, 1 Jan 2023 16:29:25 +0000 (UTC) From: "Jason A. Donenfeld" To: linux-kernel@vger.kernel.org, patches@lists.linux.dev, tglx@linutronix.de Cc: "Jason A. Donenfeld" , linux-crypto@vger.kernel.org, linux-api@vger.kernel.org, x86@kernel.org, Greg Kroah-Hartman , Adhemerval Zanella Netto , Carlos O'Donell , Florian Weimer , Arnd Bergmann , Jann Horn , Christian Brauner , linux-mm@kvack.org Subject: [PATCH v14 3/7] x86: mm: Skip faulting instruction for VM_DROPPABLE faults Date: Sun, 1 Jan 2023 17:29:06 +0100 Message-Id: <20230101162910.710293-4-Jason@zx2c4.com> In-Reply-To: <20230101162910.710293-1-Jason@zx2c4.com> References: <20230101162910.710293-1-Jason@zx2c4.com> MIME-Version: 1.0 X-Stat-Signature: ax6ie6qhhwd9kfo6gsd4qt5hdk7kwwjt X-Rspam-User: X-Rspamd-Queue-Id: 72643160006 X-Rspamd-Server: rspam06 X-HE-Tag: 1672590569-745849 X-HE-Meta: U2FsdGVkX18dzAKqQu0A3Bc2BE79y3sLV4AeO4JBu/Rw27GtpfG/dP3Lj0ryyUUxhRdR1ARo/c2u9D3HfU4YHrnehp9v1g698GKJq07w5JZLG97H2FbrnHzv1VmE/82SqJQ0DDCw9Dohs93xVmDTf/mZm1CeZlcbUKJvajmpAUE7bbk0UGM1P89qsZ+qbyQV2EP5w/1B1xkSIX5Do9CIkb+wOcsLRnullQw4zGVq36Q9wAI98yViCDp99fyoelQnjJtyXVo9sl0ZOzx1HguoPTpE8b6uTiikmZ6gMkkUnWPEqIYF87pt/NxUcHn+cF/Mcek/reY0DqlxgHGLU0piR4MUMgantoyOvtbSaRQzx+V4jHhAU6lJWxGoKFcZfzUQXUHMhFYamMj1e/KEnEN4nR4doImvKRYPvS4s2iRqyhLC8gZLEmOlcD9YzVATBtAFE2jbHDNDZrkjc45D7TrneAapOgpfk6Fu3/jTSViQgSS7qprB6QhvqiN5l6RY/Bk+J+oSFTcwGXAbRIsE+4w1cqlyMIDufjSDMiW6gDVkjnrDcXeeZsjllu3VRz2WF6dsCTdS+s+qE9obV/ZH2mGDD1XsLRXh21jG7gi5KIMw/vb326XxnQjd6ezYrxxX2+oSDYY6ksP9JQvlMCIBDR8geZJfmKvJ5B5vDpQSt+vJP2EMglxAHd7w0BC5lyunTbHdlf85LCK52e++PkxqJaMLTxodshA3Tqzz4ZBJXAMF26z07gIF7nCGT+LdK1d7cIpYuxo5yrtqkg7gUyo25DQKOb6nL2WlcMR98mlOU3bkA5wCsS7Zl3MMjm83O1eQOL5NsiEwYzjh5R0sPE7A9I/bKTU3uW25eEuh51MtPgsykgeBu9Crgc7lGkVM1pV1I9mlBuoj6gmLfCCgsRS4IXh1rYAFd+Mac4HGEDuB/7R+bBkI3ZhPQ8irVk/2LsMaZpZ06ZE9zxNzQG0NzwEin23 ETPxBl6I aajBqUlzLckUkh67IqnI1t4Xak405dRg9+P4b0v/lwGmJMsnXGyUMCEHmzGa56a7oAnfpUuXczF2WjtxLoE92qr/YnlfueCpEWJ9AIF4V62vVd7IWwg53j+d/rjyi4F87Clf+BFQMluE9ANxDxAAVeF//zZ47YYp8VB7ySvoiP/oRmvG/n2kTKUoKV3HCa52qj2OozdTom0y+eXbqvaDdRs4GJ3s50zAKeQig9WSDfs460ND7YX5+/IdsjsCCmeL83u/U3q3iJE6Amr1X45nLM/A7MdsRi3BkYxDiRWOnZHdJ3OU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The prior commit introduced VM_DROPPABLE, but in a limited form where the faulting instruction was retried instead of skipped. Finish that up with the platform-specific aspect of skipping the actual instruction. This works by copying userspace's %rip to a stack buffer of size MAX_INSN_SIZE, decoding it, and then adding the length of the decoded instruction to userspace's %rip. In the event any of these fail, just fallback to not advancing %rip and trying again. Cc: linux-mm@kvack.org Signed-off-by: Jason A. Donenfeld --- arch/x86/mm/fault.c | 19 +++++++++++++++++++ include/linux/mm_types.h | 5 ++++- mm/memory.c | 4 +++- 3 files changed, 26 insertions(+), 2 deletions(-) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 7b0d4ab894c8..76ca99ab6eb7 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -33,6 +33,8 @@ #include /* kvm_handle_async_pf */ #include /* fixup_vdso_exception() */ #include +#include /* struct insn */ +#include /* insn_fetch_from_user(), ... */ #define CREATE_TRACE_POINTS #include @@ -1454,6 +1456,23 @@ void do_user_addr_fault(struct pt_regs *regs, } mmap_read_unlock(mm); + + if (fault & VM_FAULT_SKIP_INSN) { + u8 buf[MAX_INSN_SIZE]; + struct insn insn; + int nr_copied; + + nr_copied = insn_fetch_from_user(regs, buf); + if (nr_copied <= 0) + return; + + if (!insn_decode_from_regs(&insn, regs, buf, nr_copied)) + return; + + regs->ip += insn.length; + return; + } + if (likely(!(fault & VM_FAULT_ERROR))) return; diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 3b8475007734..e76ab9ad555c 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -945,6 +945,7 @@ typedef __bitwise unsigned int vm_fault_t; * fsync() to complete (for synchronous page faults * in DAX) * @VM_FAULT_COMPLETED: ->fault completed, meanwhile mmap lock released + * @VM_FAULT_SKIP_INSN: ->handle the fault by skipping faulting instruction * @VM_FAULT_HINDEX_MASK: mask HINDEX value * */ @@ -962,6 +963,7 @@ enum vm_fault_reason { VM_FAULT_DONE_COW = (__force vm_fault_t)0x001000, VM_FAULT_NEEDDSYNC = (__force vm_fault_t)0x002000, VM_FAULT_COMPLETED = (__force vm_fault_t)0x004000, + VM_FAULT_SKIP_INSN = (__force vm_fault_t)0x008000, VM_FAULT_HINDEX_MASK = (__force vm_fault_t)0x0f0000, }; @@ -985,7 +987,8 @@ enum vm_fault_reason { { VM_FAULT_RETRY, "RETRY" }, \ { VM_FAULT_FALLBACK, "FALLBACK" }, \ { VM_FAULT_DONE_COW, "DONE_COW" }, \ - { VM_FAULT_NEEDDSYNC, "NEEDDSYNC" } + { VM_FAULT_NEEDDSYNC, "NEEDDSYNC" }, \ + { VM_FAULT_SKIP_INSN, "SKIP_INSN" } struct vm_special_mapping { const char *name; /* The name, e.g. "[vdso]". */ diff --git a/mm/memory.c b/mm/memory.c index 1ade407ccbf9..62ba9b7b713e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5221,8 +5221,10 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, lru_gen_exit_fault(); /* If the mapping is droppable, then errors due to OOM aren't fatal. */ - if (vma->vm_flags & VM_DROPPABLE) + if ((ret & VM_FAULT_OOM) && (vma->vm_flags & VM_DROPPABLE)) { ret &= ~VM_FAULT_OOM; + ret |= VM_FAULT_SKIP_INSN; + } if (flags & FAULT_FLAG_USER) { mem_cgroup_exit_user_fault();