From patchwork Fri Nov 4 22:35:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 13032641 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50829C3A5A3 for ; Fri, 4 Nov 2022 22:39:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E29606B0075; Fri, 4 Nov 2022 18:39:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D8A226B0078; Fri, 4 Nov 2022 18:39:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B8E4F8E0001; Fri, 4 Nov 2022 18:39:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A3E466B0075 for ; Fri, 4 Nov 2022 18:39:29 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 7F97AAC1A6 for ; Fri, 4 Nov 2022 22:39:29 +0000 (UTC) X-FDA: 80097227658.30.BCB88C5 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by imf03.hostedemail.com (Postfix) with ESMTP id B451920004 for ; Fri, 4 Nov 2022 22:39:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667601569; x=1699137569; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=4/pxGAyl97+5ZD5/gTFtyEZLTubfAza59sVFqeoJNMY=; b=KUJzmhjFPiouVAA/GZDVisDKzh7DvLOvZ24MKe08NjBHCMQY8nP0fb6X lKN2ybNkCSeoe2h4ClOWwmKCtYGZLkyGz0ldrJ9kgyaFaECEsB3d3qtXL cn4XgskyLFj0BGQur0EcAPAJB+HNkHuCG9U0n3pWkVcTXBYfy9XVM4CtK RaMaKpLfHnOsZoVaUxCDYhci/Q9ulycA6mYjvqY2vty6pufYTNjprPeW6 mNB4jqY0oBYbf0i5BGSk8c7YuyAgEReiK7SiRqs/ShtOfawXSOEJafFYP wmb29JzLpCnocIXh/Y+nXZz6Y17/Yrk34QGCyL4P/9lC6yWTP6Ws7oRrf A==; X-IronPort-AV: E=McAfee;i="6500,9779,10521"; a="289816262" X-IronPort-AV: E=Sophos;i="5.96,138,1665471600"; d="scan'208";a="289816262" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Nov 2022 15:39:27 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10521"; a="668513959" X-IronPort-AV: E=Sophos;i="5.96,138,1665471600"; d="scan'208";a="668513959" Received: from adhjerms-mobl1.amr.corp.intel.com (HELO rpedgeco-desk.amr.corp.intel.com) ([10.212.227.68]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Nov 2022 15:39:26 -0700 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V . Shankar" , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org Cc: rick.p.edgecombe@intel.com Subject: [PATCH v3 06/37] x86/fpu: Add helper for modifying xstate Date: Fri, 4 Nov 2022 15:35:33 -0700 Message-Id: <20221104223604.29615-7-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221104223604.29615-1-rick.p.edgecombe@intel.com> References: <20221104223604.29615-1-rick.p.edgecombe@intel.com> ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=KUJzmhjF; spf=pass (imf03.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667601569; a=rsa-sha256; cv=none; b=QQWVwKQKffw4HV3vlvRjPd38Cxmid6yN5r/WEXFVbBOZDhGrtfL62zGhVqu178MMRGc8da KYNj37GSURUfZer+cWFewxBRAlh1fYjOO9Lrg5kt6MScdifA4FfCaEk1omurZitAnl9caX kL/yYrQn+yV8jg44x8H1E0hei3/wvf0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667601569; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references:dkim-signature; bh=AbmGId16tK0t2rGVD1R/lf6dwe+cYapm4W+Qlo/nZ2Y=; b=VLKqqmCPq4bmRIwIN5f7mF7kRyPlqdHPeweGOofxN4Xd/CgADW/MmYguRb/NwuzOFxQrSn O9zCug49Id5Y0m1z40Xq6XpbsdqVSGwJkW/4QdepFzQnabPfagpm0JYmmHQmDmX9FOEux0 pqj4EkFmHoIhhM6OTOY+CF9BWjOH28w= X-Rspam-User: Authentication-Results: imf03.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=KUJzmhjF; spf=pass (imf03.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B451920004 X-Stat-Signature: k4xypyducr48hdtmewhirzj9drkz4mjw X-HE-Tag: 1667601568-540302 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like user xfeatures, supervisor xfeatures can be active in the registers or present in the task FPU buffer. If the registers are active, the registers can be modified directly. If the registers are not active, the modification must be performed on the task FPU buffer. When the state is not active, the kernel could perform modifications directly to the buffer. But in order for it to do that, it needs to know where in the buffer the specific state it wants to modify is located. Doing this is not robust against optimizations that compact the FPU buffer, as each access would require computing where in the buffer it is. The easiest way to modify supervisor xfeature data is to force restore the registers and write directly to the MSRs. Often times this is just fine anyway as the registers need to be restored before returning to userspace. Do this for now, leaving buffer writing optimizations for the future. Add a new function fpregs_lock_and_load() that can simultaneously call fpregs_lock() and do this restore. Also perform some extra sanity checks in this function since this will be used in non-fpu focused code. Tested-by: Pengfei Xu Tested-by: John Allen Suggested-by: Thomas Gleixner Signed-off-by: Rick Edgecombe --- v3: - Rename to fpregs_lock_and_load() to match the unlocking fpregs_unlock(). (Kees) - Elaborate in comment about helper. (Dave) v2: - Drop optimization of writing directly the buffer, and change API accordingly. - fpregs_lock_and_load() suggested by tglx - Some commit log verbiage from dhansen v1: - New patch. arch/x86/include/asm/fpu/api.h | 9 +++++++++ arch/x86/kernel/fpu/core.c | 19 +++++++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/arch/x86/include/asm/fpu/api.h b/arch/x86/include/asm/fpu/api.h index 503a577814b2..aadc6893dcaa 100644 --- a/arch/x86/include/asm/fpu/api.h +++ b/arch/x86/include/asm/fpu/api.h @@ -82,6 +82,15 @@ static inline void fpregs_unlock(void) preempt_enable(); } +/* + * FPU state gets lazily restored before returning to userspace. So when in the + * kernel, the valid FPU state may be kept in the buffer. This function will force + * restore all the fpu state to the registers early if needed, and lock them from + * being automatically saved/restored. Then FPU state can be modified safely in the + * registers, before unlocking with fpregs_unlock(). + */ +void fpregs_lock_and_load(void); + #ifdef CONFIG_X86_DEBUG_FPU extern void fpregs_assert_state_consistent(void); #else diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index 3b28c5b25e12..8b3162badab7 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -756,6 +756,25 @@ void switch_fpu_return(void) } EXPORT_SYMBOL_GPL(switch_fpu_return); +void fpregs_lock_and_load(void) +{ + /* + * fpregs_lock() only disables preemption (mostly). So modifing state + * in an interrupt could screw up some in progress fpregs operation, + * but appear to work. Warn about it. + */ + WARN_ON_ONCE(!irq_fpu_usable()); + WARN_ON_ONCE(current->flags & PF_KTHREAD); + + fpregs_lock(); + + fpregs_assert_state_consistent(); + + if (test_thread_flag(TIF_NEED_FPU_LOAD)) + fpregs_restore_userregs(); +} +EXPORT_SYMBOL_GPL(fpregs_lock_and_load); + #ifdef CONFIG_X86_DEBUG_FPU /* * If current FPU state according to its tracking (loaded FPU context on this