From patchwork Mon Sep 28 15:17:24 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Metcalf X-Patchwork-Id: 7278561 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 39A3A9F36A for ; Mon, 28 Sep 2015 15:22:03 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 2EBAD2073C for ; Mon, 28 Sep 2015 15:22:02 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1BF61206E1 for ; Mon, 28 Sep 2015 15:22:01 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1ZgaDm-0004mb-G4; Mon, 28 Sep 2015 15:20:06 +0000 Received: from mail-db3on0077.outbound.protection.outlook.com ([157.55.234.77] helo=emea01-db3-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1ZgaCN-0004PW-JT for linux-arm-kernel@lists.infradead.org; Mon, 28 Sep 2015 15:18:41 +0000 Received: from AM2PR02MB0420.eurprd02.prod.outlook.com (10.160.54.26) by AM2PR02MB0452.eurprd02.prod.outlook.com (10.160.55.18) with Microsoft SMTP Server (TLS) id 15.1.280.20; Mon, 28 Sep 2015 15:18:16 +0000 Received: from AM2PR02CA0015.eurprd02.prod.outlook.com (10.160.28.153) by AM2PR02MB0420.eurprd02.prod.outlook.com (10.160.54.26) with Microsoft SMTP Server (TLS) id 15.1.280.20; Mon, 28 Sep 2015 15:18:14 +0000 Received: from DB3FFO11FD004.protection.gbl (2a01:111:f400:7e04::103) by AM2PR02CA0015.outlook.office365.com (2a01:111:e400:8400::25) with Microsoft SMTP Server (TLS) id 15.1.280.20 via Frontend Transport; Mon, 28 Sep 2015 15:18:14 +0000 Authentication-Results: spf=fail (sender IP is 12.216.194.146) smtp.mailfrom=ezchip.com; ezchip.com; dkim=none (message not signed) header.d=none; ezchip.com; dmarc=none action=none header.from=ezchip.com; Received-SPF: Fail (protection.outlook.com: domain of ezchip.com does not designate 12.216.194.146 as permitted sender) receiver=protection.outlook.com; client-ip=12.216.194.146; helo=ld-1.internal.tilera.com; Received: from ld-1.internal.tilera.com (12.216.194.146) by DB3FFO11FD004.mail.protection.outlook.com (10.47.216.93) with Microsoft SMTP Server (TLS) id 15.1.274.4 via Frontend Transport; Mon, 28 Sep 2015 15:18:13 +0000 Received: (from cmetcalf@localhost) by ld-1.internal.tilera.com (8.14.4/8.14.4/Submit) id t8SFICtR007897; Mon, 28 Sep 2015 11:18:12 -0400 From: Chris Metcalf To: Gilad Ben Yossef , Steven Rostedt , Ingo Molnar , Peter Zijlstra , Andrew Morton , "Rik van Riel" , Tejun Heo , Frederic Weisbecker , Thomas Gleixner , "Paul E. McKenney" , Christoph Lameter , Viresh Kumar , Catalin Marinas , Will Deacon , Andy Lutomirski , , Subject: [PATCH v7 09/11] arch/arm64: enable task isolation functionality Date: Mon, 28 Sep 2015 11:17:24 -0400 Message-ID: <1443453446-7827-10-git-send-email-cmetcalf@ezchip.com> X-Mailer: git-send-email 2.1.2 In-Reply-To: <1443453446-7827-1-git-send-email-cmetcalf@ezchip.com> References: <1443453446-7827-1-git-send-email-cmetcalf@ezchip.com> X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1; DB3FFO11FD004; 1:7eUSOSP5kXuAyPw7I67E+3X2fAXIznAcdqF/Nnb02PTL+I4OK4KngTMKzHnlARhHBDpizffHC5Q6z54E+dTMBjbLbF/5r/M94X3RfTvcucTbADVqD0uZkmckZ659WYhYYVni5x6yT8j42YmzFWnkhXCxqL0lnrxnNMjZH5noICvSM0wsvv3BeYZSEQAhIlZeg+T4SDOm6oPtP8Ti2OuudBeQxFY1RvkgcIXOokTICYgTWwZyea2dC3X2G2Ev30iuXOSWQ6E4YlSggg46ybiGmIHKDSgrjFlwz0FESKupoxLWQnwrjNh9s1+ei+o4qC12uPaWhUD7niadQuqCEa6v5TJDph2ESsmQWZ3FRbIEIf0= X-Forefront-Antispam-Report: CIP:12.216.194.146; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(2980300002)(1110001)(1109001)(339900001)(199003)(189002)(6806005)(19580395003)(48376002)(42186005)(50466002)(87936001)(64706001)(47776003)(19580405001)(92566002)(5001960100002)(50986999)(104016004)(76176999)(33646002)(46102003)(85426001)(107886002)(4001540100001)(50226001)(97736004)(2201001)(5001860100001)(5001770100001)(5001830100001)(81156007)(106476002)(86362001)(106466001)(62966003)(36756003)(189998001)(106356001)(68736005)(105606002)(77156002)(11100500001)(5007970100001)(229853001)(5003940100001)(2950100001)(9376005)(921003)(83996005)(1121003)(4001430100001)(2101003); DIR:OUT; SFP:1101; SCL:1; SRVR:AM2PR02MB0420; H:ld-1.internal.tilera.com; FPR:; SPF:Fail; PTR:InfoNoRecords; A:1; MX:1; LANG:en; MIME-Version: 1.0 X-Microsoft-Exchange-Diagnostics: 1; AM2PR02MB0420; 2:saoIUMQDgAEh7Y6Od1Jif+v6sEGoIvLFPczQJAjPdeZtmc2GVn3ujtg8gv+J6DufR5CusI4ABgAQVOJRxCkuB/wzCJ1V+xI+4mTsJUxlhlK2XivRovolu1CZFnJ/0EbkeFCybOWVy4jPxo7jOUev+mXMvYVUv7CwlazYjQmJBWI=; 3:d78KS9sKd2OcKSVlSitvea1lkMwmur1iPIFUoGp/0L+3nTroE0ZhwGB2ALqRYQfxNHoTF1d9jcwpR0DHX/nyOYj/LGplwkj2N4XUSHLFuTLcoBbin2W/dGFXZDgnce1Mwy3xljM295UJ2HYrPo6LRfRtzy07/LXUoNScpYT7GNHAv7ztCmeeamcj0t6QXwxS67V9m9mp0X77YhxjYX/NXMqEw/lXajZOevTsnk6conqRozK9I4Yq2q1DooeMpxNu; 25:wIgRQA0H/6VUCfhn0jE3jdWrPZCyb5gvbCnidhI8GBObxj3RC751CUUk9vWyXwnEPWkcPftOutNGoAcbO6AFJ7fQjkITLiY11GGL6e7Qv7OCcEVcfRcLJ3QDDfOsh/cTGLODVFm1pvs2XR2kkK43xMtcAn7HWfXGnjsORW06LBu+zbgu/+Fs+LlM+EL5AdhrkSHmgqOEaejqOfgi0hOY7MHktef6AvoZ4ZMoAkF9W51v7EBTlL8sq7cM1ceW/leJihzXwSYJb1EAJt6zWeSxSQ==; 20:6AgKLmn/Nwo1F33QKEaAcI+KIgNcwz2JEyERc1SWuhOqxHyesrOi+1Fl75cpD78erQlFNYrTo4ek/MALd7A7OhKPb93XxGgKgv9APc1asRvq+cSXYc8yZqnelp+E6TRBa7VhW58qfMVg5tQjn/lJxBIZUjDj1DrAhZ5GDx3UZzw= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AM2PR02MB0420; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(520078)(8121501046)(3002001); SRVR:AM2PR02MB0420; BCL:0; PCL:0; RULEID:; SRVR:AM2PR02MB0420; X-Microsoft-Exchange-Diagnostics: 1; AM2PR02MB0420; 4:3qK5DRvIYfDvMDlOXwci54Z51bqV1PiuG2AkcuPx12b0CITfNalekLDv6DNklYo494TvRaFPF8Z0z3EKRmPAhxzbAasR331ht/whVJGEEOAEYwx2knNsMHm7givVgPQ3KyhqT1bU6bx4DlyeA5UqQP5adBKE2yiR+72eu6G+6waqe21LkfEHLtUF61r1BdCdppzStdpdosu2f96xQEm1lDjF4fTbGxrAkyaaptrcyOWHu/YSdhem7+4SaOGCcR08+5gLEiVinfhXOarCSocX5VByvvYv/RkLohUjM6LusrKxLqVEl2yGZ2bcr0XqZYT3ZRF+/UrPwa4ss4WuWDsFR+c2oLwLLJ4OiZNmZ/p7kIU= X-Forefront-PRVS: 0713BC207F X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM2PR02MB0420; 23:ZTgAg/V9Y+FRTGM1nZUvkBfSLxLdVvdg6PWduVnOm?= =?us-ascii?Q?2TzJwY97AH6QgdfVX9iSeSuqug/+BIz7BQdvWb59rDhmc512i18vUZhM9aZB?= =?us-ascii?Q?gIsC9a40gKmn7p/dvT15sblWHVvwEdvLRPK5Ua+8BDGrDpYXqwt12PTZqu28?= =?us-ascii?Q?X/uEAYh9+esG64ffBBiSHKtgFswDjU79xrxHopMA+OGYoNKQatFb/tMXjbhp?= =?us-ascii?Q?eVi2btTIobCRoK8v0fYpRafk9FQLi7EgsgPvmM27VOMzRCOvgdX37nS2VrX9?= =?us-ascii?Q?wguyz12FGBxDmUIPuo+whQ6SE1mVfY9uEKiOhiIAwTOah3hMGHCYS/s8r0e3?= =?us-ascii?Q?ycuL2w8B3m1/fCvZMWAehkXd6MmfaY8KvlZ/fjQ8mYYp86qujtlEUFflm+wd?= =?us-ascii?Q?IP4jF4zuObHWrqmb95xkMYVKDhoOyTdV1odl2bKXxjdwAKj9Wk+UbKpXiCuJ?= =?us-ascii?Q?oN+wGixePtDvTiTjKsGoa/3GCI9CBqiZHP/VNYEqgEJHYk8rj01qiKvx6gT6?= =?us-ascii?Q?vflI76b5MZExJYrkBK474pm5hslScO4GY+CMhwkhEtoNd9TfqBHapP9iy03i?= =?us-ascii?Q?mglUaQl8ctvMpiqqcmtKWLVVQwgZfuSkh1ETDrkRiKrhRxXRQbSnkPSen6Tp?= =?us-ascii?Q?j31xoRjW5gJbmSprJObjM9MvdkC8wGZnFjXbjPysFWETO09LDTMaHUQSy6V9?= =?us-ascii?Q?/LFS4iaUcag81+fvtoOQMp8VOyaMi8sFGTrdytvip1ySxWEJ9Qx8uFN++c1W?= =?us-ascii?Q?DBL4okY/HPmuwkrawu4MWk97jzVacJgkQfkrzS3ZGc7tBAYB3Hg9Seyg3u1A?= =?us-ascii?Q?jAtJboJyktQ+f4ChHdgnrBtpj4kyhdOATARQyMA1f5iQhXNtPwqfrIa1GNT0?= =?us-ascii?Q?Gw8MIbS5d2PiDzSuXp2do1eGPcCE/iVEC8kW+rQcrTsNTon7F2LZa1s1arEz?= =?us-ascii?Q?8WzY7apNdNOwaa0gaPGFzLPzmTSJWfDXfUNsDswFchuw8BNwjbNUmmzMifO8?= =?us-ascii?Q?Q0mT+AlQTHU2GWMCibmRfucpxZ5FAezfqYAupNtkfp3o945YHRqX/MiIUjql?= =?us-ascii?Q?4EiSQe88mHJ4OURly9WjIBvjOHM9sJXgXAsF53xWmsv4n/XXZ+8RRI7Qy1Qh?= =?us-ascii?Q?CPXb1ZDmo6ouKm95kvk5ZrCpveCWpPvrtILgfDuDXfCodf9BHx35oFeHciwR?= =?us-ascii?Q?SmGeNWvn/qBbX8Fv+DCO+usW+a4sNdSLRKKMVF8bWe/Ad80kLhXaXXEZ8Z1d?= =?us-ascii?Q?y5MasMQMiixpSQY4sy2j0cT/hKmFYgCUA5M7gxLA8u2awHt/BCpRtTGNJdnZ?= =?us-ascii?Q?Xpa3Bxdlc2buJlF4h9mbOzhbpqqMU9ogzvI1lGpT8OIAdtzUyGT2BDDzs701?= =?us-ascii?Q?2cI6vqoCMeK1z/w+WWJ//cCmPtm2RhMZnZ9qoiXkaySFkBGLKFyMfRPTTEdq?= =?us-ascii?Q?lxrI5C1dQ=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; AM2PR02MB0420; 5:HHhB+CpM09AT/XIOBeGq2uTciSFSLyShn7zgQfWCS+mse1/ZHdQQClrB/ixzPan6TmP7tLTJlJ7M8dbo3mac9RT8QZtrF0gUcR3+VoX4CeP3n1MlJ0+9SYy0D6JlQfTCgdC7XSIcqgyP2BLqADzwPQ==; 24:bxP2FBFNcKp8OwDgea3w7JC94h2FfpxJPGv4QLRyWftkK4tdSe2ZIq5G+63Suv1LkEJ5oQgMx339dNWwSsnHF+AGeh7FajkOsyeBSV8vFxk=; 20:33EDlB2oo58ehigUHhVHSVHbOAQWJEFp83dplq0VQGLvouyQ9PQKV7yCs4mhnB4hh8cwXrbVzb6fIRabnQBV5g== SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2015 15:18:13.8943 (UTC) X-MS-Exchange-CrossTenant-Id: 0fc16e0a-3cd3-4092-8b2f-0a42cff122c3 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=0fc16e0a-3cd3-4092-8b2f-0a42cff122c3; Ip=[12.216.194.146]; Helo=[ld-1.internal.tilera.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM2PR02MB0420 X-Microsoft-Exchange-Diagnostics: 1; AM2PR02MB0452; 2:IA8nAR3P++QB2gQpIZJI542lJk+OLScIy+9Sb4OAzCjHbsvqlMd1NV+nkvoQt1jNXj/y03f7S+HACDB+xZfTiOrBIhnFUfNtyEPhBbBvGwFW5khYqDbQXGwUDiTUf5/lUy8dYoPvn1mN5SSEjYufxB9U6zW7fFnfFOY/2oKg0Xw=; 23:dG/xWHnzVjScKvgiR7IdNbPxdqDqm3SRjwoG66YYt6R2jB25ZCdU8ENV6gxLoGaq6aKEpwLAnu2ULj1VAivcXOakiOQkX5o4oo9YIoQE8up4T/2fuhEyxVwWTHt5Iy+os201iZx+ErrhYZS+B4TwLP+TCVtm0X0yT6wbQo+/27067aiCPTAeCZdg1kCxmHVd X-OriginatorOrg: ezchip.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20150928_081840_262281_60D0FDA9 X-CRM114-Status: GOOD ( 18.33 ) X-Spam-Score: -1.9 (-) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Chris Metcalf Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-4.2 required=5.0 tests=BAD_ENC_HEADER,BAYES_00, RCVD_IN_DNSWL_MED, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We need to call task_isolation_enter() from prepare_exit_to_usermode(), so that we can both ensure we do it last before returning to userspace, and we also are able to re-run signal handling, etc., if something occurs while task_isolation_enter() has interrupts enabled. To do this we add _TIF_NOHZ to the _TIF_WORK_MASK if we have CONFIG_TASK_ISOLATION enabled, which brings us into prepare_exit_to_usermode() on all return to userspace. But we don't put _TIF_NOHZ in the flags that we use to loop back and recheck, since we don't need to loop back only because the flag is set. Instead we unconditionally call task_isolation_enter() at the end of the loop if any other work is done. To make the assembly code continue to be as optimized as before, we renumber the _TIF flags so that both _TIF_WORK_MASK and _TIF_SYSCALL_WORK still have contiguous runs of bits in the immediate operand for the "and" instruction, as required by the ARM64 ISA. Since TIF_NOHZ is in both masks, it must be the middle bit in the contiguous run that starts with the _TIF_WORK_MASK bits and ends with the _TIF_SYSCALL_WORK bits. We tweak syscall_trace_enter() slightly to carry the "flags" value from current_thread_info()->flags for each of the tests, rather than doing a volatile read from memory for each one. This avoids a small overhead for each test, and in particular avoids that overhead for TIF_NOHZ when TASK_ISOLATION is not enabled. Also, we have to add an explicit check for STRICT mode in do_mem_abort() to handle the case of page faults, since arm64 does not use the exception_enter() mechanism. Signed-off-by: Chris Metcalf --- arch/arm64/include/asm/thread_info.h | 18 ++++++++++++------ arch/arm64/kernel/ptrace.c | 10 ++++++++-- arch/arm64/kernel/signal.c | 6 +++++- arch/arm64/mm/fault.c | 8 ++++++++ 4 files changed, 33 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index dcd06d18a42a..4c36c4ee3528 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -101,11 +101,11 @@ static inline struct thread_info *current_thread_info(void) #define TIF_NEED_RESCHED 1 #define TIF_NOTIFY_RESUME 2 /* callback before returning to user */ #define TIF_FOREIGN_FPSTATE 3 /* CPU's FP state is not current's */ -#define TIF_NOHZ 7 -#define TIF_SYSCALL_TRACE 8 -#define TIF_SYSCALL_AUDIT 9 -#define TIF_SYSCALL_TRACEPOINT 10 -#define TIF_SECCOMP 11 +#define TIF_NOHZ 4 +#define TIF_SYSCALL_TRACE 5 +#define TIF_SYSCALL_AUDIT 6 +#define TIF_SYSCALL_TRACEPOINT 7 +#define TIF_SECCOMP 8 #define TIF_MEMDIE 18 /* is terminating due to OOM killer */ #define TIF_FREEZE 19 #define TIF_RESTORE_SIGMASK 20 @@ -124,9 +124,15 @@ static inline struct thread_info *current_thread_info(void) #define _TIF_SECCOMP (1 << TIF_SECCOMP) #define _TIF_32BIT (1 << TIF_32BIT) -#define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ +#define _TIF_WORK_LOOP_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE) +#ifdef CONFIG_TASK_ISOLATION +# define _TIF_WORK_MASK (_TIF_WORK_LOOP_MASK | _TIF_NOHZ) +#else +# define _TIF_WORK_MASK _TIF_WORK_LOOP_MASK +#endif + #define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ _TIF_NOHZ) diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 1971f491bb90..9113789e9486 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -37,6 +37,7 @@ #include #include #include +#include #include #include @@ -1240,14 +1241,19 @@ static void tracehook_report_syscall(struct pt_regs *regs, asmlinkage int syscall_trace_enter(struct pt_regs *regs) { + unsigned long work = ACCESS_ONCE(current_thread_info()->flags); + /* Do the secure computing check first; failures should be fast. */ if (secure_computing() == -1) return -1; - if (test_thread_flag(TIF_SYSCALL_TRACE)) + if ((work & _TIF_NOHZ) && task_isolation_strict()) + task_isolation_syscall(regs->syscallno); + + if (work & _TIF_SYSCALL_TRACE) tracehook_report_syscall(regs, PTRACE_SYSCALL_ENTER); - if (test_thread_flag(TIF_SYSCALL_TRACEPOINT)) + if (work & _TIF_SYSCALL_TRACEPOINT) trace_sys_enter(regs, regs->syscallno); audit_syscall_entry(regs->syscallno, regs->orig_x0, regs->regs[1], diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index fde59c1139a9..def9166eac9e 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include @@ -419,10 +420,13 @@ asmlinkage void prepare_exit_to_usermode(struct pt_regs *regs, if (thread_flags & _TIF_FOREIGN_FPSTATE) fpsimd_restore_current_state(); + if (task_isolation_enabled()) + task_isolation_enter(); + local_irq_disable(); thread_flags = READ_ONCE(current_thread_info()->flags) & - _TIF_WORK_MASK; + _TIF_WORK_LOOP_MASK; } while (thread_flags); } diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index aba9ead1384c..01c9ae336887 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include @@ -465,6 +466,13 @@ asmlinkage void __exception do_mem_abort(unsigned long addr, unsigned int esr, const struct fault_info *inf = fault_info + (esr & 63); struct siginfo info; + /* We don't use exception_enter(), so we check strict isolation here. */ + if (IS_ENABLED(CONFIG_TASK_ISOLATION) && + test_thread_flag(TIF_NOHZ) && + task_isolation_strict() && + user_mode(regs)) + task_isolation_exception(); + if (!inf->fn(addr, esr, regs)) return;