From patchwork Mon Feb 7 08:46:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jisheng Zhang X-Patchwork-Id: 12737041 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7B684C433F5 for ; Mon, 7 Feb 2022 08:48:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Subject:From:Cc:To:Date: Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=k/j24jOUifEI/rxRBtQsOSgyfLorZP3LjsL552H2eIg=; b=YN1xCu3E3ea2r3 pQo7QXce8ZhyML9BoEEsAWMutIj0lo8IF0YFHnQ9Tzc2pLCNIUJgpWifKKI9kcVYAm3Ze8O1/FM1h Ir3qZuB0T6qR7UNEEo39BO0QYtOfZ5xaCwKyUYpHAoMsODCV+JqnNynhRUv2m60XtIWUh/pf+T5vB Gu5mI0uIOm0/f8rGTVT+FFzw9Ue8dD2ZwHNDRVzFIIJTrwz5dfrKtedvfhDLUFNirCFeBr4g7NYH6 /WAiD6LDt5nORyRL5FYyxI/j8WJBk1lk1Cpo69Gv882S0A0zK6GG1exxRV68d8XMx8a9yGzVsN1K/ tmtBZ9JkGd92FJPI3M3A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nGzft-009PTe-DO; Mon, 07 Feb 2022 08:47:06 +0000 Received: from mail-dm6nam10on2058.outbound.protection.outlook.com ([40.107.93.58] helo=NAM10-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nGzfo-009POy-6a for linux-arm-kernel@lists.infradead.org; Mon, 07 Feb 2022 08:47:01 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Yi6o16cIMVhO9v7PGhcOaUsVN20PnG6MkpNHWthiRx4gXBL3AAW7HkOP9UdZzH7W6wrn7vAXde5cMB3CD7i8Vczbkdl2Y7BWguDDjtFiCKN07HtchWkPsYmLkq6CBSpvJCt9f2pZxogQHeW0YuS7wKySsp5htqaUlwWeuqHPruQbgs8aLPhMvXNZapL+uTFr9hS+V4W/CzVuMLcXWwanIBnZsskX4S4x3N45s+BD7PE4bCvB4PyykM5WnSaEm0IznoPyrM7iQkJqyLhAuXn0TuR0WAIvssv8diTepuWhHFWzN9B/GOishg5UAz1scU0p0McwIDRTZY4sjI5b8RRDHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yEfq/3Z1dBv84RXpsQPDPQE2z2tIZHT262L+V1iGoV8=; b=FKQFFY3pHPBRRDPOFlBvZAxNahr4JHXivTDr2LmIoAFAx7e44ZoRbvf1bujhuZ3pGAx+pu7iqo5Pfa0jJTjaURfnUl8KvZsFVpY6IUleYmtcmjceOZ0jRghfL80qCJH+d+vWtkbN+rcPDyMQiX+MyzZVnvfXNG5tEC4Ju9yORuS/cPi6GKvvhjGb/8LgH8nKHolsrYI5/eU1BhdTodMeC5QJ5bktlyZalRpwIWvlezhucR2EZNUvzNwocCPcJLuSj6IQD2S0c6dgAX9U8/asGDVn3UN7rROcx0RRcGoSAT6Q4wk//pA37zuxxMNwGnQHYpXgW5NjHwZpOCxY01KD8A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Synaptics.onmicrosoft.com; s=selector2-Synaptics-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yEfq/3Z1dBv84RXpsQPDPQE2z2tIZHT262L+V1iGoV8=; b=hGtNNO6rPjONW/nj8O+FyWJSfGE3Gu7WSXq7quxjlW0qGyQohlTTsTXwWPBq6Lg3iA/I4FqoNeyfmHTCgr9Px2kmgSvh4LchlobbTyJbh3Md2B94thAfzlUYBhcigLxfwJx60qtzPlZqKUd/3M/XOmqhEUoJYTZX4xj8WmLkeHU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=synaptics.com; Received: from BN9PR03MB6058.namprd03.prod.outlook.com (2603:10b6:408:137::15) by MWHPR03MB2813.namprd03.prod.outlook.com (2603:10b6:300:11a::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.12; Mon, 7 Feb 2022 08:46:51 +0000 Received: from BN9PR03MB6058.namprd03.prod.outlook.com ([fe80::fd91:f2c5:8345:cc78]) by BN9PR03MB6058.namprd03.prod.outlook.com ([fe80::fd91:f2c5:8345:cc78%3]) with mapi id 15.20.4951.018; Mon, 7 Feb 2022 08:46:51 +0000 Message-ID: <8798a3cc-6e64-6966-d5ad-fadca79d92ba@synaptics.com> Date: Mon, 7 Feb 2022 16:46:42 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 Content-Language: en-US To: Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org From: Jisheng Zhang Subject: [PATCH] arm64: Add read_mostly declaration/definition to irq stack ptr X-ClientProxiedBy: TY2PR02CA0009.apcprd02.prod.outlook.com (2603:1096:404:56::21) To BN9PR03MB6058.namprd03.prod.outlook.com (2603:10b6:408:137::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 10b36d49-414a-4388-a5b0-08d9ea166239 X-MS-TrafficTypeDiagnostic: MWHPR03MB2813:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: I7uBUkLuhLyLJt8XKEK8wvgCOmj3biTDJLHCy1/7/tNV0+KQ+sLAZqBZbWiLFGhYnhg39l0rJ1wbcUZ2fPtb0F3NrAT3iqNOr/WsEkb+8fWm9YQLIFaQz6xMvJ8bxoja0ELlIyczf2Y6Y3gA96ME3ko5BBdaYwQEPqX2cZfNYsUPFpcF0Y57m5dQmznLlhgftlcm/CnyvLDXfEgjuIwUouLr4/kGnQDV6OI4ppTq8RbkmTs3WodwSeM69lSuy1QSDcvSAjzgSqon7P2tkcaJBZV7O/oDK4uG4LlJ7RyOJ8byeRnm1d1Axuc1nzwTX7vXMpPYaZBDM8p3lE/IN9OsHVRn6v1klUGuS9NWXMEUhYVvuv6i5Y9vDkbiF+ecAqWx1UdlrPBzlkeffytTNSAkiSJOQtLt17HHcJPBUyg9B+/QQq+WIIuGWhsgvS7CbQQRi4L77fAGtUYoTulXyFG18ba49jckw7KrnPfPYysFtj+iV32fsJJfVh5a9oFtcUKAE+htCW252r7yJrekQrbkWoZ0n4VuKJRtWEqDWFWXR/zuApFXZ3dRwuwW1UfgnKICvCqbu4CxQXadlmi/Rm4A9qTWvrmQuPs0srkjAvVQsGyoJ9sisXZgzmbThgAiM07pgdmMAgIT77V6+UpihKAbFeKDIDF8zqRjDlG2KMem1GSl76PLq1WA47mm1feM46E+2Im826zWO9tBp73cvU3AeAyF9WdZa5Fvug+SQ2OQMTpzu2yE8aaeSmEOQp1+B7eW6B0Nndhv0KG16p1LLOp7TA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN9PR03MB6058.namprd03.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(52116002)(508600001)(36756003)(66476007)(86362001)(31696002)(66556008)(6512007)(6666004)(6506007)(83380400001)(2616005)(26005)(186003)(4326008)(5660300002)(316002)(66946007)(6486002)(38350700002)(38100700002)(110136005)(8676002)(8936002)(2906002)(31686004)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?7iPLgvdl0OrwDhoS54qL04yTBl5c?= =?utf-8?q?3CswuIqt1s5tZ5paSZj+m37w9UH4m7dx2RwMMVuMffvrz/aiRpcOYDA//veXKBFeP?= =?utf-8?q?DlTy/5IJoLZF6QHfyb0mQw/YbzVUfsld6izg8ho9X4gJJxze1H1nlcsOyzB5Mh+6w?= =?utf-8?q?GcPBmWxk4l4RleWJtNL55EuwN0zKzUhHA3Hb6ED5u2o1wCpKU1QPi9c73hFG82U0z?= =?utf-8?q?VsWTcIf1p6JH3/c+zgUBo0a6gs1HephThDnUhnX3sdPt53z09ZOo7m7v4fTEZ/LWN?= =?utf-8?q?IxNerH2Mq4PpdlkEH3D4I1t89QNl5fErVxDicxy6ojSq+tDYiRUz603jzdJeeKisO?= =?utf-8?q?b5dpjO4WbdBKZIIIl92yYNQIOgfyiqH9ThEEGjzhecSWOx9Uc4ID0SyYm1nbSxX24?= =?utf-8?q?SkzCh4fbgI6XZR0tWgT6v2wq+EjZ87ajHqPLDFaFecDrWBBjD4n6LSF9L4XY3dWrV?= =?utf-8?q?hppkbq4j+tqYIBqgOVdbPOjcK2Yi5vaPqiV3/L07EDVQmTqXSVzHl0HRcoG3n6Wly?= =?utf-8?q?qmWbt7CJhlzd5iytSjtuYEVypwJ2GCu1RunESUM0Aa8hpvSN3Kk1naSQkP2tADe8X?= =?utf-8?q?rczOTYiU3YZrv2O8D9zPjcNLiIdLQ1wNSFS4CKeQK9SZ6lJJJ473ISNEagZx8AucR?= =?utf-8?q?ImjdA9BKEvwUMhdzMj7Vm3Kzy8tT6OczkdgnErGgkj4hG+Cecpyku0FTbfvVb7pe0?= =?utf-8?q?pqldY+twbKnOEWDMedjwc+nbvIJOAGpvjCEmZft5zN9K/E0tFN/ORG/dAt5MG+R6D?= =?utf-8?q?EBAsh1CisqDLBSmylwqdU8t5X+Tnyli4gtnuQjG56/KMlbvHVIj4pFiatD4srct1q?= =?utf-8?q?lQfcL7smjurQ9RJ50gZiPncelUrdz/zPCefYFqUZCxgcFfd5Eo3Ep+u7bfKWdfriF?= =?utf-8?q?uPx7hbUFeEehQY04mEKfsv9pH8JIloD6xFqP1OlBEDX8EzhbnK/p4Gesl99Iu1/Hq?= =?utf-8?q?OeCidT4GMWFJWen/4RdrTeVPgzKtdPfP0VPEX/JIYDdgdKKQXPw6Bd35MWoW3E/TJ?= =?utf-8?q?m0N2FOAd1ZV08FxtCsGhGAoZCMdaNS0YYb5ijoqBQkJ1kdD3FuOuMEUtsRHvf9nHl?= =?utf-8?q?4zhqelaSg1/iJWHVU5mZRDZRH2F6zdteLTWdmvaUn8e0WvRIOtNWWic/n1udfMJYx?= =?utf-8?q?n4dOjj1ccayCDIZdVUnOV0XwRpv7KHVIE2JS0Gp9c50gcXiWQLI1SFG1DtIJ39wNu?= =?utf-8?q?ZdWK4yA4G7u/OgS/kUb07TofcCYkmYYHLr4cav6vMPXfZvQbyGW4x4MCkNH7xnqs8?= =?utf-8?q?MfzbaYVBLXSEag4wyJ38bkMvIwIP/uq5tSP82llVfURUqICVLEtwDj9OGtM8R4guF?= =?utf-8?q?Dm1r5eitMHVJSQc+379HssQfguJgsJtoTFESzK0F0NDjhek+akCLfIAlb4mo9KL/K?= =?utf-8?q?QUJeIBJS9A3HjkMVq231Nsk33eS66pBKRkSxpIqjhZMSKALPm9c/m+ZjjeDvbJq2S?= =?utf-8?q?ziseXh9O6SUMYoJCEpD1xfvH3gQG66YvmweoPQGjAtrsXRJs6w7msE2FHLeWVIQNJ?= =?utf-8?q?j1ozW0+KJfsY4VQOCdkdgvYBvG/7v5ghwymYXCZwp8x2v0QQ5OigX8w=3D?= X-OriginatorOrg: synaptics.com X-MS-Exchange-CrossTenant-Network-Message-Id: 10b36d49-414a-4388-a5b0-08d9ea166239 X-MS-Exchange-CrossTenant-AuthSource: BN9PR03MB6058.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Feb 2022 08:46:51.0260 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 335d1fbc-2124-4173-9863-17e7051a2a0e X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: kRQItADrkUgATxS+9dNJ0UrE6j6WrNc+HULmWj71mbT5whm8Yk9SS3VGwdwRQkSpynEq3VG+yMPn9vs2nHYhkA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR03MB2813 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220207_004700_359804_99AC1E16 X-CRM114-Status: GOOD ( 11.13 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add "read-mostly" qualifier to irq_stack_ptr and irq_shadow_call_stack_ptr. This is to prevent the false sharing. Before the patch, I got below percpu layout with one defconfig: ffffffc008723050 : ffffffc008723050: 00 00 00 00 .... ffffffc008723054 : ffffffc008723054: 00 00 00 00 .... ffffffc008723058 : ... ffffffc008723060 : ... ffffffc008723070 : As can be seen, the irq_stack_ptr sits with the heavy read/write percpu vars such as fpsimd_last_state etc. at the same cacheline. After the patch: ffffffc008723000 : ... ffffffc008723008 : ... ffffffc008723010 : ... ffffffc008723018 : ... Now, the irq_stack_ptr sits with read mostly percpu vars such as cpu_number etc. at the same cacheline. Signed-off-by: Jisheng Zhang --- arch/arm64/include/asm/stacktrace.h | 2 +- arch/arm64/kernel/irq.c | 7 +++---- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h index e77cdef9ca29..75c142bfdffe 100644 --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -66,7 +66,7 @@ struct stackframe { extern void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk, const char *loglvl); -DECLARE_PER_CPU(unsigned long *, irq_stack_ptr); +DECLARE_PER_CPU_READ_MOSTLY(unsigned long *, irq_stack_ptr); static inline bool on_stack(unsigned long sp, unsigned long size, unsigned long low, unsigned long high, diff --git a/arch/arm64/kernel/irq.c b/arch/arm64/kernel/irq.c index bda49430c9ea..d2e75e9bb826 100644 --- a/arch/arm64/kernel/irq.c +++ b/arch/arm64/kernel/irq.c @@ -26,13 +26,12 @@ /* Only access this in an NMI enter/exit */ DEFINE_PER_CPU(struct nmi_ctx, nmi_contexts); -DEFINE_PER_CPU(unsigned long *, irq_stack_ptr); +DEFINE_PER_CPU_READ_MOSTLY(unsigned long *, irq_stack_ptr); - -DECLARE_PER_CPU(unsigned long *, irq_shadow_call_stack_ptr); +DECLARE_PER_CPU_READ_MOSTLY(unsigned long *, irq_shadow_call_stack_ptr); #ifdef CONFIG_SHADOW_CALL_STACK -DEFINE_PER_CPU(unsigned long *, irq_shadow_call_stack_ptr); +DEFINE_PER_CPU_READ_MOSTLY(unsigned long *, irq_shadow_call_stack_ptr); #endif static void init_irq_scs(void)