From patchwork Sun Apr 21 18:28:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13637478 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AE5B9C4345F for ; Sun, 21 Apr 2024 18:28:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9r8MOC1yaITNU744pcXLuxkYGKtTtIuQP3zm/FYcSv8=; b=mjc0+9vcWt5LWk 9zjqcek5CUlEtcPD4L0Ob6FdHpxu5kfvraFf+0TL4byevLx8Wsv7LMXB/s9ppsld6RMEBvF14WSR0 jZHRjfYwGVUkhILamK2YArDYFHApI96vXK9Ay2em9kxR05WMqItplZSRqIUpiNF9NvuQdZi/pTN8g A1fOXqSLXRTUo626bIuE+1NraDwYbIxYSJ/s0DXFgEdlK+4VGUqGGrCRkKRXIiB0xNKC/6JZ+gDAh /gPtSjeRtgOSSnXF2ONGJp5Uh7rJiEwNIX5CJtHaJIvLoHB3D+EhjJx5JX4UbWx1Vm9mpuIsYWaG/ GsVigSnU+4wEG5XWqsaw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvY-0000000B41p-3HO2; Sun, 21 Apr 2024 18:28:36 +0000 Received: from mail-lf1-x12a.google.com ([2a00:1450:4864:20::12a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvV-0000000B40I-0bSI for linux-arm-kernel@lists.infradead.org; Sun, 21 Apr 2024 18:28:34 +0000 Received: by mail-lf1-x12a.google.com with SMTP id 2adb3069b0e04-516d4d80d00so4631935e87.0 for ; Sun, 21 Apr 2024 11:28:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1713724110; x=1714328910; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=XDN5FRmJQobK+MX9/UAQHb6uzGwTwVFq2SIWEuBGTZ8=; b=qV2jQ57lFQybdt9nbsRQ0eEvVBGvD4G9s5Si1jkOqL27XUEUguzQvueDctDg/bAibV TgwoIRST4NBU0GrwG1gNw5QkrgY9GvecT4TlS+pFtUkVIiIKJ7HBhkng9sEG8zmxHrUv UAmsOevnWFhlOXXS5vbSBywvdStHccWBcbNTCgW01jZYzWUc4sUHP2rPywMuai/LuKF9 Jk0l5pXgbFD/W032vGZqj4lCMQVbMDbKBcQftiTkiBY0DI8MVkas2NmhllIeGGf3SrWO xj9BvdZy7HMH5zEa8SGtPaHaMHSzBBg1kK040Y0dv3uGZlxBJdLDDZzFwI8V705swZG/ IHXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713724110; x=1714328910; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XDN5FRmJQobK+MX9/UAQHb6uzGwTwVFq2SIWEuBGTZ8=; b=IK+bNiouYEHTC6A1NRc2S1RJWAQq9/oURmoqiHXqLFLWbUMzMkmtNDYyOrxk/CjV38 fWOrrC4l6aGqJN2XhcuOSSfXEOxZLiNmSDSjI7A0eYoEPSVECxZpOFLcciAuxtHJrwuo mrnN05sgVTH3bmSOyS6hSH6kw2m0mdb3+Dv9eSGorw2NAl81N4q0r7kup0JGrdD1TlBm epXutZoN6+Mby2Do8OuWeCnfdrCyE9xxf6Ro2QuoVicO/24WnaAzNGtXWeX5+etwJw97 9Mmgwhx9EJ+yZyGwIZWWl7EJ9WHhlwevJ2augGAby2B9DvTyBMt1TUenOJ7f7fXR6A14 leQA== X-Gm-Message-State: AOJu0YygKswQWVnb3X/3UUxUiYEkiyzkkeaJsTbSmlN+UCSMzYe/4Zga +URGmGLQOwyT0qvQrhSw4F4/yUphVVq6Q0NFnhiwaTpbGW/ULnKSkIB450BQ4AI= X-Google-Smtp-Source: AGHT+IF1kFCvJZyBCCjnyY8onMy4sCEISbjCHyqmS3da1hYqvcFYcwFVkAZPOvSJ5O6gz/WKOXWcOQ== X-Received: by 2002:a05:6512:23a8:b0:51a:ca75:9ffe with SMTP id c40-20020a05651223a800b0051aca759ffemr6207388lfv.42.1713724110277; Sun, 21 Apr 2024 11:28:30 -0700 (PDT) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id s14-20020a19770e000000b005188ea5d2b5sm1549641lfc.303.2024.04.21.11.28.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Apr 2024 11:28:29 -0700 (PDT) From: Linus Walleij Date: Sun, 21 Apr 2024 20:28:27 +0200 Subject: [PATCH v7 01/11] ARM: bugs: Check in the vtable instead of defined aliases MIME-Version: 1.0 Message-Id: <20240421-arm32-cfi-v7-1-6e132a948cc8@linaro.org> References: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> In-Reply-To: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> To: Russell King , Sami Tolvanen , Kees Cook , Nathan Chancellor , Nick Desaulniers , Ard Biesheuvel , Arnd Bergmann Cc: linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev, Linus Walleij X-Mailer: b4 0.13.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240421_112833_208167_F7EA4027 X-CRM114-Status: GOOD ( 18.05 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Instead of checking if cpu_check_bugs() exist, check for this callback directly in the CPU vtable: this is better because the function is just a define to the vtable entry and this is why the code works. But we want to be able to specify a proper function for cpu_check_bugs() so look into the vtable instead. In bugs.c assign PROC_VTABLE(switch_mm) instead of assigning cpu_do_switch_mm where again this is just a define into the vtable: this makes it possible to make cpu_do_switch_mm() into a real function. Tested-by: Kees Cook Signed-off-by: Linus Walleij --- arch/arm/kernel/bugs.c | 2 +- arch/arm/mm/proc-v7-bugs.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/arm/kernel/bugs.c b/arch/arm/kernel/bugs.c index 087bce6ec8e9..35d39efb51ed 100644 --- a/arch/arm/kernel/bugs.c +++ b/arch/arm/kernel/bugs.c @@ -7,7 +7,7 @@ void check_other_bugs(void) { #ifdef MULTI_CPU - if (cpu_check_bugs) + if (PROC_VTABLE(check_bugs)) cpu_check_bugs(); #endif } diff --git a/arch/arm/mm/proc-v7-bugs.c b/arch/arm/mm/proc-v7-bugs.c index 8bc7a2d6d6c7..ea3ee2bd7b56 100644 --- a/arch/arm/mm/proc-v7-bugs.c +++ b/arch/arm/mm/proc-v7-bugs.c @@ -87,14 +87,14 @@ static unsigned int spectre_v2_install_workaround(unsigned int method) case SPECTRE_V2_METHOD_HVC: per_cpu(harden_branch_predictor_fn, cpu) = call_hvc_arch_workaround_1; - cpu_do_switch_mm = cpu_v7_hvc_switch_mm; + PROC_VTABLE(switch_mm) = cpu_v7_hvc_switch_mm; spectre_v2_method = "hypervisor"; break; case SPECTRE_V2_METHOD_SMC: per_cpu(harden_branch_predictor_fn, cpu) = call_smc_arch_workaround_1; - cpu_do_switch_mm = cpu_v7_smc_switch_mm; + PROC_VTABLE(switch_mm) = cpu_v7_smc_switch_mm; spectre_v2_method = "firmware"; break; } From patchwork Sun Apr 21 18:28:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13637480 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CDC62C4345F for ; Sun, 21 Apr 2024 18:29:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1lERpYS8ac+IFHXZDhuYafpNWuSAzCnHwP5KbIjLudM=; b=ImM1S/bEyaXXqM T+RME2Wu+v6G2RGfG53TTH/vv9TdtA35eingiIH7bPwGYluZcj4XG8tm52AAz9gKaivRp5roQHseI lRQVrwz2rfKLXAI7ZJcR8vGuqhYEMacRlGiCHPvPfbWpSkwUYb46V2OEQASIK2hupyFyLcm11j4df zKJVlPAts68POFZ0RSfKR0LoA7r6AAza3+RbamK9MHhizsDGp+zEdRxHQmHdIre8gnV8I4WzZ3c/T mfA6xYG8Jr8GZxaVSVSDvgDqD2YNC7ur8nmX1G3IlZJU3EuLswYMiftSLdM4JHeHtd2k3QHRNr67K wK+ADmHBGgZvpa/a6H8w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvq-0000000B47u-0MmH; Sun, 21 Apr 2024 18:28:54 +0000 Received: from mail-lf1-x12d.google.com ([2a00:1450:4864:20::12d]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvV-0000000B40K-1clk for linux-arm-kernel@lists.infradead.org; Sun, 21 Apr 2024 18:28:35 +0000 Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-516d4d80d00so4631939e87.0 for ; Sun, 21 Apr 2024 11:28:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1713724111; x=1714328911; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=B8O1l9DG1nr6tC/T09XllcDqc2JZrTHVCm6s1PJNYaY=; b=KSv7w3yETvnsmAXRFEh3l+kCzxeu+YDVrIbmNYCNA2vfGSll1jDbkHeJaSDuE3s9nu WygsfxkRiFHPX0fOfwLkWss9W3/8PnlwE9JBms71gXw0UT4hSN+wtrVc6Z3GFgmlAxa6 hVfYX5bJwO4sZPVYsb5tuVDyDpMorbgRhzcNjpO+xAjXUkO3xjJLl5VrV6YRjFdINRdI u3KvNA1o3xnnXuwIE1YL7vHl8m5d8URL2egEte/Vbe89pZNnOFZmwaveQOMnVSDR37JK VJIa7TdgE4VhZ2xUbWYOLkKOaFTwODKzAZZfIs3KHnodOXATyVhyqWv9FMb5dSvMDw/Z k68Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713724111; x=1714328911; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B8O1l9DG1nr6tC/T09XllcDqc2JZrTHVCm6s1PJNYaY=; b=ijuESlN8giXxFWzDiLYbNbu9lvlJEZbtHJwyjfByxpGmw+ORE0R1kK3sGOyO2geDLz us3JOpn+P9y7ZIHjOK5a5oTxStHaTV8jLDrIVhqV7kKz7JJdXhGW6REtWLGizBLf1AXn pJ2/vmyXCj0HrmnEDyfA22jnTzEUBdDo3yY14DIQFTzPu0xmHF5+kwCO0gQC7wQTfrm1 xDmj43mq4J6dzIF2AXn3PP95D/fyV7YWs+eiPAi0R5+/UCMJST1P0z70jBD6aj3oAy1G i73YbFwJ+znVttTc5izJ+MP4fLk7bkokbymxJbOc3XzpGxXpdeq0jx5lKXb5h5TAsehy Mm8w== X-Gm-Message-State: AOJu0YxsLqFgP4MDi5a4Mhu6Z9+05uOCYVN/1sLuMkv4rpeJjisbxz/H TI1BGX9CRX5UujaWulA5gmQdmoJ86mj7qvdq9e9/vb+e+HYohFpFitfmX43TQB0= X-Google-Smtp-Source: AGHT+IGUFYUliEnd1IWACpXaMye/ynirYNYqr75NfYKj0gCPLVipXdzKt9Z42rZSYd+VATr2Qtox7Q== X-Received: by 2002:a05:6512:10c6:b0:51a:c44d:b95f with SMTP id k6-20020a05651210c600b0051ac44db95fmr6479537lfg.28.1713724111153; Sun, 21 Apr 2024 11:28:31 -0700 (PDT) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id s14-20020a19770e000000b005188ea5d2b5sm1549641lfc.303.2024.04.21.11.28.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Apr 2024 11:28:30 -0700 (PDT) From: Linus Walleij Date: Sun, 21 Apr 2024 20:28:28 +0200 Subject: [PATCH v7 02/11] ARM: ftrace: Define ftrace_stub_graph MIME-Version: 1.0 Message-Id: <20240421-arm32-cfi-v7-2-6e132a948cc8@linaro.org> References: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> In-Reply-To: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> To: Russell King , Sami Tolvanen , Kees Cook , Nathan Chancellor , Nick Desaulniers , Ard Biesheuvel , Arnd Bergmann Cc: linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev, Linus Walleij X-Mailer: b4 0.13.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240421_112833_478750_BD57F465 X-CRM114-Status: GOOD ( 10.42 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Several architectures defines this stub for the graph tracer, and it is needed for CFI, as it needs a separate symbol for it. The trick from include/asm-generic/vmlinux.lds.h to define ftrace_stub_graph to ftrace_stub isn't working when using CFI. Commit 883bbbffa5a4 contains the details. Tested-by: Kees Cook Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-ftrace.S | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/arm/kernel/entry-ftrace.S b/arch/arm/kernel/entry-ftrace.S index 3e7bcaca5e07..bc598e3d8dd2 100644 --- a/arch/arm/kernel/entry-ftrace.S +++ b/arch/arm/kernel/entry-ftrace.S @@ -271,6 +271,10 @@ ENTRY(ftrace_stub) ret lr ENDPROC(ftrace_stub) +ENTRY(ftrace_stub_graph) + ret lr +ENDPROC(ftrace_stub_graph) + #ifdef CONFIG_DYNAMIC_FTRACE __INIT From patchwork Sun Apr 21 18:28:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13637482 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 58A4DC04FF8 for ; Sun, 21 Apr 2024 18:29:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=6lIi4tMOI55nDeb79B61DeIBtifIFahgj35u7fyXtFQ=; b=ZP3m7C7C/M+uUQ 7i86Zaopp1RtfYoxfwNLa0co/BowixCrwIQduuar5y5VusjJU5Ru1Hu1bQxQqT5yC6aGdC2AubvMr jSfiFGNyJ8ZuIQUZCZ03H/WnInzJ5S1dTHjYJ+rU0Bm2FwYTsPSAtlu1pk0ZK6I8kS5lLGnNPo+fh ybcZ6PyFybkW8LocBBMR7BjYcSV0Biz7g50ZFw/t51MzJtAanTzu2yCbYTCjsFyoNl79JiDnoh2V2 B/Q8/PlKRdF6PgGC22ARVF/gCSRUrlhPDfxWhUz/Me9n0IJXUkLPb8QKXTbUrk+mXRcy1Z77Qg7qG j8rWMF0OYXQNyIXGpUBQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvq-0000000B48Q-43ft; Sun, 21 Apr 2024 18:28:54 +0000 Received: from mail-lf1-x132.google.com ([2a00:1450:4864:20::132]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvX-0000000B40Z-31MM for linux-arm-kernel@lists.infradead.org; Sun, 21 Apr 2024 18:28:37 +0000 Received: by mail-lf1-x132.google.com with SMTP id 2adb3069b0e04-51abd580902so2063412e87.1 for ; Sun, 21 Apr 2024 11:28:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1713724112; x=1714328912; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=3RTNeV4li+DU0s3fUIcbK2lqyEFO9l8a9eRKgSj1QCA=; b=dJjUqbum2bZjktluWy3HEBQ9u9NK8B6zB9Ysc/+WwjfSBy1WEZ12+ezy10ZZDyvEVU xAMWyuFMVwI62Q6lGf0jj6tC2VsIayy5mS3DSPELwP5StPtZAtd+t0ZL/n0APb1b+S5H hAwdOhi39pWwhL1ghI9GSP1XVmU9qbGP56IlAXlNfbslBuqUUwhf+kSm0ZiUL4QVE/vp ZTpeZxkKykmLz/zoBnENLLirEUh5CITyeuiqkBVT/tPWOIPESMeaGmUlYwRP6tbepszO +CeFitnoAa2Ch61IvHGukn1CePE/5f2Rs90nkaePzrQRas/Q0evWg07bQfci4n5VU0gV WP7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713724112; x=1714328912; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3RTNeV4li+DU0s3fUIcbK2lqyEFO9l8a9eRKgSj1QCA=; b=i4GR74+3RU0iiXqfsJpQ+VyQb0dTXfd4T+2jM8/YjRKLD24DVrbCB5Ew+sRG9Xi+6t xI9wVsmbxntPFFdg9L/QV3wMsnGSGob/XBj6wex755M/54wLPXO9N95qgK2eu1wovS10 CMZcpyy7RMQ47Ncei3UmeoPm+msUJtkL/0IL96jyzah5/Cg/CKgxq3j9dNCwZdIR7T1f fepCETI81m1qjQ6yl1vIJl5pMJAVfhQihaajek0/b+BHnWg42eTMNqnFydTceN99l4jv iKHtzSzMq1TP2mswFU966aYLrwJIiFO8YxPeb3SHVY3GSptjy7FHYoi1HCT4i9LQd5dz lkoQ== X-Gm-Message-State: AOJu0Yy4qlS7hsTCroh+Zm6R0LSgghawNB+0o9GP4wVBEZX8P62DniTx U7qkxOmpa3kiuR8DGq02EW3qrMi7ascgqg7qmamdYTt4iaDv6X6ZOSjuf2f5zHw= X-Google-Smtp-Source: AGHT+IGEri83XqiaNY3JP7cfMUlDk+IIgO92+uFqPCCqOh9Tdj12LCfiAaI8A8GP2fJdSDtQ3Ir5Mw== X-Received: by 2002:a05:6512:1101:b0:51a:b7ee:deed with SMTP id l1-20020a056512110100b0051ab7eedeedmr2827140lfg.14.1713724112051; Sun, 21 Apr 2024 11:28:32 -0700 (PDT) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id s14-20020a19770e000000b005188ea5d2b5sm1549641lfc.303.2024.04.21.11.28.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Apr 2024 11:28:31 -0700 (PDT) From: Linus Walleij Date: Sun, 21 Apr 2024 20:28:29 +0200 Subject: [PATCH v7 03/11] ARM: mm: Make tlbflush routines CFI safe MIME-Version: 1.0 Message-Id: <20240421-arm32-cfi-v7-3-6e132a948cc8@linaro.org> References: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> In-Reply-To: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> To: Russell King , Sami Tolvanen , Kees Cook , Nathan Chancellor , Nick Desaulniers , Ard Biesheuvel , Arnd Bergmann Cc: linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev, Linus Walleij X-Mailer: b4 0.13.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240421_112835_789353_81A6BD84 X-CRM114-Status: GOOD ( 18.88 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel Instead of avoiding CFI entirely on the TLB flush helpers, reorganize the code so that the CFI machinery can deal with it. The important things to take into account are: - functions in asm called indirectly from C need to be defined using SYM_TYPED_FUNC_START() - a reference to the asm function needs to be visible to the compiler, in order to get it to emit the typeid symbol. The latter means that defining the cpu_tlb_fns structs is best done from C code, so that the references in the static initializers will be visible to the compiler. Signed-off-by: Ard Biesheuvel Tested-by: Kees Cook Signed-off-by: Linus Walleij --- arch/arm/mm/Makefile | 1 + arch/arm/mm/proc-macros.S | 15 --------- arch/arm/mm/tlb-fa.S | 12 +++---- arch/arm/mm/tlb-v4.S | 15 +++++---- arch/arm/mm/tlb-v4wb.S | 12 +++---- arch/arm/mm/tlb-v4wbi.S | 12 +++---- arch/arm/mm/tlb-v6.S | 12 +++---- arch/arm/mm/tlb-v7.S | 14 +++----- arch/arm/mm/tlb.c | 84 +++++++++++++++++++++++++++++++++++++++++++++++ 9 files changed, 119 insertions(+), 58 deletions(-) diff --git a/arch/arm/mm/Makefile b/arch/arm/mm/Makefile index 71b858c9b10c..cc8255fdf56e 100644 --- a/arch/arm/mm/Makefile +++ b/arch/arm/mm/Makefile @@ -62,6 +62,7 @@ obj-$(CONFIG_CPU_TLB_FEROCEON) += tlb-v4wbi.o # reuse v4wbi TLB functions obj-$(CONFIG_CPU_TLB_V6) += tlb-v6.o obj-$(CONFIG_CPU_TLB_V7) += tlb-v7.o obj-$(CONFIG_CPU_TLB_FA) += tlb-fa.o +obj-y += tlb.o obj-$(CONFIG_CPU_ARM7TDMI) += proc-arm7tdmi.o obj-$(CONFIG_CPU_ARM720T) += proc-arm720.o diff --git a/arch/arm/mm/proc-macros.S b/arch/arm/mm/proc-macros.S index e43f6d716b4b..c0acfeac3e84 100644 --- a/arch/arm/mm/proc-macros.S +++ b/arch/arm/mm/proc-macros.S @@ -338,21 +338,6 @@ ENTRY(\name\()_cache_fns) .size \name\()_cache_fns, . - \name\()_cache_fns .endm -.macro define_tlb_functions name:req, flags_up:req, flags_smp - .type \name\()_tlb_fns, #object - .align 2 -ENTRY(\name\()_tlb_fns) - .long \name\()_flush_user_tlb_range - .long \name\()_flush_kern_tlb_range - .ifnb \flags_smp - ALT_SMP(.long \flags_smp ) - ALT_UP(.long \flags_up ) - .else - .long \flags_up - .endif - .size \name\()_tlb_fns, . - \name\()_tlb_fns -.endm - .macro globl_equ x, y .globl \x .equ \x, \y diff --git a/arch/arm/mm/tlb-fa.S b/arch/arm/mm/tlb-fa.S index def6161ec452..85a6fe766b21 100644 --- a/arch/arm/mm/tlb-fa.S +++ b/arch/arm/mm/tlb-fa.S @@ -15,6 +15,7 @@ */ #include #include +#include #include #include #include @@ -31,7 +32,7 @@ * - mm - mm_struct describing address space */ .align 4 -ENTRY(fa_flush_user_tlb_range) +SYM_TYPED_FUNC_START(fa_flush_user_tlb_range) vma_vm_mm ip, r2 act_mm r3 @ get current->active_mm eors r3, ip, r3 @ == mm ? @@ -46,9 +47,10 @@ ENTRY(fa_flush_user_tlb_range) blo 1b mcr p15, 0, r3, c7, c10, 4 @ data write barrier ret lr +SYM_FUNC_END(fa_flush_user_tlb_range) -ENTRY(fa_flush_kern_tlb_range) +SYM_TYPED_FUNC_START(fa_flush_kern_tlb_range) mov r3, #0 mcr p15, 0, r3, c7, c10, 4 @ drain WB bic r0, r0, #0x0ff @@ -60,8 +62,4 @@ ENTRY(fa_flush_kern_tlb_range) mcr p15, 0, r3, c7, c10, 4 @ data write barrier mcr p15, 0, r3, c7, c5, 4 @ prefetch flush (isb) ret lr - - __INITDATA - - /* define struct cpu_tlb_fns (see and proc-macros.S) */ - define_tlb_functions fa, fa_tlb_flags +SYM_FUNC_END(fa_flush_kern_tlb_range) diff --git a/arch/arm/mm/tlb-v4.S b/arch/arm/mm/tlb-v4.S index b962b4e75158..09ff69008d94 100644 --- a/arch/arm/mm/tlb-v4.S +++ b/arch/arm/mm/tlb-v4.S @@ -11,6 +11,7 @@ */ #include #include +#include #include #include #include @@ -27,7 +28,7 @@ * - mm - mm_struct describing address space */ .align 5 -ENTRY(v4_flush_user_tlb_range) +SYM_TYPED_FUNC_START(v4_flush_user_tlb_range) vma_vm_mm ip, r2 act_mm r3 @ get current->active_mm eors r3, ip, r3 @ == mm ? @@ -40,6 +41,7 @@ ENTRY(v4_flush_user_tlb_range) cmp r0, r1 blo 1b ret lr +SYM_FUNC_END(v4_flush_user_tlb_range) /* * v4_flush_kern_tlb_range(start, end) @@ -50,10 +52,11 @@ ENTRY(v4_flush_user_tlb_range) * - start - virtual address (may not be aligned) * - end - virtual address (may not be aligned) */ +#ifdef CONFIG_CFI_CLANG +SYM_TYPED_FUNC_START(v4_flush_kern_tlb_range) + b .v4_flush_kern_tlb_range +SYM_FUNC_END(v4_flush_kern_tlb_range) +#else .globl v4_flush_kern_tlb_range .equ v4_flush_kern_tlb_range, .v4_flush_kern_tlb_range - - __INITDATA - - /* define struct cpu_tlb_fns (see and proc-macros.S) */ - define_tlb_functions v4, v4_tlb_flags +#endif diff --git a/arch/arm/mm/tlb-v4wb.S b/arch/arm/mm/tlb-v4wb.S index 9348bba7586a..04e46c359e75 100644 --- a/arch/arm/mm/tlb-v4wb.S +++ b/arch/arm/mm/tlb-v4wb.S @@ -11,6 +11,7 @@ */ #include #include +#include #include #include #include @@ -27,7 +28,7 @@ * - mm - mm_struct describing address space */ .align 5 -ENTRY(v4wb_flush_user_tlb_range) +SYM_TYPED_FUNC_START(v4wb_flush_user_tlb_range) vma_vm_mm ip, r2 act_mm r3 @ get current->active_mm eors r3, ip, r3 @ == mm ? @@ -43,6 +44,7 @@ ENTRY(v4wb_flush_user_tlb_range) cmp r0, r1 blo 1b ret lr +SYM_FUNC_END(v4wb_flush_user_tlb_range) /* * v4_flush_kern_tlb_range(start, end) @@ -53,7 +55,7 @@ ENTRY(v4wb_flush_user_tlb_range) * - start - virtual address (may not be aligned) * - end - virtual address (may not be aligned) */ -ENTRY(v4wb_flush_kern_tlb_range) +SYM_TYPED_FUNC_START(v4wb_flush_kern_tlb_range) mov r3, #0 mcr p15, 0, r3, c7, c10, 4 @ drain WB bic r0, r0, #0x0ff @@ -64,8 +66,4 @@ ENTRY(v4wb_flush_kern_tlb_range) cmp r0, r1 blo 1b ret lr - - __INITDATA - - /* define struct cpu_tlb_fns (see and proc-macros.S) */ - define_tlb_functions v4wb, v4wb_tlb_flags +SYM_FUNC_END(v4wb_flush_kern_tlb_range) diff --git a/arch/arm/mm/tlb-v4wbi.S b/arch/arm/mm/tlb-v4wbi.S index d4f9040a4111..502dfe5628a3 100644 --- a/arch/arm/mm/tlb-v4wbi.S +++ b/arch/arm/mm/tlb-v4wbi.S @@ -11,6 +11,7 @@ */ #include #include +#include #include #include #include @@ -26,7 +27,7 @@ * - mm - mm_struct describing address space */ .align 5 -ENTRY(v4wbi_flush_user_tlb_range) +SYM_TYPED_FUNC_START(v4wbi_flush_user_tlb_range) vma_vm_mm ip, r2 act_mm r3 @ get current->active_mm eors r3, ip, r3 @ == mm ? @@ -43,8 +44,9 @@ ENTRY(v4wbi_flush_user_tlb_range) cmp r0, r1 blo 1b ret lr +SYM_FUNC_END(v4wbi_flush_user_tlb_range) -ENTRY(v4wbi_flush_kern_tlb_range) +SYM_TYPED_FUNC_START(v4wbi_flush_kern_tlb_range) mov r3, #0 mcr p15, 0, r3, c7, c10, 4 @ drain WB bic r0, r0, #0x0ff @@ -55,8 +57,4 @@ ENTRY(v4wbi_flush_kern_tlb_range) cmp r0, r1 blo 1b ret lr - - __INITDATA - - /* define struct cpu_tlb_fns (see and proc-macros.S) */ - define_tlb_functions v4wbi, v4wbi_tlb_flags +SYM_FUNC_END(v4wbi_flush_kern_tlb_range) diff --git a/arch/arm/mm/tlb-v6.S b/arch/arm/mm/tlb-v6.S index 1d91e49b2c2d..8256a67ac654 100644 --- a/arch/arm/mm/tlb-v6.S +++ b/arch/arm/mm/tlb-v6.S @@ -9,6 +9,7 @@ */ #include #include +#include #include #include #include @@ -32,7 +33,7 @@ * - the "Invalidate single entry" instruction will invalidate * both the I and the D TLBs on Harvard-style TLBs */ -ENTRY(v6wbi_flush_user_tlb_range) +SYM_TYPED_FUNC_START(v6wbi_flush_user_tlb_range) vma_vm_mm r3, r2 @ get vma->vm_mm mov ip, #0 mmid r3, r3 @ get vm_mm->context.id @@ -56,6 +57,7 @@ ENTRY(v6wbi_flush_user_tlb_range) blo 1b mcr p15, 0, ip, c7, c10, 4 @ data synchronization barrier ret lr +SYM_FUNC_END(v6wbi_flush_user_tlb_range) /* * v6wbi_flush_kern_tlb_range(start,end) @@ -65,7 +67,7 @@ ENTRY(v6wbi_flush_user_tlb_range) * - start - start address (may not be aligned) * - end - end address (exclusive, may not be aligned) */ -ENTRY(v6wbi_flush_kern_tlb_range) +SYM_TYPED_FUNC_START(v6wbi_flush_kern_tlb_range) mov r2, #0 mcr p15, 0, r2, c7, c10, 4 @ drain write buffer mov r0, r0, lsr #PAGE_SHIFT @ align address @@ -85,8 +87,4 @@ ENTRY(v6wbi_flush_kern_tlb_range) mcr p15, 0, r2, c7, c10, 4 @ data synchronization barrier mcr p15, 0, r2, c7, c5, 4 @ prefetch flush (isb) ret lr - - __INIT - - /* define struct cpu_tlb_fns (see and proc-macros.S) */ - define_tlb_functions v6wbi, v6wbi_tlb_flags +SYM_FUNC_END(v6wbi_flush_kern_tlb_range) diff --git a/arch/arm/mm/tlb-v7.S b/arch/arm/mm/tlb-v7.S index 35fd6d4f0d03..f1aa0764a2cc 100644 --- a/arch/arm/mm/tlb-v7.S +++ b/arch/arm/mm/tlb-v7.S @@ -10,6 +10,7 @@ */ #include #include +#include #include #include #include @@ -31,7 +32,7 @@ * - the "Invalidate single entry" instruction will invalidate * both the I and the D TLBs on Harvard-style TLBs */ -ENTRY(v7wbi_flush_user_tlb_range) +SYM_TYPED_FUNC_START(v7wbi_flush_user_tlb_range) vma_vm_mm r3, r2 @ get vma->vm_mm mmid r3, r3 @ get vm_mm->context.id dsb ish @@ -57,7 +58,7 @@ ENTRY(v7wbi_flush_user_tlb_range) blo 1b dsb ish ret lr -ENDPROC(v7wbi_flush_user_tlb_range) +SYM_FUNC_END(v7wbi_flush_user_tlb_range) /* * v7wbi_flush_kern_tlb_range(start,end) @@ -67,7 +68,7 @@ ENDPROC(v7wbi_flush_user_tlb_range) * - start - start address (may not be aligned) * - end - end address (exclusive, may not be aligned) */ -ENTRY(v7wbi_flush_kern_tlb_range) +SYM_TYPED_FUNC_START(v7wbi_flush_kern_tlb_range) dsb ish mov r0, r0, lsr #PAGE_SHIFT @ align address mov r1, r1, lsr #PAGE_SHIFT @@ -86,9 +87,4 @@ ENTRY(v7wbi_flush_kern_tlb_range) dsb ish isb ret lr -ENDPROC(v7wbi_flush_kern_tlb_range) - - __INIT - - /* define struct cpu_tlb_fns (see and proc-macros.S) */ - define_tlb_functions v7wbi, v7wbi_tlb_flags_up, flags_smp=v7wbi_tlb_flags_smp +SYM_FUNC_END(v7wbi_flush_kern_tlb_range) diff --git a/arch/arm/mm/tlb.c b/arch/arm/mm/tlb.c new file mode 100644 index 000000000000..42359793120b --- /dev/null +++ b/arch/arm/mm/tlb.c @@ -0,0 +1,84 @@ +// SPDX-License-Identifier: GPL-2.0-only +// Copyright 2024 Google LLC +// Author: Ard Biesheuvel + +#include +#include + +#ifdef CONFIG_CPU_TLB_V4WT +void v4_flush_user_tlb_range(unsigned long, unsigned long, struct vm_area_struct *); +void v4_flush_kern_tlb_range(unsigned long, unsigned long); + +struct cpu_tlb_fns v4_tlb_fns __initconst = { + .flush_user_range = v4_flush_user_tlb_range, + .flush_kern_range = v4_flush_kern_tlb_range, + .tlb_flags = v4_tlb_flags, +}; +#endif + +#ifdef CONFIG_CPU_TLB_V4WB +void v4wb_flush_user_tlb_range(unsigned long, unsigned long, struct vm_area_struct *); +void v4wb_flush_kern_tlb_range(unsigned long, unsigned long); + +struct cpu_tlb_fns v4wb_tlb_fns __initconst = { + .flush_user_range = v4wb_flush_user_tlb_range, + .flush_kern_range = v4wb_flush_kern_tlb_range, + .tlb_flags = v4wb_tlb_flags, +}; +#endif + +#if defined(CONFIG_CPU_TLB_V4WBI) || defined(CONFIG_CPU_TLB_FEROCEON) +void v4wbi_flush_user_tlb_range(unsigned long, unsigned long, struct vm_area_struct *); +void v4wbi_flush_kern_tlb_range(unsigned long, unsigned long); + +struct cpu_tlb_fns v4wbi_tlb_fns __initconst = { + .flush_user_range = v4wbi_flush_user_tlb_range, + .flush_kern_range = v4wbi_flush_kern_tlb_range, + .tlb_flags = v4wbi_tlb_flags, +}; +#endif + +#ifdef CONFIG_CPU_TLB_V6 +void v6wbi_flush_user_tlb_range(unsigned long, unsigned long, struct vm_area_struct *); +void v6wbi_flush_kern_tlb_range(unsigned long, unsigned long); + +struct cpu_tlb_fns v6wbi_tlb_fns __initconst = { + .flush_user_range = v6wbi_flush_user_tlb_range, + .flush_kern_range = v6wbi_flush_kern_tlb_range, + .tlb_flags = v6wbi_tlb_flags, +}; +#endif + +#ifdef CONFIG_CPU_TLB_V7 +void v7wbi_flush_user_tlb_range(unsigned long, unsigned long, struct vm_area_struct *); +void v7wbi_flush_kern_tlb_range(unsigned long, unsigned long); + +struct cpu_tlb_fns v7wbi_tlb_fns __initconst = { + .flush_user_range = v7wbi_flush_user_tlb_range, + .flush_kern_range = v7wbi_flush_kern_tlb_range, + .tlb_flags = IS_ENABLED(CONFIG_SMP) ? v7wbi_tlb_flags_smp + : v7wbi_tlb_flags_up, +}; + +#ifdef CONFIG_SMP_ON_UP +/* This will be run-time patched so the offset better be right */ +static_assert(offsetof(struct cpu_tlb_fns, tlb_flags) == 8); + +asm(" .pushsection \".alt.smp.init\", \"a\" \n" \ + " .align 2 \n" \ + " .long v7wbi_tlb_fns + 8 - . \n" \ + " .long " __stringify(v7wbi_tlb_flags_up) " \n" \ + " .popsection \n"); +#endif +#endif + +#ifdef CONFIG_CPU_TLB_FA +void fa_flush_user_tlb_range(unsigned long, unsigned long, struct vm_area_struct *); +void fa_flush_kern_tlb_range(unsigned long, unsigned long); + +struct cpu_tlb_fns fa_tlb_fns __initconst = { + .flush_user_range = fa_flush_user_tlb_range, + .flush_kern_range = fa_flush_kern_tlb_range, + .tlb_flags = fa_tlb_flags, +}; +#endif From patchwork Sun Apr 21 18:28:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13637481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E15C7C4345F for ; Sun, 21 Apr 2024 18:29:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UnqATYBaURyd0Kazm1XyRpTSYZW2O/5FmhuFYRj3n4U=; b=WlS4szfRIgzLdo Cn2swgaOc8iVDoVBGQ25kB+3zcYJlC9ees5HYHpDrGMWj/mfeeH2zezEwzz4cNni0aL8gJAIzQyU+ F1dhFsZQOnSCAvp5/7mJxBkrC75EQaTQLIIa0SnO/d4ge5r+QtD70tET2Zn4FbXoctxFueDZhnfT2 V7VHRG3m2l9OYnCPXhqTCyhDHBbwI9G0mmNIbMahGnEGBh+SIjrTYkVWmlQDi4T64niv/0VKUzj2l 8P+xfejcfLRc13dMh/17l7R+n0F4znc8dUgUO5J5xmJUyQaR1ubhtCsh2Q/NmFnsg5Cl3t8B/M3f1 CtpIAaqd1bxVrenH16gQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvr-0000000B48y-3dZG; Sun, 21 Apr 2024 18:28:55 +0000 Received: from mail-lj1-x22c.google.com ([2a00:1450:4864:20::22c]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvZ-0000000B41B-0mNW for linux-arm-kernel@lists.infradead.org; Sun, 21 Apr 2024 18:28:40 +0000 Received: by mail-lj1-x22c.google.com with SMTP id 38308e7fff4ca-2d8b2389e73so42837021fa.3 for ; Sun, 21 Apr 2024 11:28:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1713724114; x=1714328914; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=CXHI6Nsz1rAVeXFpM1eB04t0yjhK7F1Xu95evrEKmZs=; b=TRoImoFGVY63q3cWfuY2awx9i4wyYlyH+u1rsEqGp0EyKfljI0oW86/YmxYE9bc6dA 0n/ymaplyQe/uXv2wIqErl6TEtuKFL15k9x1Bl8Bgt03Sns/dZlC+h9UEPKyLwKC/HxM CYtDVz5kaihdmVNME1aFxYkpaq1grOK0sqHML91SXtUxWAEgqIjJznd1PjZ9c585Er6N Q3G+B6UpuGMHldW1I2twd5l902zTRgNF+op7FBZz13jTiuiK+eUJuDXD84Kb1zXGxwon Z/XGsHxouNKTM/Drah0VZoZ12uyjA5TYEeEKbjJ4+HE6kj9HaqCMrFXgiu1AsOPdIrVS EikQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713724114; x=1714328914; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CXHI6Nsz1rAVeXFpM1eB04t0yjhK7F1Xu95evrEKmZs=; b=jiyPbggi023BKg5tDNhyDBU4Omp6Y70eIR3BZi9ZNQsCobIUg7Od2eGNysgLmxZ5/m 9IXw3UhS9B646Q5kqEaP3Bf7Rn7TTZ4R9jEkLw7gw7/m5BSpbuyBTrWv3lA3MEvRBOSo rejkAo8STw1WUhtBBsdEAMCnFvA6ELTXPCyxkTKqwuP0wnP2mYUPGHGrSviG675OVRuF 8EVjpQJ8tDkFvukedcw1y3lmzQ+oRkC6bkf2Lfq17SZrtQbKfbW2TBPzz1+FX/Mn5ypj 6TB/quBduYB7trmSerKEGZlMjql5QadOmjnTDWS7bNZmgYTYKXudKsVbdxX/xuaR+0VY /LGw== X-Gm-Message-State: AOJu0YyQd5V6S5NBYP2AIMLm7u9PALxgfG/M+/cMg/zS8irFbMRpn8QW mPcwA1Op7/x5c557JnWCxXJapbXfV03lca7JqnMl71KmlihYdMf+uQo3y2yrYffG6HMmFfFIUwF q X-Google-Smtp-Source: AGHT+IHI8ubfJRuhzDC3wOhjJfX8IdVvqx4QDIuynL2C6QRXvbonShxzT1osPb6N5EYCnQ8weeSVfA== X-Received: by 2002:a05:6512:98c:b0:516:e8b1:9ed5 with SMTP id w12-20020a056512098c00b00516e8b19ed5mr4373789lft.59.1713724113900; Sun, 21 Apr 2024 11:28:33 -0700 (PDT) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id s14-20020a19770e000000b005188ea5d2b5sm1549641lfc.303.2024.04.21.11.28.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Apr 2024 11:28:33 -0700 (PDT) From: Linus Walleij Date: Sun, 21 Apr 2024 20:28:31 +0200 Subject: [PATCH v7 05/11] ARM: mm: Use symbol alias for two cache functions MIME-Version: 1.0 Message-Id: <20240421-arm32-cfi-v7-5-6e132a948cc8@linaro.org> References: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> In-Reply-To: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> To: Russell King , Sami Tolvanen , Kees Cook , Nathan Chancellor , Nick Desaulniers , Ard Biesheuvel , Arnd Bergmann Cc: linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev, Linus Walleij X-Mailer: b4 0.13.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240421_112837_412484_C1AE4D0A X-CRM114-Status: GOOD ( 14.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The cache functions to flush user cache (*_flush_user_cache_all) and coherent kernel range (*_coherent_kern_range) are in many cases just a branch to the corresponfing userspace or kernelspace function. These functions also have the same arguments. Simplify these two by using SYM_FUNC_ALIAS() in all affected sites. The NOP cache has very many similar calls which are just returns, but it would be confusing to use aliases here, so leave all the explicit returns and drop a comment on why we are not using aliases. Signed-off-by: Linus Walleij --- arch/arm/mm/cache-fa.S | 8 ++------ arch/arm/mm/cache-nop.S | 4 ++++ arch/arm/mm/cache-v4.S | 4 +--- arch/arm/mm/cache-v4wb.S | 8 ++------ arch/arm/mm/cache-v4wt.S | 8 ++------ arch/arm/mm/cache-v6.S | 4 +--- arch/arm/mm/cache-v7.S | 4 +--- arch/arm/mm/proc-arm1020.S | 8 ++------ arch/arm/mm/proc-arm1020e.S | 8 ++------ arch/arm/mm/proc-arm1022.S | 8 ++------ arch/arm/mm/proc-arm1026.S | 8 ++------ arch/arm/mm/proc-arm920.S | 8 ++------ arch/arm/mm/proc-arm922.S | 8 ++------ arch/arm/mm/proc-arm925.S | 8 ++------ arch/arm/mm/proc-arm926.S | 8 ++------ arch/arm/mm/proc-arm940.S | 8 ++------ arch/arm/mm/proc-arm946.S | 8 ++------ arch/arm/mm/proc-feroceon.S | 8 ++------ arch/arm/mm/proc-mohawk.S | 8 ++------ arch/arm/mm/proc-xsc3.S | 8 ++------ arch/arm/mm/proc-xscale.S | 4 +--- 21 files changed, 40 insertions(+), 108 deletions(-) diff --git a/arch/arm/mm/cache-fa.S b/arch/arm/mm/cache-fa.S index c3642d5daf38..6fe06608f34e 100644 --- a/arch/arm/mm/cache-fa.S +++ b/arch/arm/mm/cache-fa.S @@ -52,9 +52,7 @@ SYM_FUNC_END(fa_flush_icache_all) * Clean and invalidate all cache entries in a particular address * space. */ -SYM_TYPED_FUNC_START(fa_flush_user_cache_all) - b fa_flush_kern_cache_all -SYM_FUNC_END(fa_flush_user_cache_all) +SYM_FUNC_ALIAS(fa_flush_user_cache_all, fa_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -113,9 +111,7 @@ SYM_FUNC_END(fa_flush_user_cache_range) * - start - virtual start address * - end - virtual end address */ -SYM_TYPED_FUNC_START(fa_coherent_kern_range) - b fa_coherent_user_range -SYM_FUNC_END(fa_coherent_kern_range) +SYM_FUNC_ALIAS(fa_coherent_kern_range, fa_coherent_user_range) /* * coherent_user_range(start, end) diff --git a/arch/arm/mm/cache-nop.S b/arch/arm/mm/cache-nop.S index 56e94091a55f..cd191aa90313 100644 --- a/arch/arm/mm/cache-nop.S +++ b/arch/arm/mm/cache-nop.S @@ -6,6 +6,10 @@ #include "proc-macros.S" +/* + * These are all open-coded instead of aliased, to make clear + * what is going on here: all functions are stubbed out. + */ SYM_TYPED_FUNC_START(nop_flush_icache_all) ret lr SYM_FUNC_END(nop_flush_icache_all) diff --git a/arch/arm/mm/cache-v4.S b/arch/arm/mm/cache-v4.S index 22d9c9d9e0d7..f7b7e498d3b6 100644 --- a/arch/arm/mm/cache-v4.S +++ b/arch/arm/mm/cache-v4.S @@ -28,9 +28,7 @@ SYM_FUNC_END(v4_flush_icache_all) * * - mm - mm_struct describing address space */ -SYM_TYPED_FUNC_START(v4_flush_user_cache_all) - b v4_flush_kern_cache_all -SYM_FUNC_END(v4_flush_user_cache_all) +SYM_FUNC_ALIAS(v4_flush_user_cache_all, v4_flush_kern_cache_all) /* * flush_kern_cache_all() diff --git a/arch/arm/mm/cache-v4wb.S b/arch/arm/mm/cache-v4wb.S index 0d97b594e23f..19fae44b89cd 100644 --- a/arch/arm/mm/cache-v4wb.S +++ b/arch/arm/mm/cache-v4wb.S @@ -66,9 +66,7 @@ SYM_FUNC_END(v4wb_flush_icache_all) * Clean and invalidate all cache entries in a particular address * space. */ -SYM_TYPED_FUNC_START(v4wb_flush_user_cache_all) - b v4wb_flush_kern_cache_all -SYM_FUNC_END(v4wb_flush_user_cache_all) +SYM_FUNC_ALIAS(v4wb_flush_user_cache_all, v4wb_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -151,9 +149,7 @@ SYM_FUNC_END(v4wb_flush_kern_dcache_area) * - start - virtual start address * - end - virtual end address */ -SYM_TYPED_FUNC_START(v4wb_coherent_kern_range) - b v4wb_coherent_user_range -SYM_FUNC_END(v4wb_coherent_kern_range) +SYM_FUNC_ALIAS(v4wb_coherent_kern_range, v4wb_coherent_user_range) /* * coherent_user_range(start, end) diff --git a/arch/arm/mm/cache-v4wt.S b/arch/arm/mm/cache-v4wt.S index eee6d8f06b4d..5be76ff861d7 100644 --- a/arch/arm/mm/cache-v4wt.S +++ b/arch/arm/mm/cache-v4wt.S @@ -56,9 +56,7 @@ SYM_FUNC_END(v4wt_flush_icache_all) * Invalidate all cache entries in a particular address * space. */ -SYM_TYPED_FUNC_START(v4wt_flush_user_cache_all) - b v4wt_flush_kern_cache_all -SYM_FUNC_END(v4wt_flush_user_cache_all) +SYM_FUNC_ALIAS(v4wt_flush_user_cache_all, v4wt_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -109,9 +107,7 @@ SYM_FUNC_END(v4wt_flush_user_cache_range) * - start - virtual start address * - end - virtual end address */ -SYM_TYPED_FUNC_START(v4wt_coherent_kern_range) - b v4wt_coherent_user_range -SYM_FUNC_END(v4wt_coherent_kern_range) +SYM_FUNC_ALIAS(v4wt_coherent_kern_range, v4wt_coherent_user_range) /* * coherent_user_range(start, end) diff --git a/arch/arm/mm/cache-v6.S b/arch/arm/mm/cache-v6.S index 5c7549a49db5..a590044b7282 100644 --- a/arch/arm/mm/cache-v6.S +++ b/arch/arm/mm/cache-v6.S @@ -116,9 +116,7 @@ SYM_FUNC_END(v6_flush_user_cache_range) * It is assumed that: * - the Icache does not read data from the write buffer */ -SYM_TYPED_FUNC_START(v6_coherent_kern_range) - b v6_coherent_user_range -SYM_FUNC_END(v6_coherent_kern_range) +SYM_FUNC_ALIAS(v6_coherent_kern_range, v6_coherent_user_range) /* * v6_coherent_user_range(start,end) diff --git a/arch/arm/mm/cache-v7.S b/arch/arm/mm/cache-v7.S index 5908dd54de47..6c0bc756d29a 100644 --- a/arch/arm/mm/cache-v7.S +++ b/arch/arm/mm/cache-v7.S @@ -260,9 +260,7 @@ SYM_FUNC_END(v7_flush_user_cache_range) * It is assumed that: * - the Icache does not read data from the write buffer */ -SYM_TYPED_FUNC_START(v7_coherent_kern_range) - b v7_coherent_user_range -SYM_FUNC_END(v7_coherent_kern_range) +SYM_FUNC_ALIAS(v7_coherent_kern_range, v7_coherent_user_range) /* * v7_coherent_user_range(start,end) diff --git a/arch/arm/mm/proc-arm1020.S b/arch/arm/mm/proc-arm1020.S index a3f99e1c1186..379628e8ef4e 100644 --- a/arch/arm/mm/proc-arm1020.S +++ b/arch/arm/mm/proc-arm1020.S @@ -127,9 +127,7 @@ SYM_FUNC_END(arm1020_flush_icache_all) * Invalidate all cache entries in a particular address * space. */ -SYM_TYPED_FUNC_START(arm1020_flush_user_cache_all) - b arm1020_flush_kern_cache_all -SYM_FUNC_END(arm1020_flush_user_cache_all) +SYM_FUNC_ALIAS(arm1020_flush_user_cache_all, arm1020_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -201,9 +199,7 @@ SYM_FUNC_END(arm1020_flush_user_cache_range) * - start - virtual start address * - end - virtual end address */ -SYM_TYPED_FUNC_START(arm1020_coherent_kern_range) - b arm1020_coherent_user_range -SYM_FUNC_END(arm1020_coherent_kern_range) +SYM_FUNC_ALIAS(arm1020_coherent_kern_range, arm1020_coherent_user_range) /* * coherent_user_range(start, end) diff --git a/arch/arm/mm/proc-arm1020e.S b/arch/arm/mm/proc-arm1020e.S index 64c63eb5d830..b5846fbea040 100644 --- a/arch/arm/mm/proc-arm1020e.S +++ b/arch/arm/mm/proc-arm1020e.S @@ -127,9 +127,7 @@ SYM_FUNC_END(arm1020e_flush_icache_all) * Invalidate all cache entries in a particular address * space. */ -SYM_TYPED_FUNC_START(arm1020e_flush_user_cache_all) - b arm1020e_flush_kern_cache_all -SYM_FUNC_END(arm1020e_flush_user_cache_all) +SYM_FUNC_ALIAS(arm1020e_flush_user_cache_all, arm1020e_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -198,9 +196,7 @@ SYM_FUNC_END(arm1020e_flush_user_cache_range) * - start - virtual start address * - end - virtual end address */ -SYM_TYPED_FUNC_START(arm1020e_coherent_kern_range) - b arm1020e_coherent_user_range -SYM_FUNC_END(arm1020e_coherent_kern_range) +SYM_FUNC_ALIAS(arm1020e_coherent_kern_range, arm1020e_coherent_user_range) /* * coherent_user_range(start, end) diff --git a/arch/arm/mm/proc-arm1022.S b/arch/arm/mm/proc-arm1022.S index e170497353ae..c40b268cc274 100644 --- a/arch/arm/mm/proc-arm1022.S +++ b/arch/arm/mm/proc-arm1022.S @@ -127,9 +127,7 @@ SYM_FUNC_END(arm1022_flush_icache_all) * Invalidate all cache entries in a particular address * space. */ -SYM_TYPED_FUNC_START(arm1022_flush_user_cache_all) - b arm1022_flush_kern_cache_all -SYM_FUNC_END(arm1022_flush_user_cache_all) +SYM_FUNC_ALIAS(arm1022_flush_user_cache_all, arm1022_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -197,9 +195,7 @@ SYM_FUNC_END(arm1022_flush_user_cache_range) * - start - virtual start address * - end - virtual end address */ -SYM_TYPED_FUNC_START(arm1022_coherent_kern_range) - b arm1022_coherent_user_range -SYM_FUNC_END(arm1022_coherent_kern_range) +SYM_FUNC_ALIAS(arm1022_coherent_kern_range, arm1022_coherent_user_range) /* * coherent_user_range(start, end) diff --git a/arch/arm/mm/proc-arm1026.S b/arch/arm/mm/proc-arm1026.S index 4b5a4849ad85..7ef2c6d88dc0 100644 --- a/arch/arm/mm/proc-arm1026.S +++ b/arch/arm/mm/proc-arm1026.S @@ -127,9 +127,7 @@ SYM_FUNC_END(arm1026_flush_icache_all) * Invalidate all cache entries in a particular address * space. */ -SYM_TYPED_FUNC_START(arm1026_flush_user_cache_all) - b arm1026_flush_kern_cache_all -SYM_FUNC_END(arm1026_flush_user_cache_all) +SYM_FUNC_ALIAS(arm1026_flush_user_cache_all, arm1026_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -192,9 +190,7 @@ SYM_FUNC_END(arm1026_flush_user_cache_range) * - start - virtual start address * - end - virtual end address */ -SYM_TYPED_FUNC_START(arm1026_coherent_kern_range) - b arm1026_coherent_user_range -SYM_FUNC_END(arm1026_coherent_kern_range) +SYM_FUNC_ALIAS(arm1026_coherent_kern_range, arm1026_coherent_user_range) /* * coherent_user_range(start, end) diff --git a/arch/arm/mm/proc-arm920.S b/arch/arm/mm/proc-arm920.S index fbf8937eae85..eb89a322a534 100644 --- a/arch/arm/mm/proc-arm920.S +++ b/arch/arm/mm/proc-arm920.S @@ -116,9 +116,7 @@ SYM_FUNC_END(arm920_flush_icache_all) * Invalidate all cache entries in a particular address * space. */ -SYM_TYPED_FUNC_START(arm920_flush_user_cache_all) - b arm920_flush_kern_cache_all -SYM_FUNC_END(arm920_flush_user_cache_all) +SYM_FUNC_ALIAS(arm920_flush_user_cache_all, arm920_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -179,9 +177,7 @@ SYM_FUNC_END(arm920_flush_user_cache_range) * - start - virtual start address * - end - virtual end address */ -SYM_TYPED_FUNC_START(arm920_coherent_kern_range) - b arm920_coherent_user_range -SYM_FUNC_END(arm920_coherent_kern_range) +SYM_FUNC_ALIAS(arm920_coherent_kern_range, arm920_coherent_user_range) /* * coherent_user_range(start, end) diff --git a/arch/arm/mm/proc-arm922.S b/arch/arm/mm/proc-arm922.S index ccfff2b65f49..035a1d1a26b0 100644 --- a/arch/arm/mm/proc-arm922.S +++ b/arch/arm/mm/proc-arm922.S @@ -118,9 +118,7 @@ SYM_FUNC_END(arm922_flush_icache_all) * Clean and invalidate all cache entries in a particular * address space. */ -SYM_TYPED_FUNC_START(arm922_flush_user_cache_all) - b arm922_flush_kern_cache_all -SYM_FUNC_END(arm922_flush_user_cache_all) +SYM_FUNC_ALIAS(arm922_flush_user_cache_all, arm922_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -181,9 +179,7 @@ SYM_FUNC_END(arm922_flush_user_cache_range) * - start - virtual start address * - end - virtual end address */ -SYM_TYPED_FUNC_START(arm922_coherent_kern_range) - b arm922_coherent_user_range -SYM_FUNC_END(arm922_coherent_kern_range) +SYM_FUNC_ALIAS(arm922_coherent_kern_range, arm922_coherent_user_range) /* * coherent_user_range(start, end) diff --git a/arch/arm/mm/proc-arm925.S b/arch/arm/mm/proc-arm925.S index d0f73242f70a..2510722647b4 100644 --- a/arch/arm/mm/proc-arm925.S +++ b/arch/arm/mm/proc-arm925.S @@ -151,9 +151,7 @@ SYM_FUNC_END(arm925_flush_icache_all) * Clean and invalidate all cache entries in a particular * address space. */ -SYM_TYPED_FUNC_START(arm925_flush_user_cache_all) - b arm925_flush_kern_cache_all -SYM_FUNC_END(arm925_flush_user_cache_all) +SYM_FUNC_ALIAS(arm925_flush_user_cache_all, arm925_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -227,9 +225,7 @@ SYM_FUNC_END(arm925_flush_user_cache_range) * - start - virtual start address * - end - virtual end address */ -SYM_TYPED_FUNC_START(arm925_coherent_kern_range) - b arm925_coherent_user_range -SYM_FUNC_END(arm925_coherent_kern_range) +SYM_FUNC_ALIAS(arm925_coherent_kern_range, arm925_coherent_user_range) /* * coherent_user_range(start, end) diff --git a/arch/arm/mm/proc-arm926.S b/arch/arm/mm/proc-arm926.S index 00f953dee122..dac4a22369ba 100644 --- a/arch/arm/mm/proc-arm926.S +++ b/arch/arm/mm/proc-arm926.S @@ -117,9 +117,7 @@ SYM_FUNC_END(arm926_flush_icache_all) * Clean and invalidate all cache entries in a particular * address space. */ -SYM_TYPED_FUNC_START(arm926_flush_user_cache_all) - b arm926_flush_kern_cache_all -SYM_FUNC_END(arm926_flush_user_cache_all) +SYM_FUNC_ALIAS(arm926_flush_user_cache_all, arm926_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -190,9 +188,7 @@ SYM_FUNC_END(arm926_flush_user_cache_range) * - start - virtual start address * - end - virtual end address */ -SYM_TYPED_FUNC_START(arm926_coherent_kern_range) - b arm926_coherent_user_range -SYM_FUNC_END(arm926_coherent_kern_range) +SYM_FUNC_ALIAS(arm926_coherent_kern_range, arm926_coherent_user_range) /* * coherent_user_range(start, end) diff --git a/arch/arm/mm/proc-arm940.S b/arch/arm/mm/proc-arm940.S index 7e32ec271e8a..7c2268059536 100644 --- a/arch/arm/mm/proc-arm940.S +++ b/arch/arm/mm/proc-arm940.S @@ -81,9 +81,7 @@ SYM_FUNC_END(arm940_flush_icache_all) /* * flush_user_cache_all() */ -SYM_TYPED_FUNC_START(arm940_flush_user_cache_all) - b arm940_flush_kern_cache_all -SYM_FUNC_END(arm940_flush_user_cache_all) +SYM_FUNC_ALIAS(arm940_flush_user_cache_all, arm940_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -134,9 +132,7 @@ SYM_FUNC_END(arm940_flush_user_cache_range) * - start - virtual start address * - end - virtual end address */ -SYM_TYPED_FUNC_START(arm940_coherent_kern_range) - b arm940_flush_kern_dcache_area -SYM_FUNC_END(arm940_coherent_kern_range) +SYM_FUNC_ALIAS(arm940_coherent_kern_range, arm940_flush_kern_dcache_area) /* * coherent_user_range(start, end) diff --git a/arch/arm/mm/proc-arm946.S b/arch/arm/mm/proc-arm946.S index 4fc883572e19..3955be1f4521 100644 --- a/arch/arm/mm/proc-arm946.S +++ b/arch/arm/mm/proc-arm946.S @@ -88,9 +88,7 @@ SYM_FUNC_END(arm946_flush_icache_all) /* * flush_user_cache_all() */ -SYM_TYPED_FUNC_START(arm946_flush_user_cache_all) - b arm946_flush_kern_cache_all -SYM_FUNC_END(arm946_flush_user_cache_all) +SYM_FUNC_ALIAS(arm946_flush_user_cache_all, arm946_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -168,9 +166,7 @@ SYM_FUNC_END(arm946_flush_user_cache_range) * - start - virtual start address * - end - virtual end address */ -SYM_TYPED_FUNC_START(arm946_coherent_kern_range) - b arm946_coherent_user_range -SYM_FUNC_END(arm946_coherent_kern_range) +SYM_FUNC_ALIAS(arm946_coherent_kern_range, arm946_coherent_user_range) /* * coherent_user_range(start, end) diff --git a/arch/arm/mm/proc-feroceon.S b/arch/arm/mm/proc-feroceon.S index ee936c23cac5..9b1570ea6858 100644 --- a/arch/arm/mm/proc-feroceon.S +++ b/arch/arm/mm/proc-feroceon.S @@ -136,9 +136,7 @@ SYM_FUNC_END(feroceon_flush_icache_all) * address space. */ .align 5 -SYM_TYPED_FUNC_START(feroceon_flush_user_cache_all) - b feroceon_flush_kern_cache_all -SYM_FUNC_END(feroceon_flush_user_cache_all) +SYM_FUNC_ALIAS(feroceon_flush_user_cache_all, feroceon_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -206,9 +204,7 @@ SYM_FUNC_END(feroceon_flush_user_cache_range) * - end - virtual end address */ .align 5 -SYM_TYPED_FUNC_START(feroceon_coherent_kern_range) - b feroceon_coherent_user_range -SYM_FUNC_END(feroceon_coherent_kern_range) +SYM_FUNC_ALIAS(feroceon_coherent_kern_range, feroceon_coherent_user_range) /* * coherent_user_range(start, end) diff --git a/arch/arm/mm/proc-mohawk.S b/arch/arm/mm/proc-mohawk.S index 519b7ff2c589..0a94cb0464d8 100644 --- a/arch/arm/mm/proc-mohawk.S +++ b/arch/arm/mm/proc-mohawk.S @@ -100,9 +100,7 @@ SYM_FUNC_END(mohawk_flush_icache_all) * Clean and invalidate all cache entries in a particular * address space. */ -SYM_TYPED_FUNC_START(mohawk_flush_user_cache_all) - b mohawk_flush_kern_cache_all -SYM_FUNC_END(mohawk_flush_user_cache_all) +SYM_FUNC_ALIAS(mohawk_flush_user_cache_all, mohawk_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -161,9 +159,7 @@ SYM_FUNC_END(mohawk_flush_user_cache_range) * - start - virtual start address * - end - virtual end address */ -SYM_TYPED_FUNC_START(mohawk_coherent_kern_range) - b mohawk_coherent_user_range -SYM_FUNC_END(mohawk_coherent_kern_range) +SYM_FUNC_ALIAS(mohawk_coherent_kern_range, mohawk_coherent_user_range) /* * coherent_user_range(start, end) diff --git a/arch/arm/mm/proc-xsc3.S b/arch/arm/mm/proc-xsc3.S index f08b3fce4c95..b2d907d748e9 100644 --- a/arch/arm/mm/proc-xsc3.S +++ b/arch/arm/mm/proc-xsc3.S @@ -157,9 +157,7 @@ SYM_FUNC_END(xsc3_flush_icache_all) * Invalidate all cache entries in a particular address * space. */ -SYM_TYPED_FUNC_START(xsc3_flush_user_cache_all) - b xsc3_flush_kern_cache_all -SYM_FUNC_END(xsc3_flush_user_cache_all) +SYM_FUNC_ALIAS(xsc3_flush_user_cache_all, xsc3_flush_kern_cache_all) /* * flush_kern_cache_all() @@ -221,9 +219,7 @@ SYM_FUNC_END(xsc3_flush_user_cache_range) * Note: single I-cache line invalidation isn't used here since * it also trashes the mini I-cache used by JTAG debuggers. */ -SYM_TYPED_FUNC_START(xsc3_coherent_kern_range) - b xsc3_coherent_user_range -SYM_FUNC_END(xsc3_coherent_kern_range) +SYM_FUNC_ALIAS(xsc3_coherent_kern_range, xsc3_coherent_user_range) SYM_TYPED_FUNC_START(xsc3_coherent_user_range) bic r0, r0, #CACHELINESIZE - 1 diff --git a/arch/arm/mm/proc-xscale.S b/arch/arm/mm/proc-xscale.S index 3e427db18d5b..05d9ed952983 100644 --- a/arch/arm/mm/proc-xscale.S +++ b/arch/arm/mm/proc-xscale.S @@ -199,9 +199,7 @@ SYM_FUNC_END(xscale_flush_icache_all) * Invalidate all cache entries in a particular address * space. */ -SYM_TYPED_FUNC_START(xscale_flush_user_cache_all) - b xscale_flush_kern_cache_all -SYM_FUNC_END(xscale_flush_user_cache_all) +SYM_FUNC_ALIAS(xscale_flush_user_cache_all, xscale_flush_kern_cache_all) /* * flush_kern_cache_all() From patchwork Sun Apr 21 18:28:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13637487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B111AC07E8E for ; Sun, 21 Apr 2024 18:29:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Pm4kWR65S9usQXln0+JOsBQcNqRqDdHvU4jh+aqlHuo=; b=2imN8wKgMOMbeP zz3tYNSf3UBkWgvM7otHBrAc53u7CHz5s3OIUhJhSy0vy4la/ZdhfP62HeswsuSONCW3UBH2S3hw0 o51zEzL0tgYEGSlgGHJlNbApCLp7VbBsb9uJxGcz3Wi9GtY8cefI7Y8sGCOl98YEtG7qSLI7pxeov PTIHl241hrLw8fxRGzRc1w77aXT7zpU6z8FHEdMJSeJFTzlxdn/jpZ9orjt1Zzynr3FGbPtEdge+E SWYbSskPVy675OwPehRiNRIVDEfiGuCjI1MP98TADX4CkYpVumLR1GBS5P4rrEG6qpecFA/Mcrs6K yvNdquQfqJ4yJxS8so0g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvv-0000000B4BN-1Tud; Sun, 21 Apr 2024 18:28:59 +0000 Received: from mail-lf1-x12a.google.com ([2a00:1450:4864:20::12a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvY-0000000B41R-325E for linux-arm-kernel@lists.infradead.org; Sun, 21 Apr 2024 18:28:44 +0000 Received: by mail-lf1-x12a.google.com with SMTP id 2adb3069b0e04-51ac5923ef6so1758375e87.0 for ; Sun, 21 Apr 2024 11:28:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1713724115; x=1714328915; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=0rmUoilvRHDOfrOC8IoOhQyzd0IjuLq/jUsxEzabH94=; b=fqR08LO5BjQG/5XonAbdxqyQBZ1Su8sDuHBKS+tjeDpem4sccMMlSd9mZqU0NnWSFo ZtKt+2bUEEBa2H26WBwejwXEmARm9UK8iUyMT5UxCFWmfymC2cKr4qRdazG6dYiU/SMz v8I6GDZFNm/Q0q+i6dpi1+hYxnizXwJbaLCa8pe1gKS1CYURFpCbvfxzGVkEwp+DDz5m EaBBgE01/W3Lt6HKF+LAli9O1Npg98ilg62tUaHLK3GJEDlXl3uRtmHlCNt0iX0aRmwJ Xw6RMJPjibqh+oxUO8/KWYakf3Yz+zPnFIv6DSedFjCrvpUHLjKhUS0VbrB0CSheSBTi 0Z6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713724115; x=1714328915; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0rmUoilvRHDOfrOC8IoOhQyzd0IjuLq/jUsxEzabH94=; b=RiXz3dP5liBdMJedeoE8aTnfa4gPtcomoZyRpITmYTobEs813mvXvh54tcpi50gy00 jaSm+znawdW/TrQHGsq7Ovp4Y832giHpGSeRn8f0Yx0wvCqKObxQ2n08e8GTUf1TxEFa y/0wWi4oCrdGWZn79wSlRtulEGYpHPAkuKzID4418kjRStgZ/09e02FAElc6NGlQW5sY gbnavoqQj1WMWFKr1+qYhPt7AVZgI2XeNuHF3ECgTc8CAcBO5Sys4BioSSx3tNAcYZoS TqrxxNMmMdeIA3xKIBhjUHOACjDOhPqsmZRDEGF+Ba1pWXncpo7IE5lTL9R+f7UzbsuP W5XQ== X-Gm-Message-State: AOJu0YwKFuNZQYAnnvWfb/sqDAPlqNcUGzknwfQkmuBRUAjely2+djbv 9OYS/5FiaaCRSd70e6nx8Vh75ob3pzn+0QAUVAQ3kVuQj/BulbXpV+R9uF1zDPE= X-Google-Smtp-Source: AGHT+IGhTUuBHruEyrHZHmUzWMkh/ZfNYErGvLKuETbF9KqYeaGjwRQtuYF8MHZBZsrHEF3KVNciDQ== X-Received: by 2002:a05:6512:31c9:b0:516:9f65:fcf with SMTP id j9-20020a05651231c900b005169f650fcfmr2875213lfe.26.1713724114823; Sun, 21 Apr 2024 11:28:34 -0700 (PDT) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id s14-20020a19770e000000b005188ea5d2b5sm1549641lfc.303.2024.04.21.11.28.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Apr 2024 11:28:34 -0700 (PDT) From: Linus Walleij Date: Sun, 21 Apr 2024 20:28:32 +0200 Subject: [PATCH v7 06/11] ARM: mm: Rewrite cacheflush vtables in CFI safe C MIME-Version: 1.0 Message-Id: <20240421-arm32-cfi-v7-6-6e132a948cc8@linaro.org> References: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> In-Reply-To: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> To: Russell King , Sami Tolvanen , Kees Cook , Nathan Chancellor , Nick Desaulniers , Ard Biesheuvel , Arnd Bergmann Cc: linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev, Linus Walleij X-Mailer: b4 0.13.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240421_112837_340113_533FF240 X-CRM114-Status: GOOD ( 22.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Instead of defining all cache flush operations with an assembly macro in proc-macros.S, provide an explicit struct cpu_cache_fns for each CPU cache type in mm/cache.c. As a side effect from rewriting the vtables in C, we can avoid the aliasing for the "louis" cache callback, instead we can just assign the NN_flush_kern_cache_all() function to the louis callback in the C vtable. As the louis cache callback is called explicitly (not through the vtable) if we only have one type of cache support compiled in, we need an ifdef quirk for this in the !MULTI_CACHE case. Feroceon and XScale have some dma mapping quirk, in this case we can just define two structs and assign all but one callback to the main implementation; since each of them invoked define_cache_functions twice they require MULTI_CACHE by definition so the compiled-in shortcut is not used on these variants. Tested-by: Kees Cook Signed-off-by: Linus Walleij --- arch/arm/include/asm/glue-cache.h | 28 +- arch/arm/mm/Makefile | 1 + arch/arm/mm/cache-b15-rac.c | 1 + arch/arm/mm/cache-fa.S | 8 - arch/arm/mm/cache-nop.S | 8 - arch/arm/mm/cache-v4.S | 8 - arch/arm/mm/cache-v4wb.S | 8 - arch/arm/mm/cache-v4wt.S | 8 - arch/arm/mm/cache-v6.S | 8 - arch/arm/mm/cache-v7.S | 25 -- arch/arm/mm/cache-v7m.S | 8 - arch/arm/mm/cache.c | 663 ++++++++++++++++++++++++++++++++++++++ arch/arm/mm/proc-arm1020.S | 6 - arch/arm/mm/proc-arm1020e.S | 6 - arch/arm/mm/proc-arm1022.S | 6 - arch/arm/mm/proc-arm1026.S | 6 - arch/arm/mm/proc-arm920.S | 5 - arch/arm/mm/proc-arm922.S | 6 - arch/arm/mm/proc-arm925.S | 6 - arch/arm/mm/proc-arm926.S | 6 - arch/arm/mm/proc-arm940.S | 6 - arch/arm/mm/proc-arm946.S | 6 - arch/arm/mm/proc-feroceon.S | 27 -- arch/arm/mm/proc-macros.S | 18 -- arch/arm/mm/proc-mohawk.S | 6 - arch/arm/mm/proc-xsc3.S | 6 - arch/arm/mm/proc-xscale.S | 57 +--- 27 files changed, 688 insertions(+), 259 deletions(-) diff --git a/arch/arm/include/asm/glue-cache.h b/arch/arm/include/asm/glue-cache.h index 724f8dac1e5b..4186fbf7341f 100644 --- a/arch/arm/include/asm/glue-cache.h +++ b/arch/arm/include/asm/glue-cache.h @@ -118,6 +118,10 @@ # define MULTI_CACHE 1 #endif +#ifdef CONFIG_CPU_CACHE_NOP +# define MULTI_CACHE 1 +#endif + #if defined(CONFIG_CPU_V7M) # define MULTI_CACHE 1 #endif @@ -126,29 +130,15 @@ #error Unknown cache maintenance model #endif -#ifndef __ASSEMBLER__ -static inline void nop_flush_icache_all(void) { } -static inline void nop_flush_kern_cache_all(void) { } -static inline void nop_flush_kern_cache_louis(void) { } -static inline void nop_flush_user_cache_all(void) { } -static inline void nop_flush_user_cache_range(unsigned long a, - unsigned long b, unsigned int c) { } - -static inline void nop_coherent_kern_range(unsigned long a, unsigned long b) { } -static inline int nop_coherent_user_range(unsigned long a, - unsigned long b) { return 0; } -static inline void nop_flush_kern_dcache_area(void *a, size_t s) { } - -static inline void nop_dma_flush_range(const void *a, const void *b) { } - -static inline void nop_dma_map_area(const void *s, size_t l, int f) { } -static inline void nop_dma_unmap_area(const void *s, size_t l, int f) { } -#endif - #ifndef MULTI_CACHE #define __cpuc_flush_icache_all __glue(_CACHE,_flush_icache_all) #define __cpuc_flush_kern_all __glue(_CACHE,_flush_kern_cache_all) +/* This function only has a dedicated assembly callback on the v7 cache */ +#ifdef CONFIG_CPU_CACHE_V7 #define __cpuc_flush_kern_louis __glue(_CACHE,_flush_kern_cache_louis) +#else +#define __cpuc_flush_kern_louis __glue(_CACHE,_flush_kern_cache_all) +#endif #define __cpuc_flush_user_all __glue(_CACHE,_flush_user_cache_all) #define __cpuc_flush_user_range __glue(_CACHE,_flush_user_cache_range) #define __cpuc_coherent_kern_range __glue(_CACHE,_coherent_kern_range) diff --git a/arch/arm/mm/Makefile b/arch/arm/mm/Makefile index cc8255fdf56e..17665381be96 100644 --- a/arch/arm/mm/Makefile +++ b/arch/arm/mm/Makefile @@ -45,6 +45,7 @@ obj-$(CONFIG_CPU_CACHE_V7) += cache-v7.o obj-$(CONFIG_CPU_CACHE_FA) += cache-fa.o obj-$(CONFIG_CPU_CACHE_NOP) += cache-nop.o obj-$(CONFIG_CPU_CACHE_V7M) += cache-v7m.o +obj-y += cache.o obj-$(CONFIG_CPU_COPY_V4WT) += copypage-v4wt.o obj-$(CONFIG_CPU_COPY_V4WB) += copypage-v4wb.o diff --git a/arch/arm/mm/cache-b15-rac.c b/arch/arm/mm/cache-b15-rac.c index 9c1172f26885..6f63b90f9e1a 100644 --- a/arch/arm/mm/cache-b15-rac.c +++ b/arch/arm/mm/cache-b15-rac.c @@ -5,6 +5,7 @@ * Copyright (C) 2015-2016 Broadcom */ +#include #include #include #include diff --git a/arch/arm/mm/cache-fa.S b/arch/arm/mm/cache-fa.S index 6fe06608f34e..4610105e058c 100644 --- a/arch/arm/mm/cache-fa.S +++ b/arch/arm/mm/cache-fa.S @@ -241,11 +241,3 @@ SYM_FUNC_END(fa_dma_map_area) SYM_TYPED_FUNC_START(fa_dma_unmap_area) ret lr SYM_FUNC_END(fa_dma_unmap_area) - - .globl fa_flush_kern_cache_louis - .equ fa_flush_kern_cache_louis, fa_flush_kern_cache_all - - __INITDATA - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions fa diff --git a/arch/arm/mm/cache-nop.S b/arch/arm/mm/cache-nop.S index cd191aa90313..f68dde2014ee 100644 --- a/arch/arm/mm/cache-nop.S +++ b/arch/arm/mm/cache-nop.S @@ -18,9 +18,6 @@ SYM_TYPED_FUNC_START(nop_flush_kern_cache_all) ret lr SYM_FUNC_END(nop_flush_kern_cache_all) - .globl nop_flush_kern_cache_louis - .equ nop_flush_kern_cache_louis, nop_flush_icache_all - SYM_TYPED_FUNC_START(nop_flush_user_cache_all) ret lr SYM_FUNC_END(nop_flush_user_cache_all) @@ -50,11 +47,6 @@ SYM_TYPED_FUNC_START(nop_dma_map_area) ret lr SYM_FUNC_END(nop_dma_map_area) - __INITDATA - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions nop - SYM_TYPED_FUNC_START(nop_dma_unmap_area) ret lr SYM_FUNC_END(nop_dma_unmap_area) diff --git a/arch/arm/mm/cache-v4.S b/arch/arm/mm/cache-v4.S index f7b7e498d3b6..0df97a610026 100644 --- a/arch/arm/mm/cache-v4.S +++ b/arch/arm/mm/cache-v4.S @@ -144,11 +144,3 @@ SYM_FUNC_END(v4_dma_unmap_area) SYM_TYPED_FUNC_START(v4_dma_map_area) ret lr SYM_FUNC_END(v4_dma_map_area) - - .globl v4_flush_kern_cache_louis - .equ v4_flush_kern_cache_louis, v4_flush_kern_cache_all - - __INITDATA - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions v4 diff --git a/arch/arm/mm/cache-v4wb.S b/arch/arm/mm/cache-v4wb.S index 19fae44b89cd..945a7881cc94 100644 --- a/arch/arm/mm/cache-v4wb.S +++ b/arch/arm/mm/cache-v4wb.S @@ -251,11 +251,3 @@ SYM_FUNC_END(v4wb_dma_map_area) SYM_TYPED_FUNC_START(v4wb_dma_unmap_area) ret lr SYM_FUNC_END(v4wb_dma_unmap_area) - - .globl v4wb_flush_kern_cache_louis - .equ v4wb_flush_kern_cache_louis, v4wb_flush_kern_cache_all - - __INITDATA - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions v4wb diff --git a/arch/arm/mm/cache-v4wt.S b/arch/arm/mm/cache-v4wt.S index 5be76ff861d7..d788962e2ed8 100644 --- a/arch/arm/mm/cache-v4wt.S +++ b/arch/arm/mm/cache-v4wt.S @@ -198,11 +198,3 @@ SYM_FUNC_END(v4wt_dma_unmap_area) SYM_TYPED_FUNC_START(v4wt_dma_map_area) ret lr SYM_FUNC_END(v4wt_dma_map_area) - - .globl v4wt_flush_kern_cache_louis - .equ v4wt_flush_kern_cache_louis, v4wt_flush_kern_cache_all - - __INITDATA - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions v4wt diff --git a/arch/arm/mm/cache-v6.S b/arch/arm/mm/cache-v6.S index a590044b7282..ebe96d40907a 100644 --- a/arch/arm/mm/cache-v6.S +++ b/arch/arm/mm/cache-v6.S @@ -296,11 +296,3 @@ SYM_TYPED_FUNC_START(v6_dma_unmap_area) bne v6_dma_inv_range ret lr SYM_FUNC_END(v6_dma_unmap_area) - - .globl v6_flush_kern_cache_louis - .equ v6_flush_kern_cache_louis, v6_flush_kern_cache_all - - __INITDATA - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions v6 diff --git a/arch/arm/mm/cache-v7.S b/arch/arm/mm/cache-v7.S index 6c0bc756d29a..c0ebe1aa0f02 100644 --- a/arch/arm/mm/cache-v7.S +++ b/arch/arm/mm/cache-v7.S @@ -454,28 +454,3 @@ SYM_TYPED_FUNC_START(v7_dma_unmap_area) bne v7_dma_inv_range ret lr SYM_FUNC_END(v7_dma_unmap_area) - - __INITDATA - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions v7 - - /* The Broadcom Brahma-B15 read-ahead cache requires some modifications - * to the v7_cache_fns, we only override the ones we need - */ -#ifndef CONFIG_CACHE_B15_RAC - globl_equ b15_flush_kern_cache_all, v7_flush_kern_cache_all -#endif - globl_equ b15_flush_icache_all, v7_flush_icache_all - globl_equ b15_flush_kern_cache_louis, v7_flush_kern_cache_louis - globl_equ b15_flush_user_cache_all, v7_flush_user_cache_all - globl_equ b15_flush_user_cache_range, v7_flush_user_cache_range - globl_equ b15_coherent_kern_range, v7_coherent_kern_range - globl_equ b15_coherent_user_range, v7_coherent_user_range - globl_equ b15_flush_kern_dcache_area, v7_flush_kern_dcache_area - - globl_equ b15_dma_map_area, v7_dma_map_area - globl_equ b15_dma_unmap_area, v7_dma_unmap_area - globl_equ b15_dma_flush_range, v7_dma_flush_range - - define_cache_functions b15 diff --git a/arch/arm/mm/cache-v7m.S b/arch/arm/mm/cache-v7m.S index 5a62b9a224e1..4e670697eabc 100644 --- a/arch/arm/mm/cache-v7m.S +++ b/arch/arm/mm/cache-v7m.S @@ -447,11 +447,3 @@ SYM_TYPED_FUNC_START(v7m_dma_unmap_area) bne v7m_dma_inv_range ret lr SYM_FUNC_END(v7m_dma_unmap_area) - - .globl v7m_flush_kern_cache_louis - .equ v7m_flush_kern_cache_louis, v7m_flush_kern_cache_all - - __INITDATA - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions v7m diff --git a/arch/arm/mm/cache.c b/arch/arm/mm/cache.c new file mode 100644 index 000000000000..e6fbc599c9ed --- /dev/null +++ b/arch/arm/mm/cache.c @@ -0,0 +1,663 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * This file defines C prototypes for the low-level cache assembly functions + * and populates a vtable for each selected ARM CPU cache type. + */ + +#include +#include + +#ifdef CONFIG_CPU_CACHE_V4 +void v4_flush_icache_all(void); +void v4_flush_kern_cache_all(void); +void v4_flush_user_cache_all(void); +void v4_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void v4_coherent_kern_range(unsigned long, unsigned long); +int v4_coherent_user_range(unsigned long, unsigned long); +void v4_flush_kern_dcache_area(void *, size_t); +void v4_dma_map_area(const void *, size_t, int); +void v4_dma_unmap_area(const void *, size_t, int); +void v4_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns v4_cache_fns __initconst = { + .flush_icache_all = v4_flush_icache_all, + .flush_kern_all = v4_flush_kern_cache_all, + .flush_kern_louis = v4_flush_kern_cache_all, + .flush_user_all = v4_flush_user_cache_all, + .flush_user_range = v4_flush_user_cache_range, + .coherent_kern_range = v4_coherent_kern_range, + .coherent_user_range = v4_coherent_user_range, + .flush_kern_dcache_area = v4_flush_kern_dcache_area, + .dma_map_area = v4_dma_map_area, + .dma_unmap_area = v4_dma_unmap_area, + .dma_flush_range = v4_dma_flush_range, +}; +#endif + +/* V4 write-back cache "V4WB" */ +#ifdef CONFIG_CPU_CACHE_V4WB +void v4wb_flush_icache_all(void); +void v4wb_flush_kern_cache_all(void); +void v4wb_flush_user_cache_all(void); +void v4wb_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void v4wb_coherent_kern_range(unsigned long, unsigned long); +int v4wb_coherent_user_range(unsigned long, unsigned long); +void v4wb_flush_kern_dcache_area(void *, size_t); +void v4wb_dma_map_area(const void *, size_t, int); +void v4wb_dma_unmap_area(const void *, size_t, int); +void v4wb_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns v4wb_cache_fns __initconst = { + .flush_icache_all = v4wb_flush_icache_all, + .flush_kern_all = v4wb_flush_kern_cache_all, + .flush_kern_louis = v4wb_flush_kern_cache_all, + .flush_user_all = v4wb_flush_user_cache_all, + .flush_user_range = v4wb_flush_user_cache_range, + .coherent_kern_range = v4wb_coherent_kern_range, + .coherent_user_range = v4wb_coherent_user_range, + .flush_kern_dcache_area = v4wb_flush_kern_dcache_area, + .dma_map_area = v4wb_dma_map_area, + .dma_unmap_area = v4wb_dma_unmap_area, + .dma_flush_range = v4wb_dma_flush_range, +}; +#endif + +/* V4 write-through cache "V4WT" */ +#ifdef CONFIG_CPU_CACHE_V4WT +void v4wt_flush_icache_all(void); +void v4wt_flush_kern_cache_all(void); +void v4wt_flush_user_cache_all(void); +void v4wt_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void v4wt_coherent_kern_range(unsigned long, unsigned long); +int v4wt_coherent_user_range(unsigned long, unsigned long); +void v4wt_flush_kern_dcache_area(void *, size_t); +void v4wt_dma_map_area(const void *, size_t, int); +void v4wt_dma_unmap_area(const void *, size_t, int); +void v4wt_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns v4wt_cache_fns __initconst = { + .flush_icache_all = v4wt_flush_icache_all, + .flush_kern_all = v4wt_flush_kern_cache_all, + .flush_kern_louis = v4wt_flush_kern_cache_all, + .flush_user_all = v4wt_flush_user_cache_all, + .flush_user_range = v4wt_flush_user_cache_range, + .coherent_kern_range = v4wt_coherent_kern_range, + .coherent_user_range = v4wt_coherent_user_range, + .flush_kern_dcache_area = v4wt_flush_kern_dcache_area, + .dma_map_area = v4wt_dma_map_area, + .dma_unmap_area = v4wt_dma_unmap_area, + .dma_flush_range = v4wt_dma_flush_range, +}; +#endif + +/* Faraday FA526 cache */ +#ifdef CONFIG_CPU_CACHE_FA +void fa_flush_icache_all(void); +void fa_flush_kern_cache_all(void); +void fa_flush_user_cache_all(void); +void fa_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void fa_coherent_kern_range(unsigned long, unsigned long); +int fa_coherent_user_range(unsigned long, unsigned long); +void fa_flush_kern_dcache_area(void *, size_t); +void fa_dma_map_area(const void *, size_t, int); +void fa_dma_unmap_area(const void *, size_t, int); +void fa_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns fa_cache_fns __initconst = { + .flush_icache_all = fa_flush_icache_all, + .flush_kern_all = fa_flush_kern_cache_all, + .flush_kern_louis = fa_flush_kern_cache_all, + .flush_user_all = fa_flush_user_cache_all, + .flush_user_range = fa_flush_user_cache_range, + .coherent_kern_range = fa_coherent_kern_range, + .coherent_user_range = fa_coherent_user_range, + .flush_kern_dcache_area = fa_flush_kern_dcache_area, + .dma_map_area = fa_dma_map_area, + .dma_unmap_area = fa_dma_unmap_area, + .dma_flush_range = fa_dma_flush_range, +}; +#endif + +#ifdef CONFIG_CPU_CACHE_V6 +void v6_flush_icache_all(void); +void v6_flush_kern_cache_all(void); +void v6_flush_user_cache_all(void); +void v6_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void v6_coherent_kern_range(unsigned long, unsigned long); +int v6_coherent_user_range(unsigned long, unsigned long); +void v6_flush_kern_dcache_area(void *, size_t); +void v6_dma_map_area(const void *, size_t, int); +void v6_dma_unmap_area(const void *, size_t, int); +void v6_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns v6_cache_fns __initconst = { + .flush_icache_all = v6_flush_icache_all, + .flush_kern_all = v6_flush_kern_cache_all, + .flush_kern_louis = v6_flush_kern_cache_all, + .flush_user_all = v6_flush_user_cache_all, + .flush_user_range = v6_flush_user_cache_range, + .coherent_kern_range = v6_coherent_kern_range, + .coherent_user_range = v6_coherent_user_range, + .flush_kern_dcache_area = v6_flush_kern_dcache_area, + .dma_map_area = v6_dma_map_area, + .dma_unmap_area = v6_dma_unmap_area, + .dma_flush_range = v6_dma_flush_range, +}; +#endif + +#ifdef CONFIG_CPU_CACHE_V7 +void v7_flush_icache_all(void); +void v7_flush_kern_cache_all(void); +void v7_flush_kern_cache_louis(void); +void v7_flush_user_cache_all(void); +void v7_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void v7_coherent_kern_range(unsigned long, unsigned long); +int v7_coherent_user_range(unsigned long, unsigned long); +void v7_flush_kern_dcache_area(void *, size_t); +void v7_dma_map_area(const void *, size_t, int); +void v7_dma_unmap_area(const void *, size_t, int); +void v7_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns v7_cache_fns __initconst = { + .flush_icache_all = v7_flush_icache_all, + .flush_kern_all = v7_flush_kern_cache_all, + .flush_kern_louis = v7_flush_kern_cache_louis, + .flush_user_all = v7_flush_user_cache_all, + .flush_user_range = v7_flush_user_cache_range, + .coherent_kern_range = v7_coherent_kern_range, + .coherent_user_range = v7_coherent_user_range, + .flush_kern_dcache_area = v7_flush_kern_dcache_area, + .dma_map_area = v7_dma_map_area, + .dma_unmap_area = v7_dma_unmap_area, + .dma_flush_range = v7_dma_flush_range, +}; + +/* Special quirky cache flush function for Broadcom B15 v7 caches */ +void b15_flush_kern_cache_all(void); + +struct cpu_cache_fns b15_cache_fns __initconst = { + .flush_icache_all = v7_flush_icache_all, +#ifdef CONFIG_CACHE_B15_RAC + .flush_kern_all = b15_flush_kern_cache_all, +#else + .flush_kern_all = v7_flush_kern_cache_all, +#endif + .flush_kern_louis = v7_flush_kern_cache_louis, + .flush_user_all = v7_flush_user_cache_all, + .flush_user_range = v7_flush_user_cache_range, + .coherent_kern_range = v7_coherent_kern_range, + .coherent_user_range = v7_coherent_user_range, + .flush_kern_dcache_area = v7_flush_kern_dcache_area, + .dma_map_area = v7_dma_map_area, + .dma_unmap_area = v7_dma_unmap_area, + .dma_flush_range = v7_dma_flush_range, +}; +#endif + +/* The NOP cache is just a set of dummy stubs that by definition does nothing */ +#ifdef CONFIG_CPU_CACHE_NOP +void nop_flush_icache_all(void); +void nop_flush_kern_cache_all(void); +void nop_flush_user_cache_all(void); +void nop_flush_user_cache_range(unsigned long start, unsigned long end, unsigned int flags); +void nop_coherent_kern_range(unsigned long start, unsigned long end); +int nop_coherent_user_range(unsigned long, unsigned long); +void nop_flush_kern_dcache_area(void *kaddr, size_t size); +void nop_dma_map_area(const void *start, size_t size, int flags); +void nop_dma_unmap_area(const void *start, size_t size, int flags); +void nop_dma_flush_range(const void *start, const void *end); + +struct cpu_cache_fns nop_cache_fns __initconst = { + .flush_icache_all = nop_flush_icache_all, + .flush_kern_all = nop_flush_kern_cache_all, + .flush_kern_louis = nop_flush_kern_cache_all, + .flush_user_all = nop_flush_user_cache_all, + .flush_user_range = nop_flush_user_cache_range, + .coherent_kern_range = nop_coherent_kern_range, + .coherent_user_range = nop_coherent_user_range, + .flush_kern_dcache_area = nop_flush_kern_dcache_area, + .dma_map_area = nop_dma_map_area, + .dma_unmap_area = nop_dma_unmap_area, + .dma_flush_range = nop_dma_flush_range, +}; +#endif + +#ifdef CONFIG_CPU_CACHE_V7M +void v7m_flush_icache_all(void); +void v7m_flush_kern_cache_all(void); +void v7m_flush_user_cache_all(void); +void v7m_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void v7m_coherent_kern_range(unsigned long, unsigned long); +int v7m_coherent_user_range(unsigned long, unsigned long); +void v7m_flush_kern_dcache_area(void *, size_t); +void v7m_dma_map_area(const void *, size_t, int); +void v7m_dma_unmap_area(const void *, size_t, int); +void v7m_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns v7m_cache_fns __initconst = { + .flush_icache_all = v7m_flush_icache_all, + .flush_kern_all = v7m_flush_kern_cache_all, + .flush_kern_louis = v7m_flush_kern_cache_all, + .flush_user_all = v7m_flush_user_cache_all, + .flush_user_range = v7m_flush_user_cache_range, + .coherent_kern_range = v7m_coherent_kern_range, + .coherent_user_range = v7m_coherent_user_range, + .flush_kern_dcache_area = v7m_flush_kern_dcache_area, + .dma_map_area = v7m_dma_map_area, + .dma_unmap_area = v7m_dma_unmap_area, + .dma_flush_range = v7m_dma_flush_range, +}; +#endif + +#ifdef CONFIG_CPU_ARM1020 +void arm1020_flush_icache_all(void); +void arm1020_flush_kern_cache_all(void); +void arm1020_flush_user_cache_all(void); +void arm1020_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void arm1020_coherent_kern_range(unsigned long, unsigned long); +int arm1020_coherent_user_range(unsigned long, unsigned long); +void arm1020_flush_kern_dcache_area(void *, size_t); +void arm1020_dma_map_area(const void *, size_t, int); +void arm1020_dma_unmap_area(const void *, size_t, int); +void arm1020_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns arm1020_cache_fns __initconst = { + .flush_icache_all = arm1020_flush_icache_all, + .flush_kern_all = arm1020_flush_kern_cache_all, + .flush_kern_louis = arm1020_flush_kern_cache_all, + .flush_user_all = arm1020_flush_user_cache_all, + .flush_user_range = arm1020_flush_user_cache_range, + .coherent_kern_range = arm1020_coherent_kern_range, + .coherent_user_range = arm1020_coherent_user_range, + .flush_kern_dcache_area = arm1020_flush_kern_dcache_area, + .dma_map_area = arm1020_dma_map_area, + .dma_unmap_area = arm1020_dma_unmap_area, + .dma_flush_range = arm1020_dma_flush_range, +}; +#endif + +#ifdef CONFIG_CPU_ARM1020E +void arm1020e_flush_icache_all(void); +void arm1020e_flush_kern_cache_all(void); +void arm1020e_flush_user_cache_all(void); +void arm1020e_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void arm1020e_coherent_kern_range(unsigned long, unsigned long); +int arm1020e_coherent_user_range(unsigned long, unsigned long); +void arm1020e_flush_kern_dcache_area(void *, size_t); +void arm1020e_dma_map_area(const void *, size_t, int); +void arm1020e_dma_unmap_area(const void *, size_t, int); +void arm1020e_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns arm1020e_cache_fns __initconst = { + .flush_icache_all = arm1020e_flush_icache_all, + .flush_kern_all = arm1020e_flush_kern_cache_all, + .flush_kern_louis = arm1020e_flush_kern_cache_all, + .flush_user_all = arm1020e_flush_user_cache_all, + .flush_user_range = arm1020e_flush_user_cache_range, + .coherent_kern_range = arm1020e_coherent_kern_range, + .coherent_user_range = arm1020e_coherent_user_range, + .flush_kern_dcache_area = arm1020e_flush_kern_dcache_area, + .dma_map_area = arm1020e_dma_map_area, + .dma_unmap_area = arm1020e_dma_unmap_area, + .dma_flush_range = arm1020e_dma_flush_range, +}; +#endif + +#ifdef CONFIG_CPU_ARM1022 +void arm1022_flush_icache_all(void); +void arm1022_flush_kern_cache_all(void); +void arm1022_flush_user_cache_all(void); +void arm1022_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void arm1022_coherent_kern_range(unsigned long, unsigned long); +int arm1022_coherent_user_range(unsigned long, unsigned long); +void arm1022_flush_kern_dcache_area(void *, size_t); +void arm1022_dma_map_area(const void *, size_t, int); +void arm1022_dma_unmap_area(const void *, size_t, int); +void arm1022_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns arm1022_cache_fns __initconst = { + .flush_icache_all = arm1022_flush_icache_all, + .flush_kern_all = arm1022_flush_kern_cache_all, + .flush_kern_louis = arm1022_flush_kern_cache_all, + .flush_user_all = arm1022_flush_user_cache_all, + .flush_user_range = arm1022_flush_user_cache_range, + .coherent_kern_range = arm1022_coherent_kern_range, + .coherent_user_range = arm1022_coherent_user_range, + .flush_kern_dcache_area = arm1022_flush_kern_dcache_area, + .dma_map_area = arm1022_dma_map_area, + .dma_unmap_area = arm1022_dma_unmap_area, + .dma_flush_range = arm1022_dma_flush_range, +}; +#endif + +#ifdef CONFIG_CPU_ARM1026 +void arm1026_flush_icache_all(void); +void arm1026_flush_kern_cache_all(void); +void arm1026_flush_user_cache_all(void); +void arm1026_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void arm1026_coherent_kern_range(unsigned long, unsigned long); +int arm1026_coherent_user_range(unsigned long, unsigned long); +void arm1026_flush_kern_dcache_area(void *, size_t); +void arm1026_dma_map_area(const void *, size_t, int); +void arm1026_dma_unmap_area(const void *, size_t, int); +void arm1026_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns arm1026_cache_fns __initconst = { + .flush_icache_all = arm1026_flush_icache_all, + .flush_kern_all = arm1026_flush_kern_cache_all, + .flush_kern_louis = arm1026_flush_kern_cache_all, + .flush_user_all = arm1026_flush_user_cache_all, + .flush_user_range = arm1026_flush_user_cache_range, + .coherent_kern_range = arm1026_coherent_kern_range, + .coherent_user_range = arm1026_coherent_user_range, + .flush_kern_dcache_area = arm1026_flush_kern_dcache_area, + .dma_map_area = arm1026_dma_map_area, + .dma_unmap_area = arm1026_dma_unmap_area, + .dma_flush_range = arm1026_dma_flush_range, +}; +#endif + +#if defined(CONFIG_CPU_ARM920T) && !defined(CONFIG_CPU_DCACHE_WRITETHROUGH) +void arm920_flush_icache_all(void); +void arm920_flush_kern_cache_all(void); +void arm920_flush_user_cache_all(void); +void arm920_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void arm920_coherent_kern_range(unsigned long, unsigned long); +int arm920_coherent_user_range(unsigned long, unsigned long); +void arm920_flush_kern_dcache_area(void *, size_t); +void arm920_dma_map_area(const void *, size_t, int); +void arm920_dma_unmap_area(const void *, size_t, int); +void arm920_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns arm920_cache_fns __initconst = { + .flush_icache_all = arm920_flush_icache_all, + .flush_kern_all = arm920_flush_kern_cache_all, + .flush_kern_louis = arm920_flush_kern_cache_all, + .flush_user_all = arm920_flush_user_cache_all, + .flush_user_range = arm920_flush_user_cache_range, + .coherent_kern_range = arm920_coherent_kern_range, + .coherent_user_range = arm920_coherent_user_range, + .flush_kern_dcache_area = arm920_flush_kern_dcache_area, + .dma_map_area = arm920_dma_map_area, + .dma_unmap_area = arm920_dma_unmap_area, + .dma_flush_range = arm920_dma_flush_range, +}; +#endif + +#if defined(CONFIG_CPU_ARM922T) && !defined(CONFIG_CPU_DCACHE_WRITETHROUGH) +void arm922_flush_icache_all(void); +void arm922_flush_kern_cache_all(void); +void arm922_flush_user_cache_all(void); +void arm922_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void arm922_coherent_kern_range(unsigned long, unsigned long); +int arm922_coherent_user_range(unsigned long, unsigned long); +void arm922_flush_kern_dcache_area(void *, size_t); +void arm922_dma_map_area(const void *, size_t, int); +void arm922_dma_unmap_area(const void *, size_t, int); +void arm922_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns arm922_cache_fns __initconst = { + .flush_icache_all = arm922_flush_icache_all, + .flush_kern_all = arm922_flush_kern_cache_all, + .flush_kern_louis = arm922_flush_kern_cache_all, + .flush_user_all = arm922_flush_user_cache_all, + .flush_user_range = arm922_flush_user_cache_range, + .coherent_kern_range = arm922_coherent_kern_range, + .coherent_user_range = arm922_coherent_user_range, + .flush_kern_dcache_area = arm922_flush_kern_dcache_area, + .dma_map_area = arm922_dma_map_area, + .dma_unmap_area = arm922_dma_unmap_area, + .dma_flush_range = arm922_dma_flush_range, +}; +#endif + +#ifdef CONFIG_CPU_ARM925T +void arm925_flush_icache_all(void); +void arm925_flush_kern_cache_all(void); +void arm925_flush_user_cache_all(void); +void arm925_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void arm925_coherent_kern_range(unsigned long, unsigned long); +int arm925_coherent_user_range(unsigned long, unsigned long); +void arm925_flush_kern_dcache_area(void *, size_t); +void arm925_dma_map_area(const void *, size_t, int); +void arm925_dma_unmap_area(const void *, size_t, int); +void arm925_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns arm925_cache_fns __initconst = { + .flush_icache_all = arm925_flush_icache_all, + .flush_kern_all = arm925_flush_kern_cache_all, + .flush_kern_louis = arm925_flush_kern_cache_all, + .flush_user_all = arm925_flush_user_cache_all, + .flush_user_range = arm925_flush_user_cache_range, + .coherent_kern_range = arm925_coherent_kern_range, + .coherent_user_range = arm925_coherent_user_range, + .flush_kern_dcache_area = arm925_flush_kern_dcache_area, + .dma_map_area = arm925_dma_map_area, + .dma_unmap_area = arm925_dma_unmap_area, + .dma_flush_range = arm925_dma_flush_range, +}; +#endif + +#ifdef CONFIG_CPU_ARM926T +void arm926_flush_icache_all(void); +void arm926_flush_kern_cache_all(void); +void arm926_flush_user_cache_all(void); +void arm926_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void arm926_coherent_kern_range(unsigned long, unsigned long); +int arm926_coherent_user_range(unsigned long, unsigned long); +void arm926_flush_kern_dcache_area(void *, size_t); +void arm926_dma_map_area(const void *, size_t, int); +void arm926_dma_unmap_area(const void *, size_t, int); +void arm926_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns arm926_cache_fns __initconst = { + .flush_icache_all = arm926_flush_icache_all, + .flush_kern_all = arm926_flush_kern_cache_all, + .flush_kern_louis = arm926_flush_kern_cache_all, + .flush_user_all = arm926_flush_user_cache_all, + .flush_user_range = arm926_flush_user_cache_range, + .coherent_kern_range = arm926_coherent_kern_range, + .coherent_user_range = arm926_coherent_user_range, + .flush_kern_dcache_area = arm926_flush_kern_dcache_area, + .dma_map_area = arm926_dma_map_area, + .dma_unmap_area = arm926_dma_unmap_area, + .dma_flush_range = arm926_dma_flush_range, +}; +#endif + +#ifdef CONFIG_CPU_ARM940T +void arm940_flush_icache_all(void); +void arm940_flush_kern_cache_all(void); +void arm940_flush_user_cache_all(void); +void arm940_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void arm940_coherent_kern_range(unsigned long, unsigned long); +int arm940_coherent_user_range(unsigned long, unsigned long); +void arm940_flush_kern_dcache_area(void *, size_t); +void arm940_dma_map_area(const void *, size_t, int); +void arm940_dma_unmap_area(const void *, size_t, int); +void arm940_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns arm940_cache_fns __initconst = { + .flush_icache_all = arm940_flush_icache_all, + .flush_kern_all = arm940_flush_kern_cache_all, + .flush_kern_louis = arm940_flush_kern_cache_all, + .flush_user_all = arm940_flush_user_cache_all, + .flush_user_range = arm940_flush_user_cache_range, + .coherent_kern_range = arm940_coherent_kern_range, + .coherent_user_range = arm940_coherent_user_range, + .flush_kern_dcache_area = arm940_flush_kern_dcache_area, + .dma_map_area = arm940_dma_map_area, + .dma_unmap_area = arm940_dma_unmap_area, + .dma_flush_range = arm940_dma_flush_range, +}; +#endif + +#ifdef CONFIG_CPU_ARM946E +void arm946_flush_icache_all(void); +void arm946_flush_kern_cache_all(void); +void arm946_flush_user_cache_all(void); +void arm946_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void arm946_coherent_kern_range(unsigned long, unsigned long); +int arm946_coherent_user_range(unsigned long, unsigned long); +void arm946_flush_kern_dcache_area(void *, size_t); +void arm946_dma_map_area(const void *, size_t, int); +void arm946_dma_unmap_area(const void *, size_t, int); +void arm946_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns arm946_cache_fns __initconst = { + .flush_icache_all = arm946_flush_icache_all, + .flush_kern_all = arm946_flush_kern_cache_all, + .flush_kern_louis = arm946_flush_kern_cache_all, + .flush_user_all = arm946_flush_user_cache_all, + .flush_user_range = arm946_flush_user_cache_range, + .coherent_kern_range = arm946_coherent_kern_range, + .coherent_user_range = arm946_coherent_user_range, + .flush_kern_dcache_area = arm946_flush_kern_dcache_area, + .dma_map_area = arm946_dma_map_area, + .dma_unmap_area = arm946_dma_unmap_area, + .dma_flush_range = arm946_dma_flush_range, +}; +#endif + +#ifdef CONFIG_CPU_XSCALE +void xscale_flush_icache_all(void); +void xscale_flush_kern_cache_all(void); +void xscale_flush_user_cache_all(void); +void xscale_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void xscale_coherent_kern_range(unsigned long, unsigned long); +int xscale_coherent_user_range(unsigned long, unsigned long); +void xscale_flush_kern_dcache_area(void *, size_t); +void xscale_dma_map_area(const void *, size_t, int); +void xscale_dma_unmap_area(const void *, size_t, int); +void xscale_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns xscale_cache_fns __initconst = { + .flush_icache_all = xscale_flush_icache_all, + .flush_kern_all = xscale_flush_kern_cache_all, + .flush_kern_louis = xscale_flush_kern_cache_all, + .flush_user_all = xscale_flush_user_cache_all, + .flush_user_range = xscale_flush_user_cache_range, + .coherent_kern_range = xscale_coherent_kern_range, + .coherent_user_range = xscale_coherent_user_range, + .flush_kern_dcache_area = xscale_flush_kern_dcache_area, + .dma_map_area = xscale_dma_map_area, + .dma_unmap_area = xscale_dma_unmap_area, + .dma_flush_range = xscale_dma_flush_range, +}; + +/* The 80200 A0 and A1 need a special quirk for dma_map_area() */ +void xscale_80200_A0_A1_dma_map_area(const void *, size_t, int); + +struct cpu_cache_fns xscale_80200_A0_A1_cache_fns __initconst = { + .flush_icache_all = xscale_flush_icache_all, + .flush_kern_all = xscale_flush_kern_cache_all, + .flush_kern_louis = xscale_flush_kern_cache_all, + .flush_user_all = xscale_flush_user_cache_all, + .flush_user_range = xscale_flush_user_cache_range, + .coherent_kern_range = xscale_coherent_kern_range, + .coherent_user_range = xscale_coherent_user_range, + .flush_kern_dcache_area = xscale_flush_kern_dcache_area, + .dma_map_area = xscale_80200_A0_A1_dma_map_area, + .dma_unmap_area = xscale_dma_unmap_area, + .dma_flush_range = xscale_dma_flush_range, +}; +#endif + +#ifdef CONFIG_CPU_XSC3 +void xsc3_flush_icache_all(void); +void xsc3_flush_kern_cache_all(void); +void xsc3_flush_user_cache_all(void); +void xsc3_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void xsc3_coherent_kern_range(unsigned long, unsigned long); +int xsc3_coherent_user_range(unsigned long, unsigned long); +void xsc3_flush_kern_dcache_area(void *, size_t); +void xsc3_dma_map_area(const void *, size_t, int); +void xsc3_dma_unmap_area(const void *, size_t, int); +void xsc3_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns xsc3_cache_fns __initconst = { + .flush_icache_all = xsc3_flush_icache_all, + .flush_kern_all = xsc3_flush_kern_cache_all, + .flush_kern_louis = xsc3_flush_kern_cache_all, + .flush_user_all = xsc3_flush_user_cache_all, + .flush_user_range = xsc3_flush_user_cache_range, + .coherent_kern_range = xsc3_coherent_kern_range, + .coherent_user_range = xsc3_coherent_user_range, + .flush_kern_dcache_area = xsc3_flush_kern_dcache_area, + .dma_map_area = xsc3_dma_map_area, + .dma_unmap_area = xsc3_dma_unmap_area, + .dma_flush_range = xsc3_dma_flush_range, +}; +#endif + +#ifdef CONFIG_CPU_MOHAWK +void mohawk_flush_icache_all(void); +void mohawk_flush_kern_cache_all(void); +void mohawk_flush_user_cache_all(void); +void mohawk_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void mohawk_coherent_kern_range(unsigned long, unsigned long); +int mohawk_coherent_user_range(unsigned long, unsigned long); +void mohawk_flush_kern_dcache_area(void *, size_t); +void mohawk_dma_map_area(const void *, size_t, int); +void mohawk_dma_unmap_area(const void *, size_t, int); +void mohawk_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns mohawk_cache_fns __initconst = { + .flush_icache_all = mohawk_flush_icache_all, + .flush_kern_all = mohawk_flush_kern_cache_all, + .flush_kern_louis = mohawk_flush_kern_cache_all, + .flush_user_all = mohawk_flush_user_cache_all, + .flush_user_range = mohawk_flush_user_cache_range, + .coherent_kern_range = mohawk_coherent_kern_range, + .coherent_user_range = mohawk_coherent_user_range, + .flush_kern_dcache_area = mohawk_flush_kern_dcache_area, + .dma_map_area = mohawk_dma_map_area, + .dma_unmap_area = mohawk_dma_unmap_area, + .dma_flush_range = mohawk_dma_flush_range, +}; +#endif + +#ifdef CONFIG_CPU_FEROCEON +void feroceon_flush_icache_all(void); +void feroceon_flush_kern_cache_all(void); +void feroceon_flush_user_cache_all(void); +void feroceon_flush_user_cache_range(unsigned long, unsigned long, unsigned int); +void feroceon_coherent_kern_range(unsigned long, unsigned long); +int feroceon_coherent_user_range(unsigned long, unsigned long); +void feroceon_flush_kern_dcache_area(void *, size_t); +void feroceon_dma_map_area(const void *, size_t, int); +void feroceon_dma_unmap_area(const void *, size_t, int); +void feroceon_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns feroceon_cache_fns __initconst = { + .flush_icache_all = feroceon_flush_icache_all, + .flush_kern_all = feroceon_flush_kern_cache_all, + .flush_kern_louis = feroceon_flush_kern_cache_all, + .flush_user_all = feroceon_flush_user_cache_all, + .flush_user_range = feroceon_flush_user_cache_range, + .coherent_kern_range = feroceon_coherent_kern_range, + .coherent_user_range = feroceon_coherent_user_range, + .flush_kern_dcache_area = feroceon_flush_kern_dcache_area, + .dma_map_area = feroceon_dma_map_area, + .dma_unmap_area = feroceon_dma_unmap_area, + .dma_flush_range = feroceon_dma_flush_range, +}; + +void feroceon_range_flush_kern_dcache_area(void *, size_t); +void feroceon_range_dma_map_area(const void *, size_t, int); +void feroceon_range_dma_flush_range(const void *, const void *); + +struct cpu_cache_fns feroceon_range_cache_fns __initconst = { + .flush_icache_all = feroceon_flush_icache_all, + .flush_kern_all = feroceon_flush_kern_cache_all, + .flush_kern_louis = feroceon_flush_kern_cache_all, + .flush_user_all = feroceon_flush_user_cache_all, + .flush_user_range = feroceon_flush_user_cache_range, + .coherent_kern_range = feroceon_coherent_kern_range, + .coherent_user_range = feroceon_coherent_user_range, + .flush_kern_dcache_area = feroceon_range_flush_kern_dcache_area, + .dma_map_area = feroceon_range_dma_map_area, + .dma_unmap_area = feroceon_dma_unmap_area, + .dma_flush_range = feroceon_range_dma_flush_range, +}; +#endif diff --git a/arch/arm/mm/proc-arm1020.S b/arch/arm/mm/proc-arm1020.S index 379628e8ef4e..1e014cc5b4d1 100644 --- a/arch/arm/mm/proc-arm1020.S +++ b/arch/arm/mm/proc-arm1020.S @@ -357,12 +357,6 @@ SYM_TYPED_FUNC_START(arm1020_dma_unmap_area) ret lr SYM_FUNC_END(arm1020_dma_unmap_area) - .globl arm1020_flush_kern_cache_louis - .equ arm1020_flush_kern_cache_louis, arm1020_flush_kern_cache_all - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions arm1020 - .align 5 ENTRY(cpu_arm1020_dcache_clean_area) #ifndef CONFIG_CPU_DCACHE_DISABLE diff --git a/arch/arm/mm/proc-arm1020e.S b/arch/arm/mm/proc-arm1020e.S index b5846fbea040..7d80761f207a 100644 --- a/arch/arm/mm/proc-arm1020e.S +++ b/arch/arm/mm/proc-arm1020e.S @@ -344,12 +344,6 @@ SYM_TYPED_FUNC_START(arm1020e_dma_unmap_area) ret lr SYM_FUNC_END(arm1020e_dma_unmap_area) - .globl arm1020e_flush_kern_cache_louis - .equ arm1020e_flush_kern_cache_louis, arm1020e_flush_kern_cache_all - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions arm1020e - .align 5 ENTRY(cpu_arm1020e_dcache_clean_area) #ifndef CONFIG_CPU_DCACHE_DISABLE diff --git a/arch/arm/mm/proc-arm1022.S b/arch/arm/mm/proc-arm1022.S index c40b268cc274..53b1541c50d8 100644 --- a/arch/arm/mm/proc-arm1022.S +++ b/arch/arm/mm/proc-arm1022.S @@ -343,12 +343,6 @@ SYM_TYPED_FUNC_START(arm1022_dma_unmap_area) ret lr SYM_FUNC_END(arm1022_dma_unmap_area) - .globl arm1022_flush_kern_cache_louis - .equ arm1022_flush_kern_cache_louis, arm1022_flush_kern_cache_all - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions arm1022 - .align 5 ENTRY(cpu_arm1022_dcache_clean_area) #ifndef CONFIG_CPU_DCACHE_DISABLE diff --git a/arch/arm/mm/proc-arm1026.S b/arch/arm/mm/proc-arm1026.S index 7ef2c6d88dc0..6c6ea0357a77 100644 --- a/arch/arm/mm/proc-arm1026.S +++ b/arch/arm/mm/proc-arm1026.S @@ -338,12 +338,6 @@ SYM_TYPED_FUNC_START(arm1026_dma_unmap_area) ret lr SYM_FUNC_END(arm1026_dma_unmap_area) - .globl arm1026_flush_kern_cache_louis - .equ arm1026_flush_kern_cache_louis, arm1026_flush_kern_cache_all - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions arm1026 - .align 5 ENTRY(cpu_arm1026_dcache_clean_area) #ifndef CONFIG_CPU_DCACHE_DISABLE diff --git a/arch/arm/mm/proc-arm920.S b/arch/arm/mm/proc-arm920.S index eb89a322a534..08a5bac0d89d 100644 --- a/arch/arm/mm/proc-arm920.S +++ b/arch/arm/mm/proc-arm920.S @@ -309,11 +309,6 @@ SYM_TYPED_FUNC_START(arm920_dma_unmap_area) ret lr SYM_FUNC_END(arm920_dma_unmap_area) - .globl arm920_flush_kern_cache_louis - .equ arm920_flush_kern_cache_louis, arm920_flush_kern_cache_all - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions arm920 #endif /* !CONFIG_CPU_DCACHE_WRITETHROUGH */ diff --git a/arch/arm/mm/proc-arm922.S b/arch/arm/mm/proc-arm922.S index 035a1d1a26b0..8bcc0b913ba0 100644 --- a/arch/arm/mm/proc-arm922.S +++ b/arch/arm/mm/proc-arm922.S @@ -311,12 +311,6 @@ SYM_TYPED_FUNC_START(arm922_dma_unmap_area) ret lr SYM_FUNC_END(arm922_dma_unmap_area) - .globl arm922_flush_kern_cache_louis - .equ arm922_flush_kern_cache_louis, arm922_flush_kern_cache_all - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions arm922 - #endif /* !CONFIG_CPU_DCACHE_WRITETHROUGH */ ENTRY(cpu_arm922_dcache_clean_area) diff --git a/arch/arm/mm/proc-arm925.S b/arch/arm/mm/proc-arm925.S index 2510722647b4..d0d87f9705d3 100644 --- a/arch/arm/mm/proc-arm925.S +++ b/arch/arm/mm/proc-arm925.S @@ -366,12 +366,6 @@ SYM_TYPED_FUNC_START(arm925_dma_unmap_area) ret lr SYM_FUNC_END(arm925_dma_unmap_area) - .globl arm925_flush_kern_cache_louis - .equ arm925_flush_kern_cache_louis, arm925_flush_kern_cache_all - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions arm925 - ENTRY(cpu_arm925_dcache_clean_area) #ifndef CONFIG_CPU_DCACHE_WRITETHROUGH 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry diff --git a/arch/arm/mm/proc-arm926.S b/arch/arm/mm/proc-arm926.S index dac4a22369ba..6cb98b7a0fee 100644 --- a/arch/arm/mm/proc-arm926.S +++ b/arch/arm/mm/proc-arm926.S @@ -329,12 +329,6 @@ SYM_TYPED_FUNC_START(arm926_dma_unmap_area) ret lr SYM_FUNC_END(arm926_dma_unmap_area) - .globl arm926_flush_kern_cache_louis - .equ arm926_flush_kern_cache_louis, arm926_flush_kern_cache_all - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions arm926 - ENTRY(cpu_arm926_dcache_clean_area) #ifndef CONFIG_CPU_DCACHE_WRITETHROUGH 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry diff --git a/arch/arm/mm/proc-arm940.S b/arch/arm/mm/proc-arm940.S index 7c2268059536..527f1c044683 100644 --- a/arch/arm/mm/proc-arm940.S +++ b/arch/arm/mm/proc-arm940.S @@ -267,12 +267,6 @@ SYM_TYPED_FUNC_START(arm940_dma_unmap_area) ret lr SYM_FUNC_END(arm940_dma_unmap_area) - .globl arm940_flush_kern_cache_louis - .equ arm940_flush_kern_cache_louis, arm940_flush_kern_cache_all - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions arm940 - .type __arm940_setup, #function __arm940_setup: mov r0, #0 diff --git a/arch/arm/mm/proc-arm946.S b/arch/arm/mm/proc-arm946.S index 3955be1f4521..3155e819ae5f 100644 --- a/arch/arm/mm/proc-arm946.S +++ b/arch/arm/mm/proc-arm946.S @@ -310,12 +310,6 @@ SYM_TYPED_FUNC_START(arm946_dma_unmap_area) ret lr SYM_FUNC_END(arm946_dma_unmap_area) - .globl arm946_flush_kern_cache_louis - .equ arm946_flush_kern_cache_louis, arm946_flush_kern_cache_all - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions arm946 - ENTRY(cpu_arm946_dcache_clean_area) #ifndef CONFIG_CPU_DCACHE_WRITETHROUGH 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry diff --git a/arch/arm/mm/proc-feroceon.S b/arch/arm/mm/proc-feroceon.S index 9b1570ea6858..af9482b07a4f 100644 --- a/arch/arm/mm/proc-feroceon.S +++ b/arch/arm/mm/proc-feroceon.S @@ -412,33 +412,6 @@ SYM_TYPED_FUNC_START(feroceon_dma_unmap_area) ret lr SYM_FUNC_END(feroceon_dma_unmap_area) - .globl feroceon_flush_kern_cache_louis - .equ feroceon_flush_kern_cache_louis, feroceon_flush_kern_cache_all - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions feroceon - -.macro range_alias basename - .globl feroceon_range_\basename - .type feroceon_range_\basename , %function - .equ feroceon_range_\basename , feroceon_\basename -.endm - -/* - * Most of the cache functions are unchanged for this case. - * Export suitable alias symbols for the unchanged functions: - */ - range_alias flush_icache_all - range_alias flush_user_cache_all - range_alias flush_kern_cache_all - range_alias flush_kern_cache_louis - range_alias flush_user_cache_range - range_alias coherent_kern_range - range_alias coherent_user_range - range_alias dma_unmap_area - - define_cache_functions feroceon_range - .align 5 ENTRY(cpu_feroceon_dcache_clean_area) #if defined(CONFIG_CACHE_FEROCEON_L2) && \ diff --git a/arch/arm/mm/proc-macros.S b/arch/arm/mm/proc-macros.S index c0acfeac3e84..e388c4cc0c44 100644 --- a/arch/arm/mm/proc-macros.S +++ b/arch/arm/mm/proc-macros.S @@ -320,24 +320,6 @@ ENTRY(\name\()_processor_functions) #endif .endm -.macro define_cache_functions name:req - .align 2 - .type \name\()_cache_fns, #object -ENTRY(\name\()_cache_fns) - .long \name\()_flush_icache_all - .long \name\()_flush_kern_cache_all - .long \name\()_flush_kern_cache_louis - .long \name\()_flush_user_cache_all - .long \name\()_flush_user_cache_range - .long \name\()_coherent_kern_range - .long \name\()_coherent_user_range - .long \name\()_flush_kern_dcache_area - .long \name\()_dma_map_area - .long \name\()_dma_unmap_area - .long \name\()_dma_flush_range - .size \name\()_cache_fns, . - \name\()_cache_fns -.endm - .macro globl_equ x, y .globl \x .equ \x, \y diff --git a/arch/arm/mm/proc-mohawk.S b/arch/arm/mm/proc-mohawk.S index 0a94cb0464d8..be3a1a997838 100644 --- a/arch/arm/mm/proc-mohawk.S +++ b/arch/arm/mm/proc-mohawk.S @@ -294,12 +294,6 @@ SYM_TYPED_FUNC_START(mohawk_dma_unmap_area) ret lr SYM_FUNC_END(mohawk_dma_unmap_area) - .globl mohawk_flush_kern_cache_louis - .equ mohawk_flush_kern_cache_louis, mohawk_flush_kern_cache_all - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions mohawk - ENTRY(cpu_mohawk_dcache_clean_area) 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry add r0, r0, #CACHE_DLINESIZE diff --git a/arch/arm/mm/proc-xsc3.S b/arch/arm/mm/proc-xsc3.S index b2d907d748e9..7975f93b1e14 100644 --- a/arch/arm/mm/proc-xsc3.S +++ b/arch/arm/mm/proc-xsc3.S @@ -339,12 +339,6 @@ SYM_TYPED_FUNC_START(xsc3_dma_unmap_area) ret lr SYM_FUNC_END(xsc3_dma_unmap_area) - .globl xsc3_flush_kern_cache_louis - .equ xsc3_flush_kern_cache_louis, xsc3_flush_kern_cache_all - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions xsc3 - ENTRY(cpu_xsc3_dcache_clean_area) 1: mcr p15, 0, r0, c7, c10, 1 @ clean L1 D line add r0, r0, #CACHELINESIZE diff --git a/arch/arm/mm/proc-xscale.S b/arch/arm/mm/proc-xscale.S index 05d9ed952983..bbf1e94ba554 100644 --- a/arch/arm/mm/proc-xscale.S +++ b/arch/arm/mm/proc-xscale.S @@ -391,6 +391,20 @@ SYM_TYPED_FUNC_START(xscale_dma_map_area) b xscale_dma_flush_range SYM_FUNC_END(xscale_dma_map_area) +/* + * On stepping A0/A1 of the 80200, invalidating D-cache by line doesn't + * clear the dirty bits, which means that if we invalidate a dirty line, + * the dirty data can still be written back to external memory later on. + * + * The recommended workaround is to always do a clean D-cache line before + * doing an invalidate D-cache line, so on the affected processors, + * dma_inv_range() is implemented as dma_flush_range(). + * + * See erratum #25 of "Intel 80200 Processor Specification Update", + * revision January 22, 2003, available at: + * http://www.intel.com/design/iio/specupdt/273415.htm + */ + /* * dma_map_area(start, size, dir) * - start - kernel virtual start address @@ -414,49 +428,6 @@ SYM_TYPED_FUNC_START(xscale_dma_unmap_area) ret lr SYM_FUNC_END(xscale_dma_unmap_area) - .globl xscale_flush_kern_cache_louis - .equ xscale_flush_kern_cache_louis, xscale_flush_kern_cache_all - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions xscale - -/* - * On stepping A0/A1 of the 80200, invalidating D-cache by line doesn't - * clear the dirty bits, which means that if we invalidate a dirty line, - * the dirty data can still be written back to external memory later on. - * - * The recommended workaround is to always do a clean D-cache line before - * doing an invalidate D-cache line, so on the affected processors, - * dma_inv_range() is implemented as dma_flush_range(). - * - * See erratum #25 of "Intel 80200 Processor Specification Update", - * revision January 22, 2003, available at: - * http://www.intel.com/design/iio/specupdt/273415.htm - */ -.macro a0_alias basename - .globl xscale_80200_A0_A1_\basename - .type xscale_80200_A0_A1_\basename , %function - .equ xscale_80200_A0_A1_\basename , xscale_\basename -.endm - -/* - * Most of the cache functions are unchanged for these processor revisions. - * Export suitable alias symbols for the unchanged functions: - */ - a0_alias flush_icache_all - a0_alias flush_user_cache_all - a0_alias flush_kern_cache_all - a0_alias flush_kern_cache_louis - a0_alias flush_user_cache_range - a0_alias coherent_kern_range - a0_alias coherent_user_range - a0_alias flush_kern_dcache_area - a0_alias dma_flush_range - a0_alias dma_unmap_area - - @ define struct cpu_cache_fns (see and proc-macros.S) - define_cache_functions xscale_80200_A0_A1 - ENTRY(cpu_xscale_dcache_clean_area) 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry add r0, r0, #CACHELINESIZE From patchwork Sun Apr 21 18:28:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13637488 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C6FE8C4345F for ; Sun, 21 Apr 2024 18:29:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=12hkiJC4JfNxGNpF5CkND4nEuS4bdYFeo+W3me2LhvQ=; b=i1XoBVSMhIu8fx dskZeHcUuB4J9UUCU2mlIVD32VaHijoKbBRQ/L70L4m5vX1g1dzd0tDQEvguVNGpuFIVAiUdfx7Sd 5kRDCvJZjtUhOYjTdKk5MoYtToJquVkHgZi4ukvbWu8V9ezHOcmHzTPaRy3XzVw26BILesrE0+Mbz GjnSxf2ZAJt+1ADXGt31JDaMMfhvPOg7y752ugDcTTkxod55ShRXQoH4/BMHU05GCtA77bIUwU7SU pWjUiIReS1rTEaZrGsKV7Mf3WYAuUnwOLlOG+jBwW19+Jn5Yl97bHNAJQgTh8bBQ0m/e84gflXg9H XMdILlhLIuQGUxt92LMA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybw1-0000000B4Fq-0Vdd; Sun, 21 Apr 2024 18:29:05 +0000 Received: from mail-lf1-x12e.google.com ([2a00:1450:4864:20::12e]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvb-0000000B41x-1L8U for linux-arm-kernel@lists.infradead.org; Sun, 21 Apr 2024 18:28:46 +0000 Received: by mail-lf1-x12e.google.com with SMTP id 2adb3069b0e04-51967f75763so4298322e87.2 for ; Sun, 21 Apr 2024 11:28:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1713724116; x=1714328916; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=IuvuYq8tTzca01z5gyfyOF2vL2NvZBnLK7rlGaLjsTY=; b=U4ZQY/vIZYgH0YdG3Oi+dRTrpOY+v1grZPEFGs2PqSQc3AL65OKi+JhNso+kZ7knKb PHIm38MK5N3EzLpLzF+B6HMq8f3Xe1CNCUX2nnU3kiuVbYw399qODG4tZ3ge2gULY7gz HPHW2wU/lD1VD3o2B528dQwLPRAt6FdGI2oam95YZxpr7P+g9APwoUbbJDTPS2RGLwcZ cwuOoogMEgbr1FJKqUYeQ3nv9+GqZ2JS1B++s+Sqfyirrwk839nOi357CtmgIg1x5s7p 7Nz6h8yghegOEIbVOEM7LZcVDghoUXTw0aVcbkPIkgfop95dtd6iNBF4RTb3dyjcwDJT hQGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713724116; x=1714328916; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IuvuYq8tTzca01z5gyfyOF2vL2NvZBnLK7rlGaLjsTY=; b=sMptUHVyAik16hWiHbQxE53aL/UemcpRgqQt/U3cy2A4f2R86j08JKP/HIVe2FpeE0 KjrvRZmZtOrjs+G6GSBxiElyaNtKBAcxMcO/no+zF4uPtMNQA4LwsFFXXQvEoQumBfek HQmjiB77CYLbuKvsmi84ZByf/LYQMbz1x5caGet424pEZnVTRNmouoLGHCJRjVmOg6yU 5f+I9MbKtRfpiO6QslyD5gMYGy7rTC7Ir/9objeZV5cdEo2KaGd4hM4wa94lFZ1j2kDX /0KlQWwDCvAcV63udEUHve9Pr/D7OK12KqXHaapWNlNuImQ7QCXecd0zwMOriF2u7qRb ok9g== X-Gm-Message-State: AOJu0Yx3Zldz/tWCAqqrTU29XMs35K5P+zABYTHBw7yJ9fbhlviD+Azn bNExToMfsDvzB4zFww/7uaFWZCwYLsrnGrhAmkpZYJIhD/Qf8A5EWS2MyTb7Y/U= X-Google-Smtp-Source: AGHT+IEw4g/szBzQn1d5DavIjNAbBdvJf4OpV2oEQKAcrUb7IlQIuljT3vySWkm9jyFei5QOG+5ZEg== X-Received: by 2002:ac2:4ad9:0:b0:51a:fe16:cf45 with SMTP id m25-20020ac24ad9000000b0051afe16cf45mr2106107lfp.52.1713724115825; Sun, 21 Apr 2024 11:28:35 -0700 (PDT) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id s14-20020a19770e000000b005188ea5d2b5sm1549641lfc.303.2024.04.21.11.28.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Apr 2024 11:28:35 -0700 (PDT) From: Linus Walleij Date: Sun, 21 Apr 2024 20:28:33 +0200 Subject: [PATCH v7 07/11] ARM: mm: Type-annotate all per-processor assembly routines MIME-Version: 1.0 Message-Id: <20240421-arm32-cfi-v7-7-6e132a948cc8@linaro.org> References: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> In-Reply-To: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> To: Russell King , Sami Tolvanen , Kees Cook , Nathan Chancellor , Nick Desaulniers , Ard Biesheuvel , Arnd Bergmann Cc: linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev, Linus Walleij X-Mailer: b4 0.13.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240421_112839_794901_FB38B499 X-CRM114-Status: GOOD ( 11.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Type tag the remaining per-processor assembly using the CFI symbol macros, in addition to those that were previously tagged for cache maintenance calls. This will be used to finally provide proper C prototypes for all these calls as well so that CFI can be made to work. Tested-by: Kees Cook Signed-off-by: Linus Walleij Acked-by: Arnd Bergmann --- arch/arm/mm/proc-arm1020.S | 24 ++++++++++------ arch/arm/mm/proc-arm1020e.S | 24 ++++++++++------ arch/arm/mm/proc-arm1022.S | 24 ++++++++++------ arch/arm/mm/proc-arm1026.S | 24 ++++++++++------ arch/arm/mm/proc-arm720.S | 25 +++++++++++------ arch/arm/mm/proc-arm740.S | 26 ++++++++++++----- arch/arm/mm/proc-arm7tdmi.S | 34 +++++++++++++++-------- arch/arm/mm/proc-arm920.S | 31 ++++++++++++--------- arch/arm/mm/proc-arm922.S | 23 +++++++++------ arch/arm/mm/proc-arm925.S | 22 +++++++++------ arch/arm/mm/proc-arm926.S | 31 +++++++++++++-------- arch/arm/mm/proc-arm940.S | 21 +++++++++----- arch/arm/mm/proc-arm946.S | 21 +++++++++----- arch/arm/mm/proc-arm9tdmi.S | 26 ++++++++++++----- arch/arm/mm/proc-fa526.S | 24 ++++++++++------ arch/arm/mm/proc-feroceon.S | 30 ++++++++++++-------- arch/arm/mm/proc-mohawk.S | 30 ++++++++++++-------- arch/arm/mm/proc-sa110.S | 23 +++++++++------ arch/arm/mm/proc-sa1100.S | 31 +++++++++++++-------- arch/arm/mm/proc-v6.S | 31 +++++++++++++-------- arch/arm/mm/proc-v7-2level.S | 8 +++--- arch/arm/mm/proc-v7-3level.S | 8 +++--- arch/arm/mm/proc-v7.S | 66 +++++++++++++++++++++++--------------------- arch/arm/mm/proc-v7m.S | 41 +++++++++++++-------------- arch/arm/mm/proc-xsc3.S | 30 ++++++++++++-------- arch/arm/mm/proc-xscale.S | 30 ++++++++++++-------- 26 files changed, 434 insertions(+), 274 deletions(-) diff --git a/arch/arm/mm/proc-arm1020.S b/arch/arm/mm/proc-arm1020.S index 1e014cc5b4d1..e6944ecd23ab 100644 --- a/arch/arm/mm/proc-arm1020.S +++ b/arch/arm/mm/proc-arm1020.S @@ -57,18 +57,20 @@ /* * cpu_arm1020_proc_init() */ -ENTRY(cpu_arm1020_proc_init) +SYM_TYPED_FUNC_START(cpu_arm1020_proc_init) ret lr +SYM_FUNC_END(cpu_arm1020_proc_init) /* * cpu_arm1020_proc_fin() */ -ENTRY(cpu_arm1020_proc_fin) +SYM_TYPED_FUNC_START(cpu_arm1020_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x1000 @ ...i............ bic r0, r0, #0x000e @ ............wca. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_arm1020_proc_fin) /* * cpu_arm1020_reset(loc) @@ -81,7 +83,7 @@ ENTRY(cpu_arm1020_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_arm1020_reset) +SYM_TYPED_FUNC_START(cpu_arm1020_reset) mov ip, #0 mcr p15, 0, ip, c7, c7, 0 @ invalidate I,D caches mcr p15, 0, ip, c7, c10, 4 @ drain WB @@ -93,16 +95,17 @@ ENTRY(cpu_arm1020_reset) bic ip, ip, #0x1100 @ ...i...s........ mcr p15, 0, ip, c1, c0, 0 @ ctrl register ret r0 -ENDPROC(cpu_arm1020_reset) +SYM_FUNC_END(cpu_arm1020_reset) .popsection /* * cpu_arm1020_do_idle() */ .align 5 -ENTRY(cpu_arm1020_do_idle) +SYM_TYPED_FUNC_START(cpu_arm1020_do_idle) mcr p15, 0, r0, c7, c0, 4 @ Wait for interrupt ret lr +SYM_FUNC_END(cpu_arm1020_do_idle) /* ================================= CACHE ================================ */ @@ -358,7 +361,7 @@ SYM_TYPED_FUNC_START(arm1020_dma_unmap_area) SYM_FUNC_END(arm1020_dma_unmap_area) .align 5 -ENTRY(cpu_arm1020_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_arm1020_dcache_clean_area) #ifndef CONFIG_CPU_DCACHE_DISABLE mov ip, #0 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry @@ -368,6 +371,7 @@ ENTRY(cpu_arm1020_dcache_clean_area) bhi 1b #endif ret lr +SYM_FUNC_END(cpu_arm1020_dcache_clean_area) /* =============================== PageTable ============================== */ @@ -379,7 +383,7 @@ ENTRY(cpu_arm1020_dcache_clean_area) * pgd: new page tables */ .align 5 -ENTRY(cpu_arm1020_switch_mm) +SYM_TYPED_FUNC_START(cpu_arm1020_switch_mm) #ifdef CONFIG_MMU #ifndef CONFIG_CPU_DCACHE_DISABLE mcr p15, 0, r3, c7, c10, 4 @@ -407,14 +411,15 @@ ENTRY(cpu_arm1020_switch_mm) mcr p15, 0, r1, c8, c7, 0 @ invalidate I & D TLBs #endif /* CONFIG_MMU */ ret lr - +SYM_FUNC_END(cpu_arm1020_switch_mm) + /* * cpu_arm1020_set_pte(ptep, pte) * * Set a PTE and flush it out */ .align 5 -ENTRY(cpu_arm1020_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_arm1020_set_pte_ext) #ifdef CONFIG_MMU armv3_set_pte_ext mov r0, r0 @@ -425,6 +430,7 @@ ENTRY(cpu_arm1020_set_pte_ext) mcr p15, 0, r0, c7, c10, 4 @ drain WB #endif /* CONFIG_MMU */ ret lr +SYM_FUNC_END(cpu_arm1020_set_pte_ext) .type __arm1020_setup, #function __arm1020_setup: diff --git a/arch/arm/mm/proc-arm1020e.S b/arch/arm/mm/proc-arm1020e.S index 7d80761f207a..5fae6e28c7a3 100644 --- a/arch/arm/mm/proc-arm1020e.S +++ b/arch/arm/mm/proc-arm1020e.S @@ -57,18 +57,20 @@ /* * cpu_arm1020e_proc_init() */ -ENTRY(cpu_arm1020e_proc_init) +SYM_TYPED_FUNC_START(cpu_arm1020e_proc_init) ret lr +SYM_FUNC_END(cpu_arm1020e_proc_init) /* * cpu_arm1020e_proc_fin() */ -ENTRY(cpu_arm1020e_proc_fin) +SYM_TYPED_FUNC_START(cpu_arm1020e_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x1000 @ ...i............ bic r0, r0, #0x000e @ ............wca. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_arm1020e_proc_fin) /* * cpu_arm1020e_reset(loc) @@ -81,7 +83,7 @@ ENTRY(cpu_arm1020e_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_arm1020e_reset) +SYM_TYPED_FUNC_START(cpu_arm1020e_reset) mov ip, #0 mcr p15, 0, ip, c7, c7, 0 @ invalidate I,D caches mcr p15, 0, ip, c7, c10, 4 @ drain WB @@ -93,16 +95,17 @@ ENTRY(cpu_arm1020e_reset) bic ip, ip, #0x1100 @ ...i...s........ mcr p15, 0, ip, c1, c0, 0 @ ctrl register ret r0 -ENDPROC(cpu_arm1020e_reset) +SYM_FUNC_END(cpu_arm1020e_reset) .popsection /* * cpu_arm1020e_do_idle() */ .align 5 -ENTRY(cpu_arm1020e_do_idle) +SYM_TYPED_FUNC_START(cpu_arm1020e_do_idle) mcr p15, 0, r0, c7, c0, 4 @ Wait for interrupt ret lr +SYM_FUNC_END(cpu_arm1020e_do_idle) /* ================================= CACHE ================================ */ @@ -345,7 +348,7 @@ SYM_TYPED_FUNC_START(arm1020e_dma_unmap_area) SYM_FUNC_END(arm1020e_dma_unmap_area) .align 5 -ENTRY(cpu_arm1020e_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_arm1020e_dcache_clean_area) #ifndef CONFIG_CPU_DCACHE_DISABLE mov ip, #0 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry @@ -354,6 +357,7 @@ ENTRY(cpu_arm1020e_dcache_clean_area) bhi 1b #endif ret lr +SYM_FUNC_END(cpu_arm1020e_dcache_clean_area) /* =============================== PageTable ============================== */ @@ -365,7 +369,7 @@ ENTRY(cpu_arm1020e_dcache_clean_area) * pgd: new page tables */ .align 5 -ENTRY(cpu_arm1020e_switch_mm) +SYM_TYPED_FUNC_START(cpu_arm1020e_switch_mm) #ifdef CONFIG_MMU #ifndef CONFIG_CPU_DCACHE_DISABLE mcr p15, 0, r3, c7, c10, 4 @@ -392,14 +396,15 @@ ENTRY(cpu_arm1020e_switch_mm) mcr p15, 0, r1, c8, c7, 0 @ invalidate I & D TLBs #endif ret lr - +SYM_FUNC_END(cpu_arm1020e_switch_mm) + /* * cpu_arm1020e_set_pte(ptep, pte) * * Set a PTE and flush it out */ .align 5 -ENTRY(cpu_arm1020e_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_arm1020e_set_pte_ext) #ifdef CONFIG_MMU armv3_set_pte_ext mov r0, r0 @@ -408,6 +413,7 @@ ENTRY(cpu_arm1020e_set_pte_ext) #endif #endif /* CONFIG_MMU */ ret lr +SYM_FUNC_END(cpu_arm1020e_set_pte_ext) .type __arm1020e_setup, #function __arm1020e_setup: diff --git a/arch/arm/mm/proc-arm1022.S b/arch/arm/mm/proc-arm1022.S index 53b1541c50d8..05a7f14b2751 100644 --- a/arch/arm/mm/proc-arm1022.S +++ b/arch/arm/mm/proc-arm1022.S @@ -57,18 +57,20 @@ /* * cpu_arm1022_proc_init() */ -ENTRY(cpu_arm1022_proc_init) +SYM_TYPED_FUNC_START(cpu_arm1022_proc_init) ret lr +SYM_FUNC_END(cpu_arm1022_proc_init) /* * cpu_arm1022_proc_fin() */ -ENTRY(cpu_arm1022_proc_fin) +SYM_TYPED_FUNC_START(cpu_arm1022_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x1000 @ ...i............ bic r0, r0, #0x000e @ ............wca. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_arm1022_proc_fin) /* * cpu_arm1022_reset(loc) @@ -81,7 +83,7 @@ ENTRY(cpu_arm1022_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_arm1022_reset) +SYM_TYPED_FUNC_START(cpu_arm1022_reset) mov ip, #0 mcr p15, 0, ip, c7, c7, 0 @ invalidate I,D caches mcr p15, 0, ip, c7, c10, 4 @ drain WB @@ -93,16 +95,17 @@ ENTRY(cpu_arm1022_reset) bic ip, ip, #0x1100 @ ...i...s........ mcr p15, 0, ip, c1, c0, 0 @ ctrl register ret r0 -ENDPROC(cpu_arm1022_reset) +SYM_FUNC_END(cpu_arm1022_reset) .popsection /* * cpu_arm1022_do_idle() */ .align 5 -ENTRY(cpu_arm1022_do_idle) +SYM_TYPED_FUNC_START(cpu_arm1022_do_idle) mcr p15, 0, r0, c7, c0, 4 @ Wait for interrupt ret lr +SYM_FUNC_END(cpu_arm1022_do_idle) /* ================================= CACHE ================================ */ @@ -344,7 +347,7 @@ SYM_TYPED_FUNC_START(arm1022_dma_unmap_area) SYM_FUNC_END(arm1022_dma_unmap_area) .align 5 -ENTRY(cpu_arm1022_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_arm1022_dcache_clean_area) #ifndef CONFIG_CPU_DCACHE_DISABLE mov ip, #0 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry @@ -353,6 +356,7 @@ ENTRY(cpu_arm1022_dcache_clean_area) bhi 1b #endif ret lr +SYM_FUNC_END(cpu_arm1022_dcache_clean_area) /* =============================== PageTable ============================== */ @@ -364,7 +368,7 @@ ENTRY(cpu_arm1022_dcache_clean_area) * pgd: new page tables */ .align 5 -ENTRY(cpu_arm1022_switch_mm) +SYM_TYPED_FUNC_START(cpu_arm1022_switch_mm) #ifdef CONFIG_MMU #ifndef CONFIG_CPU_DCACHE_DISABLE mov r1, #(CACHE_DSEGMENTS - 1) << 5 @ 16 segments @@ -384,14 +388,15 @@ ENTRY(cpu_arm1022_switch_mm) mcr p15, 0, r1, c8, c7, 0 @ invalidate I & D TLBs #endif ret lr - +SYM_FUNC_END(cpu_arm1022_switch_mm) + /* * cpu_arm1022_set_pte_ext(ptep, pte, ext) * * Set a PTE and flush it out */ .align 5 -ENTRY(cpu_arm1022_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_arm1022_set_pte_ext) #ifdef CONFIG_MMU armv3_set_pte_ext mov r0, r0 @@ -400,6 +405,7 @@ ENTRY(cpu_arm1022_set_pte_ext) #endif #endif /* CONFIG_MMU */ ret lr +SYM_FUNC_END(cpu_arm1022_set_pte_ext) .type __arm1022_setup, #function __arm1022_setup: diff --git a/arch/arm/mm/proc-arm1026.S b/arch/arm/mm/proc-arm1026.S index 6c6ea0357a77..6800dd7c73f8 100644 --- a/arch/arm/mm/proc-arm1026.S +++ b/arch/arm/mm/proc-arm1026.S @@ -57,18 +57,20 @@ /* * cpu_arm1026_proc_init() */ -ENTRY(cpu_arm1026_proc_init) +SYM_TYPED_FUNC_START(cpu_arm1026_proc_init) ret lr +SYM_FUNC_END(cpu_arm1026_proc_init) /* * cpu_arm1026_proc_fin() */ -ENTRY(cpu_arm1026_proc_fin) +SYM_TYPED_FUNC_START(cpu_arm1026_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x1000 @ ...i............ bic r0, r0, #0x000e @ ............wca. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_arm1026_proc_fin) /* * cpu_arm1026_reset(loc) @@ -81,7 +83,7 @@ ENTRY(cpu_arm1026_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_arm1026_reset) +SYM_TYPED_FUNC_START(cpu_arm1026_reset) mov ip, #0 mcr p15, 0, ip, c7, c7, 0 @ invalidate I,D caches mcr p15, 0, ip, c7, c10, 4 @ drain WB @@ -93,16 +95,17 @@ ENTRY(cpu_arm1026_reset) bic ip, ip, #0x1100 @ ...i...s........ mcr p15, 0, ip, c1, c0, 0 @ ctrl register ret r0 -ENDPROC(cpu_arm1026_reset) +SYM_FUNC_END(cpu_arm1026_reset) .popsection /* * cpu_arm1026_do_idle() */ .align 5 -ENTRY(cpu_arm1026_do_idle) +SYM_TYPED_FUNC_START(cpu_arm1026_do_idle) mcr p15, 0, r0, c7, c0, 4 @ Wait for interrupt ret lr +SYM_FUNC_END(cpu_arm1026_do_idle) /* ================================= CACHE ================================ */ @@ -339,7 +342,7 @@ SYM_TYPED_FUNC_START(arm1026_dma_unmap_area) SYM_FUNC_END(arm1026_dma_unmap_area) .align 5 -ENTRY(cpu_arm1026_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_arm1026_dcache_clean_area) #ifndef CONFIG_CPU_DCACHE_DISABLE mov ip, #0 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry @@ -348,6 +351,7 @@ ENTRY(cpu_arm1026_dcache_clean_area) bhi 1b #endif ret lr +SYM_FUNC_END(cpu_arm1026_dcache_clean_area) /* =============================== PageTable ============================== */ @@ -359,7 +363,7 @@ ENTRY(cpu_arm1026_dcache_clean_area) * pgd: new page tables */ .align 5 -ENTRY(cpu_arm1026_switch_mm) +SYM_TYPED_FUNC_START(cpu_arm1026_switch_mm) #ifdef CONFIG_MMU mov r1, #0 #ifndef CONFIG_CPU_DCACHE_DISABLE @@ -374,14 +378,15 @@ ENTRY(cpu_arm1026_switch_mm) mcr p15, 0, r1, c8, c7, 0 @ invalidate I & D TLBs #endif ret lr - +SYM_FUNC_END(cpu_arm1026_switch_mm) + /* * cpu_arm1026_set_pte_ext(ptep, pte, ext) * * Set a PTE and flush it out */ .align 5 -ENTRY(cpu_arm1026_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_arm1026_set_pte_ext) #ifdef CONFIG_MMU armv3_set_pte_ext mov r0, r0 @@ -390,6 +395,7 @@ ENTRY(cpu_arm1026_set_pte_ext) #endif #endif /* CONFIG_MMU */ ret lr +SYM_FUNC_END(cpu_arm1026_set_pte_ext) .type __arm1026_setup, #function __arm1026_setup: diff --git a/arch/arm/mm/proc-arm720.S b/arch/arm/mm/proc-arm720.S index 3b687e6dd9fd..59732c334e1d 100644 --- a/arch/arm/mm/proc-arm720.S +++ b/arch/arm/mm/proc-arm720.S @@ -20,6 +20,7 @@ */ #include #include +#include #include #include #include @@ -35,24 +36,30 @@ * * Notes : This processor does not require these */ -ENTRY(cpu_arm720_dcache_clean_area) -ENTRY(cpu_arm720_proc_init) +SYM_TYPED_FUNC_START(cpu_arm720_dcache_clean_area) ret lr +SYM_FUNC_END(cpu_arm720_dcache_clean_area) -ENTRY(cpu_arm720_proc_fin) +SYM_TYPED_FUNC_START(cpu_arm720_proc_init) + ret lr +SYM_FUNC_END(cpu_arm720_proc_init) + +SYM_TYPED_FUNC_START(cpu_arm720_proc_fin) mrc p15, 0, r0, c1, c0, 0 bic r0, r0, #0x1000 @ ...i............ bic r0, r0, #0x000e @ ............wca. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_arm720_proc_fin) /* * Function: arm720_proc_do_idle(void) * Params : r0 = unused * Purpose : put the processor in proper idle mode */ -ENTRY(cpu_arm720_do_idle) +SYM_TYPED_FUNC_START(cpu_arm720_do_idle) ret lr +SYM_FUNC_END(cpu_arm720_do_idle) /* * Function: arm720_switch_mm(unsigned long pgd_phys) @@ -60,7 +67,7 @@ ENTRY(cpu_arm720_do_idle) * Purpose : Perform a task switch, saving the old process' state and restoring * the new. */ -ENTRY(cpu_arm720_switch_mm) +SYM_TYPED_FUNC_START(cpu_arm720_switch_mm) #ifdef CONFIG_MMU mov r1, #0 mcr p15, 0, r1, c7, c7, 0 @ invalidate cache @@ -68,6 +75,7 @@ ENTRY(cpu_arm720_switch_mm) mcr p15, 0, r1, c8, c7, 0 @ flush TLB (v4) #endif ret lr +SYM_FUNC_END(cpu_arm720_switch_mm) /* * Function: arm720_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext) @@ -76,11 +84,12 @@ ENTRY(cpu_arm720_switch_mm) * Purpose : Set a PTE and flush it out of any WB cache */ .align 5 -ENTRY(cpu_arm720_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_arm720_set_pte_ext) #ifdef CONFIG_MMU armv3_set_pte_ext wc_disable=0 #endif ret lr +SYM_FUNC_END(cpu_arm720_set_pte_ext) /* * Function: arm720_reset @@ -88,7 +97,7 @@ ENTRY(cpu_arm720_set_pte_ext) * Notes : This sets up everything for a reset */ .pushsection .idmap.text, "ax" -ENTRY(cpu_arm720_reset) +SYM_TYPED_FUNC_START(cpu_arm720_reset) mov ip, #0 mcr p15, 0, ip, c7, c7, 0 @ invalidate cache #ifdef CONFIG_MMU @@ -99,7 +108,7 @@ ENTRY(cpu_arm720_reset) bic ip, ip, #0x2100 @ ..v....s........ mcr p15, 0, ip, c1, c0, 0 @ ctrl register ret r0 -ENDPROC(cpu_arm720_reset) +SYM_FUNC_END(cpu_arm720_reset) .popsection .type __arm710_setup, #function diff --git a/arch/arm/mm/proc-arm740.S b/arch/arm/mm/proc-arm740.S index f2ec3bc60874..78854df63964 100644 --- a/arch/arm/mm/proc-arm740.S +++ b/arch/arm/mm/proc-arm740.S @@ -6,6 +6,7 @@ */ #include #include +#include #include #include #include @@ -24,21 +25,32 @@ * * These are not required. */ -ENTRY(cpu_arm740_proc_init) -ENTRY(cpu_arm740_do_idle) -ENTRY(cpu_arm740_dcache_clean_area) -ENTRY(cpu_arm740_switch_mm) +SYM_TYPED_FUNC_START(cpu_arm740_proc_init) ret lr +SYM_FUNC_END(cpu_arm740_proc_init) + +SYM_TYPED_FUNC_START(cpu_arm740_do_idle) + ret lr +SYM_FUNC_END(cpu_arm740_do_idle) + +SYM_TYPED_FUNC_START(cpu_arm740_dcache_clean_area) + ret lr +SYM_FUNC_END(cpu_arm740_dcache_clean_area) + +SYM_TYPED_FUNC_START(cpu_arm740_switch_mm) + ret lr +SYM_FUNC_END(cpu_arm740_switch_mm) /* * cpu_arm740_proc_fin() */ -ENTRY(cpu_arm740_proc_fin) +SYM_TYPED_FUNC_START(cpu_arm740_proc_fin) mrc p15, 0, r0, c1, c0, 0 bic r0, r0, #0x3f000000 @ bank/f/lock/s bic r0, r0, #0x0000000c @ w-buffer/cache mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_arm740_proc_fin) /* * cpu_arm740_reset(loc) @@ -46,14 +58,14 @@ ENTRY(cpu_arm740_proc_fin) * Notes : This sets up everything for a reset */ .pushsection .idmap.text, "ax" -ENTRY(cpu_arm740_reset) +SYM_TYPED_FUNC_START(cpu_arm740_reset) mov ip, #0 mcr p15, 0, ip, c7, c0, 0 @ invalidate cache mrc p15, 0, ip, c1, c0, 0 @ get ctrl register bic ip, ip, #0x0000000c @ ............wc.. mcr p15, 0, ip, c1, c0, 0 @ ctrl register ret r0 -ENDPROC(cpu_arm740_reset) +SYM_FUNC_END(cpu_arm740_reset) .popsection .type __arm740_setup, #function diff --git a/arch/arm/mm/proc-arm7tdmi.S b/arch/arm/mm/proc-arm7tdmi.S index 01bbe7576c1c..baa3d4472147 100644 --- a/arch/arm/mm/proc-arm7tdmi.S +++ b/arch/arm/mm/proc-arm7tdmi.S @@ -6,6 +6,7 @@ */ #include #include +#include #include #include #include @@ -23,18 +24,29 @@ * cpu_arm7tdmi_switch_mm() * * These are not required. - */ -ENTRY(cpu_arm7tdmi_proc_init) -ENTRY(cpu_arm7tdmi_do_idle) -ENTRY(cpu_arm7tdmi_dcache_clean_area) -ENTRY(cpu_arm7tdmi_switch_mm) - ret lr +*/ +SYM_TYPED_FUNC_START(cpu_arm7tdmi_proc_init) + ret lr +SYM_FUNC_END(cpu_arm7tdmi_proc_init) + +SYM_TYPED_FUNC_START(cpu_arm7tdmi_do_idle) + ret lr +SYM_FUNC_END(cpu_arm7tdmi_do_idle) + +SYM_TYPED_FUNC_START(cpu_arm7tdmi_dcache_clean_area) + ret lr +SYM_FUNC_END(cpu_arm7tdmi_dcache_clean_area) + +SYM_TYPED_FUNC_START(cpu_arm7tdmi_switch_mm) + ret lr +SYM_FUNC_END(cpu_arm7tdmi_switch_mm) /* * cpu_arm7tdmi_proc_fin() - */ -ENTRY(cpu_arm7tdmi_proc_fin) - ret lr +*/ +SYM_TYPED_FUNC_START(cpu_arm7tdmi_proc_fin) + ret lr +SYM_FUNC_END(cpu_arm7tdmi_proc_fin) /* * Function: cpu_arm7tdmi_reset(loc) @@ -42,9 +54,9 @@ ENTRY(cpu_arm7tdmi_proc_fin) * Purpose : Sets up everything for a reset and jump to the location for soft reset. */ .pushsection .idmap.text, "ax" -ENTRY(cpu_arm7tdmi_reset) +SYM_TYPED_FUNC_START(cpu_arm7tdmi_reset) ret r0 -ENDPROC(cpu_arm7tdmi_reset) +SYM_FUNC_END(cpu_arm7tdmi_reset) .popsection .type __arm7tdmi_setup, #function diff --git a/arch/arm/mm/proc-arm920.S b/arch/arm/mm/proc-arm920.S index 08a5bac0d89d..a1eec82070e5 100644 --- a/arch/arm/mm/proc-arm920.S +++ b/arch/arm/mm/proc-arm920.S @@ -49,18 +49,20 @@ /* * cpu_arm920_proc_init() */ -ENTRY(cpu_arm920_proc_init) +SYM_TYPED_FUNC_START(cpu_arm920_proc_init) ret lr +SYM_FUNC_END(cpu_arm920_proc_init) /* * cpu_arm920_proc_fin() */ -ENTRY(cpu_arm920_proc_fin) +SYM_TYPED_FUNC_START(cpu_arm920_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x1000 @ ...i............ bic r0, r0, #0x000e @ ............wca. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_arm920_proc_fin) /* * cpu_arm920_reset(loc) @@ -73,7 +75,7 @@ ENTRY(cpu_arm920_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_arm920_reset) +SYM_TYPED_FUNC_START(cpu_arm920_reset) mov ip, #0 mcr p15, 0, ip, c7, c7, 0 @ invalidate I,D caches mcr p15, 0, ip, c7, c10, 4 @ drain WB @@ -85,17 +87,17 @@ ENTRY(cpu_arm920_reset) bic ip, ip, #0x1100 @ ...i...s........ mcr p15, 0, ip, c1, c0, 0 @ ctrl register ret r0 -ENDPROC(cpu_arm920_reset) +SYM_FUNC_END(cpu_arm920_reset) .popsection /* * cpu_arm920_do_idle() */ .align 5 -ENTRY(cpu_arm920_do_idle) +SYM_TYPED_FUNC_START(cpu_arm920_do_idle) mcr p15, 0, r0, c7, c0, 4 @ Wait for interrupt ret lr - +SYM_FUNC_END(cpu_arm920_do_idle) #ifndef CONFIG_CPU_DCACHE_WRITETHROUGH @@ -312,12 +314,13 @@ SYM_FUNC_END(arm920_dma_unmap_area) #endif /* !CONFIG_CPU_DCACHE_WRITETHROUGH */ -ENTRY(cpu_arm920_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_arm920_dcache_clean_area) 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry add r0, r0, #CACHE_DLINESIZE subs r1, r1, #CACHE_DLINESIZE bhi 1b ret lr +SYM_FUNC_END(cpu_arm920_dcache_clean_area) /* =============================== PageTable ============================== */ @@ -329,7 +332,7 @@ ENTRY(cpu_arm920_dcache_clean_area) * pgd: new page tables */ .align 5 -ENTRY(cpu_arm920_switch_mm) +SYM_TYPED_FUNC_START(cpu_arm920_switch_mm) #ifdef CONFIG_MMU mov ip, #0 #ifdef CONFIG_CPU_DCACHE_WRITETHROUGH @@ -353,6 +356,7 @@ ENTRY(cpu_arm920_switch_mm) mcr p15, 0, ip, c8, c7, 0 @ invalidate I & D TLBs #endif ret lr +SYM_FUNC_END(cpu_arm920_switch_mm) /* * cpu_arm920_set_pte(ptep, pte, ext) @@ -360,7 +364,7 @@ ENTRY(cpu_arm920_switch_mm) * Set a PTE and flush it out */ .align 5 -ENTRY(cpu_arm920_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_arm920_set_pte_ext) #ifdef CONFIG_MMU armv3_set_pte_ext mov r0, r0 @@ -368,21 +372,22 @@ ENTRY(cpu_arm920_set_pte_ext) mcr p15, 0, r0, c7, c10, 4 @ drain WB #endif ret lr +SYM_FUNC_END(cpu_arm920_set_pte_ext) /* Suspend/resume support: taken from arch/arm/plat-s3c24xx/sleep.S */ .globl cpu_arm920_suspend_size .equ cpu_arm920_suspend_size, 4 * 3 #ifdef CONFIG_ARM_CPU_SUSPEND -ENTRY(cpu_arm920_do_suspend) +SYM_TYPED_FUNC_START(cpu_arm920_do_suspend) stmfd sp!, {r4 - r6, lr} mrc p15, 0, r4, c13, c0, 0 @ PID mrc p15, 0, r5, c3, c0, 0 @ Domain ID mrc p15, 0, r6, c1, c0, 0 @ Control register stmia r0, {r4 - r6} ldmfd sp!, {r4 - r6, pc} -ENDPROC(cpu_arm920_do_suspend) +SYM_FUNC_END(cpu_arm920_do_suspend) -ENTRY(cpu_arm920_do_resume) +SYM_TYPED_FUNC_START(cpu_arm920_do_resume) mov ip, #0 mcr p15, 0, ip, c8, c7, 0 @ invalidate I+D TLBs mcr p15, 0, ip, c7, c7, 0 @ invalidate I+D caches @@ -392,7 +397,7 @@ ENTRY(cpu_arm920_do_resume) mcr p15, 0, r1, c2, c0, 0 @ TTB address mov r0, r6 @ control register b cpu_resume_mmu -ENDPROC(cpu_arm920_do_resume) +SYM_FUNC_END(cpu_arm920_do_resume) #endif .type __arm920_setup, #function diff --git a/arch/arm/mm/proc-arm922.S b/arch/arm/mm/proc-arm922.S index 8bcc0b913ba0..aeafac5143f6 100644 --- a/arch/arm/mm/proc-arm922.S +++ b/arch/arm/mm/proc-arm922.S @@ -51,18 +51,20 @@ /* * cpu_arm922_proc_init() */ -ENTRY(cpu_arm922_proc_init) +SYM_TYPED_FUNC_START(cpu_arm922_proc_init) ret lr +SYM_FUNC_END(cpu_arm922_proc_init) /* * cpu_arm922_proc_fin() */ -ENTRY(cpu_arm922_proc_fin) +SYM_TYPED_FUNC_START(cpu_arm922_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x1000 @ ...i............ bic r0, r0, #0x000e @ ............wca. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_arm922_proc_fin) /* * cpu_arm922_reset(loc) @@ -75,7 +77,7 @@ ENTRY(cpu_arm922_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_arm922_reset) +SYM_TYPED_FUNC_START(cpu_arm922_reset) mov ip, #0 mcr p15, 0, ip, c7, c7, 0 @ invalidate I,D caches mcr p15, 0, ip, c7, c10, 4 @ drain WB @@ -87,17 +89,17 @@ ENTRY(cpu_arm922_reset) bic ip, ip, #0x1100 @ ...i...s........ mcr p15, 0, ip, c1, c0, 0 @ ctrl register ret r0 -ENDPROC(cpu_arm922_reset) +SYM_FUNC_END(cpu_arm922_reset) .popsection /* * cpu_arm922_do_idle() */ .align 5 -ENTRY(cpu_arm922_do_idle) +SYM_TYPED_FUNC_START(cpu_arm922_do_idle) mcr p15, 0, r0, c7, c0, 4 @ Wait for interrupt ret lr - +SYM_FUNC_END(cpu_arm922_do_idle) #ifndef CONFIG_CPU_DCACHE_WRITETHROUGH @@ -313,7 +315,7 @@ SYM_FUNC_END(arm922_dma_unmap_area) #endif /* !CONFIG_CPU_DCACHE_WRITETHROUGH */ -ENTRY(cpu_arm922_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_arm922_dcache_clean_area) #ifndef CONFIG_CPU_DCACHE_WRITETHROUGH 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry add r0, r0, #CACHE_DLINESIZE @@ -321,6 +323,7 @@ ENTRY(cpu_arm922_dcache_clean_area) bhi 1b #endif ret lr +SYM_FUNC_END(cpu_arm922_dcache_clean_area) /* =============================== PageTable ============================== */ @@ -332,7 +335,7 @@ ENTRY(cpu_arm922_dcache_clean_area) * pgd: new page tables */ .align 5 -ENTRY(cpu_arm922_switch_mm) +SYM_TYPED_FUNC_START(cpu_arm922_switch_mm) #ifdef CONFIG_MMU mov ip, #0 #ifdef CONFIG_CPU_DCACHE_WRITETHROUGH @@ -356,6 +359,7 @@ ENTRY(cpu_arm922_switch_mm) mcr p15, 0, ip, c8, c7, 0 @ invalidate I & D TLBs #endif ret lr +SYM_FUNC_END(cpu_arm922_switch_mm) /* * cpu_arm922_set_pte_ext(ptep, pte, ext) @@ -363,7 +367,7 @@ ENTRY(cpu_arm922_switch_mm) * Set a PTE and flush it out */ .align 5 -ENTRY(cpu_arm922_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_arm922_set_pte_ext) #ifdef CONFIG_MMU armv3_set_pte_ext mov r0, r0 @@ -371,6 +375,7 @@ ENTRY(cpu_arm922_set_pte_ext) mcr p15, 0, r0, c7, c10, 4 @ drain WB #endif /* CONFIG_MMU */ ret lr +SYM_FUNC_END(cpu_arm922_set_pte_ext) .type __arm922_setup, #function __arm922_setup: diff --git a/arch/arm/mm/proc-arm925.S b/arch/arm/mm/proc-arm925.S index d0d87f9705d3..191f4fa606c7 100644 --- a/arch/arm/mm/proc-arm925.S +++ b/arch/arm/mm/proc-arm925.S @@ -72,18 +72,20 @@ /* * cpu_arm925_proc_init() */ -ENTRY(cpu_arm925_proc_init) +SYM_TYPED_FUNC_START(cpu_arm925_proc_init) ret lr +SYM_FUNC_END(cpu_arm925_proc_init) /* * cpu_arm925_proc_fin() */ -ENTRY(cpu_arm925_proc_fin) +SYM_TYPED_FUNC_START(cpu_arm925_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x1000 @ ...i............ bic r0, r0, #0x000e @ ............wca. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_arm925_proc_fin) /* * cpu_arm925_reset(loc) @@ -96,14 +98,14 @@ ENTRY(cpu_arm925_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_arm925_reset) +SYM_TYPED_FUNC_START(cpu_arm925_reset) /* Send software reset to MPU and DSP */ mov ip, #0xff000000 orr ip, ip, #0x00fe0000 orr ip, ip, #0x0000ce00 mov r4, #1 strh r4, [ip, #0x10] -ENDPROC(cpu_arm925_reset) +SYM_FUNC_END(cpu_arm925_reset) .popsection mov ip, #0 @@ -124,7 +126,7 @@ ENDPROC(cpu_arm925_reset) * Called with IRQs disabled */ .align 10 -ENTRY(cpu_arm925_do_idle) +SYM_TYPED_FUNC_START(cpu_arm925_do_idle) mov r0, #0 mrc p15, 0, r1, c1, c0, 0 @ Read control register mcr p15, 0, r0, c7, c10, 4 @ Drain write buffer @@ -133,6 +135,7 @@ ENTRY(cpu_arm925_do_idle) mcr p15, 0, r0, c7, c0, 4 @ Wait for interrupt mcr p15, 0, r1, c1, c0, 0 @ Restore ICache enable ret lr +SYM_FUNC_END(cpu_arm925_do_idle) /* * flush_icache_all() @@ -366,7 +369,7 @@ SYM_TYPED_FUNC_START(arm925_dma_unmap_area) ret lr SYM_FUNC_END(arm925_dma_unmap_area) -ENTRY(cpu_arm925_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_arm925_dcache_clean_area) #ifndef CONFIG_CPU_DCACHE_WRITETHROUGH 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry add r0, r0, #CACHE_DLINESIZE @@ -375,6 +378,7 @@ ENTRY(cpu_arm925_dcache_clean_area) #endif mcr p15, 0, r0, c7, c10, 4 @ drain WB ret lr +SYM_FUNC_END(cpu_arm925_dcache_clean_area) /* =============================== PageTable ============================== */ @@ -386,7 +390,7 @@ ENTRY(cpu_arm925_dcache_clean_area) * pgd: new page tables */ .align 5 -ENTRY(cpu_arm925_switch_mm) +SYM_TYPED_FUNC_START(cpu_arm925_switch_mm) #ifdef CONFIG_MMU mov ip, #0 #ifdef CONFIG_CPU_DCACHE_WRITETHROUGH @@ -404,6 +408,7 @@ ENTRY(cpu_arm925_switch_mm) mcr p15, 0, ip, c8, c7, 0 @ invalidate I & D TLBs #endif ret lr +SYM_FUNC_END(cpu_arm925_switch_mm) /* * cpu_arm925_set_pte_ext(ptep, pte, ext) @@ -411,7 +416,7 @@ ENTRY(cpu_arm925_switch_mm) * Set a PTE and flush it out */ .align 5 -ENTRY(cpu_arm925_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_arm925_set_pte_ext) #ifdef CONFIG_MMU armv3_set_pte_ext mov r0, r0 @@ -421,6 +426,7 @@ ENTRY(cpu_arm925_set_pte_ext) mcr p15, 0, r0, c7, c10, 4 @ drain WB #endif /* CONFIG_MMU */ ret lr +SYM_FUNC_END(cpu_arm925_set_pte_ext) .type __arm925_setup, #function __arm925_setup: diff --git a/arch/arm/mm/proc-arm926.S b/arch/arm/mm/proc-arm926.S index 6cb98b7a0fee..3bf1d4072283 100644 --- a/arch/arm/mm/proc-arm926.S +++ b/arch/arm/mm/proc-arm926.S @@ -41,18 +41,20 @@ /* * cpu_arm926_proc_init() */ -ENTRY(cpu_arm926_proc_init) +SYM_TYPED_FUNC_START(cpu_arm926_proc_init) ret lr +SYM_FUNC_END(cpu_arm926_proc_init) /* * cpu_arm926_proc_fin() */ -ENTRY(cpu_arm926_proc_fin) +SYM_TYPED_FUNC_START(cpu_arm926_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x1000 @ ...i............ bic r0, r0, #0x000e @ ............wca. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_arm926_proc_fin) /* * cpu_arm926_reset(loc) @@ -65,7 +67,7 @@ ENTRY(cpu_arm926_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_arm926_reset) +SYM_TYPED_FUNC_START(cpu_arm926_reset) mov ip, #0 mcr p15, 0, ip, c7, c7, 0 @ invalidate I,D caches mcr p15, 0, ip, c7, c10, 4 @ drain WB @@ -77,7 +79,7 @@ ENTRY(cpu_arm926_reset) bic ip, ip, #0x1100 @ ...i...s........ mcr p15, 0, ip, c1, c0, 0 @ ctrl register ret r0 -ENDPROC(cpu_arm926_reset) +SYM_FUNC_END(cpu_arm926_reset) .popsection /* @@ -86,7 +88,7 @@ ENDPROC(cpu_arm926_reset) * Called with IRQs disabled */ .align 10 -ENTRY(cpu_arm926_do_idle) +SYM_TYPED_FUNC_START(cpu_arm926_do_idle) mov r0, #0 mrc p15, 0, r1, c1, c0, 0 @ Read control register mcr p15, 0, r0, c7, c10, 4 @ Drain write buffer @@ -99,6 +101,7 @@ ENTRY(cpu_arm926_do_idle) mcr p15, 0, r1, c1, c0, 0 @ Restore ICache enable msr cpsr_c, r3 @ Restore FIQ state ret lr +SYM_FUNC_END(cpu_arm926_do_idle) /* * flush_icache_all() @@ -329,7 +332,7 @@ SYM_TYPED_FUNC_START(arm926_dma_unmap_area) ret lr SYM_FUNC_END(arm926_dma_unmap_area) -ENTRY(cpu_arm926_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_arm926_dcache_clean_area) #ifndef CONFIG_CPU_DCACHE_WRITETHROUGH 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry add r0, r0, #CACHE_DLINESIZE @@ -338,6 +341,7 @@ ENTRY(cpu_arm926_dcache_clean_area) #endif mcr p15, 0, r0, c7, c10, 4 @ drain WB ret lr +SYM_FUNC_END(cpu_arm926_dcache_clean_area) /* =============================== PageTable ============================== */ @@ -349,7 +353,8 @@ ENTRY(cpu_arm926_dcache_clean_area) * pgd: new page tables */ .align 5 -ENTRY(cpu_arm926_switch_mm) + +SYM_TYPED_FUNC_START(cpu_arm926_switch_mm) #ifdef CONFIG_MMU mov ip, #0 #ifdef CONFIG_CPU_DCACHE_WRITETHROUGH @@ -365,6 +370,7 @@ ENTRY(cpu_arm926_switch_mm) mcr p15, 0, ip, c8, c7, 0 @ invalidate I & D TLBs #endif ret lr +SYM_FUNC_END(cpu_arm926_switch_mm) /* * cpu_arm926_set_pte_ext(ptep, pte, ext) @@ -372,7 +378,7 @@ ENTRY(cpu_arm926_switch_mm) * Set a PTE and flush it out */ .align 5 -ENTRY(cpu_arm926_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_arm926_set_pte_ext) #ifdef CONFIG_MMU armv3_set_pte_ext mov r0, r0 @@ -382,21 +388,22 @@ ENTRY(cpu_arm926_set_pte_ext) mcr p15, 0, r0, c7, c10, 4 @ drain WB #endif ret lr +SYM_FUNC_END(cpu_arm926_set_pte_ext) /* Suspend/resume support: taken from arch/arm/plat-s3c24xx/sleep.S */ .globl cpu_arm926_suspend_size .equ cpu_arm926_suspend_size, 4 * 3 #ifdef CONFIG_ARM_CPU_SUSPEND -ENTRY(cpu_arm926_do_suspend) +SYM_TYPED_FUNC_START(cpu_arm926_do_suspend) stmfd sp!, {r4 - r6, lr} mrc p15, 0, r4, c13, c0, 0 @ PID mrc p15, 0, r5, c3, c0, 0 @ Domain ID mrc p15, 0, r6, c1, c0, 0 @ Control register stmia r0, {r4 - r6} ldmfd sp!, {r4 - r6, pc} -ENDPROC(cpu_arm926_do_suspend) +SYM_FUNC_END(cpu_arm926_do_suspend) -ENTRY(cpu_arm926_do_resume) +SYM_TYPED_FUNC_START(cpu_arm926_do_resume) mov ip, #0 mcr p15, 0, ip, c8, c7, 0 @ invalidate I+D TLBs mcr p15, 0, ip, c7, c7, 0 @ invalidate I+D caches @@ -406,7 +413,7 @@ ENTRY(cpu_arm926_do_resume) mcr p15, 0, r1, c2, c0, 0 @ TTB address mov r0, r6 @ control register b cpu_resume_mmu -ENDPROC(cpu_arm926_do_resume) +SYM_FUNC_END(cpu_arm926_do_resume) #endif .type __arm926_setup, #function diff --git a/arch/arm/mm/proc-arm940.S b/arch/arm/mm/proc-arm940.S index 527f1c044683..cd95fca4656f 100644 --- a/arch/arm/mm/proc-arm940.S +++ b/arch/arm/mm/proc-arm940.S @@ -26,19 +26,24 @@ * * These are not required. */ -ENTRY(cpu_arm940_proc_init) -ENTRY(cpu_arm940_switch_mm) +SYM_TYPED_FUNC_START(cpu_arm940_proc_init) ret lr +SYM_FUNC_END(cpu_arm940_proc_init) + +SYM_TYPED_FUNC_START(cpu_arm940_switch_mm) + ret lr +SYM_FUNC_END(cpu_arm940_switch_mm) /* * cpu_arm940_proc_fin() */ -ENTRY(cpu_arm940_proc_fin) +SYM_TYPED_FUNC_START(cpu_arm940_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x00001000 @ i-cache bic r0, r0, #0x00000004 @ d-cache mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_arm940_proc_fin) /* * cpu_arm940_reset(loc) @@ -46,7 +51,7 @@ ENTRY(cpu_arm940_proc_fin) * Notes : This sets up everything for a reset */ .pushsection .idmap.text, "ax" -ENTRY(cpu_arm940_reset) +SYM_TYPED_FUNC_START(cpu_arm940_reset) mov ip, #0 mcr p15, 0, ip, c7, c5, 0 @ flush I cache mcr p15, 0, ip, c7, c6, 0 @ flush D cache @@ -56,16 +61,17 @@ ENTRY(cpu_arm940_reset) bic ip, ip, #0x00001000 @ i-cache mcr p15, 0, ip, c1, c0, 0 @ ctrl register ret r0 -ENDPROC(cpu_arm940_reset) +SYM_FUNC_END(cpu_arm940_reset) .popsection /* * cpu_arm940_do_idle() */ .align 5 -ENTRY(cpu_arm940_do_idle) +SYM_TYPED_FUNC_START(cpu_arm940_do_idle) mcr p15, 0, r0, c7, c0, 4 @ Wait for interrupt ret lr +SYM_FUNC_END(cpu_arm940_do_idle) /* * flush_icache_all() @@ -202,7 +208,7 @@ arm940_dma_inv_range: * - end - virtual end address */ arm940_dma_clean_range: -ENTRY(cpu_arm940_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_arm940_dcache_clean_area) mov ip, #0 #ifndef CONFIG_CPU_DCACHE_WRITETHROUGH mov r1, #(CACHE_DSEGMENTS - 1) << 4 @ 4 segments @@ -215,6 +221,7 @@ ENTRY(cpu_arm940_dcache_clean_area) #endif mcr p15, 0, ip, c7, c10, 4 @ drain WB ret lr +SYM_FUNC_END(cpu_arm940_dcache_clean_area) /* * dma_flush_range(start, end) diff --git a/arch/arm/mm/proc-arm946.S b/arch/arm/mm/proc-arm946.S index 3155e819ae5f..7df7c6e5598a 100644 --- a/arch/arm/mm/proc-arm946.S +++ b/arch/arm/mm/proc-arm946.S @@ -33,19 +33,24 @@ * * These are not required. */ -ENTRY(cpu_arm946_proc_init) -ENTRY(cpu_arm946_switch_mm) +SYM_TYPED_FUNC_START(cpu_arm946_proc_init) ret lr +SYM_FUNC_END(cpu_arm946_proc_init) + +SYM_TYPED_FUNC_START(cpu_arm946_switch_mm) + ret lr +SYM_FUNC_END(cpu_arm946_switch_mm) /* * cpu_arm946_proc_fin() */ -ENTRY(cpu_arm946_proc_fin) +SYM_TYPED_FUNC_START(cpu_arm946_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x00001000 @ i-cache bic r0, r0, #0x00000004 @ d-cache mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_arm946_proc_fin) /* * cpu_arm946_reset(loc) @@ -53,7 +58,7 @@ ENTRY(cpu_arm946_proc_fin) * Notes : This sets up everything for a reset */ .pushsection .idmap.text, "ax" -ENTRY(cpu_arm946_reset) +SYM_TYPED_FUNC_START(cpu_arm946_reset) mov ip, #0 mcr p15, 0, ip, c7, c5, 0 @ flush I cache mcr p15, 0, ip, c7, c6, 0 @ flush D cache @@ -63,16 +68,17 @@ ENTRY(cpu_arm946_reset) bic ip, ip, #0x00001000 @ i-cache mcr p15, 0, ip, c1, c0, 0 @ ctrl register ret r0 -ENDPROC(cpu_arm946_reset) +SYM_FUNC_END(cpu_arm946_reset) .popsection /* * cpu_arm946_do_idle() */ .align 5 -ENTRY(cpu_arm946_do_idle) +SYM_TYPED_FUNC_START(cpu_arm946_do_idle) mcr p15, 0, r0, c7, c0, 4 @ Wait for interrupt ret lr +SYM_FUNC_END(cpu_arm946_do_idle) /* * flush_icache_all() @@ -310,7 +316,7 @@ SYM_TYPED_FUNC_START(arm946_dma_unmap_area) ret lr SYM_FUNC_END(arm946_dma_unmap_area) -ENTRY(cpu_arm946_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_arm946_dcache_clean_area) #ifndef CONFIG_CPU_DCACHE_WRITETHROUGH 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry add r0, r0, #CACHE_DLINESIZE @@ -319,6 +325,7 @@ ENTRY(cpu_arm946_dcache_clean_area) #endif mcr p15, 0, r0, c7, c10, 4 @ drain WB ret lr +SYM_FUNC_END(cpu_arm946_dcache_clean_area) .type __arm946_setup, #function __arm946_setup: diff --git a/arch/arm/mm/proc-arm9tdmi.S b/arch/arm/mm/proc-arm9tdmi.S index a054c0e9c034..c480a8400eff 100644 --- a/arch/arm/mm/proc-arm9tdmi.S +++ b/arch/arm/mm/proc-arm9tdmi.S @@ -6,6 +6,7 @@ */ #include #include +#include #include #include #include @@ -24,17 +25,28 @@ * * These are not required. */ -ENTRY(cpu_arm9tdmi_proc_init) -ENTRY(cpu_arm9tdmi_do_idle) -ENTRY(cpu_arm9tdmi_dcache_clean_area) -ENTRY(cpu_arm9tdmi_switch_mm) +SYM_TYPED_FUNC_START(cpu_arm9tdmi_proc_init) ret lr +SYM_FUNC_END(cpu_arm9tdmi_proc_init) + +SYM_TYPED_FUNC_START(cpu_arm9tdmi_do_idle) + ret lr +SYM_FUNC_END(cpu_arm9tdmi_do_idle) + +SYM_TYPED_FUNC_START(cpu_arm9tdmi_dcache_clean_area) + ret lr +SYM_FUNC_END(cpu_arm9tdmi_dcache_clean_area) + +SYM_TYPED_FUNC_START(cpu_arm9tdmi_switch_mm) + ret lr +SYM_FUNC_END(cpu_arm9tdmi_switch_mm) /* * cpu_arm9tdmi_proc_fin() */ -ENTRY(cpu_arm9tdmi_proc_fin) +SYM_TYPED_FUNC_START(cpu_arm9tdmi_proc_fin) ret lr +SYM_FUNC_END(cpu_arm9tdmi_proc_fin) /* * Function: cpu_arm9tdmi_reset(loc) @@ -42,9 +54,9 @@ ENTRY(cpu_arm9tdmi_proc_fin) * Purpose : Sets up everything for a reset and jump to the location for soft reset. */ .pushsection .idmap.text, "ax" -ENTRY(cpu_arm9tdmi_reset) +SYM_TYPED_FUNC_START(cpu_arm9tdmi_reset) ret r0 -ENDPROC(cpu_arm9tdmi_reset) +SYM_FUNC_END(cpu_arm9tdmi_reset) .popsection .type __arm9tdmi_setup, #function diff --git a/arch/arm/mm/proc-fa526.S b/arch/arm/mm/proc-fa526.S index 2c73e0d47d08..7c16ccac8a05 100644 --- a/arch/arm/mm/proc-fa526.S +++ b/arch/arm/mm/proc-fa526.S @@ -11,6 +11,7 @@ */ #include #include +#include #include #include #include @@ -26,13 +27,14 @@ /* * cpu_fa526_proc_init() */ -ENTRY(cpu_fa526_proc_init) +SYM_TYPED_FUNC_START(cpu_fa526_proc_init) ret lr +SYM_FUNC_END(cpu_fa526_proc_init) /* * cpu_fa526_proc_fin() */ -ENTRY(cpu_fa526_proc_fin) +SYM_TYPED_FUNC_START(cpu_fa526_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x1000 @ ...i............ bic r0, r0, #0x000e @ ............wca. @@ -40,6 +42,7 @@ ENTRY(cpu_fa526_proc_fin) nop nop ret lr +SYM_FUNC_END(cpu_fa526_proc_fin) /* * cpu_fa526_reset(loc) @@ -52,7 +55,7 @@ ENTRY(cpu_fa526_proc_fin) */ .align 4 .pushsection .idmap.text, "ax" -ENTRY(cpu_fa526_reset) +SYM_TYPED_FUNC_START(cpu_fa526_reset) /* TODO: Use CP8 if possible... */ mov ip, #0 mcr p15, 0, ip, c7, c7, 0 @ invalidate I,D caches @@ -68,24 +71,25 @@ ENTRY(cpu_fa526_reset) nop nop ret r0 -ENDPROC(cpu_fa526_reset) +SYM_FUNC_END(cpu_fa526_reset) .popsection /* * cpu_fa526_do_idle() */ .align 4 -ENTRY(cpu_fa526_do_idle) +SYM_TYPED_FUNC_START(cpu_fa526_do_idle) ret lr +SYM_FUNC_END(cpu_fa526_do_idle) - -ENTRY(cpu_fa526_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_fa526_dcache_clean_area) 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry add r0, r0, #CACHE_DLINESIZE subs r1, r1, #CACHE_DLINESIZE bhi 1b mcr p15, 0, r0, c7, c10, 4 @ drain WB ret lr +SYM_FUNC_END(cpu_fa526_dcache_clean_area) /* =============================== PageTable ============================== */ @@ -97,7 +101,7 @@ ENTRY(cpu_fa526_dcache_clean_area) * pgd: new page tables */ .align 4 -ENTRY(cpu_fa526_switch_mm) +SYM_TYPED_FUNC_START(cpu_fa526_switch_mm) #ifdef CONFIG_MMU mov ip, #0 #ifdef CONFIG_CPU_DCACHE_WRITETHROUGH @@ -113,6 +117,7 @@ ENTRY(cpu_fa526_switch_mm) mcr p15, 0, ip, c8, c7, 0 @ invalidate UTLB #endif ret lr +SYM_FUNC_END(cpu_fa526_switch_mm) /* * cpu_fa526_set_pte_ext(ptep, pte, ext) @@ -120,7 +125,7 @@ ENTRY(cpu_fa526_switch_mm) * Set a PTE and flush it out */ .align 4 -ENTRY(cpu_fa526_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_fa526_set_pte_ext) #ifdef CONFIG_MMU armv3_set_pte_ext mov r0, r0 @@ -129,6 +134,7 @@ ENTRY(cpu_fa526_set_pte_ext) mcr p15, 0, r0, c7, c10, 4 @ drain WB #endif ret lr +SYM_FUNC_END(cpu_fa526_set_pte_ext) .type __fa526_setup, #function __fa526_setup: diff --git a/arch/arm/mm/proc-feroceon.S b/arch/arm/mm/proc-feroceon.S index af9482b07a4f..4c70eb0cc0d5 100644 --- a/arch/arm/mm/proc-feroceon.S +++ b/arch/arm/mm/proc-feroceon.S @@ -44,7 +44,7 @@ __cache_params: /* * cpu_feroceon_proc_init() */ -ENTRY(cpu_feroceon_proc_init) +SYM_TYPED_FUNC_START(cpu_feroceon_proc_init) mrc p15, 0, r0, c0, c0, 1 @ read cache type register ldr r1, __cache_params mov r2, #(16 << 5) @@ -62,11 +62,12 @@ ENTRY(cpu_feroceon_proc_init) str_l r1, VFP_arch_feroceon, r2 #endif ret lr +SYM_FUNC_END(cpu_feroceon_proc_init) /* * cpu_feroceon_proc_fin() */ -ENTRY(cpu_feroceon_proc_fin) +SYM_TYPED_FUNC_START(cpu_feroceon_proc_fin) #if defined(CONFIG_CACHE_FEROCEON_L2) && \ !defined(CONFIG_CACHE_FEROCEON_L2_WRITETHROUGH) mov r0, #0 @@ -79,6 +80,7 @@ ENTRY(cpu_feroceon_proc_fin) bic r0, r0, #0x000e @ ............wca. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_feroceon_proc_fin) /* * cpu_feroceon_reset(loc) @@ -91,7 +93,7 @@ ENTRY(cpu_feroceon_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_feroceon_reset) +SYM_TYPED_FUNC_START(cpu_feroceon_reset) mov ip, #0 mcr p15, 0, ip, c7, c7, 0 @ invalidate I,D caches mcr p15, 0, ip, c7, c10, 4 @ drain WB @@ -103,7 +105,7 @@ ENTRY(cpu_feroceon_reset) bic ip, ip, #0x1100 @ ...i...s........ mcr p15, 0, ip, c1, c0, 0 @ ctrl register ret r0 -ENDPROC(cpu_feroceon_reset) +SYM_FUNC_END(cpu_feroceon_reset) .popsection /* @@ -112,11 +114,12 @@ ENDPROC(cpu_feroceon_reset) * Called with IRQs disabled */ .align 5 -ENTRY(cpu_feroceon_do_idle) +SYM_TYPED_FUNC_START(cpu_feroceon_do_idle) mov r0, #0 mcr p15, 0, r0, c7, c10, 4 @ Drain write buffer mcr p15, 0, r0, c7, c0, 4 @ Wait for interrupt ret lr +SYM_FUNC_END(cpu_feroceon_do_idle) /* * flush_icache_all() @@ -413,7 +416,7 @@ SYM_TYPED_FUNC_START(feroceon_dma_unmap_area) SYM_FUNC_END(feroceon_dma_unmap_area) .align 5 -ENTRY(cpu_feroceon_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_feroceon_dcache_clean_area) #if defined(CONFIG_CACHE_FEROCEON_L2) && \ !defined(CONFIG_CACHE_FEROCEON_L2_WRITETHROUGH) mov r2, r0 @@ -432,6 +435,7 @@ ENTRY(cpu_feroceon_dcache_clean_area) #endif mcr p15, 0, r0, c7, c10, 4 @ drain WB ret lr +SYM_FUNC_END(cpu_feroceon_dcache_clean_area) /* =============================== PageTable ============================== */ @@ -443,7 +447,7 @@ ENTRY(cpu_feroceon_dcache_clean_area) * pgd: new page tables */ .align 5 -ENTRY(cpu_feroceon_switch_mm) +SYM_TYPED_FUNC_START(cpu_feroceon_switch_mm) #ifdef CONFIG_MMU /* * Note: we wish to call __flush_whole_cache but we need to preserve @@ -464,6 +468,7 @@ ENTRY(cpu_feroceon_switch_mm) #else ret lr #endif +SYM_FUNC_END(cpu_feroceon_switch_mm) /* * cpu_feroceon_set_pte_ext(ptep, pte, ext) @@ -471,7 +476,7 @@ ENTRY(cpu_feroceon_switch_mm) * Set a PTE and flush it out */ .align 5 -ENTRY(cpu_feroceon_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_feroceon_set_pte_ext) #ifdef CONFIG_MMU armv3_set_pte_ext wc_disable=0 mov r0, r0 @@ -483,21 +488,22 @@ ENTRY(cpu_feroceon_set_pte_ext) mcr p15, 0, r0, c7, c10, 4 @ drain WB #endif ret lr +SYM_FUNC_END(cpu_feroceon_set_pte_ext) /* Suspend/resume support: taken from arch/arm/mm/proc-arm926.S */ .globl cpu_feroceon_suspend_size .equ cpu_feroceon_suspend_size, 4 * 3 #ifdef CONFIG_ARM_CPU_SUSPEND -ENTRY(cpu_feroceon_do_suspend) +SYM_TYPED_FUNC_START(cpu_feroceon_do_suspend) stmfd sp!, {r4 - r6, lr} mrc p15, 0, r4, c13, c0, 0 @ PID mrc p15, 0, r5, c3, c0, 0 @ Domain ID mrc p15, 0, r6, c1, c0, 0 @ Control register stmia r0, {r4 - r6} ldmfd sp!, {r4 - r6, pc} -ENDPROC(cpu_feroceon_do_suspend) +SYM_FUNC_END(cpu_feroceon_do_suspend) -ENTRY(cpu_feroceon_do_resume) +SYM_TYPED_FUNC_START(cpu_feroceon_do_resume) mov ip, #0 mcr p15, 0, ip, c8, c7, 0 @ invalidate I+D TLBs mcr p15, 0, ip, c7, c7, 0 @ invalidate I+D caches @@ -507,7 +513,7 @@ ENTRY(cpu_feroceon_do_resume) mcr p15, 0, r1, c2, c0, 0 @ TTB address mov r0, r6 @ control register b cpu_resume_mmu -ENDPROC(cpu_feroceon_do_resume) +SYM_FUNC_END(cpu_feroceon_do_resume) #endif .type __feroceon_setup, #function diff --git a/arch/arm/mm/proc-mohawk.S b/arch/arm/mm/proc-mohawk.S index be3a1a997838..6871395958ce 100644 --- a/arch/arm/mm/proc-mohawk.S +++ b/arch/arm/mm/proc-mohawk.S @@ -32,18 +32,20 @@ /* * cpu_mohawk_proc_init() */ -ENTRY(cpu_mohawk_proc_init) +SYM_TYPED_FUNC_START(cpu_mohawk_proc_init) ret lr +SYM_FUNC_END(cpu_mohawk_proc_init) /* * cpu_mohawk_proc_fin() */ -ENTRY(cpu_mohawk_proc_fin) +SYM_TYPED_FUNC_START(cpu_mohawk_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x1800 @ ...iz........... bic r0, r0, #0x0006 @ .............ca. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_mohawk_proc_fin) /* * cpu_mohawk_reset(loc) @@ -58,7 +60,7 @@ ENTRY(cpu_mohawk_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_mohawk_reset) +SYM_TYPED_FUNC_START(cpu_mohawk_reset) mov ip, #0 mcr p15, 0, ip, c7, c7, 0 @ invalidate I,D caches mcr p15, 0, ip, c7, c10, 4 @ drain WB @@ -68,7 +70,7 @@ ENTRY(cpu_mohawk_reset) bic ip, ip, #0x1100 @ ...i...s........ mcr p15, 0, ip, c1, c0, 0 @ ctrl register ret r0 -ENDPROC(cpu_mohawk_reset) +SYM_FUNC_END(cpu_mohawk_reset) .popsection /* @@ -77,11 +79,12 @@ ENDPROC(cpu_mohawk_reset) * Called with IRQs disabled */ .align 5 -ENTRY(cpu_mohawk_do_idle) +SYM_TYPED_FUNC_START(cpu_mohawk_do_idle) mov r0, #0 mcr p15, 0, r0, c7, c10, 4 @ drain write buffer mcr p15, 0, r0, c7, c0, 4 @ wait for interrupt ret lr +SYM_FUNC_END(cpu_mohawk_do_idle) /* * flush_icache_all() @@ -294,13 +297,14 @@ SYM_TYPED_FUNC_START(mohawk_dma_unmap_area) ret lr SYM_FUNC_END(mohawk_dma_unmap_area) -ENTRY(cpu_mohawk_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_mohawk_dcache_clean_area) 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry add r0, r0, #CACHE_DLINESIZE subs r1, r1, #CACHE_DLINESIZE bhi 1b mcr p15, 0, r0, c7, c10, 4 @ drain WB ret lr +SYM_FUNC_END(cpu_mohawk_dcache_clean_area) /* * cpu_mohawk_switch_mm(pgd) @@ -310,7 +314,7 @@ ENTRY(cpu_mohawk_dcache_clean_area) * pgd: new page tables */ .align 5 -ENTRY(cpu_mohawk_switch_mm) +SYM_TYPED_FUNC_START(cpu_mohawk_switch_mm) mov ip, #0 mcr p15, 0, ip, c7, c14, 0 @ clean & invalidate all D cache mcr p15, 0, ip, c7, c5, 0 @ invalidate I cache @@ -319,6 +323,7 @@ ENTRY(cpu_mohawk_switch_mm) mcr p15, 0, r0, c2, c0, 0 @ load page table pointer mcr p15, 0, ip, c8, c7, 0 @ invalidate I & D TLBs ret lr +SYM_FUNC_END(cpu_mohawk_switch_mm) /* * cpu_mohawk_set_pte_ext(ptep, pte, ext) @@ -326,7 +331,7 @@ ENTRY(cpu_mohawk_switch_mm) * Set a PTE and flush it out */ .align 5 -ENTRY(cpu_mohawk_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_mohawk_set_pte_ext) #ifdef CONFIG_MMU armv3_set_pte_ext mov r0, r0 @@ -334,11 +339,12 @@ ENTRY(cpu_mohawk_set_pte_ext) mcr p15, 0, r0, c7, c10, 4 @ drain WB ret lr #endif +SYM_FUNC_END(cpu_mohawk_set_pte_ext) .globl cpu_mohawk_suspend_size .equ cpu_mohawk_suspend_size, 4 * 6 #ifdef CONFIG_ARM_CPU_SUSPEND -ENTRY(cpu_mohawk_do_suspend) +SYM_TYPED_FUNC_START(cpu_mohawk_do_suspend) stmfd sp!, {r4 - r9, lr} mrc p14, 0, r4, c6, c0, 0 @ clock configuration, for turbo mode mrc p15, 0, r5, c15, c1, 0 @ CP access reg @@ -349,9 +355,9 @@ ENTRY(cpu_mohawk_do_suspend) bic r4, r4, #2 @ clear frequency change bit stmia r0, {r4 - r9} @ store cp regs ldmia sp!, {r4 - r9, pc} -ENDPROC(cpu_mohawk_do_suspend) +SYM_FUNC_END(cpu_mohawk_do_suspend) -ENTRY(cpu_mohawk_do_resume) +SYM_TYPED_FUNC_START(cpu_mohawk_do_resume) ldmia r0, {r4 - r9} @ load cp regs mov ip, #0 mcr p15, 0, ip, c7, c7, 0 @ invalidate I & D caches, BTB @@ -367,7 +373,7 @@ ENTRY(cpu_mohawk_do_resume) mcr p15, 0, r8, c1, c0, 1 @ auxiliary control reg mov r0, r9 @ control register b cpu_resume_mmu -ENDPROC(cpu_mohawk_do_resume) +SYM_FUNC_END(cpu_mohawk_do_resume) #endif .type __mohawk_setup, #function diff --git a/arch/arm/mm/proc-sa110.S b/arch/arm/mm/proc-sa110.S index 4071f7a61cb6..3da76fab8ac3 100644 --- a/arch/arm/mm/proc-sa110.S +++ b/arch/arm/mm/proc-sa110.S @@ -12,6 +12,7 @@ */ #include #include +#include #include #include #include @@ -32,15 +33,16 @@ /* * cpu_sa110_proc_init() */ -ENTRY(cpu_sa110_proc_init) +SYM_TYPED_FUNC_START(cpu_sa110_proc_init) mov r0, #0 mcr p15, 0, r0, c15, c1, 2 @ Enable clock switching ret lr +SYM_FUNC_END(cpu_sa110_proc_init) /* * cpu_sa110_proc_fin() */ -ENTRY(cpu_sa110_proc_fin) +SYM_TYPED_FUNC_START(cpu_sa110_proc_fin) mov r0, #0 mcr p15, 0, r0, c15, c2, 2 @ Disable clock switching mrc p15, 0, r0, c1, c0, 0 @ ctrl register @@ -48,6 +50,7 @@ ENTRY(cpu_sa110_proc_fin) bic r0, r0, #0x000e @ ............wca. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_sa110_proc_fin) /* * cpu_sa110_reset(loc) @@ -60,7 +63,7 @@ ENTRY(cpu_sa110_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_sa110_reset) +SYM_TYPED_FUNC_START(cpu_sa110_reset) mov ip, #0 mcr p15, 0, ip, c7, c7, 0 @ invalidate I,D caches mcr p15, 0, ip, c7, c10, 4 @ drain WB @@ -72,7 +75,7 @@ ENTRY(cpu_sa110_reset) bic ip, ip, #0x1100 @ ...i...s........ mcr p15, 0, ip, c1, c0, 0 @ ctrl register ret r0 -ENDPROC(cpu_sa110_reset) +SYM_FUNC_END(cpu_sa110_reset) .popsection /* @@ -88,7 +91,7 @@ ENDPROC(cpu_sa110_reset) */ .align 5 -ENTRY(cpu_sa110_do_idle) +SYM_TYPED_FUNC_START(cpu_sa110_do_idle) mcr p15, 0, ip, c15, c2, 2 @ disable clock switching ldr r1, =UNCACHEABLE_ADDR @ load from uncacheable loc ldr r1, [r1, #0] @ force switch to MCLK @@ -101,6 +104,7 @@ ENTRY(cpu_sa110_do_idle) mov r0, r0 @ safety mcr p15, 0, r0, c15, c1, 2 @ enable clock switching ret lr +SYM_FUNC_END(cpu_sa110_do_idle) /* ================================= CACHE ================================ */ @@ -113,12 +117,13 @@ ENTRY(cpu_sa110_do_idle) * addr: cache-unaligned virtual address */ .align 5 -ENTRY(cpu_sa110_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_sa110_dcache_clean_area) 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry add r0, r0, #DCACHELINESIZE subs r1, r1, #DCACHELINESIZE bhi 1b ret lr +SYM_FUNC_END(cpu_sa110_dcache_clean_area) /* =============================== PageTable ============================== */ @@ -130,7 +135,7 @@ ENTRY(cpu_sa110_dcache_clean_area) * pgd: new page tables */ .align 5 -ENTRY(cpu_sa110_switch_mm) +SYM_TYPED_FUNC_START(cpu_sa110_switch_mm) #ifdef CONFIG_MMU str lr, [sp, #-4]! bl v4wb_flush_kern_cache_all @ clears IP @@ -140,6 +145,7 @@ ENTRY(cpu_sa110_switch_mm) #else ret lr #endif +SYM_FUNC_END(cpu_sa110_switch_mm) /* * cpu_sa110_set_pte_ext(ptep, pte, ext) @@ -147,7 +153,7 @@ ENTRY(cpu_sa110_switch_mm) * Set a PTE and flush it out */ .align 5 -ENTRY(cpu_sa110_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_sa110_set_pte_ext) #ifdef CONFIG_MMU armv3_set_pte_ext wc_disable=0 mov r0, r0 @@ -155,6 +161,7 @@ ENTRY(cpu_sa110_set_pte_ext) mcr p15, 0, r0, c7, c10, 4 @ drain WB #endif ret lr +SYM_FUNC_END(cpu_sa110_set_pte_ext) .type __sa110_setup, #function __sa110_setup: diff --git a/arch/arm/mm/proc-sa1100.S b/arch/arm/mm/proc-sa1100.S index e723bd4119d3..7c496195e440 100644 --- a/arch/arm/mm/proc-sa1100.S +++ b/arch/arm/mm/proc-sa1100.S @@ -17,6 +17,7 @@ */ #include #include +#include #include #include #include @@ -36,11 +37,12 @@ /* * cpu_sa1100_proc_init() */ -ENTRY(cpu_sa1100_proc_init) +SYM_TYPED_FUNC_START(cpu_sa1100_proc_init) mov r0, #0 mcr p15, 0, r0, c15, c1, 2 @ Enable clock switching mcr p15, 0, r0, c9, c0, 5 @ Allow read-buffer operations from userland ret lr +SYM_FUNC_END(cpu_sa1100_proc_init) /* * cpu_sa1100_proc_fin() @@ -49,13 +51,14 @@ ENTRY(cpu_sa1100_proc_init) * - Disable interrupts * - Clean and turn off caches. */ -ENTRY(cpu_sa1100_proc_fin) +SYM_TYPED_FUNC_START(cpu_sa1100_proc_fin) mcr p15, 0, ip, c15, c2, 2 @ Disable clock switching mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x1000 @ ...i............ bic r0, r0, #0x000e @ ............wca. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_sa1100_proc_fin) /* * cpu_sa1100_reset(loc) @@ -68,7 +71,7 @@ ENTRY(cpu_sa1100_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_sa1100_reset) +SYM_TYPED_FUNC_START(cpu_sa1100_reset) mov ip, #0 mcr p15, 0, ip, c7, c7, 0 @ invalidate I,D caches mcr p15, 0, ip, c7, c10, 4 @ drain WB @@ -80,7 +83,7 @@ ENTRY(cpu_sa1100_reset) bic ip, ip, #0x1100 @ ...i...s........ mcr p15, 0, ip, c1, c0, 0 @ ctrl register ret r0 -ENDPROC(cpu_sa1100_reset) +SYM_FUNC_END(cpu_sa1100_reset) .popsection /* @@ -95,7 +98,7 @@ ENDPROC(cpu_sa1100_reset) * 3 = switch to fast processor clock */ .align 5 -ENTRY(cpu_sa1100_do_idle) +SYM_TYPED_FUNC_START(cpu_sa1100_do_idle) mov r0, r0 @ 4 nop padding mov r0, r0 mov r0, r0 @@ -111,6 +114,7 @@ ENTRY(cpu_sa1100_do_idle) mov r0, r0 @ safety mcr p15, 0, r0, c15, c1, 2 @ enable clock switching ret lr +SYM_FUNC_END(cpu_sa1100_do_idle) /* ================================= CACHE ================================ */ @@ -123,12 +127,13 @@ ENTRY(cpu_sa1100_do_idle) * addr: cache-unaligned virtual address */ .align 5 -ENTRY(cpu_sa1100_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_sa1100_dcache_clean_area) 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry add r0, r0, #DCACHELINESIZE subs r1, r1, #DCACHELINESIZE bhi 1b ret lr +SYM_FUNC_END(cpu_sa1100_dcache_clean_area) /* =============================== PageTable ============================== */ @@ -140,7 +145,7 @@ ENTRY(cpu_sa1100_dcache_clean_area) * pgd: new page tables */ .align 5 -ENTRY(cpu_sa1100_switch_mm) +SYM_TYPED_FUNC_START(cpu_sa1100_switch_mm) #ifdef CONFIG_MMU str lr, [sp, #-4]! bl v4wb_flush_kern_cache_all @ clears IP @@ -151,6 +156,7 @@ ENTRY(cpu_sa1100_switch_mm) #else ret lr #endif +SYM_FUNC_END(cpu_sa1100_switch_mm) /* * cpu_sa1100_set_pte_ext(ptep, pte, ext) @@ -158,7 +164,7 @@ ENTRY(cpu_sa1100_switch_mm) * Set a PTE and flush it out */ .align 5 -ENTRY(cpu_sa1100_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_sa1100_set_pte_ext) #ifdef CONFIG_MMU armv3_set_pte_ext wc_disable=0 mov r0, r0 @@ -166,20 +172,21 @@ ENTRY(cpu_sa1100_set_pte_ext) mcr p15, 0, r0, c7, c10, 4 @ drain WB #endif ret lr +SYM_FUNC_END(cpu_sa1100_set_pte_ext) .globl cpu_sa1100_suspend_size .equ cpu_sa1100_suspend_size, 4 * 3 #ifdef CONFIG_ARM_CPU_SUSPEND -ENTRY(cpu_sa1100_do_suspend) +SYM_TYPED_FUNC_START(cpu_sa1100_do_suspend) stmfd sp!, {r4 - r6, lr} mrc p15, 0, r4, c3, c0, 0 @ domain ID mrc p15, 0, r5, c13, c0, 0 @ PID mrc p15, 0, r6, c1, c0, 0 @ control reg stmia r0, {r4 - r6} @ store cp regs ldmfd sp!, {r4 - r6, pc} -ENDPROC(cpu_sa1100_do_suspend) +SYM_FUNC_END(cpu_sa1100_do_suspend) -ENTRY(cpu_sa1100_do_resume) +SYM_TYPED_FUNC_START(cpu_sa1100_do_resume) ldmia r0, {r4 - r6} @ load cp regs mov ip, #0 mcr p15, 0, ip, c8, c7, 0 @ flush I+D TLBs @@ -192,7 +199,7 @@ ENTRY(cpu_sa1100_do_resume) mcr p15, 0, r5, c13, c0, 0 @ PID mov r0, r6 @ control register b cpu_resume_mmu -ENDPROC(cpu_sa1100_do_resume) +SYM_FUNC_END(cpu_sa1100_do_resume) #endif .type __sa1100_setup, #function diff --git a/arch/arm/mm/proc-v6.S b/arch/arm/mm/proc-v6.S index 203dff89ab1a..90a01f5950b9 100644 --- a/arch/arm/mm/proc-v6.S +++ b/arch/arm/mm/proc-v6.S @@ -8,6 +8,7 @@ * This is the "shell" of the ARMv6 processor support. */ #include +#include #include #include #include @@ -34,15 +35,17 @@ .arch armv6 -ENTRY(cpu_v6_proc_init) +SYM_TYPED_FUNC_START(cpu_v6_proc_init) ret lr +SYM_FUNC_END(cpu_v6_proc_init) -ENTRY(cpu_v6_proc_fin) +SYM_TYPED_FUNC_START(cpu_v6_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x1000 @ ...i............ bic r0, r0, #0x0006 @ .............ca. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_v6_proc_fin) /* * cpu_v6_reset(loc) @@ -55,14 +58,14 @@ ENTRY(cpu_v6_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_v6_reset) +SYM_TYPED_FUNC_START(cpu_v6_reset) mrc p15, 0, r1, c1, c0, 0 @ ctrl register bic r1, r1, #0x1 @ ...............m mcr p15, 0, r1, c1, c0, 0 @ disable MMU mov r1, #0 mcr p15, 0, r1, c7, c5, 4 @ ISB ret r0 -ENDPROC(cpu_v6_reset) +SYM_FUNC_END(cpu_v6_reset) .popsection /* @@ -72,18 +75,20 @@ ENDPROC(cpu_v6_reset) * * IRQs are already disabled. */ -ENTRY(cpu_v6_do_idle) +SYM_TYPED_FUNC_START(cpu_v6_do_idle) mov r1, #0 mcr p15, 0, r1, c7, c10, 4 @ DWB - WFI may enter a low-power mode mcr p15, 0, r1, c7, c0, 4 @ wait for interrupt ret lr +SYM_FUNC_END(cpu_v6_do_idle) -ENTRY(cpu_v6_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_v6_dcache_clean_area) 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry add r0, r0, #D_CACHE_LINE_SIZE subs r1, r1, #D_CACHE_LINE_SIZE bhi 1b ret lr +SYM_FUNC_END(cpu_v6_dcache_clean_area) /* * cpu_v6_switch_mm(pgd_phys, tsk) @@ -95,7 +100,7 @@ ENTRY(cpu_v6_dcache_clean_area) * It is assumed that: * - we are not using split page tables */ -ENTRY(cpu_v6_switch_mm) +SYM_TYPED_FUNC_START(cpu_v6_switch_mm) #ifdef CONFIG_MMU mov r2, #0 mmid r1, r1 @ get mm->context.id @@ -113,6 +118,7 @@ ENTRY(cpu_v6_switch_mm) mcr p15, 0, r1, c13, c0, 1 @ set context ID #endif ret lr +SYM_FUNC_END(cpu_v6_switch_mm) /* * cpu_v6_set_pte_ext(ptep, pte, ext) @@ -126,17 +132,18 @@ ENTRY(cpu_v6_switch_mm) */ armv6_mt_table cpu_v6 -ENTRY(cpu_v6_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_v6_set_pte_ext) #ifdef CONFIG_MMU armv6_set_pte_ext cpu_v6 #endif ret lr +SYM_FUNC_END(cpu_v6_set_pte_ext) /* Suspend/resume support: taken from arch/arm/mach-s3c64xx/sleep.S */ .globl cpu_v6_suspend_size .equ cpu_v6_suspend_size, 4 * 6 #ifdef CONFIG_ARM_CPU_SUSPEND -ENTRY(cpu_v6_do_suspend) +SYM_TYPED_FUNC_START(cpu_v6_do_suspend) stmfd sp!, {r4 - r9, lr} mrc p15, 0, r4, c13, c0, 0 @ FCSE/PID #ifdef CONFIG_MMU @@ -148,9 +155,9 @@ ENTRY(cpu_v6_do_suspend) mrc p15, 0, r9, c1, c0, 0 @ control register stmia r0, {r4 - r9} ldmfd sp!, {r4- r9, pc} -ENDPROC(cpu_v6_do_suspend) +SYM_FUNC_END(cpu_v6_do_suspend) -ENTRY(cpu_v6_do_resume) +SYM_TYPED_FUNC_START(cpu_v6_do_resume) mov ip, #0 mcr p15, 0, ip, c7, c14, 0 @ clean+invalidate D cache mcr p15, 0, ip, c7, c5, 0 @ invalidate I cache @@ -172,7 +179,7 @@ ENTRY(cpu_v6_do_resume) mcr p15, 0, ip, c7, c5, 4 @ ISB mov r0, r9 @ control register b cpu_resume_mmu -ENDPROC(cpu_v6_do_resume) +SYM_FUNC_END(cpu_v6_do_resume) #endif string cpu_v6_name, "ARMv6-compatible processor" diff --git a/arch/arm/mm/proc-v7-2level.S b/arch/arm/mm/proc-v7-2level.S index 0a3083ad19c2..1007702fcaf3 100644 --- a/arch/arm/mm/proc-v7-2level.S +++ b/arch/arm/mm/proc-v7-2level.S @@ -40,7 +40,7 @@ * even on Cortex-A8 revisions not affected by 430973. * If IBE is not set, the flush BTAC/BTB won't do anything. */ -ENTRY(cpu_v7_switch_mm) +SYM_TYPED_FUNC_START(cpu_v7_switch_mm) #ifdef CONFIG_MMU mmid r1, r1 @ get mm->context.id ALT_SMP(orr r0, r0, #TTB_FLAGS_SMP) @@ -59,7 +59,7 @@ ENTRY(cpu_v7_switch_mm) isb #endif bx lr -ENDPROC(cpu_v7_switch_mm) +SYM_FUNC_END(cpu_v7_switch_mm) /* * cpu_v7_set_pte_ext(ptep, pte) @@ -71,7 +71,7 @@ ENDPROC(cpu_v7_switch_mm) * - pte - PTE value to store * - ext - value for extended PTE bits */ -ENTRY(cpu_v7_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_v7_set_pte_ext) #ifdef CONFIG_MMU str r1, [r0] @ linux version @@ -106,7 +106,7 @@ ENTRY(cpu_v7_set_pte_ext) ALT_UP (mcr p15, 0, r0, c7, c10, 1) @ flush_pte #endif bx lr -ENDPROC(cpu_v7_set_pte_ext) +SYM_FUNC_END(cpu_v7_set_pte_ext) /* * Memory region attributes with SCTLR.TRE=1 diff --git a/arch/arm/mm/proc-v7-3level.S b/arch/arm/mm/proc-v7-3level.S index 131984462d0d..bdabc15cde56 100644 --- a/arch/arm/mm/proc-v7-3level.S +++ b/arch/arm/mm/proc-v7-3level.S @@ -42,7 +42,7 @@ * Set the translation table base pointer to be pgd_phys (physical address of * the new TTB). */ -ENTRY(cpu_v7_switch_mm) +SYM_TYPED_FUNC_START(cpu_v7_switch_mm) #ifdef CONFIG_MMU mmid r2, r2 asid r2, r2 @@ -51,7 +51,7 @@ ENTRY(cpu_v7_switch_mm) isb #endif ret lr -ENDPROC(cpu_v7_switch_mm) +SYM_FUNC_END(cpu_v7_switch_mm) #ifdef __ARMEB__ #define rl r3 @@ -68,7 +68,7 @@ ENDPROC(cpu_v7_switch_mm) * - ptep - pointer to level 3 translation table entry * - pte - PTE value to store (64-bit in r2 and r3) */ -ENTRY(cpu_v7_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_v7_set_pte_ext) #ifdef CONFIG_MMU tst rl, #L_PTE_VALID beq 1f @@ -87,7 +87,7 @@ ENTRY(cpu_v7_set_pte_ext) ALT_UP (mcr p15, 0, r0, c7, c10, 1) @ flush_pte #endif ret lr -ENDPROC(cpu_v7_set_pte_ext) +SYM_FUNC_END(cpu_v7_set_pte_ext) /* * Memory region attributes for LPAE (defined in pgtable-3level.h): diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S index 193c7aeb6703..5fb9a6aecb00 100644 --- a/arch/arm/mm/proc-v7.S +++ b/arch/arm/mm/proc-v7.S @@ -7,6 +7,7 @@ * This is the "shell" of the ARMv7 processor support. */ #include +#include #include #include #include @@ -26,17 +27,17 @@ .arch armv7-a -ENTRY(cpu_v7_proc_init) +SYM_TYPED_FUNC_START(cpu_v7_proc_init) ret lr -ENDPROC(cpu_v7_proc_init) +SYM_FUNC_END(cpu_v7_proc_init) -ENTRY(cpu_v7_proc_fin) +SYM_TYPED_FUNC_START(cpu_v7_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x1000 @ ...i............ bic r0, r0, #0x0006 @ .............ca. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr -ENDPROC(cpu_v7_proc_fin) +SYM_FUNC_END(cpu_v7_proc_fin) /* * cpu_v7_reset(loc, hyp) @@ -53,7 +54,7 @@ ENDPROC(cpu_v7_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_v7_reset) +SYM_TYPED_FUNC_START(cpu_v7_reset) mrc p15, 0, r2, c1, c0, 0 @ ctrl register bic r2, r2, #0x1 @ ...............m THUMB( bic r2, r2, #1 << 30 ) @ SCTLR.TE (Thumb exceptions) @@ -64,7 +65,7 @@ ENTRY(cpu_v7_reset) bne __hyp_soft_restart #endif bx r0 -ENDPROC(cpu_v7_reset) +SYM_FUNC_END(cpu_v7_reset) .popsection /* @@ -74,13 +75,13 @@ ENDPROC(cpu_v7_reset) * * IRQs are already disabled. */ -ENTRY(cpu_v7_do_idle) +SYM_TYPED_FUNC_START(cpu_v7_do_idle) dsb @ WFI may enter a low-power mode wfi ret lr -ENDPROC(cpu_v7_do_idle) +SYM_FUNC_END(cpu_v7_do_idle) -ENTRY(cpu_v7_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_v7_dcache_clean_area) ALT_SMP(W(nop)) @ MP extensions imply L1 PTW ALT_UP_B(1f) ret lr @@ -91,38 +92,39 @@ ENTRY(cpu_v7_dcache_clean_area) bhi 2b dsb ishst ret lr -ENDPROC(cpu_v7_dcache_clean_area) +SYM_FUNC_END(cpu_v7_dcache_clean_area) #ifdef CONFIG_ARM_PSCI .arch_extension sec -ENTRY(cpu_v7_smc_switch_mm) +SYM_TYPED_FUNC_START(cpu_v7_smc_switch_mm) stmfd sp!, {r0 - r3} movw r0, #:lower16:ARM_SMCCC_ARCH_WORKAROUND_1 movt r0, #:upper16:ARM_SMCCC_ARCH_WORKAROUND_1 smc #0 ldmfd sp!, {r0 - r3} b cpu_v7_switch_mm -ENDPROC(cpu_v7_smc_switch_mm) +SYM_FUNC_END(cpu_v7_smc_switch_mm) .arch_extension virt -ENTRY(cpu_v7_hvc_switch_mm) +SYM_TYPED_FUNC_START(cpu_v7_hvc_switch_mm) stmfd sp!, {r0 - r3} movw r0, #:lower16:ARM_SMCCC_ARCH_WORKAROUND_1 movt r0, #:upper16:ARM_SMCCC_ARCH_WORKAROUND_1 hvc #0 ldmfd sp!, {r0 - r3} b cpu_v7_switch_mm -ENDPROC(cpu_v7_hvc_switch_mm) +SYM_FUNC_END(cpu_v7_hvc_switch_mm) #endif -ENTRY(cpu_v7_iciallu_switch_mm) + +SYM_TYPED_FUNC_START(cpu_v7_iciallu_switch_mm) mov r3, #0 mcr p15, 0, r3, c7, c5, 0 @ ICIALLU b cpu_v7_switch_mm -ENDPROC(cpu_v7_iciallu_switch_mm) -ENTRY(cpu_v7_bpiall_switch_mm) +SYM_FUNC_END(cpu_v7_iciallu_switch_mm) +SYM_TYPED_FUNC_START(cpu_v7_bpiall_switch_mm) mov r3, #0 mcr p15, 0, r3, c7, c5, 6 @ flush BTAC/BTB b cpu_v7_switch_mm -ENDPROC(cpu_v7_bpiall_switch_mm) +SYM_FUNC_END(cpu_v7_bpiall_switch_mm) string cpu_v7_name, "ARMv7 Processor" .align @@ -131,7 +133,7 @@ ENDPROC(cpu_v7_bpiall_switch_mm) .globl cpu_v7_suspend_size .equ cpu_v7_suspend_size, 4 * 9 #ifdef CONFIG_ARM_CPU_SUSPEND -ENTRY(cpu_v7_do_suspend) +SYM_TYPED_FUNC_START(cpu_v7_do_suspend) stmfd sp!, {r4 - r11, lr} mrc p15, 0, r4, c13, c0, 0 @ FCSE/PID mrc p15, 0, r5, c13, c0, 3 @ User r/o thread ID @@ -150,9 +152,9 @@ ENTRY(cpu_v7_do_suspend) mrc p15, 0, r10, c1, c0, 2 @ Co-processor access control stmia r0, {r5 - r11} ldmfd sp!, {r4 - r11, pc} -ENDPROC(cpu_v7_do_suspend) +SYM_FUNC_END(cpu_v7_do_suspend) -ENTRY(cpu_v7_do_resume) +SYM_TYPED_FUNC_START(cpu_v7_do_resume) mov ip, #0 mcr p15, 0, ip, c7, c5, 0 @ invalidate I cache mcr p15, 0, ip, c13, c0, 1 @ set reserved context ID @@ -186,22 +188,22 @@ ENTRY(cpu_v7_do_resume) dsb mov r0, r8 @ control register b cpu_resume_mmu -ENDPROC(cpu_v7_do_resume) +SYM_FUNC_END(cpu_v7_do_resume) #endif .globl cpu_ca9mp_suspend_size .equ cpu_ca9mp_suspend_size, cpu_v7_suspend_size + 4 * 2 #ifdef CONFIG_ARM_CPU_SUSPEND -ENTRY(cpu_ca9mp_do_suspend) +SYM_TYPED_FUNC_START(cpu_ca9mp_do_suspend) stmfd sp!, {r4 - r5} mrc p15, 0, r4, c15, c0, 1 @ Diagnostic register mrc p15, 0, r5, c15, c0, 0 @ Power register stmia r0!, {r4 - r5} ldmfd sp!, {r4 - r5} b cpu_v7_do_suspend -ENDPROC(cpu_ca9mp_do_suspend) +SYM_FUNC_END(cpu_ca9mp_do_suspend) -ENTRY(cpu_ca9mp_do_resume) +SYM_TYPED_FUNC_START(cpu_ca9mp_do_resume) ldmia r0!, {r4 - r5} mrc p15, 0, r10, c15, c0, 1 @ Read Diagnostic register teq r4, r10 @ Already restored? @@ -210,7 +212,7 @@ ENTRY(cpu_ca9mp_do_resume) teq r5, r10 @ Already restored? mcrne p15, 0, r5, c15, c0, 0 @ No, so restore it b cpu_v7_do_resume -ENDPROC(cpu_ca9mp_do_resume) +SYM_FUNC_END(cpu_ca9mp_do_resume) #endif #ifdef CONFIG_CPU_PJ4B @@ -220,18 +222,18 @@ ENDPROC(cpu_ca9mp_do_resume) globl_equ cpu_pj4b_proc_fin, cpu_v7_proc_fin globl_equ cpu_pj4b_reset, cpu_v7_reset #ifdef CONFIG_PJ4B_ERRATA_4742 -ENTRY(cpu_pj4b_do_idle) +SYM_TYPED_FUNC_START(cpu_pj4b_do_idle) dsb @ WFI may enter a low-power mode wfi dsb @barrier ret lr -ENDPROC(cpu_pj4b_do_idle) +SYM_FUNC_END(cpu_pj4b_do_idle) #else globl_equ cpu_pj4b_do_idle, cpu_v7_do_idle #endif globl_equ cpu_pj4b_dcache_clean_area, cpu_v7_dcache_clean_area #ifdef CONFIG_ARM_CPU_SUSPEND -ENTRY(cpu_pj4b_do_suspend) +SYM_TYPED_FUNC_START(cpu_pj4b_do_suspend) stmfd sp!, {r6 - r10} mrc p15, 1, r6, c15, c1, 0 @ save CP15 - extra features mrc p15, 1, r7, c15, c2, 0 @ save CP15 - Aux Func Modes Ctrl 0 @@ -241,9 +243,9 @@ ENTRY(cpu_pj4b_do_suspend) stmia r0!, {r6 - r10} ldmfd sp!, {r6 - r10} b cpu_v7_do_suspend -ENDPROC(cpu_pj4b_do_suspend) +SYM_FUNC_END(cpu_pj4b_do_suspend) -ENTRY(cpu_pj4b_do_resume) +SYM_TYPED_FUNC_START(cpu_pj4b_do_resume) ldmia r0!, {r6 - r10} mcr p15, 1, r6, c15, c1, 0 @ restore CP15 - extra features mcr p15, 1, r7, c15, c2, 0 @ restore CP15 - Aux Func Modes Ctrl 0 @@ -251,7 +253,7 @@ ENTRY(cpu_pj4b_do_resume) mcr p15, 1, r9, c15, c1, 1 @ restore CP15 - Aux Debug Modes Ctrl 1 mcr p15, 0, r10, c9, c14, 0 @ restore CP15 - PMC b cpu_v7_do_resume -ENDPROC(cpu_pj4b_do_resume) +SYM_FUNC_END(cpu_pj4b_do_resume) #endif .globl cpu_pj4b_suspend_size .equ cpu_pj4b_suspend_size, cpu_v7_suspend_size + 4 * 5 diff --git a/arch/arm/mm/proc-v7m.S b/arch/arm/mm/proc-v7m.S index d65a12f851a9..d4675603593b 100644 --- a/arch/arm/mm/proc-v7m.S +++ b/arch/arm/mm/proc-v7m.S @@ -8,18 +8,19 @@ * This is the "shell" of the ARMv7-M processor support. */ #include +#include #include #include #include #include "proc-macros.S" -ENTRY(cpu_v7m_proc_init) +SYM_TYPED_FUNC_START(cpu_v7m_proc_init) ret lr -ENDPROC(cpu_v7m_proc_init) +SYM_FUNC_END(cpu_v7m_proc_init) -ENTRY(cpu_v7m_proc_fin) +SYM_TYPED_FUNC_START(cpu_v7m_proc_fin) ret lr -ENDPROC(cpu_v7m_proc_fin) +SYM_FUNC_END(cpu_v7m_proc_fin) /* * cpu_v7m_reset(loc) @@ -31,9 +32,9 @@ ENDPROC(cpu_v7m_proc_fin) * - loc - location to jump to for soft reset */ .align 5 -ENTRY(cpu_v7m_reset) +SYM_TYPED_FUNC_START(cpu_v7m_reset) ret r0 -ENDPROC(cpu_v7m_reset) +SYM_FUNC_END(cpu_v7m_reset) /* * cpu_v7m_do_idle() @@ -42,36 +43,36 @@ ENDPROC(cpu_v7m_reset) * * IRQs are already disabled. */ -ENTRY(cpu_v7m_do_idle) +SYM_TYPED_FUNC_START(cpu_v7m_do_idle) wfi ret lr -ENDPROC(cpu_v7m_do_idle) +SYM_FUNC_END(cpu_v7m_do_idle) -ENTRY(cpu_v7m_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_v7m_dcache_clean_area) ret lr -ENDPROC(cpu_v7m_dcache_clean_area) +SYM_FUNC_END(cpu_v7m_dcache_clean_area) /* * There is no MMU, so here is nothing to do. */ -ENTRY(cpu_v7m_switch_mm) +SYM_TYPED_FUNC_START(cpu_v7m_switch_mm) ret lr -ENDPROC(cpu_v7m_switch_mm) +SYM_FUNC_END(cpu_v7m_switch_mm) .globl cpu_v7m_suspend_size .equ cpu_v7m_suspend_size, 0 #ifdef CONFIG_ARM_CPU_SUSPEND -ENTRY(cpu_v7m_do_suspend) +SYM_TYPED_FUNC_START(cpu_v7m_do_suspend) ret lr -ENDPROC(cpu_v7m_do_suspend) +SYM_FUNC_END(cpu_v7m_do_suspend) -ENTRY(cpu_v7m_do_resume) +SYM_TYPED_FUNC_START(cpu_v7m_do_resume) ret lr -ENDPROC(cpu_v7m_do_resume) +SYM_FUNC_END(cpu_v7m_do_resume) #endif -ENTRY(cpu_cm7_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_cm7_dcache_clean_area) dcache_line_size r2, r3 movw r3, #:lower16:BASEADDR_V7M_SCB + V7M_SCB_DCCMVAC movt r3, #:upper16:BASEADDR_V7M_SCB + V7M_SCB_DCCMVAC @@ -82,16 +83,16 @@ ENTRY(cpu_cm7_dcache_clean_area) bhi 1b dsb ret lr -ENDPROC(cpu_cm7_dcache_clean_area) +SYM_FUNC_END(cpu_cm7_dcache_clean_area) -ENTRY(cpu_cm7_proc_fin) +SYM_TYPED_FUNC_START(cpu_cm7_proc_fin) movw r2, #:lower16:(BASEADDR_V7M_SCB + V7M_SCB_CCR) movt r2, #:upper16:(BASEADDR_V7M_SCB + V7M_SCB_CCR) ldr r0, [r2] bic r0, r0, #(V7M_SCB_CCR_DC | V7M_SCB_CCR_IC) str r0, [r2] ret lr -ENDPROC(cpu_cm7_proc_fin) +SYM_FUNC_END(cpu_cm7_proc_fin) .section ".init.text", "ax" diff --git a/arch/arm/mm/proc-xsc3.S b/arch/arm/mm/proc-xsc3.S index 7975f93b1e14..0e3d8e76376a 100644 --- a/arch/arm/mm/proc-xsc3.S +++ b/arch/arm/mm/proc-xsc3.S @@ -80,18 +80,20 @@ * * Nothing too exciting at the moment */ -ENTRY(cpu_xsc3_proc_init) +SYM_TYPED_FUNC_START(cpu_xsc3_proc_init) ret lr +SYM_FUNC_END(cpu_xsc3_proc_init) /* * cpu_xsc3_proc_fin() */ -ENTRY(cpu_xsc3_proc_fin) +SYM_TYPED_FUNC_START(cpu_xsc3_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x1800 @ ...IZ........... bic r0, r0, #0x0006 @ .............CA. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_xsc3_proc_fin) /* * cpu_xsc3_reset(loc) @@ -104,7 +106,7 @@ ENTRY(cpu_xsc3_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_xsc3_reset) +SYM_TYPED_FUNC_START(cpu_xsc3_reset) mov r1, #PSR_F_BIT|PSR_I_BIT|SVC_MODE msr cpsr_c, r1 @ reset CPSR mrc p15, 0, r1, c1, c0, 0 @ ctrl register @@ -118,7 +120,7 @@ ENTRY(cpu_xsc3_reset) @ already containing those two last instructions to survive. mcr p15, 0, ip, c8, c7, 0 @ invalidate I and D TLBs ret r0 -ENDPROC(cpu_xsc3_reset) +SYM_FUNC_END(cpu_xsc3_reset) .popsection /* @@ -133,10 +135,11 @@ ENDPROC(cpu_xsc3_reset) */ .align 5 -ENTRY(cpu_xsc3_do_idle) +SYM_TYPED_FUNC_START(cpu_xsc3_do_idle) mov r0, #1 mcr p14, 0, r0, c7, c0, 0 @ go to idle ret lr +SYM_FUNC_END(cpu_xsc3_do_idle) /* ================================= CACHE ================================ */ @@ -339,12 +342,13 @@ SYM_TYPED_FUNC_START(xsc3_dma_unmap_area) ret lr SYM_FUNC_END(xsc3_dma_unmap_area) -ENTRY(cpu_xsc3_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_xsc3_dcache_clean_area) 1: mcr p15, 0, r0, c7, c10, 1 @ clean L1 D line add r0, r0, #CACHELINESIZE subs r1, r1, #CACHELINESIZE bhi 1b ret lr +SYM_FUNC_END(cpu_xsc3_dcache_clean_area) /* =============================== PageTable ============================== */ @@ -356,7 +360,7 @@ ENTRY(cpu_xsc3_dcache_clean_area) * pgd: new page tables */ .align 5 -ENTRY(cpu_xsc3_switch_mm) +SYM_TYPED_FUNC_START(cpu_xsc3_switch_mm) clean_d_cache r1, r2 mcr p15, 0, ip, c7, c5, 0 @ invalidate L1 I cache and BTB mcr p15, 0, ip, c7, c10, 4 @ data write barrier @@ -365,6 +369,7 @@ ENTRY(cpu_xsc3_switch_mm) mcr p15, 0, r0, c2, c0, 0 @ load page table pointer mcr p15, 0, ip, c8, c7, 0 @ invalidate I and D TLBs cpwait_ret lr, ip +SYM_FUNC_END(cpu_xsc3_switch_mm) /* * cpu_xsc3_set_pte_ext(ptep, pte, ext) @@ -390,7 +395,7 @@ cpu_xsc3_mt_table: .long 0x00 @ unused .align 5 -ENTRY(cpu_xsc3_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_xsc3_set_pte_ext) xscale_set_pte_ext_prologue tst r1, #L_PTE_SHARED @ shared? @@ -403,6 +408,7 @@ ENTRY(cpu_xsc3_set_pte_ext) xscale_set_pte_ext_epilogue ret lr +SYM_FUNC_END(cpu_xsc3_set_pte_ext) .ltorg .align @@ -410,7 +416,7 @@ ENTRY(cpu_xsc3_set_pte_ext) .globl cpu_xsc3_suspend_size .equ cpu_xsc3_suspend_size, 4 * 6 #ifdef CONFIG_ARM_CPU_SUSPEND -ENTRY(cpu_xsc3_do_suspend) +SYM_TYPED_FUNC_START(cpu_xsc3_do_suspend) stmfd sp!, {r4 - r9, lr} mrc p14, 0, r4, c6, c0, 0 @ clock configuration, for turbo mode mrc p15, 0, r5, c15, c1, 0 @ CP access reg @@ -421,9 +427,9 @@ ENTRY(cpu_xsc3_do_suspend) bic r4, r4, #2 @ clear frequency change bit stmia r0, {r4 - r9} @ store cp regs ldmia sp!, {r4 - r9, pc} -ENDPROC(cpu_xsc3_do_suspend) +SYM_FUNC_END(cpu_xsc3_do_suspend) -ENTRY(cpu_xsc3_do_resume) +SYM_TYPED_FUNC_START(cpu_xsc3_do_resume) ldmia r0, {r4 - r9} @ load cp regs mov ip, #0 mcr p15, 0, ip, c7, c7, 0 @ invalidate I & D caches, BTB @@ -439,7 +445,7 @@ ENTRY(cpu_xsc3_do_resume) mcr p15, 0, r8, c1, c0, 1 @ auxiliary control reg mov r0, r9 @ control register b cpu_resume_mmu -ENDPROC(cpu_xsc3_do_resume) +SYM_FUNC_END(cpu_xsc3_do_resume) #endif .type __xsc3_setup, #function diff --git a/arch/arm/mm/proc-xscale.S b/arch/arm/mm/proc-xscale.S index bbf1e94ba554..d8462df8020b 100644 --- a/arch/arm/mm/proc-xscale.S +++ b/arch/arm/mm/proc-xscale.S @@ -112,22 +112,24 @@ clean_addr: .word CLEAN_ADDR * * Nothing too exciting at the moment */ -ENTRY(cpu_xscale_proc_init) +SYM_TYPED_FUNC_START(cpu_xscale_proc_init) @ enable write buffer coalescing. Some bootloader disable it mrc p15, 0, r1, c1, c0, 1 bic r1, r1, #1 mcr p15, 0, r1, c1, c0, 1 ret lr +SYM_FUNC_END(cpu_xscale_proc_init) /* * cpu_xscale_proc_fin() */ -ENTRY(cpu_xscale_proc_fin) +SYM_TYPED_FUNC_START(cpu_xscale_proc_fin) mrc p15, 0, r0, c1, c0, 0 @ ctrl register bic r0, r0, #0x1800 @ ...IZ........... bic r0, r0, #0x0006 @ .............CA. mcr p15, 0, r0, c1, c0, 0 @ disable caches ret lr +SYM_FUNC_END(cpu_xscale_proc_fin) /* * cpu_xscale_reset(loc) @@ -142,7 +144,7 @@ ENTRY(cpu_xscale_proc_fin) */ .align 5 .pushsection .idmap.text, "ax" -ENTRY(cpu_xscale_reset) +SYM_TYPED_FUNC_START(cpu_xscale_reset) mov r1, #PSR_F_BIT|PSR_I_BIT|SVC_MODE msr cpsr_c, r1 @ reset CPSR mcr p15, 0, r1, c10, c4, 1 @ unlock I-TLB @@ -160,7 +162,7 @@ ENTRY(cpu_xscale_reset) @ already containing those two last instructions to survive. mcr p15, 0, ip, c8, c7, 0 @ invalidate I & D TLBs ret r0 -ENDPROC(cpu_xscale_reset) +SYM_FUNC_END(cpu_xscale_reset) .popsection /* @@ -175,10 +177,11 @@ ENDPROC(cpu_xscale_reset) */ .align 5 -ENTRY(cpu_xscale_do_idle) +SYM_TYPED_FUNC_START(cpu_xscale_do_idle) mov r0, #1 mcr p14, 0, r0, c7, c0, 0 @ Go to IDLE ret lr +SYM_FUNC_END(cpu_xscale_do_idle) /* ================================= CACHE ================================ */ @@ -428,12 +431,13 @@ SYM_TYPED_FUNC_START(xscale_dma_unmap_area) ret lr SYM_FUNC_END(xscale_dma_unmap_area) -ENTRY(cpu_xscale_dcache_clean_area) +SYM_TYPED_FUNC_START(cpu_xscale_dcache_clean_area) 1: mcr p15, 0, r0, c7, c10, 1 @ clean D entry add r0, r0, #CACHELINESIZE subs r1, r1, #CACHELINESIZE bhi 1b ret lr +SYM_FUNC_END(cpu_xscale_dcache_clean_area) /* =============================== PageTable ============================== */ @@ -445,13 +449,14 @@ ENTRY(cpu_xscale_dcache_clean_area) * pgd: new page tables */ .align 5 -ENTRY(cpu_xscale_switch_mm) +SYM_TYPED_FUNC_START(cpu_xscale_switch_mm) clean_d_cache r1, r2 mcr p15, 0, ip, c7, c5, 0 @ Invalidate I cache & BTB mcr p15, 0, ip, c7, c10, 4 @ Drain Write (& Fill) Buffer mcr p15, 0, r0, c2, c0, 0 @ load page table pointer mcr p15, 0, ip, c8, c7, 0 @ invalidate I & D TLBs cpwait_ret lr, ip +SYM_FUNC_END(cpu_xscale_switch_mm) /* * cpu_xscale_set_pte_ext(ptep, pte, ext) @@ -479,7 +484,7 @@ cpu_xscale_mt_table: .long 0x00 @ unused .align 5 -ENTRY(cpu_xscale_set_pte_ext) +SYM_TYPED_FUNC_START(cpu_xscale_set_pte_ext) xscale_set_pte_ext_prologue @ @@ -497,6 +502,7 @@ ENTRY(cpu_xscale_set_pte_ext) xscale_set_pte_ext_epilogue ret lr +SYM_FUNC_END(cpu_xscale_set_pte_ext) .ltorg .align @@ -504,7 +510,7 @@ ENTRY(cpu_xscale_set_pte_ext) .globl cpu_xscale_suspend_size .equ cpu_xscale_suspend_size, 4 * 6 #ifdef CONFIG_ARM_CPU_SUSPEND -ENTRY(cpu_xscale_do_suspend) +SYM_TYPED_FUNC_START(cpu_xscale_do_suspend) stmfd sp!, {r4 - r9, lr} mrc p14, 0, r4, c6, c0, 0 @ clock configuration, for turbo mode mrc p15, 0, r5, c15, c1, 0 @ CP access reg @@ -515,9 +521,9 @@ ENTRY(cpu_xscale_do_suspend) bic r4, r4, #2 @ clear frequency change bit stmia r0, {r4 - r9} @ store cp regs ldmfd sp!, {r4 - r9, pc} -ENDPROC(cpu_xscale_do_suspend) +SYM_FUNC_END(cpu_xscale_do_suspend) -ENTRY(cpu_xscale_do_resume) +SYM_TYPED_FUNC_START(cpu_xscale_do_resume) ldmia r0, {r4 - r9} @ load cp regs mov ip, #0 mcr p15, 0, ip, c8, c7, 0 @ invalidate I & D TLBs @@ -530,7 +536,7 @@ ENTRY(cpu_xscale_do_resume) mcr p15, 0, r8, c1, c0, 1 @ auxiliary control reg mov r0, r9 @ control register b cpu_resume_mmu -ENDPROC(cpu_xscale_do_resume) +SYM_FUNC_END(cpu_xscale_do_resume) #endif .type __xscale_setup, #function From patchwork Sun Apr 21 18:28:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13637485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D31D2C04FF8 for ; Sun, 21 Apr 2024 18:29:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1RpG4roOBlMZKoayVJoR3gTE7zslE8vyndH8jki6dRI=; b=EnWAgeXX9erblJ q8kovBjsuv2oxpR4XkRH1RP6x8Pao0vLX/rdzPHbbEGTJukwlNFNu7nD+HNMlxwjVqd3BuYBlscrK EGdmOVLL1qFFA4aOwY4rl6kmT0B2zjCkN13n371U28Ki8UA7LE0aflwYMG5lpyMS+FJ6BMy9syCNS ZxaetcNWu76GJgtm9DSmWVBjF5v/I9ZmmfsZZb5WPYlWuLPcjSpwq79uQ+BX+eIfLYJUy20Yn+3xZ Bm85/VsG/d30Z+LrVtDwsijrZsEYZCpEQ1gx5WkxKGSleI26+d0ueh+TS10RthYQ1tOJEdEBYNJaF OyI3r2zrGspbX0IOqxvA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvu-0000000B4AZ-1VuQ; Sun, 21 Apr 2024 18:28:58 +0000 Received: from mail-lj1-x22e.google.com ([2a00:1450:4864:20::22e]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvb-0000000B425-2Y8J for linux-arm-kernel@lists.infradead.org; Sun, 21 Apr 2024 18:28:43 +0000 Received: by mail-lj1-x22e.google.com with SMTP id 38308e7fff4ca-2dd7e56009cso3738741fa.3 for ; Sun, 21 Apr 2024 11:28:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1713724117; x=1714328917; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=kEWqcVwxPihHlHVj+XEGbwe/qqfujZwXRKtTnbCyyHc=; b=wWEDvG5u3zf8ljQOkyd8d4Rre2/b3HXF1WVnCv5XkX13kB9QQUAkV6jPAMpco2iqdQ QfgFF/iGQS1dl5psQkX5GZU5Eb+32zLJ2YJ0dYIWCAL9/jAfMwSGJHQgK2JzehIYTa8x MfEaITkPTHkWW7NfrEwIacth4tX5DC83Wo705JJ6exZYBZ7DoEGg/6+2300zajYSD8Gr dfJRvPJqvjUggV+s5H3nHFqC3M0gy20900wVoZ4iyLNC2tx1X7YsayIKIraxtyBzfXwJ myf4LbFoI4lvUHROhg0o7tMohlSKQSsNywUkH08ena1vJi1EIl+yL8aIUJuL28O6p6LO cdaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713724117; x=1714328917; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kEWqcVwxPihHlHVj+XEGbwe/qqfujZwXRKtTnbCyyHc=; b=iRlYhwAvP6L7fGP2HNP4nohSKm+XWEK10jDmMeOvhq4RPk8sGzyIznAAKnimPQvqf5 vEZQAF6Qoe0s5qaevJBvExry3IiMi6vbNBp16t0dkB/WuG110laUEj8e+59OJBBhNjuO 8MbBWv0W1S0HFUjxXbLVsBkLVE54IO4o1Dhr6v4m0toC83yYfC9K9wbEbIwz6rSwHI/k JBdLE2uoHJRnwkpB4QB3j98SYOTgWldv3eGRrDhw/dgAs4Jj1812qwG45j958oPTOxOD NcpVFT+ZTQ83lNM2s/1nBCl7QWsBwWAAkP0e6aRiRdKOH2plgFuh5YT03DCwY/WZnXf7 VFtA== X-Gm-Message-State: AOJu0Yzkn6bk8MqIpn6HwLbtn+Hz+Z3C+9AowOb1P7hIv1Kk9O18FCUn xm8VUFWQcrtCCNgPfUTw561U+236/cSd1JrTF0fpFDcDbqWKzSoDV+JfBCYNpvQ= X-Google-Smtp-Source: AGHT+IGA4musC120/nEn3KbZqN9qBcHMCv+O7LiRyTSUOKKfkCRXo3VzJbyCXXcy/clee9rUUCH43w== X-Received: by 2002:ac2:4f82:0:b0:519:7392:5b56 with SMTP id z2-20020ac24f82000000b0051973925b56mr4982518lfs.41.1713724116646; Sun, 21 Apr 2024 11:28:36 -0700 (PDT) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id s14-20020a19770e000000b005188ea5d2b5sm1549641lfc.303.2024.04.21.11.28.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Apr 2024 11:28:36 -0700 (PDT) From: Linus Walleij Date: Sun, 21 Apr 2024 20:28:34 +0200 Subject: [PATCH v7 08/11] ARM: mm: Define prototypes for all per-processor calls MIME-Version: 1.0 Message-Id: <20240421-arm32-cfi-v7-8-6e132a948cc8@linaro.org> References: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> In-Reply-To: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> To: Russell King , Sami Tolvanen , Kees Cook , Nathan Chancellor , Nick Desaulniers , Ard Biesheuvel , Arnd Bergmann Cc: linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev, Linus Walleij X-Mailer: b4 0.13.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240421_112839_819757_EF4141D2 X-CRM114-Status: GOOD ( 21.13 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Each CPU type ("proc") has assembly calls for initializing and setting up the MM context, idle and so forth. These calls have the C form of e.g.: void cpu_arm920_init(void); However this prototype is not really specified, instead it is generated by the glue code in and the prototype is implicit from the generic prototype defined in such as cpu_proc_init() in this case. (This is a bit similar to the "interface" or inheritance concept in other languages.) To be able to annotate these assembly calls for CFI, they all need to have a proper C prototype per CPU call. Define these in a new C file that is only compiled when we use CFI, and add __ADDRESSABLE() to each so the compiler knows that these will be addressed (they are not explicitly called in C, they are called by way of cpu_proc_init() etc). It is a bit of definitions, but we do not expect new ARM32 CPUs to appear very much so it should be pretty static. Tested-by: Kees Cook Signed-off-by: Linus Walleij --- arch/arm/mm/Makefile | 1 + arch/arm/mm/proc.c | 500 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 501 insertions(+) diff --git a/arch/arm/mm/Makefile b/arch/arm/mm/Makefile index 17665381be96..f1f231f20ff9 100644 --- a/arch/arm/mm/Makefile +++ b/arch/arm/mm/Makefile @@ -90,6 +90,7 @@ obj-$(CONFIG_CPU_V6) += proc-v6.o obj-$(CONFIG_CPU_V6K) += proc-v6.o obj-$(CONFIG_CPU_V7) += proc-v7.o proc-v7-bugs.o obj-$(CONFIG_CPU_V7M) += proc-v7m.o +obj-$(CONFIG_CFI_CLANG) += proc.o obj-$(CONFIG_OUTER_CACHE) += l2c-common.o obj-$(CONFIG_CACHE_B15_RAC) += cache-b15-rac.o diff --git a/arch/arm/mm/proc.c b/arch/arm/mm/proc.c new file mode 100644 index 000000000000..bdbbf65d1b36 --- /dev/null +++ b/arch/arm/mm/proc.c @@ -0,0 +1,500 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * This file defines C prototypes for the low-level processor assembly functions + * and creates a reference for CFI. This needs to be done for every assembly + * processor ("proc") function that is called from C but does not have a + * corresponding C implementation. + * + * Processors are listed in the order they appear in the Makefile. + * + * Functions are listed if and only if they see use on the target CPU, and in + * the order they are defined in struct processor. + */ +#include + +#ifdef CONFIG_CPU_ARM7TDMI +void cpu_arm7tdmi_proc_init(void); +__ADDRESSABLE(cpu_arm7tdmi_proc_init); +void cpu_arm7tdmi_proc_fin(void); +__ADDRESSABLE(cpu_arm7tdmi_proc_fin); +void cpu_arm7tdmi_reset(void); +__ADDRESSABLE(cpu_arm7tdmi_reset); +int cpu_arm7tdmi_do_idle(void); +__ADDRESSABLE(cpu_arm7tdmi_do_idle); +void cpu_arm7tdmi_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_arm7tdmi_dcache_clean_area); +void cpu_arm7tdmi_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_arm7tdmi_switch_mm); +#endif + +#ifdef CONFIG_CPU_ARM720T +void cpu_arm720_proc_init(void); +__ADDRESSABLE(cpu_arm720_proc_init); +void cpu_arm720_proc_fin(void); +__ADDRESSABLE(cpu_arm720_proc_fin); +void cpu_arm720_reset(void); +__ADDRESSABLE(cpu_arm720_reset); +int cpu_arm720_do_idle(void); +__ADDRESSABLE(cpu_arm720_do_idle); +void cpu_arm720_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_arm720_dcache_clean_area); +void cpu_arm720_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_arm720_switch_mm); +void cpu_arm720_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_arm720_set_pte_ext); +#endif + +#ifdef CONFIG_CPU_ARM740T +void cpu_arm740_proc_init(void); +__ADDRESSABLE(cpu_arm740_proc_init); +void cpu_arm740_proc_fin(void); +__ADDRESSABLE(cpu_arm740_proc_fin); +void cpu_arm740_reset(void); +__ADDRESSABLE(cpu_arm740_reset); +int cpu_arm740_do_idle(void); +__ADDRESSABLE(cpu_arm740_do_idle); +void cpu_arm740_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_arm740_dcache_clean_area); +void cpu_arm740_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_arm740_switch_mm); +#endif + +#ifdef CONFIG_CPU_ARM9TDMI +void cpu_arm9tdmi_proc_init(void); +__ADDRESSABLE(cpu_arm9tdmi_proc_init); +void cpu_arm9tdmi_proc_fin(void); +__ADDRESSABLE(cpu_arm9tdmi_proc_fin); +void cpu_arm9tdmi_reset(void); +__ADDRESSABLE(cpu_arm9tdmi_reset); +int cpu_arm9tdmi_do_idle(void); +__ADDRESSABLE(cpu_arm9tdmi_do_idle); +void cpu_arm9tdmi_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_arm9tdmi_dcache_clean_area); +void cpu_arm9tdmi_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_arm9tdmi_switch_mm); +#endif + +#ifdef CONFIG_CPU_ARM920T +void cpu_arm920_proc_init(void); +__ADDRESSABLE(cpu_arm920_proc_init); +void cpu_arm920_proc_fin(void); +__ADDRESSABLE(cpu_arm920_proc_fin); +void cpu_arm920_reset(void); +__ADDRESSABLE(cpu_arm920_reset); +int cpu_arm920_do_idle(void); +__ADDRESSABLE(cpu_arm920_do_idle); +void cpu_arm920_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_arm920_dcache_clean_area); +void cpu_arm920_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_arm920_switch_mm); +void cpu_arm920_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_arm920_set_pte_ext); +#ifdef CONFIG_ARM_CPU_SUSPEND +void cpu_arm920_do_suspend(void *); +__ADDRESSABLE(cpu_arm920_do_suspend); +void cpu_arm920_do_resume(void *); +__ADDRESSABLE(cpu_arm920_do_resume); +#endif /* CONFIG_ARM_CPU_SUSPEND */ +#endif /* CONFIG_CPU_ARM920T */ + +#ifdef CONFIG_CPU_ARM922T +void cpu_arm922_proc_init(void); +__ADDRESSABLE(cpu_arm922_proc_init); +void cpu_arm922_proc_fin(void); +__ADDRESSABLE(cpu_arm922_proc_fin); +void cpu_arm922_reset(void); +__ADDRESSABLE(cpu_arm922_reset); +int cpu_arm922_do_idle(void); +__ADDRESSABLE(cpu_arm922_do_idle); +void cpu_arm922_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_arm922_dcache_clean_area); +void cpu_arm922_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_arm922_switch_mm); +void cpu_arm922_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_arm922_set_pte_ext); +#endif + +#ifdef CONFIG_CPU_ARM925T +void cpu_arm925_proc_init(void); +__ADDRESSABLE(cpu_arm925_proc_init); +void cpu_arm925_proc_fin(void); +__ADDRESSABLE(cpu_arm925_proc_fin); +void cpu_arm925_reset(void); +__ADDRESSABLE(cpu_arm925_reset); +int cpu_arm925_do_idle(void); +__ADDRESSABLE(cpu_arm925_do_idle); +void cpu_arm925_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_arm925_dcache_clean_area); +void cpu_arm925_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_arm925_switch_mm); +void cpu_arm925_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_arm925_set_pte_ext); +#endif + +#ifdef CONFIG_CPU_ARM926T +void cpu_arm926_proc_init(void); +__ADDRESSABLE(cpu_arm926_proc_init); +void cpu_arm926_proc_fin(void); +__ADDRESSABLE(cpu_arm926_proc_fin); +void cpu_arm926_reset(unsigned long addr, bool hvc); +__ADDRESSABLE(cpu_arm926_reset); +int cpu_arm926_do_idle(void); +__ADDRESSABLE(cpu_arm926_do_idle); +void cpu_arm926_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_arm926_dcache_clean_area); +void cpu_arm926_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_arm926_switch_mm); +void cpu_arm926_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_arm926_set_pte_ext); +#ifdef CONFIG_ARM_CPU_SUSPEND +void cpu_arm926_do_suspend(void *); +__ADDRESSABLE(cpu_arm926_do_suspend); +void cpu_arm926_do_resume(void *); +__ADDRESSABLE(cpu_arm926_do_resume); +#endif /* CONFIG_ARM_CPU_SUSPEND */ +#endif /* CONFIG_CPU_ARM926T */ + +#ifdef CONFIG_CPU_ARM940T +void cpu_arm940_proc_init(void); +__ADDRESSABLE(cpu_arm940_proc_init); +void cpu_arm940_proc_fin(void); +__ADDRESSABLE(cpu_arm940_proc_fin); +void cpu_arm940_reset(void); +__ADDRESSABLE(cpu_arm940_reset); +int cpu_arm940_do_idle(void); +__ADDRESSABLE(cpu_arm940_do_idle); +void cpu_arm940_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_arm940_dcache_clean_area); +void cpu_arm940_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_arm940_switch_mm); +#endif + +#ifdef CONFIG_CPU_ARM946E +void cpu_arm946_proc_init(void); +__ADDRESSABLE(cpu_arm946_proc_init); +void cpu_arm946_proc_fin(void); +__ADDRESSABLE(cpu_arm946_proc_fin); +void cpu_arm946_reset(void); +__ADDRESSABLE(cpu_arm946_reset); +int cpu_arm946_do_idle(void); +__ADDRESSABLE(cpu_arm946_do_idle); +void cpu_arm946_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_arm946_dcache_clean_area); +void cpu_arm946_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_arm946_switch_mm); +#endif + +#ifdef CONFIG_CPU_FA526 +void cpu_fa526_proc_init(void); +__ADDRESSABLE(cpu_fa526_proc_init); +void cpu_fa526_proc_fin(void); +__ADDRESSABLE(cpu_fa526_proc_fin); +void cpu_fa526_reset(unsigned long addr, bool hvc); +__ADDRESSABLE(cpu_fa526_reset); +int cpu_fa526_do_idle(void); +__ADDRESSABLE(cpu_fa526_do_idle); +void cpu_fa526_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_fa526_dcache_clean_area); +void cpu_fa526_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_fa526_switch_mm); +void cpu_fa526_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_fa526_set_pte_ext); +#endif + +#ifdef CONFIG_CPU_ARM1020 +void cpu_arm1020_proc_init(void); +__ADDRESSABLE(cpu_arm1020_proc_init); +void cpu_arm1020_proc_fin(void); +__ADDRESSABLE(cpu_arm1020_proc_fin); +void cpu_arm1020_reset(unsigned long addr, bool hvc); +__ADDRESSABLE(cpu_arm1020_reset); +int cpu_arm1020_do_idle(void); +__ADDRESSABLE(cpu_arm1020_do_idle); +void cpu_arm1020_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_arm1020_dcache_clean_area); +void cpu_arm1020_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_arm1020_switch_mm); +void cpu_arm1020_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_arm1020_set_pte_ext); +#endif + +#ifdef CONFIG_CPU_ARM1020E +void cpu_arm1020e_proc_init(void); +__ADDRESSABLE(cpu_arm1020e_proc_init); +void cpu_arm1020e_proc_fin(void); +__ADDRESSABLE(cpu_arm1020e_proc_fin); +void cpu_arm1020e_reset(unsigned long addr, bool hvc); +__ADDRESSABLE(cpu_arm1020e_reset); +int cpu_arm1020e_do_idle(void); +__ADDRESSABLE(cpu_arm1020e_do_idle); +void cpu_arm1020e_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_arm1020e_dcache_clean_area); +void cpu_arm1020e_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_arm1020e_switch_mm); +void cpu_arm1020e_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_arm1020e_set_pte_ext); +#endif + +#ifdef CONFIG_CPU_ARM1022 +void cpu_arm1022_proc_init(void); +__ADDRESSABLE(cpu_arm1022_proc_init); +void cpu_arm1022_proc_fin(void); +__ADDRESSABLE(cpu_arm1022_proc_fin); +void cpu_arm1022_reset(unsigned long addr, bool hvc); +__ADDRESSABLE(cpu_arm1022_reset); +int cpu_arm1022_do_idle(void); +__ADDRESSABLE(cpu_arm1022_do_idle); +void cpu_arm1022_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_arm1022_dcache_clean_area); +void cpu_arm1022_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_arm1022_switch_mm); +void cpu_arm1022_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_arm1022_set_pte_ext); +#endif + +#ifdef CONFIG_CPU_ARM1026 +void cpu_arm1026_proc_init(void); +__ADDRESSABLE(cpu_arm1026_proc_init); +void cpu_arm1026_proc_fin(void); +__ADDRESSABLE(cpu_arm1026_proc_fin); +void cpu_arm1026_reset(unsigned long addr, bool hvc); +__ADDRESSABLE(cpu_arm1026_reset); +int cpu_arm1026_do_idle(void); +__ADDRESSABLE(cpu_arm1026_do_idle); +void cpu_arm1026_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_arm1026_dcache_clean_area); +void cpu_arm1026_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_arm1026_switch_mm); +void cpu_arm1026_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_arm1026_set_pte_ext); +#endif + +#ifdef CONFIG_CPU_SA110 +void cpu_sa110_proc_init(void); +__ADDRESSABLE(cpu_sa110_proc_init); +void cpu_sa110_proc_fin(void); +__ADDRESSABLE(cpu_sa110_proc_fin); +void cpu_sa110_reset(unsigned long addr, bool hvc); +__ADDRESSABLE(cpu_sa110_reset); +int cpu_sa110_do_idle(void); +__ADDRESSABLE(cpu_sa110_do_idle); +void cpu_sa110_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_sa110_dcache_clean_area); +void cpu_sa110_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_sa110_switch_mm); +void cpu_sa110_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_sa110_set_pte_ext); +#endif + +#ifdef CONFIG_CPU_SA1100 +void cpu_sa1100_proc_init(void); +__ADDRESSABLE(cpu_sa1100_proc_init); +void cpu_sa1100_proc_fin(void); +__ADDRESSABLE(cpu_sa1100_proc_fin); +void cpu_sa1100_reset(unsigned long addr, bool hvc); +__ADDRESSABLE(cpu_sa1100_reset); +int cpu_sa1100_do_idle(void); +__ADDRESSABLE(cpu_sa1100_do_idle); +void cpu_sa1100_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_sa1100_dcache_clean_area); +void cpu_sa1100_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_sa1100_switch_mm); +void cpu_sa1100_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_sa1100_set_pte_ext); +#ifdef CONFIG_ARM_CPU_SUSPEND +void cpu_sa1100_do_suspend(void *); +__ADDRESSABLE(cpu_sa1100_do_suspend); +void cpu_sa1100_do_resume(void *); +__ADDRESSABLE(cpu_sa1100_do_resume); +#endif /* CONFIG_ARM_CPU_SUSPEND */ +#endif /* CONFIG_CPU_SA1100 */ + +#ifdef CONFIG_CPU_XSCALE +void cpu_xscale_proc_init(void); +__ADDRESSABLE(cpu_xscale_proc_init); +void cpu_xscale_proc_fin(void); +__ADDRESSABLE(cpu_xscale_proc_fin); +void cpu_xscale_reset(unsigned long addr, bool hvc); +__ADDRESSABLE(cpu_xscale_reset); +int cpu_xscale_do_idle(void); +__ADDRESSABLE(cpu_xscale_do_idle); +void cpu_xscale_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_xscale_dcache_clean_area); +void cpu_xscale_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_xscale_switch_mm); +void cpu_xscale_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_xscale_set_pte_ext); +#ifdef CONFIG_ARM_CPU_SUSPEND +void cpu_xscale_do_suspend(void *); +__ADDRESSABLE(cpu_xscale_do_suspend); +void cpu_xscale_do_resume(void *); +__ADDRESSABLE(cpu_xscale_do_resume); +#endif /* CONFIG_ARM_CPU_SUSPEND */ +#endif /* CONFIG_CPU_XSCALE */ + +#ifdef CONFIG_CPU_XSC3 +void cpu_xsc3_proc_init(void); +__ADDRESSABLE(cpu_xsc3_proc_init); +void cpu_xsc3_proc_fin(void); +__ADDRESSABLE(cpu_xsc3_proc_fin); +void cpu_xsc3_reset(unsigned long addr, bool hvc); +__ADDRESSABLE(cpu_xsc3_reset); +int cpu_xsc3_do_idle(void); +__ADDRESSABLE(cpu_xsc3_do_idle); +void cpu_xsc3_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_xsc3_dcache_clean_area); +void cpu_xsc3_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_xsc3_switch_mm); +void cpu_xsc3_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_xsc3_set_pte_ext); +#ifdef CONFIG_ARM_CPU_SUSPEND +void cpu_xsc3_do_suspend(void *); +__ADDRESSABLE(cpu_xsc3_do_suspend); +void cpu_xsc3_do_resume(void *); +__ADDRESSABLE(cpu_xsc3_do_resume); +#endif /* CONFIG_ARM_CPU_SUSPEND */ +#endif /* CONFIG_CPU_XSC3 */ + +#ifdef CONFIG_CPU_MOHAWK +void cpu_mohawk_proc_init(void); +__ADDRESSABLE(cpu_mohawk_proc_init); +void cpu_mohawk_proc_fin(void); +__ADDRESSABLE(cpu_mohawk_proc_fin); +void cpu_mohawk_reset(unsigned long addr, bool hvc); +__ADDRESSABLE(cpu_mohawk_reset); +int cpu_mohawk_do_idle(void); +__ADDRESSABLE(cpu_mohawk_do_idle); +void cpu_mohawk_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_mohawk_dcache_clean_area); +void cpu_mohawk_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_mohawk_switch_mm); +void cpu_mohawk_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_mohawk_set_pte_ext); +#ifdef CONFIG_ARM_CPU_SUSPEND +void cpu_mohawk_do_suspend(void *); +__ADDRESSABLE(cpu_mohawk_do_suspend); +void cpu_mohawk_do_resume(void *); +__ADDRESSABLE(cpu_mohawk_do_resume); +#endif /* CONFIG_ARM_CPU_SUSPEND */ +#endif /* CONFIG_CPU_MOHAWK */ + +#ifdef CONFIG_CPU_FEROCEON +void cpu_feroceon_proc_init(void); +__ADDRESSABLE(cpu_feroceon_proc_init); +void cpu_feroceon_proc_fin(void); +__ADDRESSABLE(cpu_feroceon_proc_fin); +void cpu_feroceon_reset(unsigned long addr, bool hvc); +__ADDRESSABLE(cpu_feroceon_reset); +int cpu_feroceon_do_idle(void); +__ADDRESSABLE(cpu_feroceon_do_idle); +void cpu_feroceon_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_feroceon_dcache_clean_area); +void cpu_feroceon_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_feroceon_switch_mm); +void cpu_feroceon_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_feroceon_set_pte_ext); +#ifdef CONFIG_ARM_CPU_SUSPEND +void cpu_feroceon_do_suspend(void *); +__ADDRESSABLE(cpu_feroceon_do_suspend); +void cpu_feroceon_do_resume(void *); +__ADDRESSABLE(cpu_feroceon_do_resume); +#endif /* CONFIG_ARM_CPU_SUSPEND */ +#endif /* CONFIG_CPU_FEROCEON */ + +#if defined(CONFIG_CPU_V6) || defined(CONFIG_CPU_V6K) +void cpu_v6_proc_init(void); +__ADDRESSABLE(cpu_v6_proc_init); +void cpu_v6_proc_fin(void); +__ADDRESSABLE(cpu_v6_proc_fin); +void cpu_v6_reset(unsigned long addr, bool hvc); +__ADDRESSABLE(cpu_v6_reset); +int cpu_v6_do_idle(void); +__ADDRESSABLE(cpu_v6_do_idle); +void cpu_v6_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_v6_dcache_clean_area); +void cpu_v6_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_v6_switch_mm); +void cpu_v6_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_v6_set_pte_ext); +#ifdef CONFIG_ARM_CPU_SUSPEND +void cpu_v6_do_suspend(void *); +__ADDRESSABLE(cpu_v6_do_suspend); +void cpu_v6_do_resume(void *); +__ADDRESSABLE(cpu_v6_do_resume); +#endif /* CONFIG_ARM_CPU_SUSPEND */ +#endif /* CPU_V6 */ + +#ifdef CONFIG_CPU_V7 +void cpu_v7_proc_init(void); +__ADDRESSABLE(cpu_v7_proc_init); +void cpu_v7_proc_fin(void); +__ADDRESSABLE(cpu_v7_proc_fin); +void cpu_v7_reset(void); +__ADDRESSABLE(cpu_v7_reset); +int cpu_v7_do_idle(void); +__ADDRESSABLE(cpu_v7_do_idle); +#ifdef CONFIG_PJ4B_ERRATA_4742 +int cpu_pj4b_do_idle(void); +__ADDRESSABLE(cpu_pj4b_do_idle); +#endif +void cpu_v7_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_v7_dcache_clean_area); +void cpu_v7_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +/* Special switch_mm() callbacks to work around bugs in v7 */ +__ADDRESSABLE(cpu_v7_switch_mm); +void cpu_v7_iciallu_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_v7_iciallu_switch_mm); +void cpu_v7_bpiall_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_v7_bpiall_switch_mm); +#ifdef CONFIG_ARM_LPAE +void cpu_v7_set_pte_ext(pte_t *ptep, pte_t pte); +#else +void cpu_v7_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +#endif +__ADDRESSABLE(cpu_v7_set_pte_ext); +#ifdef CONFIG_ARM_CPU_SUSPEND +void cpu_v7_do_suspend(void *); +__ADDRESSABLE(cpu_v7_do_suspend); +void cpu_v7_do_resume(void *); +__ADDRESSABLE(cpu_v7_do_resume); +/* Special versions of suspend and resume for the CA9MP cores */ +void cpu_ca9mp_do_suspend(void *); +__ADDRESSABLE(cpu_ca9mp_do_suspend); +void cpu_ca9mp_do_resume(void *); +__ADDRESSABLE(cpu_ca9mp_do_resume); +/* Special versions of suspend and resume for the Marvell PJ4B cores */ +#ifdef CONFIG_CPU_PJ4B +void cpu_pj4b_do_suspend(void *); +__ADDRESSABLE(cpu_pj4b_do_suspend); +void cpu_pj4b_do_resume(void *); +__ADDRESSABLE(cpu_pj4b_do_resume); +#endif /* CONFIG_CPU_PJ4B */ +#endif /* CONFIG_ARM_CPU_SUSPEND */ +#endif /* CONFIG_CPU_V7 */ + +#ifdef CONFIG_CPU_V7M +void cpu_v7m_proc_init(void); +__ADDRESSABLE(cpu_v7m_proc_init); +void cpu_v7m_proc_fin(void); +__ADDRESSABLE(cpu_v7m_proc_fin); +void cpu_v7m_reset(unsigned long addr, bool hvc); +__ADDRESSABLE(cpu_v7m_reset); +int cpu_v7m_do_idle(void); +__ADDRESSABLE(cpu_v7m_do_idle); +void cpu_v7m_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_v7m_dcache_clean_area); +void cpu_v7m_switch_mm(phys_addr_t pgd_phys, struct mm_struct *mm); +__ADDRESSABLE(cpu_v7m_switch_mm); +void cpu_v7m_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext); +__ADDRESSABLE(cpu_v7m_set_pte_ext); +#ifdef CONFIG_ARM_CPU_SUSPEND +void cpu_v7m_do_suspend(void *); +__ADDRESSABLE(cpu_v7m_do_suspend); +void cpu_v7m_do_resume(void *); +__ADDRESSABLE(cpu_v7m_do_resume); +#endif /* CONFIG_ARM_CPU_SUSPEND */ +void cpu_cm7_proc_fin(void); +__ADDRESSABLE(cpu_cm7_proc_fin); +void cpu_cm7_dcache_clean_area(void *addr, int size); +__ADDRESSABLE(cpu_cm7_dcache_clean_area); +#endif /* CONFIG_CPU_V7M */ From patchwork Sun Apr 21 18:28:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13637483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B0270C4345F for ; Sun, 21 Apr 2024 18:29:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wtbxcCOjNqKikZ4+vs0wLkSrZ5iSy3Vgjj4tfm2zv+o=; b=WJNzfCX3mYTdbA TWH4+lWHrYkvF+4MgGWJ7wS/B3uDXzR6COsdMPG1VldxMpoArqZXvIhf0I23yq9R5IbYrCxVHjRby w7XGKkukJ2Ek1kz1FoqWKeG4WEOT+YsJpC9OJfze45hLxdwEYMFkjKBYcFeJ3JFslcv8fXMbVGNQ1 PzrBSd+yUSO2YM8im8wmxrVSdtzEG+ohduCgY7jLpcPwPx9GpwG7jlqSg25jQdu6qcXs54QwnuPck /7RAefk0Zl4i0usSsRuj/d3YrvFzhGhk1AloSbeIj8QwfBYUHoREg7TwsxHLoSQCexNzhhxzvk3of MTIHaYj84dB64RBoA1sw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvt-0000000B49l-0Od7; Sun, 21 Apr 2024 18:28:57 +0000 Received: from mail-lf1-x134.google.com ([2a00:1450:4864:20::134]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvc-0000000B42u-36og for linux-arm-kernel@lists.infradead.org; Sun, 21 Apr 2024 18:28:42 +0000 Received: by mail-lf1-x134.google.com with SMTP id 2adb3069b0e04-518a56cdbcfso5715489e87.2 for ; Sun, 21 Apr 2024 11:28:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1713724118; x=1714328918; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=NrM2dGN+EheWjKnt5zlJvOk5v0Ei3knf8+WuyH2chzo=; b=QLoyY4XR3nhhwra1CCHOBjIymplWqPaq2zzpbOH3WKDe37gyLMTVmvu3rshOu/lfcW vMngEOzfC0q9fac9l06RT4b9hFZwuZ3FmaF12PCo5ye7PXwrY1KYG7csakjzOL5SXTmT EsW9XW10iZJ0Hc7V+X6f5QJNu4MJGORKerOK0bmr0ptbUErEHiJYm3BD7/9SX2/42rTE RoIXyP1zxLlqmQaw9zzNvkTcqadnvK3uidnZrLCPBveafDykY3lIEg25tLDSuBP1wxki tI6n5UUpJp9qpXBJZPrN/5t7KqJFNWh1f/9ckDqvvArNymik8KC6YWSbfF0j4FEg0tRp WCnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713724118; x=1714328918; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NrM2dGN+EheWjKnt5zlJvOk5v0Ei3knf8+WuyH2chzo=; b=EHhVklqCoBUBNogUYwmvH32kCGcbQfTk+3Dse8Lx76jggMjb6J6cWX2A119vv63o/+ Vv91Ya6Og9q/KGVeNGP9yzVkiTbm8ctkbCige5FeoUdxHILj0hZ7QoWiW4Zt9fpPm/IE wG2z7ndKgyCy2Ez1HJz62gebgorkH5ZTzu//T5kAf+KAKJDuDP651MrFOZCI6tGRqiUN 05Qsv2HJ1mGzTHMWf/Cz9J9CbIZGWg1iHcOD0u5Nsjm/eb2v7UpBGs2XbBAn7DOxdppN GWajNMhHDrQGZkZp/Re7U78CWk+bI9VNusiWYBmubm0jAb0YRlIoSc1TMdPia2QJ53a8 5pAQ== X-Gm-Message-State: AOJu0YyXy3GrTWgZKPBmXiWscBQmaD9dmQs4fVLIgX1o1tMedO1i8sb+ 97VEqTQTGZTXth6jynPK5Ohjn3Mqqzv6cUuz1uo98dT/hq9jTb9Y1fPnEXSY2ao= X-Google-Smtp-Source: AGHT+IEh4egod2woZlvsaC0LomDmn5pY2lASSJpITdB0Oc8MC5cFfYxA/L8olJpiH4vs6LxsOlev1w== X-Received: by 2002:a05:6512:1319:b0:516:d0c2:3ff9 with SMTP id x25-20020a056512131900b00516d0c23ff9mr6886919lfu.67.1713724117939; Sun, 21 Apr 2024 11:28:37 -0700 (PDT) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id s14-20020a19770e000000b005188ea5d2b5sm1549641lfc.303.2024.04.21.11.28.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Apr 2024 11:28:37 -0700 (PDT) From: Linus Walleij Date: Sun, 21 Apr 2024 20:28:35 +0200 Subject: [PATCH v7 09/11] ARM: lib: Annotate loop delay instructions for CFI MIME-Version: 1.0 Message-Id: <20240421-arm32-cfi-v7-9-6e132a948cc8@linaro.org> References: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> In-Reply-To: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> To: Russell King , Sami Tolvanen , Kees Cook , Nathan Chancellor , Nick Desaulniers , Ard Biesheuvel , Arnd Bergmann Cc: linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev, Linus Walleij X-Mailer: b4 0.13.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240421_112840_978659_3AF8F6BB X-CRM114-Status: GOOD ( 14.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When we annotate the loop delay code with SYM_TYPED_FUNC_START() a function prototype signature will be emitted into the object file above each site called from C, and the delay loop code is using "fallthroughs" from the different assembly callbacks. This will not work as the execution flow will run into the prototype signatures. Rewrite the code to use explicit branches to the other code segments and annotate the code using SYM_TYPED_FUNC_START(). Tested on the ARM Versatile which uses the calibrated loop delay. Tested-by: Kees Cook Signed-off-by: Linus Walleij --- arch/arm/lib/delay-loop.S | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/arch/arm/lib/delay-loop.S b/arch/arm/lib/delay-loop.S index 3ac05177d097..33b08ca1c242 100644 --- a/arch/arm/lib/delay-loop.S +++ b/arch/arm/lib/delay-loop.S @@ -5,6 +5,7 @@ * Copyright (C) 1995, 1996 Russell King */ #include +#include #include #include @@ -24,21 +25,26 @@ * HZ <= 1000 */ -ENTRY(__loop_udelay) +SYM_TYPED_FUNC_START(__loop_udelay) ldr r2, .LC1 mul r0, r2, r0 @ r0 = delay_us * UDELAY_MULT -ENTRY(__loop_const_udelay) @ 0 <= r0 <= 0xfffffaf0 + b __loop_const_udelay +SYM_FUNC_END(__loop_udelay) + +SYM_TYPED_FUNC_START(__loop_const_udelay) @ 0 <= r0 <= 0xfffffaf0 ldr r2, .LC0 ldr r2, [r2] umull r1, r0, r2, r0 @ r0-r1 = r0 * loops_per_jiffy adds r1, r1, #0xffffffff @ rounding up ... adcs r0, r0, r0 @ and right shift by 31 reteq lr + b __loop_delay +SYM_FUNC_END(__loop_const_udelay) .align 3 @ Delay routine -ENTRY(__loop_delay) +SYM_TYPED_FUNC_START(__loop_delay) subs r0, r0, #1 #if 0 retls lr @@ -58,6 +64,4 @@ ENTRY(__loop_delay) #endif bhi __loop_delay ret lr -ENDPROC(__loop_udelay) -ENDPROC(__loop_const_udelay) -ENDPROC(__loop_delay) +SYM_FUNC_END(__loop_delay) From patchwork Sun Apr 21 18:28:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13637484 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 61AF1C07E8D for ; Sun, 21 Apr 2024 18:29:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=J+ffYSf5RvSyep6AjwWRenGOqQ/+ACjX8ofAoOZthww=; b=pDCo7YLbinrSvr GyQgUVn2+8Y9NvWffvKxVuZpJi/OkTkCLHzTUhwkhDyslOJnOfUmoVyESUN2sfJ9Q1hpTgVL/3yzE 6l/f2Za+AI4CK+qTrpAsRSayIhF7wLSowIXKcaqIfJepf697Y6pa4XdxxTw1/pcImFlpQ4yaKhz2G p2r9hiyinSM4byv/lbv8UOp+Hcjp5BUDY12ikZVXicD7YG+6b3WVD4gLcvBxb6RbQXj1M5/eTaGxJ pFCsj+phZnqEMHqeRsR/5vky4A5GpfU2P/Lenp9bsN8pP+e4LqRGNSijGfC8wj3BCD9DMaxZwjfK7 87yU0yccxBX6oEGpTcoA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvy-0000000B4Dp-3FkD; Sun, 21 Apr 2024 18:29:02 +0000 Received: from mail-lf1-x133.google.com ([2a00:1450:4864:20::133]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvd-0000000B43D-3IOh for linux-arm-kernel@lists.infradead.org; Sun, 21 Apr 2024 18:28:44 +0000 Received: by mail-lf1-x133.google.com with SMTP id 2adb3069b0e04-5193363d255so4835162e87.3 for ; Sun, 21 Apr 2024 11:28:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1713724119; x=1714328919; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=ecw0uWRNm3i8fpJCtFnOl72Sdbbv98c8IiHQj1PGVlc=; b=aW1tDkkDnaeO+8pYv0VqY/MXEInh7/ntHvFB/qgH7OHvDjt2x2ZcsxdoJKWz+22a16 mzOHm33GoSpwZu+xqF6RQRqUM3QJ0IWiuBm1HniqGw91t6RK7cUd3UlOArbMIY3Rttmb 5ukl1ctnmKrqxUNYByk5eGjP0mffFUfzAXuOrzvT4ospdfr78nU2VLkfXpUWx8AX4dAQ 44k7ytHt64GWx9P0l8qxl9EdG1b9b2rVOMAemdy+mmnMQ0QII4fZH0Xe61xNyXjJuR7V YxQ8DnAmrATZsPVtJBMNILinmcuCZEGK3V7uf6t1AbBiLZKHcMa2KAm6uzgIbUCyl4oL UjFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713724119; x=1714328919; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ecw0uWRNm3i8fpJCtFnOl72Sdbbv98c8IiHQj1PGVlc=; b=gVebIzoVj3O42TFBPNEUqEOTuAGq8j/eG2VuArm2OvSSksg3d3zxciaBbbfxgnDxcV xVSFnIpuwRBTyK+aWSiWGj7oHLXusWPcoh+fHw+eTv6Pde05o9YQPjbje6XF2GHC/mJP qDTPmQXFB2oS71fkWl0G4KnxqzUVorDe/U9gSaThM6H2a09AiGK7cilGqovbHdEPaJuH CZSq7JnpTN6Tn5uUXw5JKRifz0QUyUpxtcMc8RgT8HmgQNOTb+XtFSJiH0p9mRWHCkNE EfvOdjUxdTXHyArDCGdqKibRsMcvx2Aqz0hqkIj5ZOtrrzNo8DFWJ8EeacjdbW7ZmE+A GyJQ== X-Gm-Message-State: AOJu0Yy+cFoW52Qu3eLhj/hx142hwYc8RT3+dJGA0oPyX2r30h6NE5iH VvlDsMwVuSUa2psmb69kiEbqfJTJN8oxAqFirEonHUAM4hak7c/r/O73AxRjWMc= X-Google-Smtp-Source: AGHT+IFQEQOCTGYb+sj338RlX5DuJLbp/3EO6h8z6/lx8OWssMUSvRxd4tjvA+IzpvfzYdtFnvy/vw== X-Received: by 2002:ac2:41d6:0:b0:515:a417:334 with SMTP id d22-20020ac241d6000000b00515a4170334mr4597907lfi.46.1713724118796; Sun, 21 Apr 2024 11:28:38 -0700 (PDT) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id s14-20020a19770e000000b005188ea5d2b5sm1549641lfc.303.2024.04.21.11.28.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Apr 2024 11:28:38 -0700 (PDT) From: Linus Walleij Date: Sun, 21 Apr 2024 20:28:36 +0200 Subject: [PATCH v7 10/11] ARM: hw_breakpoint: Handle CFI breakpoints MIME-Version: 1.0 Message-Id: <20240421-arm32-cfi-v7-10-6e132a948cc8@linaro.org> References: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> In-Reply-To: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> To: Russell King , Sami Tolvanen , Kees Cook , Nathan Chancellor , Nick Desaulniers , Ard Biesheuvel , Arnd Bergmann Cc: linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev, Linus Walleij X-Mailer: b4 0.13.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240421_112842_152863_978886D9 X-CRM114-Status: GOOD ( 18.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This registers a breakpoint handler for the new breakpoint type (0x03) inserted by LLVM CLANG for CFI breakpoints. If we are in permissive mode, just print a backtrace and continue. Example with CONFIG_CFI_PERMISSIVE enabled: > echo CFI_FORWARD_PROTO > /sys/kernel/debug/provoke-crash/DIRECT lkdtm: Performing direct entry CFI_FORWARD_PROTO lkdtm: Calling matched prototype ... lkdtm: Calling mismatched prototype ... CFI failure at lkdtm_indirect_call+0x40/0x4c (target: 0x0; expected type: 0x00000000) WARNING: CPU: 1 PID: 112 at lkdtm_indirect_call+0x40/0x4c CPU: 1 PID: 112 Comm: sh Not tainted 6.8.0-rc1+ #150 Hardware name: ARM-Versatile Express (...) lkdtm: FAIL: survived mismatched prototype function call! lkdtm: Unexpected! This kernel (6.8.0-rc1+ armv7l) was built with CONFIG_CFI_CLANG=y As you can see the LKDTM test fails, but I expect that this would be expected behaviour in the permissive mode. We are currently not implementing target and type for the CFI breakpoint as this requires additional operand bundling compiler extensions. CPUs without breakpoint support cannot handle breakpoints naturally, in these cases the permissive mode will not work, CFI will fall over on an undefined instruction: Internal error: Oops - undefined instruction: 0 [#1] PREEMPT ARM CPU: 0 PID: 186 Comm: ash Tainted: G W 6.9.0-rc1+ #7 Hardware name: Gemini (Device Tree) PC is at lkdtm_indirect_call+0x38/0x4c LR is at lkdtm_CFI_FORWARD_PROTO+0x30/0x6c This is reasonable I think: it's the best CFI can do to ascertain the the control flow is not broken on these CPUs. Reviewed-by: Kees Cook Tested-by: Kees Cook Signed-off-by: Linus Walleij --- arch/arm/include/asm/hw_breakpoint.h | 1 + arch/arm/kernel/hw_breakpoint.c | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 36 insertions(+) diff --git a/arch/arm/include/asm/hw_breakpoint.h b/arch/arm/include/asm/hw_breakpoint.h index 62358d3ca0a8..e7f9961c53b2 100644 --- a/arch/arm/include/asm/hw_breakpoint.h +++ b/arch/arm/include/asm/hw_breakpoint.h @@ -84,6 +84,7 @@ static inline void decode_ctrl_reg(u32 reg, #define ARM_DSCR_MOE(x) ((x >> 2) & 0xf) #define ARM_ENTRY_BREAKPOINT 0x1 #define ARM_ENTRY_ASYNC_WATCHPOINT 0x2 +#define ARM_ENTRY_CFI_BREAKPOINT 0x3 #define ARM_ENTRY_SYNC_WATCHPOINT 0xa /* DSCR monitor/halting bits. */ diff --git a/arch/arm/kernel/hw_breakpoint.c b/arch/arm/kernel/hw_breakpoint.c index dc0fb7a81371..93f9034e4e72 100644 --- a/arch/arm/kernel/hw_breakpoint.c +++ b/arch/arm/kernel/hw_breakpoint.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include @@ -903,6 +904,37 @@ static void breakpoint_handler(unsigned long unknown, struct pt_regs *regs) watchpoint_single_step_handler(addr); } +#ifdef CONFIG_CFI_CLANG +static void hw_breakpoint_cfi_handler(struct pt_regs *regs) +{ + /* + * TODO: implementing target and type to pass to CFI using the more + * elaborate report_cfi_failure() requires compiler work. To be able + * to properly extract target information the compiler needs to + * emit a stable instructions sequence for the CFI checks so we can + * decode the instructions preceding the trap and figure out which + * registers were used. + */ + + switch (report_cfi_failure_noaddr(regs, instruction_pointer(regs))) { + case BUG_TRAP_TYPE_BUG: + die("Oops - CFI", regs, 0); + break; + case BUG_TRAP_TYPE_WARN: + /* Skip the breaking instruction */ + instruction_pointer(regs) += 4; + break; + default: + die("Unknown CFI error", regs, 0); + break; + } +} +#else +static void hw_breakpoint_cfi_handler(struct pt_regs *regs) +{ +} +#endif + /* * Called from either the Data Abort Handler [watchpoint] or the * Prefetch Abort Handler [breakpoint] with interrupts disabled. @@ -932,6 +964,9 @@ static int hw_breakpoint_pending(unsigned long addr, unsigned int fsr, case ARM_ENTRY_SYNC_WATCHPOINT: watchpoint_handler(addr, fsr, regs); break; + case ARM_ENTRY_CFI_BREAKPOINT: + hw_breakpoint_cfi_handler(regs); + break; default: ret = 1; /* Unhandled fault. */ } From patchwork Sun Apr 21 18:28:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13637486 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2DB0BC4345F for ; Sun, 21 Apr 2024 18:29:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=0TbLH7wKKbrcYnx9DbMBx4XVWZuvFXrho3TkRkJPtqQ=; b=Bb2/EBwV5tR+4B NrGZXyDq1TsxVlPdEiWh2FZf+eWBL1VH+jhZgkSgL7hyIsTRSHxnvnx9GpDEcQ0OET7jPYneO+Vg7 h7rbzgWj/PMNMi2NGdFOayDgZ8TFl/708wzIiBz0WMJGKCeOtEc8UOjuR9zL0iFFCpuMG7ScD4s26 YxZ/qDviw12GM/Vdfq8oMWXGCTcVaU3U6uDO1H/U8sj2OJlMnrl+RCeNfTg7uEiZqZylPQO6E4QnA YWRh2FcjQCrrkiWQee28GDfSaU/pYZf0+CD85s38DJwWsli6t2jbqRtrnuF6MNA7u93VXit4hFlns uGRpzkOym11rESQxPQqg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvx-0000000B4Ca-0Q2Z; Sun, 21 Apr 2024 18:29:01 +0000 Received: from mail-lf1-x132.google.com ([2a00:1450:4864:20::132]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rybvd-0000000B43t-0jos for linux-arm-kernel@lists.infradead.org; Sun, 21 Apr 2024 18:28:44 +0000 Received: by mail-lf1-x132.google.com with SMTP id 2adb3069b0e04-516d2600569so4629786e87.0 for ; Sun, 21 Apr 2024 11:28:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1713724120; x=1714328920; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=Y70OTHVL5ivIan0b4DqgD01dRiya5SKhfRNunxoXdLs=; b=MUqHVgI96PXD/GMoLIxnYTLQ486YUtGODWq5OXXkhol7TndZkCqv09uQwmM6o6DqVV h32EsKV4QIfPQef9Q8OYf7fMuDPW868w2mrvXbecrQ0M4j82H00psMSfRJff5vZEJ8ec nJp32R7AFBMsrXCErHThuCESGDG0wurgIszGzyjBfpDoyXK2CFlram8xWMv0hXKlvsVm W3EeVSG8Ma+EVExACurT25iQuDAFUXnN8+IE4y0WZ8yB3ZVdJe373TLfsFQSSOtc+vYl t8GG5kCmxJe6V8rfmJlJrZwZ7qoJpsbC6TF48lFk/G3t02AUpXblK/jHdZXqToC7bf3Y wJZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713724120; x=1714328920; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Y70OTHVL5ivIan0b4DqgD01dRiya5SKhfRNunxoXdLs=; b=qakpNyAF6r16klApnlSIobLNtOt+16MjctiHQgQiEu98dkqmegfCLKaooWkaEQqA2M 7CcjRjz9GchugowVFoB5bynC0JJcWIlZnzVK4ZsSEzgU6nNQKTd+btXTObl7XThdV/Ay p4W3JWhy3YHDzEFOXeQKt9JLI1odOefSAUMle43N5LX7/hMG9tO2mKylYGe8sCOhI3uU Kg216a2Et8KpzwWa14eaw8oSjTwxfuJ8k/J/zpVqnemFSDaVDBGbtQC6/0mA8k3BeJKC z/I3YDCc2AT8KNnP7F3nU7FoI0wQJ5rwakvkCSe9mkwe+/0E1eOgOI4FR3cMyU1NrH1n feVg== X-Gm-Message-State: AOJu0YzUIHvx9glCuTG7kx6u3+izgEnpeGijr/PtcR42hhaq61B1RYiW yxJMpOphqm3vr6OnKabPot+sLXHKYK4yp5cJ4obfiULo/oo6UafZy9nFBtl0FqI= X-Google-Smtp-Source: AGHT+IH7reJlxAOcCyXD+MSHuuDYG0YRVdcMUSgTIS46/BdvW/4HC0w9owbY9Cg+Mub5s+Jo3qDzbA== X-Received: by 2002:a05:6512:3054:b0:516:e8b1:177d with SMTP id b20-20020a056512305400b00516e8b1177dmr6437575lfb.65.1713724119734; Sun, 21 Apr 2024 11:28:39 -0700 (PDT) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id s14-20020a19770e000000b005188ea5d2b5sm1549641lfc.303.2024.04.21.11.28.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Apr 2024 11:28:39 -0700 (PDT) From: Linus Walleij Date: Sun, 21 Apr 2024 20:28:37 +0200 Subject: [PATCH v7 11/11] ARM: Support CLANG CFI MIME-Version: 1.0 Message-Id: <20240421-arm32-cfi-v7-11-6e132a948cc8@linaro.org> References: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> In-Reply-To: <20240421-arm32-cfi-v7-0-6e132a948cc8@linaro.org> To: Russell King , Sami Tolvanen , Kees Cook , Nathan Chancellor , Nick Desaulniers , Ard Biesheuvel , Arnd Bergmann Cc: linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev, Linus Walleij X-Mailer: b4 0.13.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240421_112841_435536_F3C59A08 X-CRM114-Status: GOOD ( 10.99 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Support Control Flow Integrity (CFI) when compiling with CLANG. In the as-of-writing LLVM CLANG implementation (v17) the 32-bit ARM platform is supported by the generic CFI implementation, which isn't tailored specifically for ARM32 but works well enough to enable the feature. Tested-by: Kees Cook Signed-off-by: Linus Walleij --- arch/arm/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index b14aed3a17ab..df7bd07ad0d4 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -35,6 +35,7 @@ config ARM select ARCH_OPTIONAL_KERNEL_RWX if ARCH_HAS_STRICT_KERNEL_RWX select ARCH_OPTIONAL_KERNEL_RWX_DEFAULT if CPU_V7 select ARCH_SUPPORTS_ATOMIC_RMW + select ARCH_SUPPORTS_CFI_CLANG select ARCH_SUPPORTS_HUGETLBFS if ARM_LPAE select ARCH_SUPPORTS_PER_VMA_LOCK select ARCH_USE_BUILTIN_BSWAP