From patchwork Mon Feb 3 21:49:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ankur Arora X-Patchwork-Id: 13958311 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BF683C02193 for ; Mon, 3 Feb 2025 21:56:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:Content-Type: Content-Transfer-Encoding:References:In-Reply-To:Message-Id:Date:Subject:Cc: To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Y2i0UFtOm8h8GVwyJybfRKu5genIQj3zDzS5If8eYcE=; b=dzmpV/GCUCZ//fF+QfwvIgcJz0 1I8j3gVPvFt9ofwqUdZZ4k1dzhY/GXc98GvHRsxP08euosZUP2Vt9ghY9lsuQuKD7onzMM+MCGXcV GggCne5KgzKI+lqZo4r3MKA8EktV9uFgRi3g59HTVVVa2+bNqSWQy3ZSgpCRunuuU+NrgZJXV7+IL naISYwiHpPRm1c4djvi6vNUj4d43vGqHPHxm8xkmgMWOw5HLIrd2dzN9TzNrkYEuIGflisDNXuQli a3dddyZyxp3T+JfIroKNoeOMgbHuqasUuAX7KMYoMvuj1wt/CxPlCDOaYW2he8qPIaSNxMdfe9oTd LQOL6ymA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tf4QZ-0000000GjwY-3LCr; Mon, 03 Feb 2025 21:56:23 +0000 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tf4K5-0000000Gitu-4Ahr for linux-arm-kernel@lists.infradead.org; Mon, 03 Feb 2025 21:49:43 +0000 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 513JMtHA001047; Mon, 3 Feb 2025 21:49:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=Y2i0UFtOm8h8GVwyJybfRKu5genIQj3zDzS5If8eYcE=; b= Q6dWTm99796JurSr6TL+oxA/jTsEBYfM4BzEgdi0k7xXu9N5Q5ioJTRHOoogjkLY HM+jk+0LL9HTQhnah+7iFcdaqSPpmY24oOSqYk6znj0lKxJCZr09QWvu06djqUCt HnazvbyxA8fkrjpKltf3pFyV5LNKHcXasec23Mf3/f8Sejl29cTlJwyRZkgpb3ZO +tZmmLQFWbyLSgweXA3VFtzUtYz/TLzcMfwX3oi2+TCpsVfHkxyUD21lwjpJ45Wq 3ZCSwdOk62OldaWdxtWRqfZL57ZQKr01L3bmAUm1VXPEXLAYkAQTCCBeLrP5gVWE q9IkBwne0UVORukNXwPCHw== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 44hhjtumu1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Feb 2025 21:49:27 +0000 (GMT) Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 513JlCRS029079; Mon, 3 Feb 2025 21:49:26 GMT Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2170.outbound.protection.outlook.com [104.47.59.170]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 44j8p27131-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Feb 2025 21:49:26 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=H62RyAECqAINgClBxFMFWRN/z/ge1hu/f/gKMwAy3eZZIFm3I1ceCLhWrF90oA+kOnDXVrct/yKDu/m0pIJIedq/rHkvDbhFlCz5ZdGEiSGe6ksaS9OYxn9AipxL64Pr4a2m+gqraxLfIIhZqwbs8NnJb8upwDmiVganv3qzSQmALaoz+jzpTBEHyej3jym3KwcBs1GkUimhlVTem84aNOMtZzx0YPB242Zrx7fXfvYkpFvFT3pi0pHJwmlgJiBkKcXxdZVn5tLGGdKksxjj5ABZnrPYK8iCSVWHkj5/7xW3/4pclEt9cE0TvrTAEOLg8ttgEHQkJOBbNMnb79UGuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Y2i0UFtOm8h8GVwyJybfRKu5genIQj3zDzS5If8eYcE=; b=YThBnri1nYjmmNxRENmjnhoIFW7ff9LqAyAZN1n3OSQarzBMS+nv3QB9IMqoT2B21oCFaUPw/XUDslwy+ScrSc5Ugjk/7QkUtqYM3xGH7ARGiuYNL00SO4Ba3xXOKLwxDV9HkG1vEFI4bKVbM+Qd/7CcAOTMpTqR88Rg9OENNqeysjgHGu0o0qClsml/+kzbgtSRQXQe10oDNROjInXi3zwns5CFi2YEf335Etuq83ImwYLCxBhYXymTraL6ugm9eQJ2n4WTtGhctmBbZ+g3aPWqVYiAGUahjC2fJcHDNI4eILxcL3Afgts4XjbtTOEC0PLq7juV/F4AJlfTa1sxag== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Y2i0UFtOm8h8GVwyJybfRKu5genIQj3zDzS5If8eYcE=; b=Oq8TvaAZC8Ay9/TtYiwhfDW/r+CkSAt0VNWNp8Jem9hq7N+M3r7XpzK1CI875Eig4oAdEuurGj/KHAavJHAIa85Kz4T+KooAlZdlCmA2qVYoDSHYXWByRzd50wVFM95SbCe/FnfxpWkf9T3xTkgTOnlYGr3AV89l3KiZaBJsKZw= Received: from CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) by IA1PR10MB6854.namprd10.prod.outlook.com (2603:10b6:208:425::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8398.23; Mon, 3 Feb 2025 21:49:24 +0000 Received: from CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::25a9:32c2:a7b0:de9e]) by CO6PR10MB5409.namprd10.prod.outlook.com ([fe80::25a9:32c2:a7b0:de9e%6]) with mapi id 15.20.8398.020; Mon, 3 Feb 2025 21:49:24 +0000 From: Ankur Arora To: linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: arnd@arndb.de, catalin.marinas@arm.com, will@kernel.org, peterz@infradead.org, mark.rutland@arm.com, harisokn@amazon.com, cl@gentwo.org, memxor@gmail.com, zhenglifeng1@huawei.com, joao.m.martins@oracle.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, Ankur Arora Subject: [PATCH 4/4] arm64: barrier: Add smp_cond_load_acquire_timewait() Date: Mon, 3 Feb 2025 13:49:11 -0800 Message-Id: <20250203214911.898276-5-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20250203214911.898276-1-ankur.a.arora@oracle.com> References: <20250203214911.898276-1-ankur.a.arora@oracle.com> X-ClientProxiedBy: MW4PR02CA0021.namprd02.prod.outlook.com (2603:10b6:303:16d::31) To CO6PR10MB5409.namprd10.prod.outlook.com (2603:10b6:5:357::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR10MB5409:EE_|IA1PR10MB6854:EE_ X-MS-Office365-Filtering-Correlation-Id: 7799147e-1bc4-4615-fe36-08dd449c9f90 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|7416014|366016; X-Microsoft-Antispam-Message-Info: PXCxF+i0+HHqAwZxw1ZxAK9Bo0YvVsnat64FCrYWCsNsIgi2VG/WQq6/Z20OD8/S20R0iDM635vna9/pFTfrdi7L0xXIjUgOYNYv9a+//sP1Vs2fRIgSRfILAW8VwrHkESZQtVa1O4hek0yqhnBTu+Y5c5Sq0QLPimbexL+pjCObyt3GGEJcpz5Lj3m1nxytUqaJR+DV3FeS/98Jd+vbJbpukM2iRAsDpEYusMDJZLSUy7At9/cPQgTNO8ulAHXV9nDfZrtt3DYuHi1RHNulLu5mdwM0iwTsSMdCArRLmwdX5jQmUIK00bPcbvyz/Zm6P4EbgVLiePWZ29S80bjziCQ+UUkiZbV4bqSnnH40GKPmfP/baBcySE+3wsrzPxYBwn7QwNLyFAMEe5BFSY2seJBNs9vnPCzfyAB9bV6IrpQkaOGVgb/GEJniS7claHFnqDs9nrYSyHBM6HvUJ79d6uX0b9u4nB9HEgLEC3bBgNlQqwe6UScgzxLS+qgD0ttF6hTAqO80LsP08mPmgv8gRpCn7D3IJQiX9xHeMykgTDM4sV6v9Jo9MW4l6GTfkC7S2ZrcsbTEdWQHqRvE1FvB4EvUz82oOgV8mqqf74Y3kL+cx6Nz+FAooDGq9vPU2q4Lj2QUx2e+goiZhtsaydgWRg5PJkBR18wGUacd1zhfZH30X3yJ1yLWWThn6PLrNBhdvsmg1Y/7ghPi+qdOJfBwHBK6kYTQaVC3h6jlTXX56p/AM+Qnkx0XRe5zv1ODNW2sSEYjZYy/yOvlGJCe1RqWziopbzSQqLK95xe1I8qcu97/Tn37f0Pl96RoeCksXAO7bx2hB4p5UQnLqWRZnZtsSPtAO9tAnzLL0BlJmvnIuEFbRdcMHyAaQzBMPEYe+pPhLdgR2IcOIbinS02YVNx4XEhAzv/obokUuxmQCL0ldXt2UK1jR8Q8QqkgeZkZXku7EYzQMv3ci8xBE/4GAv4znrCJvHXJtmwT8GRlh8S73tB3+1wh/wOuj5bfKNm+VbZWXtzgE8UUnB7TeQenLjN6N3EGwEk9Z06rYHa0EGrTSxM4T2l04doDcoTkKLFzRA/LwDdykL8dw3xqPzVglN9I+txpOTwWtUt320tCk0kEPTB5HpXnTAnubpcovb0hBBva/5FjCFqY+lfKBQSPv9jaTt6UoO8MSdLexbc32YBKDKfR2YS1Fs1pkSh+8a88JqQJVuEUeFMi7JcQoSP4mvWeSs7f/crrtdI0oF/Uiv+MZRoA4jpybt7m75WhW5+HSpkOkn++lxNcA543AubVW6XrGTXhJORuUEdDfTIqn0or1ilHJdanXWLqHxSNFc9hDo29R6Ja40+2Ve/L7/07/bjRV3HSvXPWpbmdK4Hbd0HZHCP9C3FeRbPXlJFal9ohftST X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR10MB5409.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(7416014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: oEz5ijhQcIOV3BCEwrxpcTN/CrWj1tD3gSCK3ui6WMiZyxvHQXMexTNTSr7KsC5Nxz5NS+0f5cykgLp9dZbIQPeIR3bmZXlrcScIDQO1NXIRTLvSnOXX2GMSi3tJ5rLQ/h+QcA9FULgSrOB5SxU4QM28QrrjLCrXQ8ZsPP7FzEk7TkUM+HifPwzapKvvwRJTLIwGXhqEf0/nPRblKOBB2HHkeOoLkfjawlCSP7Dxqar0QSpuVUdVPCyvsLMUdV3OKrSiPhGofCi3RUF0+5q2Q1a/jiyRJNlbbkID9j+MySdeYC1RurfD195o6G3BGywBe33YBG094/4tybXGf3akdYXnqh6DqGvlWQH8A13IKmz/x/OVNedj6EoL+39jFqP5+jD78Hnlon13mv2BBElK3xkjE5r4e/QdvbbUUXWxzAByxT57kb3U7nDYSwWDtyjiCwgDwvfDZ6XW55pI+zzV+jvQlYZh/xvaAxGT0E2mok4akmExY9vGX0xzTVSCKe8cB4eRu8ZjMNYjPNrX8iSwOVO8vjfI43pdvuvv6DwihE4wxnaF+bZb4mORrwTgj5QEzTY09/0ENJZNkAPGoeZTZQzc9y+aumAl13ZTw4a6GGChQtAYT3DA/tH6U6TyAt8vJTsyragbTuyPU2WCpELpStwdavjV+e66uCGmJOdSV7NSHQpfG7+6ZQu7l6onfkI1A96NblyzJpSMkA9CcLub+js/tVVvQTfuUjdEoisusKOb7zKZGU1OB9IwMHwn2Z06mAR7pxVNh1gVNBMgYdTQD0C2CbY8DytdAMfDOZ7xJlUl+Co1wMCg5g+prIGt3GwE3XzANDevM+RwUFhBhYjmJ5nxrm8StFC27P+oJKvkY4TNs/0FvQ3JeEytVZe8/p16qsBG/ptnstPb13ebxgBYrKyg6vaJ2jJkyphzD5dGwAslEnHWKfST7M/dNrjn3/288PJBTwKX4hwRfYjOMJN/MteGdwagkMSwmHQ//wr7WP2+9ZeLHGJW2f5QtIl6ub8606Aya8rmTb/3ieesPcnBnWCZ8SVMOgGxthUB77sp7jDJFEsPuY727BwEHwAPBTzdIy3GtNSoa1OMcXIvGTRPEunQaT0AoCD6Bgd9rFu6HH7tjnWjTuD3KY1M/df95Xm3BGg4DYu6YtKwr5hj7zRVO+7ITLycnDfjuA7J0bEWxSYu3no9ZQ2k7QDU16Y4yrFiJaZ0rggMFRIoBgyo4J6YadbhClJs0RhAKf1/i/+cXxbJewvlt4umNPJs/5AoYGJJBJfMNyPIU4Pyb5kX6C+TdLLxUWGArIWDA5uzrWjxNbjDa5mYjtCs+uGC9BDXapJRkzru5d0Nac0V6QStYN0HtDrS/Skurt6tMyPZVvfnAdgdftOV2WybmzYjkTEBZVKYyK1FQiBK8Yp7+URLbRj2XK8rPbEfr1OKXTLZiWKYZBJ4sNDHObLT5xQ7S03Mhc3R1gY+eYBubl50RaICnXMW91dNn/mTY+vcRdaMXxCiqh/yIbbiXkoyWJ9xJE0QjqexhBGYCuEqJqI46FcKZS61/Q5RLKB02S/tOmqqL7aHqrMzZ8wX4bmHw07yS6W/X26pFhb3m1GDD3qlC0FzemxjnQ== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: lifen2R5PDmmQ+p07tI7YPnmmaP7Z0QR+q5T7yFh23387tbXPkjqbwAQtSNVkomIiXqS5wy2klh6myNO339Q9QkcgOrExmC7Zj7z5yWX+uDrFGmpGrh5WB9b4cqo0ogZdu4IKOy6OK0PPbUyt2rEDwlZiQTgRGW4Jkag2+u4fz2rcRCg/5FgdRgSOEn4a4rpPLlm8GymvL7DPTeWaeAwt1uITz5o/Db/BHkoRccCrsqvSIP+z0iofhvvrxgELIM6T+b1qU25OLf6NQqFOzO3fo/C4uPDCH02SdDUtwLWFZXuKv/YdjXqbVaKL99uNS9ReEswsR9Q5iBs7GyzUYFziW02IUZFGxQqD5QVYxsyoNfWLcbKOH79O80jNId+2YLtBUfd4QCyxckj0CW19o/2mgkeywLtoqLod1crS/Yi8d85O2OSRQBAH74oOjLqmDprZcx45REQgcrsmiRrUqKsSNzcSUVpuGVsQLQW3P816wqEyjMcdtdbHFM5K0UjP1b9/eb+0H8J2OKGvlz3B45xhCmDuOVJTyDK44IkQXRmpyrxHlddJBNgFkaXjaUjMygjRaWmD8czpcsT7RogxSg6MnXPbIvRb6zYDMZPg/0hN18= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7799147e-1bc4-4615-fe36-08dd449c9f90 X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Feb 2025 21:49:24.0606 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nVQsUxNj0ehuZCTwo5ZcJJXGQSGaUJ/ZyzISYNe1s21AWJRHg8FB9OO9ywBdlUk2rKdSRvADtHrJU7CmjsEM+4rYuqMISbBkULBBEz8n42c= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR10MB6854 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-02-03_09,2025-01-31_02,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 phishscore=0 suspectscore=0 adultscore=0 mlxscore=0 spamscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2501170000 definitions=main-2502030158 X-Proofpoint-GUID: QdacrseF6w2tloYXDR-VjdGOJRI2ZTnZ X-Proofpoint-ORIG-GUID: QdacrseF6w2tloYXDR-VjdGOJRI2ZTnZ X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250203_134942_033162_A0574DFF X-CRM114-Status: GOOD ( 14.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add smp_cond_load_acquire_timewait(). This is substantially similar to smp_cond_load_acquire() where we use a load-acquire in the loop and avoid an smp_rmb() later. To handle the unlikely case of the event-stream being unavailable, keep the implementation simple by falling back to the generic __smp_cond_load_relaxed_spinwait() with an smp_rmb() to follow (via smp_acquire__after_ctrl_dep().) Cc: Will Deacon Cc: Catalin Marinas Cc: linux-arm-kernel@lists.infradead.org Signed-off-by: Ankur Arora --- arch/arm64/include/asm/barrier.h | 36 ++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h index 25721275a5a2..22d9291aee8d 100644 --- a/arch/arm64/include/asm/barrier.h +++ b/arch/arm64/include/asm/barrier.h @@ -232,6 +232,22 @@ do { \ (typeof(*ptr))VAL; \ }) +#define __smp_cond_load_acquire_timewait(ptr, cond_expr, \ + time_expr_ns, time_limit_ns) \ +({ \ + typeof(ptr) __PTR = (ptr); \ + __unqual_scalar_typeof(*ptr) VAL; \ + for (;;) { \ + VAL = smp_load_acquire(__PTR); \ + if (cond_expr) \ + break; \ + __cmpwait_relaxed(__PTR, VAL); \ + if ((time_expr_ns) >= (time_limit_ns)) \ + break; \ + } \ + (typeof(*ptr))VAL; \ +}) + /* * For the unlikely case that the event-stream is unavailable, * ward off the possibility of waiting forever by falling back @@ -254,6 +270,26 @@ do { \ (typeof(*ptr))_val; \ }) +#define smp_cond_load_acquire_timewait(ptr, cond_expr, \ + time_expr_ns, time_limit_ns) \ +({ \ + __unqual_scalar_typeof(*ptr) _val; \ + int __wfe = arch_timer_evtstrm_available(); \ + \ + if (likely(__wfe)) { \ + _val = __smp_cond_load_acquire_timewait(ptr, cond_expr, \ + time_expr_ns, \ + time_limit_ns); \ + } else { \ + _val = __smp_cond_load_relaxed_spinwait(ptr, cond_expr, \ + time_expr_ns, \ + time_limit_ns); \ + smp_acquire__after_ctrl_dep(); \ + } \ + (typeof(*ptr))_val; \ +}) + + #include #endif /* __ASSEMBLY__ */