From patchwork Wed Aug 28 22:03:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13782016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A0A78C71150 for ; Wed, 28 Aug 2024 22:04:13 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.785165.1194579 (Exim 4.92) (envelope-from ) id 1sjQll-0006ME-Ba; Wed, 28 Aug 2024 22:04:01 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 785165.1194579; Wed, 28 Aug 2024 22:04:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQll-0006L2-58; Wed, 28 Aug 2024 22:04:01 +0000 Received: by outflank-mailman (input) for mailman id 785165; Wed, 28 Aug 2024 22:03:59 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlj-0006E8-R2 for xen-devel@lists.xenproject.org; Wed, 28 Aug 2024 22:03:59 +0000 Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [2a00:1450:4864:20::534]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 6c704332-6589-11ef-99a0-01e77a169b0f; Thu, 29 Aug 2024 00:03:57 +0200 (CEST) Received: by mail-ed1-x534.google.com with SMTP id 4fb4d7f45d1cf-5bf006f37daso18895a12.1 for ; Wed, 28 Aug 2024 15:03:57 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c0bb1c2d2esm2695898a12.16.2024.08.28.15.03.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Aug 2024 15:03:55 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 6c704332-6589-11ef-99a0-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1724882636; x=1725487436; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TA9zJhMMiP1cl4HW7H7RUH127emY4xJNbEczm6wLMBU=; b=lyzXYIpTlZfTLoaktw/83gq7NnZCWtARmJkEIoQg5D4Pib7zbZpYyn2u5yNA7FpXnz tgBz+3kgRjR0N/XlEzTqG5XYZGp365TraSWmolI/7gjXgeE8C8TZDB/VEqIXXYGxbtx8 zlYVTJKqy0j3GqSJmgtowx7G+ZG9qmpb/nU00= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724882636; x=1725487436; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TA9zJhMMiP1cl4HW7H7RUH127emY4xJNbEczm6wLMBU=; b=hCh7KmDEMvZnpyoauCDBT2Fai5iMShq726JkLH3u2aTt4FsiOio8JEE3nYZ3S6NKxn 4Pakmnjv1PGhr8DZIr5B2N+Lw/dlmCrgY+7JInTd0iDNOnCRFOHpTYZqyVLoOnsoMKlO SMpK6Y0/zsMkRxc1rgRpW+fSFzwG9JxcatwV/G4zZYIG/Xc7CajEmU5xQZSlbTzOWXID +1mIMd4vUxSqQ1MLUq/QdKIg5TcjNH5c/fjqf37xwJbS2mLxP0IEB2J41FlwYGj+MnbM DPbssmse1Q4whgtutIwahJMa7s6vijoprRZ9E6CcGHEyg3Oo5UHUem7/9/hfxJTKVnpU ugUg== X-Gm-Message-State: AOJu0YwOD4rO/GR266l1+QibihGXDwHUDLwBpHkP/q1Y+LarA0qbbXIi ecVEifQvb4UW2wgGfB5OdQztNtkI0mgOOqovnzpwLAj1Ws8g8MmEB+EK2kl+oOr7VgH+Z4SWuhI l X-Google-Smtp-Source: AGHT+IGD9DKTjQXlDEnryzLj/nD4dCJYNsZnTxN+I9P6dA3O2JfEGef4Tj0eX54rIlOSfqwvDZUzNg== X-Received: by 2002:a05:6402:3506:b0:5c0:a8b8:dd6b with SMTP id 4fb4d7f45d1cf-5c22018ad3dmr620843a12.14.1724882636093; Wed, 28 Aug 2024 15:03:56 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Bertrand Marquis , Michal Orzel , Oleksii Kurochko , Shawn Anastasio Subject: [PATCH v2 01/11] xen/compiler: Rename __attribute_const__ to attr_const Date: Wed, 28 Aug 2024 23:03:41 +0100 Message-Id: <20240828220351.2686408-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240828220351.2686408-1-andrew.cooper3@citrix.com> References: <20240828220351.2686408-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 There's no need for the name to be so verbose. No functional change. Suggest-by: Jan Beulich Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Stefano Stabellini CC: Julien Grall CC: Volodymyr Babchuk CC: Bertrand Marquis CC: Michal Orzel CC: Oleksii Kurochko CC: Shawn Anastasio v2: * New --- tools/libs/guest/xg_dom_decompress_unsafe_zstd.c | 2 +- xen/arch/arm/include/asm/arm64/cpufeature.h | 12 ++++++------ xen/arch/x86/include/asm/byteorder.h | 4 ++-- xen/arch/x86/include/asm/endbr.h | 4 ++-- xen/include/xen/byteorder/swab.h | 6 +++--- xen/include/xen/compiler.h | 2 +- 6 files changed, 15 insertions(+), 15 deletions(-) diff --git a/tools/libs/guest/xg_dom_decompress_unsafe_zstd.c b/tools/libs/guest/xg_dom_decompress_unsafe_zstd.c index 7cd266444bb1..ff45732a3616 100644 --- a/tools/libs/guest/xg_dom_decompress_unsafe_zstd.c +++ b/tools/libs/guest/xg_dom_decompress_unsafe_zstd.c @@ -22,7 +22,7 @@ typedef uint16_t __be16; typedef uint32_t __be32; typedef uint64_t __be64; -#define __attribute_const__ +#define attr_const #define __force #define always_inline #define noinline diff --git a/xen/arch/arm/include/asm/arm64/cpufeature.h b/xen/arch/arm/include/asm/arm64/cpufeature.h index d9b9fa77cbd2..1bb503e857e9 100644 --- a/xen/arch/arm/include/asm/arm64/cpufeature.h +++ b/xen/arch/arm/include/asm/arm64/cpufeature.h @@ -44,25 +44,25 @@ struct arm64_ftr_bits { s64 safe_val; /* safe value for FTR_EXACT features */ }; -static inline int __attribute_const__ +static inline int attr_const cpuid_feature_extract_signed_field_width(u64 features, int field, int width) { return (s64)(features << (64 - width - field)) >> (64 - width); } -static inline int __attribute_const__ +static inline int attr_const cpuid_feature_extract_signed_field(u64 features, int field) { return cpuid_feature_extract_signed_field_width(features, field, 4); } -static inline unsigned int __attribute_const__ +static inline unsigned int attr_const cpuid_feature_extract_unsigned_field_width(u64 features, int field, int width) { return (u64)(features << (64 - width - field)) >> (64 - width); } -static inline unsigned int __attribute_const__ +static inline unsigned int attr_const cpuid_feature_extract_unsigned_field(u64 features, int field) { return cpuid_feature_extract_unsigned_field_width(features, field, 4); @@ -73,7 +73,7 @@ static inline u64 arm64_ftr_mask(const struct arm64_ftr_bits *ftrp) return (u64)GENMASK(ftrp->shift + ftrp->width - 1, ftrp->shift); } -static inline int __attribute_const__ +static inline int attr_const cpuid_feature_extract_field_width(u64 features, int field, int width, bool sign) { return (sign) ? @@ -81,7 +81,7 @@ cpuid_feature_extract_field_width(u64 features, int field, int width, bool sign) cpuid_feature_extract_unsigned_field_width(features, field, width); } -static inline int __attribute_const__ +static inline int attr_const cpuid_feature_extract_field(u64 features, int field, bool sign) { return cpuid_feature_extract_field_width(features, field, 4, sign); diff --git a/xen/arch/x86/include/asm/byteorder.h b/xen/arch/x86/include/asm/byteorder.h index e935f7b2b03b..a877c07f6796 100644 --- a/xen/arch/x86/include/asm/byteorder.h +++ b/xen/arch/x86/include/asm/byteorder.h @@ -4,13 +4,13 @@ #include #include -static inline __attribute_const__ __u32 ___arch__swab32(__u32 x) +static inline attr_const __u32 ___arch__swab32(__u32 x) { asm("bswap %0" : "=r" (x) : "0" (x)); return x; } -static inline __attribute_const__ __u64 ___arch__swab64(__u64 val) +static inline attr_const __u64 ___arch__swab64(__u64 val) { union { struct { __u32 a,b; } s; diff --git a/xen/arch/x86/include/asm/endbr.h b/xen/arch/x86/include/asm/endbr.h index 3033e40d29a8..ba3bae714787 100644 --- a/xen/arch/x86/include/asm/endbr.h +++ b/xen/arch/x86/include/asm/endbr.h @@ -19,7 +19,7 @@ * gen_endbr64() is written deliberately to avoid the problematic operand, and * marked __const__ as it is safe for the optimiser to hoist/merge/etc. */ -static inline uint32_t __attribute_const__ gen_endbr64(void) +static inline uint32_t attr_const gen_endbr64(void) { uint32_t res; @@ -45,7 +45,7 @@ static inline void place_endbr64(void *ptr) * contain an ENDBR64 instruction. Use an encoding which isn't the default * P6_NOP4. Specifically, nopw (%rcx) */ -static inline uint32_t __attribute_const__ gen_endbr64_poison(void) +static inline uint32_t attr_const gen_endbr64_poison(void) { uint32_t res; diff --git a/xen/include/xen/byteorder/swab.h b/xen/include/xen/byteorder/swab.h index b7e30f050385..9f817e3c4492 100644 --- a/xen/include/xen/byteorder/swab.h +++ b/xen/include/xen/byteorder/swab.h @@ -123,7 +123,7 @@ #endif /* OPTIMIZE */ -static inline __attribute_const__ __u16 __fswab16(__u16 x) +static inline attr_const __u16 __fswab16(__u16 x) { return __arch__swab16(x); } @@ -136,7 +136,7 @@ static inline void __swab16s(__u16 *addr) __arch__swab16s(addr); } -static inline __attribute_const__ __u32 __fswab32(__u32 x) +static inline attr_const __u32 __fswab32(__u32 x) { return __arch__swab32(x); } @@ -150,7 +150,7 @@ static inline void __swab32s(__u32 *addr) } #ifdef __BYTEORDER_HAS_U64__ -static inline __attribute_const__ __u64 __fswab64(__u64 x) +static inline attr_const __u64 __fswab64(__u64 x) { # ifdef __SWAB_64_THRU_32__ __u32 h = x >> 32; diff --git a/xen/include/xen/compiler.h b/xen/include/xen/compiler.h index 444bf80142c7..b118e4ba62eb 100644 --- a/xen/include/xen/compiler.h +++ b/xen/include/xen/compiler.h @@ -88,7 +88,7 @@ #define __constructor __attribute__((__constructor__)) cf_check #define __pure __attribute__((__pure__)) -#define __attribute_const__ __attribute__((__const__)) +#define attr_const __attribute__((__const__)) #define __transparent__ __attribute__((__transparent_union__)) /* From patchwork Wed Aug 28 22:03:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13782012 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 92494C7114C for ; Wed, 28 Aug 2024 22:04:13 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.785164.1194571 (Exim 4.92) (envelope-from ) id 1sjQll-0006HE-2D; Wed, 28 Aug 2024 22:04:01 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 785164.1194571; Wed, 28 Aug 2024 22:04:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlk-0006Fq-TV; Wed, 28 Aug 2024 22:04:00 +0000 Received: by outflank-mailman (input) for mailman id 785164; Wed, 28 Aug 2024 22:03:59 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlj-0006E0-Ig for xen-devel@lists.xenproject.org; Wed, 28 Aug 2024 22:03:59 +0000 Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [2a00:1450:4864:20::532]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 6cff8213-6589-11ef-a0b0-8be0dac302b0; Thu, 29 Aug 2024 00:03:58 +0200 (CEST) Received: by mail-ed1-x532.google.com with SMTP id 4fb4d7f45d1cf-5bf0261f162so8588209a12.0 for ; Wed, 28 Aug 2024 15:03:58 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c0bb1c2d2esm2695898a12.16.2024.08.28.15.03.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Aug 2024 15:03:56 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 6cff8213-6589-11ef-a0b0-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1724882637; x=1725487437; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uiSYv9zaENNOFB/jb2t7LrfrF1ZHW6+ZFjjQ3yb40Ek=; b=LMPUC3MnPKJV7324noSpfQG0pAxQKQJhuYPfYeJ31TavdAs7Jxa2pqenUnBGyaS3sb ok8QdClTdQoSjGELTwaWPNNAfmPxmvWrhnGGsE3rapP06R4DMySkSsdlvGsa87nUFjY0 UWRuoEBRNslNz1NcTjJTcYbKTXnsdNpresrbg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724882637; x=1725487437; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uiSYv9zaENNOFB/jb2t7LrfrF1ZHW6+ZFjjQ3yb40Ek=; b=Km9tPHpJv1nzLDzy4wc1QXFEO1AxW3lYYsARG7k6rUzhMfSoajRsbzsXYvLY0Fz2ad ltxZhfky9hZMprmCkmO8K2Q5yH5FY84T0S7ZuhctJyDXIexSwPz+CZL1X1RgcsO/ZeVz BpX6QNwlZfWxQqlHMuvHbIk9YtF1NtSNpbFJ4pxwVNmfKMeIs/kWUUcsveGhhK9Rtvq6 ZGphpN2RnRuUIE3Us6MzRXHJcZP7LnzV8S7C+uR4/gGWm6+NVcxHdIAFn+UbzzUWq4/l BjzpIRdkPZ/J2WOmKUVqUOsThpFI5s2Vy8OdyS3QewMhYOUb7vloCqfKfJLDC7tlo+pH wPxg== X-Gm-Message-State: AOJu0YwM6aSQo45fcKnidNEIdmXdPhr2YcPkeFInuzPKRhVrLZaJpugt 3fpuq1kxoCDP4LB0BSSNfU38DEKS8S2OeKwqR0ZFwctsEHJxciGEGk+8Y6T/amitum61EZHUD8m d X-Google-Smtp-Source: AGHT+IEtdemzdGxMgWOSMC2OOmCx1wf030Xl+NsCnh9UAs1rIvfUrQcYgpBhIgnas5MaVAkOPz96nA== X-Received: by 2002:a05:6402:518e:b0:5c0:bba6:860e with SMTP id 4fb4d7f45d1cf-5c21ed52c4dmr755306a12.18.1724882637093; Wed, 28 Aug 2024 15:03:57 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Bertrand Marquis , Michal Orzel , Oleksii Kurochko , Shawn Anastasio Subject: [PATCH v2 02/11] xen/bitops: Switch from __pure to attr_const Date: Wed, 28 Aug 2024 23:03:42 +0100 Message-Id: <20240828220351.2686408-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240828220351.2686408-1-andrew.cooper3@citrix.com> References: <20240828220351.2686408-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 All of the ffs()/fls() infrastructure is in fact (attr) const, because it doesn't even read global state. This allows the compiler even more flexibility to optimise. No functional change. Reported-by: Jan Beulich Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Stefano Stabellini CC: Julien Grall CC: Volodymyr Babchuk CC: Bertrand Marquis CC: Michal Orzel CC: Oleksii Kurochko CC: Shawn Anastasio v2: * New --- xen/include/xen/bitops.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h index 1cd43e464d9e..94af6da18b9b 100644 --- a/xen/include/xen/bitops.h +++ b/xen/include/xen/bitops.h @@ -32,8 +32,8 @@ extern void __bitop_bad_size(void); * * Bits are labelled from 1. Returns 0 if given 0. */ -unsigned int __pure generic_ffsl(unsigned long x); -unsigned int __pure generic_flsl(unsigned long x); +unsigned int attr_const generic_ffsl(unsigned long x); +unsigned int attr_const generic_flsl(unsigned long x); /** * generic__test_and_set_bit - Set a bit and return its old value @@ -204,7 +204,7 @@ static always_inline bool test_bit(int nr, const volatile void *addr) test_bit(nr, addr); \ }) -static always_inline __pure unsigned int ffs(unsigned int x) +static always_inline attr_const unsigned int ffs(unsigned int x) { if ( __builtin_constant_p(x) ) return __builtin_ffs(x); @@ -216,7 +216,7 @@ static always_inline __pure unsigned int ffs(unsigned int x) #endif } -static always_inline __pure unsigned int ffsl(unsigned long x) +static always_inline attr_const unsigned int ffsl(unsigned long x) { if ( __builtin_constant_p(x) ) return __builtin_ffsl(x); @@ -228,7 +228,7 @@ static always_inline __pure unsigned int ffsl(unsigned long x) #endif } -static always_inline __pure unsigned int ffs64(uint64_t x) +static always_inline attr_const unsigned int ffs64(uint64_t x) { if ( BITS_PER_LONG == 64 ) return ffsl(x); @@ -246,7 +246,7 @@ static always_inline __pure unsigned int ffs64(uint64_t x) sizeof(x) <= sizeof(uint64_t) ? ffs64(x) : \ ({ BUILD_ERROR("ffs_g() Bad input type"); 0; })) -static always_inline __pure unsigned int fls(unsigned int x) +static always_inline attr_const unsigned int fls(unsigned int x) { if ( __builtin_constant_p(x) ) return x ? 32 - __builtin_clz(x) : 0; @@ -258,7 +258,7 @@ static always_inline __pure unsigned int fls(unsigned int x) #endif } -static always_inline __pure unsigned int flsl(unsigned long x) +static always_inline attr_const unsigned int flsl(unsigned long x) { if ( __builtin_constant_p(x) ) return x ? BITS_PER_LONG - __builtin_clzl(x) : 0; @@ -270,7 +270,7 @@ static always_inline __pure unsigned int flsl(unsigned long x) #endif } -static always_inline __pure unsigned int fls64(uint64_t x) +static always_inline attr_const unsigned int fls64(uint64_t x) { if ( BITS_PER_LONG == 64 ) return flsl(x); From patchwork Wed Aug 28 22:03:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13782011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8D70BC7113C for ; Wed, 28 Aug 2024 22:04:12 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.785166.1194596 (Exim 4.92) (envelope-from ) id 1sjQln-0006vg-HC; Wed, 28 Aug 2024 22:04:03 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 785166.1194596; Wed, 28 Aug 2024 22:04:03 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQln-0006vZ-EB; Wed, 28 Aug 2024 22:04:03 +0000 Received: by outflank-mailman (input) for mailman id 785166; Wed, 28 Aug 2024 22:04:01 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQll-0006E8-Ka for xen-devel@lists.xenproject.org; Wed, 28 Aug 2024 22:04:01 +0000 Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [2a00:1450:4864:20::52b]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 6dd22981-6589-11ef-99a0-01e77a169b0f; Thu, 29 Aug 2024 00:04:00 +0200 (CEST) Received: by mail-ed1-x52b.google.com with SMTP id 4fb4d7f45d1cf-5bed05c0a2fso8767537a12.3 for ; Wed, 28 Aug 2024 15:04:00 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c0bb1c2d2esm2695898a12.16.2024.08.28.15.03.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Aug 2024 15:03:57 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 6dd22981-6589-11ef-99a0-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1724882638; x=1725487438; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NylPTSK4uUZZH3htPDJReCwwETRg+F4qLCEB3OzPaac=; b=ZmdcLbqtqrxGpwBHHuGMX/0xYiXXcNxUmJSLJDXuLPO0qaeeAAWLBxYB2Ab4QTgKSF U6pDwE4ZI1gkuItnjNp12C+5LYrXoogHtniOuRbmvijZeKK1LvI7RywwQeOx7tJd4mwD 6jEZwExQx7OTQqnXvFKhBbKwghEHNuDsM0O60= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724882638; x=1725487438; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NylPTSK4uUZZH3htPDJReCwwETRg+F4qLCEB3OzPaac=; b=ZIx4TUimE3vUh6AUOokC94G5nSCwOOeYaVxsXr+r17fNDTO9Jfe5vhCTawQ8UyRqRF 2vBDbUdHdHRDJeIQyXhsKsueSsp+Y1ItSw5XEvvBFH/vIzo9eb5gHMWP92LRKqBGdf0o RrEZhlPxJ+vAS4sgANYLCPjr8I10ty1rKmEnqjS0q3k1tHRvbd5xK/+hs+zkM6nSyH36 mnyQEAgRV0vlgpZzxP0tCofnGAWE3wCXg5zA76gQu7L5dWV7RVTfVHsRrXyL9n/vX04v fOLPCtF2viKYYphbdduaq4TI0gY0C89mQ8z4SKXYNY50ENknlflgGuiy66lZoRvGsyvj VdAA== X-Gm-Message-State: AOJu0YzBgsyInrsxITmoHZ2WuhG6iggn9YrKC+emkI5qQwnMnW3xN/4Q fvY7NG7qonaWgtls3011EnxyAqiGYLamkYGmO7yRjiAvcRquAsy5AyL8XkVcZ1WK5dh/yuVUtgI o X-Google-Smtp-Source: AGHT+IE2eu8kKMFH5UPCGR6d+u0pnhhnKtzrehZJO4eQoOpFhy8qAaNfa1P2UR0oHN/qkBQGs1V35Q== X-Received: by 2002:a05:6402:3705:b0:5c0:aadc:8b51 with SMTP id 4fb4d7f45d1cf-5c21ed5758emr695859a12.22.1724882638135; Wed, 28 Aug 2024 15:03:58 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Bertrand Marquis , Michal Orzel , Oleksii Kurochko , Shawn Anastasio Subject: [PATCH v2 03/11] xen/bitops: Reinstate the please tidy message Date: Wed, 28 Aug 2024 23:03:43 +0100 Message-Id: <20240828220351.2686408-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240828220351.2686408-1-andrew.cooper3@citrix.com> References: <20240828220351.2686408-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Recent additions have undone prior tidying at the top of the file. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné CC: Stefano Stabellini CC: Julien Grall CC: Volodymyr Babchuk CC: Bertrand Marquis CC: Michal Orzel CC: Oleksii Kurochko CC: Shawn Anastasio --- xen/include/xen/bitops.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h index 94af6da18b9b..9f0a0ce4a73b 100644 --- a/xen/include/xen/bitops.h +++ b/xen/include/xen/bitops.h @@ -204,6 +204,8 @@ static always_inline bool test_bit(int nr, const volatile void *addr) test_bit(nr, addr); \ }) +/* --------------------- Please tidy above here --------------------- */ + static always_inline attr_const unsigned int ffs(unsigned int x) { if ( __builtin_constant_p(x) ) From patchwork Wed Aug 28 22:03:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13782015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8A5E1C71151 for ; Wed, 28 Aug 2024 22:04:14 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.785167.1194606 (Exim 4.92) (envelope-from ) id 1sjQlo-0007BU-WF; Wed, 28 Aug 2024 22:04:05 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 785167.1194606; Wed, 28 Aug 2024 22:04:04 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlo-0007BJ-R7; Wed, 28 Aug 2024 22:04:04 +0000 Received: by outflank-mailman (input) for mailman id 785167; Wed, 28 Aug 2024 22:04:03 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQln-0006E8-Nb for xen-devel@lists.xenproject.org; Wed, 28 Aug 2024 22:04:03 +0000 Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com [2a00:1450:4864:20::22e]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 6f09f2af-6589-11ef-99a0-01e77a169b0f; Thu, 29 Aug 2024 00:04:02 +0200 (CEST) Received: by mail-lj1-x22e.google.com with SMTP id 38308e7fff4ca-2f3e9fb6ee9so84001561fa.3 for ; Wed, 28 Aug 2024 15:04:02 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c0bb1c2d2esm2695898a12.16.2024.08.28.15.03.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Aug 2024 15:03:58 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 6f09f2af-6589-11ef-99a0-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1724882641; x=1725487441; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jE7MF4AvTxU0huYOPOMOwWO7uwI5OHylKNn5LBEsDiE=; b=Fyud8T8eUx8itFWQ9uj6AEwijMh64XFVIwQo1D2Ku0LpVc2FXHP1Wh0n+foZU+K1Ru pY0Nl1fvQzqLqm9/yS9UrU/8UpsvKbQR2BgtBS0i7Gz1dZe7Cpvs3jLl4Teip/2+NQhD aGQCbjwJLHw13B21XWPSIlFlF+awbJ2y7JK7o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724882641; x=1725487441; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jE7MF4AvTxU0huYOPOMOwWO7uwI5OHylKNn5LBEsDiE=; b=jtZMiZ/GrGT6yVY98Cqm4Z1B3uE8fX02uYv2tXAuDPBIklmpOwGWWZlAWiKcZF2NIC Is7ccXLiPYc40u6IjHadDGtdzDdo2XYclSjKj5F3ItZH4Pyag10DRusw3MxMkRkI/w8W rO1Yi68SLbtyIn9twki5T5S1kvuuQyT/rZJ3czChtniOst9pqo8Glyfk/q/6YpdyotA9 9+oPtRqGdyUM+J6zvDo0xtxtjr2dubgt9t8Sd6JQgmbl/U/RP0SPz7opV6dw8Z/IcS5O VSNiZIcaU0RPqxNsMvYpVgt1g4l7U4P8cx1b0W52lxPMb9UYFNcez1dpon8O411evRe3 Uh0g== X-Gm-Message-State: AOJu0YyGWT/b87zZYcd59eIJyRTOcSdCIYf+In452ja1sh3UuxN7/ygh 7rAdEPPmjUL9SSdOWwftmY7T1+iT3QVHOGWlcbkbBKN4tm8fTCCXG39IBkPKkPLXsKtxQOOGt8Y / X-Google-Smtp-Source: AGHT+IGskY5mUpIjN+AdOMHhdI+obRxCcgVMM986wSrN/PLyjC7vqCbNurLLvgdunNzRC4wX05XAQg== X-Received: by 2002:a05:651c:1543:b0:2ef:2dfd:15e3 with SMTP id 38308e7fff4ca-2f6103a28femr5671851fa.19.1724882640541; Wed, 28 Aug 2024 15:04:00 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Bertrand Marquis , Michal Orzel , Oleksii Kurochko , Shawn Anastasio Subject: [PATCH v2 04/11] xen/bitops: Introduce a multiple_bits_set() helper Date: Wed, 28 Aug 2024 23:03:44 +0100 Message-Id: <20240828220351.2686408-5-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240828220351.2686408-1-andrew.cooper3@citrix.com> References: <20240828220351.2686408-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 This will be used to simplify real logic in the following patch. Add compile and boot time testing as with other bitops. Because the expression is so simple, implement it as a function-like macro which is generic on the type of it's argument, rather than having multiple variants. Testing function-like macros needs a minor adjustments to the infrastructure in xen/self-tests.h to avoid bracketing the fn parameter. The utility of this outweighs the associated risks. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Stefano Stabellini CC: Julien Grall CC: Volodymyr Babchuk CC: Bertrand Marquis CC: Michal Orzel CC: Oleksii Kurochko CC: Shawn Anastasio Name inevitably subject to nitpicking. I'd prefer it to be shorter but I can't think of anything suitable. v2: * Drop redundant CHECK() in the BITS_PER_LONG > 32 case. --- xen/common/bitops.c | 23 +++++++++++++++++++++++ xen/include/xen/bitops.h | 10 ++++++++++ xen/include/xen/self-tests.h | 10 ++++++++-- 3 files changed, 41 insertions(+), 2 deletions(-) diff --git a/xen/common/bitops.c b/xen/common/bitops.c index 9e532f0d87aa..b504dd1308b8 100644 --- a/xen/common/bitops.c +++ b/xen/common/bitops.c @@ -112,9 +112,32 @@ static void __init test_for_each_set_bit(void) panic("for_each_set_bit(uint64) expected %#"PRIx64", got %#"PRIx64"\n", ull, ull_res); } +static void __init test_multiple_bits_set(void) +{ + /* + * multiple_bits_set() is generic on the type of it's parameter, as the + * internal expression is so simple. + */ + + CHECK(multiple_bits_set, 0, false); + CHECK(multiple_bits_set, 1, false); + CHECK(multiple_bits_set, 2, false); + CHECK(multiple_bits_set, 3, true); + + CHECK(multiple_bits_set, 1 | (1UL << (BITS_PER_LONG - 1)), true); +#if BITS_PER_LONG > 32 + CHECK(multiple_bits_set, 1 | (1UL << 32), true); +#endif + + CHECK(multiple_bits_set, 0x8000000000000001ULL, true); + CHECK(multiple_bits_set, 0xc000000000000000ULL, true); +} + static void __init __constructor test_bitops(void) { test_ffs(); test_fls(); test_for_each_set_bit(); + + test_multiple_bits_set(); } diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h index 9f0a0ce4a73b..e7c2c5598275 100644 --- a/xen/include/xen/bitops.h +++ b/xen/include/xen/bitops.h @@ -298,6 +298,16 @@ static always_inline attr_const unsigned int fls64(uint64_t x) __v && ((iter) = ffs_g(__v) - 1, true); \ __v &= __v - 1 ) +/* + * Calculate if a value has two or more bits set. Always use this in + * preference to an expression of the form 'hweight(x) > 1'. + */ +#define multiple_bits_set(x) \ + ({ \ + typeof(x) _v = (x); \ + (_v & (_v - 1)) != 0; \ + }) + /* --------------------- Please tidy below here --------------------- */ #ifndef find_next_bit diff --git a/xen/include/xen/self-tests.h b/xen/include/xen/self-tests.h index e9a879489363..bd8a4867aa40 100644 --- a/xen/include/xen/self-tests.h +++ b/xen/include/xen/self-tests.h @@ -15,11 +15,14 @@ * * Clang < 8 can't fold constants through static inlines, causing this to * fail. Simply skip it for incredibly old compilers. + * + * N.B. fn is intentionally not bracketed to allow us to test function-like + * macros too. */ #if !defined(CONFIG_CC_IS_CLANG) || CONFIG_CLANG_VERSION >= 80000 #define COMPILE_CHECK(fn, val, res) \ do { \ - typeof((fn)(val)) real = (fn)(val); \ + typeof(fn(val)) real = fn(val); \ \ if ( !__builtin_constant_p(real) ) \ BUILD_ERROR("'" STR(fn(val)) "' not compile-time constant"); \ @@ -34,10 +37,13 @@ * Check that Xen's runtime logic for fn(val) gives the expected answer. This * requires using HIDE() to prevent the optimiser from collapsing the logic * into a constant. + * + * N.B. fn is intentionally not bracketed to allow us to test function-like + * macros too. */ #define RUNTIME_CHECK(fn, val, res) \ do { \ - typeof((fn)(val)) real = (fn)(HIDE(val)); \ + typeof(fn(val)) real = fn(HIDE(val)); \ \ if ( real != (res) ) \ panic("%s: %s(%s) expected %u, got %u\n", \ From patchwork Wed Aug 28 22:03:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13782017 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2B118C71156 for ; Wed, 28 Aug 2024 22:04:14 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.785169.1194620 (Exim 4.92) (envelope-from ) id 1sjQlq-0007Wd-Kq; Wed, 28 Aug 2024 22:04:06 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 785169.1194620; Wed, 28 Aug 2024 22:04:06 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlq-0007Tu-Cb; Wed, 28 Aug 2024 22:04:06 +0000 Received: by outflank-mailman (input) for mailman id 785169; Wed, 28 Aug 2024 22:04:05 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlp-0006E8-4G for xen-devel@lists.xenproject.org; Wed, 28 Aug 2024 22:04:05 +0000 Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [2a00:1450:4864:20::531]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 6fe06872-6589-11ef-99a0-01e77a169b0f; Thu, 29 Aug 2024 00:04:03 +0200 (CEST) Received: by mail-ed1-x531.google.com with SMTP id 4fb4d7f45d1cf-5a10835487fso10803030a12.1 for ; Wed, 28 Aug 2024 15:04:03 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c0bb1c2d2esm2695898a12.16.2024.08.28.15.04.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Aug 2024 15:04:00 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 6fe06872-6589-11ef-99a0-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1724882642; x=1725487442; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZXiFJWRq4x5FPe198sZU9vAtmoQscirnVku7zrIp0cM=; b=AbMiCzaNw0J+7PjWB8aCJV6Pn2eztdFDTL0VPEFHeN5cmMrJEsdc+h5hCo9J3j1rOU IvAtx2Pfn3/TcNM53PABbHTCqlZ6FI0KTNFO7RO2claYanfwzYz2HGPR0rSKWImmuayU QyfOQ09YXX9fM6QMEj65IUOlKjD9mRABasOcE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724882642; x=1725487442; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZXiFJWRq4x5FPe198sZU9vAtmoQscirnVku7zrIp0cM=; b=UcOnKcIAAbA0kM7V+zt05kWYQozjxCHv+lZzliJlSHHAWFfVdb/ZFrF63YS9Ecsjwg C1v4FJ2naCpG311WdEn3qrE1kRErfFZTqaa8q6LZnHUWisZm+LrIswq+oqKwF2t5DMMO SZMGl922841IXMJuqAXAb8iOvyqsgc4ZfMv/wSJVdg50Qw2kk8ZysaIsj/AnpL8Uq6BQ UQXg4cH9/PeR0UUmhIVeA9a/vrlo6WWWkMUPnGcC4rTOYxCeOoCVjwLCw5GlWECaUVdL yNkCUULjPt7wMgJlkquYwdidMd0tELav56BCX+E4v7KqvnFeqPP1MvPogyJahzfeJ6u3 450w== X-Gm-Message-State: AOJu0YwaG8sQ8wbbucrUDitjyK9A3vGdTs6ed8llZYU6BiRsTvpU4Gkf jwQoz1dBu7Cx5EMrqsUHr5Fny2uDOpkhlNhqpLTxc3jSh8/smI7ugECp59pigKG+ny5x+Zgoskj f X-Google-Smtp-Source: AGHT+IEvBWczXJywqu93GhfTA0hun1Xm5VwCqG1AFE2wyPe5kLOh7xiV1ipXRad4bLf+4WSvk1yCJw== X-Received: by 2002:a05:6402:40c4:b0:5be:fbce:939e with SMTP id 4fb4d7f45d1cf-5c21ec5b94dmr882236a12.0.1724882641403; Wed, 28 Aug 2024 15:04:01 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Bertrand Marquis , Michal Orzel , Oleksii Kurochko , Shawn Anastasio Subject: [PATCH v2 05/11] xen/bitops: Convert 'hweight(x) > 1' to new multiple_bits_set() Date: Wed, 28 Aug 2024 23:03:45 +0100 Message-Id: <20240828220351.2686408-6-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240828220351.2686408-1-andrew.cooper3@citrix.com> References: <20240828220351.2686408-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Using hweight() is an especially expensive way of determining simply if multiple bits are set in a value. Worse, 4 of the 10 hweight() calls in Xen are of this form. Switch to the new multiple_bits_set() helper. This is far more efficient than the longhand hweight() algorithm and, owing to its simplicity, likely more efficient than even a dedicated instruction on a superscalar processor. No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Stefano Stabellini CC: Julien Grall CC: Volodymyr Babchuk CC: Bertrand Marquis CC: Michal Orzel CC: Oleksii Kurochko CC: Shawn Anastasio For future work, do related work for bitmap_weight() so cpumask_weight() in particular can stop being used for so many 0/1 special cases. On x86, the code reduction speaks for itself: add/remove: 0/0 grow/shrink: 0/3 up/down: 0/-240 (-240) Function old new delta vlapic_ipi 722 650 -72 numa_emulation 577 497 -80 do_xenpmu_op 1665 1577 -88 That's an aweful lot of wasted calculation for the same answer. I can't find a way of enabling CONFIG_NUMA on ARM (yet?) so right now there's no change in any other architecture. However, a synthetic helper shows the following on ARM32: add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-128 (-128) Function old new delta test_mbs 176 48 -128 and on ARM64: add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-44 (-44) Function old new delta test_mbs 60 16 -44 PPC64 has POPCNTD in a default build of Xen and by chance both algorithms compile to the same number of instructions. RISC-V doesn't have hweight() wired up yet at all. --- xen/arch/x86/cpu/vpmu.c | 2 +- xen/arch/x86/hvm/vlapic.c | 10 ++++++---- xen/common/numa.c | 2 +- 3 files changed, 8 insertions(+), 6 deletions(-) diff --git a/xen/arch/x86/cpu/vpmu.c b/xen/arch/x86/cpu/vpmu.c index b2ba9994129b..a5bb1689c7d5 100644 --- a/xen/arch/x86/cpu/vpmu.c +++ b/xen/arch/x86/cpu/vpmu.c @@ -673,7 +673,7 @@ long do_xenpmu_op( { if ( (pmu_params.val & ~(XENPMU_MODE_SELF | XENPMU_MODE_HV | XENPMU_MODE_ALL)) || - (hweight64(pmu_params.val) > 1) ) + multiple_bits_set(pmu_params.val) ) return -EINVAL; /* 32-bit dom0 can only sample itself. */ diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c index 2ec95942713e..4a3e21a65f9d 100644 --- a/xen/arch/x86/hvm/vlapic.c +++ b/xen/arch/x86/hvm/vlapic.c @@ -467,12 +467,14 @@ static bool is_multicast_dest(struct vlapic *vlapic, unsigned int short_hand, return short_hand != APIC_DEST_SELF; if ( vlapic_x2apic_mode(vlapic) ) - return dest_mode ? hweight16(dest) > 1 : dest == 0xffffffffU; + return dest_mode ? multiple_bits_set((uint16_t)dest) + : dest == 0xffffffffU; if ( dest_mode ) - return hweight8(dest & - GET_xAPIC_DEST_FIELD(vlapic_get_reg(vlapic, - APIC_DFR))) > 1; + { + dest &= GET_xAPIC_DEST_FIELD(vlapic_get_reg(vlapic, APIC_DFR)); + return multiple_bits_set((uint8_t)dest); + } return dest == 0xff; } diff --git a/xen/common/numa.c b/xen/common/numa.c index 28a09766fabc..ce3991929ce5 100644 --- a/xen/common/numa.c +++ b/xen/common/numa.c @@ -546,7 +546,7 @@ static int __init numa_emulation(unsigned long start_pfn, uint64_t sz = pfn_to_paddr(end_pfn - start_pfn) / numa_fake; /* Kludge needed for the hash function */ - if ( hweight64(sz) > 1 ) + if ( multiple_bits_set(sz) ) { uint64_t x = 1; From patchwork Wed Aug 28 22:03:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13782013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AFF1CC71153 for ; Wed, 28 Aug 2024 22:04:13 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.785168.1194616 (Exim 4.92) (envelope-from ) id 1sjQlq-0007Ss-A2; Wed, 28 Aug 2024 22:04:06 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 785168.1194616; Wed, 28 Aug 2024 22:04:06 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlq-0007Sc-5I; Wed, 28 Aug 2024 22:04:06 +0000 Received: by outflank-mailman (input) for mailman id 785168; Wed, 28 Aug 2024 22:04:04 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlo-0006E0-Fb for xen-devel@lists.xenproject.org; Wed, 28 Aug 2024 22:04:04 +0000 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [2a00:1450:4864:20::533]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 702a4b3d-6589-11ef-a0b0-8be0dac302b0; Thu, 29 Aug 2024 00:04:04 +0200 (CEST) Received: by mail-ed1-x533.google.com with SMTP id 4fb4d7f45d1cf-5c0abaae174so4210006a12.1 for ; Wed, 28 Aug 2024 15:04:04 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c0bb1c2d2esm2695898a12.16.2024.08.28.15.04.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Aug 2024 15:04:01 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 702a4b3d-6589-11ef-a0b0-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1724882643; x=1725487443; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1d4T7bgcWZFGEOhnXIFX9rzTrMxxv0mMcntif1gRWqE=; b=rDw/v8bnzRTZHYzsgx9qQyUe+lDE7zoQbJXY3qfHCrH/mg8bbfkUd0Su/1Vu/0MNGg j0zR7Pib5OFCMbqvpBxVz8fH+MM8gC19U9Jt27euoeltoTuwOJVPxK3xtos5A8Q30ho0 z4nwJ4qqNGLBjyvaYWxs0PgLPFSVcLyolk734= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724882643; x=1725487443; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1d4T7bgcWZFGEOhnXIFX9rzTrMxxv0mMcntif1gRWqE=; b=cql2E8ZC2XHcazHTP+ukJ/6zv42aBKEZDF9+RUsnxgF5rDCprgaL869n1M4XpWzIN1 9Xiok5uktFD5X0KAzrkmDJbKtO13Qd2StZpZAfyMRa9m97LcJ6Eoantt7OQgje8W7oXG VLiAoYOju5VDpZ7BTORNGmshxp0VEPOXXs0PXVkeoWkkX4ei+7yCG07mlr/LNX6M4bRH JAXJvnXE3C1Xsxju0gZ+S6T4nhS/p/dYFNwTviJR6/UMxD1AvEY3j10znrF3FAIAe0Qv jR2rZKjlAdCTY7cSen0VRF21B/uN+FsJY+NRVl0AAmF3IQcZPKK5oQcTXpONAB0xWQqj MWaQ== X-Gm-Message-State: AOJu0YzYNc8jeASN8jAl4zcoKGUWB7XU+lJAtw/uXkVVMJmWTycaD8vz OStrptz00JAD8Fc7SVT0EnomPlXSfG2iHtmh1m1O8IdaDY0C8vFI/NcjIrIFdCjQPNjdcQR77Lx k X-Google-Smtp-Source: AGHT+IF7BfYv8A77TM6Js7u46H/5nrrqGXZxWpYnBfYsiP2mHja+VUkq+v83YoMCSlV9Z3MbjxLEzg== X-Received: by 2002:a05:6402:518e:b0:5be:fc44:d159 with SMTP id 4fb4d7f45d1cf-5c21ed52ba1mr605962a12.22.1724882642446; Wed, 28 Aug 2024 15:04:02 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Bertrand Marquis , Michal Orzel , Oleksii Kurochko , Shawn Anastasio Subject: [PATCH v2 06/11] xen/bitops: Drop the remnants of hweight{8,16}() Date: Wed, 28 Aug 2024 23:03:46 +0100 Message-Id: <20240828220351.2686408-7-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240828220351.2686408-1-andrew.cooper3@citrix.com> References: <20240828220351.2686408-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 They are no more. No functional change. Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Stefano Stabellini CC: Julien Grall CC: Volodymyr Babchuk CC: Bertrand Marquis CC: Michal Orzel CC: Oleksii Kurochko CC: Shawn Anastasio --- xen/arch/arm/include/asm/bitops.h | 2 -- xen/arch/ppc/include/asm/bitops.h | 2 -- xen/arch/x86/include/asm/bitops.h | 2 -- xen/include/xen/bitops.h | 17 ----------------- 4 files changed, 23 deletions(-) diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h index 3c023103f734..91cd167b6bbb 100644 --- a/xen/arch/arm/include/asm/bitops.h +++ b/xen/arch/arm/include/asm/bitops.h @@ -86,8 +86,6 @@ bool clear_mask16_timeout(uint16_t mask, volatile void *p, */ #define hweight64(x) generic_hweight64(x) #define hweight32(x) generic_hweight32(x) -#define hweight16(x) generic_hweight16(x) -#define hweight8(x) generic_hweight8(x) #endif /* _ARM_BITOPS_H */ /* diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h index eb3355812ea3..a62c4f99c3bb 100644 --- a/xen/arch/ppc/include/asm/bitops.h +++ b/xen/arch/ppc/include/asm/bitops.h @@ -132,7 +132,5 @@ static inline int test_and_set_bit(unsigned int nr, volatile void *addr) */ #define hweight64(x) __builtin_popcountll(x) #define hweight32(x) __builtin_popcount(x) -#define hweight16(x) __builtin_popcount((uint16_t)(x)) -#define hweight8(x) __builtin_popcount((uint8_t)(x)) #endif /* _ASM_PPC_BITOPS_H */ diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h index 8c0403405aa2..4c5b21907a64 100644 --- a/xen/arch/x86/include/asm/bitops.h +++ b/xen/arch/x86/include/asm/bitops.h @@ -483,7 +483,5 @@ static always_inline unsigned int arch_flsl(unsigned long x) */ #define hweight64(x) generic_hweight64(x) #define hweight32(x) generic_hweight32(x) -#define hweight16(x) generic_hweight16(x) -#define hweight8(x) generic_hweight8(x) #endif /* _X86_BITOPS_H */ diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h index e7c2c5598275..1c160b643ed6 100644 --- a/xen/include/xen/bitops.h +++ b/xen/include/xen/bitops.h @@ -395,23 +395,6 @@ static inline unsigned int generic_hweight32(unsigned int w) return (w + (w >> 16)) & 0xff; } -static inline unsigned int generic_hweight16(unsigned int w) -{ - w -= ((w >> 1) & 0x5555); - w = (w & 0x3333) + ((w >> 2) & 0x3333); - w = (w + (w >> 4)) & 0x0f0f; - - return (w + (w >> 8)) & 0xff; -} - -static inline unsigned int generic_hweight8(unsigned int w) -{ - w -= ((w >> 1) & 0x55); - w = (w & 0x33) + ((w >> 2) & 0x33); - - return (w + (w >> 4)) & 0x0f; -} - static inline unsigned int generic_hweight64(uint64_t w) { if ( BITS_PER_LONG < 64 ) From patchwork Wed Aug 28 22:03:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13782018 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 37396C71157 for ; Wed, 28 Aug 2024 22:04:16 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.785170.1194636 (Exim 4.92) (envelope-from ) id 1sjQls-00080O-P4; Wed, 28 Aug 2024 22:04:08 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 785170.1194636; Wed, 28 Aug 2024 22:04:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQls-000808-LF; Wed, 28 Aug 2024 22:04:08 +0000 Received: by outflank-mailman (input) for mailman id 785170; Wed, 28 Aug 2024 22:04:07 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlq-0006E0-U7 for xen-devel@lists.xenproject.org; Wed, 28 Aug 2024 22:04:06 +0000 Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [2a00:1450:4864:20::534]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 717cec95-6589-11ef-a0b0-8be0dac302b0; Thu, 29 Aug 2024 00:04:06 +0200 (CEST) Received: by mail-ed1-x534.google.com with SMTP id 4fb4d7f45d1cf-5bed72ff2f2so9048435a12.2 for ; Wed, 28 Aug 2024 15:04:06 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c0bb1c2d2esm2695898a12.16.2024.08.28.15.04.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Aug 2024 15:04:02 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 717cec95-6589-11ef-a0b0-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1724882645; x=1725487445; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uz9gF0kJ1ww5Xl2kbFnbXvQPW3lQQ7rY+fD3ahbYndw=; b=cD2ielAqEZaS2j5hMPfNsDbJHvhVBtoNOjRxh9zYLUMwJHxVrhznwXtC7OaKu11UV2 TnY+CUXaLS8qSluASp75oWH45GL+EXPFmZlFE3xWsW7Tctq6yt3ey9D4X/LMXVbtwTfg fOnJcZK/e1CeJUJmeSBauK47f3I7reFLjWg0U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724882645; x=1725487445; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uz9gF0kJ1ww5Xl2kbFnbXvQPW3lQQ7rY+fD3ahbYndw=; b=cQ6F2vwa15laAUFUndfNlvD6vg1KnZB4Ri0T2w4bRecy7zJkF1G2S56SohTr8B6Y9D mjcDOGRau0yd344j804XqIn3Vk5x4yb8l8IPQv4T9DiGwkhA5lYnzH9sAkChzsoOkV4P pMFs/Zqfcs8A9vwHrC4nI2FEkhe1n0XYtXO5Q3RK950epguYBWRfOJH/dzu+TaQfyVsC bV5yoqLpZU4/2c58QgqtFye8Sz056wRWw57FQFbNUdGjJB+G0VbneL+kD/cQowPxjEDr npJkZUgu4SH5zZuq50ahf+U2IthOUYvs3ONE795bXUs8Bj6k9j8Dunhe2naEf/4ds9aq /4kA== X-Gm-Message-State: AOJu0YxxwdVVj+doHyAo+EWhYXMoiNcTX+hKHKIFpwM6JwdSMNw2G/gl GGK1wllbBoOKI+1zoLeniDrsOg27o0rWMuxfyqQ1jO7i6J3itXjU607vOY6lOhYsBLEfOdGIVeZ 3 X-Google-Smtp-Source: AGHT+IFPnPelOGlHkbSiWoLUTbbBX0ocH1aTm9zD7+Aeom2DD0tWujvgwCU1PAM7VBJ3aUgzB/qqxA== X-Received: by 2002:a05:6402:1d48:b0:5b8:34a9:7fd9 with SMTP id 4fb4d7f45d1cf-5c21ed86b5amr661354a12.27.1724882644363; Wed, 28 Aug 2024 15:04:04 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Bertrand Marquis , Michal Orzel , Oleksii Kurochko , Shawn Anastasio Subject: [PATCH v2 07/11] xen/bitops: Introduce generic_hweightl() and hweightl() Date: Wed, 28 Aug 2024 23:03:47 +0100 Message-Id: <20240828220351.2686408-8-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240828220351.2686408-1-andrew.cooper3@citrix.com> References: <20240828220351.2686408-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 There are 6 remaining callers in Xen: * The two hweight32() calls, _domain_struct_bits() and efi_find_gop_mode(), are __init only. * The two hweight_long() calls are both in bitmap_weight(). * The two hweight64() calls are hv_vpset_nr_banks() and x86_emulate(). Only bitmap_weight() and possibly hv_vpset_nr_banks() can be considered fast paths, and they're all of GPR-width form. Furthermore, the differences between a generic int and generic long form is only an ADD and SHIFT, and only in !CONFIG_HAS_FAST_MULTIPLY builds. Therefore, it is definitely not worth having both generic implemenations. Implement generic_hweightl() based on the current generic_hweight64(), adjusted to be compatible with ARM32, along with standard SELF_TESTS. Implement hweightl() with usual constant-folding and arch opt-in support. PPC is the only architecture that devates from generic, and it simply uses the builtin. No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Stefano Stabellini CC: Julien Grall CC: Volodymyr Babchuk CC: Bertrand Marquis CC: Michal Orzel CC: Oleksii Kurochko CC: Shawn Anastasio v2: * s/MASK/BCST/. Extend testing * s/__pure/attr_const/. --- xen/arch/ppc/include/asm/bitops.h | 2 ++ xen/common/bitops.c | 14 +++++++++ xen/include/xen/bitops.h | 18 ++++++++++++ xen/lib/Makefile | 1 + xen/lib/generic-hweightl.c | 49 +++++++++++++++++++++++++++++++ 5 files changed, 84 insertions(+) create mode 100644 xen/lib/generic-hweightl.c diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h index a62c4f99c3bb..64512e949530 100644 --- a/xen/arch/ppc/include/asm/bitops.h +++ b/xen/arch/ppc/include/asm/bitops.h @@ -124,6 +124,8 @@ static inline int test_and_set_bit(unsigned int nr, volatile void *addr) #define arch_fls(x) ((x) ? 32 - __builtin_clz(x) : 0) #define arch_flsl(x) ((x) ? BITS_PER_LONG - __builtin_clzl(x) : 0) +#define arch_hweightl(x) __builtin_popcountl(x) + /** * hweightN - returns the hamming weight of a N-bit word * @x: the word to weigh diff --git a/xen/common/bitops.c b/xen/common/bitops.c index b504dd1308b8..5e5d20d225d7 100644 --- a/xen/common/bitops.c +++ b/xen/common/bitops.c @@ -133,6 +133,19 @@ static void __init test_multiple_bits_set(void) CHECK(multiple_bits_set, 0xc000000000000000ULL, true); } +static void __init test_hweight(void) +{ + /* unsigned int hweightl(unsigned long) */ + CHECK(hweightl, 0, 0); + CHECK(hweightl, 1, 1); + CHECK(hweightl, 3, 2); + CHECK(hweightl, 7, 3); + CHECK(hweightl, 0xff, 8); + + CHECK(hweightl, 1 | (1UL << (BITS_PER_LONG - 1)), 2); + CHECK(hweightl, -1UL, BITS_PER_LONG); +} + static void __init __constructor test_bitops(void) { test_ffs(); @@ -140,4 +153,5 @@ static void __init __constructor test_bitops(void) test_for_each_set_bit(); test_multiple_bits_set(); + test_hweight(); } diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h index 1c160b643ed6..96dfe0f2c71a 100644 --- a/xen/include/xen/bitops.h +++ b/xen/include/xen/bitops.h @@ -35,6 +35,12 @@ extern void __bitop_bad_size(void); unsigned int attr_const generic_ffsl(unsigned long x); unsigned int attr_const generic_flsl(unsigned long x); +/* + * Hamming Weight, also called Population Count. Returns the number of set + * bits in @x. + */ +unsigned int attr_const generic_hweightl(unsigned long x); + /** * generic__test_and_set_bit - Set a bit and return its old value * @nr: Bit to set @@ -308,6 +314,18 @@ static always_inline attr_const unsigned int fls64(uint64_t x) (_v & (_v - 1)) != 0; \ }) +static always_inline attr_const unsigned int hweightl(unsigned long x) +{ + if ( __builtin_constant_p(x) ) + return __builtin_popcountl(x); + +#ifdef arch_hweightl + return arch_hweightl(x); +#else + return generic_hweightl(x); +#endif +} + /* --------------------- Please tidy below here --------------------- */ #ifndef find_next_bit diff --git a/xen/lib/Makefile b/xen/lib/Makefile index a48541596470..b6558e108bd9 100644 --- a/xen/lib/Makefile +++ b/xen/lib/Makefile @@ -6,6 +6,7 @@ lib-y += ctype.o lib-y += find-next-bit.o lib-y += generic-ffsl.o lib-y += generic-flsl.o +lib-y += generic-hweightl.o lib-y += list-sort.o lib-y += memchr.o lib-y += memchr_inv.o diff --git a/xen/lib/generic-hweightl.c b/xen/lib/generic-hweightl.c new file mode 100644 index 000000000000..c242d4c2d9ab --- /dev/null +++ b/xen/lib/generic-hweightl.c @@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#include +#include +#include + +/* Value @b broadcast to every byte in a long */ +#if BITS_PER_LONG == 32 +# define BCST(b) ((b) * 0x01010101UL) +#elif BITS_PER_LONG == 64 +# define BCST(b) ((b) * 0x0101010101010101UL) +#else +# error Extend me please +#endif + +unsigned int generic_hweightl(unsigned long x) +{ + x -= (x >> 1) & BCST(0x55); + x = (x & BCST(0x33)) + ((x >> 2) & BCST(0x33)); + x = (x + (x >> 4)) & BCST(0x0f); + + if ( IS_ENABLED(CONFIG_HAS_FAST_MULTIPLY) ) + return (x * BCST(0x01)) >> (BITS_PER_LONG - 8); + + x += x >> 8; + x += x >> 16; +#if BITS_PER_LONG > 32 + x += x >> 32; +#endif + + return x & 0xff; +} + +#ifdef CONFIG_SELF_TESTS +static void __init __constructor test_generic_hweightl(void) +{ + RUNTIME_CHECK(generic_hweightl, 0, 0); + RUNTIME_CHECK(generic_hweightl, 1, 1); + RUNTIME_CHECK(generic_hweightl, 3, 2); + RUNTIME_CHECK(generic_hweightl, 7, 3); + RUNTIME_CHECK(generic_hweightl, 0xff, 8); + + RUNTIME_CHECK(generic_hweightl, BCST(0x55), BITS_PER_LONG / 2); + RUNTIME_CHECK(generic_hweightl, BCST(0xaa), BITS_PER_LONG / 2); + + RUNTIME_CHECK(generic_hweightl, 1 | (1UL << (BITS_PER_LONG - 1)), 2); + RUNTIME_CHECK(generic_hweightl, -1UL, BITS_PER_LONG); +} +#endif /* CONFIG_SELF_TESTS */ From patchwork Wed Aug 28 22:03:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13782019 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6ABA2C7114C for ; Wed, 28 Aug 2024 22:04:17 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.785171.1194641 (Exim 4.92) (envelope-from ) id 1sjQlt-00087g-Dx; Wed, 28 Aug 2024 22:04:09 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 785171.1194641; Wed, 28 Aug 2024 22:04:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlt-00086B-5r; Wed, 28 Aug 2024 22:04:09 +0000 Received: by outflank-mailman (input) for mailman id 785171; Wed, 28 Aug 2024 22:04:07 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlr-0006E0-Oz for xen-devel@lists.xenproject.org; Wed, 28 Aug 2024 22:04:07 +0000 Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com [2a00:1450:4864:20::22c]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 723a1c2b-6589-11ef-a0b0-8be0dac302b0; Thu, 29 Aug 2024 00:04:07 +0200 (CEST) Received: by mail-lj1-x22c.google.com with SMTP id 38308e7fff4ca-2f4f8742138so66205671fa.0 for ; Wed, 28 Aug 2024 15:04:07 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c0bb1c2d2esm2695898a12.16.2024.08.28.15.04.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Aug 2024 15:04:04 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 723a1c2b-6589-11ef-a0b0-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1724882646; x=1725487446; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3wEH6spuDcWI9g3H1/wKwZf6UslM/CRGIF6MqUG9Fo4=; b=SYh+VbLbtd+UcTE5IWCL1WUQ/K70VLsSZcyt75G1gXMwWAG9916peHIG8rr1KgmqM2 rpm7FNtH3X5P/w3uJlp5RyUl+KvgPTE1G6OfLlqmAjIjm0JgV2S7RbVNwmyxhJu9RclS DkPiiiPNruOXnjluUbjL3da+NAqdCpIuTRe2A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724882646; x=1725487446; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3wEH6spuDcWI9g3H1/wKwZf6UslM/CRGIF6MqUG9Fo4=; b=XvI65YU1qZG3QHaF8z7dcqEJhu+ePEy4a0TJSoYZi0eamiTP3EpXEKknB5pOT1ql2b mZ3Td/cdD8z7RVynU6ZkS9D+GU2fC4aDDTaxtTncWFt56iDnKSqol9laBEhqWKvp0xQD vd1lG61N+D7xoBufahTObrx26JHi3D68KG8VqJSNmXidfc0yH9dDC3Puk7t566SQRYdo xSHDs9TQbUiWmtOTD7q2jc/+25pYEITkz9lHkyIrOs8LI1AuYCU6rxsW/7laxT/ca+UP 3q738UjiI3j5u0c/1lDaYTsuk5gC6PoJ1C6qErgEiv47Usiq+N1u5C6oh/7tFuyyPr4M /ZZA== X-Gm-Message-State: AOJu0YxoBP+I3BMYYsk2c6h1JLUO9v5oFwV0v9WZCTOLE30b2iyV4Fv5 lnSiFkNW44atpEIzPnU0Yec0TTp2EzTGYwi6ksVtjVTDalAQfpRp5cMZ8EASTvYFf0b4E4CTmiD r X-Google-Smtp-Source: AGHT+IE5UtUMGD1wRggXBe+xzppDvxCyJEivzUcLNMKfgMxac6dmJDS5xX3A4MiKzTBV/xGdZyLdlw== X-Received: by 2002:a05:651c:19ac:b0:2ef:2247:987b with SMTP id 38308e7fff4ca-2f6108908d2mr6546591fa.32.1724882645193; Wed, 28 Aug 2024 15:04:05 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Bertrand Marquis , Michal Orzel , Oleksii Kurochko , Shawn Anastasio Subject: [PATCH v2 08/11] xen/bitops: Drop hweight_long() and use hweightl() Date: Wed, 28 Aug 2024 23:03:48 +0100 Message-Id: <20240828220351.2686408-9-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240828220351.2686408-1-andrew.cooper3@citrix.com> References: <20240828220351.2686408-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Stefano Stabellini CC: Julien Grall CC: Volodymyr Babchuk CC: Bertrand Marquis CC: Michal Orzel CC: Oleksii Kurochko CC: Shawn Anastasio --- xen/common/bitmap.c | 4 ++-- xen/include/xen/bitops.h | 5 ----- 2 files changed, 2 insertions(+), 7 deletions(-) diff --git a/xen/common/bitmap.c b/xen/common/bitmap.c index 9d9ff09f3dde..3da63a32a680 100644 --- a/xen/common/bitmap.c +++ b/xen/common/bitmap.c @@ -191,10 +191,10 @@ unsigned int __bitmap_weight(const unsigned long *bitmap, unsigned int bits) unsigned int k, w = 0, lim = bits / BITS_PER_LONG; for (k = 0; k < lim; k++) - w += hweight_long(bitmap[k]); + w += hweightl(bitmap[k]); if (bits % BITS_PER_LONG) - w += hweight_long(bitmap[k] & BITMAP_LAST_WORD_MASK(bits)); + w += hweightl(bitmap[k] & BITMAP_LAST_WORD_MASK(bits)); return w; } diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h index 96dfe0f2c71a..58c600155f7e 100644 --- a/xen/include/xen/bitops.h +++ b/xen/include/xen/bitops.h @@ -431,11 +431,6 @@ static inline unsigned int generic_hweight64(uint64_t w) return (w + (w >> 32)) & 0xFF; } -static inline unsigned int hweight_long(unsigned long w) -{ - return sizeof(w) == 4 ? generic_hweight32(w) : generic_hweight64(w); -} - /* * rol32 - rotate a 32-bit value left * From patchwork Wed Aug 28 22:03:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13782020 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 15C8BC7113C for ; Wed, 28 Aug 2024 22:04:20 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.785172.1194656 (Exim 4.92) (envelope-from ) id 1sjQlw-0000DV-Ml; Wed, 28 Aug 2024 22:04:12 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 785172.1194656; Wed, 28 Aug 2024 22:04:12 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlw-0000DJ-JQ; Wed, 28 Aug 2024 22:04:12 +0000 Received: by outflank-mailman (input) for mailman id 785172; Wed, 28 Aug 2024 22:04:10 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlu-0006E8-U2 for xen-devel@lists.xenproject.org; Wed, 28 Aug 2024 22:04:10 +0000 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [2a00:1450:4864:20::52a]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 735b5ad8-6589-11ef-99a0-01e77a169b0f; Thu, 29 Aug 2024 00:04:09 +0200 (CEST) Received: by mail-ed1-x52a.google.com with SMTP id 4fb4d7f45d1cf-5a108354819so9317577a12.0 for ; Wed, 28 Aug 2024 15:04:09 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c0bb1c2d2esm2695898a12.16.2024.08.28.15.04.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Aug 2024 15:04:05 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 735b5ad8-6589-11ef-99a0-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1724882648; x=1725487448; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+a9rZjwjT4T/KMzHim0E6pRI1Xb37U68VQt9wRm391Y=; b=b/lenRI932SyEytxCAFEiO/qsUMJQcj46yK/2EQwJV5yqOiR4EfYhmBFIUvNGcNERG uJ1yO725B4mqWpd2WorZFUuLsOjeCfPq0bBp/mPW7olusr7UpKTfgIf9dDVjUByXqDx7 Ba61YuBnA8xLFcCSQT+7LZHwjVPopcpjTBd/8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724882648; x=1725487448; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+a9rZjwjT4T/KMzHim0E6pRI1Xb37U68VQt9wRm391Y=; b=YKHg9Aain2ThG+UogaAxRg+bjMXfbWFsu2HDKDRvJplSmD7OqR5yfrLS59SeVhnbIc iwu36hPf9h1yifargx9fmWpQpdB3mmOuDnx5MbKcD/XiE1UhC1gCOCIX12d31H97AS/5 JtCeAG+bryp6aTIPqYMIf3EtT6K1GPEO20RPY2cAr2PxlMB5vuF5f+U5jn6j29RYhUnn i3DBJay4rI5/B6Y2B5mJ6AUQsez9y2vvBHJl+9fMwf74CEkzexf1pbGW2Fx+U18ibXJD vEnac9DNCxBY3z4XM8MKXRiu3Gy89JF5oFJV6M8FdE6lBqJVUW29tH4/0zORbiysg2S4 27Pg== X-Gm-Message-State: AOJu0YzXKkhqB9ejgcm1yKZHKeniB9pHZu+VgO/2vpvRJjKC8/xBfotM zOVHzGf1K53hnUYhxOKhPnd3YaRw5u4VZqaSzk6+6yeS09/ygecOCRb2hArcL9QwbI9KH6EB0I/ e X-Google-Smtp-Source: AGHT+IE3qOrSiwEVqCW9Hnp03aRaF70/QvtqPk4oYtkklqTrDEe2lMIHKCK2PJjCz4Asj6oY3RjTrQ== X-Received: by 2002:a05:6402:40c1:b0:5c2:17b7:5a7e with SMTP id 4fb4d7f45d1cf-5c21ed9fcbcmr609830a12.36.1724882647591; Wed, 28 Aug 2024 15:04:07 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Bertrand Marquis , Michal Orzel , Oleksii Kurochko , Shawn Anastasio Subject: [PATCH v2 09/11] xen/bitops: Implement hweight32() in terms of hweightl() Date: Wed, 28 Aug 2024 23:03:49 +0100 Message-Id: <20240828220351.2686408-10-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240828220351.2686408-1-andrew.cooper3@citrix.com> References: <20240828220351.2686408-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 ... and drop generic_hweight32(). As noted previously, the only two users of hweight32() are in __init paths. The int-optimised form of generic_hweight() is only two instructions shorter than the long-optimised form, and even then only on architectures which lack fast multiplication, so there's no point providing an int-optimised form. No functional change. Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Stefano Stabellini CC: Julien Grall CC: Volodymyr Babchuk CC: Bertrand Marquis CC: Michal Orzel CC: Oleksii Kurochko CC: Shawn Anastasio v2: * Reorder with respect to the hweight64() patch * Rerwrite the commit message * s/__pure/attr_const/ --- xen/arch/arm/include/asm/bitops.h | 1 - xen/arch/ppc/include/asm/bitops.h | 1 - xen/arch/x86/include/asm/bitops.h | 1 - xen/include/xen/bitops.h | 5 +++++ 4 files changed, 5 insertions(+), 3 deletions(-) diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h index 91cd167b6bbb..b28c25b3d52d 100644 --- a/xen/arch/arm/include/asm/bitops.h +++ b/xen/arch/arm/include/asm/bitops.h @@ -85,7 +85,6 @@ bool clear_mask16_timeout(uint16_t mask, volatile void *p, * The Hamming Weight of a number is the total number of bits set in it. */ #define hweight64(x) generic_hweight64(x) -#define hweight32(x) generic_hweight32(x) #endif /* _ARM_BITOPS_H */ /* diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h index 64512e949530..f488a7c03425 100644 --- a/xen/arch/ppc/include/asm/bitops.h +++ b/xen/arch/ppc/include/asm/bitops.h @@ -133,6 +133,5 @@ static inline int test_and_set_bit(unsigned int nr, volatile void *addr) * The Hamming Weight of a number is the total number of bits set in it. */ #define hweight64(x) __builtin_popcountll(x) -#define hweight32(x) __builtin_popcount(x) #endif /* _ASM_PPC_BITOPS_H */ diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h index 4c5b21907a64..507b043b8a86 100644 --- a/xen/arch/x86/include/asm/bitops.h +++ b/xen/arch/x86/include/asm/bitops.h @@ -482,6 +482,5 @@ static always_inline unsigned int arch_flsl(unsigned long x) * The Hamming Weight of a number is the total number of bits set in it. */ #define hweight64(x) generic_hweight64(x) -#define hweight32(x) generic_hweight32(x) #endif /* _X86_BITOPS_H */ diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h index 58c600155f7e..a462c3065158 100644 --- a/xen/include/xen/bitops.h +++ b/xen/include/xen/bitops.h @@ -326,6 +326,11 @@ static always_inline attr_const unsigned int hweightl(unsigned long x) #endif } +static always_inline attr_const unsigned int hweight32(uint32_t x) +{ + return hweightl(x); +} + /* --------------------- Please tidy below here --------------------- */ #ifndef find_next_bit From patchwork Wed Aug 28 22:03:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13782021 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8B8C8C7114C for ; Wed, 28 Aug 2024 22:04:23 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.785173.1194666 (Exim 4.92) (envelope-from ) id 1sjQlz-0000bD-0V; Wed, 28 Aug 2024 22:04:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 785173.1194666; Wed, 28 Aug 2024 22:04:14 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQly-0000b1-Sn; Wed, 28 Aug 2024 22:04:14 +0000 Received: by outflank-mailman (input) for mailman id 785173; Wed, 28 Aug 2024 22:04:13 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlw-0006E8-Sa for xen-devel@lists.xenproject.org; Wed, 28 Aug 2024 22:04:12 +0000 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [2a00:1450:4864:20::333]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 7474ae6a-6589-11ef-99a0-01e77a169b0f; Thu, 29 Aug 2024 00:04:11 +0200 (CEST) Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-4280ca0791bso65455495e9.1 for ; Wed, 28 Aug 2024 15:04:11 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c0bb1c2d2esm2695898a12.16.2024.08.28.15.04.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Aug 2024 15:04:08 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 7474ae6a-6589-11ef-99a0-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1724882650; x=1725487450; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RsrqcBVubseMkc2czg+d/Lcx765jVnGIyUSplcSJKR4=; b=D3CKCn9t1BNiwkxkuLB+WwvPpw6Z8+6Lv73F3ZbjFcxLTqE7sJsLZXmbf3DoYTqCBv pFBV+/xjXWrPpVPbltFHpDpiM4efF25Ja+hA5Tqd6c5Tcm+Zuvdb7jNGHv0FQZIo9zLW 5jOY3XwvAu/sqs3HIbRyRL+0/AiSociLXnE2A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724882650; x=1725487450; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RsrqcBVubseMkc2czg+d/Lcx765jVnGIyUSplcSJKR4=; b=RZfJnFEddSefUo7KtSwOQHkhZ2FeqKTsr8trj2g5EuLa0pLTm4iksrfKcgcopDQbhV l44moESqNRBBIeXsrCeYQuGyXEcgy5KPa6Dfpfd6zwWYKZO9qKrYi1wjhypqIAy7cTmy 8FMR8cTC0CdJBZVZZzcIkNZhLUtUQ4TKn4tn0HFt/AKEAqJCC83R299kISL37nSo8pZ2 zWPUXTkinkt7XtMpyYH2sTWQNNy9clqqHKU4o91HICDEc2YmTkS3logEQ5vHyyiaPHtW Hfs0+EWoca+FgayY/UR5gTiZMMzZUGI8pT1MU979+ZcVVM7D3sd3+H2XjsUtRax6Vg5N 2Tcg== X-Gm-Message-State: AOJu0YwioL/bi11+y0f3b4GyZui6fhckNTdZ8iy8WOQ+8qQHMKudbx2X /GxQonCu7IOvsLcd6aaNWtinV1Ccmlvfi3m3JmKWY1fB490EoqfaXIvataDmcAi1ZExW0OQmzih K X-Google-Smtp-Source: AGHT+IH9thU/Kzrpd446ZuHrwCQxl0T9VP7RRAW0814zcCW2MU0Qfv9Jbyb29dJP8AwWWDR2moSOtQ== X-Received: by 2002:a05:6000:b89:b0:371:846d:12b3 with SMTP id ffacd0b85a97d-3749b54d1abmr469774f8f.28.1724882649798; Wed, 28 Aug 2024 15:04:09 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Bertrand Marquis , Michal Orzel , Oleksii Kurochko , Shawn Anastasio Subject: [PATCH v2 10/11] xen/bitops: Implement hweight64() in terms of hweight{l,32}() Date: Wed, 28 Aug 2024 23:03:50 +0100 Message-Id: <20240828220351.2686408-11-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240828220351.2686408-1-andrew.cooper3@citrix.com> References: <20240828220351.2686408-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 ... and drop generic_hweight{32,64}(). This is identical on all architectures except ARM32. Add one extra SELF_TEST to check that hweight64() works when the input is split in half. No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich Acked-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Stefano Stabellini CC: Julien Grall CC: Volodymyr Babchuk CC: Bertrand Marquis CC: Michal Orzel CC: Oleksii Kurochko CC: Shawn Anastasio v2: * Reorder with respect to the hweight32() patch * s/__pure/attr_const/ --- xen/arch/arm/include/asm/bitops.h | 8 ------ xen/arch/ppc/include/asm/bitops.h | 8 ------ xen/arch/x86/include/asm/bitops.h | 8 ------ xen/common/bitops.c | 3 +++ xen/include/xen/bitops.h | 45 ++++++------------------------- 5 files changed, 11 insertions(+), 61 deletions(-) diff --git a/xen/arch/arm/include/asm/bitops.h b/xen/arch/arm/include/asm/bitops.h index b28c25b3d52d..f163d9bb4578 100644 --- a/xen/arch/arm/include/asm/bitops.h +++ b/xen/arch/arm/include/asm/bitops.h @@ -78,14 +78,6 @@ bool clear_mask16_timeout(uint16_t mask, volatile void *p, #define arch_fls(x) ((x) ? 32 - __builtin_clz(x) : 0) #define arch_flsl(x) ((x) ? BITS_PER_LONG - __builtin_clzl(x) : 0) -/** - * hweightN - returns the hamming weight of a N-bit word - * @x: the word to weigh - * - * The Hamming Weight of a number is the total number of bits set in it. - */ -#define hweight64(x) generic_hweight64(x) - #endif /* _ARM_BITOPS_H */ /* * Local variables: diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h index f488a7c03425..c942e9432e20 100644 --- a/xen/arch/ppc/include/asm/bitops.h +++ b/xen/arch/ppc/include/asm/bitops.h @@ -126,12 +126,4 @@ static inline int test_and_set_bit(unsigned int nr, volatile void *addr) #define arch_hweightl(x) __builtin_popcountl(x) -/** - * hweightN - returns the hamming weight of a N-bit word - * @x: the word to weigh - * - * The Hamming Weight of a number is the total number of bits set in it. - */ -#define hweight64(x) __builtin_popcountll(x) - #endif /* _ASM_PPC_BITOPS_H */ diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h index 507b043b8a86..642d8e58b288 100644 --- a/xen/arch/x86/include/asm/bitops.h +++ b/xen/arch/x86/include/asm/bitops.h @@ -475,12 +475,4 @@ static always_inline unsigned int arch_flsl(unsigned long x) } #define arch_flsl arch_flsl -/** - * hweightN - returns the hamming weight of a N-bit word - * @x: the word to weigh - * - * The Hamming Weight of a number is the total number of bits set in it. - */ -#define hweight64(x) generic_hweight64(x) - #endif /* _X86_BITOPS_H */ diff --git a/xen/common/bitops.c b/xen/common/bitops.c index 5e5d20d225d7..91ae961440af 100644 --- a/xen/common/bitops.c +++ b/xen/common/bitops.c @@ -144,6 +144,9 @@ static void __init test_hweight(void) CHECK(hweightl, 1 | (1UL << (BITS_PER_LONG - 1)), 2); CHECK(hweightl, -1UL, BITS_PER_LONG); + + /* unsigned int hweight64(uint64_t) */ + CHECK(hweight64, -1ULL, 64); } static void __init __constructor test_bitops(void) diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h index a462c3065158..c4cb432eed23 100644 --- a/xen/include/xen/bitops.h +++ b/xen/include/xen/bitops.h @@ -331,6 +331,14 @@ static always_inline attr_const unsigned int hweight32(uint32_t x) return hweightl(x); } +static always_inline attr_const unsigned int hweight64(uint64_t x) +{ + if ( BITS_PER_LONG == 64 ) + return hweightl(x); + else + return hweight32(x >> 32) + hweight32(x); +} + /* --------------------- Please tidy below here --------------------- */ #ifndef find_next_bit @@ -399,43 +407,6 @@ static inline int get_count_order(unsigned int count) return order; } -/* - * hweightN: returns the hamming weight (i.e. the number - * of bits set) of a N-bit word - */ - -static inline unsigned int generic_hweight32(unsigned int w) -{ - w -= (w >> 1) & 0x55555555; - w = (w & 0x33333333) + ((w >> 2) & 0x33333333); - w = (w + (w >> 4)) & 0x0f0f0f0f; - - if ( IS_ENABLED(CONFIG_HAS_FAST_MULTIPLY) ) - return (w * 0x01010101) >> 24; - - w += w >> 8; - - return (w + (w >> 16)) & 0xff; -} - -static inline unsigned int generic_hweight64(uint64_t w) -{ - if ( BITS_PER_LONG < 64 ) - return generic_hweight32(w >> 32) + generic_hweight32(w); - - w -= (w >> 1) & 0x5555555555555555UL; - w = (w & 0x3333333333333333UL) + ((w >> 2) & 0x3333333333333333UL); - w = (w + (w >> 4)) & 0x0f0f0f0f0f0f0f0fUL; - - if ( IS_ENABLED(CONFIG_HAS_FAST_MULTIPLY) ) - return (w * 0x0101010101010101UL) >> 56; - - w += w >> 8; - w += w >> 16; - - return (w + (w >> 32)) & 0xFF; -} - /* * rol32 - rotate a 32-bit value left * From patchwork Wed Aug 28 22:03:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13782022 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 69E3CC71151 for ; Wed, 28 Aug 2024 22:04:25 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.785174.1194671 (Exim 4.92) (envelope-from ) id 1sjQlz-0000f2-Hj; Wed, 28 Aug 2024 22:04:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 785174.1194671; Wed, 28 Aug 2024 22:04:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlz-0000dx-9d; Wed, 28 Aug 2024 22:04:15 +0000 Received: by outflank-mailman (input) for mailman id 785174; Wed, 28 Aug 2024 22:04:13 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sjQlw-0006E0-Ub for xen-devel@lists.xenproject.org; Wed, 28 Aug 2024 22:04:12 +0000 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [2a00:1450:4864:20::535]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 7505a605-6589-11ef-a0b0-8be0dac302b0; Thu, 29 Aug 2024 00:04:12 +0200 (CEST) Received: by mail-ed1-x535.google.com with SMTP id 4fb4d7f45d1cf-5c210e23573so1833151a12.0 for ; Wed, 28 Aug 2024 15:04:12 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c0bb1c2d2esm2695898a12.16.2024.08.28.15.04.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Aug 2024 15:04:10 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 7505a605-6589-11ef-a0b0-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1724882651; x=1725487451; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Sb7srVii8m7D5bOp7Td/YLt3PcEfAn83ipVBC8MF2p4=; b=CHV7GAiOq4okjKbi0Rb2cYBHQMAiXHFQzvejNaWJrcZ2pwRnodad6wjrwPTaVxOeui GqTpyUeVKoPjQ4LYDZp8l6Fejdg0NceeWEowYdTbAZ5X0JFV+xW7IbZ8yB2imj9vtK2T wT4wY+Fw0IhkMqriJSbT+COm/GmYM4+ymyW7k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724882651; x=1725487451; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Sb7srVii8m7D5bOp7Td/YLt3PcEfAn83ipVBC8MF2p4=; b=ACu1DtvWIdsVUd28HBx3S/HL8FOLk7ETO54iqr7twuirZh1wmcimFTkox9gRy9YLea CTk3BN7l1B4PcvFNmfagmY3TlVDArqCOFKVdFccUV74vzic5Qnw5cWeGI47ktym8oZhE MGuH43f05YWxOhyEiVa7mSS2vcTN0V4BEyucfD+YawMYZaJkb/4hzv36+Lw/yMTEmTq9 WMRQdNU95O9+2APAMRb00qfYOz9LrtgEPFrfcqcDtVLN9Yj0SAmedEV7kmHmWUBP+UMX C/VT5oLagu+Wvs+iKW8SPTh4XTShA+pZdtzS8emMpMnZXA7JBaBc4PAIoGJYZQ3MVe1O fLbg== X-Gm-Message-State: AOJu0YxbwWqF+aMTFhyZrruV/uiQ/a++sHmpkCNnxFUUfn/au9LHP2xl HYIGe0fluUVHdgzt1RlxOG1r9oWygZttoqjGIgUNU9m5K+w6nEIWnQnvDb+IfkblsdY77lEvNgZ O X-Google-Smtp-Source: AGHT+IGKW3SvmGx+8Qmh8o8prGpaIiHhPr6Oq+77bbpVyS4CCBGopTL/2h4wnYqadHJpcMBYRKfyqw== X-Received: by 2002:a05:6402:2554:b0:5be:ef1f:c679 with SMTP id 4fb4d7f45d1cf-5c21ed8e6c7mr686193a12.23.1724882650506; Wed, 28 Aug 2024 15:04:10 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v2 11/11] x86/bitops: Use the POPCNT instruction when available Date: Wed, 28 Aug 2024 23:03:51 +0100 Message-Id: <20240828220351.2686408-12-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240828220351.2686408-1-andrew.cooper3@citrix.com> References: <20240828220351.2686408-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 It has existed in x86 CPUs since 2008, so we're only 16 years late adding support. With all the other scafolding in place, implement arch_hweightl() for x86. The only complication is that the call to arch_generic_hweightl() is behind the compilers back. Address this by writing it in ASM and ensure that it preserves all registers. Copy the code generation from generic_hweightl(). It's not a complicated algorithm, and is easy to regenerate if needs be, but cover it with the same unit tests as test_generic_hweightl() just for piece of mind. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné v2: * Fix MISRA 8.2 (parameter name) and 8.5 (single declaration) regressions. * Rename {arch->x86}-generic-hweightl.{S->c} * Adjust ASM formating The __constructor trick to cause any reference to $foo() to pull in test_$foo() only works when both are in the same TU. i.e. what I did in v1 (putting test_arch_generic_hweightl() in the regular generic-hweightl.c) didn't work. This in turn means that arch_generic_hweightl() needs writing in a global asm block, and also that we can't use FUNC()/END(). While we could adjust it to work for GCC/binutils, we can't have CPP macros in Clang-IAS strings. I don't particularly like opencoding FUNC()/END(), but I can't think of anything better. --- xen/arch/x86/include/asm/bitops.h | 23 +++++++++++ xen/lib/Makefile | 1 + xen/lib/x86-generic-hweightl.c | 69 +++++++++++++++++++++++++++++++ 3 files changed, 93 insertions(+) create mode 100644 xen/lib/x86-generic-hweightl.c diff --git a/xen/arch/x86/include/asm/bitops.h b/xen/arch/x86/include/asm/bitops.h index 642d8e58b288..39e37f1cbe55 100644 --- a/xen/arch/x86/include/asm/bitops.h +++ b/xen/arch/x86/include/asm/bitops.h @@ -6,6 +6,7 @@ */ #include +#include #include /* @@ -475,4 +476,26 @@ static always_inline unsigned int arch_flsl(unsigned long x) } #define arch_flsl arch_flsl +unsigned int arch_generic_hweightl(unsigned long x); + +static always_inline unsigned int arch_hweightl(unsigned long x) +{ + unsigned int r; + + /* + * arch_generic_hweightl() is written in ASM in order to preserve all + * registers, as the compiler can't see the call. + * + * This limits the POPCNT instruction to using the same ABI as a function + * call (input in %rdi, output in %eax) but that's fine. + */ + alternative_io("call arch_generic_hweightl", + "popcnt %[val], %q[res]", X86_FEATURE_POPCNT, + ASM_OUTPUT2([res] "=a" (r) ASM_CALL_CONSTRAINT), + [val] "D" (x)); + + return r; +} +#define arch_hweightl arch_hweightl + #endif /* _X86_BITOPS_H */ diff --git a/xen/lib/Makefile b/xen/lib/Makefile index b6558e108bd9..54440f628aae 100644 --- a/xen/lib/Makefile +++ b/xen/lib/Makefile @@ -36,6 +36,7 @@ lib-y += strtol.o lib-y += strtoll.o lib-y += strtoul.o lib-y += strtoull.o +lib-$(CONFIG_X86) += x86-generic-hweightl.o lib-$(CONFIG_X86) += xxhash32.o lib-$(CONFIG_X86) += xxhash64.o diff --git a/xen/lib/x86-generic-hweightl.c b/xen/lib/x86-generic-hweightl.c new file mode 100644 index 000000000000..e0be25a01c1d --- /dev/null +++ b/xen/lib/x86-generic-hweightl.c @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#include +#include +#include + +/* + * An implementation of generic_hweightl() used on hardware without the POPCNT + * instruction. + * + * This function is called from within an ALTERNATIVE in arch_hweightl(). + * i.e. behind the back of the compiler. Therefore all registers are callee + * preserved. + * + * The ASM is what GCC-12 emits for generic_hweightl() in a release build of + * Xen, with spilling of %rdi/%rdx to preserve the callers registers. + * + * Note: When we can use __attribute__((no_caller_saved_registers)) + * unconditionally (GCC 7, Clang 5), we can implement this in plain C. + */ +asm ( + ".type arch_generic_hweightl, STT_FUNC\n\t" + ".globl arch_generic_hweightl\n\t" + ".hidden arch_generic_hweightl\n\t" + ".balign " STR(CONFIG_FUNCTION_ALIGNMENT) ", 0x90\n\t" + "arch_generic_hweightl:\n\t" + + "push %rdi\n\t" + "push %rdx\n\t" + + "movabs $0x5555555555555555, %rdx\n\t" + "mov %rdi, %rax\n\t" + "shr $1, %rax\n\t" + "and %rdx, %rax\n\t" + "sub %rax, %rdi\n\t" + "movabs $0x3333333333333333, %rax\n\t" + "mov %rdi, %rdx\n\t" + "shr $2, %rdi\n\t" + "and %rax, %rdx\n\t" + "and %rax, %rdi\n\t" + "add %rdi, %rdx\n\t" + "mov %rdx, %rax\n\t" + "shr $4, %rax\n\t" + "add %rdx, %rax\n\t" + "movabs $0x0f0f0f0f0f0f0f0f, %rdx\n\t" + "and %rdx, %rax\n\t" + "movabs $0x0101010101010101, %rdx\n\t" + "imul %rdx, %rax\n\t" + "shr $" STR(BITS_PER_LONG) "- 8, %rax\n\t" + + "pop %rdx\n\t" + "pop %rdi\n\t" + + "ret\n\t" + + ".size arch_generic_hweightl, . - arch_generic_hweightl\n\t" +); + +static void __init __constructor test_arch_generic_hweightl(void) +{ + RUNTIME_CHECK(arch_generic_hweightl, 0, 0); + RUNTIME_CHECK(arch_generic_hweightl, 1, 1); + RUNTIME_CHECK(arch_generic_hweightl, 3, 2); + RUNTIME_CHECK(arch_generic_hweightl, 7, 3); + RUNTIME_CHECK(arch_generic_hweightl, 0xff, 8); + + RUNTIME_CHECK(arch_generic_hweightl, 1 | (1UL << (BITS_PER_LONG - 1)), 2); + RUNTIME_CHECK(arch_generic_hweightl, -1UL, BITS_PER_LONG); +}