From patchwork Tue Dec 15 18:28:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uros Bizjak X-Patchwork-Id: 11975581 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 952E4C4361B for ; Tue, 15 Dec 2020 18:29:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 61F5A22B2D for ; Tue, 15 Dec 2020 18:29:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731564AbgLOS3i (ORCPT ); Tue, 15 Dec 2020 13:29:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51258 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731330AbgLOS3h (ORCPT ); Tue, 15 Dec 2020 13:29:37 -0500 Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E830AC06179C; Tue, 15 Dec 2020 10:28:26 -0800 (PST) Received: by mail-ed1-x52b.google.com with SMTP id v22so22079662edt.9; Tue, 15 Dec 2020 10:28:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BxbWIyImdzA25H+jox/xdDou7lPmQDmqOAV4NhorkHw=; b=anIFo0HqTUq6Q8OaS1iLz+7sCpz1Y4IOXhJShH09+NYu0z9Rr0qoEuOoDxbrJ7gzyb 2a62VVwMNDulYctUXaNPfBDjKX50+5yIqhSUXnJz/cZefSNG7nlkbHKbWCvjt5cg+jY1 6TeVsaZa2rrMsBB0GnJXUNPw7kKlXJkMOJpiucQdaCgmg/ooxGyzw6hViXA2EnW000j/ M4rvd0QxEI7vCFFRXZYNvc2sx43swlLmKhMKkxtojgpTrNQI9qt1pbbKo2PgtNTAhhdp PDvIBld6sAdaVONM7RLJ/oPYJZSpWwzvaHHcxhLpE5hP8d4J+wc3uucVsI5eQlzWTfvZ gKgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BxbWIyImdzA25H+jox/xdDou7lPmQDmqOAV4NhorkHw=; b=BzlUoHHgyrlTkMykKVGgrGLMvb0VAswGgLfCaPDpUwKWaw152NF0ZQV3+3SBARhZY6 +GsmCujFEceapsgEi7cRQU3H1PDvHPT76iy/jNw496Q5bMZ8tbVPWAyZncy2i8JVQ2Db +Ya/S+ygi6b3hweRL5RDZxYSu8su4l55ZOkp0WGxu+m/JrpYzaQrAR4ihS9U4kPGX2Qk YXoNiP7amzigEN5x1jL9tbSEnf3Xcxw+Do3AtMR5nwRcKFa8D86gFGg8mg52n99/NcPN Gn4CJpmmbomk6I8z15DTLbupgNaMxSNDlNWXS/ue2298v6KTuoa852Qk7X10+V/3X32O wuMQ== X-Gm-Message-State: AOAM533+3qzm9gOx5NGwjK31UM5o+A/Bmpka2vPOjLCVBgfjtb3Bwd92 HH/2tubl7bQWssQeehawH14= X-Google-Smtp-Source: ABdhPJyox0k1n+HV1FwaAD5wwobWoXwd0wa0XsldkrCgtmGdlyI4j0i36Yg+iy3nzoqOjwxlEchmOQ== X-Received: by 2002:aa7:d750:: with SMTP id a16mr31144433eds.252.1608056905681; Tue, 15 Dec 2020 10:28:25 -0800 (PST) Received: from localhost.localdomain (93-103-18-160.static.t-2.net. [93.103.18.160]) by smtp.gmail.com with ESMTPSA id r21sm7369228eds.91.2020.12.15.10.28.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Dec 2020 10:28:25 -0800 (PST) From: Uros Bizjak To: x86@kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Uros Bizjak , Will Deacon , Peter Zijlstra , Boqun Feng Subject: [PATCH 1/3] asm-generic/atomic: Add try_cmpxchg64() instrumentation Date: Tue, 15 Dec 2020 19:28:03 +0100 Message-Id: <20201215182805.53913-2-ubizjak@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201215182805.53913-1-ubizjak@gmail.com> References: <20201215182805.53913-1-ubizjak@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Instrument try_cmpxchg64() similar to try_cmpxchg(). Signed-off-by: Uros Bizjak Cc: Will Deacon Cc: Peter Zijlstra Cc: Boqun Feng --- include/asm-generic/atomic-instrumented.h | 46 ++++++++++++++++++++++- scripts/atomic/gen-atomic-instrumented.sh | 2 +- 2 files changed, 46 insertions(+), 2 deletions(-) diff --git a/include/asm-generic/atomic-instrumented.h b/include/asm-generic/atomic-instrumented.h index 888b6cfeed91..396d4484963a 100644 --- a/include/asm-generic/atomic-instrumented.h +++ b/include/asm-generic/atomic-instrumented.h @@ -1793,6 +1793,50 @@ atomic64_dec_if_positive(atomic64_t *v) }) #endif +#if !defined(arch_try_cmpxchg64_relaxed) || defined(arch_try_cmpxchg64) +#define try_cmpxchg64(ptr, oldp, ...) \ +({ \ + typeof(ptr) __ai_ptr = (ptr); \ + typeof(oldp) __ai_oldp = (oldp); \ + instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \ + instrument_atomic_write(__ai_oldp, sizeof(*__ai_oldp)); \ + arch_try_cmpxchg64(__ai_ptr, __ai_oldp, __VA_ARGS__); \ +}) +#endif + +#if defined(arch_try_cmpxchg64_acquire) +#define try_cmpxchg64_acquire(ptr, oldp, ...) \ +({ \ + typeof(ptr) __ai_ptr = (ptr); \ + typeof(oldp) __ai_oldp = (oldp); \ + instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \ + instrument_atomic_write(__ai_oldp, sizeof(*__ai_oldp)); \ + arch_try_cmpxchg64_acquire(__ai_ptr, __ai_oldp, __VA_ARGS__); \ +}) +#endif + +#if defined(arch_try_cmpxchg64_release) +#define try_cmpxchg64_release(ptr, oldp, ...) \ +({ \ + typeof(ptr) __ai_ptr = (ptr); \ + typeof(oldp) __ai_oldp = (oldp); \ + instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \ + instrument_atomic_write(__ai_oldp, sizeof(*__ai_oldp)); \ + arch_try_cmpxchg64_release(__ai_ptr, __ai_oldp, __VA_ARGS__); \ +}) +#endif + +#if defined(arch_try_cmpxchg64_relaxed) +#define try_cmpxchg64_relaxed(ptr, oldp, ...) \ +({ \ + typeof(ptr) __ai_ptr = (ptr); \ + typeof(oldp) __ai_oldp = (oldp); \ + instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \ + instrument_atomic_write(__ai_oldp, sizeof(*__ai_oldp)); \ + arch_try_cmpxchg64_relaxed(__ai_ptr, __ai_oldp, __VA_ARGS__); \ +}) +#endif + #define cmpxchg_local(ptr, ...) \ ({ \ typeof(ptr) __ai_ptr = (ptr); \ @@ -1830,4 +1874,4 @@ atomic64_dec_if_positive(atomic64_t *v) }) #endif /* _ASM_GENERIC_ATOMIC_INSTRUMENTED_H */ -// 4bec382e44520f4d8267e42620054db26a659ea3 +// 28b0549ca119ed58a9b61da6e277ee30de18bdfc diff --git a/scripts/atomic/gen-atomic-instrumented.sh b/scripts/atomic/gen-atomic-instrumented.sh index 5766ffcec7c5..ade2e509b5eb 100755 --- a/scripts/atomic/gen-atomic-instrumented.sh +++ b/scripts/atomic/gen-atomic-instrumented.sh @@ -186,7 +186,7 @@ grep '^[a-z]' "$1" | while read name meta args; do gen_proto "${meta}" "${name}" "atomic64" "s64" ${args} done -for xchg in "xchg" "cmpxchg" "cmpxchg64" "try_cmpxchg"; do +for xchg in "xchg" "cmpxchg" "cmpxchg64" "try_cmpxchg" "try_cmpxchg64"; do for order in "" "_acquire" "_release" "_relaxed"; do gen_optional_xchg "${xchg}" "${order}" done From patchwork Tue Dec 15 18:28:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uros Bizjak X-Patchwork-Id: 11975595 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9A3DC4361B for ; Tue, 15 Dec 2020 18:32:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A84AD227C3 for ; Tue, 15 Dec 2020 18:32:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731409AbgLOS3e (ORCPT ); Tue, 15 Dec 2020 13:29:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730346AbgLOS3W (ORCPT ); Tue, 15 Dec 2020 13:29:22 -0500 Received: from mail-ed1-x543.google.com (mail-ed1-x543.google.com [IPv6:2a00:1450:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 76378C0611CE; Tue, 15 Dec 2020 10:28:30 -0800 (PST) Received: by mail-ed1-x543.google.com with SMTP id i24so22060223edj.8; Tue, 15 Dec 2020 10:28:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=C/f96L+Qey1SIv38JhHlB/Gh2jbhdjxdYrDSCqGmPs4=; b=GRG7AbYTBer5jpfXbhM3wUrl3QJMvscjSBggN7duh9Fhm4RsKTw8PHfQHKOA2znL3z pN3+LB+kVCIdyHOmFLOmQSWS3p036gDp0B9I4tQfs43ju+Bh3Iby9ACFbWl67pPoSd2P NQ1niL4otPZDso5tI23KnkOtStq2juV4a6wsLNjvdjxEL6IH7taBBwKGaBd98XXaxDLt lVKYWdn0fxg9b2ynQlUsRysrujzt4cWogoepI31hwCPg7q0egFEGz4PgdGhzoFh16XaA lPDDel5pk3cbfX1TDajdXqz4r1r7Eaoukw/n+MLC+cvM1QcI5l7UuEXSEUc59q3CQ7YP SPtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=C/f96L+Qey1SIv38JhHlB/Gh2jbhdjxdYrDSCqGmPs4=; b=HZH1wSQWBbFsr2fA17WdXIoLfLn1FbErqzRBH79WIJJhDgwKPuMFQejCBrfsAy+2l0 dJ+xLH+h+vz9KXEouUMVL5IQZ+ItEqWoonoKzogSaTAVsqRqX8nXGaebqX4y4ZBCyeWO ryw5AHbu18VwXTXnijS0VRvUUdzVShv8vz/PclZBBbx1arsIxGTttuHV4nky11F6MQC9 wB9i6YusvDc8rOAcEUoybRknHAx6bQooTjVldbfaW4U04gisiSSShHvj7n5v4B95PXxI u6r6LqxWs08kldvxL3RdV0E3i86GlC2AVqKUhc6RAbS93+vtKRLG5z8UgD3afAJoKkHn BoVg== X-Gm-Message-State: AOAM533dep4N2jqGOPUHGSmKj4GI1f3cnqEEzQhWJIyFZLNDHRlHOcQa T4XOLmKyV+erfKvoUj9V48g= X-Google-Smtp-Source: ABdhPJxgSdvIleJuuGkLQ7F6EYdxASBIk7AjEcW6asusXT5HS15COUxjJM+w9octiJriD6AS/MAR0g== X-Received: by 2002:a05:6402:1d9a:: with SMTP id dk26mr31091139edb.283.1608056909156; Tue, 15 Dec 2020 10:28:29 -0800 (PST) Received: from localhost.localdomain (93-103-18-160.static.t-2.net. [93.103.18.160]) by smtp.gmail.com with ESMTPSA id r21sm7369228eds.91.2020.12.15.10.28.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Dec 2020 10:28:28 -0800 (PST) From: Uros Bizjak To: x86@kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Uros Bizjak , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" Subject: [PATCH 2/3] locking/atomic/x86: Introduce arch_try_cmpxchg64() Date: Tue, 15 Dec 2020 19:28:04 +0100 Message-Id: <20201215182805.53913-3-ubizjak@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201215182805.53913-1-ubizjak@gmail.com> References: <20201215182805.53913-1-ubizjak@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add arch_try_cmpxchg64(), similar to arch_try_cmpxchg(), that operates with 64 bit operands. This function provides the same interface for 32 bit and 64 bit targets. Signed-off-by: Uros Bizjak Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: "H. Peter Anvin" Reported-by: kernel test robot --- arch/x86/include/asm/cmpxchg_32.h | 62 ++++++++++++++++++++++++++----- arch/x86/include/asm/cmpxchg_64.h | 6 +++ 2 files changed, 59 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/cmpxchg_32.h b/arch/x86/include/asm/cmpxchg_32.h index 0a7fe0321613..8dcde400244e 100644 --- a/arch/x86/include/asm/cmpxchg_32.h +++ b/arch/x86/include/asm/cmpxchg_32.h @@ -35,15 +35,6 @@ static inline void set_64bit(volatile u64 *ptr, u64 value) : "memory"); } -#ifdef CONFIG_X86_CMPXCHG64 -#define arch_cmpxchg64(ptr, o, n) \ - ((__typeof__(*(ptr)))__cmpxchg64((ptr), (unsigned long long)(o), \ - (unsigned long long)(n))) -#define arch_cmpxchg64_local(ptr, o, n) \ - ((__typeof__(*(ptr)))__cmpxchg64_local((ptr), (unsigned long long)(o), \ - (unsigned long long)(n))) -#endif - static inline u64 __cmpxchg64(volatile u64 *ptr, u64 old, u64 new) { u64 prev; @@ -71,6 +62,39 @@ static inline u64 __cmpxchg64_local(volatile u64 *ptr, u64 old, u64 new) } #ifndef CONFIG_X86_CMPXCHG64 +#define arch_cmpxchg64(ptr, o, n) \ + ((__typeof__(*(ptr)))__cmpxchg64((ptr), (unsigned long long)(o), \ + (unsigned long long)(n))) +#define arch_cmpxchg64_local(ptr, o, n) \ + ((__typeof__(*(ptr)))__cmpxchg64_local((ptr), (unsigned long long)(o), \ + +#define __raw_try_cmpxchg64(_ptr, _pold, _new, lock) \ +({ \ + bool success; \ + __typeof__(_ptr) _old = (__typeof__(_ptr))(_pold); \ + __typeof__(*(_ptr)) __old = *_old; \ + __typeof__(*(_ptr)) __new = (_new); \ + asm volatile(lock "cmpxchg8b %1" \ + CC_SET(z) \ + : CC_OUT(z) (success), \ + "+m" (*_ptr), \ + "+A" (__old) \ + : "b" ((unsigned int)__new), \ + "c" ((unsigned int)(__new>>32)) \ + : "memory"); \ + if (unlikely(!success)) \ + *_old = __old; \ + likely(success); \ +}) + +#define __try_cmpxchg64(ptr, pold, new) \ + __raw_try_cmpxchg64((ptr), (pold), (new), LOCK_PREFIX) + +#define arch_try_cmpxchg64(ptr, pold, new) \ + __try_cmpxchg64((ptr), (pold), (new)) + +#else + /* * Building a kernel capable running on 80386 and 80486. It may be necessary * to simulate the cmpxchg8b on the 80386 and 80486 CPU. @@ -108,6 +132,26 @@ static inline u64 __cmpxchg64_local(volatile u64 *ptr, u64 old, u64 new) : "memory"); \ __ret; }) +#define arch_try_cmpxchg64(ptr, po, n) \ +({ \ + bool success; \ + __typeof__(ptr) _old = (__typeof__(ptr))(po); \ + __typeof__(*(ptr)) __old = *_old; \ + __typeof__(*(ptr)) __new = (n); \ + alternative_io(LOCK_PREFIX_HERE \ + "call cmpxchg8b_emu", \ + "lock; cmpxchg8b (%%esi)" , \ + X86_FEATURE_CX8, \ + "+A" (__old), \ + "S" ((ptr)), \ + "b" ((unsigned int)__new), \ + "c" ((unsigned int)(__new>>32)) \ + : "memory"); \ + success = (__old == *_old); \ + if (unlikely(!success)) \ + *_old = __old; \ + likely(success); \ +}) #endif #define system_has_cmpxchg_double() boot_cpu_has(X86_FEATURE_CX8) diff --git a/arch/x86/include/asm/cmpxchg_64.h b/arch/x86/include/asm/cmpxchg_64.h index 072e5459fe2f..250187ac8248 100644 --- a/arch/x86/include/asm/cmpxchg_64.h +++ b/arch/x86/include/asm/cmpxchg_64.h @@ -19,6 +19,12 @@ static inline void set_64bit(volatile u64 *ptr, u64 val) arch_cmpxchg_local((ptr), (o), (n)); \ }) +#define arch_try_cmpxchg64(ptr, po, n) \ +({ \ + BUILD_BUG_ON(sizeof(*(ptr)) != 8); \ + arch_try_cmpxchg((ptr), (po), (n)); \ +}) + #define system_has_cmpxchg_double() boot_cpu_has(X86_FEATURE_CX16) #endif /* _ASM_X86_CMPXCHG_64_H */ From patchwork Tue Dec 15 18:28:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uros Bizjak X-Patchwork-Id: 11975583 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99A9BC2BBCA for ; Tue, 15 Dec 2020 18:31:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 59604227C3 for ; Tue, 15 Dec 2020 18:31:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731498AbgLOS3h (ORCPT ); Tue, 15 Dec 2020 13:29:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51162 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730472AbgLOS3W (ORCPT ); Tue, 15 Dec 2020 13:29:22 -0500 Received: from mail-ed1-x541.google.com (mail-ed1-x541.google.com [IPv6:2a00:1450:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AC092C0611CF; Tue, 15 Dec 2020 10:28:33 -0800 (PST) Received: by mail-ed1-x541.google.com with SMTP id b73so22064861edf.13; Tue, 15 Dec 2020 10:28:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=doIMCM/CToq9Jbv6jz6zVoEhuL9pFBHCD93XAYTZiak=; b=e+vgd4s4pLeiGcmeLRWukldHDTnE5CMva8Y5p/xy5be4OToCaYiDvlDO6stecGIbyh zj7Kzi5OMpHFHrFOdl8ogEgAp1zHLaNMZSV/I57wZanLYJOi8u82Gt1m5W2oFmHS40iD 3q+YV9KFBfzV6jd/ZNo7SzzKETYN8lme8ZehYinQevV9HCMbepBVw9AY24+hY84JFMJG jzBzKJfnqm9qV8VRiY9DC7xuoGypjsxHhuEtmvWXvKz0RitW5DcdpMF19BrXdgqU9hdv Q4+Bg8EiDylm1NWCP8zTyRFHvHIntL6WeA1efifRxOmJ/IdEBREW75CvNIYyv70GbAwI Ll4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=doIMCM/CToq9Jbv6jz6zVoEhuL9pFBHCD93XAYTZiak=; b=Nuh6sILpHS593dlsajhXWUy6AvOrvEufrmfm660KGHQuwCFa4BxdM2C1JhKH45BtOG +9SAP9p5ZO5q9AWMZCn/6IpZa0vg/PGdw2Bec/ayv/5Vrj0SA96Tq1NMI+/zHF6wJnud banAOgaMwuJ7LYXECY9imNBEMZ81rWQhZqW6azb7f2tC1t0cgQe5D/mJVN51urILV9qB HoltuUdhr8TzEXMHlXZKLzvsrXdKGR2H9Ttf4x9y9UKxB7rvpTXojOg5BZF6QIAMXZED p+jjjHLuTXtYSI/EWPx1vEYixZ3cY8IWNlcvKJfWPbWTW35VluEzO9JWgaTy2qSqB5nL VIaA== X-Gm-Message-State: AOAM531ZYQvaNMoZLE5tHQJd81vRpl+Q9iRp0BseKyJ7aHUtApY3kXJw YYznXtJtAkzNvERxd9bg4+Q= X-Google-Smtp-Source: ABdhPJwnj92sUfDJzxNZoIDbn3gTJ53XjZjYpEbpUTUwa5ohjSsFzr7O3uH+RR5alWgDSOxFszCFNg== X-Received: by 2002:a50:d484:: with SMTP id s4mr30952192edi.13.1608056912382; Tue, 15 Dec 2020 10:28:32 -0800 (PST) Received: from localhost.localdomain (93-103-18-160.static.t-2.net. [93.103.18.160]) by smtp.gmail.com with ESMTPSA id r21sm7369228eds.91.2020.12.15.10.28.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Dec 2020 10:28:31 -0800 (PST) From: Uros Bizjak To: x86@kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Uros Bizjak , Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel Subject: [PATCH 3/3] KVM/VMX: Use try_cmpxchg64() in posted_intr.c Date: Tue, 15 Dec 2020 19:28:05 +0100 Message-Id: <20201215182805.53913-4-ubizjak@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201215182805.53913-1-ubizjak@gmail.com> References: <20201215182805.53913-1-ubizjak@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Use try_cmpxchg64() instead of cmpxchg64() to reuse flags from cmpxchg/cmpxchg8b instruction. For 64 bit targets flags reuse avoids a CMP instruction, while for 32 bit targets flags reuse avoids XOR/XOR/OR instruction sequence. Signed-off-by: Uros Bizjak Cc: Paolo Bonzini Cc: Sean Christopherson Cc: Vitaly Kuznetsov Cc: Wanpeng Li Cc: Jim Mattson Cc: Joerg Roedel Reviewed-by: Sean Christopherson --- arch/x86/kvm/vmx/posted_intr.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/vmx/posted_intr.c b/arch/x86/kvm/vmx/posted_intr.c index f02962dcc72c..47b47970fb46 100644 --- a/arch/x86/kvm/vmx/posted_intr.c +++ b/arch/x86/kvm/vmx/posted_intr.c @@ -60,8 +60,7 @@ void vmx_vcpu_pi_load(struct kvm_vcpu *vcpu, int cpu) new.ndst = (dest << 8) & 0xFF00; new.sn = 0; - } while (cmpxchg64(&pi_desc->control, old.control, - new.control) != old.control); + } while (!try_cmpxchg64(&pi_desc->control, &old.control, new.control)); after_clear_sn: @@ -111,8 +110,7 @@ static void __pi_post_block(struct kvm_vcpu *vcpu) /* set 'NV' to 'notification vector' */ new.nv = POSTED_INTR_VECTOR; - } while (cmpxchg64(&pi_desc->control, old.control, - new.control) != old.control); + } while (!try_cmpxchg64(&pi_desc->control, &old.control, new.control)); if (!WARN_ON_ONCE(vcpu->pre_pcpu == -1)) { spin_lock(&per_cpu(blocked_vcpu_on_cpu_lock, vcpu->pre_pcpu)); @@ -181,8 +179,7 @@ int pi_pre_block(struct kvm_vcpu *vcpu) /* set 'NV' to 'wakeup vector' */ new.nv = POSTED_INTR_WAKEUP_VECTOR; - } while (cmpxchg64(&pi_desc->control, old.control, - new.control) != old.control); + } while (!try_cmpxchg64(&pi_desc->control, &old.control, new.control)); /* We should not block the vCPU if an interrupt is posted for it. */ if (pi_test_on(pi_desc) == 1)