From patchwork Thu Sep 26 01:34:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Houghton X-Patchwork-Id: 13812678 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2938CCF9EB for ; Thu, 26 Sep 2024 01:35:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 050806B0083; Wed, 25 Sep 2024 21:35:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EEBA06B00B6; Wed, 25 Sep 2024 21:35:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D67DA6B00B8; Wed, 25 Sep 2024 21:35:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B29396B00B5 for ; Wed, 25 Sep 2024 21:35:21 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2AA661411BB for ; Thu, 26 Sep 2024 01:35:21 +0000 (UTC) X-FDA: 82605171642.27.AB45ECC Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf13.hostedemail.com (Postfix) with ESMTP id 55F8E20003 for ; Thu, 26 Sep 2024 01:35:19 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=O+4W2xzc; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of 3Vrr0ZgoKCNsGQELRDEQLKDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--jthoughton.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3Vrr0ZgoKCNsGQELRDEQLKDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--jthoughton.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727314503; a=rsa-sha256; cv=none; b=x7E02KM8FTj1y4NW3g6wFQE3eqszTQ323WMTvFI6FeMqNKzOkUexMZRXg585OX6C3WZjcH 0UAouQ/MEbzTJV5F315U8AT7/D4kMho3TqI6zte1xVJy+yG0JgINswadsjy+FajLedeRWD eBStCGeAcekd9pMVh9N44glqYzz3NGM= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=O+4W2xzc; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of 3Vrr0ZgoKCNsGQELRDEQLKDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--jthoughton.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3Vrr0ZgoKCNsGQELRDEQLKDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--jthoughton.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727314503; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CIRgMt/tPt8dDHjlNVeDudvk1PSxwGvmBpyKYz5f4us=; b=GJS7pmTjynfGT/5LY23Z7YhFIsKOK4Ylk05XXkd5pnvJJBfQLMoRrb10El7fAlduZx4wdf 0srjyX7sA+YsxFk7ijLzsQ6fMxX38KphkIq9HZIyCzHN3IFBpTtBbc0mbn8KEYfPTtsjv1 w0QmSkgfdojnW4wu96L6f6I456kZpNk= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6d5235d1bcaso9766907b3.2 for ; Wed, 25 Sep 2024 18:35:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1727314518; x=1727919318; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=CIRgMt/tPt8dDHjlNVeDudvk1PSxwGvmBpyKYz5f4us=; b=O+4W2xzcXxIAJtkp9LBRIVdLqSfSbu2Q6B4OMddwcmWQ4eQcP6b692wXHOlKOT4cZD WHLIdwQAcyGYzDQLVflCsDeEBsfwDd+KRFu9MQ3mBkaMZIL5q3yVWzJKa1P33olGmbfN sllNAmYEmMGXfF3a92lWhr3rPPphkV2QBHK5wsqDB8mg4jQIQ73CUAyb6hbKq9BKoeST Jzu1taCVnK18xbf6Oxh3nNQV+dljDfxLmr28XoFr2E7GOcrVSKfVE2Cj3nvU35KmN6EF RUvaSVgx6A0fMCxqh4YuCtNAmVNfTSg76WbsL3CG9ChMuFlcQ0vxHwgTIa5jrXb0KZJX pT2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727314518; x=1727919318; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=CIRgMt/tPt8dDHjlNVeDudvk1PSxwGvmBpyKYz5f4us=; b=I7Fx/9rrBIftVSTBLef2VH9EwMWyicJZFSZ+KYHGyp6SiKuWhMwDDGHmmSr94DCSBr izsvzMwQ/3Y2CZMp9syqBm7qHyKsWq3/YgwP2gwqj0sn4npLT4kEbQ+6/cRq9S0bvoxr IUBRzKSqqMked8vikwuo+7xW++A3XhnWFeFw4fp2iMqPQnQz7pCbU6L/6aNorYvPbPpf EczbWsR27uUC0EoGqROqg2/COqhRdiJr977MEiASY2d3Kf7duYuxFXGQxUiVeZbUqBON 01MjgZS1ODTgygljqFfvtCltpH0APPsK5vFhFwfh4IpVOz4tN6Zvf84re4W4fpJSO1Uc VPZQ== X-Forwarded-Encrypted: i=1; AJvYcCVk6GjS/M5Pcuv4cds/v8Zu3iPxB2d0ldFTXKZFmUhHcvqeH1thFOWkGIQGLt0RRITi0sJDJ1HTyA==@kvack.org X-Gm-Message-State: AOJu0YwWZpoAV7VYBJyDlFzWHg9hM4o3GCZTeTSMjOQp/XvSPZVFDIvz z3p04NX9Jc9c6s/79W6yqCu8jHMs5VOsAQMtch5jz0GlC6c307Ni57N90L3rF2ZWqfbtq1T7jOz qE51mEAiAkJBD4G7liA== X-Google-Smtp-Source: AGHT+IHo9npkiN5cwqAiCiI3e58k0AahBEai2pKIEYXOCcGxS2Z6C3puLSeMWFdZt66z08Hill7horZ6pvVyPTNq X-Received: from jthoughton.c.googlers.com ([fda3:e722:ac3:cc00:13d:fb22:ac12:a84b]) (user=jthoughton job=sendgmr) by 2002:a05:690c:4a08:b0:6e2:371f:4afe with SMTP id 00721157ae682-6e2371f4d7dmr1417b3.4.1727314518344; Wed, 25 Sep 2024 18:35:18 -0700 (PDT) Date: Thu, 26 Sep 2024 01:34:51 +0000 In-Reply-To: <20240926013506.860253-1-jthoughton@google.com> Mime-Version: 1.0 References: <20240926013506.860253-1-jthoughton@google.com> X-Mailer: git-send-email 2.46.0.792.g87dc391469-goog Message-ID: <20240926013506.860253-4-jthoughton@google.com> Subject: [PATCH v7 03/18] KVM: x86/mmu: Factor out spte atomic bit clearing routine From: James Houghton To: Sean Christopherson , Paolo Bonzini Cc: Andrew Morton , David Matlack , David Rientjes , James Houghton , Jason Gunthorpe , Jonathan Corbet , Marc Zyngier , Oliver Upton , Wei Xu , Yu Zhao , Axel Rasmussen , kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org X-Rspam-User: X-Stat-Signature: dw66adrjkt9we85i8ozka3rjbpt3zzez X-Rspamd-Queue-Id: 55F8E20003 X-Rspamd-Server: rspam02 X-HE-Tag: 1727314519-213099 X-HE-Meta: U2FsdGVkX1+z7LGH4zoaE2PgAcjLDvG12+0JcoaxLgjQyR6kG4WeU2PEVHmktNHbWc3ts7DgVSUqi7wP7tsNceSH9DV1b1x7/cSKddJHzWR2KDWDlmvXywb4l+HTxvrsbY28XBvo8ggPD5Yos1997UyxcrQ5r44AmL5lVzYmZagYK4Ff2MkKGSEsoo65prNi7QOwURLgqqycbAn5mH2TWQluMzzJG2PAaoFo3eCicplwZB/3dySQ7A6LUYWJGvZjS1pw/mT5KUDdmtWBq71X3hok0gDuBKCqA59WT/MU9hiqpcV3gZHPgs6L142hXUUOzJ2yrRoyPDkQSUiWxXJzLirOgbeOkJioWDnWJ0Jl+dGYvF9WpmEhYbRbxyqSEjjztQZmWgUxad4o5d4u8GfHzublgJMzH91CrHyauQTS1vZ1AV5UxE3S7HF8I/D0dpH0k1QIcmnQvIHV8f0ac+ghFoDEVVauDtQPJEJ5C2Koi24MvaWApYgNLCKJ1mhMlCtjo2yvRGsnMsTFTIJUsQnB/9WmqrbMWSdjUkvoG7aZn4mG9D3PSiBkjuhUKbyzR1XeRUGZJlOfiOrGPnUmbWM2scY91M3WQXR+fo0h+gnHb5648d7W93FWJtZdDI5rKm7dPrbwqezwhvoQGuTGHrpR6n5Wf0TjMuFrpYImf6gcLj6mAZRxkTWbwW1jRLKzYyv7gv4yOT+/xHusnu1CIWJ8WJo9OcY41smvXMxmfhQ33J2rMinei5TpCPSkftnFU2dFKLc2/08wYX7bthgjWb6QiRngF1+fofPBepzuwK3t7Q7q3ZiZmsoULtTfN+rBDkoea31LfkhzV57RwqDW00ceRnnkvdwv+rvYTL7XW2wju6zgmxonvZlBgrHFyAW1+OneRouzaFkmX83fWGLXoQIxQHi0bM8d1rWkLmXuOn74qpBvxaepH5SPaDUhGi5aHL32MNy4an1qnQsVCU5l3Fa l0llHbL4 k5mgdBv80RgOAYhN5d9H6UvPn1sfAmvemmS9XLFJPM1qmV1d6OIwXH9n7ziNXeWwIprrUyfCTg5JnhXNNEiUSj4jeamweuyFN56TQdtAjGGC8S4wKpwPwtHZRGHOK3Y9EuLPoMTWv/gzu+ZE0OMp6BJWZFZ9IPM1M3+6dwfcF7QOOYAnDDrXOGU/VlGERFoF+vdpjeSysYk5T4qM3OzVuLU2aq2I99UK9UfLASZA66O1ynOcqy1mO+l1COX1xnO/dEBJ4boeq+rpJJeu93hhFds2eLHnIa8EddE2JTK2LxP3g+2xUTyHa5hdvzrI6bOvPSN+1yaPwTNCpF5BnLrYESBxyTFPoSE5gWKawc1wzJ3zvooFO4KwLRpZ3ssUh2xFkaJuTOk7Yq19uo1I8v4DmjJvxsOYCCCeYy683VyahhejWg8O38kEpqbw6SIS57pSmDZM57vUSyNcTeIja6EL0sJlV67O5+BqLhQTm7qTo3bQbcYBaf0HBdeKY3NyUmF39sNYeKa1ngsurQm+4Q5CyASa+Ne6UV89PgSSuj09iIU7p7GnBpZzTOC1A81/qWFM/iHs7cHALT+Apk24= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000211, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This new function, tdp_mmu_clear_spte_bits_atomic(), will be used in a follow-up patch to enable lockless Accessed and R/W/X bit clearing. Signed-off-by: James Houghton --- arch/x86/kvm/mmu/tdp_iter.h | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_iter.h b/arch/x86/kvm/mmu/tdp_iter.h index 2880fd392e0c..ec171568487c 100644 --- a/arch/x86/kvm/mmu/tdp_iter.h +++ b/arch/x86/kvm/mmu/tdp_iter.h @@ -25,6 +25,13 @@ static inline u64 kvm_tdp_mmu_write_spte_atomic(tdp_ptep_t sptep, u64 new_spte) return xchg(rcu_dereference(sptep), new_spte); } +static inline u64 tdp_mmu_clear_spte_bits_atomic(tdp_ptep_t sptep, u64 mask) +{ + atomic64_t *sptep_atomic = (atomic64_t *)rcu_dereference(sptep); + + return (u64)atomic64_fetch_and(~mask, sptep_atomic); +} + static inline void __kvm_tdp_mmu_write_spte(tdp_ptep_t sptep, u64 new_spte) { KVM_MMU_WARN_ON(is_ept_ve_possible(new_spte)); @@ -65,10 +72,8 @@ static inline u64 tdp_mmu_clear_spte_bits(tdp_ptep_t sptep, u64 old_spte, { atomic64_t *sptep_atomic; - if (kvm_tdp_mmu_spte_need_atomic_write(old_spte, level)) { - sptep_atomic = (atomic64_t *)rcu_dereference(sptep); - return (u64)atomic64_fetch_and(~mask, sptep_atomic); - } + if (kvm_tdp_mmu_spte_need_atomic_write(old_spte, level)) + return tdp_mmu_clear_spte_bits_atomic(sptep, mask); __kvm_tdp_mmu_write_spte(sptep, old_spte & ~mask); return old_spte;