From patchwork Tue Aug 27 13:57:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13779594 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 43B6FC54734 for ; Tue, 27 Aug 2024 13:58:04 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.784069.1193440 (Exim 4.92) (envelope-from ) id 1siwho-0004A0-3D; Tue, 27 Aug 2024 13:57:56 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 784069.1193440; Tue, 27 Aug 2024 13:57:56 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1siwho-00049Y-07; Tue, 27 Aug 2024 13:57:56 +0000 Received: by outflank-mailman (input) for mailman id 784069; Tue, 27 Aug 2024 13:57:55 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1siwhn-00047p-G2 for xen-devel@lists.xenproject.org; Tue, 27 Aug 2024 13:57:55 +0000 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [2a00:1450:4864:20::62e]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 5b7e8350-647c-11ef-a0b0-8be0dac302b0; Tue, 27 Aug 2024 15:57:54 +0200 (CEST) Received: by mail-ej1-x62e.google.com with SMTP id a640c23a62f3a-a86859e2fc0so687341566b.3 for ; Tue, 27 Aug 2024 06:57:54 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a86e588adf2sm113898966b.173.2024.08.27.06.57.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Aug 2024 06:57:52 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 5b7e8350-647c-11ef-a0b0-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1724767073; x=1725371873; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lWli70jhXRT+1zJ4cUaW5oqtYkf8vd8SYDQrICxm6yE=; b=GL+XsUdCoKBKeDoVN5cq+4tEMei5ooCUDi9pbzHFIow3n67/+r3Ae6ojAL/u5nozU0 heAwoM91N2n+x1a4aq/AxUqd2ld+2lY9lT8w9EH90uWI4yZHGt80KUOFvHivyY0Z0D98 kOXzdoM/2Rr7LQBR41mEQcuVJCtOynCSik5FI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724767073; x=1725371873; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lWli70jhXRT+1zJ4cUaW5oqtYkf8vd8SYDQrICxm6yE=; b=txEYLNAbcM4RnLk+Kp52JVFhckpTk7I9hKYTHZV7Xi7KxRdMMb+8WwgnrIcdfNzm9r AcrYDX/bsjI6izUi1DJXrTeb1v+jL96Ooes8kgqI4F2Yg4eV73HPHtfy+1bqwlAlKt4w gy/j6QNXEQnFDo1Jttr7HP1kLOtVicKIswijt/gHHqbu9sISj1AIbJ+2sVAPb3aLRLuW g2x0DrMq0Fd97N3Uw3GK3zxNh0zzYpVnps3kPTgGkvQgGPRDS44y5oZo0EuJnwCeCk7L dHPq/LqypqfveHQbxYx0pYdRyXqrVsyaghnEISG8N+fkZu0GZRGUW91nmqijZgkZaY1Q oIWg== X-Gm-Message-State: AOJu0Ywj/7r2FE0nqiByu46D2GUu+Io0Hrf0y2Rr9ICKewsN9/4QkEk3 Gy/rTx7PAVnoGai9W46FbbvAOBdmTqLcyk0a0yU9BAEklI2ZFTtb62Atd9Sd+jbKfLSDZZJFm4T X X-Google-Smtp-Source: AGHT+IFfGcZL/pIGlbHYGOiP8yeTKhRtLOJUjxssu5EVYTfskAuuK/Tmx6RF7IRtiN49AhGs0Q7hKw== X-Received: by 2002:a17:907:7d94:b0:a86:b504:42ea with SMTP id a640c23a62f3a-a86e3bdbd2bmr197554766b.45.1724767073100; Tue, 27 Aug 2024 06:57:53 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , Stefano Stabellini , Julien Grall Subject: [PATCH 1/4] xen/evtchn: Use bitmap_for_each() in evtchn_check_pollers() Date: Tue, 27 Aug 2024 14:57:43 +0100 Message-Id: <20240827135746.1908070-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240827135746.1908070-1-andrew.cooper3@citrix.com> References: <20240827135746.1908070-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 It is a preprocessor-identical opencoding. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Stefano Stabellini CC: Julien Grall --- xen/common/event_channel.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c index 13b97c94d7fa..8db2ca4ba23e 100644 --- a/xen/common/event_channel.c +++ b/xen/common/event_channel.c @@ -1517,9 +1517,7 @@ void evtchn_check_pollers(struct domain *d, unsigned int port) return; /* Wake any interested (or potentially interested) pollers. */ - for ( vcpuid = find_first_bit(d->poll_mask, d->max_vcpus); - vcpuid < d->max_vcpus; - vcpuid = find_next_bit(d->poll_mask, d->max_vcpus, vcpuid+1) ) + bitmap_for_each ( vcpuid, d->poll_mask, d->max_vcpus ) { v = d->vcpu[vcpuid]; if ( ((v->poll_evtchn <= 0) || (v->poll_evtchn == port)) && From patchwork Tue Aug 27 13:57:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13779595 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7EF81C54739 for ; Tue, 27 Aug 2024 13:58:04 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.784070.1193456 (Exim 4.92) (envelope-from ) id 1siwhp-0004bs-F0; Tue, 27 Aug 2024 13:57:57 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 784070.1193456; Tue, 27 Aug 2024 13:57:57 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1siwhp-0004bl-BL; Tue, 27 Aug 2024 13:57:57 +0000 Received: by outflank-mailman (input) for mailman id 784070; Tue, 27 Aug 2024 13:57:56 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1siwho-00047p-5u for xen-devel@lists.xenproject.org; Tue, 27 Aug 2024 13:57:56 +0000 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [2a00:1450:4864:20::52e]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 5c13d309-647c-11ef-a0b0-8be0dac302b0; Tue, 27 Aug 2024 15:57:55 +0200 (CEST) Received: by mail-ed1-x52e.google.com with SMTP id 4fb4d7f45d1cf-5bed83487aeso5778025a12.2 for ; Tue, 27 Aug 2024 06:57:55 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a86e588adf2sm113898966b.173.2024.08.27.06.57.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Aug 2024 06:57:53 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 5c13d309-647c-11ef-a0b0-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1724767074; x=1725371874; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GG6I/ExZxOSYobQF/vdPu38zR0FWgZTstjCiLrrIFG0=; b=MCmjjpNMXpD0a2g1Hc4BAkYGMNPAKitkxzoNKlaS4XtaHY8gjS6Xf/taKgENk2VN9Y V0hUknZLMt9TcBXfmP2eGSPGw1Mrrb/7+2/MCI8SUSESn5HrWpN8lQ6ALeydWIhiQyfC 9d7QEyHofboSqHVDP51OhyAco38xAg0ux+trQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724767074; x=1725371874; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GG6I/ExZxOSYobQF/vdPu38zR0FWgZTstjCiLrrIFG0=; b=fkuhc/SZOgd9C/o8XIvO1sUS7pyexqkTXVzVgs2wcs56bb5+cHt/TAcyXWsxmMdCjT lWy0lky6DI3KL8DhVRRMPABLnIalFY8nn4SwRorO2EN6RJhL6vl07oLeR7ac/lN9Yztq LCVSNeXXlxI0JqAzMTtGPjEJIleIoVUaceO25UrXQ2aotzQQ+JkqxUc45ynO8zt6FMMi aMTv465ptmrWsKwhTWXVTRqr1UKtMj2EjDOUFVwiAPI8FK58BPj1MFx+r3ENPnTByHth 9tRwx/hdMeK0KLgpsYxsHKTPSCtV2kcPYqqqFpK0VKe6M0SZ7auQopueL4GqXIS7jSGJ vjcw== X-Gm-Message-State: AOJu0YyyAURZ/sZkjTmRVAE2TpRGMOWUgLgV7efgwFD6LUxxygHWVdD6 3nSNbpkG2DgLh2hNbgqEDIkLqmNE9AU+pDYud3w53KQRuHxGeg/8bQiLyNVsahN3RYWUBAYoD7B / X-Google-Smtp-Source: AGHT+IGPFcMG0+WuSEvzaLfVo4yvDOnp6oCLBMJgVrqDnI5Yvr5lFEvM6AhodWIgFxEHdcFp/6Tj+A== X-Received: by 2002:a17:907:7d94:b0:a86:b46b:860a with SMTP id a640c23a62f3a-a86e3be5cf1mr197371566b.54.1724767074158; Tue, 27 Aug 2024 06:57:54 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH 2/4] x86/hvm: Use for_each_set_bit() in hvm_emulate_writeback() Date: Tue, 27 Aug 2024 14:57:44 +0100 Message-Id: <20240827135746.1908070-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240827135746.1908070-1-andrew.cooper3@citrix.com> References: <20240827135746.1908070-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 ... which is more consise than the opencoded form. Also, for production VMs, ~100% of emulations are simple MOVs, so it is likely that there are no segments to write back. Furthermore, now that find_{first,next}_bit() are no longer in use, the seg_reg_{accessed,dirty} fields aren't forced to be unsigned long, although they do need to remain unsigned int because of __set_bit() elsewhere. No practical change. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné Pulling current out into curr is good for code generation. When using current in the loop, GCC can't retain the calculation across the call to hvm_set_segment_register() and is forced to re-read from the cpu_info block. However, if curr is initialised, it's calculated even in the likely path... --- xen/arch/x86/hvm/emulate.c | 20 ++++++++++---------- xen/arch/x86/include/asm/hvm/emulate.h | 4 ++-- 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c index feb4792cc567..732bdbab25b0 100644 --- a/xen/arch/x86/hvm/emulate.c +++ b/xen/arch/x86/hvm/emulate.c @@ -2908,18 +2908,18 @@ void hvm_emulate_init_per_insn( void hvm_emulate_writeback( struct hvm_emulate_ctxt *hvmemul_ctxt) { - enum x86_segment seg; + struct vcpu *curr; + unsigned int dirty = hvmemul_ctxt->seg_reg_dirty; - seg = find_first_bit(&hvmemul_ctxt->seg_reg_dirty, - ARRAY_SIZE(hvmemul_ctxt->seg_reg)); + if ( likely(!dirty) ) + return; - while ( seg < ARRAY_SIZE(hvmemul_ctxt->seg_reg) ) - { - hvm_set_segment_register(current, seg, &hvmemul_ctxt->seg_reg[seg]); - seg = find_next_bit(&hvmemul_ctxt->seg_reg_dirty, - ARRAY_SIZE(hvmemul_ctxt->seg_reg), - seg+1); - } + curr = current; + + for_each_set_bit ( seg, dirty ) + hvm_set_segment_register(curr, seg, &hvmemul_ctxt->seg_reg[seg]); + + hvmemul_ctxt->seg_reg_dirty = 0; } /* diff --git a/xen/arch/x86/include/asm/hvm/emulate.h b/xen/arch/x86/include/asm/hvm/emulate.h index 29d679442e10..972cdf1fa0cf 100644 --- a/xen/arch/x86/include/asm/hvm/emulate.h +++ b/xen/arch/x86/include/asm/hvm/emulate.h @@ -36,8 +36,8 @@ struct hvm_emulate_ctxt { unsigned int insn_buf_bytes; struct segment_register seg_reg[10]; - unsigned long seg_reg_accessed; - unsigned long seg_reg_dirty; + unsigned int seg_reg_accessed; + unsigned int seg_reg_dirty; /* * MFNs behind temporary mappings in the write callback. The length is From patchwork Tue Aug 27 13:57:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13779597 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 586F7C54734 for ; Tue, 27 Aug 2024 13:58:07 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.784072.1193470 (Exim 4.92) (envelope-from ) id 1siwhs-0004xV-1m; Tue, 27 Aug 2024 13:58:00 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 784072.1193470; Tue, 27 Aug 2024 13:57:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1siwhr-0004wO-RO; Tue, 27 Aug 2024 13:57:59 +0000 Received: by outflank-mailman (input) for mailman id 784072; Tue, 27 Aug 2024 13:57:58 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1siwhq-0003sI-6S for xen-devel@lists.xenproject.org; Tue, 27 Aug 2024 13:57:58 +0000 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [2a00:1450:4864:20::62e]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 5ca9fc5d-647c-11ef-99a0-01e77a169b0f; Tue, 27 Aug 2024 15:57:56 +0200 (CEST) Received: by mail-ej1-x62e.google.com with SMTP id a640c23a62f3a-a7aa086b077so493904266b.0 for ; Tue, 27 Aug 2024 06:57:56 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a86e588adf2sm113898966b.173.2024.08.27.06.57.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Aug 2024 06:57:54 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 5ca9fc5d-647c-11ef-99a0-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1724767076; x=1725371876; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nGUfx2q1+jWgi8jP5MwYlvDEDt5Tq995qxqKh3B4e+k=; b=DHLiiCoBRMXFoVCaBxdOlLd8jCuJ7jxvy1jvJg81HN0sZuf1lwh7cRYKOm2Y2IfXAO dUtgyBfBI+GPZkmb6+NpJNk+pOa256bpX+2tAVipjjk2fFL+UCyRVh9tX2WS235UYPDm vUVD3Zbn4gT4YUA4EHC/yGqik3u4RFZPc1bSs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724767076; x=1725371876; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nGUfx2q1+jWgi8jP5MwYlvDEDt5Tq995qxqKh3B4e+k=; b=ZBsXRgC7NTkQqkwIToFL/xtBLTfT6/33hwP7+7HzyxV0o5xHAWacl+kLStVHkizGDq M9JIiHy3lmLiBksNwYgsmnqDQy+jRWgT9NRHn84Qk9YYZZI7YJamEVmCMVWbN11+M/+o dwb0fRun+u0igKtay51RPCI6IdQLWo2QqxfkDTMovl2uY3EEExpWqTJ+HfaRD8AjHQbn kanAhLeKVBPwTy3lJmsrilb30ldbk+8zYHQVhpC6EedLtUGbdWvj0yIQyjkvjNE7Pic6 PEmuUuDBXDw3DF1vmxeHCVvrGfzF4YQOiPD4I4v6rBAoZwRA7qAl/SJ0DQcSCtU8QtOz rGPw== X-Gm-Message-State: AOJu0Yz6HxO13k8do/s9Jn1kOEIV1dHXr7LYeMYlCtcgPveJLglF5aRf tQXhYLO/QN05ayschIp6z8qQGEu7N2BAkkxHtY+OJvoT9vrVkPN68+niO9GN5kO/sydnm7qQQwn a X-Google-Smtp-Source: AGHT+IF+lE+VmLKB8gnLP/DI0Jsj0j8worRIiuKlHU8MrF0gDq9Q5hJgWjAvzf9gxSKUf9uxlYP8Dw== X-Received: by 2002:a17:907:7e9a:b0:a86:a4b1:d2c0 with SMTP id a640c23a62f3a-a86a54b8ce4mr1167977166b.53.1724767075121; Tue, 27 Aug 2024 06:57:55 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH 3/4] x86/hvm: Rework hpet_write() for improved code generation Date: Tue, 27 Aug 2024 14:57:45 +0100 Message-Id: <20240827135746.1908070-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240827135746.1908070-1-andrew.cooper3@citrix.com> References: <20240827135746.1908070-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 In the HPET_STATUS handling, the use of __clear_bit(i, &new_val) is the only thing causing it to be spilled to the stack. Furthemore we only care about the bottom 3 bits, so rewrite it to be a plain for loop. For the {start,stop}_timer variables, these are spilled to the stack despite the __{set,clear}_bit() calls. Again we only care about the bottom 3 bits, so shrink the variables from long to int. Use for_each_set_bit() rather than opencoding it at the end which amongst other things means the loop predicate is no longer forced to the stack by the loop body. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné All in all, it's modest according to bloat-o-meter: add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-29 (-29) Function old new delta hpet_write 2225 2196 -29 but we have shrunk the stack frame by 8 bytes; 0x28 as opposed to 0x30 before. --- xen/arch/x86/hvm/hpet.c | 29 ++++++++--------------------- 1 file changed, 8 insertions(+), 21 deletions(-) diff --git a/xen/arch/x86/hvm/hpet.c b/xen/arch/x86/hvm/hpet.c index 87642575f9cd..e3981d5e467c 100644 --- a/xen/arch/x86/hvm/hpet.c +++ b/xen/arch/x86/hvm/hpet.c @@ -349,8 +349,7 @@ static int cf_check hpet_write( unsigned int tn, i; /* Acculumate a bit mask of timers whos state is changed by this write. */ - unsigned long start_timers = 0; - unsigned long stop_timers = 0; + unsigned int start_timers = 0, stop_timers = 0; #define set_stop_timer(n) (__set_bit((n), &stop_timers)) #define set_start_timer(n) (__set_bit((n), &start_timers)) #define set_restart_timer(n) (set_stop_timer(n),set_start_timer(n)) @@ -405,16 +404,12 @@ static int cf_check hpet_write( case HPET_STATUS: /* write 1 to clear. */ - while ( new_val ) + for ( i = 0; i < HPET_TIMER_NUM; i++ ) { - bool active; - - i = ffsl(new_val) - 1; - if ( i >= HPET_TIMER_NUM ) - break; - __clear_bit(i, &new_val); - active = __test_and_clear_bit(i, &h->hpet.isr); - if ( active ) + if ( !(new_val & (1U << i)) ) + continue; + + if ( __test_and_clear_bit(i, &h->hpet.isr) ) { hvm_ioapic_deassert(v->domain, timer_int_route(h, i)); if ( hpet_enabled(h) && timer_enabled(h, i) && @@ -533,19 +528,11 @@ static int cf_check hpet_write( } /* stop/start timers whos state was changed by this write. */ - while (stop_timers) - { - i = ffsl(stop_timers) - 1; - __clear_bit(i, &stop_timers); + for_each_set_bit ( i, stop_timers ) hpet_stop_timer(h, i, guest_time); - } - while (start_timers) - { - i = ffsl(start_timers) - 1; - __clear_bit(i, &start_timers); + for_each_set_bit ( i, start_timers ) hpet_set_timer(h, i, guest_time); - } #undef set_stop_timer #undef set_start_timer From patchwork Tue Aug 27 13:57:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13779596 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 25C93C5472F for ; Tue, 27 Aug 2024 13:58:07 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.784071.1193466 (Exim 4.92) (envelope-from ) id 1siwhr-0004tc-ME; Tue, 27 Aug 2024 13:57:59 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 784071.1193466; Tue, 27 Aug 2024 13:57:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1siwhr-0004tL-IY; Tue, 27 Aug 2024 13:57:59 +0000 Received: by outflank-mailman (input) for mailman id 784071; Tue, 27 Aug 2024 13:57:58 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1siwhq-00047p-4h for xen-devel@lists.xenproject.org; Tue, 27 Aug 2024 13:57:58 +0000 Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [2a00:1450:4864:20::62c]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 5d3fa71f-647c-11ef-a0b0-8be0dac302b0; Tue, 27 Aug 2024 15:57:57 +0200 (CEST) Received: by mail-ej1-x62c.google.com with SMTP id a640c23a62f3a-a868d7f92feso700694466b.2 for ; Tue, 27 Aug 2024 06:57:57 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a86e588adf2sm113898966b.173.2024.08.27.06.57.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Aug 2024 06:57:55 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 5d3fa71f-647c-11ef-a0b0-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1724767076; x=1725371876; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=onbkRJjonSldeX+TKHvWoalIHglLaxnF1KDpl/v4wiA=; b=uT8enpQLoIrW8DSrhlqvctPlFAqWtGJTBIKzU71f/nDyntq2WNJZ7P/CP9qehyD9aV wbljrgpk+YhLg38EVpJrbG0n29+BYGMb4o8YEeNEt+ypHA80jhLu+j1gHNPwantqntOO BBrocgVovvIqUgzQ1m0f+QO02ztjRdxUH/HsM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724767076; x=1725371876; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=onbkRJjonSldeX+TKHvWoalIHglLaxnF1KDpl/v4wiA=; b=TvVdn/g1X/JlsZNnfuuwGzYJh7bBL9ImJsYQxaqAU5XwI9+gJLMl9uNe7d8GqIeTu2 XYh1C0ru8iP+crKR1wshdIYmmGAq2qbbCj2EZwtrBdp/hve94NG7Btpnvq2bCJtIebLs PdsfDl7jqeMzjZF30khSzJqPHY4gAFWtTlVHnqrOiqRK/F3tNIRx4VEAPp2IhFOLZquJ 1T2bDzrukgsVcJ7JuHEPVoOf1DhK2EIZlbqCJSbErBxPBkK+I7Za1sNlYbIpJmHRbbjP 3O2yQhAQuqTwkoIlBCV0sycdgxuYiVT94UOLNdFN3JfkqrD+yh+PvIpLWrf/PL7v81C6 G18Q== X-Gm-Message-State: AOJu0YzFst2ANLV8wnjMGsnuEa+BL3L6IKsw69Erk4w7j6aL1IUeRGXQ LwywPqkW2vr6m7lQG6pooCvbznI7K0PVDhsMBtMTxDsb9+PKFGeQTIIwfi8h+No3EUN3PtWEz6/ K X-Google-Smtp-Source: AGHT+IH7hZ1uEw4kAXnd6m6eSnLzFwncr1ABIEpVvj5g0O6NferXHSInSooeNemWXtI/6AQO/5hpRg== X-Received: by 2002:a17:907:6d0f:b0:a86:7e7f:69ab with SMTP id a640c23a62f3a-a86a52b367emr931155566b.15.1724767076098; Tue, 27 Aug 2024 06:57:56 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Bertrand Marquis , Michal Orzel , Oleksii Kurochko Subject: [PATCH v2 4/4] x86/vmx: Rewrite vmx_sync_pir_to_irr() to be more efficient Date: Tue, 27 Aug 2024 14:57:46 +0100 Message-Id: <20240827135746.1908070-5-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240827135746.1908070-1-andrew.cooper3@citrix.com> References: <20240827135746.1908070-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 There are two issues. First, pi_test_and_clear_on() pulls the cache-line to the CPU and dirties it even if there's nothing outstanding, but the final for_each_set_bit() is O(256) when O(8) would do, and would avoid multiple atomic updates to the same IRR word. Rewrite it from scratch, explaining what's going on at each step. Bloat-o-meter reports 177 -> 145 (net -32), but the better aspect is the removal calls to __find_{first,next}_bit() hidden behind for_each_set_bit(). No functional change. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné CC: Stefano Stabellini CC: Julien Grall CC: Volodymyr Babchuk CC: Bertrand Marquis CC: Michal Orzel CC: Oleksii Kurochko The main purpose of this is to get rid of bitmap_for_each(). v2: * Extend the comments --- xen/arch/x86/hvm/vmx/vmx.c | 70 +++++++++++++++++++++++++++++++++----- 1 file changed, 62 insertions(+), 8 deletions(-) diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index 12f8a66458db..1baed7e816c4 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -2317,18 +2317,72 @@ static void cf_check vmx_deliver_posted_intr(struct vcpu *v, u8 vector) static void cf_check vmx_sync_pir_to_irr(struct vcpu *v) { - struct vlapic *vlapic = vcpu_vlapic(v); - unsigned int group, i; - DECLARE_BITMAP(pending_intr, X86_NR_VECTORS); + struct pi_desc *desc = &v->arch.hvm.vmx.pi_desc; + union { + uint64_t _64[X86_NR_VECTORS / (sizeof(uint64_t) * 8)]; + uint32_t _32[X86_NR_VECTORS / (sizeof(uint32_t) * 8)]; + } vec; + uint32_t *irr; + bool on; - if ( !pi_test_and_clear_on(&v->arch.hvm.vmx.pi_desc) ) + /* + * The PIR is a contended cacheline which bounces between the CPU(s) and + * IOMMU(s). An IOMMU updates the entire PIR atomically, but we can't + * express the same on the CPU side, so care has to be taken. + * + * First, do a plain read of ON. If the PIR hasn't been modified, this + * will keep the cacheline Shared and not pull it Excusive on the current + * CPU. + */ + if ( !pi_test_on(desc) ) return; - for ( group = 0; group < ARRAY_SIZE(pending_intr); group++ ) - pending_intr[group] = pi_get_pir(&v->arch.hvm.vmx.pi_desc, group); + /* + * Second, if the plain read said that ON was set, we must clear it with + * an atomic action. This will bring the cachline to Exclusive on the + * current CPU. + * + * This should always succeed because noone else should be playing with + * the PIR behind our back, but assert so just in case. + */ + on = pi_test_and_clear_on(desc); + ASSERT(on); + + /* + * The cacheline is now Exclusive on the current CPU, and because ON was + * set, some other entity (an IOMMU, or Xen on another CPU) has indicated + * that at PIR needs re-scanning. + * + * Note: Entities which can't update the entire cacheline atomically + * (i.e. Xen on another CPU) are required to update PIR first, then + * set ON. Therefore, there is a corner case where we may have + * found and processed the PIR updates "last time around" and only + * found ON this time around. This is fine; the logic still + * operates correctly. + * + * Atomically read and clear the entire pending bitmap as fast as we, to + * reduce the window where another entity may steal the cacheline back + * from us. This is a performance concern, not a correctness concern. If + * the another entity does steal the cacheline back, we'll just wait to + * get it back again. + */ + for ( unsigned int i = 0; i < ARRAY_SIZE(vec._64); ++i ) + vec._64[i] = xchg(&desc->pir[i], 0); + + /* + * Finally, merge the pending vectors into IRR. The IRR register is + * scattered in memory, so we have to do this 32 bits at a time. + */ + irr = (uint32_t *)&vcpu_vlapic(v)->regs->data[APIC_IRR]; + for ( unsigned int i = 0; i < ARRAY_SIZE(vec._32); ++i ) + { + if ( !vec._32[i] ) + continue; - bitmap_for_each ( i, pending_intr, X86_NR_VECTORS ) - vlapic_set_vector(i, &vlapic->regs->data[APIC_IRR]); + asm ( "lock or %[val], %[irr]" + : [irr] "+m" (irr[i * 0x10]) + : [val] "r" (vec._32[i]) ); + } } static bool cf_check vmx_test_pir(const struct vcpu *v, uint8_t vec)