From patchwork Mon May 20 08:20:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gautam Menghani X-Patchwork-Id: 13668117 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F06221CD32; Mon, 20 May 2024 08:20:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716193257; cv=none; b=dVGSu1yhPfQL0i4xpoy9PRE3RYTROSlsIu4t9DY50GeA/2pC3E1nbIrp7ywmeSWBEqr61nlSF6J+uBk/6HsxZzZWe3WmvyR7u7wHpeDl5Dm9wAmDKNpuE93j1zo63qr311OKCDwTRr704OgpST3sKBTyafpdf69bQh73mgdaY+I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716193257; c=relaxed/simple; bh=Yudd+LbphmqwuVEHJwSLVDg9XFbhqEuPzRJAqIoAD5Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jtTqsQOSNQHP90MrkHCSID6hCA1mK6ortmpMSJlW+lBni42ojQ84fQ0B+r+4Lq+HLUUVcijDMx3P2o5TZ6KQM7U7ExTtyRy0wGjD0Jkmg7F8KSKCfjY65c8n1EJt2/MzeGGVNzvIh0BcpUYyHCqqWuNWRT734XhchwsUKB3yVzQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=kLjmWEEp; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="kLjmWEEp" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44K6U4eE032441; Mon, 20 May 2024 08:20:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=1ZljFRPQkJliM89Aj1uqay/WLHKarY1XQa8XB660YSs=; b=kLjmWEEpRo2pFdXEWqRTKMBqpWQ7iyW+HvVoMFRz0SYPA/5TcEEn/MTeCtCWqZtEkVC7 Wc9tb9zKc7N07xQuGUXxJqE1m9QOqSmV79jT3+9qWnCvl4MbHt6udAaNyYSiA00MLYi5 8tniWoYWpRwmJcHCA5Viet2aD/cUvMKbvk8U4ud2bKOuiFQZUbnxWXdbcZmtUy2zqjk4 ngO6dXDqigWwJWvxy/PsxoLQGw+bhnRCj583tIBmXTWSXXuOOlEiYV9w3b5mOkbnp0fp pp6VnIakf5Qa2pJX3XwRKq1cWuif+16hujY7456dy/uVMjruEfs7l3ExDU7P1ShW7kep Ag== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3y7wnxgnqr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 20 May 2024 08:20:37 +0000 Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44K8KbpV008202; Mon, 20 May 2024 08:20:37 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3y7wnxgnqp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 20 May 2024 08:20:37 +0000 Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 44K5178e000878; Mon, 20 May 2024 08:20:35 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 3y771yxw0h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 20 May 2024 08:20:35 +0000 Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 44K8KTp941943530 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 20 May 2024 08:20:31 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CF65C20040; Mon, 20 May 2024 08:20:29 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0D4212004D; Mon, 20 May 2024 08:20:28 +0000 (GMT) Received: from li-c6426e4c-27cf-11b2-a85c-95d65bc0de0e.in.ibm.com (unknown [9.204.206.66]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 20 May 2024 08:20:27 +0000 (GMT) From: Gautam Menghani To: mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, naveen.n.rao@linux.ibm.com, clg@kaod.org Cc: Gautam Menghani , linuxppc-dev@lists.ozlabs.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RESEND PATCH 1/3] arch/powerpc/kvm: Use bitmap to speed up resend of irqs in ICS Date: Mon, 20 May 2024 13:50:08 +0530 Message-ID: <20240520082014.140697-2-gautam@linux.ibm.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240520082014.140697-1-gautam@linux.ibm.com> References: <20240520082014.140697-1-gautam@linux.ibm.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: md7XFMpY2sLfiYBBRvq-vqILGD4IWWBZ X-Proofpoint-GUID: mGIm1GLoZKNh7eUYRFOJVIOyPzd7QlHI X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-20_04,2024-05-17_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 mlxlogscore=333 priorityscore=1501 impostorscore=0 adultscore=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 mlxscore=0 spamscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405200068 When an irq is to be resent, all 1024 irqs in an ICS are scanned and the irqs having 'resend' flag set are resent. Optimize this flow using bitmap array to speed up the resends. Signed-off-by: Gautam Menghani --- arch/powerpc/kvm/book3s_xics.c | 22 +++++++++++----------- arch/powerpc/kvm/book3s_xics.h | 1 + 2 files changed, 12 insertions(+), 11 deletions(-) diff --git a/arch/powerpc/kvm/book3s_xics.c b/arch/powerpc/kvm/book3s_xics.c index 589a8f257120..12de526f04c4 100644 --- a/arch/powerpc/kvm/book3s_xics.c +++ b/arch/powerpc/kvm/book3s_xics.c @@ -47,9 +47,6 @@ * TODO * ==== * - * - To speed up resends, keep a bitmap of "resend" set bits in the - * ICS - * * - Speed up server# -> ICP lookup (array ? hash table ?) * * - Make ICS lockless as well, or at least a per-interrupt lock or hashed @@ -125,15 +122,17 @@ static int ics_deliver_irq(struct kvmppc_xics *xics, u32 irq, u32 level) static void ics_check_resend(struct kvmppc_xics *xics, struct kvmppc_ics *ics, struct kvmppc_icp *icp) { - int i; + u32 irq; + struct ics_irq_state *state; - for (i = 0; i < KVMPPC_XICS_IRQ_PER_ICS; i++) { - struct ics_irq_state *state = &ics->irq_state[i]; - if (state->resend) { - XICS_DBG("resend %#x prio %#x\n", state->number, - state->priority); - icp_deliver_irq(xics, icp, state->number, true); - } + for_each_set_bit(irq, ics->resend_map, KVMPPC_XICS_IRQ_PER_ICS) { + state = &ics->irq_state[irq]; + + if (!test_and_clear_bit(irq, ics->resend_map)) + continue; + if (!state) + continue; + icp_deliver_irq(xics, icp, state->number, true); } } @@ -489,6 +488,7 @@ static void icp_deliver_irq(struct kvmppc_xics *xics, struct kvmppc_icp *icp, */ smp_wmb(); set_bit(ics->icsid, icp->resend_map); + set_bit(src, ics->resend_map); /* * If the need_resend flag got cleared in the ICP some time diff --git a/arch/powerpc/kvm/book3s_xics.h b/arch/powerpc/kvm/book3s_xics.h index 08fb0843faf5..8fcb34ea47a4 100644 --- a/arch/powerpc/kvm/book3s_xics.h +++ b/arch/powerpc/kvm/book3s_xics.h @@ -98,6 +98,7 @@ struct kvmppc_ics { arch_spinlock_t lock; u16 icsid; struct ics_irq_state irq_state[KVMPPC_XICS_IRQ_PER_ICS]; + DECLARE_BITMAP(resend_map, KVMPPC_XICS_IRQ_PER_ICS); }; struct kvmppc_xics { From patchwork Mon May 20 08:20:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gautam Menghani X-Patchwork-Id: 13668118 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4C58208A7; Mon, 20 May 2024 08:20:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716193260; cv=none; b=CvujqDfSvCetZst9w9s76A63o//s9l07tXUxPoE3c8dK7/bE4Cep7ug44VeOtkmH26bW8qqR0C3z5gc3fOBThWwwDkKiX3cjrBB39uZ5KId3RIDNmaYNYVwVPgXhWk1s8lj7uzclIrWuHLxD0FdlsG4rfLHSRkGg6TJVX4RO3Hw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716193260; c=relaxed/simple; bh=fxNefskKpD2OANrNxXrnrAAT0Q0jJDzhonaTjHySxxo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SC47B8pr/eP+GcWDO3XkJD9zShzqJJQFThcGHwEsdU0rfWgaIuD4+6QOcxte+ulLoJxbftASBbdDs37vhrN0rSzdmpkXDABNgxrtiSzgTS8lUWDOMn284/xPshO1tb/LlpKc41HlZyD1UNhfutT+NYj/QnUePIIraZQPCKp4dN4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=GwSaaqAx; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="GwSaaqAx" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44K684Ft008783; Mon, 20 May 2024 08:20:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=kMBRKF0m+ZIYkax/WD20be0LrYx6C1oQnvg851lGN1U=; b=GwSaaqAxU7fBS45CbPzdk+Eh1M2bMy1+RCeu+HwXyRFB30p/7e0ZSb4rnYe7S/GJd4Wd Aa1+6Zv1UCvtzOSX/kqhbOvbJ2EyiEhO80MxHz7Uq1EzpTir57ahLKc1ACDlY+Fv9UFM lQ0wdniDzT0qzd4j1hMjM9HQwJUnOZI9rqkBN+mJhyDDE3ZucSede9AFFoN5grad5mgy GJnok1psKZc12jLXb5m5iyJskvX2vbteA01NkfGSk2GLhWW34sZ+Qx+hVDgh12oa1HgQ hxLVikcWK5Yo5jT2T7wGPYNLyOx/aWI+SmJ7nKdhrx9XUVGX1oXir67V2RMd6jrM2tfw 0w== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3y80ypg8w5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 20 May 2024 08:20:43 +0000 Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44K8KT6c014300; Mon, 20 May 2024 08:20:42 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3y80ypg8w3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 20 May 2024 08:20:42 +0000 Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 44K6bAZB026443; Mon, 20 May 2024 08:20:41 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 3y785m6kjv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 20 May 2024 08:20:41 +0000 Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 44K8KaJe40042904 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 20 May 2024 08:20:38 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0203120040; Mon, 20 May 2024 08:20:36 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 350FC2004B; Mon, 20 May 2024 08:20:34 +0000 (GMT) Received: from li-c6426e4c-27cf-11b2-a85c-95d65bc0de0e.in.ibm.com (unknown [9.204.206.66]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 20 May 2024 08:20:34 +0000 (GMT) From: Gautam Menghani To: mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, naveen.n.rao@linux.ibm.com, clg@kaod.org Cc: Gautam Menghani , linuxppc-dev@lists.ozlabs.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RESEND PATCH 2/3] arch/powerpc/kvm: Optimize the server number -> ICP lookup Date: Mon, 20 May 2024 13:50:09 +0530 Message-ID: <20240520082014.140697-3-gautam@linux.ibm.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240520082014.140697-1-gautam@linux.ibm.com> References: <20240520082014.140697-1-gautam@linux.ibm.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: PltlRD2anSDEkJRtIzHJsy7VWpiniW7Z X-Proofpoint-ORIG-GUID: UMKHA2d_CikoIcN3Wo1bJNJlLclWVs95 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-20_04,2024-05-17_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=19 impostorscore=0 mlxlogscore=62 clxscore=1015 bulkscore=0 suspectscore=0 adultscore=0 malwarescore=0 spamscore=19 phishscore=0 lowpriorityscore=0 priorityscore=1501 mlxscore=19 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405200068 Given a server number, kvmppc_xics_find_server() does a linear search over the vcpus of a VM. Optimize this logic by using an array to maintain the mapping between server number -> icp. Signed-off-by: Gautam Menghani --- arch/powerpc/kvm/book3s_xics.c | 4 ++-- arch/powerpc/kvm/book3s_xics.h | 10 ++-------- 2 files changed, 4 insertions(+), 10 deletions(-) diff --git a/arch/powerpc/kvm/book3s_xics.c b/arch/powerpc/kvm/book3s_xics.c index 12de526f04c4..1dc2f77571e7 100644 --- a/arch/powerpc/kvm/book3s_xics.c +++ b/arch/powerpc/kvm/book3s_xics.c @@ -47,8 +47,6 @@ * TODO * ==== * - * - Speed up server# -> ICP lookup (array ? hash table ?) - * * - Make ICS lockless as well, or at least a per-interrupt lock or hashed * locks array to improve scalability */ @@ -1062,6 +1060,7 @@ static struct kvmppc_ics *kvmppc_xics_create_ics(struct kvm *kvm, static int kvmppc_xics_create_icp(struct kvm_vcpu *vcpu, unsigned long server_num) { struct kvmppc_icp *icp; + struct kvm *kvm = vcpu->kvm; if (!vcpu->kvm->arch.xics) return -ENODEV; @@ -1078,6 +1077,7 @@ static int kvmppc_xics_create_icp(struct kvm_vcpu *vcpu, unsigned long server_nu icp->state.mfrr = MASKED; icp->state.pending_pri = MASKED; vcpu->arch.icp = icp; + kvm->arch.xics->icps[server_num] = icp; XICS_DBG("created server for vcpu %d\n", vcpu->vcpu_id); diff --git a/arch/powerpc/kvm/book3s_xics.h b/arch/powerpc/kvm/book3s_xics.h index 8fcb34ea47a4..feeb0897d555 100644 --- a/arch/powerpc/kvm/book3s_xics.h +++ b/arch/powerpc/kvm/book3s_xics.h @@ -111,19 +111,13 @@ struct kvmppc_xics { u32 err_noics; u32 err_noicp; struct kvmppc_ics *ics[KVMPPC_XICS_MAX_ICS_ID + 1]; + DECLARE_FLEX_ARRAY(struct kvmppc_icp *, icps); }; static inline struct kvmppc_icp *kvmppc_xics_find_server(struct kvm *kvm, u32 nr) { - struct kvm_vcpu *vcpu = NULL; - unsigned long i; - - kvm_for_each_vcpu(i, vcpu, kvm) { - if (vcpu->arch.icp && nr == vcpu->arch.icp->server_num) - return vcpu->arch.icp; - } - return NULL; + return kvm->arch.xics->icps[nr]; } static inline struct kvmppc_ics *kvmppc_xics_find_ics(struct kvmppc_xics *xics, From patchwork Mon May 20 08:20:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gautam Menghani X-Patchwork-Id: 13668119 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 509182CCBE; Mon, 20 May 2024 08:21:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716193267; cv=none; b=S2UMg9syyEQB+xy/ppbO7mPC/OoqsAIshlzG7kKuSCuYilxcTCsEPsQmu+XtyZ9M4kJBYhQ9n/1ZCshTVDkbrmOVgEsRHGZt12g40nFNztKC4OUslXHW/wu4+peZTLs3/AanG8ryKVCflWIaus3mLaJC5q6MyGykzAvWh5OJveI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716193267; c=relaxed/simple; bh=nYOXwfkD4Aagx4HIXM0yufDLbQFaKlb+z99rD9eDC1I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VlegGgau9pJLnnjwEqQhSYlQkkQGtR/CI6R0f89niS9ez+5aXavpq2/RquOTe8AvYSDN5hI/OA4kpiNbf21QdI0KNDbtkvE4MSns3r9zLPJgfMT48PoQy+WIXa03UdT1s4nUjsNkAarBGEZQvI0JIUxwnZLGgrkVDCPjAsFpAQk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=PJmc7lYp; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="PJmc7lYp" Received: from pps.filterd (m0353723.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44K62wlh003173; Mon, 20 May 2024 08:20:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=YI+s/9jNr6WRySWWHUvNpGEk6ucoohYRHUK1CkL4ZOQ=; b=PJmc7lYpDNLN572Zl4hjgyvUDv5VUmcPjU9FMoH0E98LyD3hD4kIKry4rnx0v1+HTckA zyLtuG8+nqUipw4NlZ+PI7k3j/9ops0pJBzczrvqom3Lf+tPWTg+TC7WzI7s3kClNvla QMBqmWsKlUE7dAaEL5cT39V6aERxlLd8BRzd3TOZstl34OXAg5UbmmpqYpddK54sxRBX vo7QXu/OcPHEnMKQwg/ofjADjc6hga1pm/XGzSdnxGci1yFr4UYoZrw8PCMDQYYJbTY9 5ZxExJdX1rVQEcu18mVk1gE4ZgxAcNP0aJcHz0hYFglIv682grntB0C+15X2FNLrvI11 TA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3y80w908u4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 20 May 2024 08:20:49 +0000 Received: from m0353723.ppops.net (m0353723.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44K8KmQY013615; Mon, 20 May 2024 08:20:48 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3y80w908u0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 20 May 2024 08:20:48 +0000 Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 44K6bAZE026443; Mon, 20 May 2024 08:20:48 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 3y785m6kkv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 20 May 2024 08:20:47 +0000 Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 44K8KfIK54329746 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 20 May 2024 08:20:43 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9CD192004D; Mon, 20 May 2024 08:20:41 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6DB3020040; Mon, 20 May 2024 08:20:39 +0000 (GMT) Received: from li-c6426e4c-27cf-11b2-a85c-95d65bc0de0e.in.ibm.com (unknown [9.204.206.66]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 20 May 2024 08:20:39 +0000 (GMT) From: Gautam Menghani To: mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, naveen.n.rao@linux.ibm.com, clg@kaod.org Cc: Gautam Menghani , linuxppc-dev@lists.ozlabs.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RESEND PATCH 3/3] arch/powerpc/kvm: Reduce lock contention by moving spinlock from ics to irq_state Date: Mon, 20 May 2024 13:50:10 +0530 Message-ID: <20240520082014.140697-4-gautam@linux.ibm.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240520082014.140697-1-gautam@linux.ibm.com> References: <20240520082014.140697-1-gautam@linux.ibm.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: ByXVqzFTru2cFNVeoqW3aaiIQL3nYDo0 X-Proofpoint-GUID: -nwvkA9zgALh1NB1I92k0ZlrqQJ16Ln8 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-20_04,2024-05-17_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 suspectscore=0 mlxscore=0 lowpriorityscore=0 impostorscore=0 phishscore=0 bulkscore=0 malwarescore=0 clxscore=1015 adultscore=0 spamscore=0 mlxlogscore=390 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405200068 Take a spinlock on state of an IRQ instead of an entire ICS. This improves scalability by reducing contention. Signed-off-by: Gautam Menghani --- arch/powerpc/kvm/book3s_hv_rm_xics.c | 8 ++--- arch/powerpc/kvm/book3s_xics.c | 44 ++++++++++++---------------- arch/powerpc/kvm/book3s_xics.h | 2 +- 3 files changed, 23 insertions(+), 31 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv_rm_xics.c b/arch/powerpc/kvm/book3s_hv_rm_xics.c index e42984878503..178bc869b519 100644 --- a/arch/powerpc/kvm/book3s_hv_rm_xics.c +++ b/arch/powerpc/kvm/book3s_hv_rm_xics.c @@ -308,7 +308,7 @@ static void icp_rm_deliver_irq(struct kvmppc_xics *xics, struct kvmppc_icp *icp, state = &ics->irq_state[src]; /* Get a lock on the ICS */ - arch_spin_lock(&ics->lock); + arch_spin_lock(&state->lock); /* Get our server */ if (!icp || state->server != icp->server_num) { @@ -368,7 +368,7 @@ static void icp_rm_deliver_irq(struct kvmppc_xics *xics, struct kvmppc_icp *icp, * Delivery was successful, did we reject somebody else ? */ if (reject && reject != XICS_IPI) { - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); icp->n_reject++; new_irq = reject; check_resend = 0; @@ -397,13 +397,13 @@ static void icp_rm_deliver_irq(struct kvmppc_xics *xics, struct kvmppc_icp *icp, smp_mb(); if (!icp->state.need_resend) { state->resend = 0; - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); check_resend = 0; goto again; } } out: - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); } static void icp_rm_down_cppr(struct kvmppc_xics *xics, struct kvmppc_icp *icp, diff --git a/arch/powerpc/kvm/book3s_xics.c b/arch/powerpc/kvm/book3s_xics.c index 1dc2f77571e7..466c92cf49fb 100644 --- a/arch/powerpc/kvm/book3s_xics.c +++ b/arch/powerpc/kvm/book3s_xics.c @@ -36,21 +36,13 @@ * LOCKING * ======= * - * Each ICS has a spin lock protecting the information about the IRQ - * sources and avoiding simultaneous deliveries of the same interrupt. + * Each IRQ has a spin lock protecting its state sources and avoiding + * simultaneous deliveries of the same interrupt. * * ICP operations are done via a single compare & swap transaction * (most ICP state fits in the union kvmppc_icp_state) */ -/* - * TODO - * ==== - * - * - Make ICS lockless as well, or at least a per-interrupt lock or hashed - * locks array to improve scalability - */ - /* -- ICS routines -- */ static void icp_deliver_irq(struct kvmppc_xics *xics, struct kvmppc_icp *icp, @@ -142,7 +134,7 @@ static bool write_xive(struct kvmppc_xics *xics, struct kvmppc_ics *ics, unsigned long flags; local_irq_save(flags); - arch_spin_lock(&ics->lock); + arch_spin_lock(&state->lock); state->server = server; state->priority = priority; @@ -154,7 +146,7 @@ static bool write_xive(struct kvmppc_xics *xics, struct kvmppc_ics *ics, deliver = true; } - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); local_irq_restore(flags); return deliver; @@ -207,10 +199,10 @@ int kvmppc_xics_get_xive(struct kvm *kvm, u32 irq, u32 *server, u32 *priority) state = &ics->irq_state[src]; local_irq_save(flags); - arch_spin_lock(&ics->lock); + arch_spin_lock(&state->lock); *server = state->server; *priority = state->priority; - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); local_irq_restore(flags); return 0; @@ -406,7 +398,7 @@ static void icp_deliver_irq(struct kvmppc_xics *xics, struct kvmppc_icp *icp, /* Get a lock on the ICS */ local_irq_save(flags); - arch_spin_lock(&ics->lock); + arch_spin_lock(&state->lock); /* Get our server */ if (!icp || state->server != icp->server_num) { @@ -467,7 +459,7 @@ static void icp_deliver_irq(struct kvmppc_xics *xics, struct kvmppc_icp *icp, * Delivery was successful, did we reject somebody else ? */ if (reject && reject != XICS_IPI) { - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); local_irq_restore(flags); new_irq = reject; check_resend = false; @@ -497,14 +489,14 @@ static void icp_deliver_irq(struct kvmppc_xics *xics, struct kvmppc_icp *icp, smp_mb(); if (!icp->state.need_resend) { state->resend = 0; - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); local_irq_restore(flags); check_resend = false; goto again; } } out: - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&state->lock); local_irq_restore(flags); } @@ -992,20 +984,20 @@ static int xics_debug_show(struct seq_file *m, void *private) seq_printf(m, "=========\nICS state for ICS 0x%x\n=========\n", icsid); - local_irq_save(flags); - arch_spin_lock(&ics->lock); for (i = 0; i < KVMPPC_XICS_IRQ_PER_ICS; i++) { struct ics_irq_state *irq = &ics->irq_state[i]; + local_irq_save(flags); + arch_spin_lock(&irq->lock); seq_printf(m, "irq 0x%06x: server %#x prio %#x save prio %#x pq_state %d resend %d masked pending %d\n", irq->number, irq->server, irq->priority, irq->saved_priority, irq->pq_state, irq->resend, irq->masked_pending); + arch_spin_unlock(&irq->lock); + local_irq_restore(flags); } - arch_spin_unlock(&ics->lock); - local_irq_restore(flags); } return 0; } @@ -1189,7 +1181,7 @@ static int xics_get_source(struct kvmppc_xics *xics, long irq, u64 addr) irqp = &ics->irq_state[idx]; local_irq_save(flags); - arch_spin_lock(&ics->lock); + arch_spin_lock(&irqp->lock); ret = -ENOENT; if (irqp->exists) { val = irqp->server; @@ -1214,7 +1206,7 @@ static int xics_get_source(struct kvmppc_xics *xics, long irq, u64 addr) ret = 0; } - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&irqp->lock); local_irq_restore(flags); if (!ret && put_user(val, ubufp)) @@ -1254,7 +1246,7 @@ static int xics_set_source(struct kvmppc_xics *xics, long irq, u64 addr) return -EINVAL; local_irq_save(flags); - arch_spin_lock(&ics->lock); + arch_spin_lock(&irqp->lock); irqp->server = server; irqp->saved_priority = prio; if (val & KVM_XICS_MASKED) @@ -1272,7 +1264,7 @@ static int xics_set_source(struct kvmppc_xics *xics, long irq, u64 addr) if (val & KVM_XICS_QUEUED) irqp->pq_state |= PQ_QUEUED; irqp->exists = 1; - arch_spin_unlock(&ics->lock); + arch_spin_unlock(&irqp->lock); local_irq_restore(flags); if (val & KVM_XICS_PENDING) diff --git a/arch/powerpc/kvm/book3s_xics.h b/arch/powerpc/kvm/book3s_xics.h index feeb0897d555..1ee62b7a8fdf 100644 --- a/arch/powerpc/kvm/book3s_xics.h +++ b/arch/powerpc/kvm/book3s_xics.h @@ -45,6 +45,7 @@ struct ics_irq_state { u8 exists; int intr_cpu; u32 host_irq; + arch_spinlock_t lock; }; /* Atomic ICP state, updated with a single compare & swap */ @@ -95,7 +96,6 @@ struct kvmppc_icp { }; struct kvmppc_ics { - arch_spinlock_t lock; u16 icsid; struct ics_irq_state irq_state[KVMPPC_XICS_IRQ_PER_ICS]; DECLARE_BITMAP(resend_map, KVMPPC_XICS_IRQ_PER_ICS);