From patchwork Thu Dec 8 09:44:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13068175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7125C4332F for ; Thu, 8 Dec 2022 09:45:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230307AbiLHJpH (ORCPT ); Thu, 8 Dec 2022 04:45:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39160 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230293AbiLHJo5 (ORCPT ); Thu, 8 Dec 2022 04:44:57 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 564997061A for ; Thu, 8 Dec 2022 01:44:55 -0800 (PST) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B89h8hT030168; Thu, 8 Dec 2022 09:44:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=S9MtH+bP97hDKsTxBpwY7AWGyysFbg1Gj9dPqllDCj8=; b=KNONbU4SbsJxCOG91z8ckrY5EXdC4t8Uh2ppkeTfMS15EgLX6LK4v04dNfbGwW0IZoGO jn9uYEBTHce6NOAiu7cv8mpYqubt7ykHCD/Fbzmikuu/T1XBv2YXR1xN+f760gbeT/jU MUPdMlvs/hbTGptdpiNMPEJ/8niY6g4b3oLVAIkGCH3c0SFWjVGnWlVXD58cXEAcbE01 XlHOjQTOIcS55kxCiVTJbDdVHbbIIKXBMjB7An+mje1mH0FFGrpZ7wrKxBR3FRav3Ykl gG3FTa86IBmoQNOOlg/GFERs6uV+3WUOujfZ6JOLZmA1cGbNpkYBxrSTwg7LfNeysREo ww== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3mbdhq00nm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:40 +0000 Received: from m0098419.ppops.net (m0098419.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2B89iedt002642; Thu, 8 Dec 2022 09:44:40 GMT Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3mbdhq00mu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:40 +0000 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 2B88tiMB016423; Thu, 8 Dec 2022 09:44:38 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma03ams.nl.ibm.com (PPS) with ESMTPS id 3m9m5y46aj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:37 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2B89iY0Y23986502 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 8 Dec 2022 09:44:34 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DFE1720040; Thu, 8 Dec 2022 09:44:33 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6DB122004B; Thu, 8 Dec 2022 09:44:33 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com (unknown [9.152.222.245]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 8 Dec 2022 09:44:33 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v13 1/7] s390x/cpu topology: Creating CPU topology device Date: Thu, 8 Dec 2022 10:44:26 +0100 Message-Id: <20221208094432.9732-2-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221208094432.9732-1-pmorel@linux.ibm.com> References: <20221208094432.9732-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: Le--2uoos67mM5RlHsMcnf2bR3RjxhuZ X-Proofpoint-GUID: RGNKwkKQ-ueYWbo8vv3z_xswdbKSS3WF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_04,2022-12-07_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 clxscore=1015 lowpriorityscore=0 bulkscore=0 impostorscore=0 priorityscore=1501 malwarescore=0 spamscore=0 adultscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080077 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org We will need a Topology device to transfer the topology during migration and to implement machine reset. The device creation is fenced by s390_has_topology(). Signed-off-by: Pierre Morel --- include/hw/s390x/cpu-topology.h | 44 ++++++++++ hw/s390x/cpu-topology.c | 149 ++++++++++++++++++++++++++++++++ hw/s390x/s390-virtio-ccw.c | 6 ++ hw/s390x/meson.build | 1 + 4 files changed, 200 insertions(+) create mode 100644 include/hw/s390x/cpu-topology.h create mode 100644 hw/s390x/cpu-topology.c diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h new file mode 100644 index 0000000000..6c3d2d080f --- /dev/null +++ b/include/hw/s390x/cpu-topology.h @@ -0,0 +1,44 @@ +/* + * CPU Topology + * + * Copyright IBM Corp. 2022 + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ +#ifndef HW_S390X_CPU_TOPOLOGY_H +#define HW_S390X_CPU_TOPOLOGY_H + +#include "hw/sysbus.h" +#include "hw/qdev-core.h" +#include "qom/object.h" + +#define S390_TOPOLOGY_CPU_IFL 0x03 +#define S390_TOPOLOGY_MAX_ORIGIN ((63 + S390_MAX_CPUS) / 64) + +#define S390_TOPOLOGY_POLARITY_HORIZONTAL 0x00 +#define S390_TOPOLOGY_POLARITY_VERTICAL_LOW 0x01 +#define S390_TOPOLOGY_POLARITY_VERTICAL_MEDIUM 0x02 +#define S390_TOPOLOGY_POLARITY_VERTICAL_HIGH 0x03 + +typedef struct S390TopoSocket { + int active_count; + uint64_t mask[S390_TOPOLOGY_MAX_ORIGIN]; +} S390TopoSocket; + +struct S390Topology { + SysBusDevice parent_obj; + uint32_t num_cores; + uint32_t num_sockets; + S390TopoSocket *socket; +}; + +#define TYPE_S390_CPU_TOPOLOGY "s390-topology" +OBJECT_DECLARE_SIMPLE_TYPE(S390Topology, S390_CPU_TOPOLOGY) + +void s390_init_topology(MachineState *machine, Error **errp); +bool s390_has_topology(void); +S390Topology *s390_get_topology(void); + +#endif diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c new file mode 100644 index 0000000000..b3e59873f6 --- /dev/null +++ b/hw/s390x/cpu-topology.c @@ -0,0 +1,149 @@ +/* + * CPU Topology + * + * Copyright IBM Corp. 2022 + * Author(s): Pierre Morel + + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "hw/qdev-properties.h" +#include "hw/boards.h" +#include "qemu/typedefs.h" +#include "target/s390x/cpu.h" +#include "hw/s390x/s390-virtio-ccw.h" +#include "hw/s390x/cpu-topology.h" + +/** + * s390_has_topology + * + * Return false until the commit activating the topology is + * commited. + */ +bool s390_has_topology(void) +{ + return false; +} + +/** + * s390_get_topology + * + * Returns a pointer to the topology. + * + * This function is called when we know the topology exist. + * Testing if the topology exist is done with s390_has_topology() + */ +S390Topology *s390_get_topology(void) +{ + static S390Topology *s390Topology; + + if (!s390Topology) { + s390Topology = S390_CPU_TOPOLOGY( + object_resolve_path(TYPE_S390_CPU_TOPOLOGY, NULL)); + } + + assert(s390Topology); + + return s390Topology; +} + +/** + * s390_init_topology + * @machine: The Machine state, used to retrieve the SMP parameters + * @errp: the error pointer in case of problem + * + * This function creates and initialize the S390Topology with + * the QEMU -smp parameters we will use during adding cores to the + * topology. + */ +void s390_init_topology(MachineState *machine, Error **errp) +{ + DeviceState *dev; + + if (machine->smp.threads > 1) { + error_setg(errp, "CPU Topology do not support multithreading"); + return; + } + + dev = qdev_new(TYPE_S390_CPU_TOPOLOGY); + + object_property_add_child(&machine->parent_obj, + TYPE_S390_CPU_TOPOLOGY, OBJECT(dev)); + object_property_set_int(OBJECT(dev), "num-cores", + machine->smp.cores, errp); + object_property_set_int(OBJECT(dev), "num-sockets", + machine->smp.sockets, errp); + + sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), errp); +} + +/** + * s390_topology_realize: + * @dev: the device state + * + * We free the socket array allocated in realize. + */ +static void s390_topology_unrealize(DeviceState *dev) +{ + S390Topology *topo = S390_CPU_TOPOLOGY(dev); + + g_free(topo->socket); +} + +/** + * s390_topology_realize: + * @dev: the device state + * @errp: the error pointer (not used) + * + * During realize the machine CPU topology is initialized with the + * QEMU -smp parameters. + * The maximum count of CPU TLE in the all Topology can not be greater + * than the maximum CPUs. + */ +static void s390_topology_realize(DeviceState *dev, Error **errp) +{ + S390Topology *topo = S390_CPU_TOPOLOGY(dev); + + topo->socket = g_new0(S390TopoSocket, topo->num_sockets); +} + +static Property s390_topology_properties[] = { + DEFINE_PROP_UINT32("num-cores", S390Topology, num_cores, 1), + DEFINE_PROP_UINT32("num-sockets", S390Topology, num_sockets, 1), + DEFINE_PROP_END_OF_LIST(), +}; + +/** + * topology_class_init: + * @oc: Object class + * @data: (not used) + * + * A very simple object we will need for reset and migration. + */ +static void topology_class_init(ObjectClass *oc, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(oc); + + dc->realize = s390_topology_realize; + dc->unrealize = s390_topology_unrealize; + device_class_set_props(dc, s390_topology_properties); + set_bit(DEVICE_CATEGORY_MISC, dc->categories); +} + +static const TypeInfo cpu_topology_info = { + .name = TYPE_S390_CPU_TOPOLOGY, + .parent = TYPE_SYS_BUS_DEVICE, + .instance_size = sizeof(S390Topology), + .class_init = topology_class_init, +}; + +static void topology_register(void) +{ + type_register_static(&cpu_topology_info); +} +type_init(topology_register); diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index 2e64ffab45..8971ffb871 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -44,6 +44,7 @@ #include "hw/s390x/pv.h" #include "migration/blocker.h" #include "qapi/visitor.h" +#include "hw/s390x/cpu-topology.h" static Error *pv_mig_blocker; @@ -255,6 +256,11 @@ static void ccw_init(MachineState *machine) /* init CPUs (incl. CPU model) early so s390_has_feature() works */ s390_init_cpus(machine); + /* Need CPU model to be determined before we can set up topology */ + if (s390_has_topology()) { + s390_init_topology(machine, &error_fatal); + } + /* Need CPU model to be determined before we can set up PV */ s390_pv_init(machine->cgs, &error_fatal); diff --git a/hw/s390x/meson.build b/hw/s390x/meson.build index f291016fee..58dfbdff4f 100644 --- a/hw/s390x/meson.build +++ b/hw/s390x/meson.build @@ -24,6 +24,7 @@ s390x_ss.add(when: 'CONFIG_KVM', if_true: files( 's390-stattrib-kvm.c', 'pv.c', 's390-pci-kvm.c', + 'cpu-topology.c', )) s390x_ss.add(when: 'CONFIG_TCG', if_true: files( 'tod-tcg.c', From patchwork Thu Dec 8 09:44:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13068172 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBD2CC3A5A7 for ; Thu, 8 Dec 2022 09:45:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230289AbiLHJpC (ORCPT ); Thu, 8 Dec 2022 04:45:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39154 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230283AbiLHJo4 (ORCPT ); Thu, 8 Dec 2022 04:44:56 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 588CF6F0EB for ; Thu, 8 Dec 2022 01:44:53 -0800 (PST) Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B89g9Ee040385; Thu, 8 Dec 2022 09:44:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=jO0sfzr1+h0Gp7EfzTVqjHr85N84SkrUb9aXRtWd/L0=; b=V1Tq8J0tspKE2v9H6OVlYlUvfnZauuxIzQtT+Bw2iGY3+R3D/rzIxFDOEJm3NOA5r4ok crffsI1Kn1agho0/VJw+bOcBXoLYcjq1Sfn3ZteSC2TeEDcQhP+HBM5bV+Y+SKOHT/R3 S31kFLYsLS9cbp6xUQpn9l3zLbw4/jXAIWMkdNtMC3kaYf327TZBASSH6NKEooREVjfg nEl9bI9CjC9zHORXRL8S6/t3cPnQKowACm+8EDZMIHDyP6hPArDQlS0SOdmofPKkCye5 QVczniP0Bq1Wg9hy/ykZIphAtXkl8FRNUzh0K87sv5IbulMaklLBwRP6S1aX6OZKPZbo Pg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mbdh2g1jy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:44 +0000 Received: from m0187473.ppops.net (m0187473.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2B89hmBm006214; Thu, 8 Dec 2022 09:44:44 GMT Received: from ppma02fra.de.ibm.com (47.49.7a9f.ip4.static.sl-reverse.com [159.122.73.71]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mbdh2g1j4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:43 +0000 Received: from pps.filterd (ppma02fra.de.ibm.com [127.0.0.1]) by ppma02fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 2B7HHaoS002557; Thu, 8 Dec 2022 09:44:41 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma02fra.de.ibm.com (PPS) with ESMTPS id 3m9mb234cb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:41 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2B89iYDA23986504 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 8 Dec 2022 09:44:34 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6C58020040; Thu, 8 Dec 2022 09:44:34 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E9B8820043; Thu, 8 Dec 2022 09:44:33 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com (unknown [9.152.222.245]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 8 Dec 2022 09:44:33 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v13 2/7] s390x/cpu topology: reporting the CPU topology to the guest Date: Thu, 8 Dec 2022 10:44:27 +0100 Message-Id: <20221208094432.9732-3-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221208094432.9732-1-pmorel@linux.ibm.com> References: <20221208094432.9732-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: qR2clKJaLovXcoodGVzfgaOd0TYSeHXq X-Proofpoint-ORIG-GUID: d8xqmLGa0RSrPdQXCINFQAWPazNG7HYV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_04,2022-12-07_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0 mlxlogscore=999 clxscore=1015 bulkscore=0 lowpriorityscore=0 priorityscore=1501 mlxscore=0 adultscore=0 impostorscore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080077 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The guest uses the STSI instruction to get information on the CPU topology. Let us implement the STSI instruction for the basis CPU topology level, level 2. Signed-off-by: Pierre Morel --- target/s390x/cpu.h | 76 +++++++++++++ target/s390x/kvm/cpu_topology.c | 186 ++++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c | 6 +- target/s390x/kvm/meson.build | 3 +- 4 files changed, 269 insertions(+), 2 deletions(-) create mode 100644 target/s390x/kvm/cpu_topology.c diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index 7d6d01325b..729ace321a 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -565,6 +565,80 @@ typedef union SysIB { } SysIB; QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096); +/* + * CPU Topology List provided by STSI with fc=15 provides a list + * of two different Topology List Entries (TLE) types to specify + * the topology hierarchy. + * + * - Container Topology List Entry + * Defines a container to contain other Topology List Entries + * of any type, nested containers or CPU. + * - CPU Topology List Entry + * Specifies the CPUs position, type, entitlement and polarization + * of the CPUs contained in the last Container TLE. + * + * There can be theoretically up to five levels of containers, QEMU + * uses only one level, the socket level. + * + * A container of with a nesting level (NL) greater than 1 can only + * contain another container of nesting level NL-1. + * + * A container of nesting level 1 (socket), contains as many CPU TLE + * as needed to describe the position and qualities of all CPUs inside + * the container. + * The qualities of a CPU are polarization, entitlement and type. + * + * The CPU TLE defines the position of the CPUs of identical qualities + * using a 64bits mask which first bit has its offset defined by + * the CPU address orgin field of the CPU TLE like in: + * CPU address = origin * 64 + bit position within the mask + * + */ +/* Container type Topology List Entry */ +typedef struct SysIBTl_container { + uint8_t nl; + uint8_t reserved[6]; + uint8_t id; +} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_container; +QEMU_BUILD_BUG_ON(sizeof(SysIBTl_container) != 8); + +/* CPU type Topology List Entry */ +typedef struct SysIBTl_cpu { + uint8_t nl; + uint8_t reserved0[3]; + uint8_t reserved1:5; + uint8_t dedicated:1; + uint8_t polarity:2; + uint8_t type; + uint16_t origin; + uint64_t mask; +} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_cpu; +QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16); + +#define S390_TOPOLOGY_MAG 6 +#define S390_TOPOLOGY_MAG6 0 +#define S390_TOPOLOGY_MAG5 1 +#define S390_TOPOLOGY_MAG4 2 +#define S390_TOPOLOGY_MAG3 3 +#define S390_TOPOLOGY_MAG2 4 +#define S390_TOPOLOGY_MAG1 5 +/* Configuration topology */ +typedef struct SysIB_151x { + uint8_t reserved0[2]; + uint16_t length; + uint8_t mag[S390_TOPOLOGY_MAG]; + uint8_t reserved1; + uint8_t mnest; + uint32_t reserved2; + char tle[]; +} QEMU_PACKED QEMU_ALIGNED(8) SysIB_151x; +QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16); + +/* Max size of a SYSIB structure is when all CPU are alone in a container */ +#define S390_TOPOLOGY_SYSIB_SIZE (sizeof(SysIB_151x) + \ + S390_MAX_CPUS * (sizeof(SysIBTl_container) + \ + sizeof(SysIBTl_cpu))) + /* MMU defines */ #define ASCE_ORIGIN (~0xfffULL) /* segment table origin */ #define ASCE_SUBSPACE 0x200 /* subspace group control */ @@ -843,4 +917,6 @@ S390CPU *s390_cpu_addr2state(uint16_t cpu_addr); #include "exec/cpu-all.h" +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar); + #endif diff --git a/target/s390x/kvm/cpu_topology.c b/target/s390x/kvm/cpu_topology.c new file mode 100644 index 0000000000..1d6fd4394b --- /dev/null +++ b/target/s390x/kvm/cpu_topology.c @@ -0,0 +1,186 @@ +/* + * QEMU S390x CPU Topology + * + * Copyright IBM Corp. 2022 + * Author(s): Pierre Morel + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ +#include "qemu/osdep.h" +#include "cpu.h" +#include "hw/s390x/pv.h" +#include "hw/sysbus.h" +#include "hw/s390x/cpu-topology.h" +#include "hw/s390x/sclp.h" + +/* + * s390_topology_add_cpu: + * @topo: pointer to the topology + * @cpu : pointer to the new CPU + * + * The topology pointed by S390CPU, gives us the CPU topology + * established by the -smp QEMU arguments. + * The core-id is used to calculate the position of the CPU inside + * the topology: + * - the socket, container TLE, containing the CPU, we have one socket + * for every num_cores cores. + * - the CPU TLE inside the socket, we have potentionly up to 4 CPU TLE + * in a container TLE with the assumption that all CPU are identical + * with the same polarity and entitlement because we have maximum 256 + * CPUs and each TLE can hold up to 64 identical CPUs. + * - the bit in the 64 bit CPU TLE core mask + */ +static void s390_topology_add_cpu(S390Topology *topo, S390CPU *cpu) +{ + int core_id = cpu->env.core_id; + int bit, origin; + int socket_id; + + socket_id = core_id / topo->num_cores; +/* + * At the core level, each CPU is represented by a bit in a 64bit + * unsigned long. + * The architecture specifies that all CPU in a CPU TLE have the same + * type, polarization and are all dedicated or shared. + * In the case that a socket contains CPUs with different type, polarization + * or entitlement then they will be defined in different CPU containers. + * Currently we assume all CPU are identical IFL CPUs and that they are + * all dedicated CPUs with horizontal polarization. + * Therefore, the only reason to have several CPU TLE inside a socket is + * to support CPU id differences > 64. + * In that case, the origin field in a container represents the offset of + * the first CPU in that CPU container, thereby allowing representation + * of all CPUs via multiple containers. + */ + bit = core_id; + origin = bit / 64; + bit %= 64; + bit = 63 - bit; + + topo->socket[socket_id].active_count++; + set_bit(bit, &topo->socket[socket_id].mask[origin]); +} + +/* + * s390_prepare_topology: + * @s390ms : pointer to the S390CcwMachite State + * + * Calls s390_topology_add_cpu to organize the topology + * inside the topology device before writing the SYSIB. + * + * The topology is currently fixed on boot and does not change + * even on migration. + */ +static void s390_prepare_topology(S390CcwMachineState *s390ms) +{ + const MachineState *ms = MACHINE(s390ms); + S390Topology *topo = s390_get_topology(); + static bool done; + int i; + + if (done) { + return; + } + + for (i = 0; i < ms->possible_cpus->len; i++) { + if (ms->possible_cpus->cpus[i].cpu) { + s390_topology_add_cpu(topo, + S390_CPU(ms->possible_cpus->cpus[i].cpu)); + } + } + + done = true; +} + +static char *fill_container(char *p, int level, int id) +{ + SysIBTl_container *tle = (SysIBTl_container *)p; + + tle->nl = level; + tle->id = id; + return p + sizeof(*tle); +} + +static char *fill_tle_cpu(char *p, uint64_t mask, int origin) +{ + SysIBTl_cpu *tle = (SysIBTl_cpu *)p; + + tle->nl = 0; + tle->dedicated = 1; + tle->polarity = S390_TOPOLOGY_POLARITY_HORIZONTAL; + tle->type = S390_TOPOLOGY_CPU_IFL; + tle->origin = cpu_to_be64(origin * 64); + tle->mask = cpu_to_be64(mask); + return p + sizeof(*tle); +} + +static char *s390_top_set_level2(S390Topology *topo, char *p) +{ + int i, origin; + + for (i = 0; i < topo->num_sockets; i++) { + if (!topo->socket[i].active_count) { + continue; + } + p = fill_container(p, 1, i); + for (origin = 0; origin < S390_TOPOLOGY_MAX_ORIGIN; origin++) { + uint64_t mask = 0L; + + mask = topo->socket[i].mask[origin]; + if (mask) { + p = fill_tle_cpu(p, mask, origin); + } + } + } + return p; +} + +static int setup_stsi(S390CPU *cpu, SysIB_151x *sysib, int level) +{ + S390Topology *topo = s390_get_topology(); + char *p = sysib->tle; + + sysib->mnest = level; + switch (level) { + case 2: + sysib->mag[S390_TOPOLOGY_MAG2] = topo->num_sockets; + sysib->mag[S390_TOPOLOGY_MAG1] = topo->num_cores; + p = s390_top_set_level2(topo, p); + break; + } + + return p - (char *)sysib; +} + +#define S390_TOPOLOGY_MAX_MNEST 2 + +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar) +{ + union { + char place_holder[S390_TOPOLOGY_SYSIB_SIZE]; + SysIB_151x sysib; + } buffer QEMU_ALIGNED(8); + int len; + + if (s390_is_pv() || !s390_has_topology() || + sel2 < 2 || sel2 > S390_TOPOLOGY_MAX_MNEST) { + setcc(cpu, 3); + return; + } + + s390_prepare_topology(S390_CCW_MACHINE(current_machine)); + + len = setup_stsi(cpu, &buffer.sysib, sel2); + + if (len > 4096) { + setcc(cpu, 3); + return; + } + + buffer.sysib.length = cpu_to_be16(len); + s390_cpu_virt_mem_write(cpu, addr, ar, &buffer.sysib, len); + setcc(cpu, 0); +} + diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index 3ac7ec9acf..7dc96f3663 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -51,6 +51,7 @@ #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/s390-virtio-hcall.h" #include "hw/s390x/pv.h" +#include "hw/s390x/cpu-topology.h" #ifndef DEBUG_KVM #define DEBUG_KVM 0 @@ -1919,9 +1920,12 @@ static int handle_stsi(S390CPU *cpu) if (run->s390_stsi.sel1 != 2 || run->s390_stsi.sel2 != 2) { return 0; } - /* Only sysib 3.2.2 needs post-handling for now. */ insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar); return 0; + case 15: + insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr, + run->s390_stsi.ar); + return 0; default: return 0; } diff --git a/target/s390x/kvm/meson.build b/target/s390x/kvm/meson.build index aef52b6686..5daa5c6033 100644 --- a/target/s390x/kvm/meson.build +++ b/target/s390x/kvm/meson.build @@ -1,6 +1,7 @@ s390x_ss.add(when: 'CONFIG_KVM', if_true: files( - 'kvm.c' + 'kvm.c', + 'cpu_topology.c' ), if_false: files( 'stubs.c' )) From patchwork Thu Dec 8 09:44:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13068171 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 807B6C4332F for ; Thu, 8 Dec 2022 09:45:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229586AbiLHJpA (ORCPT ); Thu, 8 Dec 2022 04:45:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230269AbiLHJoy (ORCPT ); Thu, 8 Dec 2022 04:44:54 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F6256F0EC for ; Thu, 8 Dec 2022 01:44:53 -0800 (PST) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B89hBJn030238; Thu, 8 Dec 2022 09:44:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=f1zGJAuoCsybIfQ3bR2/a/pYE9Jqdq5zPfCZ5rSBQr8=; b=IkjFCYgbqFvLjmSk/psXTh5HxhITqYp3J1VJRHONjy9iSKzge+G72BXN4qixTTMd6i3G QU9IPEDE0kXay76ptBnc9SHWSlt/45qgj3WFHNW37ngW/XP3tNUuzp/7AKdHG7sdj7zZ 7TTs1M0nV9FYeogodKLeVSj2hDVIg2b6FRK660dDYY0xktqtOcGlJDcl5DuNzo+YCKNB zpLsBAZyrB1eMCAoKyR9qQCPe2DUqroOQMQBgSud5gLbU3ItrEXfx+48yl3nbCrK6dQt cFiSe4QBoWGnF5CG5ZiWLRfNAnrvGldFAOKo7mr0Qu8VZ9FlNJm9ItPDrjvIoXSIaiuY RQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3mbdhq00pf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:42 +0000 Received: from m0098419.ppops.net (m0098419.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2B89hTDP030646; Thu, 8 Dec 2022 09:44:41 GMT Received: from ppma04fra.de.ibm.com (6a.4a.5195.ip4.static.sl-reverse.com [149.81.74.106]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3mbdhq00n9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:41 +0000 Received: from pps.filterd (ppma04fra.de.ibm.com [127.0.0.1]) by ppma04fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 2B7HHXBR022426; Thu, 8 Dec 2022 09:44:39 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma04fra.de.ibm.com (PPS) with ESMTPS id 3m9kur32tv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:39 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2B89iY5M23397048 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 8 Dec 2022 09:44:36 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DBE7920043; Thu, 8 Dec 2022 09:44:34 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 752F120049; Thu, 8 Dec 2022 09:44:34 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com (unknown [9.152.222.245]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 8 Dec 2022 09:44:34 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v13 3/7] s390x/cpu_topology: resetting the Topology-Change-Report Date: Thu, 8 Dec 2022 10:44:28 +0100 Message-Id: <20221208094432.9732-4-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221208094432.9732-1-pmorel@linux.ibm.com> References: <20221208094432.9732-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: s7Rw40sOqe8Ijpuzlt1Uif8RSufnhqpG X-Proofpoint-GUID: OuMwv6cYBpOhUCN0ODSGJ1MsKSf2a3o_ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_04,2022-12-07_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 clxscore=1015 lowpriorityscore=0 bulkscore=0 impostorscore=0 priorityscore=1501 malwarescore=0 spamscore=0 adultscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080077 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org During a subsystem reset the Topology-Change-Report is cleared by the machine. Let's ask KVM to clear the Modified Topology Change Report (MTCR) bit of the SCA in the case of a subsystem reset. Signed-off-by: Pierre Morel Reviewed-by: Nico Boehr Reviewed-by: Janis Schoetterl-Glausch --- target/s390x/cpu.h | 1 + target/s390x/kvm/kvm_s390x.h | 1 + hw/s390x/cpu-topology.c | 12 ++++++++++++ hw/s390x/s390-virtio-ccw.c | 1 + target/s390x/cpu-sysemu.c | 13 +++++++++++++ target/s390x/kvm/kvm.c | 17 +++++++++++++++++ 6 files changed, 45 insertions(+) diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index 729ace321a..bc1a7de932 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -853,6 +853,7 @@ void s390_enable_css_support(S390CPU *cpu); void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg); int s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch_id, int vq, bool assign); +void s390_cpu_topology_reset(void); #ifndef CONFIG_USER_ONLY unsigned int s390_cpu_set_state(uint8_t cpu_state, S390CPU *cpu); #else diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h index f9785564d0..649dae5948 100644 --- a/target/s390x/kvm/kvm_s390x.h +++ b/target/s390x/kvm/kvm_s390x.h @@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void); void kvm_s390_restart_interrupt(S390CPU *cpu); void kvm_s390_stop_interrupt(S390CPU *cpu); void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info); +int kvm_s390_topology_set_mtcr(uint64_t attr); #endif /* KVM_S390X_H */ diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index b3e59873f6..f54afcf550 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -118,6 +118,17 @@ static Property s390_topology_properties[] = { DEFINE_PROP_END_OF_LIST(), }; +/** + * s390_topology_reset: + * @dev: the device + * + * Calls the sysemu topology reset + */ +static void s390_topology_reset(DeviceState *dev) +{ + s390_cpu_topology_reset(); +} + /** * topology_class_init: * @oc: Object class @@ -133,6 +144,7 @@ static void topology_class_init(ObjectClass *oc, void *data) dc->unrealize = s390_topology_unrealize; device_class_set_props(dc, s390_topology_properties); set_bit(DEVICE_CATEGORY_MISC, dc->categories); + dc->reset = s390_topology_reset; } static const TypeInfo cpu_topology_info = { diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index 8971ffb871..f03a5aaeea 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -109,6 +109,7 @@ static const char *const reset_dev_types[] = { "s390-flic", "diag288", TYPE_S390_PCI_HOST_BRIDGE, + TYPE_S390_CPU_TOPOLOGY, }; static void subsystem_reset(void) diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c index 948e4bd3e0..e27864c5f5 100644 --- a/target/s390x/cpu-sysemu.c +++ b/target/s390x/cpu-sysemu.c @@ -306,3 +306,16 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg) kvm_s390_set_diag318(cs, arg.host_ulong); } } + +void s390_cpu_topology_reset(void) +{ + int ret; + + if (kvm_enabled()) { + ret = kvm_s390_topology_set_mtcr(0); + if (ret) { + error_report("Failed to set Modified Topology Change Report: %s", + strerror(-ret)); + } + } +} diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index 7dc96f3663..5b6383eab0 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -2593,6 +2593,23 @@ int kvm_s390_get_zpci_op(void) return cap_zpci_op; } +int kvm_s390_topology_set_mtcr(uint64_t attr) +{ + struct kvm_device_attr attribute = { + .group = KVM_S390_VM_CPU_TOPOLOGY, + .attr = attr, + }; + + if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) { + return 0; + } + if (!kvm_vm_check_attr(kvm_state, KVM_S390_VM_CPU_TOPOLOGY, attr)) { + return -ENOTSUP; + } + + return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attribute); +} + void kvm_arch_accel_class_init(ObjectClass *oc) { } From patchwork Thu Dec 8 09:44:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13068173 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EF1EC63709 for ; Thu, 8 Dec 2022 09:45:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230296AbiLHJpE (ORCPT ); Thu, 8 Dec 2022 04:45:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230243AbiLHJo5 (ORCPT ); Thu, 8 Dec 2022 04:44:57 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 72C5E6F0FE for ; Thu, 8 Dec 2022 01:44:54 -0800 (PST) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B89Sut3030561; Thu, 8 Dec 2022 09:44:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=S2aNrKrNNfRw7fXFnytQVWWPiOQA7nMgqNrhmxBah2I=; b=P0Rbv/z5dBcVjY2/KFLfbsMo4ij7VjEYXL9hUyW5baUn0+i3vEXzj+EhCds/EInREcQh MS1kptQ7H3vNyYo2H2k+/tRBoAjt32ajh9xgUbVV/fUk2L0u5Fwwpbqs8tmot8iKCCbN ru4fBh+ahYtyWntAV63b65Rkxc3nQ42vAeqVbiK5EjIBDH8Y62ncaS+9HLrB9S+b11cv EC3dbI+2/5zLD5foQ6hQExBtM35T04bZEGc/8cIb3ZElQPeU6c6ZoCf2QbXVDZ+VBfA7 8gwytKmLHcN4wHPEHLPzea5xQFVplV7taf+u9BHNqX+kN0PYQvNU7RJXFFhKHVvEQ6x2 YA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mbdb40cng-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:42 +0000 Received: from m0098396.ppops.net (m0098396.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2B89bIwT002509; Thu, 8 Dec 2022 09:44:41 GMT Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mbdb40cme-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:41 +0000 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 2B88tiMC016423; Thu, 8 Dec 2022 09:44:39 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma03ams.nl.ibm.com (PPS) with ESMTPS id 3m9m5y46ak-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:39 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2B89iZWF42992018 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 8 Dec 2022 09:44:35 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 585C120049; Thu, 8 Dec 2022 09:44:35 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E4D632004B; Thu, 8 Dec 2022 09:44:34 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com (unknown [9.152.222.245]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 8 Dec 2022 09:44:34 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v13 4/7] s390x/cpu_topology: CPU topology migration Date: Thu, 8 Dec 2022 10:44:29 +0100 Message-Id: <20221208094432.9732-5-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221208094432.9732-1-pmorel@linux.ibm.com> References: <20221208094432.9732-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: Mqa_5pckH4tNmhdSxrw0csx-yS0JTb2S X-Proofpoint-ORIG-GUID: QRh0O0-U2s4W-179ysnIkGaA443Lec0y X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_04,2022-12-07_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 mlxlogscore=999 clxscore=1015 malwarescore=0 priorityscore=1501 mlxscore=0 bulkscore=0 phishscore=0 spamscore=0 impostorscore=0 suspectscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080077 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The migration can only take place if both source and destination of the migration both use or both do not use the CPU topology facility. We indicate a change in topology during migration postload for the case the topology changed between source and destination. Signed-off-by: Pierre Morel --- target/s390x/cpu.h | 1 + hw/s390x/cpu-topology.c | 49 +++++++++++++++++++++++++++++++++++++++ target/s390x/cpu-sysemu.c | 8 +++++++ 3 files changed, 58 insertions(+) diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index bc1a7de932..284c708a6c 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -854,6 +854,7 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg); int s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch_id, int vq, bool assign); void s390_cpu_topology_reset(void); +int s390_cpu_topology_mtcr_set(void); #ifndef CONFIG_USER_ONLY unsigned int s390_cpu_set_state(uint8_t cpu_state, S390CPU *cpu); #else diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index f54afcf550..8a2fe041d4 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -18,6 +18,7 @@ #include "target/s390x/cpu.h" #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" +#include "migration/vmstate.h" /** * s390_has_topology @@ -129,6 +130,53 @@ static void s390_topology_reset(DeviceState *dev) s390_cpu_topology_reset(); } +/** + * cpu_topology_postload + * @opaque: a pointer to the S390Topology + * @version_id: version identifier + * + * We check that the topology is used or is not used + * on both side identically. + * + * If the topology is in use we set the Modified Topology Change Report + * on the destination host. + */ +static int cpu_topology_postload(void *opaque, int version_id) +{ + int ret; + + /* We do not support CPU Topology, all is good */ + if (!s390_has_topology()) { + return 0; + } + + /* We support CPU Topology, set the MTCR unconditionally */ + ret = s390_cpu_topology_mtcr_set(); + if (ret) { + error_report("Failed to set MTCR: %s", strerror(-ret)); + } + return ret; +} + +/** + * cpu_topology_needed: + * @opaque: The pointer to the S390Topology + * + * We always need to know if source and destination use the topology. + */ +static bool cpu_topology_needed(void *opaque) +{ + return s390_has_topology(); +} + +const VMStateDescription vmstate_cpu_topology = { + .name = "cpu_topology", + .version_id = 1, + .post_load = cpu_topology_postload, + .minimum_version_id = 1, + .needed = cpu_topology_needed, +}; + /** * topology_class_init: * @oc: Object class @@ -145,6 +193,7 @@ static void topology_class_init(ObjectClass *oc, void *data) device_class_set_props(dc, s390_topology_properties); set_bit(DEVICE_CATEGORY_MISC, dc->categories); dc->reset = s390_topology_reset; + dc->vmsd = &vmstate_cpu_topology; } static const TypeInfo cpu_topology_info = { diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c index e27864c5f5..a8e3e6219d 100644 --- a/target/s390x/cpu-sysemu.c +++ b/target/s390x/cpu-sysemu.c @@ -319,3 +319,11 @@ void s390_cpu_topology_reset(void) } } } + +int s390_cpu_topology_mtcr_set(void) +{ + if (kvm_enabled()) { + return kvm_s390_topology_set_mtcr(1); + } + return -ENOENT; +} From patchwork Thu Dec 8 09:44:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13068176 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B6C9C63705 for ; Thu, 8 Dec 2022 09:45:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230324AbiLHJpK (ORCPT ); Thu, 8 Dec 2022 04:45:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39162 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230294AbiLHJo6 (ORCPT ); Thu, 8 Dec 2022 04:44:58 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED5F36F0F5 for ; Thu, 8 Dec 2022 01:44:55 -0800 (PST) Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B89gC2E040494; Thu, 8 Dec 2022 09:44:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=aU0MX7D/CuX0IIyBhXjo6etgY+kXRehqHWqtBY0Oamo=; b=C0QQClEE6bLjnadWp199gZ8R+OhtaDSMd4t9CY8CdSOanUkbPb6c8K8jIiqSjN0fcAOt D8iUWxRP2JqWpoZ77IcBLWXbEmhtFcMum1R/MkyqAnwDLKW+P06Hb2vTL+R8mkhtWtJH HBIhUS0lMwbaUHkTJqNoJXbRybyrtMfNRIjx/CUm//ItVkD8fzpBq2AgQPcKc/Fdpl4a oXBLZgXzXGt3BwDN6IT+Zp3RJLWSHo47yKNuvF+yc9PmV6lGJuTgQtMcuxqDQfpIxiKG H5piDpRPdMBPqJlFQe/eqFNQXqBQ6QGR/8Q5UgKTmLDhe8UcZeblGrhQI+gj1m4hLcqm 1A== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mbdh2g1jb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:42 +0000 Received: from m0187473.ppops.net (m0187473.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2B89hooH006265; Thu, 8 Dec 2022 09:44:42 GMT Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mbdh2g1h7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:41 +0000 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 2B893GDg027261; Thu, 8 Dec 2022 09:44:39 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma04ams.nl.ibm.com (PPS) with ESMTPS id 3m9ks4470x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:39 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2B89iZpZ38928740 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 8 Dec 2022 09:44:35 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C7EDE20040; Thu, 8 Dec 2022 09:44:35 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 613E22004D; Thu, 8 Dec 2022 09:44:35 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com (unknown [9.152.222.245]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 8 Dec 2022 09:44:35 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v13 5/7] s390x/cpu_topology: interception of PTF instruction Date: Thu, 8 Dec 2022 10:44:30 +0100 Message-Id: <20221208094432.9732-6-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221208094432.9732-1-pmorel@linux.ibm.com> References: <20221208094432.9732-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: u5xiGOePMKdVhZOCa7HTJ6jG_OTxNSjQ X-Proofpoint-ORIG-GUID: VifBaDXOP6kbKLy7zE_O5yS3EuxxeEgB X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_04,2022-12-07_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0 mlxlogscore=999 clxscore=1015 bulkscore=0 lowpriorityscore=0 priorityscore=1501 mlxscore=0 adultscore=0 impostorscore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080077 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When the host supports the CPU topology facility, the PTF instruction with function code 2 is interpreted by the SIE, provided that the userland hypervizor activates the interpretation by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension. The PTF instructions with function code 0 and 1 are intercepted and must be emulated by the userland hypervizor. Signed-off-by: Pierre Morel Reviewed-by: Janis Schoetterl-Glausch --- include/hw/s390x/s390-virtio-ccw.h | 6 ++++ hw/s390x/cpu-topology.c | 52 ++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c | 11 +++++++ 3 files changed, 69 insertions(+) diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h index 9bba21a916..c1d46e78af 100644 --- a/include/hw/s390x/s390-virtio-ccw.h +++ b/include/hw/s390x/s390-virtio-ccw.h @@ -30,6 +30,12 @@ struct S390CcwMachineState { uint8_t loadparm[8]; }; +#define S390_PTF_REASON_NONE (0x00 << 8) +#define S390_PTF_REASON_DONE (0x01 << 8) +#define S390_PTF_REASON_BUSY (0x02 << 8) +#define S390_TOPO_FC_MASK 0xffUL +void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra); + struct S390CcwMachineClass { /*< private >*/ MachineClass parent_class; diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index 8a2fe041d4..a2a553e362 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -19,6 +19,58 @@ #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" #include "migration/vmstate.h" +#include "target/s390x/cpu.h" +#include "hw/s390x/s390-virtio-ccw.h" + +/* + * s390_handle_ptf: + * + * @register 1: contains the function code + * + * Function codes 0 and 1 handle the CPU polarization. + * We assume an horizontal topology, the only one supported currently + * by Linux, consequently we answer to function code 0, requesting + * horizontal polarization that it is already the current polarization + * and reject vertical polarization request without further explanation. + * + * Function code 2 is handling topology changes and is interpreted + * by the SIE. + */ +void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra) +{ + CPUS390XState *env = &cpu->env; + uint64_t reg = env->regs[r1]; + uint8_t fc = reg & S390_TOPO_FC_MASK; + + if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) { + s390_program_interrupt(env, PGM_OPERATION, ra); + return; + } + + if (env->psw.mask & PSW_MASK_PSTATE) { + s390_program_interrupt(env, PGM_PRIVILEGED, ra); + return; + } + + if (reg & ~S390_TOPO_FC_MASK) { + s390_program_interrupt(env, PGM_SPECIFICATION, ra); + return; + } + + switch (fc) { + case 0: /* Horizontal polarization is already set */ + env->regs[r1] |= S390_PTF_REASON_DONE; + setcc(cpu, 2); + break; + case 1: /* Vertical polarization is not supported */ + env->regs[r1] |= S390_PTF_REASON_NONE; + setcc(cpu, 2); + break; + default: + /* Note that fc == 2 is interpreted by the SIE */ + s390_program_interrupt(env, PGM_SPECIFICATION, ra); + } +} /** * s390_has_topology diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index 5b6383eab0..a79fdf1c79 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -97,6 +97,7 @@ #define PRIV_B9_EQBS 0x9c #define PRIV_B9_CLP 0xa0 +#define PRIV_B9_PTF 0xa2 #define PRIV_B9_PCISTG 0xd0 #define PRIV_B9_PCILG 0xd2 #define PRIV_B9_RPCIT 0xd3 @@ -1465,6 +1466,13 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct kvm_run *run) } } +static void kvm_handle_ptf(S390CPU *cpu, struct kvm_run *run) +{ + uint8_t r1 = (run->s390_sieic.ipb >> 20) & 0x0f; + + s390_handle_ptf(cpu, r1, RA_IGNORED); +} + static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1) { int r = 0; @@ -1482,6 +1490,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1) case PRIV_B9_RPCIT: r = kvm_rpcit_service_call(cpu, run); break; + case PRIV_B9_PTF: + kvm_handle_ptf(cpu, run); + break; case PRIV_B9_EQBS: /* just inject exception */ r = -1; From patchwork Thu Dec 8 09:44:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13068177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E0A6C4332F for ; Thu, 8 Dec 2022 09:45:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230354AbiLHJpL (ORCPT ); Thu, 8 Dec 2022 04:45:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230295AbiLHJo6 (ORCPT ); Thu, 8 Dec 2022 04:44:58 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 01C837063C for ; Thu, 8 Dec 2022 01:44:55 -0800 (PST) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B89Sodj030372; Thu, 8 Dec 2022 09:44:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=wKDBZOQcvqo9qMIwZyG6Fwl2n08Nj1JefsyyRmaBv04=; b=SG4xhNe/15A3Dqt9vbaadIR4H9ugF8+e4/AybX1dVjmtGRVJo8Ao41UjX7usuOjZ0h99 OAL/NPGivK7UfvB039krf+xahh7g9RnfY7IcsnvdTEQTWZGP8dt9oa6eTvchQg0qvyKs JYifmDyKQJ1aHzryDcdGzlO4GuPOL0Q+GhLZgwYIX88rEIcSamC/1yLQFejaJa//qReb MuIpKNCI3l9Fux5QScs9AvV1EddTbs7dou/UOGH6lEfCY7QRRx8NGuN8wjYtKo+bg9vv 1JPWYIHsoRZyAHVm6rHytuThOhzCoCWVZFDdhUodM39MMJRNlhhxvwi0GxkNQGPBDD6O HA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mbdb40cqy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:49 +0000 Received: from m0098396.ppops.net (m0098396.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2B89TMFZ032149; Thu, 8 Dec 2022 09:44:48 GMT Received: from ppma06fra.de.ibm.com (48.49.7a9f.ip4.static.sl-reverse.com [159.122.73.72]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mbdb40cq6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:48 +0000 Received: from pps.filterd (ppma06fra.de.ibm.com [127.0.0.1]) by ppma06fra.de.ibm.com (8.17.1.19/8.16.1.2) with ESMTP id 2B7HHRdR003014; Thu, 8 Dec 2022 09:44:45 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma06fra.de.ibm.com (PPS) with ESMTPS id 3m9m6y33fk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:45 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2B89ia5u42861040 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 8 Dec 2022 09:44:37 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4394C20040; Thu, 8 Dec 2022 09:44:36 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D11D220049; Thu, 8 Dec 2022 09:44:35 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com (unknown [9.152.222.245]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 8 Dec 2022 09:44:35 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v13 6/7] s390x/cpu_topology: activating CPU topology Date: Thu, 8 Dec 2022 10:44:31 +0100 Message-Id: <20221208094432.9732-7-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221208094432.9732-1-pmorel@linux.ibm.com> References: <20221208094432.9732-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: WjOVbXSugKa0qK4Ehcs-hUKJhniIBPyn X-Proofpoint-ORIG-GUID: GB92fsjZ5TwJYfk4EnMYz_vPkYeRHxc9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_04,2022-12-07_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 mlxlogscore=999 clxscore=1015 malwarescore=0 priorityscore=1501 mlxscore=0 bulkscore=0 phishscore=0 spamscore=0 impostorscore=0 suspectscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080077 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The KVM capability, KVM_CAP_S390_CPU_TOPOLOGY is used to activate the S390_FEAT_CONFIGURATION_TOPOLOGY feature and the topology facility for the guest in the case the topology is available in QEMU and in KVM. The feature is disabled by default and fenced for SE (secure execution). Signed-off-by: Pierre Morel --- include/hw/s390x/cpu-topology.h | 10 +++++++++- hw/s390x/cpu-topology.c | 5 ++--- target/s390x/cpu_models.c | 1 + target/s390x/kvm/kvm.c | 12 ++++++++++++ 4 files changed, 24 insertions(+), 4 deletions(-) diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h index 6c3d2d080f..fe25a3bf6b 100644 --- a/include/hw/s390x/cpu-topology.h +++ b/include/hw/s390x/cpu-topology.h @@ -38,7 +38,15 @@ struct S390Topology { OBJECT_DECLARE_SIMPLE_TYPE(S390Topology, S390_CPU_TOPOLOGY) void s390_init_topology(MachineState *machine, Error **errp); -bool s390_has_topology(void); S390Topology *s390_get_topology(void); +#ifdef CONFIG_KVM +bool s390_has_topology(void); +#else +static inline bool s390_has_topology(void) +{ + return false; +} +#endif + #endif diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index a2a553e362..bf125bb4c0 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -75,12 +75,11 @@ void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra) /** * s390_has_topology * - * Return false until the commit activating the topology is - * commited. + * checks the S390_FEAT_CONFIGURATION_TOPOLOGY availability. */ bool s390_has_topology(void) { - return false; + return s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY); } /** diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c index c3a4f80633..3f05e05fd3 100644 --- a/target/s390x/cpu_models.c +++ b/target/s390x/cpu_models.c @@ -253,6 +253,7 @@ bool s390_has_feat(S390Feat feat) case S390_FEAT_SIE_CMMA: case S390_FEAT_SIE_PFMFI: case S390_FEAT_SIE_IBS: + case S390_FEAT_CONFIGURATION_TOPOLOGY: return false; break; default: diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index a79fdf1c79..abf1202c28 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -2471,6 +2471,18 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp) set_bit(S390_FEAT_UNPACK, model->features); } + /* + * If we have support for CPU Topology in KVM + * activate the CPU TOPOLOGY feature. + */ + if (kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) { + if (kvm_vm_enable_cap(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY, 0) < 0) { + error_setg(errp, "KVM: Error enabling KVM_CAP_S390_CPU_TOPOLOGY"); + return; + } + set_bit(S390_FEAT_CONFIGURATION_TOPOLOGY, model->features); + } + /* We emulate a zPCI bus and AEN, therefore we don't need HW support */ set_bit(S390_FEAT_ZPCI, model->features); set_bit(S390_FEAT_ADAPTER_EVENT_NOTIFICATION, model->features); From patchwork Thu Dec 8 09:44:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13068174 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68CBAC4332F for ; Thu, 8 Dec 2022 09:45:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230304AbiLHJpG (ORCPT ); Thu, 8 Dec 2022 04:45:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39002 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230260AbiLHJo5 (ORCPT ); Thu, 8 Dec 2022 04:44:57 -0500 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 564BD7061E for ; Thu, 8 Dec 2022 01:44:55 -0800 (PST) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2B89HSjU013456; Thu, 8 Dec 2022 09:44:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=cG8TO8J0JhjMLHeFR+gvjjd0ZIzzXmKYU9QQQjVs628=; b=ow8MZrsfQF1gZQkCVa+NhvofbIZ7XkbnTyxsNNd89YANCf4oKiiQ8JLkhrV4EPXXnA0d 8DsI6aqdvan8nrz3LiRAy+XZVid6CrECOKHvtLuHJ91qwiPj0fg6PEvwI27eZlVgmsjH 6MZtdCm+qVGvFvX/pj7/wUMX95NFFGTXHKFZCzdn4PfHqQyf+EegQgVcDA2cEGmQZ2DR vhXr98lygQhPCn7p4iw7w5DxmxFnYyg7Vu0lHteAs8uhiGA4lAZPN7Cmnj3x6yxZVM1A HWLbKb+9GXg55g3twe0mATa7oDlxuga3nwMYDVuIuuHT4455msB0+xxQlSU7DwUWKlHj wA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mbd5cgmex-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:43 +0000 Received: from m0098417.ppops.net (m0098417.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2B89h5we027592; Thu, 8 Dec 2022 09:44:42 GMT Received: from ppma02fra.de.ibm.com (47.49.7a9f.ip4.static.sl-reverse.com [159.122.73.71]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mbd5cgme9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:42 +0000 Received: from pps.filterd (ppma02fra.de.ibm.com [127.0.0.1]) by ppma02fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 2B7HHaVa002553; Thu, 8 Dec 2022 09:44:40 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma02fra.de.ibm.com (PPS) with ESMTPS id 3m9mb234ca-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Dec 2022 09:44:40 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2B89iahT40239386 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 8 Dec 2022 09:44:37 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B0B9C20043; Thu, 8 Dec 2022 09:44:36 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4B7872004D; Thu, 8 Dec 2022 09:44:36 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com (unknown [9.152.222.245]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 8 Dec 2022 09:44:36 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v13 7/7] docs/s390x: document s390x cpu topology Date: Thu, 8 Dec 2022 10:44:32 +0100 Message-Id: <20221208094432.9732-8-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221208094432.9732-1-pmorel@linux.ibm.com> References: <20221208094432.9732-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: LYdLbaQtXdsdPJ9IKDCUqK5lywi98jhi X-Proofpoint-ORIG-GUID: Xec_YaXbKLzt6rfLXJMBDnEdpcgvdywQ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-08_04,2022-12-07_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 mlxlogscore=999 malwarescore=0 spamscore=0 mlxscore=0 lowpriorityscore=0 priorityscore=1501 bulkscore=0 impostorscore=0 phishscore=0 adultscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2212080077 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add some basic examples for the definition of cpu topology in s390x. Signed-off-by: Pierre Morel --- docs/system/s390x/cpu-topology.rst | 87 ++++++++++++++++++++++++++++++ docs/system/target-s390x.rst | 1 + 2 files changed, 88 insertions(+) create mode 100644 docs/system/s390x/cpu-topology.rst diff --git a/docs/system/s390x/cpu-topology.rst b/docs/system/s390x/cpu-topology.rst new file mode 100644 index 0000000000..12f974cc54 --- /dev/null +++ b/docs/system/s390x/cpu-topology.rst @@ -0,0 +1,87 @@ +CPU Topology on s390x +===================== + +CPU Topology on S390x provides up to 5 levels of topology containers: +nodes, drawers, books, sockets and CPUs. +While the higher level containers, Containers Topology List Entries, +(Containers TLE) define a tree hierarchy, the lowest level of topology +definition, the CPU Topology List Entry (CPU TLE), provides the placement +of the CPUs inside the parent container. + +Currently QEMU CPU topology uses a single level of container: the sockets. + +For backward compatibility, threads can be declared on the ``-smp`` command +line. They will be seen as CPUs by the guest as long as multithreading +is not really supported by QEMU for S390. + +Prerequisites +------------- + +To use CPU Topology a Linux QEMU/KVM machine providing the CPU Topology facility +(STFLE bit 11) is required. + +However, since this facility has been enabled by default in an early version +of QEMU, we use a capability, ``KVM_CAP_S390_CPU_TOPOLOGY``, to notify KVM +QEMU use of the CPU Topology. + +Enabling CPU topology +--------------------- + +Currently, CPU topology is only enabled in the host model. + +Enabling CPU topology in a CPU model is done by setting the CPU flag +``ctop`` to ``on`` like in: + +.. code-block:: bash + + -cpu gen16b,ctop=on + +Having the topology disabled by default allows migration between +old and new QEMU without adding new flags. + +Indicating the CPU topology to the Virtual Machine +-------------------------------------------------- + +The CPU Topology, can be specified on the QEMU command line +with the ``-smp`` or the ``-device`` QEMU command arguments. + +In the following machine we define 8 sockets with 4 cores each. +Note that S390 QEMU machines do not implement multithreading. + +.. code-block:: bash + + $ qemu-system-s390x -m 2G \ + -cpu gen16b,ctop=on \ + -smp cpus=5,sockets=8,cores=4,maxcpus=32 \ + -device host-s390x-cpu,core-id=14 \ + +New CPUs can be plugged using the device_add hmp command like in: + +.. code-block:: bash + + (qemu) device_add host-s390x-cpu,core-id=9 + +The core-id defines the placement of the core in the topology by +starting with core 0 in socket 0 up to maxcpus. + +In the example above: + +* There are 5 CPUs provided to the guest with the ``-smp`` command line + They will take the core-ids 0,1,2,3,4 + As we have 4 cores in a socket, we have 4 CPUs provided + to the guest in socket 0, with core-ids 0,1,2,3. + The last cpu, with core-id 4, will be on socket 1. + +* the core with ID 14 provided by the ``-device`` command line will + be placed in socket 3, with core-id 14 + +* the core with ID 9 provided by the ``device_add`` qmp command will + be placed in socket 2, with core-id 9 + +Note that the core ID is machine wide and the CPU TLE masks provided +by the STSI instruction will be writen in a big endian mask: + +* in socket 0: 0xf0000000 (core id 0,1,2,3) +* in socket 1: 0x08000000 (core id 4) +* in socket 2: 0x00400000 (core id 9) +* in socket 3: 0x00020000 (core id 14) diff --git a/docs/system/target-s390x.rst b/docs/system/target-s390x.rst index c636f64113..ff0ffe04f3 100644 --- a/docs/system/target-s390x.rst +++ b/docs/system/target-s390x.rst @@ -33,3 +33,4 @@ Architectural features .. toctree:: s390x/bootdevices s390x/protvirt + s390x/cpu-topology