From patchwork Thu Jan 28 05:49:52 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bharata B Rao X-Patchwork-Id: 8146471 Return-Path: X-Original-To: patchwork-qemu-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 363E0BEEE5 for ; Thu, 28 Jan 2016 05:55:36 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 3B01220256 for ; Thu, 28 Jan 2016 05:55:35 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0BC76201CD for ; Thu, 28 Jan 2016 05:55:34 +0000 (UTC) Received: from localhost ([::1]:54122 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aOfYL-0001Ho-6R for patchwork-qemu-devel@patchwork.kernel.org; Thu, 28 Jan 2016 00:55:33 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50805) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aOfUV-0002mE-AQ for qemu-devel@nongnu.org; Thu, 28 Jan 2016 00:51:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aOfUS-0005KF-0K for qemu-devel@nongnu.org; Thu, 28 Jan 2016 00:51:35 -0500 Received: from e23smtp05.au.ibm.com ([202.81.31.147]:49945) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aOfUR-0005JW-61 for qemu-devel@nongnu.org; Thu, 28 Jan 2016 00:51:31 -0500 Received: from localhost by e23smtp05.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 28 Jan 2016 15:51:29 +1000 Received: from d23dlp03.au.ibm.com (202.81.31.214) by e23smtp05.au.ibm.com (202.81.31.211) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 28 Jan 2016 15:51:27 +1000 X-IBM-Helo: d23dlp03.au.ibm.com X-IBM-MailFrom: bharata@linux.vnet.ibm.com X-IBM-RcptTo: qemu-devel@nongnu.org;qemu-ppc@nongnu.org Received: from d23relay08.au.ibm.com (d23relay08.au.ibm.com [9.185.71.33]) by d23dlp03.au.ibm.com (Postfix) with ESMTP id 0E39B3578053; Thu, 28 Jan 2016 16:51:27 +1100 (EST) Received: from d23av02.au.ibm.com (d23av02.au.ibm.com [9.190.235.138]) by d23relay08.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id u0S5otDN35324112; Thu, 28 Jan 2016 16:51:03 +1100 Received: from d23av02.au.ibm.com (localhost [127.0.0.1]) by d23av02.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id u0S5os7f003024; Thu, 28 Jan 2016 16:50:54 +1100 Received: from bharata.in.ibm.com ([9.124.35.121]) by d23av02.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id u0S5oG89001110; Thu, 28 Jan 2016 16:50:51 +1100 From: Bharata B Rao To: qemu-devel@nongnu.org Date: Thu, 28 Jan 2016 11:19:52 +0530 Message-Id: <1453960195-15181-11-git-send-email-bharata@linux.vnet.ibm.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1453960195-15181-1-git-send-email-bharata@linux.vnet.ibm.com> References: <1453960195-15181-1-git-send-email-bharata@linux.vnet.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16012805-0017-0000-0000-000002AC3EBC X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 202.81.31.147 Cc: mjrosato@linux.vnet.ibm.com, ehabkost@redhat.com, aik@ozlabs.ru, Bharata B Rao , mdroth@linux.vnet.ibm.com, agraf@suse.de, pbonzini@redhat.com, qemu-ppc@nongnu.org, tyreld@linux.vnet.ibm.com, nfont@linux.vnet.ibm.com, imammedo@redhat.com, afaerber@suse.de, david@gibson.dropbear.id.au Subject: [Qemu-devel] [PATCH v7 10/13] spapr: CPU hotplug support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Support CPU hotplug via device-add command like this: (qemu) device_add powerpc64-cpu-core,id=core2 In response to device_add, CPU core device will be created. CPU core device creates and realizes CPU thread devices. If the machine type supports CPU hotplug, boot-time CPUs are created as CPU core devices otherwise they continue to be created as individual CPU devices. Set up device tree entries for the hotplugged CPU core and use the exising EPOW event infrastructure to send CPU hotplug notification to the guest. Signed-off-by: Bharata B Rao Reviewed-by: David Gibson --- hw/ppc/spapr.c | 145 +++++++++++++++++++++++++++++++++++++++++--- hw/ppc/spapr_events.c | 3 + hw/ppc/spapr_rtas.c | 24 ++++++++ target-ppc/translate_init.c | 8 +++ 4 files changed, 173 insertions(+), 7 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index eeea411..6ef520d 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -63,6 +63,7 @@ #include "hw/compat.h" #include "qemu-common.h" +#include "hw/ppc/cpu-core.h" #include @@ -601,6 +602,18 @@ static void spapr_populate_cpu_dt(CPUState *cs, void *fdt, int offset, size_t page_sizes_prop_size; uint32_t vcpus_per_socket = smp_threads * smp_cores; uint32_t pft_size_prop[] = {0, cpu_to_be32(spapr->htab_shift)}; + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine()); + sPAPRDRConnector *drc; + sPAPRDRConnectorClass *drck; + int drc_index; + + if (smc->dr_cpu_enabled) { + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, index); + g_assert(drc); + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc); + drc_index = drck->get_index(drc); + _FDT((fdt_setprop_cell(fdt, offset, "ibm,my-drc-index", drc_index))); + } /* Note: we keep CI large pages off for now because a 64K capable guest * provisioned with large pages might otherwise try to map a qemu @@ -1764,6 +1777,8 @@ static void ppc_spapr_init(MachineState *machine) char *filename; int smt = kvmppc_smt_threads(); int smp_max_cores = max_cpus / smp_threads; + int spapr_smp_cores = smp_cpus / smp_threads; + Object *core; msi_supported = true; @@ -1843,13 +1858,22 @@ static void ppc_spapr_init(MachineState *machine) if (machine->cpu_model == NULL) { machine->cpu_model = kvm_enabled() ? "host" : "POWER7"; } - for (i = 0; i < smp_cpus; i++) { - cpu = cpu_ppc_init(machine->cpu_model); - if (cpu == NULL) { - error_report("Unable to find PowerPC CPU definition"); - exit(1); + + if (smc->dr_cpu_enabled) { + for (i = 0; i < spapr_smp_cores; i++) { + core = object_new(TYPE_POWERPC_CPU_CORE); + object_property_set_bool(core, true, "realized", &error_abort); + } + } else { + for (i = 0; i < smp_cpus; i++) { + cpu = cpu_ppc_init(machine->cpu_model); + if (cpu == NULL) { + error_report("Unable to find PowerPC CPU definition"); + exit(1); + } + object_property_set_bool(OBJECT(cpu), true, "realized", + &error_abort); } - spapr_cpu_init(spapr, cpu, &error_fatal); } if (kvm_enabled()) { @@ -2245,10 +2269,92 @@ out: error_propagate(errp, local_err); } +static void *spapr_populate_hotplug_cpu_dt(DeviceState *dev, CPUState *cs, + int *fdt_offset, + sPAPRMachineState *spapr) +{ + PowerPCCPU *cpu = POWERPC_CPU(cs); + DeviceClass *dc = DEVICE_GET_CLASS(cs); + int id = ppc_get_vcpu_dt_id(cpu); + void *fdt; + int offset, fdt_size; + char *nodename; + + fdt = create_device_tree(&fdt_size); + nodename = g_strdup_printf("%s@%x", dc->fw_name, id); + offset = fdt_add_subnode(fdt, 0, nodename); + + spapr_populate_cpu_dt(cs, fdt, offset, spapr); + g_free(nodename); + + *fdt_offset = offset; + return fdt; +} + +static void spapr_core_plug(HotplugHandler *hotplug_dev, DeviceState *dev, + Error **errp) +{ + sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine()); + sPAPRMachineState *ms = SPAPR_MACHINE(qdev_get_machine()); + PowerPCCPUCore *core = POWERPC_CPU_CORE(OBJECT(dev)); + PowerPCCPU *cpu = core->thread0; + CPUState *cs = CPU(cpu); + int id = ppc_get_vcpu_dt_id(cpu); + sPAPRDRConnector *drc = + spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, id); + sPAPRDRConnectorClass *drck; + Error *local_err = NULL; + void *fdt = NULL; + int fdt_offset = 0; + + if (!smc->dr_cpu_enabled) { + /* + * This is a cold plugged CPU core but the machine doesn't support + * DR. So skip the hotplug path ensuring that the core is brought + * up online with out an associated DR connector. + */ + return; + } + + g_assert(drc); + + /* + * Setup CPU DT entries only for hotplugged CPUs. For boot time or + * coldplugged CPUs DT entries are setup in spapr_finalize_fdt(). + */ + if (dev->hotplugged) { + fdt = spapr_populate_hotplug_cpu_dt(dev, cs, &fdt_offset, ms); + } + + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc); + drck->attach(drc, dev, fdt, fdt_offset, !dev->hotplugged, &local_err); + if (local_err) { + g_free(fdt); + error_propagate(errp, local_err); + return; + } + + if (dev->hotplugged) { + /* + * Send hotplug notification interrupt to the guest only in case + * of hotplugged CPUs. + */ + spapr_hotplug_req_add_by_index(drc); + } else { + /* + * Set the right DRC states for cold plugged CPU. + */ + drck->set_allocation_state(drc, SPAPR_DR_ALLOCATION_STATE_USABLE); + drck->set_isolation_state(drc, SPAPR_DR_ISOLATION_STATE_UNISOLATED); + } + return; +} + static void spapr_machine_device_plug(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp) { sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine()); + sPAPRMachineState *ms = SPAPR_MACHINE(hotplug_dev); if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) { int node; @@ -2285,6 +2391,29 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev, } spapr_memory_plug(hotplug_dev, dev, node, errp); + } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) { + CPUState *cs = CPU(dev); + PowerPCCPU *cpu = POWERPC_CPU(cs); + int i; + + if (!smc->dr_cpu_enabled && dev->hotplugged) { + error_setg(errp, "CPU hotplug not supported for this machine"); + cpu_remove_sync(cs); + return; + } + + /* Set NUMA node for the added CPUs */ + for (i = 0; i < nb_numa_nodes; i++) { + if (test_bit(cs->cpu_index, numa_info[i].node_cpu)) { + cs->numa_node = i; + break; + } + } + + spapr_cpu_init(ms, cpu, errp); + spapr_cpu_reset(cpu); + } else if (object_dynamic_cast(OBJECT(dev), TYPE_POWERPC_CPU_CORE)) { + spapr_core_plug(hotplug_dev, dev, errp); } } @@ -2299,7 +2428,9 @@ static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev, static HotplugHandler *spapr_get_hotpug_handler(MachineState *machine, DeviceState *dev) { - if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) { + if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) || + object_dynamic_cast(OBJECT(dev), TYPE_CPU) || + object_dynamic_cast(OBJECT(dev), TYPE_POWERPC_CPU_CORE)) { return HOTPLUG_HANDLER(machine); } return NULL; diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c index 333e6ff..9493c7c 100644 --- a/hw/ppc/spapr_events.c +++ b/hw/ppc/spapr_events.c @@ -436,6 +436,9 @@ static void spapr_hotplug_req_event(uint8_t hp_id, uint8_t hp_action, case SPAPR_DR_CONNECTOR_TYPE_LMB: hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_MEMORY; break; + case SPAPR_DR_CONNECTOR_TYPE_CPU: + hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_CPU; + break; default: /* we shouldn't be signaling hotplug events for resources * that don't support them diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c index 130c917..cee9960 100644 --- a/hw/ppc/spapr_rtas.c +++ b/hw/ppc/spapr_rtas.c @@ -33,6 +33,7 @@ #include "hw/ppc/spapr.h" #include "hw/ppc/spapr_vio.h" +#include "hw/ppc/ppc.h" #include "qapi-event.h" #include "hw/boards.h" @@ -159,6 +160,27 @@ static void rtas_query_cpu_stopped_state(PowerPCCPU *cpu_, rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR); } +/* + * Set the timebase offset of the CPU to that of first CPU. + * This helps hotplugged CPU to have the correct timebase offset. + */ +static void spapr_cpu_update_tb_offset(PowerPCCPU *cpu) +{ + PowerPCCPU *fcpu = POWERPC_CPU(first_cpu); + + cpu->env.tb_env->tb_offset = fcpu->env.tb_env->tb_offset; +} + +static void spapr_cpu_set_endianness(PowerPCCPU *cpu) +{ + PowerPCCPU *fcpu = POWERPC_CPU(first_cpu); + PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(fcpu); + + if (!pcc->interrupts_big_endian(fcpu)) { + cpu->env.spr[SPR_LPCR] |= LPCR_ILE; + } +} + static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachineState *spapr, uint32_t token, uint32_t nargs, target_ulong args, @@ -195,6 +217,8 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachineState *spapr, env->nip = start; env->gpr[3] = r3; cs->halted = 0; + spapr_cpu_set_endianness(cpu); + spapr_cpu_update_tb_offset(cpu); qemu_cpu_kick(cs); diff --git a/target-ppc/translate_init.c b/target-ppc/translate_init.c index 3304ad1..f18489f 100644 --- a/target-ppc/translate_init.c +++ b/target-ppc/translate_init.c @@ -30,6 +30,9 @@ #include "qemu/error-report.h" #include "qapi/visitor.h" #include "hw/qdev-properties.h" +#if !defined(CONFIG_USER_ONLY) +#include "sysemu/sysemu.h" +#endif //#define PPC_DUMP_CPU //#define PPC_DEBUG_SPR @@ -8983,6 +8986,11 @@ static void ppc_cpu_realizefn(DeviceState *dev, Error **errp) } #if !defined(CONFIG_USER_ONLY) + if (cs->cpu_index >= max_cpus) { + error_setg(errp, "Cannot have more than %d CPUs", max_cpus); + return; + } + cpu->cpu_dt_id = (cs->cpu_index / smp_threads) * max_smt + (cs->cpu_index % smp_threads); #endif