From patchwork Mon Nov 12 11:57:30 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 1727881 Return-Path: X-Original-To: patchwork-kvm@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork1.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork1.kernel.org (Postfix) with ESMTP id 0C30D40D1B for ; Mon, 12 Nov 2012 11:57:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752951Ab2KLL5q (ORCPT ); Mon, 12 Nov 2012 06:57:46 -0500 Received: from cam-admin0.cambridge.arm.com ([217.140.96.50]:55182 "EHLO cam-admin0.cambridge.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752915Ab2KLL5m (ORCPT ); Mon, 12 Nov 2012 06:57:42 -0500 Received: from mudshark.cambridge.arm.com (mudshark.cambridge.arm.com [10.1.79.58]) by cam-admin0.cambridge.arm.com (8.12.6/8.12.6) with ESMTP id qACBvYED025847; Mon, 12 Nov 2012 11:57:35 GMT Received: by mudshark.cambridge.arm.com (Postfix, from userid 1000) id 7E354C2A97; Mon, 12 Nov 2012 11:57:34 +0000 (GMT) From: Will Deacon To: kvm@vger.kernel.org Cc: penberg@kernel.org, marc.zyngier@arm.com, c.dall@virtualopensystems.com, matt.evans@arm.com, peter.maydell@linaro.org, Will Deacon Subject: [RFC PATCH 16/16] kvm tools: add support for ARMv7 processors Date: Mon, 12 Nov 2012 11:57:30 +0000 Message-Id: <1352721450-11340-17-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 1.7.4.1 In-Reply-To: <1352721450-11340-1-git-send-email-will.deacon@arm.com> References: <1352721450-11340-1-git-send-email-will.deacon@arm.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This patch adds initial support for ARMv7 processors (more specifically, Cortex-A15) to kvmtool. Everything is driven by FDT, including dynamic generation of virtio nodes for MMIO devices (PCI is not used due to lack of a suitable host-bridge). The virtual timers and virtual interrupt controller (VGIC) are provided by the kernel and require very little in terms of userspace code. Signed-off-by: Will Deacon Acked-By: Matt Evans --- tools/kvm/Makefile | 20 ++- tools/kvm/arm/aarch32/cortex-a15.c | 98 ++++++++++ tools/kvm/arm/aarch32/include/kvm/barrier.h | 10 + tools/kvm/arm/aarch32/include/kvm/kvm-arch.h | 28 +++ tools/kvm/arm/aarch32/kvm-cpu.c | 111 +++++++++++ tools/kvm/arm/aarch32/smp-pen.S | 14 ++ tools/kvm/arm/fdt.c | 269 ++++++++++++++++++++++++++ tools/kvm/arm/gic.c | 86 ++++++++ tools/kvm/arm/include/arm-common/gic.h | 29 +++ tools/kvm/arm/include/arm-common/kvm-arch.h | 31 +++ tools/kvm/arm/include/arm-common/smp.h | 7 + tools/kvm/arm/include/kvm/kvm-cpu-arch.h | 47 +++++ tools/kvm/arm/ioport.c | 5 + tools/kvm/arm/irq.c | 25 +++ tools/kvm/arm/kvm-cpu.c | 112 +++++++++++ tools/kvm/arm/kvm.c | 69 +++++++ tools/kvm/arm/smp.c | 13 ++ 17 files changed, 973 insertions(+), 1 deletions(-) create mode 100644 tools/kvm/arm/aarch32/cortex-a15.c create mode 100644 tools/kvm/arm/aarch32/include/kvm/barrier.h create mode 100644 tools/kvm/arm/aarch32/include/kvm/kvm-arch.h create mode 100644 tools/kvm/arm/aarch32/kvm-cpu.c create mode 100644 tools/kvm/arm/aarch32/smp-pen.S create mode 100644 tools/kvm/arm/fdt.c create mode 100644 tools/kvm/arm/gic.c create mode 100644 tools/kvm/arm/include/arm-common/gic.h create mode 100644 tools/kvm/arm/include/arm-common/kvm-arch.h create mode 100644 tools/kvm/arm/include/arm-common/smp.h create mode 100644 tools/kvm/arm/include/kvm/kvm-cpu-arch.h create mode 100644 tools/kvm/arm/ioport.c create mode 100644 tools/kvm/arm/irq.c create mode 100644 tools/kvm/arm/kvm-cpu.c create mode 100644 tools/kvm/arm/kvm.c create mode 100644 tools/kvm/arm/smp.c diff --git a/tools/kvm/Makefile b/tools/kvm/Makefile index 5da416f..2c9c19e 100644 --- a/tools/kvm/Makefile +++ b/tools/kvm/Makefile @@ -102,7 +102,8 @@ OBJS += builtin-sandbox.o OBJS += virtio/mmio.o # Translate uname -m into ARCH string -ARCH ?= $(shell uname -m | sed -e s/i.86/i386/ -e s/ppc.*/powerpc/) +ARCH ?= $(shell uname -m | sed -e s/i.86/i386/ -e s/ppc.*/powerpc/ \ + -e s/armv7.*/arm/) ifeq ($(ARCH),i386) ARCH := x86 @@ -157,6 +158,23 @@ ifeq ($(ARCH), powerpc) CFLAGS += -m64 endif +# ARM +OBJS_ARM_COMMON := arm/fdt.o arm/gic.o arm/ioport.o arm/irq.o \ + arm/kvm.o arm/kvm-cpu.o arm/smp.o +HDRS_ARM_COMMON := arm/include +ifeq ($(ARCH), arm) + DEFINES += -DCONFIG_ARM + OBJS += $(OBJS_ARM_COMMON) + OBJS += arm/aarch32/cortex-a15.o + OBJS += arm/aarch32/kvm-cpu.o + OTHEROBJS += arm/aarch32/smp-pen.o + ARCH_INCLUDE := $(HDRS_ARM_COMMON) + ARCH_INCLUDE += -Iarm/aarch32/include + CFLAGS += -march=armv7-a + CFLAGS += -I../../scripts/dtc/libfdt + OTHEROBJS += $(LIBFDT_OBJS) +endif + ### ifeq (,$(ARCH_INCLUDE)) diff --git a/tools/kvm/arm/aarch32/cortex-a15.c b/tools/kvm/arm/aarch32/cortex-a15.c new file mode 100644 index 0000000..eac0bb9 --- /dev/null +++ b/tools/kvm/arm/aarch32/cortex-a15.c @@ -0,0 +1,98 @@ +#include "kvm/fdt.h" +#include "kvm/kvm.h" +#include "kvm/kvm-cpu.h" +#include "kvm/util.h" + +#include "arm-common/gic.h" + +#include +#include + +#define CPU_NAME_MAX_LEN 8 +static void generate_cpu_nodes(void *fdt, struct kvm *kvm) +{ + int cpu; + + _FDT(fdt_begin_node(fdt, "cpus")); + _FDT(fdt_property_cell(fdt, "#address-cells", 0x1)); + _FDT(fdt_property_cell(fdt, "#size-cells", 0x0)); + + for (cpu = 0; cpu < kvm->nrcpus; ++cpu) { + char cpu_name[CPU_NAME_MAX_LEN]; + + if (kvm->cpus[cpu]->cpu_type != KVM_ARM_TARGET_CORTEX_A15) { + pr_warning("Ignoring unknown type for CPU %d\n", cpu); + continue; + } + + snprintf(cpu_name, CPU_NAME_MAX_LEN, "cpu@%d", cpu); + + _FDT(fdt_begin_node(fdt, cpu_name)); + _FDT(fdt_property_string(fdt, "device_type", "cpu")); + _FDT(fdt_property_string(fdt, "compatible", "arm,cortex-a15")); + + if (kvm->nrcpus > 1) { + _FDT(fdt_property_string(fdt, "enable-method", + "spin-table")); + _FDT(fdt_property_cell(fdt, "cpu-release-addr", + kvm->arch.smp_jump_guest_start)); + } + + _FDT(fdt_property_cell(fdt, "reg", cpu)); + _FDT(fdt_end_node(fdt)); + } + + _FDT(fdt_end_node(fdt)); +} + +static void generate_timer_nodes(void *fdt, struct kvm *kvm) +{ + u32 cpu_mask = (((1 << kvm->nrcpus) - 1) << GIC_FDT_IRQ_PPI_CPU_SHIFT) \ + & GIC_FDT_IRQ_PPI_CPU_MASK; + u32 irq_prop[] = { + cpu_to_fdt32(GIC_FDT_IRQ_TYPE_PPI), + cpu_to_fdt32(13), + cpu_to_fdt32(cpu_mask | GIC_FDT_IRQ_FLAGS_EDGE_LO_HI), + + cpu_to_fdt32(GIC_FDT_IRQ_TYPE_PPI), + cpu_to_fdt32(14), + cpu_to_fdt32(cpu_mask | GIC_FDT_IRQ_FLAGS_EDGE_LO_HI), + + cpu_to_fdt32(GIC_FDT_IRQ_TYPE_PPI), + cpu_to_fdt32(11), + cpu_to_fdt32(cpu_mask | GIC_FDT_IRQ_FLAGS_EDGE_LO_HI), + + cpu_to_fdt32(GIC_FDT_IRQ_TYPE_PPI), + cpu_to_fdt32(10), + cpu_to_fdt32(cpu_mask | GIC_FDT_IRQ_FLAGS_EDGE_LO_HI), + }; + + _FDT(fdt_begin_node(fdt, "timer")); + _FDT(fdt_property_string(fdt, "compatible", "arm,armv7-timer")); + _FDT(fdt_property(fdt, "interrupts", irq_prop, sizeof(irq_prop))); + _FDT(fdt_end_node(fdt)); +} + +static void generate_fdt_nodes(void *fdt, struct kvm *kvm, u32 gic_phandle) +{ + generate_cpu_nodes(fdt, kvm); + gic__generate_fdt_nodes(fdt, gic_phandle); + generate_timer_nodes(fdt, kvm); +} + +static int cortex_a15__vcpu_init(struct kvm_cpu *vcpu) +{ + vcpu->generate_fdt_nodes = generate_fdt_nodes; + return 0; +} + +static struct kvm_arm_target target_cortex_a15 = { + .id = KVM_ARM_TARGET_CORTEX_A15, + .init = cortex_a15__vcpu_init, +}; + +static int cortex_a15__core_init(struct kvm *kvm) +{ + return kvm_cpu__register_kvm_arm_target(&target_cortex_a15); +} +core_init(cortex_a15__core_init); diff --git a/tools/kvm/arm/aarch32/include/kvm/barrier.h b/tools/kvm/arm/aarch32/include/kvm/barrier.h new file mode 100644 index 0000000..ee8e643 --- /dev/null +++ b/tools/kvm/arm/aarch32/include/kvm/barrier.h @@ -0,0 +1,10 @@ +#ifndef KVM__KVM_BARRIER_H +#define KVM__KVM_BARRIER_H + +#define dmb() asm volatile ("dmb" : : : "memory") + +#define mb() dmb() +#define rmb() dmb() +#define wmb() dmb() + +#endif diff --git a/tools/kvm/arm/aarch32/include/kvm/kvm-arch.h b/tools/kvm/arm/aarch32/include/kvm/kvm-arch.h new file mode 100644 index 0000000..eb51917 --- /dev/null +++ b/tools/kvm/arm/aarch32/include/kvm/kvm-arch.h @@ -0,0 +1,28 @@ +#ifndef KVM__KVM_ARCH_H +#define KVM__KVM_ARCH_H + +#define ARM_LOMAP_MMIO_AREA 0x00000000UL +#define ARM_LOMAP_AXI_AREA 0x40000000UL +#define ARM_LOMAP_MEMORY_AREA 0x80000000UL +#define ARM_LOMAP_MAX_MEMORY 0x7ffff000UL + +#define ARM_GIC_DIST_SIZE 0x1000 +#define ARM_GIC_DIST_BASE (ARM_LOMAP_AXI_AREA - ARM_GIC_DIST_SIZE) +#define ARM_GIC_CPUI_SIZE 0x2000 +#define ARM_GIC_CPUI_BASE (ARM_GIC_DIST_BASE - ARM_GIC_CPUI_SIZE) + +#define ARM_KERN_OFFSET 0x8000 + +#define ARM_SMP_PEN_SIZE PAGE_SIZE +#define ARM_VIRTIO_MMIO_SIZE (ARM_GIC_DIST_BASE - ARM_LOMAP_MMIO_AREA) +#define ARM_PCI_MMIO_SIZE (ARM_LOMAP_MEMORY_AREA - ARM_LOMAP_AXI_AREA) + +#define ARM_MEMORY_AREA ARM_LOMAP_MEMORY_AREA +#define ARM_MAX_MEMORY ARM_LOMAP_MAX_MEMORY + +#define KVM_PCI_MMIO_AREA ARM_LOMAP_AXI_AREA +#define KVM_VIRTIO_MMIO_AREA ARM_LOMAP_MMIO_AREA + +#include "arm-common/kvm-arch.h" + +#endif diff --git a/tools/kvm/arm/aarch32/kvm-cpu.c b/tools/kvm/arm/aarch32/kvm-cpu.c new file mode 100644 index 0000000..f00a2f1 --- /dev/null +++ b/tools/kvm/arm/aarch32/kvm-cpu.c @@ -0,0 +1,111 @@ +#include "kvm/kvm-cpu.h" +#include "kvm/kvm.h" + +#include + +#define ARM_CORE_REG(x) (KVM_REG_ARM | KVM_REG_SIZE_U32 | KVM_REG_ARM_CORE | \ + KVM_REG_ARM_CORE_REG(x)) + +void kvm_cpu__reset_vcpu(struct kvm_cpu *vcpu) +{ + struct kvm *kvm = vcpu->kvm; + struct kvm_one_reg reg; + u32 data; + + /* Who said future-proofing was a good idea? */ + reg.addr = (u64)(unsigned long)&data; + + /* cpsr = IRQs/FIQs masked */ + data = PSR_I_BIT | PSR_F_BIT | SVC_MODE; + reg.id = ARM_CORE_REG(usr_regs.ARM_cpsr); + if (ioctl(vcpu->vcpu_fd, KVM_SET_ONE_REG, ®) < 0) + die_perror("KVM_SET_ONE_REG failed (cpsr)"); + + if (vcpu->cpu_id == 0) { + /* r0 = 0 */ + data = 0; + reg.id = ARM_CORE_REG(usr_regs.ARM_r0); + if (ioctl(vcpu->vcpu_fd, KVM_SET_ONE_REG, ®) < 0) + die_perror("KVM_SET_ONE_REG failed (r0)"); + + /* r1 = machine type (-1) */ + data = -1; + reg.id = ARM_CORE_REG(usr_regs.ARM_r1); + if (ioctl(vcpu->vcpu_fd, KVM_SET_ONE_REG, ®) < 0) + die_perror("KVM_SET_ONE_REG failed (r1)"); + + /* r2 = physical address of the device tree blob */ + data = kvm->arch.dtb_guest_start; + reg.id = ARM_CORE_REG(usr_regs.ARM_r2); + if (ioctl(vcpu->vcpu_fd, KVM_SET_ONE_REG, ®) < 0) + die_perror("KVM_SET_ONE_REG failed (r2)"); + + /* pc = start of kernel image */ + data = kvm->arch.kern_guest_start; + reg.id = ARM_CORE_REG(usr_regs.ARM_pc); + if (ioctl(vcpu->vcpu_fd, KVM_SET_ONE_REG, ®) < 0) + die_perror("KVM_SET_ONE_REG failed (pc)"); + + } else { + /* Simply enter the pen */ + data = kvm->arch.smp_pen_guest_start; + reg.id = ARM_CORE_REG(usr_regs.ARM_pc); + if (ioctl(vcpu->vcpu_fd, KVM_SET_ONE_REG, ®) < 0) + die_perror("KVM_SET_ONE_REG failed (SMP pc)"); + } +} + +void kvm_cpu__show_code(struct kvm_cpu *vcpu) +{ + struct kvm_one_reg reg; + u32 data; + + reg.addr = (u64)(unsigned long)&data; + + printf("*pc:\n"); + reg.id = ARM_CORE_REG(usr_regs.ARM_pc); + if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, ®) < 0) + die("KVM_GET_ONE_REG failed (show_code @ PC)"); + + kvm__dump_mem(vcpu->kvm, data, 32); + printf("\n"); + + printf("*lr (svc):\n"); + reg.id = ARM_CORE_REG(svc_regs[1]); + if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, ®) < 0) + die("KVM_GET_ONE_REG failed (show_code @ LR_svc)"); + data &= ~0x1; + + kvm__dump_mem(vcpu->kvm, data, 32); + printf("\n"); +} + +void kvm_cpu__show_registers(struct kvm_cpu *vcpu) +{ + struct kvm_one_reg reg; + u32 data; + int debug_fd = kvm_cpu__get_debug_fd(); + + reg.addr = (u64)(unsigned long)&data; + dprintf(debug_fd, "\n Registers:\n"); + + reg.id = ARM_CORE_REG(usr_regs.ARM_pc); + if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, ®) < 0) + die("KVM_GET_ONE_REG failed (pc)"); + dprintf(debug_fd, " PC: 0x%x\n", data); + + reg.id = ARM_CORE_REG(usr_regs.ARM_cpsr); + if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, ®) < 0) + die("KVM_GET_ONE_REG failed (cpsr)"); + dprintf(debug_fd, " CPSR: 0x%x\n", data); + + reg.id = ARM_CORE_REG(svc_regs[0]); + if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, ®) < 0) + die("KVM_GET_ONE_REG failed (SP_svc)"); + dprintf(debug_fd, " SP_svc: 0x%x\n", data); + + reg.id = ARM_CORE_REG(svc_regs[1]); + if (ioctl(vcpu->vcpu_fd, KVM_GET_ONE_REG, ®) < 0) + die("KVM_GET_ONE_REG failed (LR_svc)"); + dprintf(debug_fd, " LR_svc: 0x%x\n", data); +} diff --git a/tools/kvm/arm/aarch32/smp-pen.S b/tools/kvm/arm/aarch32/smp-pen.S new file mode 100644 index 0000000..5e25753 --- /dev/null +++ b/tools/kvm/arm/aarch32/smp-pen.S @@ -0,0 +1,14 @@ + .arm + .globl smp_pen_start + .globl smp_pen_end + .globl smp_jump_addr + .align +smp_pen_start: + adr r0, smp_jump_addr + wfi + ldr r1, [r0] + mov pc, r1 + .align +smp_jump_addr: + .long 0xdeadc0de +smp_pen_end: diff --git a/tools/kvm/arm/fdt.c b/tools/kvm/arm/fdt.c new file mode 100644 index 0000000..97b92e0 --- /dev/null +++ b/tools/kvm/arm/fdt.c @@ -0,0 +1,269 @@ +#include "kvm/devices.h" +#include "kvm/fdt.h" +#include "kvm/kvm.h" +#include "kvm/kvm-cpu.h" +#include "kvm/virtio-mmio.h" + +#include "arm-common/gic.h" + +#include + +#include +#include +#include +#include + +#define DEBUG 0 +#define DEBUG_FDT_DUMP_FILE "/tmp/kvmtool.dtb" + +static char kern_cmdline[COMMAND_LINE_SIZE]; + +bool kvm__load_firmware(struct kvm *kvm, const char *firmware_filename) +{ + return false; +} + +int kvm__arch_setup_firmware(struct kvm *kvm) +{ + return 0; +} + +#if DEBUG +static void dump_fdt(void *fdt) +{ + int count, fd; + + fd = open(DEBUG_FDT_DUMP_FILE, O_CREAT | O_TRUNC | O_RDWR, 0666); + if (fd < 0) + die("Failed to write dtb to %s", DEBUG_FDT_DUMP_FILE); + + count = write(fd, fdt, FDT_MAX_SIZE); + if (count < 0) + die_perror("Failed to dump dtb"); + + pr_info("Wrote %d bytes to dtb %s\n", count, DEBUG_FDT_DUMP_FILE); + close(fd); +} +#else +static void dump_fdt(void *fdt) { } +#endif + +#define DEVICE_NAME_MAX_LEN 32 +static void generate_virtio_mmio_node(void *fdt, struct virtio_mmio *vmmio) +{ + char dev_name[DEVICE_NAME_MAX_LEN]; + u64 addr = vmmio->addr; + u64 reg_prop[] = { + cpu_to_fdt64(addr), + cpu_to_fdt64(VIRTIO_MMIO_IO_SIZE) + }; + u32 irq_prop[] = { + cpu_to_fdt32(GIC_FDT_IRQ_TYPE_SPI), + cpu_to_fdt32(vmmio->irq - GIC_SPI_IRQ_BASE), + cpu_to_fdt32(GIC_FDT_IRQ_FLAGS_EDGE_LO_HI), + }; + + snprintf(dev_name, DEVICE_NAME_MAX_LEN, "virtio@%llx", addr); + + _FDT(fdt_begin_node(fdt, dev_name)); + _FDT(fdt_property_string(fdt, "compatible", "virtio,mmio")); + _FDT(fdt_property(fdt, "reg", reg_prop, sizeof(reg_prop))); + _FDT(fdt_property(fdt, "interrupts", irq_prop, sizeof(irq_prop))); + _FDT(fdt_end_node(fdt)); +} + +static int setup_fdt(struct kvm *kvm) +{ + int devid; + u8 staging_fdt[FDT_MAX_SIZE]; + u32 gic_phandle = fdt__alloc_phandle(); + u64 mem_reg_prop[] = { + cpu_to_fdt64(kvm->arch.memory_guest_start), + cpu_to_fdt64(kvm->ram_size), + }; + void *fdt = staging_fdt; + void *fdt_dest = guest_flat_to_host(kvm, + kvm->arch.dtb_guest_start); + void (*generate_cpu_nodes)(void *, struct kvm *, u32) + = kvm->cpus[0]->generate_fdt_nodes; + + /* Create new tree without a reserve map */ + _FDT(fdt_create(fdt, FDT_MAX_SIZE)); + if (kvm->nrcpus > 1) + _FDT(fdt_add_reservemap_entry(fdt, + kvm->arch.smp_pen_guest_start, + ARM_SMP_PEN_SIZE)); + _FDT(fdt_finish_reservemap(fdt)); + + /* Header */ + _FDT(fdt_begin_node(fdt, "")); + _FDT(fdt_property_cell(fdt, "interrupt-parent", gic_phandle)); + _FDT(fdt_property_string(fdt, "compatible", "linux,dummy-virt")); + _FDT(fdt_property_cell(fdt, "#address-cells", 0x2)); + _FDT(fdt_property_cell(fdt, "#size-cells", 0x2)); + + /* /chosen */ + _FDT(fdt_begin_node(fdt, "chosen")); + _FDT(fdt_property_string(fdt, "bootargs", kern_cmdline)); + + /* Initrd */ + if (kvm->arch.initrd_size != 0) { + u32 ird_st_prop = cpu_to_fdt64(kvm->arch.initrd_guest_start); + u32 ird_end_prop = cpu_to_fdt64(kvm->arch.initrd_guest_start + + kvm->arch.initrd_size); + + _FDT(fdt_property(fdt, "linux,initrd-start", + &ird_st_prop, sizeof(ird_st_prop))); + _FDT(fdt_property(fdt, "linux,initrd-end", + &ird_end_prop, sizeof(ird_end_prop))); + } + _FDT(fdt_end_node(fdt)); + + /* Memory */ + _FDT(fdt_begin_node(fdt, "memory")); + _FDT(fdt_property_string(fdt, "device_type", "memory")); + _FDT(fdt_property(fdt, "reg", mem_reg_prop, sizeof(mem_reg_prop))); + _FDT(fdt_end_node(fdt)); + + /* CPU and peripherals (interrupt controller, timers, etc) */ + if (generate_cpu_nodes) + generate_cpu_nodes(fdt, kvm, gic_phandle); + + /* Virtio MMIO devices */ + for (devid = 0; devid < KVM_MAX_DEVICES; ++devid) { + struct device_header *dev_hdr = device__find_dev(devid); + + if (!dev_hdr || dev_hdr->bus_type != DEVICE_BUS_MMIO) + continue; + + generate_virtio_mmio_node(fdt, dev_hdr->data); + } + + /* Finalise. */ + _FDT(fdt_end_node(fdt)); + _FDT(fdt_finish(fdt)); + + _FDT(fdt_open_into(fdt, fdt_dest, FDT_MAX_SIZE)); + _FDT(fdt_pack(fdt_dest)); + + dump_fdt(fdt_dest); + return 0; +} +firmware_init(setup_fdt); + +static int read_image(int fd, void **pos, void *limit) +{ + int count; + + while (((count = read(fd, *pos, SZ_64K)) > 0) && *pos <= limit) + *pos += count; + + if (pos < 0) + die_perror("read"); + + return *pos < limit ? 0 : -ENOMEM; +} + +#define FDT_ALIGN SZ_2M +#define INITRD_ALIGN 4 +#define SMP_PEN_ALIGN PAGE_SIZE +int load_flat_binary(struct kvm *kvm, int fd_kernel, int fd_initrd, + const char *kernel_cmdline) +{ + void *pos, *kernel_end, *limit; + unsigned long guest_addr; + + if (lseek(fd_kernel, 0, SEEK_SET) < 0) + die_perror("lseek"); + + /* + * Linux requires the initrd, pen and dtb to be mapped inside + * lowmem, so we can't just place them at the top of memory. + */ + limit = kvm->ram_start + min(kvm->ram_size, (u64)SZ_256M) - 1; + + pos = kvm->ram_start + ARM_KERN_OFFSET; + kvm->arch.kern_guest_start = host_to_guest_flat(kvm, pos); + if (read_image(fd_kernel, &pos, limit) == -ENOMEM) + die("kernel image too big to contain in guest memory."); + + kernel_end = pos; + pr_info("Loaded kernel to 0x%lx (%ld bytes)", + kvm->arch.kern_guest_start, + host_to_guest_flat(kvm, pos) - kvm->arch.kern_guest_start); + + /* + * Now load backwards from the end of memory so the kernel + * decompressor has plenty of space to work with. First up is + * the SMP pen if we have more than one virtual CPU... + */ + pos = limit; + if (kvm->cfg.nrcpus > 1) { + pos -= (ARM_SMP_PEN_SIZE + SMP_PEN_ALIGN); + guest_addr = ALIGN(host_to_guest_flat(kvm, pos), SMP_PEN_ALIGN); + pos = guest_flat_to_host(kvm, guest_addr); + if (pos < kernel_end) + die("SMP pen overlaps with kernel image."); + + kvm->arch.smp_pen_guest_start = guest_addr; + pr_info("Placing SMP pen at 0x%lx - 0x%lx", + kvm->arch.smp_pen_guest_start, + host_to_guest_flat(kvm, limit)); + limit = pos; + } + + /* ...now the device tree blob... */ + pos -= (FDT_MAX_SIZE + FDT_ALIGN); + guest_addr = ALIGN(host_to_guest_flat(kvm, pos), FDT_ALIGN); + pos = guest_flat_to_host(kvm, guest_addr); + if (pos < kernel_end) + die("fdt overlaps with kernel image."); + + kvm->arch.dtb_guest_start = guest_addr; + pr_info("Placing fdt at 0x%lx - 0x%lx", + kvm->arch.dtb_guest_start, + host_to_guest_flat(kvm, limit)); + limit = pos; + + /* ... and finally the initrd, if we have one. */ + if (fd_initrd != -1) { + struct stat sb; + unsigned long initrd_start; + + if (lseek(fd_initrd, 0, SEEK_SET) < 0) + die_perror("lseek"); + + if (fstat(fd_initrd, &sb)) + die_perror("fstat"); + + pos -= (sb.st_size + INITRD_ALIGN); + guest_addr = ALIGN(host_to_guest_flat(kvm, pos), INITRD_ALIGN); + pos = guest_flat_to_host(kvm, guest_addr); + if (pos < kernel_end) + die("initrd overlaps with kernel image."); + + initrd_start = guest_addr; + if (read_image(fd_initrd, &pos, limit) == -ENOMEM) + die("initrd too big to contain in guest memory."); + + kvm->arch.initrd_guest_start = initrd_start; + kvm->arch.initrd_size = host_to_guest_flat(kvm, pos) - initrd_start; + pr_info("Loaded initrd to 0x%lx (%ld bytes)", + kvm->arch.initrd_guest_start, + kvm->arch.initrd_size); + } else { + kvm->arch.initrd_size = 0; + } + + strncpy(kern_cmdline, kernel_cmdline, COMMAND_LINE_SIZE); + kern_cmdline[COMMAND_LINE_SIZE - 1] = '\0'; + + return true; +} + +bool load_bzimage(struct kvm *kvm, int fd_kernel, + int fd_initrd, const char *kernel_cmdline, u16 vidmode) +{ + /* To b or not to b? That is the zImage. */ + return false; +} diff --git a/tools/kvm/arm/gic.c b/tools/kvm/arm/gic.c new file mode 100644 index 0000000..f5fa7c7 --- /dev/null +++ b/tools/kvm/arm/gic.c @@ -0,0 +1,86 @@ +#include "kvm/fdt.h" +#include "kvm/kvm.h" +#include "kvm/virtio.h" + +#include "arm-common/gic.h" + +#include +#include + +static int irq_ids; + +int gic__alloc_irqnum(void) +{ + int irq = GIC_SPI_IRQ_BASE + irq_ids++; + + if (irq > KVM_MAX_IRQ) + die("IRQ limit %d reached!", KVM_MAX_IRQ); + + return irq; +} + +int gic__init_irqchip(struct kvm *kvm) +{ + int err; + struct kvm_device_address gic_addr[] = { + [0] = { + .id = (KVM_ARM_DEVICE_VGIC_V2 << KVM_DEVICE_ID_SHIFT) |\ + KVM_VGIC_V2_ADDR_TYPE_DIST, + .addr = ARM_GIC_DIST_BASE, + }, + [1] = { + .id = (KVM_ARM_DEVICE_VGIC_V2 << KVM_DEVICE_ID_SHIFT) |\ + KVM_VGIC_V2_ADDR_TYPE_CPU, + .addr = ARM_GIC_CPUI_BASE, + } + }; + + err = ioctl(kvm->vm_fd, KVM_CREATE_IRQCHIP); + if (err) + return err; + + err = ioctl(kvm->vm_fd, KVM_SET_DEVICE_ADDRESS, &gic_addr[0]); + if (err) + return err; + + err = ioctl(kvm->vm_fd, KVM_SET_DEVICE_ADDRESS, &gic_addr[1]); + return err; +} + +void gic__generate_fdt_nodes(void *fdt, u32 phandle) +{ + u64 reg_prop[] = { + cpu_to_fdt64(ARM_GIC_DIST_BASE), cpu_to_fdt64(ARM_GIC_DIST_SIZE), + cpu_to_fdt64(ARM_GIC_CPUI_BASE), cpu_to_fdt64(ARM_GIC_CPUI_SIZE), + }; + + _FDT(fdt_begin_node(fdt, "intc")); + _FDT(fdt_property_string(fdt, "compatible", "arm,cortex-a15-gic")); + _FDT(fdt_property_cell(fdt, "#interrupt-cells", GIC_FDT_IRQ_NUM_CELLS)); + _FDT(fdt_property(fdt, "interrupt-controller", NULL, 0)); + _FDT(fdt_property(fdt, "reg", reg_prop, sizeof(reg_prop))); + _FDT(fdt_property_cell(fdt, "phandle", phandle)); + _FDT(fdt_end_node(fdt)); +} + +#define KVM_IRQCHIP_IRQ(x) (KVM_ARM_IRQ_TYPE_SPI << KVM_ARM_IRQ_TYPE_SHIFT) |\ + ((x) & KVM_ARM_IRQ_NUM_MASK) + +void kvm__irq_line(struct kvm *kvm, int irq, int level) +{ + struct kvm_irq_level irq_level = { + .irq = KVM_IRQCHIP_IRQ(irq), + .level = !!level, + }; + + if (irq < GIC_SPI_IRQ_BASE || irq > KVM_MAX_IRQ) + pr_warning("Ignoring invalid GIC IRQ %d", irq); + else if (ioctl(kvm->vm_fd, KVM_IRQ_LINE, &irq_level) < 0) + pr_warning("Could not KVM_IRQ_LINE for irq %d", irq); +} + +void kvm__irq_trigger(struct kvm *kvm, int irq) +{ + kvm__irq_line(kvm, irq, VIRTIO_IRQ_HIGH); + kvm__irq_line(kvm, irq, VIRTIO_IRQ_LOW); +} diff --git a/tools/kvm/arm/include/arm-common/gic.h b/tools/kvm/arm/include/arm-common/gic.h new file mode 100644 index 0000000..4d85dd9 --- /dev/null +++ b/tools/kvm/arm/include/arm-common/gic.h @@ -0,0 +1,29 @@ +#ifndef ARM_COMMON__GIC_H +#define ARM_COMMON__GIC_H + +#define GIC_SGI_IRQ_BASE 0 +#define GIC_PPI_IRQ_BASE 16 +#define GIC_SPI_IRQ_BASE 32 + +#define GIC_FDT_IRQ_NUM_CELLS 3 + +#define GIC_FDT_IRQ_TYPE_SPI 0 +#define GIC_FDT_IRQ_TYPE_PPI 1 + +#define GIC_FDT_IRQ_FLAGS_EDGE_LO_HI 1 +#define GIC_FDT_IRQ_FLAGS_EDGE_HI_LO 2 +#define GIC_FDT_IRQ_FLAGS_LEVEL_HI 4 +#define GIC_FDT_IRQ_FLAGS_LEVEL_LO 8 + +#define GIC_FDT_IRQ_PPI_CPU_SHIFT 8 +#define GIC_FDT_IRQ_PPI_CPU_MASK (0xff << GIC_FDT_IRQ_PPI_CPU_SHIFT) + +#define KVM_MAX_IRQ 255 + +struct kvm; + +int gic__alloc_irqnum(void); +int gic__init_irqchip(struct kvm *kvm); +void gic__generate_fdt_nodes(void *fdt, u32 phandle); + +#endif /* ARM_COMMON__GIC_H */ diff --git a/tools/kvm/arm/include/arm-common/kvm-arch.h b/tools/kvm/arm/include/arm-common/kvm-arch.h new file mode 100644 index 0000000..0b20437 --- /dev/null +++ b/tools/kvm/arm/include/arm-common/kvm-arch.h @@ -0,0 +1,31 @@ +#ifndef ARM_COMMON__KVM_ARCH_H +#define ARM_COMMON__KVM_ARCH_H + +#include +#include + +#define VIRTIO_DEFAULT_TRANS VIRTIO_MMIO + +static inline bool arm_addr_in_virtio_mmio_region(u64 phys_addr) +{ + u64 limit = KVM_VIRTIO_MMIO_AREA + ARM_VIRTIO_MMIO_SIZE; + return phys_addr >= KVM_VIRTIO_MMIO_AREA && phys_addr < limit; +} + +static inline bool arm_addr_in_pci_mmio_region(u64 phys_addr) +{ + u64 limit = KVM_PCI_MMIO_AREA + ARM_PCI_MMIO_SIZE; + return phys_addr >= KVM_PCI_MMIO_AREA && phys_addr < limit; +} + +struct kvm_arch { + unsigned long memory_guest_start; + unsigned long kern_guest_start; + unsigned long initrd_guest_start; + unsigned long initrd_size; + unsigned long dtb_guest_start; + unsigned long smp_pen_guest_start; + unsigned long smp_jump_guest_start; +}; + +#endif /* ARM_COMMON__KVM_ARCH_H */ diff --git a/tools/kvm/arm/include/arm-common/smp.h b/tools/kvm/arm/include/arm-common/smp.h new file mode 100644 index 0000000..993a3e6 --- /dev/null +++ b/tools/kvm/arm/include/arm-common/smp.h @@ -0,0 +1,7 @@ +#ifndef ARM_COMMON__SMP_H +#define ARM_COMMON__SMP_H + +extern u8 smp_pen_start, smp_pen_end, smp_jump_addr; +void smp_pen_init(struct kvm *kvm); + +#endif /* ARM_COMMON__SMP_H */ diff --git a/tools/kvm/arm/include/kvm/kvm-cpu-arch.h b/tools/kvm/arm/include/kvm/kvm-cpu-arch.h new file mode 100644 index 0000000..d4618e9 --- /dev/null +++ b/tools/kvm/arm/include/kvm/kvm-cpu-arch.h @@ -0,0 +1,47 @@ +#ifndef ARM_COMMON__KVM_CPU_ARCH_H +#define ARM_COMMON__KVM_CPU_ARCH_H + +#include +#include +#include + +struct kvm; + +struct kvm_cpu { + pthread_t thread; + + unsigned long cpu_id; + unsigned long cpu_type; + + struct kvm *kvm; + int vcpu_fd; + struct kvm_run *kvm_run; + + u8 is_running; + u8 paused; + u8 needs_nmi; + + struct kvm_coalesced_mmio_ring *ring; + + void (*generate_fdt_nodes)(void *fdt, struct kvm* kvm, + u32 gic_phandle); +}; + +struct kvm_arm_target { + u32 id; + int (*init)(struct kvm_cpu *vcpu); +}; + +int kvm_cpu__register_kvm_arm_target(struct kvm_arm_target *target); + +static inline bool kvm_cpu__emulate_io(struct kvm *kvm, u16 port, void *data, + int direction, int size, u32 count) +{ + return false; +} + +bool kvm_cpu__emulate_mmio(struct kvm *kvm, u64 phys_addr, u8 *data, u32 len, + u8 is_write); + +#endif /* ARM_COMMON__KVM_CPU_ARCH_H */ + diff --git a/tools/kvm/arm/ioport.c b/tools/kvm/arm/ioport.c new file mode 100644 index 0000000..3c03fa0 --- /dev/null +++ b/tools/kvm/arm/ioport.c @@ -0,0 +1,5 @@ +#include "kvm/ioport.h" + +void ioport__setup_arch(struct kvm *kvm) +{ +} diff --git a/tools/kvm/arm/irq.c b/tools/kvm/arm/irq.c new file mode 100644 index 0000000..a17c722 --- /dev/null +++ b/tools/kvm/arm/irq.c @@ -0,0 +1,25 @@ +#include "kvm/devices.h" +#include "kvm/irq.h" +#include "kvm/kvm.h" +#include "kvm/util.h" + +#include "arm-common/gic.h" + +static int device_ids; + +int irq__register_device(u32 dev, u8 *num, u8 *pin, u8 *line) +{ + if (device_ids >= KVM_MAX_DEVICES) + die("Device limit %d reached!", KVM_MAX_DEVICES); + + *num = device_ids++; + *line = gic__alloc_irqnum(); + + return 0; +} + +int irq__add_msix_route(struct kvm *kvm, struct msi_msg *msg) +{ + die(__FUNCTION__); + return 0; +} diff --git a/tools/kvm/arm/kvm-cpu.c b/tools/kvm/arm/kvm-cpu.c new file mode 100644 index 0000000..f659118 --- /dev/null +++ b/tools/kvm/arm/kvm-cpu.c @@ -0,0 +1,112 @@ +#include "kvm/kvm.h" +#include "kvm/kvm-cpu.h" + +#include "arm-common/smp.h" + +static int debug_fd; + +void kvm_cpu__set_debug_fd(int fd) +{ + debug_fd = fd; +} + +int kvm_cpu__get_debug_fd(void) +{ + return debug_fd; +} + +static struct kvm_arm_target *kvm_arm_targets[KVM_ARM_NUM_TARGETS]; +int kvm_cpu__register_kvm_arm_target(struct kvm_arm_target *target) +{ + unsigned int i = 0; + + for (i = 0; i < ARRAY_SIZE(kvm_arm_targets); ++i) { + if (!kvm_arm_targets[i]) { + kvm_arm_targets[i] = target; + return 0; + } + } + + return -ENOSPC; +} + +struct kvm_cpu *kvm_cpu__arch_init(struct kvm *kvm, unsigned long cpu_id) +{ + struct kvm_cpu *vcpu; + int coalesced_offset, mmap_size, err = -1; + unsigned int i; + struct kvm_vcpu_init vcpu_init = { }; + + vcpu = calloc(1, sizeof(struct kvm_cpu)); + if (!vcpu) + return NULL; + + vcpu->vcpu_fd = ioctl(kvm->vm_fd, KVM_CREATE_VCPU, cpu_id); + if (vcpu->vcpu_fd < 0) + die_perror("KVM_CREATE_VCPU ioctl"); + + mmap_size = ioctl(kvm->sys_fd, KVM_GET_VCPU_MMAP_SIZE, 0); + if (mmap_size < 0) + die_perror("KVM_GET_VCPU_MMAP_SIZE ioctl"); + + vcpu->kvm_run = mmap(NULL, mmap_size, PROT_RW, MAP_SHARED, + vcpu->vcpu_fd, 0); + if (vcpu->kvm_run == MAP_FAILED) + die("unable to mmap vcpu fd"); + + /* Find an appropriate target CPU type. */ + for (i = 0; i < ARRAY_SIZE(kvm_arm_targets); ++i) { + vcpu_init.target = kvm_arm_targets[i]->id; + err = ioctl(vcpu->vcpu_fd, KVM_ARM_VCPU_INIT, &vcpu_init); + if (!err) + break; + } + + if (err || kvm_arm_targets[i]->init(vcpu)) + die("Unable to initialise ARM vcpu"); + + coalesced_offset = ioctl(kvm->sys_fd, KVM_CHECK_EXTENSION, + KVM_CAP_COALESCED_MMIO); + if (coalesced_offset) + vcpu->ring = (void *)vcpu->kvm_run + + (coalesced_offset * PAGE_SIZE); + + if (cpu_id != 0) + smp_pen_init(kvm); + + /* Populate the vcpu structure. */ + vcpu->kvm = kvm; + vcpu->cpu_id = cpu_id; + vcpu->cpu_type = vcpu_init.target; + vcpu->is_running = true; + return vcpu; +} + +void kvm_cpu__arch_nmi(struct kvm_cpu *cpu) +{ +} + +void kvm_cpu__delete(struct kvm_cpu *vcpu) +{ + free(vcpu); +} + +bool kvm_cpu__handle_exit(struct kvm_cpu *vcpu) +{ + return false; +} + +bool kvm_cpu__emulate_mmio(struct kvm *kvm, u64 phys_addr, u8 *data, u32 len, + u8 is_write) +{ + if (arm_addr_in_virtio_mmio_region(phys_addr)) + return kvm__emulate_mmio(kvm, phys_addr, data, len, is_write); + else if (arm_addr_in_pci_mmio_region(phys_addr)) + die("PCI emulation not supported on ARM!"); + + return false; +} + +void kvm_cpu__show_page_tables(struct kvm_cpu *vcpu) +{ +} diff --git a/tools/kvm/arm/kvm.c b/tools/kvm/arm/kvm.c new file mode 100644 index 0000000..aa2673a --- /dev/null +++ b/tools/kvm/arm/kvm.c @@ -0,0 +1,69 @@ +#include "kvm/kvm.h" +#include "kvm/term.h" +#include "kvm/util.h" +#include "kvm/virtio-console.h" + +#include "arm-common/gic.h" + +#include +#include + +struct kvm_ext kvm_req_ext[] = { + { DEFINE_KVM_EXT(KVM_CAP_IRQCHIP) }, + { DEFINE_KVM_EXT(KVM_CAP_ONE_REG) }, + { 0, 0 }, +}; + +bool kvm__arch_cpu_supports_vm(void) +{ + /* The KVM capability check is enough. */ + return true; +} + +void kvm__init_ram(struct kvm *kvm) +{ + int err; + u64 phys_start, phys_size; + void *host_mem; + + phys_start = ARM_MEMORY_AREA; + phys_size = kvm->ram_size; + host_mem = kvm->ram_start; + + err = kvm__register_mem(kvm, phys_start, phys_size, host_mem); + if (err) + die("Failed to register %lld bytes of memory at physical " + "address 0x%llx [err %d]", phys_size, phys_start, err); + + kvm->arch.memory_guest_start = phys_start; +} + +void kvm__arch_delete_ram(struct kvm *kvm) +{ + munmap(kvm->ram_start, kvm->ram_size); +} + +void kvm__arch_periodic_poll(struct kvm *kvm) +{ + if (term_readable(0)) + virtio_console__inject_interrupt(kvm); +} + +void kvm__arch_set_cmdline(char *cmdline, bool video) +{ +} + +void kvm__arch_init(struct kvm *kvm, const char *hugetlbfs_path, u64 ram_size) +{ + /* Allocate guest memory. */ + kvm->ram_size = min(ram_size, (u64)ARM_MAX_MEMORY); + kvm->ram_start = mmap_anon_or_hugetlbfs(kvm, NULL, kvm->ram_size); + if (kvm->ram_start == MAP_FAILED) + die("Failed to map %lld bytes for guest memory (%d)", + kvm->ram_size, errno); + madvise(kvm->ram_start, kvm->ram_size, MADV_MERGEABLE); + + /* Initialise the virtual GIC. */ + if (gic__init_irqchip(kvm)) + die("Failed to initialise virtual GIC"); +} diff --git a/tools/kvm/arm/smp.c b/tools/kvm/arm/smp.c new file mode 100644 index 0000000..73e58bd --- /dev/null +++ b/tools/kvm/arm/smp.c @@ -0,0 +1,13 @@ +#include "kvm/kvm.h" + +#include "arm-common/smp.h" + +void smp_pen_init(struct kvm *kvm) +{ + unsigned long pen_size = &smp_pen_end - &smp_pen_start; + unsigned long pen_start = kvm->arch.smp_pen_guest_start; + unsigned long jump_offset = &smp_jump_addr - &smp_pen_start; + + kvm->arch.smp_jump_guest_start = pen_start + jump_offset; + memcpy(guest_flat_to_host(kvm, pen_start), &smp_pen_start, pen_size); +}