From patchwork Thu Jul 27 10:24:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong-Xuan Wang X-Patchwork-Id: 13330014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DD921C001DC for ; Thu, 27 Jul 2023 11:42:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qOyAw-0003Ot-JL; Thu, 27 Jul 2023 06:24:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qOyAu-0003OY-6n for qemu-devel@nongnu.org; Thu, 27 Jul 2023 06:24:52 -0400 Received: from mail-ot1-x329.google.com ([2607:f8b0:4864:20::329]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qOyAr-0003sx-UR for qemu-devel@nongnu.org; Thu, 27 Jul 2023 06:24:51 -0400 Received: by mail-ot1-x329.google.com with SMTP id 46e09a7af769-6bb0cadd3ccso656822a34.3 for ; Thu, 27 Jul 2023 03:24:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1690453489; x=1691058289; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=6yy/ilJ4eRlk81CPTCEAUhyUjpFl7zB/SF7tY6vWp9Q=; b=ZvMzixzW84dQvuWiL5JHfHBog7fZy0l4GaUZUye7vaTLYRPeY2eFe4sLVa5TNUe+mc L/BC4RtrT8v6VVLrHrAF0IkPS+FG1ADidjQLyu/I3wTEsvZLUVzr7KeUWLmBBwJKf0Ea B0eA4uYdsruEWYwYbwWXTydoTxS0jbuTPHVnEucyMu34W0jZgZgd55JDHDCS82dcfYM8 VhBXfA8eX4ipbZy/A77q2qOq7susFIBqDQLvVZP9CYiTp5noyOgmHQr/4dsvBZcLYjW3 fJEJmIg2ais5W/u1LSc8CI128PPGwHry48ts9TWSPe755WuBwH+pEPqXp1Q+vQtn4WnL TVKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690453489; x=1691058289; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6yy/ilJ4eRlk81CPTCEAUhyUjpFl7zB/SF7tY6vWp9Q=; b=D7k+Euzb3xkIpR5Zz824ZmsMnN1hbK6AAh57V4uwpH0O7Eob4rWNCGxzla1yH+D0ur /71UyYhNOnCrqsy5Hk16LCEwVhMoTxPZ9rP+lXW86CGYyoIuXs6C96vNlMyTObyTVUsU TsC92O1kmzkBvwEr+Le/tGjdONOhGRgi6iT22ZNLPh7i69AY6/00Gp+laAgnx+8IOzXY /Y1ZqHTbH2aOydO+WBxH+oZ4U8Ru4nVpU1TjRFXNsUnDUSUpEUzhbDuV/i9OYDaP7lsC 27cCyRO30qutpVAlvtYdi4cjDxSbO8+L7IY1kR7z1S8MS92CRUr72IZpdvsHEwMEC8Ed uoeg== X-Gm-Message-State: ABy/qLZrt1dGw/UMcgY0iOGVBR2K8Zbz1P93e6h1NM+zpM8BMhnuIIof mfBfcsg7acWesdwZJMAAXm1lYl6ohIfLhb6lrmYi8S3B3vYtgntlUebtCJZqBHCzmgWHUZeE/u2 Nw5JExebvX4NSip+SXi2ttTiHZLEypq9Fx7wz9ZzkAtIHwozVURDh0j/y6lym54UHI8CfNcCghA sm11lg X-Google-Smtp-Source: APBJJlEItx4fGp2NhStoWwWHVczs+3bVjo6yZrBaH71XSwXLXYElZVB+DMliOQsc+NtoHfaYKBZH2A== X-Received: by 2002:a05:6808:138d:b0:3a4:3192:1627 with SMTP id c13-20020a056808138d00b003a431921627mr3005146oiw.42.1690453488502; Thu, 27 Jul 2023 03:24:48 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id u8-20020a17090341c800b001b882880550sm1230139ple.282.2023.07.27.03.24.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Jul 2023 03:24:48 -0700 (PDT) From: Yong-Xuan Wang To: qemu-devel@nongnu.org, qemu-riscv@nongnu.org Cc: rkanwal@rivosinc.com, anup@brainfault.org, dbarboza@ventanamicro.com, ajones@ventanamicro.com, atishp@atishpatra.org, vincent.chen@sifive.com, greentime.hu@sifive.com, frank.chang@sifive.com, jim.shu@sifive.com, Yong-Xuan Wang , Palmer Dabbelt , Alistair Francis , Bin Meng , Weiwei Li , Liu Zhiwei Subject: [PATCH v7 1/5] target/riscv: support the AIA device emulation with KVM enabled Date: Thu, 27 Jul 2023 10:24:33 +0000 Message-Id: <20230727102439.22554-2-yongxuan.wang@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230727102439.22554-1-yongxuan.wang@sifive.com> References: <20230727102439.22554-1-yongxuan.wang@sifive.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::329; envelope-from=yongxuan.wang@sifive.com; helo=mail-ot1-x329.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In this patch, we create the APLIC and IMSIC FDT helper functions and remove M mode AIA devices when using KVM acceleration. Signed-off-by: Yong-Xuan Wang Reviewed-by: Jim Shu Reviewed-by: Daniel Henrique Barboza Reviewed-by: Andrew Jones --- hw/riscv/virt.c | 290 +++++++++++++++++++++++------------------------- 1 file changed, 137 insertions(+), 153 deletions(-) diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index d90286dc46..f595380be1 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -516,79 +516,28 @@ static uint32_t imsic_num_bits(uint32_t count) return ret; } -static void create_fdt_imsic(RISCVVirtState *s, const MemMapEntry *memmap, - uint32_t *phandle, uint32_t *intc_phandles, - uint32_t *msi_m_phandle, uint32_t *msi_s_phandle) +static void create_fdt_one_imsic(RISCVVirtState *s, hwaddr base_addr, + uint32_t *intc_phandles, uint32_t msi_phandle, + bool m_mode, uint32_t imsic_guest_bits) { int cpu, socket; char *imsic_name; MachineState *ms = MACHINE(s); int socket_count = riscv_socket_count(ms); - uint32_t imsic_max_hart_per_socket, imsic_guest_bits; + uint32_t imsic_max_hart_per_socket; uint32_t *imsic_cells, *imsic_regs, imsic_addr, imsic_size; - *msi_m_phandle = (*phandle)++; - *msi_s_phandle = (*phandle)++; imsic_cells = g_new0(uint32_t, ms->smp.cpus * 2); imsic_regs = g_new0(uint32_t, socket_count * 4); - /* M-level IMSIC node */ for (cpu = 0; cpu < ms->smp.cpus; cpu++) { imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]); - imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT); + imsic_cells[cpu * 2 + 1] = cpu_to_be32(m_mode ? IRQ_M_EXT : IRQ_S_EXT); } - imsic_max_hart_per_socket = 0; - for (socket = 0; socket < socket_count; socket++) { - imsic_addr = memmap[VIRT_IMSIC_M].base + - socket * VIRT_IMSIC_GROUP_MAX_SIZE; - imsic_size = IMSIC_HART_SIZE(0) * s->soc[socket].num_harts; - imsic_regs[socket * 4 + 0] = 0; - imsic_regs[socket * 4 + 1] = cpu_to_be32(imsic_addr); - imsic_regs[socket * 4 + 2] = 0; - imsic_regs[socket * 4 + 3] = cpu_to_be32(imsic_size); - if (imsic_max_hart_per_socket < s->soc[socket].num_harts) { - imsic_max_hart_per_socket = s->soc[socket].num_harts; - } - } - imsic_name = g_strdup_printf("/soc/imsics@%lx", - (unsigned long)memmap[VIRT_IMSIC_M].base); - qemu_fdt_add_subnode(ms->fdt, imsic_name); - qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible", - "riscv,imsics"); - qemu_fdt_setprop_cell(ms->fdt, imsic_name, "#interrupt-cells", - FDT_IMSIC_INT_CELLS); - qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller", - NULL, 0); - qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller", - NULL, 0); - qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended", - imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2); - qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs, - socket_count * sizeof(uint32_t) * 4); - qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids", - VIRT_IRQCHIP_NUM_MSIS); - if (socket_count > 1) { - qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,hart-index-bits", - imsic_num_bits(imsic_max_hart_per_socket)); - qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-bits", - imsic_num_bits(socket_count)); - qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-shift", - IMSIC_MMIO_GROUP_MIN_SHIFT); - } - qemu_fdt_setprop_cell(ms->fdt, imsic_name, "phandle", *msi_m_phandle); - - g_free(imsic_name); - /* S-level IMSIC node */ - for (cpu = 0; cpu < ms->smp.cpus; cpu++) { - imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]); - imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT); - } - imsic_guest_bits = imsic_num_bits(s->aia_guests + 1); imsic_max_hart_per_socket = 0; for (socket = 0; socket < socket_count; socket++) { - imsic_addr = memmap[VIRT_IMSIC_S].base + - socket * VIRT_IMSIC_GROUP_MAX_SIZE; + imsic_addr = base_addr + socket * VIRT_IMSIC_GROUP_MAX_SIZE; imsic_size = IMSIC_HART_SIZE(imsic_guest_bits) * s->soc[socket].num_harts; imsic_regs[socket * 4 + 0] = 0; @@ -599,119 +548,151 @@ static void create_fdt_imsic(RISCVVirtState *s, const MemMapEntry *memmap, imsic_max_hart_per_socket = s->soc[socket].num_harts; } } - imsic_name = g_strdup_printf("/soc/imsics@%lx", - (unsigned long)memmap[VIRT_IMSIC_S].base); + + imsic_name = g_strdup_printf("/soc/imsics@%lx", (unsigned long)base_addr); qemu_fdt_add_subnode(ms->fdt, imsic_name); - qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible", - "riscv,imsics"); + qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible", "riscv,imsics"); qemu_fdt_setprop_cell(ms->fdt, imsic_name, "#interrupt-cells", - FDT_IMSIC_INT_CELLS); - qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller", - NULL, 0); - qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller", - NULL, 0); + FDT_IMSIC_INT_CELLS); + qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller", NULL, 0); + qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller", NULL, 0); qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended", - imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2); + imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2); qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs, - socket_count * sizeof(uint32_t) * 4); + socket_count * sizeof(uint32_t) * 4); qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids", - VIRT_IRQCHIP_NUM_MSIS); + VIRT_IRQCHIP_NUM_MSIS); + if (imsic_guest_bits) { qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,guest-index-bits", - imsic_guest_bits); + imsic_guest_bits); } + if (socket_count > 1) { qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,hart-index-bits", - imsic_num_bits(imsic_max_hart_per_socket)); + imsic_num_bits(imsic_max_hart_per_socket)); qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-bits", - imsic_num_bits(socket_count)); + imsic_num_bits(socket_count)); qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-shift", - IMSIC_MMIO_GROUP_MIN_SHIFT); + IMSIC_MMIO_GROUP_MIN_SHIFT); } - qemu_fdt_setprop_cell(ms->fdt, imsic_name, "phandle", *msi_s_phandle); - g_free(imsic_name); + qemu_fdt_setprop_cell(ms->fdt, imsic_name, "phandle", msi_phandle); + g_free(imsic_name); g_free(imsic_regs); g_free(imsic_cells); } -static void create_fdt_socket_aplic(RISCVVirtState *s, - const MemMapEntry *memmap, int socket, - uint32_t msi_m_phandle, - uint32_t msi_s_phandle, - uint32_t *phandle, - uint32_t *intc_phandles, - uint32_t *aplic_phandles) +static void create_fdt_imsic(RISCVVirtState *s, const MemMapEntry *memmap, + uint32_t *phandle, uint32_t *intc_phandles, + uint32_t *msi_m_phandle, uint32_t *msi_s_phandle) +{ + *msi_m_phandle = (*phandle)++; + *msi_s_phandle = (*phandle)++; + + if (!kvm_enabled()) { + /* M-level IMSIC node */ + create_fdt_one_imsic(s, memmap[VIRT_IMSIC_M].base, intc_phandles, + *msi_m_phandle, true, 0); + } + + /* S-level IMSIC node */ + create_fdt_one_imsic(s, memmap[VIRT_IMSIC_S].base, intc_phandles, + *msi_s_phandle, false, + imsic_num_bits(s->aia_guests + 1)); + +} + +static void create_fdt_one_aplic(RISCVVirtState *s, int socket, + unsigned long aplic_addr, uint32_t aplic_size, + uint32_t msi_phandle, + uint32_t *intc_phandles, + uint32_t aplic_phandle, + uint32_t aplic_child_phandle, + bool m_mode) { int cpu; char *aplic_name; uint32_t *aplic_cells; - unsigned long aplic_addr; MachineState *ms = MACHINE(s); - uint32_t aplic_m_phandle, aplic_s_phandle; - aplic_m_phandle = (*phandle)++; - aplic_s_phandle = (*phandle)++; aplic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 2); - /* M-level APLIC node */ for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) { aplic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]); - aplic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT); + aplic_cells[cpu * 2 + 1] = cpu_to_be32(m_mode ? IRQ_M_EXT : IRQ_S_EXT); } - aplic_addr = memmap[VIRT_APLIC_M].base + - (memmap[VIRT_APLIC_M].size * socket); + aplic_name = g_strdup_printf("/soc/aplic@%lx", aplic_addr); qemu_fdt_add_subnode(ms->fdt, aplic_name); qemu_fdt_setprop_string(ms->fdt, aplic_name, "compatible", "riscv,aplic"); qemu_fdt_setprop_cell(ms->fdt, aplic_name, - "#interrupt-cells", FDT_APLIC_INT_CELLS); + "#interrupt-cells", FDT_APLIC_INT_CELLS); qemu_fdt_setprop(ms->fdt, aplic_name, "interrupt-controller", NULL, 0); + if (s->aia_type == VIRT_AIA_TYPE_APLIC) { qemu_fdt_setprop(ms->fdt, aplic_name, "interrupts-extended", - aplic_cells, s->soc[socket].num_harts * sizeof(uint32_t) * 2); + aplic_cells, + s->soc[socket].num_harts * sizeof(uint32_t) * 2); } else { - qemu_fdt_setprop_cell(ms->fdt, aplic_name, "msi-parent", - msi_m_phandle); + qemu_fdt_setprop_cell(ms->fdt, aplic_name, "msi-parent", msi_phandle); } + qemu_fdt_setprop_cells(ms->fdt, aplic_name, "reg", - 0x0, aplic_addr, 0x0, memmap[VIRT_APLIC_M].size); + 0x0, aplic_addr, 0x0, aplic_size); qemu_fdt_setprop_cell(ms->fdt, aplic_name, "riscv,num-sources", - VIRT_IRQCHIP_NUM_SOURCES); - qemu_fdt_setprop_cell(ms->fdt, aplic_name, "riscv,children", - aplic_s_phandle); - qemu_fdt_setprop_cells(ms->fdt, aplic_name, "riscv,delegate", - aplic_s_phandle, 0x1, VIRT_IRQCHIP_NUM_SOURCES); + VIRT_IRQCHIP_NUM_SOURCES); + + if (aplic_child_phandle) { + qemu_fdt_setprop_cell(ms->fdt, aplic_name, "riscv,children", + aplic_child_phandle); + qemu_fdt_setprop_cells(ms->fdt, aplic_name, "riscv,delegate", + aplic_child_phandle, 0x1, + VIRT_IRQCHIP_NUM_SOURCES); + } + riscv_socket_fdt_write_id(ms, aplic_name, socket); - qemu_fdt_setprop_cell(ms->fdt, aplic_name, "phandle", aplic_m_phandle); + qemu_fdt_setprop_cell(ms->fdt, aplic_name, "phandle", aplic_phandle); + g_free(aplic_name); + g_free(aplic_cells); +} - /* S-level APLIC node */ - for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) { - aplic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]); - aplic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT); +static void create_fdt_socket_aplic(RISCVVirtState *s, + const MemMapEntry *memmap, int socket, + uint32_t msi_m_phandle, + uint32_t msi_s_phandle, + uint32_t *phandle, + uint32_t *intc_phandles, + uint32_t *aplic_phandles) +{ + char *aplic_name; + unsigned long aplic_addr; + MachineState *ms = MACHINE(s); + uint32_t aplic_m_phandle, aplic_s_phandle; + + aplic_m_phandle = (*phandle)++; + aplic_s_phandle = (*phandle)++; + + if (!kvm_enabled()) { + /* M-level APLIC node */ + aplic_addr = memmap[VIRT_APLIC_M].base + + (memmap[VIRT_APLIC_M].size * socket); + create_fdt_one_aplic(s, socket, aplic_addr, memmap[VIRT_APLIC_M].size, + msi_m_phandle, intc_phandles, + aplic_m_phandle, aplic_s_phandle, + true); } + + /* S-level APLIC node */ aplic_addr = memmap[VIRT_APLIC_S].base + (memmap[VIRT_APLIC_S].size * socket); + create_fdt_one_aplic(s, socket, aplic_addr, memmap[VIRT_APLIC_S].size, + msi_s_phandle, intc_phandles, + aplic_s_phandle, 0, + false); + aplic_name = g_strdup_printf("/soc/aplic@%lx", aplic_addr); - qemu_fdt_add_subnode(ms->fdt, aplic_name); - qemu_fdt_setprop_string(ms->fdt, aplic_name, "compatible", "riscv,aplic"); - qemu_fdt_setprop_cell(ms->fdt, aplic_name, - "#interrupt-cells", FDT_APLIC_INT_CELLS); - qemu_fdt_setprop(ms->fdt, aplic_name, "interrupt-controller", NULL, 0); - if (s->aia_type == VIRT_AIA_TYPE_APLIC) { - qemu_fdt_setprop(ms->fdt, aplic_name, "interrupts-extended", - aplic_cells, s->soc[socket].num_harts * sizeof(uint32_t) * 2); - } else { - qemu_fdt_setprop_cell(ms->fdt, aplic_name, "msi-parent", - msi_s_phandle); - } - qemu_fdt_setprop_cells(ms->fdt, aplic_name, "reg", - 0x0, aplic_addr, 0x0, memmap[VIRT_APLIC_S].size); - qemu_fdt_setprop_cell(ms->fdt, aplic_name, "riscv,num-sources", - VIRT_IRQCHIP_NUM_SOURCES); - riscv_socket_fdt_write_id(ms, aplic_name, socket); - qemu_fdt_setprop_cell(ms->fdt, aplic_name, "phandle", aplic_s_phandle); if (!socket) { platform_bus_add_all_fdt_nodes(ms->fdt, aplic_name, @@ -722,7 +703,6 @@ static void create_fdt_socket_aplic(RISCVVirtState *s, g_free(aplic_name); - g_free(aplic_cells); aplic_phandles[socket] = aplic_s_phandle; } @@ -1163,16 +1143,20 @@ static DeviceState *virt_create_aia(RISCVVirtAIAType aia_type, int aia_guests, int i; hwaddr addr; uint32_t guest_bits; - DeviceState *aplic_m; - bool msimode = (aia_type == VIRT_AIA_TYPE_APLIC_IMSIC) ? true : false; + DeviceState *aplic_s = NULL; + DeviceState *aplic_m = NULL; + bool msimode = aia_type == VIRT_AIA_TYPE_APLIC_IMSIC; if (msimode) { - /* Per-socket M-level IMSICs */ - addr = memmap[VIRT_IMSIC_M].base + socket * VIRT_IMSIC_GROUP_MAX_SIZE; - for (i = 0; i < hart_count; i++) { - riscv_imsic_create(addr + i * IMSIC_HART_SIZE(0), - base_hartid + i, true, 1, - VIRT_IRQCHIP_NUM_MSIS); + if (!kvm_enabled()) { + /* Per-socket M-level IMSICs */ + addr = memmap[VIRT_IMSIC_M].base + + socket * VIRT_IMSIC_GROUP_MAX_SIZE; + for (i = 0; i < hart_count; i++) { + riscv_imsic_create(addr + i * IMSIC_HART_SIZE(0), + base_hartid + i, true, 1, + VIRT_IRQCHIP_NUM_MSIS); + } } /* Per-socket S-level IMSICs */ @@ -1185,29 +1169,29 @@ static DeviceState *virt_create_aia(RISCVVirtAIAType aia_type, int aia_guests, } } - /* Per-socket M-level APLIC */ - aplic_m = riscv_aplic_create( - memmap[VIRT_APLIC_M].base + socket * memmap[VIRT_APLIC_M].size, - memmap[VIRT_APLIC_M].size, - (msimode) ? 0 : base_hartid, - (msimode) ? 0 : hart_count, - VIRT_IRQCHIP_NUM_SOURCES, - VIRT_IRQCHIP_NUM_PRIO_BITS, - msimode, true, NULL); - - if (aplic_m) { - /* Per-socket S-level APLIC */ - riscv_aplic_create( - memmap[VIRT_APLIC_S].base + socket * memmap[VIRT_APLIC_S].size, - memmap[VIRT_APLIC_S].size, - (msimode) ? 0 : base_hartid, - (msimode) ? 0 : hart_count, - VIRT_IRQCHIP_NUM_SOURCES, - VIRT_IRQCHIP_NUM_PRIO_BITS, - msimode, false, aplic_m); + if (!kvm_enabled()) { + /* Per-socket M-level APLIC */ + aplic_m = riscv_aplic_create(memmap[VIRT_APLIC_M].base + + socket * memmap[VIRT_APLIC_M].size, + memmap[VIRT_APLIC_M].size, + (msimode) ? 0 : base_hartid, + (msimode) ? 0 : hart_count, + VIRT_IRQCHIP_NUM_SOURCES, + VIRT_IRQCHIP_NUM_PRIO_BITS, + msimode, true, NULL); } - return aplic_m; + /* Per-socket S-level APLIC */ + aplic_s = riscv_aplic_create(memmap[VIRT_APLIC_S].base + + socket * memmap[VIRT_APLIC_S].size, + memmap[VIRT_APLIC_S].size, + (msimode) ? 0 : base_hartid, + (msimode) ? 0 : hart_count, + VIRT_IRQCHIP_NUM_SOURCES, + VIRT_IRQCHIP_NUM_PRIO_BITS, + msimode, false, aplic_m); + + return kvm_enabled() ? aplic_s : aplic_m; } static void create_platform_bus(RISCVVirtState *s, DeviceState *irqchip) From patchwork Thu Jul 27 10:24:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong-Xuan Wang X-Patchwork-Id: 13330008 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E5B4EC41513 for ; Thu, 27 Jul 2023 11:35:43 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qOyBJ-0003Ph-MJ; Thu, 27 Jul 2023 06:25:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qOyAy-0003PN-Fo for qemu-devel@nongnu.org; Thu, 27 Jul 2023 06:24:56 -0400 Received: from mail-io1-xd2f.google.com ([2607:f8b0:4864:20::d2f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qOyAw-0003tw-Oo for qemu-devel@nongnu.org; Thu, 27 Jul 2023 06:24:56 -0400 Received: by mail-io1-xd2f.google.com with SMTP id ca18e2360f4ac-7906614bc90so5389639f.3 for ; Thu, 27 Jul 2023 03:24:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1690453493; x=1691058293; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=GfEVzfDMM3KV/MRFcXCu/fGQgJk4J1J61XVMjT70iMs=; b=cHmrInueGEngMD7UYGmB+iSXoktK/awClLPYSSKF/pxXfwAGRn+ArthLC/yhmHpuJB LswL/BReQY7IL6c2/yJXWuk09AYNEeR2QJscAvtNifHzohRaMFnzh7JCCOs+E/qsZn5n TS7CmIir0KqXVnwtD7qWEC8ZpYvZMPqiD2NQ3Vn+58/3ZCWF2Spok7iS6cAIvZaQ3h8H yMZJIHYAKXPamwly2j9kdO7kitEsCGSBFTRscnmDvQltEnV0RpJAfcpbrmq5R22FK8Q+ slnlFFAe9ukcL6dPVpCUHcvwdNNeIatrHV5mkFebfpIzyIR6/kQjINlH/Qjb8WoywZcp WcCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690453493; x=1691058293; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GfEVzfDMM3KV/MRFcXCu/fGQgJk4J1J61XVMjT70iMs=; b=Y6QQNQ+R06Ndcy/hsjp9w7MKeYDDaDKOMtxyasrJyTy5e2hr0XJ7/f2k+0vl6uk/7e Yh6bdQ+Pth6nTnnPEYNfosbWZyByV+BOJomiihN7vxfhu112eiYq6eRCtsDOT51dphqa 70yKEJCVBrEneWVbDQb6Pqa/i/Z1LQBq0JTz2dptMo1lRSShMFn9F807QN8VPxI+zeF6 Yv8MYJEQUym5oO3Lpw+PLF07rhWCv5visI38j8gcOl0p16DhrJPR6JYGCLJt85g0s3jP 8hWxvkeDoRuSue9F55e48KPb4F6aZ8zWzkmNtWdJYlFDnc/TXtgU76lVLQb6GRVUGndF f5EA== X-Gm-Message-State: ABy/qLbE4OGEuX2U5F+fbHj+XHqEPmZD2lK26Shg3PWC3N4Tzx511g+K ptO8PiPAV+XC2Su2AlHcTD9iBE812mLUK3zJPItle6W3IGTc1hFO0ArlJGGgxrID0j5juL6tXYS raeQW6vfaB8objqCHpruL5uBGlCTIHeXzPbPncgnaKNBFeuAfY272kMA+pafiU0T+nHgGjUgKnI R8vMqF X-Google-Smtp-Source: APBJJlGmpqtL4QEdAiDixYp4kNbQW1CQ9NkC8foOTmxv9KtWAPuVL9V2HinYgjq9XtD2a/1AO9/O7w== X-Received: by 2002:a05:6e02:1a6a:b0:345:fbdc:bb78 with SMTP id w10-20020a056e021a6a00b00345fbdcbb78mr5294262ilv.29.1690453493285; Thu, 27 Jul 2023 03:24:53 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id u8-20020a17090341c800b001b882880550sm1230139ple.282.2023.07.27.03.24.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Jul 2023 03:24:53 -0700 (PDT) From: Yong-Xuan Wang To: qemu-devel@nongnu.org, qemu-riscv@nongnu.org Cc: rkanwal@rivosinc.com, anup@brainfault.org, dbarboza@ventanamicro.com, ajones@ventanamicro.com, atishp@atishpatra.org, vincent.chen@sifive.com, greentime.hu@sifive.com, frank.chang@sifive.com, jim.shu@sifive.com, Yong-Xuan Wang , Palmer Dabbelt , Alistair Francis , Bin Meng , Weiwei Li , Liu Zhiwei , Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v7 2/5] target/riscv: check the in-kernel irqchip support Date: Thu, 27 Jul 2023 10:24:34 +0000 Message-Id: <20230727102439.22554-3-yongxuan.wang@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230727102439.22554-1-yongxuan.wang@sifive.com> References: <20230727102439.22554-1-yongxuan.wang@sifive.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::d2f; envelope-from=yongxuan.wang@sifive.com; helo=mail-io1-xd2f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We check the in-kernel irqchip support when using KVM acceleration. Signed-off-by: Yong-Xuan Wang Reviewed-by: Jim Shu Reviewed-by: Daniel Henrique Barboza Reviewed-by: Andrew Jones --- target/riscv/kvm.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 9d8a8982f9..005e054604 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -914,7 +914,15 @@ int kvm_arch_init(MachineState *ms, KVMState *s) int kvm_arch_irqchip_create(KVMState *s) { - return 0; + if (kvm_kernel_irqchip_split()) { + error_report("-machine kernel_irqchip=split is not supported on RISC-V."); + exit(1); + } + + /* + * We can create the VAIA using the newer device control API. + */ + return kvm_check_extension(s, KVM_CAP_DEVICE_CTRL); } int kvm_arch_process_async_events(CPUState *cs) From patchwork Thu Jul 27 10:24:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong-Xuan Wang X-Patchwork-Id: 13329968 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 54C65C0015E for ; Thu, 27 Jul 2023 11:22:58 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qOyBo-0003Yg-ND; Thu, 27 Jul 2023 06:25:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qOyB4-0003Sm-IZ for qemu-devel@nongnu.org; Thu, 27 Jul 2023 06:25:14 -0400 Received: from mail-oa1-x36.google.com ([2001:4860:4864:20::36]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qOyB2-0003uh-33 for qemu-devel@nongnu.org; Thu, 27 Jul 2023 06:25:02 -0400 Received: by mail-oa1-x36.google.com with SMTP id 586e51a60fabf-1bb98474b8cso658287fac.3 for ; Thu, 27 Jul 2023 03:24:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1690453498; x=1691058298; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=p3orrnsVpmPvdMzKCfa6SEMJluYKUnGd5kHcMJ1WXrM=; b=EQwROzf+msMOSLAm/5y6gX+AoHCG3hjukm31H/UEcZy+cDv/K/IxAF6tC0Jn447kNi WvBd1mymEdZl3hnTaEhb2l4atMqypmRR8yiHCyZjCLbPrPo9Jw0LXn52JEQrA+zguasi GdYHOF1G4+1YIPhdjZ7V7WVFJSv8VHyU6LANOlTiaEv6As5tRWt4yGx2mvtMBQFzBo4p dh97hXGkD5nonIy5JzxV6EdCFziww/Iac2BYEV2t0zD69A+zO0gqo3cSsJU4HAlCTBtu Q5RYXU6N5n1lm0koGTMVD4EXUcs4QEWKhgH1pJ8hw6JHNFAAJqaalklgQC4pD4dSWRLC xypg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690453498; x=1691058298; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=p3orrnsVpmPvdMzKCfa6SEMJluYKUnGd5kHcMJ1WXrM=; b=ASH1m0n86Mpcm2v5oZsqoWiRgsLvKPFnULr7p5Hs5E+Hn7sJSfOwRQKYrIhi+wOMPo DNCvc904H9ufe+bbp0OUsdcPEzgVVMLA3PCuOoi77bnfRgWTLAAmPetiucfQjqUT22UM eunJpG486DQiuOtBudf4oonef4nmyWOq3kJtbfuZCRNcMiU/Zl+aEfnispja5ENleGn0 5MIV2F+HW7ycuwV0KKRh0dR+URl5zZclVUatYrs/ydqt9B74BXlDpFNil9WMTu1FP26q JKOAKXZ5IUWDY7bq537HpD7SwVDFk0vXLr5f26kc/fuY2wFdXPsklisDxZ0LsG7na67w e+yA== X-Gm-Message-State: ABy/qLaBxM5EqaC1J691oZCTCmFDVnwgXd+OKH7kFcEo92hoqbrgBHbl vcS0nXxv7Z+ygQUk8RL9UC+CH9MnSZ+qtcfJAVx+DVi21423dASi4UdkfXtLQSu35bzmk6lm2+1 QHMRrSMDOxiyuQwrWZWvodYzBBxfl4ss+4AhMumqjmybPH4pZh+XTrcrMcLIdDdGgZAXFCzmo/Q 23mbVS X-Google-Smtp-Source: APBJJlF5PC70kkRRGOeQzn5k2u7WI5QwJ/1xIiC62ubRFNQuRYWlJ6zWKXRkGOLaLKOb+aFzSgKBXA== X-Received: by 2002:a05:6870:a550:b0:1bb:c236:dff with SMTP id p16-20020a056870a55000b001bbc2360dffmr2697769oal.50.1690453497854; Thu, 27 Jul 2023 03:24:57 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id u8-20020a17090341c800b001b882880550sm1230139ple.282.2023.07.27.03.24.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Jul 2023 03:24:57 -0700 (PDT) From: Yong-Xuan Wang To: qemu-devel@nongnu.org, qemu-riscv@nongnu.org Cc: rkanwal@rivosinc.com, anup@brainfault.org, dbarboza@ventanamicro.com, ajones@ventanamicro.com, atishp@atishpatra.org, vincent.chen@sifive.com, greentime.hu@sifive.com, frank.chang@sifive.com, jim.shu@sifive.com, Yong-Xuan Wang , Palmer Dabbelt , Alistair Francis , Bin Meng , Weiwei Li , Liu Zhiwei , Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v7 3/5] target/riscv: Create an KVM AIA irqchip Date: Thu, 27 Jul 2023 10:24:35 +0000 Message-Id: <20230727102439.22554-4-yongxuan.wang@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230727102439.22554-1-yongxuan.wang@sifive.com> References: <20230727102439.22554-1-yongxuan.wang@sifive.com> Received-SPF: pass client-ip=2001:4860:4864:20::36; envelope-from=yongxuan.wang@sifive.com; helo=mail-oa1-x36.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We create a vAIA chip by using the KVM_DEV_TYPE_RISCV_AIA and then set up the chip with the KVM_DEV_RISCV_AIA_GRP_* APIs. We also extend KVM accelerator to specify the KVM AIA mode. The "riscv-aia" parameter is passed along with --accel in QEMU command-line. 1) "riscv-aia=emul": IMSIC is emulated by hypervisor 2) "riscv-aia=hwaccel": use hardware guest IMSIC 3) "riscv-aia=auto": use the hardware guest IMSICs whenever available otherwise we fallback to software emulation. Signed-off-by: Yong-Xuan Wang Reviewed-by: Jim Shu Reviewed-by: Daniel Henrique Barboza Reviewed-by: Andrew Jones --- target/riscv/kvm.c | 186 +++++++++++++++++++++++++++++++++++++++ target/riscv/kvm_riscv.h | 4 + 2 files changed, 190 insertions(+) diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 005e054604..0c17e2027a 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -36,6 +36,7 @@ #include "exec/address-spaces.h" #include "hw/boards.h" #include "hw/irq.h" +#include "hw/intc/riscv_imsic.h" #include "qemu/log.h" #include "hw/loader.h" #include "kvm_riscv.h" @@ -43,6 +44,7 @@ #include "chardev/char-fe.h" #include "migration/migration.h" #include "sysemu/runstate.h" +#include "hw/riscv/numa.h" static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx) @@ -1023,6 +1025,190 @@ bool kvm_arch_cpu_check_are_resettable(void) return true; } +static int aia_mode; + +static const char *kvm_aia_mode_str(uint64_t mode) +{ + switch (mode) { + case KVM_DEV_RISCV_AIA_MODE_EMUL: + return "emul"; + case KVM_DEV_RISCV_AIA_MODE_HWACCEL: + return "hwaccel"; + case KVM_DEV_RISCV_AIA_MODE_AUTO: + default: + return "auto"; + }; +} + +static char *riscv_get_kvm_aia(Object *obj, Error **errp) +{ + return g_strdup(kvm_aia_mode_str(aia_mode)); +} + +static void riscv_set_kvm_aia(Object *obj, const char *val, Error **errp) +{ + if (!strcmp(val, "emul")) { + aia_mode = KVM_DEV_RISCV_AIA_MODE_EMUL; + } else if (!strcmp(val, "hwaccel")) { + aia_mode = KVM_DEV_RISCV_AIA_MODE_HWACCEL; + } else if (!strcmp(val, "auto")) { + aia_mode = KVM_DEV_RISCV_AIA_MODE_AUTO; + } else { + error_setg(errp, "Invalid KVM AIA mode"); + error_append_hint(errp, "Valid values are emul, hwaccel, and auto.\n"); + } +} + void kvm_arch_accel_class_init(ObjectClass *oc) { + object_class_property_add_str(oc, "riscv-aia", riscv_get_kvm_aia, + riscv_set_kvm_aia); + object_class_property_set_description(oc, "riscv-aia", + "Set KVM AIA mode. Valid values are " + "emul, hwaccel, and auto. Default " + "is auto."); + object_property_set_default_str(object_class_property_find(oc, "riscv-aia"), + "auto"); +} + +void kvm_riscv_aia_create(MachineState *machine, uint64_t group_shift, + uint64_t aia_irq_num, uint64_t aia_msi_num, + uint64_t aplic_base, uint64_t imsic_base, + uint64_t guest_num) +{ + int ret, i; + int aia_fd = -1; + uint64_t default_aia_mode; + uint64_t socket_count = riscv_socket_count(machine); + uint64_t max_hart_per_socket = 0; + uint64_t socket, base_hart, hart_count, socket_imsic_base, imsic_addr; + uint64_t socket_bits, hart_bits, guest_bits; + + aia_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_RISCV_AIA, false); + + if (aia_fd < 0) { + error_report("Unable to create in-kernel irqchip"); + exit(1); + } + + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_MODE, + &default_aia_mode, false, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to get current KVM AIA mode"); + exit(1); + } + qemu_log("KVM AIA: default mode is %s\n", + kvm_aia_mode_str(default_aia_mode)); + + if (default_aia_mode != aia_mode) { + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_MODE, + &aia_mode, true, NULL); + if (ret < 0) + warn_report("KVM AIA: failed to set KVM AIA mode"); + else + qemu_log("KVM AIA: set current mode to %s\n", + kvm_aia_mode_str(aia_mode)); + } + + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_SRCS, + &aia_irq_num, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set number of input irq lines"); + exit(1); + } + + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_IDS, + &aia_msi_num, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set number of msi"); + exit(1); + } + + socket_bits = find_last_bit(&socket_count, BITS_PER_LONG) + 1; + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_GROUP_BITS, + &socket_bits, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set group_bits"); + exit(1); + } + + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_GROUP_SHIFT, + &group_shift, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set group_shift"); + exit(1); + } + + guest_bits = guest_num == 0 ? 0 : + find_last_bit(&guest_num, BITS_PER_LONG) + 1; + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_GUEST_BITS, + &guest_bits, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set guest_bits"); + exit(1); + } + + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_ADDR, + KVM_DEV_RISCV_AIA_ADDR_APLIC, + &aplic_base, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set the base address of APLIC"); + exit(1); + } + + for (socket = 0; socket < socket_count; socket++) { + socket_imsic_base = imsic_base + socket * (1U << group_shift); + hart_count = riscv_socket_hart_count(machine, socket); + base_hart = riscv_socket_first_hartid(machine, socket); + + if (max_hart_per_socket < hart_count) { + max_hart_per_socket = hart_count; + } + + for (i = 0; i < hart_count; i++) { + imsic_addr = socket_imsic_base + i * IMSIC_HART_SIZE(guest_bits); + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_ADDR, + KVM_DEV_RISCV_AIA_ADDR_IMSIC(i + base_hart), + &imsic_addr, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set the IMSIC address for hart %d", i); + exit(1); + } + } + } + + hart_bits = find_last_bit(&max_hart_per_socket, BITS_PER_LONG) + 1; + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_HART_BITS, + &hart_bits, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set hart_bits"); + exit(1); + } + + if (kvm_has_gsi_routing()) { + for (uint64_t idx = 0; idx < aia_irq_num + 1; ++idx) { + /* KVM AIA only has one APLIC instance */ + kvm_irqchip_add_irq_route(kvm_state, idx, 0, idx); + } + kvm_gsi_routing_allowed = true; + kvm_irqchip_commit_routes(kvm_state); + } + + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CTRL, + KVM_DEV_RISCV_AIA_CTRL_INIT, + NULL, true, NULL); + if (ret < 0) { + error_report("KVM AIA: initialized fail"); + exit(1); + } + + kvm_msi_via_irqfd_allowed = kvm_irqfds_enabled(); } diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h index e3ba935808..7d4b7c60e2 100644 --- a/target/riscv/kvm_riscv.h +++ b/target/riscv/kvm_riscv.h @@ -22,5 +22,9 @@ void kvm_riscv_init_user_properties(Object *cpu_obj); void kvm_riscv_reset_vcpu(RISCVCPU *cpu); void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level); +void kvm_riscv_aia_create(MachineState *machine, uint64_t group_shift, + uint64_t aia_irq_num, uint64_t aia_msi_num, + uint64_t aplic_base, uint64_t imsic_base, + uint64_t guest_num); #endif From patchwork Thu Jul 27 10:24:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong-Xuan Wang X-Patchwork-Id: 13329974 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0F2B9C001E0 for ; Thu, 27 Jul 2023 11:30:38 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qOyC9-0003iU-1A; Thu, 27 Jul 2023 06:26:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qOyBP-0003Wu-79 for qemu-devel@nongnu.org; Thu, 27 Jul 2023 06:25:23 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qOyBD-00049T-BZ for qemu-devel@nongnu.org; Thu, 27 Jul 2023 06:25:15 -0400 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1bbc06f830aso4981225ad.0 for ; Thu, 27 Jul 2023 03:25:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1690453505; x=1691058305; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=OULN+wwisoBSnT3VMk9Y7HLLhh5KeELP9Ck02I8dOSM=; b=SkFM1Jhyqd5M80e+Hjce9UMwsCrdxraYXDT6NJnpvnnz1pf5JFUVdSWdvzN8BvQYwB k91nGrBhz2QzgVqfKxewEDSlwB91NrRLdUMFmuSQPRKkQLKbGHLM2pU0uZjC5TVvrOaM 9JuCg28UuEtQCvKkE/HmuIelNoN2qAdhfSQRKUxSv8KJlJtx7XGFyDaZxZHZSBpry0DE b26eCWl7u26IfOwzxKGOwYGGbS27TvsLeWZEP/GI3vWyp2/L5z4vCluGZxc+VhKhCst9 UflMBkGy8/6F0mF1d7suHEPza816s922eVSi4tUNrsJbJCzw6VvLtJVitnV6dh5kwQ6g aQbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690453505; x=1691058305; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=OULN+wwisoBSnT3VMk9Y7HLLhh5KeELP9Ck02I8dOSM=; b=FPYnlmrqhlaxxaDrIn0cn4YKpFlsdjLhrQguzwj7ut8NcCGsGKVnCT+V10sd7Nfd9Y VtzRPa1PgL8tr3CFOoyBJ18GXa6uDQaFpZTP2QJaebntntp7SQDMWw6ZqwJNLOfQ8aqz N26C96gDr4epZnhBf50YWp6rSQNxeebP8NWrNaY61lyEBC+mmZSql+WcIv7jvVeBdJ+T SkqSgcuTmylMwkEzQScgqp1lGtM9Vq64BC9hYfH69xn92sE1PTNrvlnETQ2aP24AyzJV sUzdsHypJmM53cZoc6bziUWpw+1DfTcNVMuxIht9/pU8HL4U67h3EF3vijb3Sy7gvR73 vs1Q== X-Gm-Message-State: ABy/qLYrfBS4gCxoboWOg2qyS1D1FkYrB6re80+YW8aBUAG0+c3Ei6x9 vxWHvB9HlowOvJcR9EdHDylRMphklDXkmtw3Pd5bs9do7Z65R8AHNHNBQA7UQBFuXdlat2GLRSx I5QOcDt4GI3GaI8fQ8N8UJJvnkRC1TFtlRUm4RkcROPaHBfHXrOHpZDPGaj6GpQF4bM0lcEz+Dr uD6O5I X-Google-Smtp-Source: APBJJlG2ZP7qwtNxQoeQJnvSWf3xpKzie1j1g+qVGFlELlVRwE77NiAW2P/zjn+f7Rk/e4CDl0wRcw== X-Received: by 2002:a17:902:64c8:b0:1b6:6b03:10e7 with SMTP id y8-20020a17090264c800b001b66b0310e7mr2648932pli.5.1690453505162; Thu, 27 Jul 2023 03:25:05 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id u8-20020a17090341c800b001b882880550sm1230139ple.282.2023.07.27.03.25.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Jul 2023 03:25:04 -0700 (PDT) From: Yong-Xuan Wang To: qemu-devel@nongnu.org, qemu-riscv@nongnu.org Cc: rkanwal@rivosinc.com, anup@brainfault.org, dbarboza@ventanamicro.com, ajones@ventanamicro.com, atishp@atishpatra.org, vincent.chen@sifive.com, greentime.hu@sifive.com, frank.chang@sifive.com, jim.shu@sifive.com, Yong-Xuan Wang , Alistair Francis , Ivan Klokov , Palmer Dabbelt Subject: [PATCH v7 4/5] target/riscv: update APLIC and IMSIC to support KVM AIA Date: Thu, 27 Jul 2023 10:24:36 +0000 Message-Id: <20230727102439.22554-5-yongxuan.wang@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230727102439.22554-1-yongxuan.wang@sifive.com> References: <20230727102439.22554-1-yongxuan.wang@sifive.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=yongxuan.wang@sifive.com; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org KVM AIA can't emulate APLIC only. When "aia=aplic" parameter is passed, APLIC devices is emulated by QEMU. For "aia=aplic-imsic", remove the mmio operations of APLIC when using KVM AIA and send wired interrupt signal via KVM_IRQ_LINE API. After KVM AIA enabled, MSI messages are delivered by KVM_SIGNAL_MSI API when the IMSICs receive mmio write requests. Signed-off-by: Yong-Xuan Wang Reviewed-by: Jim Shu Reviewed-by: Daniel Henrique Barboza Reviewed-by: Andrew Jones --- hw/intc/riscv_aplic.c | 56 ++++++++++++++++++++++++++++++------------- hw/intc/riscv_imsic.c | 25 +++++++++++++++---- 2 files changed, 61 insertions(+), 20 deletions(-) diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c index 4bdc6a5d1a..592c3ce768 100644 --- a/hw/intc/riscv_aplic.c +++ b/hw/intc/riscv_aplic.c @@ -31,6 +31,7 @@ #include "hw/irq.h" #include "target/riscv/cpu.h" #include "sysemu/sysemu.h" +#include "sysemu/kvm.h" #include "migration/vmstate.h" #define APLIC_MAX_IDC (1UL << 14) @@ -148,6 +149,15 @@ #define APLIC_IDC_CLAIMI 0x1c +/* + * KVM AIA only supports APLIC MSI, fallback to QEMU emulation if we want to use + * APLIC Wired. + */ +static bool is_kvm_aia(bool msimode) +{ + return kvm_irqchip_in_kernel() && msimode; +} + static uint32_t riscv_aplic_read_input_word(RISCVAPLICState *aplic, uint32_t word) { @@ -471,6 +481,11 @@ static uint32_t riscv_aplic_idc_claimi(RISCVAPLICState *aplic, uint32_t idc) return topi; } +static void riscv_kvm_aplic_request(void *opaque, int irq, int level) +{ + kvm_set_irq(kvm_state, irq, !!level); +} + static void riscv_aplic_request(void *opaque, int irq, int level) { bool update = false; @@ -801,29 +816,35 @@ static void riscv_aplic_realize(DeviceState *dev, Error **errp) uint32_t i; RISCVAPLICState *aplic = RISCV_APLIC(dev); - aplic->bitfield_words = (aplic->num_irqs + 31) >> 5; - aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs); - aplic->state = g_new0(uint32_t, aplic->num_irqs); - aplic->target = g_new0(uint32_t, aplic->num_irqs); - if (!aplic->msimode) { - for (i = 0; i < aplic->num_irqs; i++) { - aplic->target[i] = 1; + if (!is_kvm_aia(aplic->msimode)) { + aplic->bitfield_words = (aplic->num_irqs + 31) >> 5; + aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs); + aplic->state = g_new0(uint32_t, aplic->num_irqs); + aplic->target = g_new0(uint32_t, aplic->num_irqs); + if (!aplic->msimode) { + for (i = 0; i < aplic->num_irqs; i++) { + aplic->target[i] = 1; + } } - } - aplic->idelivery = g_new0(uint32_t, aplic->num_harts); - aplic->iforce = g_new0(uint32_t, aplic->num_harts); - aplic->ithreshold = g_new0(uint32_t, aplic->num_harts); + aplic->idelivery = g_new0(uint32_t, aplic->num_harts); + aplic->iforce = g_new0(uint32_t, aplic->num_harts); + aplic->ithreshold = g_new0(uint32_t, aplic->num_harts); - memory_region_init_io(&aplic->mmio, OBJECT(dev), &riscv_aplic_ops, aplic, - TYPE_RISCV_APLIC, aplic->aperture_size); - sysbus_init_mmio(SYS_BUS_DEVICE(dev), &aplic->mmio); + memory_region_init_io(&aplic->mmio, OBJECT(dev), &riscv_aplic_ops, + aplic, TYPE_RISCV_APLIC, aplic->aperture_size); + sysbus_init_mmio(SYS_BUS_DEVICE(dev), &aplic->mmio); + } /* * Only root APLICs have hardware IRQ lines. All non-root APLICs * have IRQ lines delegated by their parent APLIC. */ if (!aplic->parent) { - qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs); + if (is_kvm_aia(aplic->msimode)) { + qdev_init_gpio_in(dev, riscv_kvm_aplic_request, aplic->num_irqs); + } else { + qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs); + } } /* Create output IRQ lines for non-MSI mode */ @@ -958,7 +979,10 @@ DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size, qdev_prop_set_bit(dev, "mmode", mmode); sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal); - sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr); + + if (!is_kvm_aia(msimode)) { + sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr); + } if (parent) { riscv_aplic_add_child(parent, dev); diff --git a/hw/intc/riscv_imsic.c b/hw/intc/riscv_imsic.c index fea3385b51..760dbddcf7 100644 --- a/hw/intc/riscv_imsic.c +++ b/hw/intc/riscv_imsic.c @@ -32,6 +32,7 @@ #include "target/riscv/cpu.h" #include "target/riscv/cpu_bits.h" #include "sysemu/sysemu.h" +#include "sysemu/kvm.h" #include "migration/vmstate.h" #define IMSIC_MMIO_PAGE_LE 0x00 @@ -283,6 +284,20 @@ static void riscv_imsic_write(void *opaque, hwaddr addr, uint64_t value, goto err; } +#if defined(CONFIG_KVM) + if (kvm_irqchip_in_kernel()) { + struct kvm_msi msi; + + msi.address_lo = extract64(imsic->mmio.addr + addr, 0, 32); + msi.address_hi = extract64(imsic->mmio.addr + addr, 32, 32); + msi.data = le32_to_cpu(value); + + kvm_vm_ioctl(kvm_state, KVM_SIGNAL_MSI, &msi); + + return; + } +#endif + /* Writes only supported for MSI little-endian registers */ page = addr >> IMSIC_MMIO_PAGE_SHIFT; if ((addr & (IMSIC_MMIO_PAGE_SZ - 1)) == IMSIC_MMIO_PAGE_LE) { @@ -320,10 +335,12 @@ static void riscv_imsic_realize(DeviceState *dev, Error **errp) CPUState *cpu = cpu_by_arch_id(imsic->hartid); CPURISCVState *env = cpu ? cpu->env_ptr : NULL; - imsic->num_eistate = imsic->num_pages * imsic->num_irqs; - imsic->eidelivery = g_new0(uint32_t, imsic->num_pages); - imsic->eithreshold = g_new0(uint32_t, imsic->num_pages); - imsic->eistate = g_new0(uint32_t, imsic->num_eistate); + if (!kvm_irqchip_in_kernel()) { + imsic->num_eistate = imsic->num_pages * imsic->num_irqs; + imsic->eidelivery = g_new0(uint32_t, imsic->num_pages); + imsic->eithreshold = g_new0(uint32_t, imsic->num_pages); + imsic->eistate = g_new0(uint32_t, imsic->num_eistate); + } memory_region_init_io(&imsic->mmio, OBJECT(dev), &riscv_imsic_ops, imsic, TYPE_RISCV_IMSIC, From patchwork Thu Jul 27 10:24:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong-Xuan Wang X-Patchwork-Id: 13329937 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 90502C001DC for ; Thu, 27 Jul 2023 10:39:54 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qOyCB-0003j4-3m; Thu, 27 Jul 2023 06:26:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qOyBQ-0003XJ-8j for qemu-devel@nongnu.org; Thu, 27 Jul 2023 06:25:36 -0400 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qOyBF-0004Bj-7E for qemu-devel@nongnu.org; Thu, 27 Jul 2023 06:25:19 -0400 Received: by mail-pl1-x629.google.com with SMTP id d9443c01a7336-1bbc64f9a91so6252435ad.0 for ; Thu, 27 Jul 2023 03:25:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1690453510; x=1691058310; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=H7Zt4zM7mdn4Ype868SA62zfOIkFcLwtmRVr8Dp/bb8=; b=d9tzBPwHMDWBaO1swiv2Dw3LW7KVWDxxKgrMljO2n2C2ch9bvwjIxo7+wacgc/cECj Wa3qPcm1hmYSsevakmq3CWtP7xz2R6J8RdOPwnG3j3CO2Ax6qcPQ3Oqah8vvEfO3sSuB 0++tlOepW/H4ToxI5Kpwm15vjWw4ZFT7wcwLUBNF2mt0avt0VBXZfF9O7+e7rvdLLB5A dk+zUCVyOCc+8BvTu9c3AlePyo/9tNCTJagQcVGfhr/YLCy1NddId0k+lfENaB7rRKsj GXinsOu/PC89pdPt01jegja28o9fACrkazgahXzIXXED22YOdFn2ds/le0bImTLEXlZ8 X5ZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690453510; x=1691058310; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=H7Zt4zM7mdn4Ype868SA62zfOIkFcLwtmRVr8Dp/bb8=; b=EFfSK0h6SVwl8YjTTBSNB837hM9jk3mOPwH4RWYu46jLTU/pR+BMbbsE3LVVyorbiz J1fx7UkqRuPopXyIoEew9cBMrDj9oUQqS+Gk0Vhl73RnwnDBiNlLs+avV7cZoK+LiaMc KzRNCx45xFD4Fk7Os3yWmbUZNIu0X/Lj8M/URgPUdMgN8yhdECOTlkn1F3bED6uSnRs3 NNWqx+dGTjPlNe9HMt3qtnsmcdtofRZSDq8Er6lm5DBv+NwTpNiaC6phC3Ea9pOsltb2 g62MX4xVM2XSAEeElByD4SiOW1QYNWVZGal4ISlNSj2QzN9nJstASzDSxPahjQNtQa/o CUfQ== X-Gm-Message-State: ABy/qLYukEW8OjGxkHmp6wT1TntWv4dc/2zx+bldEeBxfwJJ4u4cx55j bFloywKTVKqre/w1j8n8MIzokk4GI+rLWh3oHR1z89D/toKzFU3/C7ndM9rIdYVU45y9QQyD6jR wyENv5ip2ml+C5P1KS5+SMoeV2+oXdrlnRc32zEKlRpYfLzxkr2td1Fd56l0/y3zA8oca9qfPcK 2WocK5 X-Google-Smtp-Source: APBJJlGAOCc+wbYfEp9OyBp1n97lYyv0gEYX3EEXprsSRTX1yK8rvEb1Kyy5OvsMLmx74uGh/YPsIA== X-Received: by 2002:a17:902:db09:b0:1b8:8dbd:e1a0 with SMTP id m9-20020a170902db0900b001b88dbde1a0mr5968320plx.13.1690453509815; Thu, 27 Jul 2023 03:25:09 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id u8-20020a17090341c800b001b882880550sm1230139ple.282.2023.07.27.03.25.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Jul 2023 03:25:09 -0700 (PDT) From: Yong-Xuan Wang To: qemu-devel@nongnu.org, qemu-riscv@nongnu.org Cc: rkanwal@rivosinc.com, anup@brainfault.org, dbarboza@ventanamicro.com, ajones@ventanamicro.com, atishp@atishpatra.org, vincent.chen@sifive.com, greentime.hu@sifive.com, frank.chang@sifive.com, jim.shu@sifive.com, Yong-Xuan Wang , Palmer Dabbelt , Alistair Francis , Bin Meng , Weiwei Li , Liu Zhiwei Subject: [PATCH v7 5/5] target/riscv: select KVM AIA in riscv virt machine Date: Thu, 27 Jul 2023 10:24:37 +0000 Message-Id: <20230727102439.22554-6-yongxuan.wang@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230727102439.22554-1-yongxuan.wang@sifive.com> References: <20230727102439.22554-1-yongxuan.wang@sifive.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::629; envelope-from=yongxuan.wang@sifive.com; helo=mail-pl1-x629.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Select KVM AIA when the host kernel has in-kernel AIA chip support. Since KVM AIA only has one APLIC instance, we map the QEMU APLIC devices to KVM APLIC. Signed-off-by: Yong-Xuan Wang Reviewed-by: Jim Shu Reviewed-by: Daniel Henrique Barboza Reviewed-by: Andrew Jones --- hw/riscv/virt.c | 94 +++++++++++++++++++++++++++++++++---------------- 1 file changed, 63 insertions(+), 31 deletions(-) diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index f595380be1..4af73ac1bb 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -35,6 +35,7 @@ #include "hw/riscv/virt.h" #include "hw/riscv/boot.h" #include "hw/riscv/numa.h" +#include "kvm_riscv.h" #include "hw/intc/riscv_aclint.h" #include "hw/intc/riscv_aplic.h" #include "hw/intc/riscv_imsic.h" @@ -75,6 +76,12 @@ #error "Can't accomodate all IMSIC groups in address space" #endif +/* KVM AIA only supports APLIC MSI. APLIC Wired is always emulated by QEMU. */ +static bool virt_use_kvm_aia(RISCVVirtState *s) +{ + return kvm_irqchip_in_kernel() && s->aia_type == VIRT_AIA_TYPE_APLIC_IMSIC; +} + static const MemMapEntry virt_memmap[] = { [VIRT_DEBUG] = { 0x0, 0x100 }, [VIRT_MROM] = { 0x1000, 0xf000 }, @@ -609,16 +616,16 @@ static void create_fdt_one_aplic(RISCVVirtState *s, int socket, uint32_t *intc_phandles, uint32_t aplic_phandle, uint32_t aplic_child_phandle, - bool m_mode) + bool m_mode, int num_harts) { int cpu; char *aplic_name; uint32_t *aplic_cells; MachineState *ms = MACHINE(s); - aplic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 2); + aplic_cells = g_new0(uint32_t, num_harts * 2); - for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) { + for (cpu = 0; cpu < num_harts; cpu++) { aplic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]); aplic_cells[cpu * 2 + 1] = cpu_to_be32(m_mode ? IRQ_M_EXT : IRQ_S_EXT); } @@ -632,8 +639,7 @@ static void create_fdt_one_aplic(RISCVVirtState *s, int socket, if (s->aia_type == VIRT_AIA_TYPE_APLIC) { qemu_fdt_setprop(ms->fdt, aplic_name, "interrupts-extended", - aplic_cells, - s->soc[socket].num_harts * sizeof(uint32_t) * 2); + aplic_cells, num_harts * sizeof(uint32_t) * 2); } else { qemu_fdt_setprop_cell(ms->fdt, aplic_name, "msi-parent", msi_phandle); } @@ -664,7 +670,8 @@ static void create_fdt_socket_aplic(RISCVVirtState *s, uint32_t msi_s_phandle, uint32_t *phandle, uint32_t *intc_phandles, - uint32_t *aplic_phandles) + uint32_t *aplic_phandles, + int num_harts) { char *aplic_name; unsigned long aplic_addr; @@ -681,7 +688,7 @@ static void create_fdt_socket_aplic(RISCVVirtState *s, create_fdt_one_aplic(s, socket, aplic_addr, memmap[VIRT_APLIC_M].size, msi_m_phandle, intc_phandles, aplic_m_phandle, aplic_s_phandle, - true); + true, num_harts); } /* S-level APLIC node */ @@ -690,7 +697,7 @@ static void create_fdt_socket_aplic(RISCVVirtState *s, create_fdt_one_aplic(s, socket, aplic_addr, memmap[VIRT_APLIC_S].size, msi_s_phandle, intc_phandles, aplic_s_phandle, 0, - false); + false, num_harts); aplic_name = g_strdup_printf("/soc/aplic@%lx", aplic_addr); @@ -774,34 +781,51 @@ static void create_fdt_sockets(RISCVVirtState *s, const MemMapEntry *memmap, *msi_pcie_phandle = msi_s_phandle; } - phandle_pos = ms->smp.cpus; - for (socket = (socket_count - 1); socket >= 0; socket--) { - phandle_pos -= s->soc[socket].num_harts; - - if (s->aia_type == VIRT_AIA_TYPE_NONE) { - create_fdt_socket_plic(s, memmap, socket, phandle, - &intc_phandles[phandle_pos], xplic_phandles); - } else { - create_fdt_socket_aplic(s, memmap, socket, - msi_m_phandle, msi_s_phandle, phandle, - &intc_phandles[phandle_pos], xplic_phandles); + /* KVM AIA only has one APLIC instance */ + if (virt_use_kvm_aia(s)) { + create_fdt_socket_aplic(s, memmap, 0, + msi_m_phandle, msi_s_phandle, phandle, + &intc_phandles[0], xplic_phandles, + ms->smp.cpus); + } else { + phandle_pos = ms->smp.cpus; + for (socket = (socket_count - 1); socket >= 0; socket--) { + phandle_pos -= s->soc[socket].num_harts; + + if (s->aia_type == VIRT_AIA_TYPE_NONE) { + create_fdt_socket_plic(s, memmap, socket, phandle, + &intc_phandles[phandle_pos], + xplic_phandles); + } else { + create_fdt_socket_aplic(s, memmap, socket, + msi_m_phandle, msi_s_phandle, phandle, + &intc_phandles[phandle_pos], + xplic_phandles, + s->soc[socket].num_harts); + } } } g_free(intc_phandles); - for (socket = 0; socket < socket_count; socket++) { - if (socket == 0) { - *irq_mmio_phandle = xplic_phandles[socket]; - *irq_virtio_phandle = xplic_phandles[socket]; - *irq_pcie_phandle = xplic_phandles[socket]; - } - if (socket == 1) { - *irq_virtio_phandle = xplic_phandles[socket]; - *irq_pcie_phandle = xplic_phandles[socket]; - } - if (socket == 2) { - *irq_pcie_phandle = xplic_phandles[socket]; + if (virt_use_kvm_aia(s)) { + *irq_mmio_phandle = xplic_phandles[0]; + *irq_virtio_phandle = xplic_phandles[0]; + *irq_pcie_phandle = xplic_phandles[0]; + } else { + for (socket = 0; socket < socket_count; socket++) { + if (socket == 0) { + *irq_mmio_phandle = xplic_phandles[socket]; + *irq_virtio_phandle = xplic_phandles[socket]; + *irq_pcie_phandle = xplic_phandles[socket]; + } + if (socket == 1) { + *irq_virtio_phandle = xplic_phandles[socket]; + *irq_pcie_phandle = xplic_phandles[socket]; + } + if (socket == 2) { + *irq_pcie_phandle = xplic_phandles[socket]; + } } } @@ -1432,6 +1456,14 @@ static void virt_machine_init(MachineState *machine) } } + if (virt_use_kvm_aia(s)) { + kvm_riscv_aia_create(machine, IMSIC_MMIO_GROUP_MIN_SHIFT, + VIRT_IRQCHIP_NUM_SOURCES, VIRT_IRQCHIP_NUM_MSIS, + memmap[VIRT_APLIC_S].base, + memmap[VIRT_IMSIC_S].base, + s->aia_guests); + } + if (riscv_is_32bit(&s->soc[0])) { #if HOST_LONG_BITS == 64 /* limit RAM size in a 32-bit system */