From patchwork Thu Jul 13 08:43:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong-Xuan Wang X-Patchwork-Id: 13311574 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7600EC0015E for ; Thu, 13 Jul 2023 08:46:32 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qJrw7-0006N7-6v; Thu, 13 Jul 2023 04:44:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qJrvz-0006La-J3 for qemu-devel@nongnu.org; Thu, 13 Jul 2023 04:44:24 -0400 Received: from mail-pj1-x1036.google.com ([2607:f8b0:4864:20::1036]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qJrvw-0006yF-K1 for qemu-devel@nongnu.org; Thu, 13 Jul 2023 04:44:23 -0400 Received: by mail-pj1-x1036.google.com with SMTP id 98e67ed59e1d1-2637aa1a48dso147329a91.2 for ; Thu, 13 Jul 2023 01:44:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1689237858; x=1691829858; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=oDWCXSG2jMUhg8u/j+Vsa3iyGmfK9Oon8tFWhsM/Cqk=; b=VbUEr3pkjJzjaFiGhxfHMmDaNo57pUKpo1fo/R51Rh9uZWACKhQqBN001vxgsz5sg+ dwNWCL4yUKiWCEUuzcXum8jo9VEk+zqJDlOsUdCRCkWqs3zUGAYPcRJZjacbMIFrC3/X vzpjGNiFR0rECKioZVof78KqyHAhYP2s6leFzt9OixUfsZR9Lun5Ub7nEukUGaqfrB5z VsxhlMydmsnGg9RLK7j/vLPWwv4jEuWlxu78j8SzyFGkQWBRUbS65ciEL2KtKWqfTI5c 7BzuqT3XydRkBO3c9b59dWmjTw4u2Qh2VVjfdyqgxZtMwFAE2E68GXMrZtPmljCcu9IZ f6vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689237858; x=1691829858; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=oDWCXSG2jMUhg8u/j+Vsa3iyGmfK9Oon8tFWhsM/Cqk=; b=KM8gXkHbZZwtArakUjuFrG2SxPcNgBCysHrRUREeHFqB1ZcO0HIVayCdg83Su5P0LH b2WHWQ/aVVJkygc920/rOc0MNjpqlq/r0tJBw6X3twYbtrctpLvQV2RK7XA9Kf/sU7YT wM/5yWxn4UAsKbvbftVcKXt5mgC0CUBk2APqgRO0AqyD/DyeLFt1NzL8IKaQ4HXgMFpN D6OS4HS65UaApXecq73fEyH3xdvzu6573HaYNgvmFvD876sOLmjKcvFtBIXmTM9aguYr RMyBV63mNKD076EiCiJP8KaJCMQwMDCV5scH2D4DtA2SDodRw62W3urkRTpokAfXRRCu aDTg== X-Gm-Message-State: ABy/qLZc5k33CD7Q6w3cdUYhZlS1CUPNxTaXiOVi3ZzxzkMEIlG9+RFc iU8pGpG4DW2Hqhsfh61EwqJjhFuMw828O+LVbn2ysp9CTzG1I1M8AQqMBPx+uj2f36hAg3PIv2e tqsPLr5QUXIN+NNOhbW0szuszNc2QAxKYmutbqwxl2ZFyvlBP2RitgRhdfnpkq3NWpi8DxLMslX omPcji X-Google-Smtp-Source: APBJJlHAXQS/h9N4fTYRXKFxDw2KpOsrwBsi7R/xwl62Wgrt6g44Ib62S8//G8WY0yLEBTgkoyR5MA== X-Received: by 2002:a17:90a:fb83:b0:263:f3ad:64af with SMTP id cp3-20020a17090afb8300b00263f3ad64afmr316041pjb.45.1689237857872; Thu, 13 Jul 2023 01:44:17 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id c10-20020a63724a000000b0055386b1415dsm4989198pgn.51.2023.07.13.01.44.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jul 2023 01:44:17 -0700 (PDT) From: Yong-Xuan Wang To: qemu-devel@nongnu.org, qemu-riscv@nongnu.org Cc: rkanwal@rivosinc.com, anup@brainfault.org, dbarboza@ventanamicro.com, ajones@ventanamicro.com, atishp@atishpatra.org, vincent.chen@sifive.com, greentime.hu@sifive.com, frank.chang@sifive.com, jim.shu@sifive.com, Yong-Xuan Wang , Palmer Dabbelt , Alistair Francis , Bin Meng , Weiwei Li , Liu Zhiwei Subject: [PATCH v5 1/5] target/riscv: support the AIA device emulation with KVM enabled Date: Thu, 13 Jul 2023 08:43:53 +0000 Message-Id: <20230713084405.24545-2-yongxuan.wang@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230713084405.24545-1-yongxuan.wang@sifive.com> References: <20230713084405.24545-1-yongxuan.wang@sifive.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::1036; envelope-from=yongxuan.wang@sifive.com; helo=mail-pj1-x1036.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In this patch, we create the APLIC and IMSIC FDT helper functions and remove M mode AIA devices when using KVM acceleration. Signed-off-by: Yong-Xuan Wang Reviewed-by: Jim Shu Reviewed-by: Daniel Henrique Barboza Reviewed-by: Andrew Jones --- hw/riscv/virt.c | 264 ++++++++++++++++++++++-------------------------- 1 file changed, 123 insertions(+), 141 deletions(-) diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index d90286dc46..26b3aff28e 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -516,79 +516,28 @@ static uint32_t imsic_num_bits(uint32_t count) return ret; } -static void create_fdt_imsic(RISCVVirtState *s, const MemMapEntry *memmap, - uint32_t *phandle, uint32_t *intc_phandles, - uint32_t *msi_m_phandle, uint32_t *msi_s_phandle) +static void create_fdt_one_imsic(RISCVVirtState *s, hwaddr base_addr, + uint32_t *intc_phandles, uint32_t msi_phandle, + bool m_mode, uint32_t imsic_guest_bits) { int cpu, socket; char *imsic_name; MachineState *ms = MACHINE(s); int socket_count = riscv_socket_count(ms); - uint32_t imsic_max_hart_per_socket, imsic_guest_bits; + uint32_t imsic_max_hart_per_socket; uint32_t *imsic_cells, *imsic_regs, imsic_addr, imsic_size; - *msi_m_phandle = (*phandle)++; - *msi_s_phandle = (*phandle)++; imsic_cells = g_new0(uint32_t, ms->smp.cpus * 2); imsic_regs = g_new0(uint32_t, socket_count * 4); - /* M-level IMSIC node */ for (cpu = 0; cpu < ms->smp.cpus; cpu++) { imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]); - imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT); - } - imsic_max_hart_per_socket = 0; - for (socket = 0; socket < socket_count; socket++) { - imsic_addr = memmap[VIRT_IMSIC_M].base + - socket * VIRT_IMSIC_GROUP_MAX_SIZE; - imsic_size = IMSIC_HART_SIZE(0) * s->soc[socket].num_harts; - imsic_regs[socket * 4 + 0] = 0; - imsic_regs[socket * 4 + 1] = cpu_to_be32(imsic_addr); - imsic_regs[socket * 4 + 2] = 0; - imsic_regs[socket * 4 + 3] = cpu_to_be32(imsic_size); - if (imsic_max_hart_per_socket < s->soc[socket].num_harts) { - imsic_max_hart_per_socket = s->soc[socket].num_harts; - } - } - imsic_name = g_strdup_printf("/soc/imsics@%lx", - (unsigned long)memmap[VIRT_IMSIC_M].base); - qemu_fdt_add_subnode(ms->fdt, imsic_name); - qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible", - "riscv,imsics"); - qemu_fdt_setprop_cell(ms->fdt, imsic_name, "#interrupt-cells", - FDT_IMSIC_INT_CELLS); - qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller", - NULL, 0); - qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller", - NULL, 0); - qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended", - imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2); - qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs, - socket_count * sizeof(uint32_t) * 4); - qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids", - VIRT_IRQCHIP_NUM_MSIS); - if (socket_count > 1) { - qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,hart-index-bits", - imsic_num_bits(imsic_max_hart_per_socket)); - qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-bits", - imsic_num_bits(socket_count)); - qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-shift", - IMSIC_MMIO_GROUP_MIN_SHIFT); + imsic_cells[cpu * 2 + 1] = cpu_to_be32(m_mode ? IRQ_M_EXT : IRQ_S_EXT); } - qemu_fdt_setprop_cell(ms->fdt, imsic_name, "phandle", *msi_m_phandle); - g_free(imsic_name); - - /* S-level IMSIC node */ - for (cpu = 0; cpu < ms->smp.cpus; cpu++) { - imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]); - imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT); - } - imsic_guest_bits = imsic_num_bits(s->aia_guests + 1); imsic_max_hart_per_socket = 0; for (socket = 0; socket < socket_count; socket++) { - imsic_addr = memmap[VIRT_IMSIC_S].base + - socket * VIRT_IMSIC_GROUP_MAX_SIZE; + imsic_addr = base_addr + socket * VIRT_IMSIC_GROUP_MAX_SIZE; imsic_size = IMSIC_HART_SIZE(imsic_guest_bits) * s->soc[socket].num_harts; imsic_regs[socket * 4 + 0] = 0; @@ -599,119 +548,149 @@ static void create_fdt_imsic(RISCVVirtState *s, const MemMapEntry *memmap, imsic_max_hart_per_socket = s->soc[socket].num_harts; } } - imsic_name = g_strdup_printf("/soc/imsics@%lx", - (unsigned long)memmap[VIRT_IMSIC_S].base); + + imsic_name = g_strdup_printf("/soc/imsics@%lx", (unsigned long)base_addr); qemu_fdt_add_subnode(ms->fdt, imsic_name); - qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible", - "riscv,imsics"); + qemu_fdt_setprop_string(ms->fdt, imsic_name, "compatible", "riscv,imsics"); qemu_fdt_setprop_cell(ms->fdt, imsic_name, "#interrupt-cells", FDT_IMSIC_INT_CELLS); - qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller", - NULL, 0); - qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller", - NULL, 0); + qemu_fdt_setprop(ms->fdt, imsic_name, "interrupt-controller", NULL, 0); + qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller", NULL, 0); qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended", imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2); qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs, socket_count * sizeof(uint32_t) * 4); qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids", VIRT_IRQCHIP_NUM_MSIS); + if (imsic_guest_bits) { qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,guest-index-bits", imsic_guest_bits); } + if (socket_count > 1) { qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,hart-index-bits", imsic_num_bits(imsic_max_hart_per_socket)); qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-bits", imsic_num_bits(socket_count)); - qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,group-index-shift", - IMSIC_MMIO_GROUP_MIN_SHIFT); + qemu_fdt_setprop_cell(ms->fdt, imsic_name, + "riscv,group-index-shift", IMSIC_MMIO_GROUP_MIN_SHIFT); } - qemu_fdt_setprop_cell(ms->fdt, imsic_name, "phandle", *msi_s_phandle); - g_free(imsic_name); + qemu_fdt_setprop_cell(ms->fdt, imsic_name, "phandle", msi_phandle); + g_free(imsic_name); g_free(imsic_regs); g_free(imsic_cells); } -static void create_fdt_socket_aplic(RISCVVirtState *s, - const MemMapEntry *memmap, int socket, - uint32_t msi_m_phandle, - uint32_t msi_s_phandle, - uint32_t *phandle, - uint32_t *intc_phandles, - uint32_t *aplic_phandles) +static void create_fdt_imsic(RISCVVirtState *s, const MemMapEntry *memmap, + uint32_t *phandle, uint32_t *intc_phandles, + uint32_t *msi_m_phandle, uint32_t *msi_s_phandle) +{ + *msi_m_phandle = (*phandle)++; + *msi_s_phandle = (*phandle)++; + + if (!kvm_enabled()) { + /* M-level IMSIC node */ + create_fdt_one_imsic(s, memmap[VIRT_IMSIC_M].base, intc_phandles, + *msi_m_phandle, true, 0); + } + + /* S-level IMSIC node */ + create_fdt_one_imsic(s, memmap[VIRT_IMSIC_S].base, intc_phandles, + *msi_s_phandle, false, + imsic_num_bits(s->aia_guests + 1)); + +} + +static void create_fdt_one_aplic(RISCVVirtState *s, int socket, + unsigned long aplic_addr, uint32_t aplic_size, + uint32_t msi_phandle, + uint32_t *intc_phandles, + uint32_t aplic_phandle, + uint32_t aplic_child_phandle, + bool m_mode) { int cpu; char *aplic_name; uint32_t *aplic_cells; - unsigned long aplic_addr; MachineState *ms = MACHINE(s); - uint32_t aplic_m_phandle, aplic_s_phandle; - aplic_m_phandle = (*phandle)++; - aplic_s_phandle = (*phandle)++; aplic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 2); - /* M-level APLIC node */ for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) { aplic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]); - aplic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT); + aplic_cells[cpu * 2 + 1] = cpu_to_be32(m_mode ? IRQ_M_EXT : IRQ_S_EXT); } - aplic_addr = memmap[VIRT_APLIC_M].base + - (memmap[VIRT_APLIC_M].size * socket); + aplic_name = g_strdup_printf("/soc/aplic@%lx", aplic_addr); qemu_fdt_add_subnode(ms->fdt, aplic_name); qemu_fdt_setprop_string(ms->fdt, aplic_name, "compatible", "riscv,aplic"); qemu_fdt_setprop_cell(ms->fdt, aplic_name, "#interrupt-cells", FDT_APLIC_INT_CELLS); qemu_fdt_setprop(ms->fdt, aplic_name, "interrupt-controller", NULL, 0); + if (s->aia_type == VIRT_AIA_TYPE_APLIC) { qemu_fdt_setprop(ms->fdt, aplic_name, "interrupts-extended", aplic_cells, s->soc[socket].num_harts * sizeof(uint32_t) * 2); } else { - qemu_fdt_setprop_cell(ms->fdt, aplic_name, "msi-parent", - msi_m_phandle); + qemu_fdt_setprop_cell(ms->fdt, aplic_name, "msi-parent", msi_phandle); } + qemu_fdt_setprop_cells(ms->fdt, aplic_name, "reg", - 0x0, aplic_addr, 0x0, memmap[VIRT_APLIC_M].size); + 0x0, aplic_addr, 0x0, aplic_size); qemu_fdt_setprop_cell(ms->fdt, aplic_name, "riscv,num-sources", VIRT_IRQCHIP_NUM_SOURCES); - qemu_fdt_setprop_cell(ms->fdt, aplic_name, "riscv,children", - aplic_s_phandle); - qemu_fdt_setprop_cells(ms->fdt, aplic_name, "riscv,delegate", - aplic_s_phandle, 0x1, VIRT_IRQCHIP_NUM_SOURCES); + + if (aplic_child_phandle) { + qemu_fdt_setprop_cell(ms->fdt, aplic_name, "riscv,children", + aplic_child_phandle); + qemu_fdt_setprop_cells(ms->fdt, aplic_name, "riscv,delegate", + aplic_child_phandle, 0x1, VIRT_IRQCHIP_NUM_SOURCES); + } + riscv_socket_fdt_write_id(ms, aplic_name, socket); - qemu_fdt_setprop_cell(ms->fdt, aplic_name, "phandle", aplic_m_phandle); + qemu_fdt_setprop_cell(ms->fdt, aplic_name, "phandle", aplic_phandle); + g_free(aplic_name); + g_free(aplic_cells); +} - /* S-level APLIC node */ - for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) { - aplic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]); - aplic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT); +static void create_fdt_socket_aplic(RISCVVirtState *s, + const MemMapEntry *memmap, int socket, + uint32_t msi_m_phandle, + uint32_t msi_s_phandle, + uint32_t *phandle, + uint32_t *intc_phandles, + uint32_t *aplic_phandles) +{ + char *aplic_name; + unsigned long aplic_addr; + MachineState *ms = MACHINE(s); + uint32_t aplic_m_phandle, aplic_s_phandle; + + aplic_m_phandle = (*phandle)++; + aplic_s_phandle = (*phandle)++; + + if (!kvm_enabled()) { + /* M-level APLIC node */ + aplic_addr = memmap[VIRT_APLIC_M].base + + (memmap[VIRT_APLIC_M].size * socket); + create_fdt_one_aplic(s, socket, aplic_addr, memmap[VIRT_APLIC_M].size, + msi_m_phandle, intc_phandles, + aplic_m_phandle, aplic_s_phandle, + true); } + + /* S-level APLIC node */ aplic_addr = memmap[VIRT_APLIC_S].base + (memmap[VIRT_APLIC_S].size * socket); + create_fdt_one_aplic(s, socket, aplic_addr, memmap[VIRT_APLIC_S].size, + msi_s_phandle, intc_phandles, + aplic_s_phandle, 0, + false); + aplic_name = g_strdup_printf("/soc/aplic@%lx", aplic_addr); - qemu_fdt_add_subnode(ms->fdt, aplic_name); - qemu_fdt_setprop_string(ms->fdt, aplic_name, "compatible", "riscv,aplic"); - qemu_fdt_setprop_cell(ms->fdt, aplic_name, - "#interrupt-cells", FDT_APLIC_INT_CELLS); - qemu_fdt_setprop(ms->fdt, aplic_name, "interrupt-controller", NULL, 0); - if (s->aia_type == VIRT_AIA_TYPE_APLIC) { - qemu_fdt_setprop(ms->fdt, aplic_name, "interrupts-extended", - aplic_cells, s->soc[socket].num_harts * sizeof(uint32_t) * 2); - } else { - qemu_fdt_setprop_cell(ms->fdt, aplic_name, "msi-parent", - msi_s_phandle); - } - qemu_fdt_setprop_cells(ms->fdt, aplic_name, "reg", - 0x0, aplic_addr, 0x0, memmap[VIRT_APLIC_S].size); - qemu_fdt_setprop_cell(ms->fdt, aplic_name, "riscv,num-sources", - VIRT_IRQCHIP_NUM_SOURCES); - riscv_socket_fdt_write_id(ms, aplic_name, socket); - qemu_fdt_setprop_cell(ms->fdt, aplic_name, "phandle", aplic_s_phandle); if (!socket) { platform_bus_add_all_fdt_nodes(ms->fdt, aplic_name, @@ -722,7 +701,6 @@ static void create_fdt_socket_aplic(RISCVVirtState *s, g_free(aplic_name); - g_free(aplic_cells); aplic_phandles[socket] = aplic_s_phandle; } @@ -1163,16 +1141,20 @@ static DeviceState *virt_create_aia(RISCVVirtAIAType aia_type, int aia_guests, int i; hwaddr addr; uint32_t guest_bits; - DeviceState *aplic_m; - bool msimode = (aia_type == VIRT_AIA_TYPE_APLIC_IMSIC) ? true : false; + DeviceState *aplic_s = NULL; + DeviceState *aplic_m = NULL; + bool msimode = aia_type == VIRT_AIA_TYPE_APLIC_IMSIC; if (msimode) { - /* Per-socket M-level IMSICs */ - addr = memmap[VIRT_IMSIC_M].base + socket * VIRT_IMSIC_GROUP_MAX_SIZE; - for (i = 0; i < hart_count; i++) { - riscv_imsic_create(addr + i * IMSIC_HART_SIZE(0), - base_hartid + i, true, 1, - VIRT_IRQCHIP_NUM_MSIS); + if (!kvm_enabled()) { + /* Per-socket M-level IMSICs */ + addr = memmap[VIRT_IMSIC_M].base + + socket * VIRT_IMSIC_GROUP_MAX_SIZE; + for (i = 0; i < hart_count; i++) { + riscv_imsic_create(addr + i * IMSIC_HART_SIZE(0), + base_hartid + i, true, 1, + VIRT_IRQCHIP_NUM_MSIS); + } } /* Per-socket S-level IMSICs */ @@ -1185,29 +1167,29 @@ static DeviceState *virt_create_aia(RISCVVirtAIAType aia_type, int aia_guests, } } - /* Per-socket M-level APLIC */ - aplic_m = riscv_aplic_create( - memmap[VIRT_APLIC_M].base + socket * memmap[VIRT_APLIC_M].size, - memmap[VIRT_APLIC_M].size, + if (!kvm_enabled()) { + /* Per-socket M-level APLIC */ + aplic_m = riscv_aplic_create( + memmap[VIRT_APLIC_M].base + socket * memmap[VIRT_APLIC_M].size, + memmap[VIRT_APLIC_M].size, + (msimode) ? 0 : base_hartid, + (msimode) ? 0 : hart_count, + VIRT_IRQCHIP_NUM_SOURCES, + VIRT_IRQCHIP_NUM_PRIO_BITS, + msimode, true, NULL); + } + + /* Per-socket S-level APLIC */ + aplic_s = riscv_aplic_create( + memmap[VIRT_APLIC_S].base + socket * memmap[VIRT_APLIC_S].size, + memmap[VIRT_APLIC_S].size, (msimode) ? 0 : base_hartid, (msimode) ? 0 : hart_count, VIRT_IRQCHIP_NUM_SOURCES, VIRT_IRQCHIP_NUM_PRIO_BITS, - msimode, true, NULL); - - if (aplic_m) { - /* Per-socket S-level APLIC */ - riscv_aplic_create( - memmap[VIRT_APLIC_S].base + socket * memmap[VIRT_APLIC_S].size, - memmap[VIRT_APLIC_S].size, - (msimode) ? 0 : base_hartid, - (msimode) ? 0 : hart_count, - VIRT_IRQCHIP_NUM_SOURCES, - VIRT_IRQCHIP_NUM_PRIO_BITS, - msimode, false, aplic_m); - } + msimode, false, aplic_m); - return aplic_m; + return kvm_enabled() ? aplic_s : aplic_m; } static void create_platform_bus(RISCVVirtState *s, DeviceState *irqchip) From patchwork Thu Jul 13 08:43:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong-Xuan Wang X-Patchwork-Id: 13311577 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 638F0C001DC for ; Thu, 13 Jul 2023 08:46:39 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qJrw8-0006No-Ve; Thu, 13 Jul 2023 04:44:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qJrw4-0006MO-9h for qemu-devel@nongnu.org; Thu, 13 Jul 2023 04:44:30 -0400 Received: from mail-oi1-x234.google.com ([2607:f8b0:4864:20::234]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qJrw1-0006yl-0o for qemu-devel@nongnu.org; Thu, 13 Jul 2023 04:44:26 -0400 Received: by mail-oi1-x234.google.com with SMTP id 5614622812f47-38c35975545so497165b6e.1 for ; Thu, 13 Jul 2023 01:44:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1689237863; x=1691829863; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=bRqQKtep7JWLgclLdUkli6/fChTnwWs+kTdTq5m30jU=; b=CTlKktPs+hY0cgE7CS2PUANVjOXYIGn9DpU2fh7mMW2HZEQy1ymS0r+QkdZU1bdjGB P2H/jtF17Wexhc+P++LOh0FtkHw9ToVJYYkmJ2R+46Pq+yCVcBYwEa6sNddZeYTAQG93 F9Re91F5WlS3VTETLFEyPWpb8ZNnRWYnopTN/k1Y1tVgM4Q3WhYSiZEb5INzk1ZCWmZy Ht6uHxlWh/bDhXre4KQ3q292Ebz8ayR2ctMZrSgTwIWEJnL9HCOIKtH7CssMxSprI1gQ basdybVGjqRuPOKpJ+sCLh4s5/DYF6oxFOrVQwIq/hKuKNAcy/cCYqRf8VrmJrBEBlX/ xwhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689237863; x=1691829863; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=bRqQKtep7JWLgclLdUkli6/fChTnwWs+kTdTq5m30jU=; b=JONJNMCwiO/Ues9F/bZGJKEcLQEbIrM1aMcEcpRRHoSuE59iOr11Ma9OxBl4aSE36a syBTh2A4kF+9+s3WRRnBbpvy1AgI7zpN2XgdMVvIIf+At2CthlcFNQn71sVTfk1yEQND xHer2KdCnJqQTODAz5hnzxYJ5pOwzXVQ4eE5YJ5o0SyIteardJol09jQp2HD8Vhse1s9 aPnhSTiRNhyVfmg/0XbgHniu9phf+dg8FrXAhwc9klilxu94vhjcuOaco8FKS6VHO8se ScJsLWKdD1AUG9lVwb+ZvALsePYzxMNfOmOvRAkYAZxinbx1WfwmyKQOPgBCq2SmCfdb 316Q== X-Gm-Message-State: ABy/qLYz0IcHhw7caD7gRyIp0n0t3pdTtF6+TvJCWFoUJA6wZFDL6KUW avZO3pu3OuubMq98QsdzpUWUhlzeny5YrQ8JrdewcSynqC5mliwCq0IcCEO66nZXGPM68B2exYb sRX9vV9/2R8GuR/reFnKJ6LqELeuxsW6h/Q2HnhhP860LeHAMcYEJv/Jcnr3N/AdJwDDQrcXZ6w aUo+LB X-Google-Smtp-Source: APBJJlEwvjQxj7TZ0LJFULoNQDbfGWViUSUPEKbUpl3Uk7rotzEbi31NqIdsDVgZwphMV9Wf+7w6Vw== X-Received: by 2002:a05:6358:70d:b0:133:26e4:afb2 with SMTP id e13-20020a056358070d00b0013326e4afb2mr2045730rwj.3.1689237863041; Thu, 13 Jul 2023 01:44:23 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id c10-20020a63724a000000b0055386b1415dsm4989198pgn.51.2023.07.13.01.44.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jul 2023 01:44:22 -0700 (PDT) From: Yong-Xuan Wang To: qemu-devel@nongnu.org, qemu-riscv@nongnu.org Cc: rkanwal@rivosinc.com, anup@brainfault.org, dbarboza@ventanamicro.com, ajones@ventanamicro.com, atishp@atishpatra.org, vincent.chen@sifive.com, greentime.hu@sifive.com, frank.chang@sifive.com, jim.shu@sifive.com, Yong-Xuan Wang , Palmer Dabbelt , Alistair Francis , Bin Meng , Weiwei Li , Liu Zhiwei , Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v5 2/5] target/riscv: check the in-kernel irqchip support Date: Thu, 13 Jul 2023 08:43:54 +0000 Message-Id: <20230713084405.24545-3-yongxuan.wang@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230713084405.24545-1-yongxuan.wang@sifive.com> References: <20230713084405.24545-1-yongxuan.wang@sifive.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::234; envelope-from=yongxuan.wang@sifive.com; helo=mail-oi1-x234.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We check the in-kernel irqchip support when using KVM acceleration. Signed-off-by: Yong-Xuan Wang Reviewed-by: Jim Shu Reviewed-by: Daniel Henrique Barboza Reviewed-by: Andrew Jones --- target/riscv/kvm.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 9d8a8982f9..005e054604 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -914,7 +914,15 @@ int kvm_arch_init(MachineState *ms, KVMState *s) int kvm_arch_irqchip_create(KVMState *s) { - return 0; + if (kvm_kernel_irqchip_split()) { + error_report("-machine kernel_irqchip=split is not supported on RISC-V."); + exit(1); + } + + /* + * We can create the VAIA using the newer device control API. + */ + return kvm_check_extension(s, KVM_CAP_DEVICE_CTRL); } int kvm_arch_process_async_events(CPUState *cs) From patchwork Thu Jul 13 08:43:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong-Xuan Wang X-Patchwork-Id: 13311576 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5E084EB64DD for ; Thu, 13 Jul 2023 08:46:38 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qJrwA-0006OI-Ck; Thu, 13 Jul 2023 04:44:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qJrw8-0006Nm-SF for qemu-devel@nongnu.org; Thu, 13 Jul 2023 04:44:32 -0400 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qJrw6-0006za-E0 for qemu-devel@nongnu.org; Thu, 13 Jul 2023 04:44:32 -0400 Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-666ed230c81so510126b3a.0 for ; Thu, 13 Jul 2023 01:44:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1689237868; x=1691829868; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=TInFTBHuF+SmzJxy3gMWFeCGYHH0/coaMMeOCtsjHH0=; b=hm77oFoTpkB0AvitGeC0SfsSWGXi5wE5Tdhh83+2NvgRNh5sPBhI49UYTX8fO5PqTr +Q1VtFS/e8XdfPB3CO84C/G3SyrTnHVvwvajtkR4mdWBdNqdtSj4PghpZixRPMsst9GS 7b4iC4YLl1e2ex46d6S1R3BFCoBNUA9YfrnEeZ0AmAb6Hi1kVMYm/spLvAN/+w7eZnXZ UTwbeCocu1K6bPbQswkxlUFtKFHXY/TCy/5ORti2wkuIisBanXKdii4UmxERmd9babN+ 7yARiAlvFvDVR0POBeYnrkqaKy8XyCwgVLIKAwU46Hl1+5dSlib1zPwQfJq2J60KfXT8 MKbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689237868; x=1691829868; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=TInFTBHuF+SmzJxy3gMWFeCGYHH0/coaMMeOCtsjHH0=; b=Mf3Hy6+iVCUernE6up/0uaYOm8bpkBPjxTLP9Bs87Zv+qlZE8IML5lwBSY3662NODw OR/DHhCIkNwD4poERY9nTy8qr8+eI+2TarJ+wtMuvI8xrHs7mvQ2ZTL+jNqgtJJGaFuY +706gA6H0U8ZeeLWrdWasrggoec0fU4DnmIWBWADiBa0Q7ZvqEMsqqe9qLvjq9xpFcUp 3ew8URKBTFq2XacpOLVJSWId9pXm068uBOXpwT2kNJCRuSxqX+fl9GzXK874vJbLpClR r0hjxReNKbq6SOmKGWGwChmoQrZeu4KhLUY7iekkyjRXZXEMYMIUbohcV0zVbQUl+V1S LMbg== X-Gm-Message-State: ABy/qLaIrSVDJ6SLWFn+3TY7OkPNiI325lALRNXXgG2w6bJc2nOP+FFy mO5Ceo0BCWm9zJeeanOgi2dCXhO7MoELWAXD1b6SVFhYXFV+1yC9aqEMp+v+ls1yY+YWInPUiby gabQ+WkcVSSZJib/P+o3E2BI30ullYYto/MtS/lqGltIndkAYddfYhu4CaOGf1mAulEW7RCpzUT p+FXOJ X-Google-Smtp-Source: APBJJlGIIS9SWvwBr06+S3cPMiWK4R0LzvHQWvnQYxA5wg/Eswfxg2NUcoGvixGAV5h74X4bGWgY2A== X-Received: by 2002:a05:6a20:244d:b0:122:4a16:dfa4 with SMTP id t13-20020a056a20244d00b001224a16dfa4mr1023154pzc.10.1689237868118; Thu, 13 Jul 2023 01:44:28 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id c10-20020a63724a000000b0055386b1415dsm4989198pgn.51.2023.07.13.01.44.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jul 2023 01:44:27 -0700 (PDT) From: Yong-Xuan Wang To: qemu-devel@nongnu.org, qemu-riscv@nongnu.org Cc: rkanwal@rivosinc.com, anup@brainfault.org, dbarboza@ventanamicro.com, ajones@ventanamicro.com, atishp@atishpatra.org, vincent.chen@sifive.com, greentime.hu@sifive.com, frank.chang@sifive.com, jim.shu@sifive.com, Yong-Xuan Wang , Palmer Dabbelt , Alistair Francis , Bin Meng , Weiwei Li , Liu Zhiwei , Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v5 3/5] target/riscv: Create an KVM AIA irqchip Date: Thu, 13 Jul 2023 08:43:55 +0000 Message-Id: <20230713084405.24545-4-yongxuan.wang@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230713084405.24545-1-yongxuan.wang@sifive.com> References: <20230713084405.24545-1-yongxuan.wang@sifive.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::42e; envelope-from=yongxuan.wang@sifive.com; helo=mail-pf1-x42e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We create a vAIA chip by using the KVM_DEV_TYPE_RISCV_AIA and then set up the chip with the KVM_DEV_RISCV_AIA_GRP_* APIs. Signed-off-by: Yong-Xuan Wang Reviewed-by: Jim Shu Reviewed-by: Daniel Henrique Barboza Reviewed-by: Andrew Jones --- target/riscv/kvm.c | 160 +++++++++++++++++++++++++++++++++++++++ target/riscv/kvm_riscv.h | 6 ++ 2 files changed, 166 insertions(+) diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c index 005e054604..64156c15ec 100644 --- a/target/riscv/kvm.c +++ b/target/riscv/kvm.c @@ -36,6 +36,7 @@ #include "exec/address-spaces.h" #include "hw/boards.h" #include "hw/irq.h" +#include "hw/intc/riscv_imsic.h" #include "qemu/log.h" #include "hw/loader.h" #include "kvm_riscv.h" @@ -43,6 +44,7 @@ #include "chardev/char-fe.h" #include "migration/migration.h" #include "sysemu/runstate.h" +#include "hw/riscv/numa.h" static uint64_t kvm_riscv_reg_id(CPURISCVState *env, uint64_t type, uint64_t idx) @@ -1026,3 +1028,161 @@ bool kvm_arch_cpu_check_are_resettable(void) void kvm_arch_accel_class_init(ObjectClass *oc) { } + +char *kvm_aia_mode_str(uint64_t aia_mode) +{ + const char *val; + + switch (aia_mode) { + case KVM_DEV_RISCV_AIA_MODE_EMUL: + return "emul"; + case KVM_DEV_RISCV_AIA_MODE_HWACCEL: + return "hwaccel"; + case KVM_DEV_RISCV_AIA_MODE_AUTO: + default: + return "auto"; + }; +} + +void kvm_riscv_aia_create(MachineState *machine, + uint64_t aia_mode, uint64_t group_shift, + uint64_t aia_irq_num, uint64_t aia_msi_num, + uint64_t aplic_base, uint64_t imsic_base, + uint64_t guest_num) +{ + int ret, i; + int aia_fd = -1; + uint64_t default_aia_mode; + uint64_t socket_count = riscv_socket_count(machine); + uint64_t max_hart_per_socket = 0; + uint64_t socket, base_hart, hart_count, socket_imsic_base, imsic_addr; + uint64_t socket_bits, hart_bits, guest_bits; + + aia_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_RISCV_AIA, false); + + if (aia_fd < 0) { + error_report("Unable to create in-kernel irqchip"); + exit(1); + } + + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_MODE, + &default_aia_mode, false, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to get current KVM AIA mode"); + exit(1); + } + qemu_log("KVM AIA: default mode is %s\n", + kvm_aia_mode_str(default_aia_mode)); + + if (default_aia_mode != aia_mode) { + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_MODE, + &aia_mode, true, NULL); + if (ret < 0) + warn_report("KVM AIA: failed to set KVM AIA mode"); + else + qemu_log("KVM AIA: set current mode to %s\n", + kvm_aia_mode_str(aia_mode)); + } + + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_SRCS, + &aia_irq_num, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set number of input irq lines"); + exit(1); + } + + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_IDS, + &aia_msi_num, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set number of msi"); + exit(1); + } + + socket_bits = find_last_bit(&socket_count, BITS_PER_LONG) + 1; + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_GROUP_BITS, + &socket_bits, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set group_bits"); + exit(1); + } + + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_GROUP_SHIFT, + &group_shift, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set group_shift"); + exit(1); + } + + guest_bits = guest_num == 0 ? 0 : + find_last_bit(&guest_num, BITS_PER_LONG) + 1; + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_GUEST_BITS, + &guest_bits, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set guest_bits"); + exit(1); + } + + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_ADDR, + KVM_DEV_RISCV_AIA_ADDR_APLIC, + &aplic_base, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set the base address of APLIC"); + exit(1); + } + + for (socket = 0; socket < socket_count; socket++) { + socket_imsic_base = imsic_base + socket * (1U << group_shift); + hart_count = riscv_socket_hart_count(machine, socket); + base_hart = riscv_socket_first_hartid(machine, socket); + + if (max_hart_per_socket < hart_count) { + max_hart_per_socket = hart_count; + } + + for (i = 0; i < hart_count; i++) { + imsic_addr = socket_imsic_base + i * IMSIC_HART_SIZE(guest_bits); + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_ADDR, + KVM_DEV_RISCV_AIA_ADDR_IMSIC(i + base_hart), + &imsic_addr, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set the address of IMSICs"); + exit(1); + } + } + } + + hart_bits = find_last_bit(&max_hart_per_socket, BITS_PER_LONG) + 1; + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG, + KVM_DEV_RISCV_AIA_CONFIG_HART_BITS, + &hart_bits, true, NULL); + if (ret < 0) { + error_report("KVM AIA: failed to set hart_bits"); + exit(1); + } + + if (kvm_has_gsi_routing()) { + for (uint64_t idx = 0; idx < aia_irq_num + 1; ++idx) { + /* KVM AIA only has one APLIC instance */ + kvm_irqchip_add_irq_route(kvm_state, idx, 0, idx); + } + kvm_gsi_routing_allowed = true; + kvm_irqchip_commit_routes(kvm_state); + } + + ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CTRL, + KVM_DEV_RISCV_AIA_CTRL_INIT, + NULL, true, NULL); + if (ret < 0) { + error_report("KVM AIA: initialized fail"); + exit(1); + } + + kvm_msi_via_irqfd_allowed = kvm_irqfds_enabled(); +} diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h index e3ba935808..c6745dd29a 100644 --- a/target/riscv/kvm_riscv.h +++ b/target/riscv/kvm_riscv.h @@ -22,5 +22,11 @@ void kvm_riscv_init_user_properties(Object *cpu_obj); void kvm_riscv_reset_vcpu(RISCVCPU *cpu); void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level); +char *kvm_aia_mode_str(uint64_t aia_mode); +void kvm_riscv_aia_create(MachineState *machine, + uint64_t aia_mode, uint64_t group_shift, + uint64_t aia_irq_num, uint64_t aia_msi_num, + uint64_t aplic_base, uint64_t imsic_base, + uint64_t guest_num); #endif From patchwork Thu Jul 13 08:43:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong-Xuan Wang X-Patchwork-Id: 13311572 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 08BCCC001DC for ; Thu, 13 Jul 2023 08:46:14 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qJrwE-0006Qy-FH; Thu, 13 Jul 2023 04:44:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qJrwC-0006Q9-Gm for qemu-devel@nongnu.org; Thu, 13 Jul 2023 04:44:36 -0400 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qJrwA-00070J-Ei for qemu-devel@nongnu.org; Thu, 13 Jul 2023 04:44:36 -0400 Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-666edfc50deso284093b3a.0 for ; Thu, 13 Jul 2023 01:44:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1689237873; x=1691829873; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=R5Lwp6JGXFIhvrTHMOVoigMo/sy9lxlSF/Jzo4dn8GE=; b=Q81w/fVFX0+9SHZnwqTWIHRPqkUYbMJ+3ib46P29sYubjO8Yet89MQNUw2OpxqYdvn YgLDgGiQrPETtGRXI7Ft2SLlRG2LXCAFJAhP2ok3W3b92qFUKUUtBXAhqQvr2HYunnuc aOjwbJScPEIncprZ3sVF5ulcMdMSNtHKuUhv03qhhDFSkT9Zi7vhtzv5K+NShko4HiBy dw1KXu4wg3fZ7EKG8HI3RZkGQrTXB8r9pLyVHuzEW5zF+qrJJn5zrN6NI9hhuwcGrq0T R32TG1ef3e4BpXfBslUU/y75an5+/Q8YVw86FSSQRt+2O/hPKCHF/g33Jmby+rLZcvAp cp+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689237873; x=1691829873; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=R5Lwp6JGXFIhvrTHMOVoigMo/sy9lxlSF/Jzo4dn8GE=; b=dx/B9jRDUUOSHrXhxHsjiafD4nvlQ3PjFSvNgA2V9Wn178EVYHJ+c0udyMggH0YzZZ JjQjb3t08fC4vwJQXr7Fjc++HI2uddn9xag0rxYAgt0Ht/CdOqOby8vL1z0VprQJAbTc wcz2V1dbDu4Ff0Jg31FADRhiNkkACYuJrd1CpXNkaf4XP9sDfQwBAWVNizoe9VX+oyQP Ya8UOWxP/zrTqo1m18zSl8dchFcki/vS1cAwucSn8drGdOJFsJJ+t60se0X1iW9FlAQU 8y96Cf0kMo606BJVxPZVYbagZgZcVRJ4rv46mxT2J/MBh+XF0EsBKT4Rjg0urLND5kt0 2nZw== X-Gm-Message-State: ABy/qLYLkocPbIUtXXreO4fZizbqJHbPodjbrsIvV26wzM9ZCV1SDDLU 4cSaPmmpgyHqFo38zym6wXM+Usg6uRSssN7smGk69iBLVK14ua1jCAVFZuMjvqw2SBI2UMfaxGT 8xjDY/2YUAGIlUEKXw4AuCIechiekFr4UWCccrjw6rF2jeD6askuwq9LZGwuC2jD/RPorp1PkIR aXssz3 X-Google-Smtp-Source: APBJJlEvxDZxnLUdCWLrGW+4Y2JnBhDhaldI8wRc/bSpP/HOyMKN3LObz/+lXQ46YibT0158AdmX5g== X-Received: by 2002:a05:6a20:12cc:b0:12f:86f1:b16f with SMTP id v12-20020a056a2012cc00b0012f86f1b16fmr5783216pzg.1.1689237872806; Thu, 13 Jul 2023 01:44:32 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id c10-20020a63724a000000b0055386b1415dsm4989198pgn.51.2023.07.13.01.44.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jul 2023 01:44:32 -0700 (PDT) From: Yong-Xuan Wang To: qemu-devel@nongnu.org, qemu-riscv@nongnu.org Cc: rkanwal@rivosinc.com, anup@brainfault.org, dbarboza@ventanamicro.com, ajones@ventanamicro.com, atishp@atishpatra.org, vincent.chen@sifive.com, greentime.hu@sifive.com, frank.chang@sifive.com, jim.shu@sifive.com, Yong-Xuan Wang , Alistair Francis , Mayuresh Chitale , Tommy Wu Subject: [PATCH v5 4/5] target/riscv: update APLIC and IMSIC to support KVM AIA Date: Thu, 13 Jul 2023 08:43:56 +0000 Message-Id: <20230713084405.24545-5-yongxuan.wang@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230713084405.24545-1-yongxuan.wang@sifive.com> References: <20230713084405.24545-1-yongxuan.wang@sifive.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::42b; envelope-from=yongxuan.wang@sifive.com; helo=mail-pf1-x42b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org KVM AIA can't emulate APLIC only. When "aia=aplic" parameter is passed, APLIC devices is emulated by QEMU. For "aia=aplic-imsic", remove the mmio operations of APLIC when using KVM AIA and send wired interrupt signal via KVM_IRQ_LINE API. After KVM AIA enabled, MSI messages are delivered by KVM_SIGNAL_MSI API when the IMSICs receive mmio write requests. Signed-off-by: Yong-Xuan Wang Reviewed-by: Jim Shu Reviewed-by: Daniel Henrique Barboza Reviewed-by: Andrew Jones --- hw/intc/riscv_aplic.c | 56 ++++++++++++++++++++++++++++++------------- hw/intc/riscv_imsic.c | 25 +++++++++++++++---- 2 files changed, 61 insertions(+), 20 deletions(-) diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c index 4bdc6a5d1a..eab5db053b 100644 --- a/hw/intc/riscv_aplic.c +++ b/hw/intc/riscv_aplic.c @@ -31,6 +31,7 @@ #include "hw/irq.h" #include "target/riscv/cpu.h" #include "sysemu/sysemu.h" +#include "sysemu/kvm.h" #include "migration/vmstate.h" #define APLIC_MAX_IDC (1UL << 14) @@ -148,6 +149,15 @@ #define APLIC_IDC_CLAIMI 0x1c +/* + * KVM AIA only supports APLIC MSI, fallback to QEMU emulation if we want to use + * APLIC Wired. + */ +static bool is_kvm_aia(bool msimode) +{ + return kvm_irqchip_in_kernel() && msimode; +} + static uint32_t riscv_aplic_read_input_word(RISCVAPLICState *aplic, uint32_t word) { @@ -471,6 +481,11 @@ static uint32_t riscv_aplic_idc_claimi(RISCVAPLICState *aplic, uint32_t idc) return topi; } +static void riscv_kvm_aplic_request(void *opaque, int irq, int level) +{ + kvm_set_irq(kvm_state, irq, !!level); +} + static void riscv_aplic_request(void *opaque, int irq, int level) { bool update = false; @@ -801,29 +816,35 @@ static void riscv_aplic_realize(DeviceState *dev, Error **errp) uint32_t i; RISCVAPLICState *aplic = RISCV_APLIC(dev); - aplic->bitfield_words = (aplic->num_irqs + 31) >> 5; - aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs); - aplic->state = g_new0(uint32_t, aplic->num_irqs); - aplic->target = g_new0(uint32_t, aplic->num_irqs); - if (!aplic->msimode) { - for (i = 0; i < aplic->num_irqs; i++) { - aplic->target[i] = 1; + if (!is_kvm_aia(aplic->msimode)) { + aplic->bitfield_words = (aplic->num_irqs + 31) >> 5; + aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs); + aplic->state = g_new0(uint32_t, aplic->num_irqs); + aplic->target = g_new0(uint32_t, aplic->num_irqs); + if (!aplic->msimode) { + for (i = 0; i < aplic->num_irqs; i++) { + aplic->target[i] = 1; + } } - } - aplic->idelivery = g_new0(uint32_t, aplic->num_harts); - aplic->iforce = g_new0(uint32_t, aplic->num_harts); - aplic->ithreshold = g_new0(uint32_t, aplic->num_harts); + aplic->idelivery = g_new0(uint32_t, aplic->num_harts); + aplic->iforce = g_new0(uint32_t, aplic->num_harts); + aplic->ithreshold = g_new0(uint32_t, aplic->num_harts); - memory_region_init_io(&aplic->mmio, OBJECT(dev), &riscv_aplic_ops, aplic, - TYPE_RISCV_APLIC, aplic->aperture_size); - sysbus_init_mmio(SYS_BUS_DEVICE(dev), &aplic->mmio); + memory_region_init_io(&aplic->mmio, OBJECT(dev), &riscv_aplic_ops, + aplic, TYPE_RISCV_APLIC, aplic->aperture_size); + sysbus_init_mmio(SYS_BUS_DEVICE(dev), &aplic->mmio); + } /* * Only root APLICs have hardware IRQ lines. All non-root APLICs * have IRQ lines delegated by their parent APLIC. */ if (!aplic->parent) { - qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs); + if (is_kvm_aia(aplic->msimode)) { + qdev_init_gpio_in(dev, riscv_kvm_aplic_request, aplic->num_irqs); + } else { + qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs); + } } /* Create output IRQ lines for non-MSI mode */ @@ -958,7 +979,10 @@ DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size, qdev_prop_set_bit(dev, "mmode", mmode); sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal); - sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr); + + if (!is_kvm_aia(msimode)) { + sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr); + } if (parent) { riscv_aplic_add_child(parent, dev); diff --git a/hw/intc/riscv_imsic.c b/hw/intc/riscv_imsic.c index fea3385b51..760dbddcf7 100644 --- a/hw/intc/riscv_imsic.c +++ b/hw/intc/riscv_imsic.c @@ -32,6 +32,7 @@ #include "target/riscv/cpu.h" #include "target/riscv/cpu_bits.h" #include "sysemu/sysemu.h" +#include "sysemu/kvm.h" #include "migration/vmstate.h" #define IMSIC_MMIO_PAGE_LE 0x00 @@ -283,6 +284,20 @@ static void riscv_imsic_write(void *opaque, hwaddr addr, uint64_t value, goto err; } +#if defined(CONFIG_KVM) + if (kvm_irqchip_in_kernel()) { + struct kvm_msi msi; + + msi.address_lo = extract64(imsic->mmio.addr + addr, 0, 32); + msi.address_hi = extract64(imsic->mmio.addr + addr, 32, 32); + msi.data = le32_to_cpu(value); + + kvm_vm_ioctl(kvm_state, KVM_SIGNAL_MSI, &msi); + + return; + } +#endif + /* Writes only supported for MSI little-endian registers */ page = addr >> IMSIC_MMIO_PAGE_SHIFT; if ((addr & (IMSIC_MMIO_PAGE_SZ - 1)) == IMSIC_MMIO_PAGE_LE) { @@ -320,10 +335,12 @@ static void riscv_imsic_realize(DeviceState *dev, Error **errp) CPUState *cpu = cpu_by_arch_id(imsic->hartid); CPURISCVState *env = cpu ? cpu->env_ptr : NULL; - imsic->num_eistate = imsic->num_pages * imsic->num_irqs; - imsic->eidelivery = g_new0(uint32_t, imsic->num_pages); - imsic->eithreshold = g_new0(uint32_t, imsic->num_pages); - imsic->eistate = g_new0(uint32_t, imsic->num_eistate); + if (!kvm_irqchip_in_kernel()) { + imsic->num_eistate = imsic->num_pages * imsic->num_irqs; + imsic->eidelivery = g_new0(uint32_t, imsic->num_pages); + imsic->eithreshold = g_new0(uint32_t, imsic->num_pages); + imsic->eistate = g_new0(uint32_t, imsic->num_eistate); + } memory_region_init_io(&imsic->mmio, OBJECT(dev), &riscv_imsic_ops, imsic, TYPE_RISCV_IMSIC, From patchwork Thu Jul 13 08:43:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yong-Xuan Wang X-Patchwork-Id: 13311575 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 06EEFC0015E for ; Thu, 13 Jul 2023 08:46:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qJrwK-0006RS-3z; Thu, 13 Jul 2023 04:44:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qJrwI-0006R3-5e for qemu-devel@nongnu.org; Thu, 13 Jul 2023 04:44:42 -0400 Received: from mail-qk1-x72c.google.com ([2607:f8b0:4864:20::72c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qJrwG-00070g-6T for qemu-devel@nongnu.org; Thu, 13 Jul 2023 04:44:41 -0400 Received: by mail-qk1-x72c.google.com with SMTP id af79cd13be357-76728ae3162so46175985a.3 for ; Thu, 13 Jul 2023 01:44:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1689237878; x=1691829878; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=djwjqUPMIlHHN6FZYr+Mw7HTbl7A9BLuFx65r3ISBNo=; b=D7GkGVU2bSU3NKD/DQr2QAXUzC50XWeTsIopMCxKNiAPUvJTWj1WqUxvL6QaqJunrL zzmIbHKn6HgLQSS0lp9Qmgxym5jAhbxloE6nupc75UX6D6Qoa2i9+Lto+JZYsqdqx/O6 DW92U7eMjUjlid0q2RGIJGJBMqqdgepV93GkiB/3GeQt0/uvJcN8B6GFUo8sZQlUTKdU J/jhYLQaK6BWAiY4RiaF2UjKAhcFDXmlUivjUs78hjS5awa5/HH/ZPsINsucH+fGq3MK BNTRNwOvZXwbwKuduyx3ZkUw4LRGZes79wPtdNdOOPRH+Y/2r+viBkstIfNJ/bRsAfLN ga2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689237878; x=1691829878; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=djwjqUPMIlHHN6FZYr+Mw7HTbl7A9BLuFx65r3ISBNo=; b=EAK4974AGV1No0iO5T6Qk+m+6TIsQft8VU86LhvwFB8riQ9013qla9n3xvEHuKZh0t xp+QNj0mC8tDYMEh//UN+xmtFhtT/cwMGjBqrR8RG4BNwF77h8kHYUjjvcxCJZf0CxVx WIS+SnX4yXDAsbtRxW47O9iEOC0E62DRp8IVZ5QViWEr8c4Yro0KVouWOmwxvtxCRjv2 EztZ0dgJK39rbHUEDswHJ9fIRlKerbGS2gzHI9vWdJYJLtdsT+3zGD9LGrdt+NMfJP/Q oX0e/+NjdM7kXvtqtKu0aFuOzkX9hpnd7Rl3xiFeaCa68q6euwpF76htU/6fdJePw+fi dwyw== X-Gm-Message-State: ABy/qLa29A7sy23Oy+I396fbIU0GCpROo92vbOoPRA7Fnuc/nmtg6vqg dk+rOTDLGD9iWLIz5+cfXlfGCZRG7i641Ylbnob1WyHu8dOlrGFMzN/2g5FR+CShjHGgTA9cJJ+ wfikI2edcjZcz6s3tktHb9G3iT/Su1ogV47DRTLXAgH0i0xRDxQgkiPpVP3mPs4kCsy+G1JCjIr Ta0VPD X-Google-Smtp-Source: APBJJlEhb9fc7ih6eqP3c6AXyvLUeig0dUJFa4R2Kz7+l0rx5LWcdB+Zn3QxGj9fi0GE2v6rmqLCcg== X-Received: by 2002:a05:620a:2546:b0:767:a1ac:3c74 with SMTP id s6-20020a05620a254600b00767a1ac3c74mr1120252qko.26.1689237877838; Thu, 13 Jul 2023 01:44:37 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id c10-20020a63724a000000b0055386b1415dsm4989198pgn.51.2023.07.13.01.44.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jul 2023 01:44:37 -0700 (PDT) From: Yong-Xuan Wang To: qemu-devel@nongnu.org, qemu-riscv@nongnu.org Cc: rkanwal@rivosinc.com, anup@brainfault.org, dbarboza@ventanamicro.com, ajones@ventanamicro.com, atishp@atishpatra.org, vincent.chen@sifive.com, greentime.hu@sifive.com, frank.chang@sifive.com, jim.shu@sifive.com, Yong-Xuan Wang , Palmer Dabbelt , Alistair Francis , Bin Meng , Weiwei Li , Liu Zhiwei Subject: [PATCH v5 5/5] target/riscv: select KVM AIA in riscv virt machine Date: Thu, 13 Jul 2023 08:43:57 +0000 Message-Id: <20230713084405.24545-6-yongxuan.wang@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230713084405.24545-1-yongxuan.wang@sifive.com> References: <20230713084405.24545-1-yongxuan.wang@sifive.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::72c; envelope-from=yongxuan.wang@sifive.com; helo=mail-qk1-x72c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Select KVM AIA when the host kernel has in-kernel AIA chip support. Since KVM AIA only has one APLIC instance, we map the QEMU APLIC devices to KVM APLIC. We also extend virt machine to specify the KVM AIA mode. The "kvm-aia" parameter is passed along with machine name in QEMU command-line. 1) "kvm-aia=emul": IMSIC is emulated by hypervisor 2) "kvm-aia=hwaccel": use hardware guest IMSIC 3) "kvm-aia=auto": use the hardware guest IMSICs whenever available otherwise we fallback to software emulation. Signed-off-by: Yong-Xuan Wang Reviewed-by: Jim Shu Reviewed-by: Daniel Henrique Barboza Reviewed-by: Andrew Jones --- hw/riscv/virt.c | 125 ++++++++++++++++++++++++++++++---------- include/hw/riscv/virt.h | 1 + 2 files changed, 97 insertions(+), 29 deletions(-) diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 26b3aff28e..b74142d978 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -35,6 +35,7 @@ #include "hw/riscv/virt.h" #include "hw/riscv/boot.h" #include "hw/riscv/numa.h" +#include "kvm_riscv.h" #include "hw/intc/riscv_aclint.h" #include "hw/intc/riscv_aplic.h" #include "hw/intc/riscv_imsic.h" @@ -75,6 +76,12 @@ #error "Can't accomodate all IMSIC groups in address space" #endif +/* KVM AIA only supports APLIC MSI. APLIC Wired is always emulated by QEMU. */ +static bool virt_use_kvm_aia(RISCVVirtState *s) +{ + return kvm_irqchip_in_kernel() && s->aia_type == VIRT_AIA_TYPE_APLIC_IMSIC; +} + static const MemMapEntry virt_memmap[] = { [VIRT_DEBUG] = { 0x0, 0x100 }, [VIRT_MROM] = { 0x1000, 0xf000 }, @@ -609,16 +616,16 @@ static void create_fdt_one_aplic(RISCVVirtState *s, int socket, uint32_t *intc_phandles, uint32_t aplic_phandle, uint32_t aplic_child_phandle, - bool m_mode) + bool m_mode, int num_harts) { int cpu; char *aplic_name; uint32_t *aplic_cells; MachineState *ms = MACHINE(s); - aplic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 2); + aplic_cells = g_new0(uint32_t, num_harts * 2); - for (cpu = 0; cpu < s->soc[socket].num_harts; cpu++) { + for (cpu = 0; cpu < num_harts; cpu++) { aplic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]); aplic_cells[cpu * 2 + 1] = cpu_to_be32(m_mode ? IRQ_M_EXT : IRQ_S_EXT); } @@ -632,7 +639,7 @@ static void create_fdt_one_aplic(RISCVVirtState *s, int socket, if (s->aia_type == VIRT_AIA_TYPE_APLIC) { qemu_fdt_setprop(ms->fdt, aplic_name, "interrupts-extended", - aplic_cells, s->soc[socket].num_harts * sizeof(uint32_t) * 2); + aplic_cells, num_harts * sizeof(uint32_t) * 2); } else { qemu_fdt_setprop_cell(ms->fdt, aplic_name, "msi-parent", msi_phandle); } @@ -662,7 +669,8 @@ static void create_fdt_socket_aplic(RISCVVirtState *s, uint32_t msi_s_phandle, uint32_t *phandle, uint32_t *intc_phandles, - uint32_t *aplic_phandles) + uint32_t *aplic_phandles, + int num_harts) { char *aplic_name; unsigned long aplic_addr; @@ -679,7 +687,7 @@ static void create_fdt_socket_aplic(RISCVVirtState *s, create_fdt_one_aplic(s, socket, aplic_addr, memmap[VIRT_APLIC_M].size, msi_m_phandle, intc_phandles, aplic_m_phandle, aplic_s_phandle, - true); + true, num_harts); } /* S-level APLIC node */ @@ -688,7 +696,7 @@ static void create_fdt_socket_aplic(RISCVVirtState *s, create_fdt_one_aplic(s, socket, aplic_addr, memmap[VIRT_APLIC_S].size, msi_s_phandle, intc_phandles, aplic_s_phandle, 0, - false); + false, num_harts); aplic_name = g_strdup_printf("/soc/aplic@%lx", aplic_addr); @@ -772,34 +780,48 @@ static void create_fdt_sockets(RISCVVirtState *s, const MemMapEntry *memmap, *msi_pcie_phandle = msi_s_phandle; } - phandle_pos = ms->smp.cpus; - for (socket = (socket_count - 1); socket >= 0; socket--) { - phandle_pos -= s->soc[socket].num_harts; + /* KVM AIA only has one APLIC instance */ + if (virt_use_kvm_aia(s)) { + create_fdt_socket_aplic(s, memmap, 0, + msi_m_phandle, msi_s_phandle, phandle, + &intc_phandles[0], xplic_phandles, ms->smp.cpus); + } else { + phandle_pos = ms->smp.cpus; + for (socket = (socket_count - 1); socket >= 0; socket--) { + phandle_pos -= s->soc[socket].num_harts; - if (s->aia_type == VIRT_AIA_TYPE_NONE) { - create_fdt_socket_plic(s, memmap, socket, phandle, - &intc_phandles[phandle_pos], xplic_phandles); - } else { - create_fdt_socket_aplic(s, memmap, socket, - msi_m_phandle, msi_s_phandle, phandle, - &intc_phandles[phandle_pos], xplic_phandles); + if (s->aia_type == VIRT_AIA_TYPE_NONE) { + create_fdt_socket_plic(s, memmap, socket, phandle, + &intc_phandles[phandle_pos], xplic_phandles); + } else { + create_fdt_socket_aplic(s, memmap, socket, + msi_m_phandle, msi_s_phandle, phandle, + &intc_phandles[phandle_pos], xplic_phandles, + s->soc[socket].num_harts); + } } } g_free(intc_phandles); - for (socket = 0; socket < socket_count; socket++) { - if (socket == 0) { - *irq_mmio_phandle = xplic_phandles[socket]; - *irq_virtio_phandle = xplic_phandles[socket]; - *irq_pcie_phandle = xplic_phandles[socket]; - } - if (socket == 1) { - *irq_virtio_phandle = xplic_phandles[socket]; - *irq_pcie_phandle = xplic_phandles[socket]; - } - if (socket == 2) { - *irq_pcie_phandle = xplic_phandles[socket]; + if (virt_use_kvm_aia(s)) { + *irq_mmio_phandle = xplic_phandles[0]; + *irq_virtio_phandle = xplic_phandles[0]; + *irq_pcie_phandle = xplic_phandles[0]; + } else { + for (socket = 0; socket < socket_count; socket++) { + if (socket == 0) { + *irq_mmio_phandle = xplic_phandles[socket]; + *irq_virtio_phandle = xplic_phandles[socket]; + *irq_pcie_phandle = xplic_phandles[socket]; + } + if (socket == 1) { + *irq_virtio_phandle = xplic_phandles[socket]; + *irq_pcie_phandle = xplic_phandles[socket]; + } + if (socket == 2) { + *irq_pcie_phandle = xplic_phandles[socket]; + } } } @@ -1430,6 +1452,14 @@ static void virt_machine_init(MachineState *machine) } } + if (virt_use_kvm_aia(s)) { + kvm_riscv_aia_create( + machine, s->kvm_aia_mode, IMSIC_MMIO_GROUP_MIN_SHIFT, + VIRT_IRQCHIP_NUM_SOURCES, VIRT_IRQCHIP_NUM_MSIS, + memmap[VIRT_APLIC_S].base, memmap[VIRT_IMSIC_S].base, + s->aia_guests); + } + if (riscv_is_32bit(&s->soc[0])) { #if HOST_LONG_BITS == 64 /* limit RAM size in a 32-bit system */ @@ -1575,6 +1605,31 @@ static void virt_set_aia(Object *obj, const char *val, Error **errp) } } +#if defined(CONFIG_KVM) +static char *virt_get_kvm_aia(Object *obj, Error **errp) +{ + RISCVVirtState *s = RISCV_VIRT_MACHINE(obj); + + return g_strdup(kvm_aia_mode_str(s->kvm_aia_mode)); +} + +static void virt_set_kvm_aia(Object *obj, const char *val, Error **errp) +{ + RISCVVirtState *s = RISCV_VIRT_MACHINE(obj); + + if (!strcmp(val, "emul")) { + s->kvm_aia_mode = KVM_DEV_RISCV_AIA_MODE_EMUL; + } else if (!strcmp(val, "hwaccel")) { + s->kvm_aia_mode = KVM_DEV_RISCV_AIA_MODE_HWACCEL; + } else if (!strcmp(val, "auto")) { + s->kvm_aia_mode = KVM_DEV_RISCV_AIA_MODE_AUTO; + } else { + error_setg(errp, "Invalid KVM AIA mode"); + error_append_hint(errp, "Valid values are emul, hwaccel, and auto.\n"); + } +} +#endif + static bool virt_get_aclint(Object *obj, Error **errp) { RISCVVirtState *s = RISCV_VIRT_MACHINE(obj); @@ -1685,6 +1740,18 @@ static void virt_machine_class_init(ObjectClass *oc, void *data) sprintf(str, "Set number of guest MMIO pages for AIA IMSIC. Valid value " "should be between 0 and %d.", VIRT_IRQCHIP_MAX_GUESTS); object_class_property_set_description(oc, "aia-guests", str); + +#if defined(CONFIG_KVM) + object_class_property_add_str(oc, "kvm-aia", virt_get_kvm_aia, + virt_set_kvm_aia); + object_class_property_set_description(oc, "kvm-aia", + "Set KVM AIA mode. Valid values are " + "emul, hwaccel, and auto. Default " + "is auto."); + object_property_set_default_str(object_class_property_find(oc, "kvm-aia"), + "auto"); + +#endif object_class_property_add(oc, "acpi", "OnOffAuto", virt_get_acpi, virt_set_acpi, NULL, NULL); diff --git a/include/hw/riscv/virt.h b/include/hw/riscv/virt.h index e5c474b26e..d0140feeff 100644 --- a/include/hw/riscv/virt.h +++ b/include/hw/riscv/virt.h @@ -56,6 +56,7 @@ struct RISCVVirtState { bool have_aclint; RISCVVirtAIAType aia_type; int aia_guests; + uint64_t kvm_aia_mode; char *oem_id; char *oem_table_id; OnOffAuto acpi;