From patchwork Tue Aug 13 06:37:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13761379 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7EE50C52D7C for ; Tue, 13 Aug 2024 06:38:07 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sdl9k-0005mO-7Q; Tue, 13 Aug 2024 02:37:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sdl9i-0005h4-95 for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:18 -0400 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sdl9g-0006OO-KR for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:17 -0400 Received: by mail-pf1-x430.google.com with SMTP id d2e1a72fcca58-7105043330aso4147770b3a.0 for ; Mon, 12 Aug 2024 23:37:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1723531035; x=1724135835; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=HcdZ79xR6R3d995UHi2uhISfC570oppLStRageGdWZU=; b=0+35AJvKFTBR7hHR6X+B3VCiuTMRlGIzJpHZAIu3lEdKgWlQ9Qeym6uWyoQWT2w5Mz 1L4NrQnkSgEv2Jz7r5COeCz+iNj3Jtu30PMLPa7WfofLDaurmEqPh+CXrPGr5bUiP80t 8X6EYlzQhUTrXAuO1apt+j+oUeywcGoG6LFgikFP9YrCpCZXq+l7hO3jDFs0PQL2BE1F U7wCp6lpbGsEY0ODaTxNqM8vgEpkD//ok3B/nPOzSUjhMKkbhO5eGpBvvn3IZgApfTme 1SMv085tGWh7xyZkzXp7acnAMq0HcRv6943ZikFmTSEzCEKUxNoHq2ZQbX8Qiijv+nVR 18Pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723531035; x=1724135835; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HcdZ79xR6R3d995UHi2uhISfC570oppLStRageGdWZU=; b=Kuwar8dNv8/RC/AtIzBEf6Ec0VUsYi5TI+4oRwBhUr0hcqXl1EPySZYXgbNNtSB1Q1 1EsuFSiGsMcjTWGQYfaiF9pPYZ6i72KN4nVt6kcrYERnmagpwr7eroZtu8lw7zxKYeKu UA9K/oz7H7+Rap4P7CDCiPiPaP3IPW0boBuSpxsVbRfy3uB9Tl3zFR1MA8meGiqj+DXg WCvYB0ngUHu7FET5huO/JTlSpEBJal7WU6oPz4jxvxfEWvnsdRH5wti9UcP/NfdATqH+ jjywIg4gPvQ+dQSd+OJKIobQ0hd4B7tfQp4GNxsMnbJmi2/5sbObR+aBlxcQ1Pfti7v/ qu7w== X-Gm-Message-State: AOJu0YwQinveCHN7Q2hYtoAhYtRozFibQ/paWJNcl2t+bcl63JEnZeT+ NnoVjYH1+4lEab6WWaAzICRbhjOHbFgAOsNw8z/6V1gT8Zwj7WEw0fAqaebghsc= X-Google-Smtp-Source: AGHT+IEch9SwD8Q+h9uVEvHbtNqM1jtzZzAcbONmiJNa0r9Z/x9ub4X0EJe1TJ53BArtoaWcbZqRuQ== X-Received: by 2002:a05:6a00:17aa:b0:70d:2693:d212 with SMTP id d2e1a72fcca58-7125510c560mr3177429b3a.6.1723531034969; Mon, 12 Aug 2024 23:37:14 -0700 (PDT) Received: from localhost ([157.82.202.230]) by smtp.gmail.com with UTF8SMTPSA id 41be03b00d2f7-7c6979ebfaesm728212a12.24.2024.08.12.23.37.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 12 Aug 2024 23:37:14 -0700 (PDT) From: Akihiko Odaki Date: Tue, 13 Aug 2024 15:37:00 +0900 Subject: [PATCH for-9.2 v7 1/9] hw/pci: Do not add ROM BAR for SR-IOV VF MIME-Version: 1.0 Message-Id: <20240813-sriov-v7-1-8515e3774df7@daynix.com> References: <20240813-sriov-v7-0-8515e3774df7@daynix.com> In-Reply-To: <20240813-sriov-v7-0-8515e3774df7@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::430; envelope-from=akihiko.odaki@daynix.com; helo=mail-pf1-x430.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A SR-IOV VF cannot have a ROM BAR. Co-developed-by: Yui Washizu Signed-off-by: Akihiko Odaki --- hw/pci/pci.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index d2eaf0c51dde..60b1747d60e6 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -2359,6 +2359,14 @@ static void pci_add_option_rom(PCIDevice *pdev, bool is_default_rom, return; } + if (pci_is_vf(pdev)) { + if (pdev->rom_bar > 0) { + error_setg(errp, "ROM BAR cannot be enabled for SR-IOV VF"); + } + + return; + } + if (load_file || pdev->romsize == UINT32_MAX) { path = qemu_find_file(QEMU_FILE_TYPE_BIOS, pdev->romfile); if (path == NULL) { From patchwork Tue Aug 13 06:37:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13761376 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 15468C52D7C for ; Tue, 13 Aug 2024 06:37:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sdl9o-00064s-NW; Tue, 13 Aug 2024 02:37:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sdl9n-0005zu-FG for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:23 -0400 Received: from mail-pl1-x62f.google.com ([2607:f8b0:4864:20::62f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sdl9l-0006PG-QO for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:23 -0400 Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1ff3d5c6e9eso35745235ad.1 for ; Mon, 12 Aug 2024 23:37:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1723531039; x=1724135839; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=ctp9i8y5VUp0+JHaTY/1UxY+d2tZ1Ss90F+E/6J1VMQ=; b=Wzhz6iZjssghYPiyJ4ZhVXmnVhgP8sY457+y1xeLL/7Tc6MmJoh6VYO9GuLVdSLCei 2JVSgXt05E9dmZUjIwaNVUz6Q54At1ks/yWxQbetL1PNoqb1w4Suyalq4HtQGmbI8bQ/ tOqZw7W16UiEPECh8CcnlZ91bwAXYk9l1Go1lb18tb2CR9hXFQcEmIvNbyfzfAHaqrdN LtQZryTLCRDVePvOiADaG7WLc9OPa+j3bB2zF6UIa/Kg+KlXo+KB+ipjCXt6AUUol3sW atqzsDUbuoqs+47JHb+e7Wj3aRgvSzISQsMjvxiVkfTs52ygMzAVSxfqyhNDdvzR2izR TWHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723531039; x=1724135839; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ctp9i8y5VUp0+JHaTY/1UxY+d2tZ1Ss90F+E/6J1VMQ=; b=NKTOfbLbF5vwRVeEV1fnhILgUDWPIPd9bF6KRa4weXYxPqyv4LAU2QheBJFLdmsFFm rjgWryGPByQ5Xhc2Jc2CO30Y230EjufgyR3mJWfTHdHN+j5ZW2PsHdxXyJuMjS5lgvPc yXzrf7QiUxSA7oUFF32IdNq8gpDKueQY/TJfvwErnQUOeAL9LuGsX3pHxEZ4Jnj/bNBL j67xLurY3BWSnDBAfSCI4g+LA6OH9/QXk5jmJB+5gRFfBvAg8Bk50dLXTlp7sOrgEldM z2tU0R6HTP2G2r/NycV9B8JN048XoPXSeWgVtE6nlQ95OPkJZdWyXLMJGR/s+tiDnJ3v DnZA== X-Gm-Message-State: AOJu0YwdavHpGZ66TLsFOaTHsBxWTBj1Lkfv9FIW+hXWcRGk5dcQvOk/ exRsUvYFDig7mSpUzB4HrF4BmRrkafYIj5GCd8wdELTDdjfZh2IKucoSS7Wyhl4= X-Google-Smtp-Source: AGHT+IFbOFZrNxeWJ/44GVzv5WgsMdOj+1HULObeS7AXl5nBKwLTlRdpoQAvVQ/0M4wN5Nwgm2bQrg== X-Received: by 2002:a17:902:ec91:b0:1fd:9c2d:2f1b with SMTP id d9443c01a7336-201ca1cb0d8mr25991515ad.52.1723531039371; Mon, 12 Aug 2024 23:37:19 -0700 (PDT) Received: from localhost ([157.82.202.230]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-201cd1a9220sm6574955ad.124.2024.08.12.23.37.16 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 12 Aug 2024 23:37:19 -0700 (PDT) From: Akihiko Odaki Date: Tue, 13 Aug 2024 15:37:01 +0900 Subject: [PATCH for-9.2 v7 2/9] hw/pci: Fix SR-IOV VF number calculation MIME-Version: 1.0 Message-Id: <20240813-sriov-v7-2-8515e3774df7@daynix.com> References: <20240813-sriov-v7-0-8515e3774df7@daynix.com> In-Reply-To: <20240813-sriov-v7-0-8515e3774df7@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::62f; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x62f.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org pci_config_get_bar_addr() had a division by vf_stride. vf_stride needs to be non-zero when there are multiple VFs, but the specification does not prohibit to make it zero when there is only one VF. Do not perform the division for the first VF to avoid division by zero. Signed-off-by: Akihiko Odaki --- hw/pci/pci.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 60b1747d60e6..0956fe5eb444 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -1437,7 +1437,11 @@ static pcibus_t pci_config_get_bar_addr(PCIDevice *d, int reg, pci_get_word(pf->config + sriov_cap + PCI_SRIOV_VF_OFFSET); uint16_t vf_stride = pci_get_word(pf->config + sriov_cap + PCI_SRIOV_VF_STRIDE); - uint32_t vf_num = (d->devfn - (pf->devfn + vf_offset)) / vf_stride; + uint32_t vf_num = d->devfn - (pf->devfn + vf_offset); + + if (vf_num) { + vf_num /= vf_stride; + } if (type & PCI_BASE_ADDRESS_MEM_TYPE_64) { new_addr = pci_get_quad(pf->config + bar); From patchwork Tue Aug 13 06:37:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13761377 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 43857C52D7B for ; Tue, 13 Aug 2024 06:38:00 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sdl9u-0006QI-8U; Tue, 13 Aug 2024 02:37:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sdl9q-0006Dz-WE for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:28 -0400 Received: from mail-pl1-x62b.google.com ([2607:f8b0:4864:20::62b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sdl9p-0006Q0-Bi for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:26 -0400 Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1fd70ba6a15so38122275ad.0 for ; Mon, 12 Aug 2024 23:37:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1723531044; x=1724135844; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=sLNYm1kEQastcr3NeEEOoStKnsUPyQiA0AB5cl9ypRI=; b=vEhPY2zkGytE8bQkUs8cCMgMzOVNDjWdHHwf69M8HOJYXQOeabIVfwn00eSlg5bhsA pOg3arwVulfs3+7PxDED9KtGxlXwd8JpGrQbpVDTU2JVw7T3ceCI0YxsEhYZ0lWYVqtv QFhtEueM+HGghnNyEQuOTdltnbi3Z+2lyB98tNkiKW1ciadpGJIqnokrzDug29p5Ckk8 PwnfACBHnhXpznrTn3oiOqnX1fcDtoZRu9Z4tTUvB8YpP9kS+t7lmP58v791kTHd2uaX TkIPqL01N2SNyi2JT8ATOw4pzJalXQPrOrK1skvv957nba0iMQnCNjFQvAtuH+K15N94 UqVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723531044; x=1724135844; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sLNYm1kEQastcr3NeEEOoStKnsUPyQiA0AB5cl9ypRI=; b=vL5rbuRwugSbDDM5y1S9XvXFcSLdSQ37L5RblFPCfpQl68IMBTM7//V6qu9QG2HCAG RjjwczdWlquMVfv+jdl1lNa9oLMWNagwlBIt3utqoXHjrUnuSe3bj/mv/y2CEf0xUU6i /F0One87k7/15BABwtgJK7LrVWPIEA4KJmeS4XnJDXqmGl5m0EXJ4maPJMfM0IMhrVIs jHscQpQEMiGTRDzSsk79uwYAUlDHfEBgVnXCLD5G4z6VHCQf35wWqlbNjoQGIrX/Dy3h foguPJ6D6A/IfFErsh9yhwQmu/iX7cKcd4p3gX4XvxRLlDB2PkNl7OUq/k239blNaeZ2 jP+Q== X-Gm-Message-State: AOJu0YznDON2ixp+77uHoLcL1kkSPUDHz/FLoJluqRAByAq7jT/279mB z7vDO/LfKp8bK1fmcYatzqEeuc9b7BrrN5vY0TOoz12PUS1IAz3rd+hAez49gO8= X-Google-Smtp-Source: AGHT+IGnQLrDp9DkpksoF5PkMiT3Wh7IcDAkoqzn4M85GHP69v09o4sfdUhgbjsX7lJMAXHDavXBmQ== X-Received: by 2002:a17:902:e88d:b0:1fb:82f5:6641 with SMTP id d9443c01a7336-201ca13ced6mr34317055ad.23.1723531043849; Mon, 12 Aug 2024 23:37:23 -0700 (PDT) Received: from localhost ([157.82.202.230]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-201ce2e7bb7sm5501525ad.3.2024.08.12.23.37.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 12 Aug 2024 23:37:23 -0700 (PDT) From: Akihiko Odaki Date: Tue, 13 Aug 2024 15:37:02 +0900 Subject: [PATCH for-9.2 v7 3/9] pcie_sriov: Ensure PF and VF are mutually exclusive MIME-Version: 1.0 Message-Id: <20240813-sriov-v7-3-8515e3774df7@daynix.com> References: <20240813-sriov-v7-0-8515e3774df7@daynix.com> In-Reply-To: <20240813-sriov-v7-0-8515e3774df7@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::62b; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x62b.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A device cannot be a SR-IOV PF and a VF at the same time. Signed-off-by: Akihiko Odaki --- hw/pci/pcie_sriov.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index 1eae9f0a0acf..e1b4ecf79ff9 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie_sriov.c @@ -42,6 +42,11 @@ bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, uint8_t *cfg = dev->config + offset; uint8_t *wmask; + if (pci_is_vf(dev)) { + error_setg(errp, "a device cannot be a SR-IOV PF and a VF at the same time"); + return false; + } + pcie_add_capability(dev, PCI_EXT_CAP_ID_SRIOV, 1, offset, PCI_EXT_CAP_SRIOV_SIZEOF); dev->exp.sriov_cap = offset; From patchwork Tue Aug 13 06:37:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13761383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 62980C52D7B for ; Tue, 13 Aug 2024 06:39:01 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sdl9y-0006e1-JK; Tue, 13 Aug 2024 02:37:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sdl9w-0006Ts-2d for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:32 -0400 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sdl9t-0006Qh-Fv for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:30 -0400 Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-1ff67158052so32851905ad.0 for ; Mon, 12 Aug 2024 23:37:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1723531048; x=1724135848; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=vA26nEU65dtwvWJQIJNPD+i6nbzwt/Hl37bWV6eE0XI=; b=O/Nd1IcrHMkzaYNfjMjaqsSps+7yBjK+7igfeNnCQwl8IbzisIhQgcRYRYuj6cp2el ETjXD7mbox1JDgTqRpsg93VYSV6UbVDQcBRp1yERKDKXaZYes536NdoSoygVfFnow9H8 TwGiP50fD0EI9Xwbu+HhkFu+ZU5kB8fJWoidZ+C2dpACUabvpmfZSZO9Dw4i04953RHn 5aRGb7bOCc6AgT9OR6ZZA5CT5eAI1c8wfgqQj8kiVgnoxO68jcTKZvPj4cm7RuU0FRLt gEosAgG1bn2gqaoBLa8bNJpCLmcyw+5Y1ZYlR6oUT55tOnAH2cRlXuoD3bmvoaIhXr1Z 9LDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723531048; x=1724135848; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vA26nEU65dtwvWJQIJNPD+i6nbzwt/Hl37bWV6eE0XI=; b=Ow+jkNkwPX1PyKZHguziaq4OzX4Zke1w/K935gtbXHux8gADQj7N17DcUOsyNWzy7A oQQGGTsso269PvZ2OrTjLme2Xr5pRa444qwr3qxhh/EXFOTqG4KQVU+HKCYyScmWaYFw Gd+3BIPHHLWee9u88UzYY5PbGorzZpQqsV3Bs8/PBdKMz9NhOgl8VhTvFEktTf/hvmFF GwG9sDE1YY0U3sTyuVpVLVPUPPHE4tFYdQtVIRHwrTQVP2X+9ZKNG8/1ZJ6RGQOObst/ NDwHu0zEXnV3cttgtsUBn1BJ4OmEhyOEkN4/ig2BblFSxjW7Eiok3Fz1X69rV847jMvR 3M5g== X-Gm-Message-State: AOJu0Yz3s16Om3MJ2BjFn/0MA0M7X14+NwHxXbPbWZJVk23TG2/CjqzX OOP44Fl9YlVgMCSbXmy4sG1sV7+tj608wkaC/jDC1WIBkiDL0K97KmR5fcvc0yM= X-Google-Smtp-Source: AGHT+IF3zbsteBl/roY0i9ig54NZa83VT030hPoJA9+VEYyaQhCYX8aLn34bWM95k6bRMMAxt3QTdw== X-Received: by 2002:a17:902:f651:b0:1fc:4763:445c with SMTP id d9443c01a7336-201ca175c46mr29679955ad.32.1723531048182; Mon, 12 Aug 2024 23:37:28 -0700 (PDT) Received: from localhost ([157.82.202.230]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-201cd1a9215sm6609885ad.153.2024.08.12.23.37.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 12 Aug 2024 23:37:27 -0700 (PDT) From: Akihiko Odaki Date: Tue, 13 Aug 2024 15:37:03 +0900 Subject: [PATCH for-9.2 v7 4/9] pcie_sriov: Check PCI Express for SR-IOV PF MIME-Version: 1.0 Message-Id: <20240813-sriov-v7-4-8515e3774df7@daynix.com> References: <20240813-sriov-v7-0-8515e3774df7@daynix.com> In-Reply-To: <20240813-sriov-v7-0-8515e3774df7@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::632; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x632.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org SR-IOV requires PCI Express. Signed-off-by: Akihiko Odaki --- hw/pci/pcie_sriov.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index e1b4ecf79ff9..2daea6ecdb6a 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie_sriov.c @@ -42,6 +42,11 @@ bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, uint8_t *cfg = dev->config + offset; uint8_t *wmask; + if (!pci_is_express(dev)) { + error_setg(errp, "PCI Express is required for SR-IOV PF"); + return false; + } + if (pci_is_vf(dev)) { error_setg(errp, "a device cannot be a SR-IOV PF and a VF at the same time"); return false; From patchwork Tue Aug 13 06:37:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13761378 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 46762C52D7F for ; Tue, 13 Aug 2024 06:38:00 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sdlA6-00070p-9m; Tue, 13 Aug 2024 02:37:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sdlA1-0006si-QF for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:37 -0400 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sdl9y-0006RQ-IT for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:36 -0400 Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-710ffaf921fso1188935b3a.1 for ; Mon, 12 Aug 2024 23:37:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1723531053; x=1724135853; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=z/MFYpSKsxurDELJiU0C9WDyT7OyUExUvk7brt45XRg=; b=zrFHdys5Vjr8ss2Z17QjD2GDB7xL8Y8BSWfn2TMS5SwArgKHPMXLGer24lYLbIMrIx VrqzvKKbAqogZZimQs0F4klxab4hlrXTEC/+JzZBgbPyioAi8sw3KoVxp9XLxUSUUm1n Wl4Md+k/2wlFSIaJK+d9i6yCW+EQTyI7oG4rYCyUtGi08cjOS6q5jogKLy/yMaxmIq6B iwBN6J31H6VuUHUSytMgM5Cd8KP1vc4kD6mmN99ylUDl9C70UqRjx2l5MRhJM0hCgNtZ QMr2lz26sAb/P5muFuFhjzJLQeHO4RVKwhgFbrtAdxJmCZU9L/7H94M33FHwtd2F9FYA 0xOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723531053; x=1724135853; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=z/MFYpSKsxurDELJiU0C9WDyT7OyUExUvk7brt45XRg=; b=Eiuus1v9dFSKOkEB16ZU6H0bLilgmwB0tMJsMZMJmtb55S/4kq8cNgpeAUdzPOPI2R ZlekB6/FqvvbFUGxgpG3LoT1/i6c+svR1y65wlNA3oZ/mSm2bkvFF9PbV1swRvsplH/2 O9RY69P1Eg1ITBnFJk1bavAsKM6LFXzHM1jeDDirO79Vu717eHSnqxAdqPIOzdOY68EM OwmfTKWgjvQH95HVfAD5Yx+UXlSXBAqWklvAcId9ky4VURx1uTdndCt0Wi2X5UpVccut vw2Y/SYNrGfiiClctkVL5nIqp0X0/gHfRjPG60GhFS6gfHYBbdX8i+aqu1RMbmSHAfnh a5Kw== X-Gm-Message-State: AOJu0YwgDqCvMnS3yZyCRYR7V21NvzUSAhMimEYPKtUjdxDByAPmNO+J PTCP94+ZKCrpP+2b43G1/yoJigRxkfdThBvlkrR0E5SV24nGLX+7UkuBNo3f5js= X-Google-Smtp-Source: AGHT+IE4CxSswGjjZxdpxn6ph8hVIMTXl+WiphPoNgJhi3EyKkmDOc1PLtcZCemtHKCmXAm1129hbA== X-Received: by 2002:a05:6a00:170a:b0:708:41c4:8849 with SMTP id d2e1a72fcca58-71256fa3249mr3203455b3a.9.1723531053087; Mon, 12 Aug 2024 23:37:33 -0700 (PDT) Received: from localhost ([157.82.202.230]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-710e5ab7ba6sm4980054b3a.209.2024.08.12.23.37.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 12 Aug 2024 23:37:32 -0700 (PDT) From: Akihiko Odaki Date: Tue, 13 Aug 2024 15:37:04 +0900 Subject: [PATCH for-9.2 v7 5/9] pcie_sriov: Allow user to create SR-IOV device MIME-Version: 1.0 Message-Id: <20240813-sriov-v7-5-8515e3774df7@daynix.com> References: <20240813-sriov-v7-0-8515e3774df7@daynix.com> In-Reply-To: <20240813-sriov-v7-0-8515e3774df7@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::42b; envelope-from=akihiko.odaki@daynix.com; helo=mail-pf1-x42b.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A user can create a SR-IOV device by specifying the PF with the sriov-pf property of the VFs. The VFs must be added before the PF. A user-creatable VF must have PCIDeviceClass::sriov_vf_user_creatable set. Such a VF cannot refer to the PF because it is created before the PF. A PF that user-creatable VFs can be attached calls pcie_sriov_pf_init_from_user_created_vfs() during realization and pcie_sriov_pf_exit() when exiting. Signed-off-by: Akihiko Odaki --- include/hw/pci/pci_device.h | 6 +- include/hw/pci/pcie_sriov.h | 18 +++ hw/pci/pci.c | 62 ++++++---- hw/pci/pcie_sriov.c | 279 +++++++++++++++++++++++++++++++++++--------- 4 files changed, 286 insertions(+), 79 deletions(-) diff --git a/include/hw/pci/pci_device.h b/include/hw/pci/pci_device.h index 8fa845beee5e..1d31099dd4dc 100644 --- a/include/hw/pci/pci_device.h +++ b/include/hw/pci/pci_device.h @@ -38,6 +38,8 @@ struct PCIDeviceClass { uint16_t subsystem_id; /* only for header type = 0 */ const char *romfile; /* rom bar */ + + bool sriov_vf_user_creatable; }; enum PCIReqIDType { @@ -167,6 +169,8 @@ struct PCIDevice { /* ID of standby device in net_failover pair */ char *failover_pair_id; uint32_t acpi_index; + + char *sriov_pf; }; static inline int pci_intx(PCIDevice *pci_dev) @@ -199,7 +203,7 @@ static inline int pci_is_express_downstream_port(const PCIDevice *d) static inline int pci_is_vf(const PCIDevice *d) { - return d->exp.sriov_vf.pf != NULL; + return d->sriov_pf || d->exp.sriov_vf.pf != NULL; } static inline uint32_t pci_config_size(const PCIDevice *d) diff --git a/include/hw/pci/pcie_sriov.h b/include/hw/pci/pcie_sriov.h index c5d2d318d330..f75b8f22ee92 100644 --- a/include/hw/pci/pcie_sriov.h +++ b/include/hw/pci/pcie_sriov.h @@ -18,6 +18,7 @@ typedef struct PCIESriovPF { uint8_t vf_bar_type[PCI_NUM_REGIONS]; /* Store type for each VF bar */ PCIDevice **vf; /* Pointer to an array of num_vfs VF devices */ + bool vf_user_created; /* If VFs are created by user */ } PCIESriovPF; typedef struct PCIESriovVF { @@ -40,6 +41,23 @@ void pcie_sriov_pf_init_vf_bar(PCIDevice *dev, int region_num, void pcie_sriov_vf_register_bar(PCIDevice *dev, int region_num, MemoryRegion *memory); +/** + * pcie_sriov_pf_init_from_user_created_vfs() - Initialize PF with user-created + * VFs. + * @dev: A PCIe device being realized. + * @offset: The offset of the SR-IOV capability. + * @errp: pointer to Error*, to store an error if it happens. + * + * Return: The size of added capability. 0 if the user did not create VFs. + * -1 if failed. + */ +int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, + uint16_t offset, + Error **errp); + +bool pcie_sriov_register_device(PCIDevice *dev, Error **errp); +void pcie_sriov_unregister_device(PCIDevice *dev); + /* * Default (minimal) page size support values * as required by the SR/IOV standard: diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 0956fe5eb444..e693f5b1e044 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -85,6 +85,7 @@ static Property pci_props[] = { QEMU_PCIE_ERR_UNC_MASK_BITNR, true), DEFINE_PROP_BIT("x-pcie-ari-nextfn-1", PCIDevice, cap_present, QEMU_PCIE_ARI_NEXTFN_1_BITNR, false), + DEFINE_PROP_STRING("sriov-pf", PCIDevice, sriov_pf), DEFINE_PROP_END_OF_LIST() }; @@ -959,13 +960,8 @@ static void pci_init_multifunction(PCIBus *bus, PCIDevice *dev, Error **errp) dev->config[PCI_HEADER_TYPE] |= PCI_HEADER_TYPE_MULTI_FUNCTION; } - /* - * With SR/IOV and ARI, a device at function 0 need not be a multifunction - * device, as it may just be a VF that ended up with function 0 in - * the legacy PCI interpretation. Avoid failing in such cases: - */ - if (pci_is_vf(dev) && - dev->exp.sriov_vf.pf->cap_present & QEMU_PCI_CAP_MULTIFUNCTION) { + /* SR/IOV is not handled here. */ + if (pci_is_vf(dev)) { return; } @@ -998,7 +994,8 @@ static void pci_init_multifunction(PCIBus *bus, PCIDevice *dev, Error **errp) } /* function 0 indicates single function, so function > 0 must be NULL */ for (func = 1; func < PCI_FUNC_MAX; ++func) { - if (bus->devices[PCI_DEVFN(slot, func)]) { + PCIDevice *device = bus->devices[PCI_DEVFN(slot, func)]; + if (device && !pci_is_vf(device)) { error_setg(errp, "PCI: %x.0 indicates single function, " "but %x.%x is already populated.", slot, slot, func); @@ -1283,6 +1280,7 @@ static void pci_qdev_unrealize(DeviceState *dev) pci_unregister_io_regions(pci_dev); pci_del_option_rom(pci_dev); + pcie_sriov_unregister_device(pci_dev); if (pc->exit) { pc->exit(pci_dev); @@ -1314,7 +1312,6 @@ void pci_register_bar(PCIDevice *pci_dev, int region_num, pcibus_t size = memory_region_size(memory); uint8_t hdr_type; - assert(!pci_is_vf(pci_dev)); /* VFs must use pcie_sriov_vf_register_bar */ assert(region_num >= 0); assert(region_num < PCI_NUM_REGIONS); assert(is_power_of_2(size)); @@ -1325,7 +1322,6 @@ void pci_register_bar(PCIDevice *pci_dev, int region_num, assert(hdr_type != PCI_HEADER_TYPE_BRIDGE || region_num < 2); r = &pci_dev->io_regions[region_num]; - r->addr = PCI_BAR_UNMAPPED; r->size = size; r->type = type; r->memory = memory; @@ -1333,22 +1329,35 @@ void pci_register_bar(PCIDevice *pci_dev, int region_num, ? pci_get_bus(pci_dev)->address_space_io : pci_get_bus(pci_dev)->address_space_mem; - wmask = ~(size - 1); - if (region_num == PCI_ROM_SLOT) { - /* ROM enable bit is writable */ - wmask |= PCI_ROM_ADDRESS_ENABLE; - } - - addr = pci_bar(pci_dev, region_num); - pci_set_long(pci_dev->config + addr, type); + if (pci_is_vf(pci_dev)) { + PCIDevice *pf = pci_dev->exp.sriov_vf.pf; + assert(!pf || type == pf->exp.sriov_pf.vf_bar_type[region_num]); - if (!(r->type & PCI_BASE_ADDRESS_SPACE_IO) && - r->type & PCI_BASE_ADDRESS_MEM_TYPE_64) { - pci_set_quad(pci_dev->wmask + addr, wmask); - pci_set_quad(pci_dev->cmask + addr, ~0ULL); + r->addr = pci_bar_address(pci_dev, region_num, r->type, r->size); + if (r->addr != PCI_BAR_UNMAPPED) { + memory_region_add_subregion_overlap(r->address_space, + r->addr, r->memory, 1); + } } else { - pci_set_long(pci_dev->wmask + addr, wmask & 0xffffffff); - pci_set_long(pci_dev->cmask + addr, 0xffffffff); + r->addr = PCI_BAR_UNMAPPED; + + wmask = ~(size - 1); + if (region_num == PCI_ROM_SLOT) { + /* ROM enable bit is writable */ + wmask |= PCI_ROM_ADDRESS_ENABLE; + } + + addr = pci_bar(pci_dev, region_num); + pci_set_long(pci_dev->config + addr, type); + + if (!(r->type & PCI_BASE_ADDRESS_SPACE_IO) && + r->type & PCI_BASE_ADDRESS_MEM_TYPE_64) { + pci_set_quad(pci_dev->wmask + addr, wmask); + pci_set_quad(pci_dev->cmask + addr, ~0ULL); + } else { + pci_set_long(pci_dev->wmask + addr, wmask & 0xffffffff); + pci_set_long(pci_dev->cmask + addr, 0xffffffff); + } } } @@ -2109,6 +2118,11 @@ static void pci_qdev_realize(DeviceState *qdev, Error **errp) } } + if (!pcie_sriov_register_device(pci_dev, errp)) { + pci_qdev_unrealize(DEVICE(pci_dev)); + return; + } + /* * A PCIe Downstream Port that do not have ARI Forwarding enabled must * associate only Device 0 with the device attached to the bus diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index 2daea6ecdb6a..f5b83a92a00c 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie_sriov.c @@ -15,11 +15,12 @@ #include "hw/pci/pcie.h" #include "hw/pci/pci_bus.h" #include "hw/qdev-properties.h" -#include "qemu/error-report.h" #include "qemu/range.h" #include "qapi/error.h" #include "trace.h" +static GHashTable *pfs; + static void unparent_vfs(PCIDevice *dev, uint16_t total_vfs) { for (uint16_t i = 0; i < total_vfs; i++) { @@ -31,14 +32,43 @@ static void unparent_vfs(PCIDevice *dev, uint16_t total_vfs) dev->exp.sriov_pf.vf = NULL; } -bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, - const char *vfname, uint16_t vf_dev_id, - uint16_t init_vfs, uint16_t total_vfs, - uint16_t vf_offset, uint16_t vf_stride, - Error **errp) +static void register_vfs(PCIDevice *dev) +{ + uint16_t num_vfs; + uint16_t i; + uint16_t sriov_cap = dev->exp.sriov_cap; + + assert(sriov_cap > 0); + num_vfs = pci_get_word(dev->config + sriov_cap + PCI_SRIOV_NUM_VF); + + trace_sriov_register_vfs(dev->name, PCI_SLOT(dev->devfn), + PCI_FUNC(dev->devfn), num_vfs); + for (i = 0; i < num_vfs; i++) { + pci_set_enabled(dev->exp.sriov_pf.vf[i], true); + } + + pci_set_word(dev->wmask + sriov_cap + PCI_SRIOV_NUM_VF, 0); +} + +static void unregister_vfs(PCIDevice *dev) +{ + uint8_t *cfg = dev->config + dev->exp.sriov_cap; + uint16_t i; + + trace_sriov_unregister_vfs(dev->name, PCI_SLOT(dev->devfn), + PCI_FUNC(dev->devfn)); + for (i = 0; i < pci_get_word(cfg + PCI_SRIOV_TOTAL_VF); i++) { + pci_set_enabled(dev->exp.sriov_pf.vf[i], false); + } + + pci_set_word(dev->wmask + dev->exp.sriov_cap + PCI_SRIOV_NUM_VF, 0xffff); +} + +static bool pcie_sriov_pf_init_common(PCIDevice *dev, uint16_t offset, + uint16_t vf_dev_id, uint16_t init_vfs, + uint16_t total_vfs, uint16_t vf_offset, + uint16_t vf_stride, Error **errp) { - BusState *bus = qdev_get_parent_bus(&dev->qdev); - int32_t devfn = dev->devfn + vf_offset; uint8_t *cfg = dev->config + offset; uint8_t *wmask; @@ -88,6 +118,28 @@ bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, qdev_prop_set_bit(&dev->qdev, "multifunction", true); + return true; +} + +bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, + const char *vfname, uint16_t vf_dev_id, + uint16_t init_vfs, uint16_t total_vfs, + uint16_t vf_offset, uint16_t vf_stride, + Error **errp) +{ + BusState *bus = qdev_get_parent_bus(&dev->qdev); + int32_t devfn = dev->devfn + vf_offset; + + if (pfs && g_hash_table_contains(pfs, dev->qdev.id)) { + error_setg(errp, "attaching user-created SR-IOV VF unsupported"); + return false; + } + + if (!pcie_sriov_pf_init_common(dev, offset, vf_dev_id, init_vfs, + total_vfs, vf_offset, vf_stride, errp)) { + return false; + } + dev->exp.sriov_pf.vf = g_new(PCIDevice *, total_vfs); for (uint16_t i = 0; i < total_vfs; i++) { @@ -117,7 +169,22 @@ void pcie_sriov_pf_exit(PCIDevice *dev) { uint8_t *cfg = dev->config + dev->exp.sriov_cap; - unparent_vfs(dev, pci_get_word(cfg + PCI_SRIOV_TOTAL_VF)); + if (dev->exp.sriov_pf.vf_user_created) { + uint16_t ven_id = pci_get_word(dev->config + PCI_VENDOR_ID); + uint16_t total_vfs = pci_get_word(dev->config + PCI_SRIOV_TOTAL_VF); + uint16_t vf_dev_id = pci_get_word(dev->config + PCI_SRIOV_VF_DID); + + unregister_vfs(dev); + + for (uint16_t i = 0; i < total_vfs; i++) { + dev->exp.sriov_pf.vf[i]->exp.sriov_vf.pf = NULL; + + pci_config_set_vendor_id(dev->exp.sriov_pf.vf[i]->config, ven_id); + pci_config_set_device_id(dev->exp.sriov_pf.vf[i]->config, vf_dev_id); + } + } else { + unparent_vfs(dev, pci_get_word(cfg + PCI_SRIOV_TOTAL_VF)); + } } void pcie_sriov_pf_init_vf_bar(PCIDevice *dev, int region_num, @@ -150,69 +217,173 @@ void pcie_sriov_pf_init_vf_bar(PCIDevice *dev, int region_num, void pcie_sriov_vf_register_bar(PCIDevice *dev, int region_num, MemoryRegion *memory) { - PCIIORegion *r; - PCIBus *bus = pci_get_bus(dev); uint8_t type; - pcibus_t size = memory_region_size(memory); - assert(pci_is_vf(dev)); /* PFs must use pci_register_bar */ - assert(region_num >= 0); - assert(region_num < PCI_NUM_REGIONS); + assert(dev->exp.sriov_vf.pf); type = dev->exp.sriov_vf.pf->exp.sriov_pf.vf_bar_type[region_num]; - if (!is_power_of_2(size)) { - error_report("%s: PCI region size must be a power" - " of two - type=0x%x, size=0x%"FMT_PCIBUS, - __func__, type, size); - exit(1); - } + return pci_register_bar(dev, region_num, type, memory); +} - r = &dev->io_regions[region_num]; - r->memory = memory; - r->address_space = - type & PCI_BASE_ADDRESS_SPACE_IO - ? bus->address_space_io - : bus->address_space_mem; - r->size = size; - r->type = type; - - r->addr = pci_bar_address(dev, region_num, r->type, r->size); - if (r->addr != PCI_BAR_UNMAPPED) { - memory_region_add_subregion_overlap(r->address_space, - r->addr, r->memory, 1); - } +static gint compare_vf_devfns(gconstpointer a, gconstpointer b) +{ + return (*(PCIDevice **)a)->devfn - (*(PCIDevice **)b)->devfn; } -static void register_vfs(PCIDevice *dev) +int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, + uint16_t offset, + Error **errp) { - uint16_t num_vfs; + GPtrArray *pf; + PCIDevice **vfs; + BusState *bus = qdev_get_parent_bus(DEVICE(dev)); + uint16_t ven_id = pci_get_word(dev->config + PCI_VENDOR_ID); + uint16_t vf_dev_id; + uint16_t vf_offset; + uint16_t vf_stride; uint16_t i; - uint16_t sriov_cap = dev->exp.sriov_cap; - assert(sriov_cap > 0); - num_vfs = pci_get_word(dev->config + sriov_cap + PCI_SRIOV_NUM_VF); + if (!pfs || !dev->qdev.id) { + return 0; + } - trace_sriov_register_vfs(dev->name, PCI_SLOT(dev->devfn), - PCI_FUNC(dev->devfn), num_vfs); - for (i = 0; i < num_vfs; i++) { - pci_set_enabled(dev->exp.sriov_pf.vf[i], true); + pf = g_hash_table_lookup(pfs, dev->qdev.id); + if (!pf) { + return 0; } - pci_set_word(dev->wmask + sriov_cap + PCI_SRIOV_NUM_VF, 0); + if (pf->len > UINT16_MAX) { + error_setg(errp, "too many VFs"); + return -1; + } + + g_ptr_array_sort(pf, compare_vf_devfns); + vfs = (void *)pf->pdata; + + if (vfs[0]->devfn <= dev->devfn) { + error_setg(errp, "a VF function number is less than the PF function number"); + return -1; + } + + vf_dev_id = pci_get_word(vfs[0]->config + PCI_DEVICE_ID); + vf_offset = vfs[0]->devfn - dev->devfn; + vf_stride = pf->len < 2 ? 0 : vfs[1]->devfn - vfs[0]->devfn; + + for (i = 0; i < pf->len; i++) { + if (bus != qdev_get_parent_bus(&vfs[i]->qdev)) { + error_setg(errp, "SR-IOV VF parent bus mismatches with PF"); + return -1; + } + + if (ven_id != pci_get_word(vfs[i]->config + PCI_VENDOR_ID)) { + error_setg(errp, "SR-IOV VF vendor ID mismatches with PF"); + return -1; + } + + if (vf_dev_id != pci_get_word(vfs[i]->config + PCI_DEVICE_ID)) { + error_setg(errp, "inconsistent SR-IOV VF device IDs"); + return -1; + } + + for (size_t j = 0; j < PCI_NUM_REGIONS; j++) { + if (vfs[i]->io_regions[j].size != vfs[0]->io_regions[j].size || + vfs[i]->io_regions[j].type != vfs[0]->io_regions[j].type) { + error_setg(errp, "inconsistent SR-IOV BARs"); + return -1; + } + } + + if (vfs[i]->devfn - vfs[0]->devfn != vf_stride * i) { + error_setg(errp, "inconsistent SR-IOV stride"); + return -1; + } + } + + if (!pcie_sriov_pf_init_common(dev, offset, vf_dev_id, pf->len, + pf->len, vf_offset, vf_stride, errp)) { + return -1; + } + + for (i = 0; i < pf->len; i++) { + vfs[i]->exp.sriov_vf.pf = dev; + vfs[i]->exp.sriov_vf.vf_number = i; + + /* set vid/did according to sr/iov spec - they are not used */ + pci_config_set_vendor_id(vfs[i]->config, 0xffff); + pci_config_set_device_id(vfs[i]->config, 0xffff); + } + + dev->exp.sriov_pf.vf = vfs; + dev->exp.sriov_pf.vf_user_created = true; + + for (i = 0; i < PCI_NUM_REGIONS; i++) { + PCIIORegion *region = &vfs[0]->io_regions[i]; + + if (region->size) { + pcie_sriov_pf_init_vf_bar(dev, i, region->type, region->size); + } + } + + return PCI_EXT_CAP_SRIOV_SIZEOF; } -static void unregister_vfs(PCIDevice *dev) +bool pcie_sriov_register_device(PCIDevice *dev, Error **errp) { - uint8_t *cfg = dev->config + dev->exp.sriov_cap; - uint16_t i; + if (!dev->exp.sriov_pf.vf && dev->qdev.id && + pfs && g_hash_table_contains(pfs, dev->qdev.id)) { + error_setg(errp, "attaching user-created SR-IOV VF unsupported"); + return false; + } - trace_sriov_unregister_vfs(dev->name, PCI_SLOT(dev->devfn), - PCI_FUNC(dev->devfn)); - for (i = 0; i < pci_get_word(cfg + PCI_SRIOV_TOTAL_VF); i++) { - pci_set_enabled(dev->exp.sriov_pf.vf[i], false); + if (dev->sriov_pf) { + PCIDevice *pci_pf; + GPtrArray *pf; + + if (!PCI_DEVICE_GET_CLASS(dev)->sriov_vf_user_creatable) { + error_setg(errp, "user cannot create SR-IOV VF with this device type"); + return false; + } + + if (!pci_is_express(dev)) { + error_setg(errp, "PCI Express is required for SR-IOV VF"); + return false; + } + + if (!pci_qdev_find_device(dev->sriov_pf, &pci_pf)) { + error_setg(errp, "PCI device specified as SR-IOV PF already exists"); + return false; + } + + if (!pfs) { + pfs = g_hash_table_new_full(g_str_hash, g_str_equal, g_free, NULL); + } + + pf = g_hash_table_lookup(pfs, dev->sriov_pf); + if (!pf) { + pf = g_ptr_array_new(); + g_hash_table_insert(pfs, g_strdup(dev->sriov_pf), pf); + } + + g_ptr_array_add(pf, dev); } - pci_set_word(dev->wmask + dev->exp.sriov_cap + PCI_SRIOV_NUM_VF, 0xffff); + return true; +} + +void pcie_sriov_unregister_device(PCIDevice *dev) +{ + if (dev->sriov_pf && pfs) { + GPtrArray *pf = g_hash_table_lookup(pfs, dev->sriov_pf); + + if (pf) { + g_ptr_array_remove_fast(pf, dev); + + if (!pf->len) { + g_hash_table_remove(pfs, dev->sriov_pf); + g_ptr_array_free(pf, FALSE); + } + } + } } void pcie_sriov_config_write(PCIDevice *dev, uint32_t address, @@ -308,7 +479,7 @@ void pcie_sriov_pf_add_sup_pgsize(PCIDevice *dev, uint16_t opt_sup_pgsize) uint16_t pcie_sriov_vf_number(PCIDevice *dev) { - assert(pci_is_vf(dev)); + assert(dev->exp.sriov_vf.pf); return dev->exp.sriov_vf.vf_number; } From patchwork Tue Aug 13 06:37:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13761384 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 95C87C52D7C for ; Tue, 13 Aug 2024 06:39:07 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sdlA7-00074M-Ah; Tue, 13 Aug 2024 02:37:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sdlA4-0006xs-Nt for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:41 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sdlA2-0006Th-Uu for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:40 -0400 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1fc5296e214so48823945ad.0 for ; Mon, 12 Aug 2024 23:37:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1723531057; x=1724135857; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=XtLHnn9U6eyio3FDZY7yCUtQSaVZBT8Hqm8aVmtB1Sg=; b=rFhCVqYu5JlasfHk7p2Jl/PWLa9if8IoAdRJJOuCEA5cUUTa9gCFq6sX6pDHDg47Nx qVa/mfK4L0Q6ySCJxD1BrCZG+1LYsnEu2h5To+PcAcBcOfbDBvKJq9zJX8HkipWeffYT 4xFnXJwCda9tuZCEhXuvoacwr7beYc6Xwj1KzMqvzSzasUcg4ePOBKh+71eFRZmdd73K es20dqsONXx8aorJjg3mspOqhFw+qaz6iNBrD6/Gt1abkofsJ5tLTZ/dBdT922xADD8z D1euK/I3pzrdY6izmNGeBjqv9PnTlBU38x6nYbwNWrRXQjPBUdk8Vs4wOLU92pSABqbI 62qQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723531057; x=1724135857; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XtLHnn9U6eyio3FDZY7yCUtQSaVZBT8Hqm8aVmtB1Sg=; b=hnrpnmr4DmTrl1RyS0NSL1/is4ebe/9El1D9cUjbin85497U3bTI/UcQnGJ95qmKOD bridYlhk32+2MenYYe66W+MS1QJGxomqE8Xoh/zFZGF7hJFGnPMs5pZedHwmJwgLzjUq Kg0lqyxHvtCChsLCes3MIojbsvDlqWxwpZ66NCqaVfJufOv8junvJYpY2x5dKZ/+HrIh q7Hsp+31685eGKpI0TnfIyMUlIr2KqwuH8Puov1KGdGgOv2cSxVEQPlTxOJd5dmi1ETc Sy5GAYdlYRIo6vQdSVCr7/skHzI3MUR+4kxExuVlWwqgsrwLq4xkGXunqVQzUa43TbBR qEKg== X-Gm-Message-State: AOJu0YwAwKR+ps9YvXjj7QZu1d4GovXJRGrbNudy7CRpRMRCq3aH6EL8 Gnq0K2EttL5O1gAiI37lcbWg0VlXp0GnK8cA1Ybp16lAGSexD6icG/z4C5g/EZ8= X-Google-Smtp-Source: AGHT+IH+oOu07OUmzoKngWVeUyLvLcA8nJ/81oRiB2mQhrarVd0nr9eorQDcaHEj+SJxOPax5VlbZQ== X-Received: by 2002:a17:902:d50f:b0:1fb:7435:c2c7 with SMTP id d9443c01a7336-201ca22924bmr27851605ad.57.1723531057606; Mon, 12 Aug 2024 23:37:37 -0700 (PDT) Received: from localhost ([157.82.202.230]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-201cd1b8dc6sm6609525ad.219.2024.08.12.23.37.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 12 Aug 2024 23:37:37 -0700 (PDT) From: Akihiko Odaki Date: Tue, 13 Aug 2024 15:37:05 +0900 Subject: [PATCH for-9.2 v7 6/9] virtio-pci: Implement SR-IOV PF MIME-Version: 1.0 Message-Id: <20240813-sriov-v7-6-8515e3774df7@daynix.com> References: <20240813-sriov-v7-0-8515e3774df7@daynix.com> In-Reply-To: <20240813-sriov-v7-0-8515e3774df7@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::634; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x634.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Allow user to attach SR-IOV VF to a virtio-pci PF. Signed-off-by: Akihiko Odaki --- include/hw/virtio/virtio-pci.h | 1 + hw/virtio/virtio-pci.c | 20 +++++++++++++++----- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/include/hw/virtio/virtio-pci.h b/include/hw/virtio/virtio-pci.h index 9e67ba38c748..34539f2f6722 100644 --- a/include/hw/virtio/virtio-pci.h +++ b/include/hw/virtio/virtio-pci.h @@ -152,6 +152,7 @@ struct VirtIOPCIProxy { uint32_t modern_io_bar_idx; uint32_t modern_mem_bar_idx; int config_cap; + uint16_t last_pcie_cap_offset; uint32_t flags; bool disable_modern; bool ignore_backend_features; diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index 9534730bba19..0c8fcc5627d5 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -1955,6 +1955,7 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp) uint8_t *config; uint32_t size; VirtIODevice *vdev = virtio_bus_get_device(bus); + int16_t res; /* * Virtio capabilities present without @@ -2100,6 +2101,14 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp) pci_register_bar(&proxy->pci_dev, proxy->legacy_io_bar_idx, PCI_BASE_ADDRESS_SPACE_IO, &proxy->bar); } + + res = pcie_sriov_pf_init_from_user_created_vfs(&proxy->pci_dev, + proxy->last_pcie_cap_offset, + errp); + if (res > 0) { + proxy->last_pcie_cap_offset += res; + virtio_add_feature(&vdev->host_features, VIRTIO_F_SR_IOV); + } } static void virtio_pci_device_unplugged(DeviceState *d) @@ -2187,7 +2196,7 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) if (pcie_port && pci_is_express(pci_dev)) { int pos; - uint16_t last_pcie_cap_offset = PCI_CONFIG_SPACE_SIZE; + proxy->last_pcie_cap_offset = PCI_CONFIG_SPACE_SIZE; pos = pcie_endpoint_cap_init(pci_dev, 0); assert(pos > 0); @@ -2207,9 +2216,9 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) pci_set_word(pci_dev->config + pos + PCI_PM_PMC, 0x3); if (proxy->flags & VIRTIO_PCI_FLAG_AER) { - pcie_aer_init(pci_dev, PCI_ERR_VER, last_pcie_cap_offset, + pcie_aer_init(pci_dev, PCI_ERR_VER, proxy->last_pcie_cap_offset, PCI_ERR_SIZEOF, NULL); - last_pcie_cap_offset += PCI_ERR_SIZEOF; + proxy->last_pcie_cap_offset += PCI_ERR_SIZEOF; } if (proxy->flags & VIRTIO_PCI_FLAG_INIT_DEVERR) { @@ -2234,9 +2243,9 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) } if (proxy->flags & VIRTIO_PCI_FLAG_ATS) { - pcie_ats_init(pci_dev, last_pcie_cap_offset, + pcie_ats_init(pci_dev, proxy->last_pcie_cap_offset, proxy->flags & VIRTIO_PCI_FLAG_ATS_PAGE_ALIGNED); - last_pcie_cap_offset += PCI_EXT_CAP_ATS_SIZEOF; + proxy->last_pcie_cap_offset += PCI_EXT_CAP_ATS_SIZEOF; } if (proxy->flags & VIRTIO_PCI_FLAG_INIT_FLR) { @@ -2263,6 +2272,7 @@ static void virtio_pci_exit(PCIDevice *pci_dev) bool pcie_port = pci_bus_is_express(pci_get_bus(pci_dev)) && !pci_bus_is_root(pci_get_bus(pci_dev)); + pcie_sriov_pf_exit(&proxy->pci_dev); msix_uninit_exclusive_bar(pci_dev); if (proxy->flags & VIRTIO_PCI_FLAG_AER && pcie_port && pci_is_express(pci_dev)) { From patchwork Tue Aug 13 06:37:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13761382 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ABF78C52D7C for ; Tue, 13 Aug 2024 06:38:55 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sdlAE-0007WM-JQ; Tue, 13 Aug 2024 02:37:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sdlAA-0007Jc-Md for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:47 -0400 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sdlA9-0006UL-7B for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:46 -0400 Received: by mail-pj1-x1029.google.com with SMTP id 98e67ed59e1d1-2cb64529a36so3440522a91.0 for ; Mon, 12 Aug 2024 23:37:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1723531063; x=1724135863; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=K5bDA3kbj4aI3tc5UHvikrRJGrmEJbDMhb9LbvuEW9c=; b=jr8pjMyD3Vj0Jkn311TfKWayGumdk+E4PigLJy+lXdpEhQoMxhHh9wJ2ABCkeXnsqq GOaQgwPGaroxr9moLNFqmhkAwRDCub+rpucLOpGlF1PIOsg+9cnaj+4NsBx3mPDmh48s XnB4eHHS3q+n1vwMOOOxneGZbDphQaRGbMKp+DXEfSgfl2d97jLauw8ZI36WUXELx23I 2s6A5HXGh9+UqTfbSpDI2bZmFb33Nnh0rc9fFxbUZOyVCtssximk/aWvObkl/ePu9vQo w/I9eBkEGV3aHQ6q6LKEpnO60kz9Fk8+wH5jdlYDCchSSlN19lI5faadpX76FSkuexbY ZcYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723531063; x=1724135863; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=K5bDA3kbj4aI3tc5UHvikrRJGrmEJbDMhb9LbvuEW9c=; b=G8xIEnMG/kKeHbQ8ZR3gMg5foIEqSCbY/ojpsLpuNZeg9JMMovhUwoSc/QY6hMg2/G NvfQ91waNgRuUNxOKX154QPCm3aoTobEwAmexcvrdPyNIrJAAlKCscyR9I9BNpOSGV39 9fyhhoGoyIX3Q9nOF8Xc+W9PRdxaC9JMNU8/ysngHsdHAbVvbnIjY5OgX1KSHAWZyor3 EIjxK48G+laBONBNq8uikuli7j/QGW7T7nRULtDptrjgTleqeZXdNoOnPMH1DgKFO1KM DVUnuWnAvFtXd7nJZiEkYWI/CB1X1vTTV0ccsZAfVFLJasNGDPFYRHevrWKwKphrcLuI r9IQ== X-Gm-Message-State: AOJu0YyAQM6gy46F7T9sAUv1uC6yLokl9dxyEJbJ8qzKlMFGGhnKU9/p nP1ZOKObpaqmvfEQRpc5p85N04KpobAYi7jFsq0MqbyOmfAIqNTggoJt24sotmg= X-Google-Smtp-Source: AGHT+IGIR+HsQ/34qPixsQzlLTmVMPhJg/+aX+bdh8S0OqdiasWISKkxO58UIeI0GJV0zikat+1HYA== X-Received: by 2002:a17:90a:dd82:b0:2c9:8a4b:b4d3 with SMTP id 98e67ed59e1d1-2d39265f681mr2449417a91.43.1723531062587; Mon, 12 Aug 2024 23:37:42 -0700 (PDT) Received: from localhost ([157.82.202.230]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2d39882bc89sm343126a91.1.2024.08.12.23.37.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 12 Aug 2024 23:37:42 -0700 (PDT) From: Akihiko Odaki Date: Tue, 13 Aug 2024 15:37:06 +0900 Subject: [PATCH for-9.2 v7 7/9] virtio-net: Implement SR-IOV VF MIME-Version: 1.0 Message-Id: <20240813-sriov-v7-7-8515e3774df7@daynix.com> References: <20240813-sriov-v7-0-8515e3774df7@daynix.com> In-Reply-To: <20240813-sriov-v7-0-8515e3774df7@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::1029; envelope-from=akihiko.odaki@daynix.com; helo=mail-pj1-x1029.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A virtio-net device can be added as a SR-IOV VF to another virtio-pci device that will be the PF. Signed-off-by: Akihiko Odaki --- hw/virtio/virtio-net-pci.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/virtio/virtio-net-pci.c b/hw/virtio/virtio-net-pci.c index e03543a70a75..dba4987d6e04 100644 --- a/hw/virtio/virtio-net-pci.c +++ b/hw/virtio/virtio-net-pci.c @@ -75,6 +75,7 @@ static void virtio_net_pci_class_init(ObjectClass *klass, void *data) k->device_id = PCI_DEVICE_ID_VIRTIO_NET; k->revision = VIRTIO_PCI_ABI_VERSION; k->class_id = PCI_CLASS_NETWORK_ETHERNET; + k->sriov_vf_user_creatable = true; set_bit(DEVICE_CATEGORY_NETWORK, dc->categories); device_class_set_props(dc, virtio_net_properties); vpciklass->realize = virtio_net_pci_realize; From patchwork Tue Aug 13 06:37:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13761380 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E9894C52D7C for ; Tue, 13 Aug 2024 06:38:40 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sdlAK-0008A8-TO; Tue, 13 Aug 2024 02:37:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sdlAF-0007gw-HY for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:53 -0400 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sdlAC-0006Up-Sz for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:37:50 -0400 Received: by mail-pg1-x530.google.com with SMTP id 41be03b00d2f7-7bcf8077742so3675161a12.0 for ; Mon, 12 Aug 2024 23:37:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1723531067; x=1724135867; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=+42c5gGiKcfaS9Ju95w5EqxMNlhh/OqheyuSjM0gous=; b=GMeet9QwIcSW7CccUi4Kb2fWUW8va2TCpa0Bnavbb+F6dW3zuGu+bokZ8fsFtm53bm w2KZWKY8vV+Fx/hkWSPoj8x1XgWjAJuIKxu/KFk5nmBKBE4bSmApfFOSsaZiILyti7vJ c4ZmDVOxMZiTYXx1Q9ChPMskXBtD8KknmxHarKyUBIbDKTlJqqaY13L4NzfPb6KJe96m wLoIn5mIC4aZrGswWWG7EP9ET62OKpBZ3zD00c4TEFpLIYSjQPR/mdx5HZR06rlI2+Mn PylLnW2KpWUw5mlRZOuDs05dtx/xRNBACqkeBsUF84ypYWTVpmhHqChfzaABcIu919Mk YFVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723531067; x=1724135867; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+42c5gGiKcfaS9Ju95w5EqxMNlhh/OqheyuSjM0gous=; b=Rhqxis6DAH0D6fIzEXoijQZzcutx79ehB8jqD9EgLzKTw84T24QvqefiyT1UP1EL1H RWDknfCLp3FybvPcjpj71gUfnJavaulKdHj1Bi0I5pljCuCq/0fROsEWqu4qPRLupZBV nh2Xg8PvLpobE+caVSsbsC3Epf1iDoOV3JSefSMBElCa9MJjZdbAJz2IuwIgnDIR3RDS JIyY8AjUZ3/VDBDVx4k9RChsB3WZhbzl1VC6XfMfMUXWA9FHLu5FRN5ABpkgg5SU2Xeq yv+6ZQ424TVb9/tQZRQrmB6TQx8ZmUvoTPcybcdw0gQUC3ez89A6IOYlVR0r5Sj9keZ+ dLZQ== X-Gm-Message-State: AOJu0YxrC0cNhbYkJ+rIFakIHtOldvg38qpQZZ9YVAQUbPhip4SMlBS8 8ScvFrA59yuOVUit2Oyja52I9SSN84p6RJxyb6r1Y6iA3W/BkaUSLALUgl98oC0= X-Google-Smtp-Source: AGHT+IGajeKLi0558siYbfsWxIpwxXWp2zMyvjMsCw/alMSaWIZ7QC3KtxAia44f7w4ZWqfDo6HT2g== X-Received: by 2002:a05:6a20:2d0c:b0:1c4:a7a0:a7d4 with SMTP id adf61e73a8af0-1c8d74456eemr3800627637.7.1723531067336; Mon, 12 Aug 2024 23:37:47 -0700 (PDT) Received: from localhost ([157.82.202.230]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-201cd130045sm6672135ad.50.2024.08.12.23.37.44 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 12 Aug 2024 23:37:46 -0700 (PDT) From: Akihiko Odaki Date: Tue, 13 Aug 2024 15:37:07 +0900 Subject: [PATCH for-9.2 v7 8/9] docs: Document composable SR-IOV device MIME-Version: 1.0 Message-Id: <20240813-sriov-v7-8-8515e3774df7@daynix.com> References: <20240813-sriov-v7-0-8515e3774df7@daynix.com> In-Reply-To: <20240813-sriov-v7-0-8515e3774df7@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::530; envelope-from=akihiko.odaki@daynix.com; helo=mail-pg1-x530.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Akihiko Odaki --- MAINTAINERS | 1 + docs/system/index.rst | 1 + docs/system/sriov.rst | 36 ++++++++++++++++++++++++++++++++++++ 3 files changed, 38 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index e34c2bd4cda2..72b3c6736088 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2011,6 +2011,7 @@ F: hw/pci-bridge/* F: qapi/pci.json F: docs/pci* F: docs/specs/*pci* +F: docs/system/sriov.rst PCIE DOE M: Huai-Cheng Kuo diff --git a/docs/system/index.rst b/docs/system/index.rst index c21065e51932..718e9d3c56bb 100644 --- a/docs/system/index.rst +++ b/docs/system/index.rst @@ -39,3 +39,4 @@ or Hypervisor.Framework. multi-process confidential-guest-support vm-templating + sriov diff --git a/docs/system/sriov.rst b/docs/system/sriov.rst new file mode 100644 index 000000000000..a851a66a4b8b --- /dev/null +++ b/docs/system/sriov.rst @@ -0,0 +1,36 @@ +.. SPDX-License-Identifier: GPL-2.0-or-later + +Compsable SR-IOV device +======================= + +SR-IOV (Single Root I/O Virtualization) is an optional extended capability of a +PCI Express device. It allows a single physical function (PF) to appear as +multiple virtual functions (VFs) for the main purpose of eliminating software +overhead in I/O from virtual machines. + +There are devices with predefined SR-IOV configurations, but it is also possible +to compose an SR-IOV device yourself. Composing an SR-IOV device is currently +only supported by virtio-net-pci. + +Users can configure an SR-IOV-capable virtio-net device by adding +virtio-net-pci functions to a bus. Below is a command line example: + +.. code-block:: shell + + -netdev user,id=n -netdev user,id=o + -netdev user,id=p -netdev user,id=q + -device pcie-root-port,id=b + -device virtio-net-pci,bus=b,addr=0x0.0x3,netdev=q,sriov-pf=f + -device virtio-net-pci,bus=b,addr=0x0.0x2,netdev=p,sriov-pf=f + -device virtio-net-pci,bus=b,addr=0x0.0x1,netdev=o,sriov-pf=f + -device virtio-net-pci,bus=b,addr=0x0.0x0,netdev=n,id=f + +The VFs specify the paired PF with ``sriov-pf`` property. The PF must be +added after all VFs. It is the user's responsibility to ensure that VFs have +function numbers larger than one of the PF, and that the function numbers +have a consistent stride. + +You may also need to perform additional steps to activate the SR-IOV feature on +your guest. For Linux, refer to [1]_. + +.. [1] https://docs.kernel.org/PCI/pci-iov-howto.html From patchwork Tue Aug 13 06:37:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13761385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C22A4C52D7C for ; Tue, 13 Aug 2024 06:39:15 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sdlAQ-0000Jk-AW; Tue, 13 Aug 2024 02:38:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sdlAO-0000CL-W9 for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:38:01 -0400 Received: from mail-oi1-x22d.google.com ([2607:f8b0:4864:20::22d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sdlAH-0006VS-Pd for qemu-devel@nongnu.org; Tue, 13 Aug 2024 02:38:00 -0400 Received: by mail-oi1-x22d.google.com with SMTP id 5614622812f47-3db1e4219f8so2902319b6e.3 for ; Mon, 12 Aug 2024 23:37:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1723531072; x=1724135872; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=XP4TPVtZh3pRSWD5lDzkymaK3OFz5YOgTXNbBmvp+To=; b=SquWLOAVdVSVDtPjCXwE3Nfry/UpqOjGe4ZUvGICQIT1DAAlEpoWN89WQv2fN2f9pK pnUU/bkN9X4rFhpIga7VlhBNAcWMyUGOProoNrMt9M7Lt7YlRLPEoQbmqZF1rW/W6mxC CT1b0Ao+cw6xldqn1zQaUinPc9MNU2PQ+OX++zeAyupV0sF9Oqtv4uRsxZ53+3jhjS7D xTByDa9A3OnOxGP0TrVBlftrh2Zi4ZafnY6PkrugUlebBAd3AEUXmjaM40hK5wAtYa+Y PX55RXSYKNdOOurnBtfgrsHoZZ5p5UOFdwACLqLORcyo8ew4iSMTVRsgZhX9AKGFFz8W wfMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723531072; x=1724135872; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XP4TPVtZh3pRSWD5lDzkymaK3OFz5YOgTXNbBmvp+To=; b=Hz89pOLsHjhK0nHO5chW8q+5saZZ5xumWf5SzqGBy+kSL02JYiVu67tT729uH72Rqy ozNY7H7wAv1Tk8HeC40UbF5dbRifIHxl3Dv+NpIcsf0mrPu79YRnd9uDcyQ6fEaGkKQ0 Hs7q57HyBMFw457O7/aYNpGz7Bwk7lzXPjMVOIVvu5k7Q7GsKKbM4r/QEqi2bP8PnTDD pODp/kZTn9YYuMHHaPNweGp9nGWCz4REPTGJZyhuvUatFIACN+pjvQnohdE4bIeQhu/b ZIe4nPdMjWDQP1Lj7AckGs112AkrHs0bd6uBL/HQeoCJzrl3CJFs4QgJPUjZzT56644G YD2g== X-Gm-Message-State: AOJu0YwHz6VzPz4k+Mj5zKd82vQH8lTRadvsQWGS0e9OYeK7/gMydp0t e22UG/fn4fqiiuQrUnecTbTL45RHsA4Cr1cK6b9tQuelmPcRyZr6BMjYIJ6NmZA= X-Google-Smtp-Source: AGHT+IF1UakvDc9WUU1KqM1Htx5gUWRlW5rOFVbF3rQspx02BvyWdKP4qDhCh4KJruxWiSebBTF2LQ== X-Received: by 2002:a05:6808:398e:b0:3dc:1b09:55c9 with SMTP id 5614622812f47-3dd1ee32eaamr3551684b6e.12.1723531071891; Mon, 12 Aug 2024 23:37:51 -0700 (PDT) Received: from localhost ([157.82.202.230]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-710e58ca645sm5141237b3a.94.2024.08.12.23.37.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 12 Aug 2024 23:37:51 -0700 (PDT) From: Akihiko Odaki Date: Tue, 13 Aug 2024 15:37:08 +0900 Subject: [PATCH for-9.2 v7 9/9] pcie_sriov: Make a PCI device with user-created VF ARI-capable MIME-Version: 1.0 Message-Id: <20240813-sriov-v7-9-8515e3774df7@daynix.com> References: <20240813-sriov-v7-0-8515e3774df7@daynix.com> In-Reply-To: <20240813-sriov-v7-0-8515e3774df7@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::22d; envelope-from=akihiko.odaki@daynix.com; helo=mail-oi1-x22d.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Akihiko Odaki --- docs/system/sriov.rst | 3 ++- include/hw/pci/pcie_sriov.h | 7 +++++-- hw/pci/pcie_sriov.c | 8 +++++++- hw/virtio/virtio-pci.c | 16 ++++++++++------ 4 files changed, 24 insertions(+), 10 deletions(-) diff --git a/docs/system/sriov.rst b/docs/system/sriov.rst index a851a66a4b8b..d12178f3c319 100644 --- a/docs/system/sriov.rst +++ b/docs/system/sriov.rst @@ -28,7 +28,8 @@ virtio-net-pci functions to a bus. Below is a command line example: The VFs specify the paired PF with ``sriov-pf`` property. The PF must be added after all VFs. It is the user's responsibility to ensure that VFs have function numbers larger than one of the PF, and that the function numbers -have a consistent stride. +have a consistent stride. Both the PF and VFs are ARI-capable so you can have +255 VFs at maximum. You may also need to perform additional steps to activate the SR-IOV feature on your guest. For Linux, refer to [1]_. diff --git a/include/hw/pci/pcie_sriov.h b/include/hw/pci/pcie_sriov.h index f75b8f22ee92..aeaa38cf3456 100644 --- a/include/hw/pci/pcie_sriov.h +++ b/include/hw/pci/pcie_sriov.h @@ -43,12 +43,15 @@ void pcie_sriov_vf_register_bar(PCIDevice *dev, int region_num, /** * pcie_sriov_pf_init_from_user_created_vfs() - Initialize PF with user-created - * VFs. + * VFs, adding ARI to PF * @dev: A PCIe device being realized. * @offset: The offset of the SR-IOV capability. * @errp: pointer to Error*, to store an error if it happens. * - * Return: The size of added capability. 0 if the user did not create VFs. + * Initializes a PF with user-created VFs, adding the ARI extended capability to + * the PF. The VFs should call pcie_ari_init() to form an ARI device. + * + * Return: The size of added capabilities. 0 if the user did not create VFs. * -1 if failed. */ int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index f5b83a92a00c..94ab92d8c80d 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie_sriov.c @@ -238,6 +238,7 @@ int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, PCIDevice **vfs; BusState *bus = qdev_get_parent_bus(DEVICE(dev)); uint16_t ven_id = pci_get_word(dev->config + PCI_VENDOR_ID); + uint16_t size = PCI_EXT_CAP_SRIOV_SIZEOF; uint16_t vf_dev_id; uint16_t vf_offset; uint16_t vf_stride; @@ -304,6 +305,11 @@ int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, return -1; } + if (!pcie_find_capability(dev, PCI_EXT_CAP_ID_ARI)) { + pcie_ari_init(dev, offset + size); + size += PCI_ARI_SIZEOF; + } + for (i = 0; i < pf->len; i++) { vfs[i]->exp.sriov_vf.pf = dev; vfs[i]->exp.sriov_vf.vf_number = i; @@ -324,7 +330,7 @@ int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, } } - return PCI_EXT_CAP_SRIOV_SIZEOF; + return size; } bool pcie_sriov_register_device(PCIDevice *dev, Error **errp) diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index 0c8fcc5627d5..b19e2983ee22 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -2102,12 +2102,16 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp) PCI_BASE_ADDRESS_SPACE_IO, &proxy->bar); } - res = pcie_sriov_pf_init_from_user_created_vfs(&proxy->pci_dev, - proxy->last_pcie_cap_offset, - errp); - if (res > 0) { - proxy->last_pcie_cap_offset += res; - virtio_add_feature(&vdev->host_features, VIRTIO_F_SR_IOV); + if (pci_is_vf(&proxy->pci_dev)) { + pcie_ari_init(&proxy->pci_dev, proxy->last_pcie_cap_offset); + proxy->last_pcie_cap_offset += PCI_ARI_SIZEOF; + } else { + res = pcie_sriov_pf_init_from_user_created_vfs( + &proxy->pci_dev, proxy->last_pcie_cap_offset, errp); + if (res > 0) { + proxy->last_pcie_cap_offset += res; + virtio_add_feature(&vdev->host_features, VIRTIO_F_SR_IOV); + } } }