From patchwork Fri Aug 2 15:35:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony PERARD X-Patchwork-Id: 11073827 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E95D81399 for ; Fri, 2 Aug 2019 15:59:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D8D4828613 for ; Fri, 2 Aug 2019 15:59:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CCB9828872; Fri, 2 Aug 2019 15:59:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 267B5286DB for ; Fri, 2 Aug 2019 15:59:47 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htZwU-0005uc-3e; Fri, 02 Aug 2019 15:58:06 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htZwS-0005tm-Bq for xen-devel@lists.xenproject.org; Fri, 02 Aug 2019 15:58:04 +0000 X-Inumbo-ID: 4e4939a0-b53e-11e9-a1ad-ffb9cc75931d Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 4e4939a0-b53e-11e9-a1ad-ffb9cc75931d; Fri, 02 Aug 2019 15:58:01 +0000 (UTC) Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=anthony.perard@citrix.com; spf=Pass smtp.mailfrom=anthony.perard@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of anthony.perard@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="anthony.perard@citrix.com"; x-sender="anthony.perard@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa4.hc3370-68.iphmx.com: domain of anthony.perard@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="anthony.perard@citrix.com"; x-sender="anthony.perard@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ~all" Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="anthony.perard@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: EZTo8lTXZmgTU+dMVYJ0BJxfwtTobgE3Ei6AvjuVQeqwtZ+w9sfzcSGDuSH3u2KdbIUDI3Frup Gac/wVGGQNFH4eqJf7lEIHW6WJva4tL/pNmBuAvDQi3vT4HeQ+gdd93eeFSQVtlT2SL0VNP9ve RMIkZcOjSxWuIAU/Vaiiq0+1MMVhiFTORYbrGNpM62r3ShF7yPOmQ0D4+wbdFuFZcc7I6JZlfp Dk3+kijOr0BfGbNpfz+T9/o9+5r30sWz4Hie6669B4rIoXo4Y2UCRO4c0XdmrsDQNMvIrS0Qly E18= X-SBRS: 2.7 X-MesageID: 3951219 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.64,338,1559534400"; d="scan'208";a="3951219" From: Anthony PERARD To: Date: Fri, 2 Aug 2019 16:35:58 +0100 Message-ID: <20190802153606.32061-28-anthony.perard@citrix.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190802153606.32061-1-anthony.perard@citrix.com> References: <20190802153606.32061-1-anthony.perard@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH 27/35] libxl_pci: Use libxl__ao_device with libxl__device_pci_add X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Anthony PERARD , Ian Jackson , Wei Liu Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Anthony PERARD Acked-by: Ian Jackson --- tools/libxl/libxl_internal.h | 7 +- tools/libxl/libxl_pci.c | 170 ++++++++++++++++++++++++++++++----- 2 files changed, 150 insertions(+), 27 deletions(-) diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index 5c15a73a08..dd9934141f 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -198,6 +198,7 @@ typedef struct libxl__json_object libxl__json_object; typedef struct libxl__carefd libxl__carefd; typedef struct libxl__ev_lock libxl__ev_lock; typedef struct libxl__dm_resume_state libxl__dm_resume_state; +typedef struct libxl__ao_device libxl__ao_device; typedef struct libxl__domain_create_state libxl__domain_create_state; typedef void libxl__domain_create_cb(struct libxl__egc *egc, @@ -1592,8 +1593,9 @@ _hidden int libxl__pci_topology_init(libxl__gc *gc, /* from libxl_pci */ -_hidden int libxl__device_pci_add(libxl__gc *gc, uint32_t domid, - libxl_device_pci *pcidev, bool starting); +_hidden void libxl__device_pci_add(libxl__egc *egc, uint32_t domid, + libxl_device_pci *pcidev, bool starting, + libxl__ao_device *aodev); _hidden int libxl__device_pci_destroy_all(libxl__gc *gc, uint32_t domid); _hidden bool libxl__is_igd_vga_passthru(libxl__gc *gc, const libxl_domain_config *d_config); @@ -2571,7 +2573,6 @@ _hidden void libxl__kill(libxl__gc *gc, pid_t pid, int sig, const char *what); /*----- device addition/removal -----*/ -typedef struct libxl__ao_device libxl__ao_device; typedef struct libxl__multidev libxl__multidev; typedef void libxl__device_callback(libxl__egc*, libxl__ao_device*); diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c index 071880b855..503db6c260 100644 --- a/tools/libxl/libxl_pci.c +++ b/tools/libxl/libxl_pci.c @@ -983,9 +983,24 @@ static int qemu_pci_add_xenstore(libxl__gc *gc, uint32_t domid, return rc; } -static int do_pci_add(libxl__gc *gc, uint32_t domid, - libxl_device_pci *pcidev, bool starting) +typedef struct pci_add_state { + /* filled by user of do_pci_add */ + libxl__ao_device *aodev; + libxl_domid domid; + bool starting; + void (*callback)(libxl__egc *, struct pci_add_state *, int rc); + + /* private to do_pci_add */ + libxl_device_pci *pcidev; + int pci_domid; +} pci_add_state; + +static void do_pci_add(libxl__egc *egc, + libxl_domid domid, + libxl_device_pci *pcidev, + pci_add_state *pas) { + STATE_AO_GC(pas->aodev->ao); libxl_ctx *ctx = libxl__gc_owner(gc); libxl_domain_type type = libxl__domain_type(gc, domid); char *sysfs_path; @@ -997,6 +1012,13 @@ static int do_pci_add(libxl__gc *gc, uint32_t domid, bool isstubdom = libxl_is_stubdom(ctx, domid, &domainid); int r; + /* Convenience aliases */ + bool starting = pas->starting; + + /* init pci_add_state */ + pas->pcidev = pcidev; + pas->pci_domid = domid; + if (type == LIBXL_DOMAIN_TYPE_INVALID) { rc = ERROR_FAIL; goto out; @@ -1123,7 +1145,7 @@ static int do_pci_add(libxl__gc *gc, uint32_t domid, else rc = 0; out: - return rc; + pas->callback(egc, pas, rc); } static int libxl__device_pci_reset(libxl__gc *gc, unsigned int domain, unsigned int bus, @@ -1177,9 +1199,14 @@ int libxl_device_pci_add(libxl_ctx *ctx, uint32_t domid, const libxl_asyncop_how *ao_how) { AO_CREATE(ctx, domid, ao_how); - int rc; - rc = libxl__device_pci_add(gc, domid, pcidev, false); - libxl__ao_complete(egc, ao, rc); + libxl__ao_device *aodev; + + GCNEW(aodev); + libxl__prepare_ao_device(ao, aodev); + aodev->action = LIBXL__DEVICE_ACTION_ADD; + aodev->callback = device_addrm_aocomplete; + aodev->update_json = true; + libxl__device_pci_add(egc, domid, pcidev, false, aodev); return AO_INPROGRESS; } @@ -1200,14 +1227,31 @@ static int libxl_pcidev_assignable(libxl_ctx *ctx, libxl_device_pci *pcidev) return i != num; } -int libxl__device_pci_add(libxl__gc *gc, uint32_t domid, - libxl_device_pci *pcidev, bool starting) +static void device_pci_add_stubdom_done(libxl__egc *egc, + pci_add_state *, int rc); +static void device_pci_add_done(libxl__egc *egc, + pci_add_state *, int rc); + +void libxl__device_pci_add(libxl__egc *egc, uint32_t domid, + libxl_device_pci *pcidev, bool starting, + libxl__ao_device *aodev) { + STATE_AO_GC(aodev->ao); libxl_ctx *ctx = libxl__gc_owner(gc); - unsigned int orig_vdev, pfunc_mask; libxl_device_pci *assigned; - int num_assigned, i, rc; + int num_assigned, rc; int stubdomid = 0; + pci_add_state *pas; + + /* Store *pcidev to be used by callbacks */ + aodev->device_config = pcidev; + aodev->device_type = &libxl__pcidev_devtype; + + GCNEW(pas); + pas->aodev = aodev; + pas->domid = domid; + pas->starting = starting; + pas->callback = device_pci_add_stubdom_done; if (libxl__domain_type(gc, domid) == LIBXL_DOMAIN_TYPE_HVM) { rc = xc_test_assign_device(ctx->xch, domid, pcidev_encode_bdf(pcidev)); @@ -1254,13 +1298,39 @@ int libxl__device_pci_add(libxl__gc *gc, uint32_t domid, stubdomid = libxl_get_stubdom_id(ctx, domid); if (stubdomid != 0) { - libxl_device_pci pcidev_s = *pcidev; + libxl_device_pci *pcidev_s; + + GCNEW(pcidev_s); + libxl_device_pci_init(pcidev_s); + libxl_device_pci_copy(CTX, pcidev_s, pcidev); /* stubdomain is always running by now, even at create time */ - rc = do_pci_add(gc, stubdomid, &pcidev_s, false); - if ( rc ) - goto out; + pas->callback = device_pci_add_stubdom_done; + do_pci_add(egc, stubdomid, pcidev_s, pas); /* must be last */ + return; } + device_pci_add_stubdom_done(egc, pas, 0); /* must be last */ + return; + +out: + device_pci_add_done(egc, pas, rc); /* must be last */ +} + +static void device_pci_add_stubdom_done(libxl__egc *egc, + pci_add_state *pas, + int rc) +{ + STATE_AO_GC(pas->aodev->ao); + unsigned int orig_vdev, pfunc_mask; + int i; + + /* Convenience aliases */ + libxl__ao_device *aodev = pas->aodev; + libxl_domid domid = pas->domid; + libxl_device_pci *pcidev = aodev->device_config; + + if (rc) goto out; + orig_vdev = pcidev->vdevfn & ~7U; if ( pcidev->vfunc_mask == LIBXL_PCI_FUNC_ALL ) { @@ -1291,31 +1361,83 @@ int libxl__device_pci_add(libxl__gc *gc, uint32_t domid, */ pcidev->vdevfn = orig_vdev; } - if ( do_pci_add(gc, domid, pcidev, starting) ) - rc = ERROR_FAIL; + pas->callback = device_pci_add_done; + do_pci_add(egc, domid, pcidev, pas); /* must be last */ + return; } } out: - return rc; + device_pci_add_done(egc, pas, rc); +} + +static void device_pci_add_done(libxl__egc *egc, + pci_add_state *pas, + int rc) +{ + EGC_GC; + libxl__ao_device *aodev = pas->aodev; + libxl_domid domid = pas->domid; + libxl_device_pci *pcidev = aodev->device_config; + + if (rc) { + LOGD(ERROR, domid, + "libxl__device_pci_add failed for " + "PCI device %x:%x:%x.%x (rc %d)", + pcidev->domain, pcidev->bus, pcidev->dev, pcidev->func, + rc); + } + aodev->rc = rc; + aodev->callback(egc, aodev); } +typedef struct { + libxl__multidev multidev; + libxl__ao_device *outer_aodev; + libxl_domain_config *d_config; + libxl_domid domid; +} add_pcidevs_state; + +static void add_pcidevs_done(libxl__egc *, libxl__multidev *, int rc); + static void libxl__add_pcidevs(libxl__egc *egc, libxl__ao *ao, uint32_t domid, libxl_domain_config *d_config, libxl__multidev *multidev) { AO_GC; - libxl__ao_device *aodev = libxl__multidev_prepare(multidev); - int i, rc = 0; + add_pcidevs_state *apds; + int i; + + /* We need to start a new multidev in order to be able to execute + * libxl__create_pci_backend only once. */ + + GCNEW(apds); + apds->outer_aodev = libxl__multidev_prepare(multidev); + apds->d_config = d_config; + apds->domid = domid; + apds->multidev.callback = add_pcidevs_done; + libxl__multidev_begin(ao, &apds->multidev); for (i = 0; i < d_config->num_pcidevs; i++) { - rc = libxl__device_pci_add(gc, domid, &d_config->pcidevs[i], true); - if (rc < 0) { - LOGD(ERROR, domid, "libxl_device_pci_add failed: %d", rc); - goto out; - } + libxl__ao_device *aodev = libxl__multidev_prepare(&apds->multidev); + libxl__device_pci_add(egc, domid, &d_config->pcidevs[i], + true, aodev); } + libxl__multidev_prepared(egc, &apds->multidev, 0); +} + +static void add_pcidevs_done(libxl__egc *egc, libxl__multidev *multidev, + int rc) +{ + EGC_GC; + add_pcidevs_state *apds = CONTAINER_OF(multidev, *apds, multidev); + + /* Convenience aliases */ + libxl_domain_config *d_config = apds->d_config; + libxl_domid domid = apds->domid; + libxl__ao_device *aodev = apds->outer_aodev; + if (d_config->num_pcidevs > 0) { rc = libxl__create_pci_backend(gc, domid, d_config->pcidevs, d_config->num_pcidevs);