From patchwork Tue Sep 5 16:31:00 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Venu Busireddy X-Patchwork-Id: 9939289 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 23F0B604D3 for ; Tue, 5 Sep 2017 16:32:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 178F91FF83 for ; Tue, 5 Sep 2017 16:32:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0CA9F289A6; Tue, 5 Sep 2017 16:32:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 00A901FF83 for ; Tue, 5 Sep 2017 16:32:00 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dpGjX-0001pS-2H; Tue, 05 Sep 2017 16:29:51 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dpGjV-0001p9-Qz for xen-devel@lists.xen.org; Tue, 05 Sep 2017 16:29:49 +0000 Received: from [85.158.143.35] (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256 bits)) by server-2.bemta-6.messagelabs.com id 5A/66-07918-DF0DEA95; Tue, 05 Sep 2017 16:29:49 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrNLMWRWlGSWpSXmKPExsXSO6nOVffPhXW RBoe/S1gs+biYxYHR4+ju30wBjFGsmXlJ+RUJrBlvdjxmK7ivW7Hg7gy2BsYXyl2MXBxCApOY JLbPvMkG4fxhlOh52MQM4WxglDg3YxILhNPJKNH27RBTFyMnB5uAgcTRwz2sIAkRgTZGidUnL jOCJJgFaiT6vq0ASnBwCAuES9z8nwBisgioSnR8swap4BWwlViy/QI7iC0hICexdPt1ZhCbU8 BO4tXmZ2C2EFDN0sdtjBA1xhLtby+yTWDkW8DIsIpRvTi1qCy1SNdYL6koMz2jJDcxM0fX0MB MLze1uDgxPTUnMalYLzk/dxMjMFAYgGAHY8c/p0OMkhxMSqK8blvXRQrxJeWnVGYkFmfEF5Xm pBYfYpTh4FCS4BU6D5QTLEpNT61Iy8wBhixMWoKDR0mEVxokzVtckJhbnJkOkTrFqCglzrvlH FBCACSRUZoH1waLk0uMslLCvIxAhwjxFKQW5WaWoMq/YhTnYFQS5mUDGc+TmVcCN/0V0GImoM VVL9eALC5JREhJNTA67d9ysS7pfFL8scrKlICqTg/PnAthH05PVry4LfXVpphg1UKBpXwCPwU /MHfn2t3U21Ky2iYj77m75ddcx7fvVJQ7VLYaJzhNUL5UULVyemNuHe/koHXa+fmJhz2cGa3k 1cI3Pfr4+PCZAxOFeOukCp2+3tzr45m4Mv+3i9c+/rflR1q2dyuxFGckGmoxFxUnAgCOTgHJj gIAAA== X-Env-Sender: venu.busireddy@oracle.com X-Msg-Ref: server-16.tower-21.messagelabs.com!1504628987!69370214!1 X-Originating-IP: [141.146.126.69] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTQxLjE0Ni4xMjYuNjkgPT4gMjc3MjE4\n X-StarScan-Received: X-StarScan-Version: 9.4.45; banners=-,-,- X-VirusChecked: Checked Received: (qmail 8168 invoked from network); 5 Sep 2017 16:29:48 -0000 Received: from aserp1040.oracle.com (HELO aserp1040.oracle.com) (141.146.126.69) by server-16.tower-21.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 5 Sep 2017 16:29:48 -0000 Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id v85GTg4h009511 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 5 Sep 2017 16:29:43 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id v85GTgIO017801 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 5 Sep 2017 16:29:42 GMT Received: from abhmp0006.oracle.com (abhmp0006.oracle.com [141.146.116.12]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id v85GTgGd022200; Tue, 5 Sep 2017 16:29:42 GMT Received: from x52.us.oracle.com (/10.153.74.183) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 05 Sep 2017 09:29:42 -0700 From: Venu Busireddy To: venu.busireddy@oracle.com, xen-devel@lists.xen.org, Ian Jackson , Wei Liu Date: Tue, 5 Sep 2017 11:31:00 -0500 Message-Id: <20170905163101.3730-2-venu.busireddy@oracle.com> X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170905163101.3730-1-venu.busireddy@oracle.com> References: <20170905163101.3730-1-venu.busireddy@oracle.com> X-Source-IP: aserv0022.oracle.com [141.146.126.234] Cc: Andrew Cooper , Jan Beulich Subject: [Xen-devel] [PATCH v4 1/2] libxl: Implement the handler to handle unrecoverable AER errors X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Implement the callback function to handle unrecoverable AER errors, and also the public APIs that can be used to register/unregister the handler. When an AER error occurs, the handler will forcibly remove the erring PCIe device from the guest. Signed-off-by: Venu Busireddy --- tools/libxl/libxl.h | 7 ++++ tools/libxl/libxl_event.h | 13 +++++++ tools/libxl/libxl_internal.h | 7 ++++ tools/libxl/libxl_pci.c | 84 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 111 insertions(+) diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index 7cf0f31..03c5565 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -1044,6 +1044,13 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src); */ #define LIBXL_HAVE_QED 1 +/* LIBXL_HAVE_AER_EVENTS_HANDLER + * + * If this is defined, libxl has the library functions called + * libxl_reg_aer_events_handler and libxl_unreg_aer_events_handler. + */ +#define LIBXL_HAVE_AER_EVENTS_HANDLER 1 + typedef char **libxl_string_list; void libxl_string_list_dispose(libxl_string_list *sl); int libxl_string_list_length(const libxl_string_list *sl); diff --git a/tools/libxl/libxl_event.h b/tools/libxl/libxl_event.h index 1ea789e..1aea906 100644 --- a/tools/libxl/libxl_event.h +++ b/tools/libxl/libxl_event.h @@ -184,6 +184,19 @@ void libxl_evdisable_domain_death(libxl_ctx *ctx, libxl_evgen_domain_death*); * may generate only a DEATH event. */ +typedef struct libxl__aer_watch libxl_aer_watch; +int libxl_reg_aer_events_handler(libxl_ctx *, uint32_t, libxl_aer_watch **) + LIBXL_EXTERNAL_CALLERS_ONLY; + /* + * Registers a handler to handle the occurrence of unrecoverable AER errors. + * This function depends on the calling application running the libxl's + * internal event loop. Toolstacks that do not use libxl's internal + * event loop must arrange to have their own event loop created and enter + * libxl (say, call libxl_event_wait()), to enable the event to be processed. + */ +void libxl_unreg_aer_events_handler(libxl_ctx *, uint32_t, libxl_aer_watch *) + LIBXL_EXTERNAL_CALLERS_ONLY; + typedef struct libxl__evgen_disk_eject libxl_evgen_disk_eject; int libxl_evenable_disk_eject(libxl_ctx *ctx, uint32_t domid, const char *vdev, libxl_ev_user, libxl_evgen_disk_eject **evgen_out); diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index afe6652..2b74286 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -352,6 +352,13 @@ struct libxl__ev_child { LIBXL_LIST_ENTRY(struct libxl__ev_child) entry; }; +/* + * Structure used for AER event handling. + */ +struct libxl__aer_watch { + uint32_t domid; + libxl__ev_xswatch watch; +}; /* * evgen structures, which are the state we use for generating diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c index 65ad5e5..d1008f8 100644 --- a/tools/libxl/libxl_pci.c +++ b/tools/libxl/libxl_pci.c @@ -1678,6 +1678,90 @@ static int libxl_device_pci_compare(libxl_device_pci *d1, return COMPARE_PCI(d1, d2); } +static void aer_backend_watch_callback(libxl__egc *egc, + libxl__ev_xswatch *watch, + const char *watch_path, + const char *event_path) +{ + EGC_GC; + libxl_aer_watch *aer_ws = CONTAINER_OF(watch, *aer_ws, watch); + int rc; + uint32_t dom, bus, dev, fn; + uint32_t domid = aer_ws->domid; + char *p, *path; + const char *aerFailedSBDF; + libxl_device_pci pcidev; + + /* Extract the backend directory. */ + path = libxl__strdup(gc, event_path); + p = strrchr(path, '/'); + if ((p == NULL) || (strcmp(p, "/aerFailedSBDF") != 0)) + return; + /* Truncate the string so it points to the backend directory. */ + *p = '\0'; + + /* Fetch the value of the failed PCI device. */ + rc = libxl__xs_read_checked(gc, XBT_NULL, + GCSPRINTF("%s/aerFailedSBDF", path), &aerFailedSBDF); + if (rc || !aerFailedSBDF) + return; + sscanf(aerFailedSBDF, "%x:%x:%x.%x", &dom, &bus, &dev, &fn); + + libxl_device_pci_init(&pcidev); + pcidev_struct_fill(&pcidev, dom, bus, dev, fn, 0); + /* Forcibly remove the device from the guest */ + rc = libxl__device_pci_remove_common(gc, domid, &pcidev, 1); + if (rc) + LOGD(ERROR, domid, " libxl__device_pci_remove_common() failed, rc=x%x", + (unsigned int)rc); + + return; +} + +int libxl_reg_aer_events_handler(libxl_ctx *ctx, + uint32_t domid, + libxl_aer_watch **aer_ws_out) +{ + int rc = 0; + uint32_t pciback_domid; + char *be_path; + libxl_aer_watch *aer_ws = NULL; + GC_INIT(ctx); + + *aer_ws_out = NULL; + + rc = libxl__get_domid(gc, (uint32_t *)(&pciback_domid)); + if (rc) { + LOGD(ERROR, domid, " libxl__get_domid() failed, rc = %d", rc); + goto out; + } + + aer_ws = libxl__calloc(NOGC, 1, sizeof(libxl_aer_watch)); + aer_ws->domid = domid; + be_path = GCSPRINTF("/local/domain/%u/backend/pci/%u/%u/%s", + pciback_domid, domid, pciback_domid, "aerFailedSBDF"); + rc = libxl__ev_xswatch_register(gc, &aer_ws->watch, + aer_backend_watch_callback, be_path); + *aer_ws_out = aer_ws; + +out: + GC_FREE; + return rc; +} + +void libxl_unreg_aer_events_handler(libxl_ctx *ctx, + uint32_t domid, + libxl_aer_watch *aer_ws) +{ + GC_INIT(ctx); + + libxl__ev_xswatch_deregister(gc, &aer_ws->watch); + + free(aer_ws); + GC_FREE; + return; +} + DEFINE_DEVICE_TYPE_STRUCT_X(pcidev, pci); /*