From patchwork Tue Oct 16 18:12:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kirti Wankhede X-Patchwork-Id: 10644047 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3DA4515E2 for ; Tue, 16 Oct 2018 18:14:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 305462A587 for ; Tue, 16 Oct 2018 18:14:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 244E92A5F1; Tue, 16 Oct 2018 18:14:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9B4F42A587 for ; Tue, 16 Oct 2018 18:14:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727476AbeJQCGC (ORCPT ); Tue, 16 Oct 2018 22:06:02 -0400 Received: from hqemgate15.nvidia.com ([216.228.121.64]:3981 "EHLO hqemgate15.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727395AbeJQCGB (ORCPT ); Tue, 16 Oct 2018 22:06:01 -0400 Received: from hqpgpgate102.nvidia.com (Not Verified[216.228.121.13]) by hqemgate15.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Tue, 16 Oct 2018 11:14:18 -0700 Received: from HQMAIL103.nvidia.com ([172.20.161.6]) by hqpgpgate102.nvidia.com (PGP Universal service); Tue, 16 Oct 2018 11:14:21 -0700 X-PGP-Universal: processed; by hqpgpgate102.nvidia.com on Tue, 16 Oct 2018 11:14:21 -0700 Received: from HQMAIL106.nvidia.com (172.18.146.12) by HQMAIL103.nvidia.com (172.20.187.11) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Tue, 16 Oct 2018 18:14:20 +0000 Received: from kwankhede-dev.nvidia.com (172.20.13.39) by HQMAIL106.nvidia.com (172.18.146.12) with Microsoft SMTP Server (TLS) id 15.0.1395.4 via Frontend Transport; Tue, 16 Oct 2018 18:14:18 +0000 From: Kirti Wankhede To: , CC: , , Kirti Wankhede Subject: [RFC PATCH v1 1/4] VFIO KABI for migration interface Date: Tue, 16 Oct 2018 23:42:35 +0530 Message-ID: <1539713558-2453-2-git-send-email-kwankhede@nvidia.com> X-Mailer: git-send-email 2.7.0 In-Reply-To: <1539713558-2453-1-git-send-email-kwankhede@nvidia.com> References: <1539713558-2453-1-git-send-email-kwankhede@nvidia.com> X-NVConfidentiality: public MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1539713658; bh=CyywSk+x9dw+XyZrPIPCCg6U8WO1bp20MNnNeaZT95I=; h=X-PGP-Universal:From:To:CC:Subject:Date:Message-ID:X-Mailer: In-Reply-To:References:X-NVConfidentiality:MIME-Version: Content-Type; b=OOJq4ELs+IUi8jWvpMKpi6qkhO6rEuGWyuWzPeC+3HIns87C2PrZBqFai4q3So8Pj 6pdeNNlgOLn7hu76TMq2JWAY9opBHoUtTkIItjFPnlYSyudQJqIMnRvd7AD1eVLQMa HXfAGhqUTLi1RGjPQPpnOCjxpVC7RiO+xKwAMaL3NQrBgKyQUnSs1Tzvh+/Kjx137T vqol4XCLEGdtkDBLtZAHvDZOw0rO/IaLUK3a0fTZEfqMK1iacvyyy7a/1ttgpMFGAQ U852pHrXXeAa27bmcopDzq5BwkYdZfoxI7pa/tN805ATAm/NJrrY9+rRPID/25p4w2 hyPyBGzmob2KA== Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP - Added vfio_device_migration_info structure to use interact with vendor driver. - Different flags are used to get or set migration related information from/to vendor driver. Flag VFIO_MIGRATION_PROBE: To query if feature is supported Flag VFIO_MIGRATION_GET_REGION: To get migration region info Flag VFIO_MIGRATION_SET_STATE: To convey device state in vendor driver Flag VFIO_MIGRATION_GET_PENDING: To get pending bytes yet to be migrated from vendor driver Flag VFIO_MIGRATION_GET_BUFFER: On this flag, vendor driver should write data to migration region and return number of bytes written in the region Flag VFIO_MIGRATION_SET_BUFFER: In migration resume path, user space app writes to migration region and communicates it to vendor driver with this ioctl with this flag. Flag VFIO_MIGRATION_GET_DIRTY_PFNS: Get bitmap of dirty pages from vendor driver from given start address - Added enum for possible device states. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia --- linux-headers/linux/vfio.h | 91 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h index 3615a269d378..8e9045ed9aa8 100644 --- a/linux-headers/linux/vfio.h +++ b/linux-headers/linux/vfio.h @@ -602,6 +602,97 @@ struct vfio_device_ioeventfd { #define VFIO_DEVICE_IOEVENTFD _IO(VFIO_TYPE, VFIO_BASE + 16) +/** + * VFIO_DEVICE_MIGRATION_INFO - _IOW(VFIO_TYPE, VFIO_BASE + 17, + * struct vfio_device_migration_info) + * Flag VFIO_MIGRATION_PROBE: + * To query if feature is supported + * + * Flag VFIO_MIGRATION_GET_REGION: + * To get migration region info + * region_index [output] : region index to be used for migration region + * size [output]: size of migration region + * + * Flag VFIO_MIGRATION_SET_STATE: + * To set device state in vendor driver + * device_state [input] : User space app sends device state to vendor + * driver on state change + * + * Flag VFIO_MIGRATION_GET_PENDING: + * To get pending bytes yet to be migrated from vendor driver + * threshold_size [Input] : threshold of buffer in User space app. + * pending_precopy_only [output] : pending data which must be migrated in + * precopy phase or in stopped state, in other words - before target + * vm start + * pending_compatible [output] : pending data which may be migrated in any + * phase + * pending_postcopy_only [output] : pending data which must be migrated in + * postcopy phase or in stopped state, in other words - after source + * vm stop + * Sum of pending_precopy_only, pending_compatible and + * pending_postcopy_only is the whole amount of pending data. + * + * Flag VFIO_MIGRATION_GET_BUFFER: + * On this flag, vendor driver should write data to migration region and + * return number of bytes written in the region. + * bytes_written [output] : number of bytes written in migration buffer by + * vendor driver + * + * Flag VFIO_MIGRATION_SET_BUFFER + * In migration resume path, user space app writes to migration region and + * communicates it to vendor driver with this ioctl with this flag. + * bytes_written [Input] : number of bytes written in migration buffer by + * user space app. + * + * Flag VFIO_MIGRATION_GET_DIRTY_PFNS + * Get bitmap of dirty pages from vendor driver from given start address. + * start_addr [Input] : start address + * pfn_count [Input] : Total pfn count from start_addr for which dirty + * bitmap is requested + * dirty_bitmap [Output] : bitmap memory allocated by user space + * application, vendor driver should return the bitmap with bits set + * only for pages to be marked dirty. + * Return: 0 on success, -errno on failure. + */ + +struct vfio_device_migration_info { + __u32 argsz; + __u32 flags; +#define VFIO_MIGRATION_PROBE (1 << 0) +#define VFIO_MIGRATION_GET_REGION (1 << 1) +#define VFIO_MIGRATION_SET_STATE (1 << 2) +#define VFIO_MIGRATION_GET_PENDING (1 << 3) +#define VFIO_MIGRATION_GET_BUFFER (1 << 4) +#define VFIO_MIGRATION_SET_BUFFER (1 << 5) +#define VFIO_MIGRATION_GET_DIRTY_PFNS (1 << 6) + __u32 region_index; /* region index */ + __u64 size; /* size */ + __u32 device_state; /* VFIO device state */ + __u64 pending_precopy_only; + __u64 pending_compatible; + __u64 pending_postcopy_only; + __u64 threshold_size; + __u64 bytes_written; + __u64 start_addr; + __u64 pfn_count; + __u8 dirty_bitmap[]; +}; + +enum { + VFIO_DEVICE_STATE_NONE, + VFIO_DEVICE_STATE_RUNNING, + VFIO_DEVICE_STATE_MIGRATION_SETUP, + VFIO_DEVICE_STATE_MIGRATION_PRECOPY_ACTIVE, + VFIO_DEVICE_STATE_MIGRATION_STOPNCOPY_ACTIVE, + VFIO_DEVICE_STATE_MIGRATION_SAVE_COMPLETED, + VFIO_DEVICE_STATE_MIGRATION_RESUME, + VFIO_DEVICE_STATE_MIGRATION_RESUME_COMPLETED, + VFIO_DEVICE_STATE_MIGRATION_FAILED, + VFIO_DEVICE_STATE_MIGRATION_CANCELLED, +}; + +#define VFIO_DEVICE_MIGRATION_INFO _IO(VFIO_TYPE, VFIO_BASE + 17) + /* -------- API for Type1 VFIO IOMMU -------- */ /**