From patchwork Thu May 4 11:44:21 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 9711789 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id AF0F660362 for ; Thu, 4 May 2017 12:04:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 90C382623C for ; Thu, 4 May 2017 12:04:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 830DB28614; Thu, 4 May 2017 12:04:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D07E82623C for ; Thu, 4 May 2017 12:04:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ZSzBqNy68b4Na3TYzvhXROo66Bzbg1BjNQb1QGSpOP4=; b=Fnnw4znL/L/dezhCYJ908jjIrh ArdQyrX6J5KFN8i+WoBF0LOUNlKspbK2WW8DbJSX3ad1mpgJMskC8el3UlxM8JXqDYkV5fWee0Xb1 kAcrIfI5H6198TJ+msSoV0dCVa8iZG+m9LiDaUccjJMGdeykVEimftNegRO2AM8wxnMeYTVAHQCt+ pvMTTAFrhYA1jr2A1p0CX4ozuDmycL8Vwzx6uaboc0PYaHNQ9IuFeQHcx/UFA78mgJZbHQ5vjDAtb e8Y9aPFIZxo8cJBPiIeT1zD4i1e0pX4FjyCkd3pvQL+dQ/MN9w4icKwSR/EOQ9HdjZOhpsygxcbDr lEIe9E9w==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1d6FUv-0002Ko-DD; Thu, 04 May 2017 12:04:41 +0000 Received: from merlin.infradead.org ([2001:4978:20e::2]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1d6FRN-0005KN-Um for linux-arm-kernel@bombadil.infradead.org; Thu, 04 May 2017 12:01:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=fxtfhkerkTKYIzGJ7zSHdMpJUI2pr+5z3cx3/7AtpvM=; b=uErqXD+MjzENjbY2p52NVgQkT k9rG+ECbzioZwMc1f9z2CyZ/3LkIsiVDiJXym2XnLJ1ORNwh1KfmiSpLrnvWs+oE4iZCK8d4SwdvS y/Uo6+hRtG8csWuI6KODYe0R2eTDv618mdUi5h+LBoY9rbg3Z7e1vZfAUlGEn9pn7E2sVuvpdELLT dPYkJlacNHo45daddJUq/QWuCzw9YdgBC4vp7IruB/hWgWUrgghEBH+ozjvn17WuWZ38fgEjuek5/ osCrfXjcDEriLY9jimRihsvoj2eus6zql5GGcaSxQxGBssr1c1xtWAtKZD/A7/4DNmw4dkZY2F2bu rm7sZjlzA==; Received: from mx1.redhat.com ([209.132.183.28]) by merlin.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1d6FCA-0004y7-Os for linux-arm-kernel@lists.infradead.org; Thu, 04 May 2017 11:45:19 +0000 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D534DC059748; Thu, 4 May 2017 11:44:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D534DC059748 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eric.auger@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com D534DC059748 Received: from localhost.localdomain.com (ovpn-116-214.ams2.redhat.com [10.36.116.214]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7F26C53CC2; Thu, 4 May 2017 11:44:51 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, marc.zyngier@arm.com, christoffer.dall@linaro.org, andre.przywara@arm.com, vijayak@caviumnetworks.com, Vijaya.Kumar@cavium.com, peter.maydell@linaro.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org Subject: [PATCH v6 01/24] KVM: arm/arm64: Add ITS save/restore API documentation Date: Thu, 4 May 2017 13:44:21 +0200 Message-Id: <1493898284-29504-2-git-send-email-eric.auger@redhat.com> In-Reply-To: <1493898284-29504-1-git-send-email-eric.auger@redhat.com> References: <1493898284-29504-1-git-send-email-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Thu, 04 May 2017 11:44:57 +0000 (UTC) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170504_074518_929939_3E33B4F4 X-CRM114-Status: GOOD ( 19.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Prasun.Kapoor@cavium.com, drjones@redhat.com, quintela@redhat.com, dgilbert@redhat.com, pbonzini@redhat.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Add description for how to access ITS registers and how to save/restore ITS tables into/from memory. Signed-off-by: Eric Auger --- v5 -> v6: - add restoration ordering - 256B -> 256 Byte aligned - DTE Size is number of bits for the EVENTID - s/GITS_READR/GITS_CREADR v4 -> v5: - take into account Christoffer's comments - pending table save on GICV3 side now v3 -> v4: - take into account Peter's comments: - typos - KVM_DEV_ARM_VGIC_GRP_ITS_TABLES kvm_device_attr = 0 - add a validity bit in DTE - document all fields in CTE and ITE - document ABI revision - take into account Andre's comments: - document restrictions about GITS_CREADR writing and GITS_IIDR - document -EBUSY error if one or more VCPUS are runnning - document 64b registers only can be accessed with 64b access - itt_addr field matches bits [51:8] of the itt_addr v1 -> v2: - DTE and ITE now are 8 bytes - DTE and ITE now indexed by deviceid/eventid - use ITE name instead of ITTE - mentions ITT_addr matches bits [51:8] of the actual address - mentions LE layout --- Documentation/virtual/kvm/devices/arm-vgic-its.txt | 117 +++++++++++++++++++++ 1 file changed, 117 insertions(+) diff --git a/Documentation/virtual/kvm/devices/arm-vgic-its.txt b/Documentation/virtual/kvm/devices/arm-vgic-its.txt index 6081a5b..863c359 100644 --- a/Documentation/virtual/kvm/devices/arm-vgic-its.txt +++ b/Documentation/virtual/kvm/devices/arm-vgic-its.txt @@ -32,7 +32,124 @@ Groups: KVM_DEV_ARM_VGIC_CTRL_INIT request the initialization of the ITS, no additional parameter in kvm_device_attr.addr. + + KVM_DEV_ARM_ITS_SAVE_TABLES + save the ITS table data into guest RAM, at the location provisioned + by the guest in corresponding registers/table entries. + + The layout of the tables in guest memory defines an ABI. The entries + are laid out in little endian format as described in the last paragraph. + + KVM_DEV_ARM_ITS_RESTORE_TABLES + restore the ITS tables from guest RAM to ITS internal structures. + + The GICV3 must be restored before the ITS and all ITS registers but + the GITS_CTLR must be restored before restoring the ITS tables. + + The GITS_IIDR read-only register must also be restored before + the table restore as the IIDR revision field encodes the ABI revision. + + The expected ordering when restoring the GICv3/ITS is described in section + "ITS Restoration Sequence". + Errors: -ENXIO: ITS not properly configured as required prior to setting this attribute -ENOMEM: Memory shortage when allocating ITS internal data + -EINVAL: Inconsistent restored data + -EFAULT: Invalid guest ram access + -EBUSY: One or more VCPUS are running + + KVM_DEV_ARM_VGIC_GRP_ITS_REGS + Attributes: + The attr field of kvm_device_attr encodes the offset of the + ITS register, relative to the ITS control frame base address + (ITS_base). + + kvm_device_attr.addr points to a __u64 value whatever the width + of the addressed register (32/64 bits). 64 bit registers can only + be accessed with full length. + + Writes to read-only registers are ignored by the kernel except for: + - GITS_CREADR. It needs to be restored otherwise commands in the queue + will be re-executed after restoring CWRITER. GITS_CREADR must be + restored before restoring the GITS_CTLR which is likely to enable the + ITS. Also it needs to be restored after GITS_CBASER since a write to + GITS_CBASER resets GITS_CREADR. + - GITS_IIDR. Its Revision field encodes the table layout ABI revision. + In the future we might implement direct injection of virtual LPIS. + This will require an upgrade of the table layout and an evolution of + the ABI. GITS_IIDR must be restored before the table restoration. + + For other registers, getting or setting a register has the same + effect as reading/writing the register on real hardware. + Errors: + -ENXIO: Offset does not correspond to any supported register + -EFAULT: Invalid user pointer for attr->addr + -EINVAL: Offset is not 64-bit aligned + -EBUSY: one or more VCPUS are running + + ITS Restoration Sequence: + ------------------------- + +The following ordering must be followed when restoring the GIC and the ITS: +a) restore all guest memory and create vcpus +b) initialize the GIC and ITS (KVM_DEV_ARM_VGIC_CTRL_INIT) +c) provide GIC and ITS base addresses (KVM_DEV_ARM_VGIC_GRP_ADDR) +d) restore all redistributors +e) restore the ITS in the following order: + 1. Restore GITS_CBASER + 2. Restore all other GITS_ registers, except GITS_CTLR! + 3. Load the ITS table data (KVM_DEV_ARM_ITS_RESTORE_TABLES) + 4. Restore GITS_CTLR +f) start vcpus + + ITS Table ABI REV0: + ------------------- + + Revision 0 of the ABI only supports physical LPIs. + + The device table and ITT are indexed by the deviceid and eventid, + respectively. The collection table is not indexed by collectionid: + CTE are written in the table in the order of collection creation. All + entries are 8 bytes. + + Device Table Entry (DTE): + + bits: | 63| 62 ... 49 | 48 ... 5 | 4 ... 0 | + values: | V | next | ITT_addr | Size | + + where; + - V indicates whether the entry is valid. If not, other fields + are not meaningful. + - next: equals to 0 if this entry is the last one; otherwise it + corresponds to the deviceid offset to the next DTE, capped by + 2^14 -1. + - ITT_addr matches bits [51:8] of the ITT address (256 Byte aligned). + - Size specifies the supported number of bits for the eventid, + minus one + + Collection Table Entry (CTE): + + bits: | 63| 62 .. 52 | 51 ... 16 | 15 ... 0 | + values: | V | RES0 | RDBase | ICID | + + where: + - V indicates whether the entry is valid. If not, other fields are + not meaningful. + - RES0: reserved field with Should-Be-Zero-or-Preserved behavior. + - RDBase is the PE number (GICR_TYPER.Processor_Number semantic), + - ICID is the collection ID + + Interrupt Translation Entry (ITE): + + bits: | 63 ... 48 | 47 ... 16 | 15 ... 0 | + values: | next | pINTID | ICID | + + where: + - next: equals to 0 if this entry is the last one; otherwise it corresponds + to the eventid offset to the next ITE capped by 2^16 -1. + - pINTID is the physical LPI ID; if zero, it means the entry is not valid + and other fields are not meaningful. + - ICID is the collection ID +