From patchwork Fri Nov 30 14:47:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Nefedov X-Patchwork-Id: 10706441 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 409A114D6 for ; Fri, 30 Nov 2018 14:49:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3040A30145 for ; Fri, 30 Nov 2018 14:49:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 24A3730154; Fri, 30 Nov 2018 14:49:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4FBBA30145 for ; Fri, 30 Nov 2018 14:49:14 +0000 (UTC) Received: from localhost ([::1]:60518 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gSk6T-0007yn-E1 for patchwork-qemu-devel@patchwork.kernel.org; Fri, 30 Nov 2018 09:49:13 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60178) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gSk4k-0005LW-QJ for qemu-devel@nongnu.org; Fri, 30 Nov 2018 09:47:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gSk4f-0003zK-PL for qemu-devel@nongnu.org; Fri, 30 Nov 2018 09:47:26 -0500 Received: from mail-eopbgr00109.outbound.protection.outlook.com ([40.107.0.109]:64560 helo=EUR02-AM5-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gSk4f-0003yU-Cl; Fri, 30 Nov 2018 09:47:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3oIHHJOu0Yf7EyP8c7i1GAvx4SdVcf8F2LyWHwCuECE=; b=djcC1i6qhy/Spocy24EShZbvLHV7KHAO0ZncYWGDTcCwLWc+INhbHZmErRB4om6OirgsDnef1NHXvjBMuiFLSQ2KMzKbxPZJzjrfnYO39wAa5Nx5aAF2mjAZFwMtPhpj/vldrEIqxA/WffkTdhzoTK/ce8HE3suaOeEJ5+KJZRI= Received: from AM0PR08MB3347.eurprd08.prod.outlook.com (52.134.94.148) by AM0PR08MB3891.eurprd08.prod.outlook.com (20.178.82.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1382.18; Fri, 30 Nov 2018 14:47:17 +0000 Received: from AM0PR08MB3347.eurprd08.prod.outlook.com ([fe80::a0bd:dda4:d9e4:1d01]) by AM0PR08MB3347.eurprd08.prod.outlook.com ([fe80::a0bd:dda4:d9e4:1d01%2]) with mapi id 15.20.1361.022; Fri, 30 Nov 2018 14:47:17 +0000 From: Anton Nefedov To: "qemu-devel@nongnu.org" Thread-Topic: [PATCH v6 0/9] discard blockstats Thread-Index: AQHUiLuWIP80K86Yf0K3X3gH7vHb+Q== Date: Fri, 30 Nov 2018 14:47:17 +0000 Message-ID: <20181130144705.77454-1-anton.nefedov@virtuozzo.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: HE1PR05CA0208.eurprd05.prod.outlook.com (2603:10a6:3:f9::32) To AM0PR08MB3347.eurprd08.prod.outlook.com (2603:10a6:208:5f::20) x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [185.231.240.5] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM0PR08MB3891; 20:6e/uJDxVZ/b1jOTSoVFDFrnbGvhQnRTicF00/uRolBQA1WNMmvUhMbqKhBVwA+BHwqYdAI4JoSt9a5m7CV7OlfD8FFktRudl+0KYTq/mggUa/rs/bVADf7RrsYE9ErJS7kz82md7Is8VVo9fyjeUlAW3tXKAomgwSpuvRubBHUU= x-ms-office365-filtering-correlation-id: 566fcf50-c756-4c3c-3243-08d656d2b90e x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390098)(7020095)(4652040)(8989299)(5600074)(711020)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7153060)(7193020); SRVR:AM0PR08MB3891; x-ms-traffictypediagnostic: AM0PR08MB3891: authentication-results: spf=none (sender IP is ) smtp.mailfrom=anton.nefedov@virtuozzo.com; x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(8121501046)(5005006)(10201501046)(3002001)(93006095)(93001095)(3231453)(999002)(944501466)(4982022)(52105112)(148016)(149066)(150057)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123564045)(20161123562045)(20161123560045)(201708071742011)(7699051)(76991095); SRVR:AM0PR08MB3891; BCL:0; PCL:0; RULEID:; SRVR:AM0PR08MB3891; x-forefront-prvs: 087223B4DA x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(136003)(396003)(346002)(39850400004)(376002)(366004)(199004)(189003)(6436002)(26005)(68736007)(52116002)(106356001)(3846002)(1076002)(2906002)(6916009)(6116002)(186003)(478600001)(99286004)(316002)(6506007)(102836004)(54906003)(256004)(25786009)(486006)(966005)(2501003)(386003)(44832011)(476003)(2616005)(7736002)(81166006)(8676002)(7416002)(305945005)(6512007)(36756003)(5660300001)(8936002)(6306002)(81156014)(4326008)(66066001)(14444005)(53936002)(107886003)(14454004)(97736004)(105586002)(6486002)(71200400001)(71190400001)(86362001)(2351001)(5640700003); DIR:OUT; SFP:1102; SCL:1; SRVR:AM0PR08MB3891; H:AM0PR08MB3347.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: virtuozzo.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: KlVkTbmy24FWUSCXVCLCY99bECCi14uDBx8sgZRlvkVcoiNcCtAyO7SFi7wzr3ZQB1/WQh0Gk2Xn3zvyluWQg67knLnowHDlKiwLFIeQPXH8PTftOtJE6YyDpKnKrAaEvbalQEAsr5yop7TtC0A9Rev64XwCdLYStONw5jQZ+FK6AUb3So/k7QI8Z4VS71GgLt/9d8FTj8JZ6LJp+tKlWRfhSIPL9jiYlR5M6EzDfovch9XSzQxqbVdG4dLat1om+Kh4DxuKgo+R3bzffhESeWWi/5fJ112TcdKhXvBB4z+gG0Yh/e/C+1Q5A8u7KT/8KXifCgszRRiU2t19gtcPqYY5Ban1LBgtPwy+0AbCb78= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 566fcf50-c756-4c3c-3243-08d656d2b90e X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Nov 2018 14:47:17.2313 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3891 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 40.107.0.109 Subject: [Qemu-devel] [PATCH v6 0/9] discard blockstats X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "kwolf@redhat.com" , Vladimir Sementsov-Ogievskiy , "famz@redhat.com" , Denis Lunev , "qemu-block@nongnu.org" , "armbru@redhat.com" , "mreitz@redhat.com" , "berto@igalia.com" , Anton Nefedov , "pbonzini@redhat.com" , "jsnow@redhat.com" Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP new in v6: - since clauses -> 4.0 - patch 9: qapi json cosmetic changes as suggested by Vladimir - patch 9: proper "driver" field setting for host_device driver v5: http://lists.nongnu.org/archive/html/qemu-devel/2018-10/msg06828.html v4: http://lists.nongnu.org/archive/html/qemu-devel/2018-08/msg04308.html v3: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg03688.html ---- qmp query-blockstats provides stats info for write/read/flush ops. Patches 1-7 implement the similar for discard (unmap) command for scsi and ide disks. Discard stat "unmap_ops / unmap_bytes" is supposed to account the ops that have completed without an error. However, discard operation is advisory. Specifically, - common block layer ignores ENOTSUP error code. That might be returned if the block driver does not support discard, or discard has been configured to be ignored. - format drivers such as qcow2 may ignore discard if they were configured to ignore that, or if the corresponding area is already marked unused (unallocated / zero clusters). And what is actually useful is the number of bytes actually discarded down on the host filesystem. To achieve that, driver-specific statistics has been added to blockstats (patch 9). With patch 8, file-posix driver accounts discard operations on its level too. query-blockstat result: (note the difference between blockdevice unmap and file discard stats. qcow2 sends fewer ops down to the file as the clusters are actually unallocated on qcow2 level) { "device": "drive-scsi0-0-0-0", "node-name": "#block159", "stats": { > "invalid_unmap_operations": 0, > "failed_unmap_operations": 0, "wr_highest_offset": 13411688448, "rd_total_time_ns": 2859566315, "rd_bytes": 103182336, "rd_merged": 0, "flush_operations": 19, "invalid_wr_operations": 0, "flush_total_time_ns": 23111608, "failed_rd_operations": 0, "failed_flush_operations": 0, "invalid_flush_operations": 0, "timed_stats": [ ], "wr_merged": 0, "wr_bytes": 1702912, > "unmap_bytes": 11954954240, > "unmap_operations": 865, "idle_time_ns": 2669508623, "account_invalid": true, > "unmap_total_time_ns": 19698002, "wr_operations": 143, "failed_wr_operations": 0, "rd_operations": 4816, "account_failed": true, > "unmap_merged": 0, "wr_total_time_ns": 1262686124, "invalid_rd_operations": 0 }, "parent": { > "driver-specific": { > "discard-nb-failed": 0, > "discard-bytes-ok": 720896, > "driver": "file", > "discard-nb-ok": 8 > }, "node-name": "#block009", "stats": { [..] } } }, { "device": "floppy0", Anton Nefedov (9): qapi: group BlockDeviceStats fields qapi: add unmap to BlockDeviceStats block: add empty account cookie type ide: account UNMAP (TRIM) operations scsi: store unmap offset and nb_sectors in request struct scsi: move unmap error checking to the complete callback scsi: account unmap operations file-posix: account discard operations qapi: query-blockstat: add driver specific file-posix stats qapi/block-core.json | 81 ++++++++++++++++++++++++++++++++------ include/block/accounting.h | 2 + include/block/block.h | 1 + include/block/block_int.h | 1 + block.c | 9 +++++ block/accounting.c | 6 +++ block/file-posix.c | 60 ++++++++++++++++++++++++++-- block/qapi.c | 11 ++++++ hw/ide/core.c | 12 ++++++ hw/scsi/scsi-disk.c | 32 +++++++++------ tests/qemu-iotests/227.out | 18 +++++++++ 11 files changed, 208 insertions(+), 25 deletions(-)