From patchwork Wed Oct 31 11:34:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Nefedov X-Patchwork-Id: 10662507 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F30D717DB for ; Wed, 31 Oct 2018 11:36:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E684D2A726 for ; Wed, 31 Oct 2018 11:36:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D92A32A903; Wed, 31 Oct 2018 11:36:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 586632A726 for ; Wed, 31 Oct 2018 11:36:44 +0000 (UTC) Received: from localhost ([::1]:58728 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gHonj-0004ZY-GI for patchwork-qemu-devel@patchwork.kernel.org; Wed, 31 Oct 2018 07:36:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34486) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gHolv-0003Iu-Ls for qemu-devel@nongnu.org; Wed, 31 Oct 2018 07:34:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gHolr-0001BJ-0V for qemu-devel@nongnu.org; Wed, 31 Oct 2018 07:34:51 -0400 Received: from mail-he1eur01on0118.outbound.protection.outlook.com ([104.47.0.118]:30018 helo=EUR01-HE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gHolq-000175-7a; Wed, 31 Oct 2018 07:34:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VRjsflUzjQtUed8DXJTqN9auk7g4abUs01LA1kivcjI=; b=GKR4FP/mfCXMy1SQPZZPYdmbxUlDbkECaqhjBNs7RaclBLAkdwDjApqvp4TnlS0r9IbdFf1qrwUeFY+Gbo80qiO5r1HeHighCvLZNAJZBlPt2PrhfWZ7MBmiY/VtNYPor+Lv2LwzpAR8lkAobAcPdpbfBo06/4MWFHVWAxiRXIY= Received: from VI1PR08MB3358.eurprd08.prod.outlook.com (52.134.31.145) by VI1PR08MB3534.eurprd08.prod.outlook.com (20.177.61.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1273.21; Wed, 31 Oct 2018 11:34:37 +0000 Received: from VI1PR08MB3358.eurprd08.prod.outlook.com ([fe80::614e:e2cd:51e1:a19c]) by VI1PR08MB3358.eurprd08.prod.outlook.com ([fe80::614e:e2cd:51e1:a19c%4]) with mapi id 15.20.1273.027; Wed, 31 Oct 2018 11:34:37 +0000 From: Anton Nefedov To: "qemu-devel@nongnu.org" Thread-Topic: [PATCH v5 0/9] discard blockstats Thread-Index: AQHUcQ20okiLF3DyKk2300bnhLbS3A== Date: Wed, 31 Oct 2018 11:34:37 +0000 Message-ID: <20181031113418.29796-1-anton.nefedov@virtuozzo.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: AM6PR05CA0028.eurprd05.prod.outlook.com (2603:10a6:20b:2e::41) To VI1PR08MB3358.eurprd08.prod.outlook.com (2603:10a6:803:3f::17) authentication-results: spf=none (sender IP is ) smtp.mailfrom=anton.nefedov@virtuozzo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [185.231.240.5] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; VI1PR08MB3534; 20:MntrdpoD38aVfSUJx0y/8ENIaS2YUpJ5VFANV/16WnIwrQkB9jksYfuT63H+iPQdPEX9DBqSJlzlrugxvewkCJQK++UznjyvTKsdfHVs/6Kmi05I+CneCDuDZgWbgbOIuXCQ45WgDjKg0fTl2UAQjzLeaytrSIsQud4uvgQFDtk= x-ms-office365-filtering-correlation-id: f8a763bb-e539-485a-0658-08d63f24d672 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(5600074)(711020)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7153060)(7193020); SRVR:VI1PR08MB3534; x-ms-traffictypediagnostic: VI1PR08MB3534: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(10201501046)(3002001)(93006095)(93001095)(3231382)(944501410)(4982022)(52105095)(148016)(149066)(150057)(6041310)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123562045)(201708071742011)(7699051)(76991095); SRVR:VI1PR08MB3534; BCL:0; PCL:0; RULEID:; SRVR:VI1PR08MB3534; x-forefront-prvs: 084285FC5C x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(346002)(376002)(366004)(396003)(39850400004)(136003)(199004)(189003)(966005)(3846002)(99286004)(386003)(52116002)(107886003)(6436002)(6486002)(14444005)(4326008)(186003)(26005)(6506007)(54906003)(2906002)(102836004)(2501003)(5250100002)(14454004)(478600001)(316002)(1076002)(256004)(6116002)(7416002)(97736004)(86362001)(25786009)(5660300001)(2616005)(36756003)(66066001)(8936002)(81166006)(476003)(44832011)(8676002)(68736007)(6916009)(6306002)(5640700003)(7736002)(6512007)(305945005)(71200400001)(71190400001)(2900100001)(2351001)(53936002)(105586002)(106356001)(81156014)(486006); DIR:OUT; SFP:1102; SCL:1; SRVR:VI1PR08MB3534; H:VI1PR08MB3358.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: virtuozzo.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: 5bUkSu8fCl55poHTfBQVi6jd4/kWraML1343NBnbE0YLswEn8+GPBldlU3aqXvjFLZJq5sCXj06H9GEOtN5HgdSeu+nZvEMREfo2RD/N+kq6IpQPyXcWRUmSgCS9Rcsl93TZ/ic4DYfdKoPHglq86BH+dEgOcglWhNKAULCyAzxyVZLVb7rkPMYIsaQzfTBac8AskMsNAS4Fq1pE6mLRhR1dFPLdr8WRoryXj/xDB2HVurYC7KzjAjC0uqko8bU2jrQtf7cJ+ipk68QotYda7KTpzIiXAdtnmN0Dg3k44E7BFfFQLdDsmQesZ2qTSteThxhxdL80mA9W85vi8k5mstJUz3YYqbCSThOVGiBRkKs= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: f8a763bb-e539-485a-0658-08d63f24d672 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Oct 2018 11:34:37.6178 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3534 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 104.47.0.118 Subject: [Qemu-devel] [PATCH v5 0/9] discard blockstats X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "kwolf@redhat.com" , Vladimir Sementsov-Ogievskiy , "famz@redhat.com" , Denis Lunev , "qemu-block@nongnu.org" , "armbru@redhat.com" , "mreitz@redhat.com" , "berto@igalia.com" , Anton Nefedov , "pbonzini@redhat.com" , "jsnow@redhat.com" Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP new in v5: - new patch 3. Protects from double-accounting IDE trim requests in ide_handle_rw_error(). - patch 4: move block_acct_invalid() for unmap to ide_issue_trim_cb() This way it doesn't affect read and write requests. - patch 7: added missing hunks as suggested by Kevin v4: http://lists.nongnu.org/archive/html/qemu-devel/2018-08/msg04308.html v3: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg03688.html ---- qmp query-blockstats provides stats info for write/read/flush ops. Patches 1-7 implement the similar for discard (unmap) command for scsi and ide disks. Discard stat "unmap_ops / unmap_bytes" is supposed to account the ops that have completed without an error. However, discard operation is advisory. Specifically, - common block layer ignores ENOTSUP error code. That might be returned if the block driver does not support discard, or discard has been configured to be ignored. - format drivers such as qcow2 may ignore discard if they were configured to ignore that, or if the corresponding area is already marked unused (unallocated / zero clusters). And what is actually useful is the number of bytes actually discarded down on the host filesystem. To achieve that, driver-specific statistics has been added to blockstats (patch 9). With patch 8, file-posix driver accounts discard operations on its level too. query-blockstat result: (note the difference between blockdevice unmap and file discard stats. qcow2 sends fewer ops down to the file as the clusters are actually unallocated on qcow2 level) { "device": "drive-scsi0-0-0-0", "node-name": "#block159", "stats": { > "invalid_unmap_operations": 0, > "failed_unmap_operations": 0, "wr_highest_offset": 13411688448, "rd_total_time_ns": 2859566315, "rd_bytes": 103182336, "rd_merged": 0, "flush_operations": 19, "invalid_wr_operations": 0, "flush_total_time_ns": 23111608, "failed_rd_operations": 0, "failed_flush_operations": 0, "invalid_flush_operations": 0, "timed_stats": [ ], "wr_merged": 0, "wr_bytes": 1702912, > "unmap_bytes": 11954954240, > "unmap_operations": 865, "idle_time_ns": 2669508623, "account_invalid": true, > "unmap_total_time_ns": 19698002, "wr_operations": 143, "failed_wr_operations": 0, "rd_operations": 4816, "account_failed": true, > "unmap_merged": 0, "wr_total_time_ns": 1262686124, "invalid_rd_operations": 0 }, "parent": { > "driver-specific": { > "discard-nb-failed": 0, > "discard-bytes-ok": 720896, > "driver": "file", > "discard-nb-ok": 8 > }, "node-name": "#block009", "stats": { [..] } } }, { "device": "floppy0", Anton Nefedov (9): qapi: group BlockDeviceStats fields qapi: add unmap to BlockDeviceStats block: add empty account cookie type ide: account UNMAP (TRIM) operations scsi: store unmap offset and nb_sectors in request struct scsi: move unmap error checking to the complete callback scsi: account unmap operations file-posix: account discard operations qapi: query-blockstat: add driver specific file-posix stats qapi/block-core.json | 82 +++++++++++++++++++++++++++++++++----- include/block/accounting.h | 2 + include/block/block.h | 1 + include/block/block_int.h | 1 + block.c | 9 +++++ block/accounting.c | 6 +++ block/file-posix.c | 45 +++++++++++++++++++-- block/qapi.c | 11 +++++ hw/ide/core.c | 12 ++++++ hw/scsi/scsi-disk.c | 32 ++++++++++----- tests/qemu-iotests/227.out | 18 +++++++++ 11 files changed, 194 insertions(+), 25 deletions(-)