From patchwork Thu Sep 15 16:48:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sarthak Kukreti X-Patchwork-Id: 12978259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A5709C54EE9 for ; Fri, 16 Sep 2022 08:45:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663317903; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=cUqs1mz25c8LkdAKXZAQTmoI4k2JgXPQ/+stMIcWx7U=; b=a9wxwILKluM9Qxz2QNSCq5KL1vrwPhHKvbDypZe4HrC1dYjtFNFCJOU7FnK/m8Fz1jMCLI +bzzpKdpeslD6H+8j43eNVSc1n9nmLP7xYE6z3HmyNY5fZALhaEEppd5Dq4ryE0ApgGEND DYxQOzZFg/J5Pa50uLKjs0bHe0aSolY= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-458-ycVt505mNHKLv3_FU0GUlg-1; Fri, 16 Sep 2022 04:45:00 -0400 X-MC-Unique: ycVt505mNHKLv3_FU0GUlg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A1C9C382F1AE; Fri, 16 Sep 2022 08:44:58 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 37D35140EBF5; Fri, 16 Sep 2022 08:44:57 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id EAAA61946587; Fri, 16 Sep 2022 08:44:56 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id AE77E1946587 for ; Thu, 15 Sep 2022 16:48:42 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 5D9D12166B29; Thu, 15 Sep 2022 16:48:42 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast09.extmail.prod.ext.rdu2.redhat.com [10.11.55.25]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 59F282166B26 for ; Thu, 15 Sep 2022 16:48:42 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3F577294EDFB for ; Thu, 15 Sep 2022 16:48:42 +0000 (UTC) Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-642-AzdVgFuLPimpViSasUF_qQ-2; Thu, 15 Sep 2022 12:48:39 -0400 X-MC-Unique: AzdVgFuLPimpViSasUF_qQ-2 Received: by mail-pf1-f176.google.com with SMTP id l65so18621624pfl.8 for ; Thu, 15 Sep 2022 09:48:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date; bh=zlA6ptTTn1LB8f3ds6R6bK3ndLgpFyQYa5/cgjmBz0k=; b=GsEXR3eKKPwWDMckOnDHIhKC8Z6U306Rb2qCk5aUccUnqI2sQzUa2dZY8/6MJHfjDV 1S+mYEOpotafwSlB3wnzeC294cBK05nRrW7pUd+F7dnMWCBOdro8PwxvaaiKrT/gLX3e 6QVBVrjZN0BAdLCyg4aw59HIcWObbYqolDwbX2Ey1k+5578+54o019wLxlte8iQdMlPO +CVB+0Of+NOIl/SqkDdfELjbfCASFMfc9Q3GACGFQZIIprNpjGaSfpP1UQ69cNZMDPXs XwugLKD31z3JQA0rWh+RicQlzT4Y/yUdIabNKo5p2MhXDf3WpBXNufZ9AHrxGyUsxj4S +csw== X-Gm-Message-State: ACrzQf29vJFI/F9mgY7c7z9CRqs/IfyZKhbsPPBakMteGUG0aEE2h8tf BaN1LA/QSXzScPBrK6HIZyGAL/egZdq4unjR X-Google-Smtp-Source: AMsMyM4FdBRmEUaoFqZ1ZX6Vh6XtevMCEGOiKjv87UB8WmxxEo9+INCb9SpLcAAa5zOhruJLuTG+QQ== X-Received: by 2002:aa7:8892:0:b0:540:edcb:4773 with SMTP id z18-20020aa78892000000b00540edcb4773mr674454pfe.19.1663260517947; Thu, 15 Sep 2022 09:48:37 -0700 (PDT) Received: from sarthakkukreti-glaptop.hsd1.ca.comcast.net ([2601:647:4200:b5b0:3af2:34b2:a98a:a652]) by smtp.gmail.com with ESMTPSA id o4-20020a170902bcc400b00177ee563b6dsm13174970pls.33.2022.09.15.09.48.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Sep 2022 09:48:37 -0700 (PDT) From: Sarthak Kukreti X-Google-Original-From: Sarthak Kukreti To: dm-devel@redhat.com, linux-block@vger.kernel.org, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org Date: Thu, 15 Sep 2022 09:48:18 -0700 Message-Id: <20220915164826.1396245-1-sarthakkukreti@google.com> MIME-Version: 1.0 X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mailman-Approved-At: Fri, 16 Sep 2022 08:44:55 +0000 Subject: [dm-devel] [PATCH RFC 0/8] Introduce provisioning primitives for thinly provisioned storage X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jens Axboe , Gwendal Grignou , Theodore Ts'o , Sarthak Kukreti , "Michael S . Tsirkin" , Jason Wang , Bart Van Assche , Mike Snitzer , Evan Green , Andreas Dilger , Daniil Lunev , Stefan Hajnoczi , Paolo Bonzini , Alasdair Kergon Errors-To: dm-devel-bounces@redhat.com Sender: "dm-devel" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com From: Sarthak Kukreti Hi, This patch series is an RFC of a mechanism to pass through provision requests on stacked thinly provisioned storage devices/filesystems. The linux kernel provides several mechanisms to set up thinly provisioned block storage abstractions (eg. dm-thin, loop devices over sparse files), either directly as block devices or backing storage for filesystems. Currently, short of writing data to either the device or filesystem, there is no way for users to pre-allocate space for use in such storage setups. Consider the following use-cases: 1) Suspend-to-disk and resume from a dm-thin device: In order to ensure that the underlying thinpool metadata is not modified during the suspend mechanism, the dm-thin device needs to be fully provisioned. 2) If a filesystem uses a loop device over a sparse file, fallocate() on the filesystem will allocate blocks for files but the underlying sparse file will remain intact. 3) Another example is virtual machine using a sparse file/dm-thin as a storage device; by default, allocations within the VM boundaries will not affect the host. 4) Several storage standards support mechanisms for thin provisioning on real hardware devices. For example: a. The NVMe spec 1.0b section 2.1.1 loosely talks about thin provisioning: "When the THINP bit in the NSFEAT field of the Identify Namespace data structure is set to ‘1’, the controller ... shall track the number of allocated blocks in the Namespace Utilization field" b. The SCSi Block Commands reference - 4 section references "Thin provisioned logical units", c. UFS 3.0 spec section 13.3.3 references "Thin provisioning". In all of the above situations, currently the only way for pre-allocating space is to issue writes (or use WRITE_ZEROES/WRITE_SAME). However, that does not scale well with larger pre-allocation sizes. This patchset introduces primitives to support block-level provisioning (note: the term 'provisioning' is used to prevent overloading the term 'allocations/pre-allocations') requests across filesystems and block devices. This allows fallocate() and file creation requests to reserve space across stacked layers of block devices and filesystems. Currently, the patchset covers a prototype on the device-mapper targets, loop device and ext4, but the same mechanism can be extended to other filesystems/block devices as well as extended for use with devices in 4 a-c. Patch 1 introduces REQ_OP_PROVISION as a new request type. The provision request acts like the inverse of a discard request; instead of notifying lower layers that the block range will no longer be used, provision acts as a request to lower layers to provision disk space for the given block range. Real hardware storage devices will currently disable the provisioing capability but for the standards listed in 4a.-c., REQ_OP_PROVISION can be overloaded for use as the provisioing primitive for future devices. Patch 2 implements REQ_OP_PROVISION handling for some of the device-mapper targets. This additionally adds support for pre-allocating space for thinly provisioned logical volumes via fallocate() Patch 3 implements the handling for virtio-blk. Patch 4 introduces an fallocate() mode (FALLOC_FL_PROVISION) that sends a provision request to the underlying block device (and beyond). This acts as the primary mechanism for file-level provisioing. Patch 5 wires up the loop device handling of REQ_OP_PROVISION. Patches 6-8 cover a prototype implementation for ext4, which includes wiring up the fallocate() implementation, introducing a filesystem level option (called 'provision') to control the default allocation behaviour and finally a file level override to retain current handling, even on filesystems mounted with 'provision' Testing: -------- - A backport of this patch series was tested on ChromiumOS using a 5.10 kernel. - File on ext4 on a thin logical volume: fallocate(FALLOC_FL_PROVISION) : 4.6s, dd if=/dev/zero of=...: 6 mins. TODOs: ------ 1) The stacked block devices (dm-*, loop etc.) currently unconditionally pass through provision requests. Add support for provision, similar to how discard handling is set up (with options to disable, passdown or passthrough requests). 2) Blktests and Xfstests for validating provisioning. --- dm-devel mailing list dm-devel@redhat.com https://listman.redhat.com/mailman/listinfo/dm-devel