From patchwork Thu Nov 15 08:52:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 10683701 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D2B9B14D6 for ; Thu, 15 Nov 2018 08:53:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0DFA82818A for ; Thu, 15 Nov 2018 08:53:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 011162856A; Thu, 15 Nov 2018 08:53:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 336D92818A for ; Thu, 15 Nov 2018 08:53:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 387AC6B026B; Thu, 15 Nov 2018 03:53:37 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3377A6B026C; Thu, 15 Nov 2018 03:53:37 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24DED6B026D; Thu, 15 Nov 2018 03:53:37 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by kanga.kvack.org (Postfix) with ESMTP id EDB786B026B for ; Thu, 15 Nov 2018 03:53:36 -0500 (EST) Received: by mail-qk1-f197.google.com with SMTP id s19so43885211qke.20 for ; Thu, 15 Nov 2018 00:53:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id; bh=5n8Uo/vC9cgKokQj2zmOIdylEAmjpmVYiaJR/+YEe8A=; b=Ad+zWWFxwJn2bth7Av7jFTRiqrBGNCc+GOsMkXTPqnAQ/SuSF4X29/jPq67S1NsLFL cWwpE3MulOwEtqPE0t/7pud7BqdHoafEPq3WFIqS5wykMvdYJZhZHUEBDGpJJISx+X0g bV1JOVSOF6SIK4wNyNte1UWE0ZJhVpBQL44+VPo//Yehc5TbnKH6+C81c9N2CgUfcktY ASxhf20Tmi2oJeC+2KJNhj80qHnc0PwohekFnCGVR8KIhM+f5MVZgU+ijRyWOqxrF35T 5H2EK0dCPwd442rEAA5LoF+ChJEkan3e1DsVtj5UNp5sasbma1WRIicODZu+SlFk8FoN sl0A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AGRZ1gLlb9in7BFxupzRTlCdJtW/tDGjqJ6pUjia+qPNf6Dgv7xUB+p4 hmgGRcsLYxXn2ymqleOAAb5WfTNIhJAMIcrw4jk7VNYtQLku4OUTp13KJV1Stsy9w5x1SRmNofp 1IWo8Ngk0gmuVaNEm+qEerc6kMkf5C2azoMGsahgK+nr44dZANGRpe0Fk2saYb3Hamg== X-Received: by 2002:a37:72c3:: with SMTP id n186mr5022584qkc.340.1542272016689; Thu, 15 Nov 2018 00:53:36 -0800 (PST) X-Google-Smtp-Source: AJdET5dUg98hVUvX5lTGJFJF5Owzs72F2BNIb5dX2bsM1mgkuWOFoZPqe+PlD0BliTm37SziEb5Z X-Received: by 2002:a37:72c3:: with SMTP id n186mr5022547qkc.340.1542272015541; Thu, 15 Nov 2018 00:53:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542272015; cv=none; d=google.com; s=arc-20160816; b=F1cs93ed0YGrv7bwmJze4s8pIkvWc4SWd2FSyZzUszzaRKd9AYDlpgrlblBARzhD// mfMJVSPT9bAfwVBobjzmp9zjxEwf6/uoJu+Tpbtgw0jGrQi8bhMfKVd0Oaz+aDB1zpAs LuIfktGOw4mfIxpwG3s/yBwCehSq8KiNEgArZ1j5PBU4mD9uJIGBqx8UilllL6qAX6o0 EgfDloa+POAgGwThFf/Gk15kV1sqTBPE9HJ66RlFE5hfItCoqVT5Xk/v+0WGpsC+i21V UviIgV1PntCKjipajLBtJZp+t+xSxn8IyQtkrF/Bc+so7DLJKx2Ka5a/o2lHxyMxSOCt M1dw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from; bh=5n8Uo/vC9cgKokQj2zmOIdylEAmjpmVYiaJR/+YEe8A=; b=uahlFSQn0bDFwAQ120weftTXCfe5md9RKijSxpfu0OEGGItyLH2eLEDpmHu/+l7gup uVeQROtclrz6kJn88yIWpUCixB1aJpz54fL88rOD90UYEFUHiokqPHjOI8+sfSpZZAue 70IcwGLFIHyVAJEmyd0X6xOjngUr5vqZpX23salBfMJZAOJD9qdpSKGF9mBYaVjKj7nF 50/+jjhajz5BRbFWnx9S179ouOVcBj5eiVN6OC6MwVRPqI1f0QzyTgs/GTJ/l1WEzxkZ 8yacCv8xWXT3WEOMXLsel9hEC2c6rg8XBOptlK2AaEscu7JXvqBEbLFaZrJ51tHN8wVl lorA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id 51si971565qvt.60.2018.11.15.00.53.35 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Nov 2018 00:53:35 -0800 (PST) Received-SPF: pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 54E2C300157B; Thu, 15 Nov 2018 08:53:34 +0000 (UTC) Received: from localhost (ovpn-8-23.pek2.redhat.com [10.72.8.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9E184608FA; Thu, 15 Nov 2018 08:53:20 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Ming Lei , Dave Chinner , Kent Overstreet , Mike Snitzer , dm-devel@redhat.com, Alexander Viro , linux-fsdevel@vger.kernel.org, Shaohua Li , linux-raid@vger.kernel.org, linux-erofs@lists.ozlabs.org, David Sterba , linux-btrfs@vger.kernel.org, "Darrick J . Wong" , linux-xfs@vger.kernel.org, Gao Xiang , Christoph Hellwig , Theodore Ts'o , linux-ext4@vger.kernel.org, Coly Li , linux-bcache@vger.kernel.org, Boaz Harrosh , Bob Peterson , cluster-devel@redhat.com Subject: [PATCH V10 00/19] block: support multi-page bvec Date: Thu, 15 Nov 2018 16:52:47 +0800 Message-Id: <20181115085306.9910-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Thu, 15 Nov 2018 08:53:34 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Hi, This patchset brings multi-page bvec into block layer: 1) what is multi-page bvec? Multipage bvecs means that one 'struct bio_bvec' can hold multiple pages which are physically contiguous instead of one single page used in linux kernel for long time. 2) why is multi-page bvec introduced? Kent proposed the idea[1] first. As system's RAM becomes much bigger than before, and huge page, transparent huge page and memory compaction are widely used, it is a bit easy now to see physically contiguous pages from fs in I/O. On the other hand, from block layer's view, it isn't necessary to store intermediate pages into bvec, and it is enough to just store the physicallly contiguous 'segment' in each io vector. Also huge pages are being brought to filesystem and swap [2][6], we can do IO on a hugepage each time[3], which requires that one bio can transfer at least one huge page one time. Turns out it isn't flexiable to change BIO_MAX_PAGES simply[3][5]. Multipage bvec can fit in this case very well. As we saw, if CONFIG_THP_SWAP is enabled, BIO_MAX_PAGES can be configured as much bigger, such as 512, which requires at least two 4K pages for holding the bvec table. With multi-page bvec: - Inside block layer, both bio splitting and sg map can become more efficient than before by just traversing the physically contiguous 'segment' instead of each page. - segment handling in block layer can be improved much in future since it should be quite easy to convert multipage bvec into segment easily. For example, we might just store segment in each bvec directly in future. - bio size can be increased and it should improve some high-bandwidth IO case in theory[4]. - there is opportunity in future to improve memory footprint of bvecs. 3) how is multi-page bvec implemented in this patchset? The patches of 1 ~ 14 implement multipage bvec in block layer: - put all tricks into bvec/bio/rq iterators, and as far as drivers and fs use these standard iterators, they are happy with multipage bvec - introduce bio_for_each_bvec() to iterate over multipage bvec for splitting bio and mapping sg - keep current bio_for_each_segment*() to itereate over singlepage bvec and make sure current users won't be broken; especailly, convert to this new helper prototype in single patch 21 given it is bascially a mechanism conversion - deal with iomap & xfs's sub-pagesize io vec in patch 13 - enalbe multipage bvec in patch 14 Patch 15 redefines BIO_MAX_PAGES as 256. Patch 16 documents usages of bio iterator helpers. Patch 17~19 kills NO_SG_MERGE. These patches can be found in the following git tree: git: https://github.com/ming1/linux.git for-4.21-block-mp-bvec-V10 Lots of test(blktest, xfstests, ltp io, ...) have been run with this patchset, and not see regression. Thanks Christoph for reviewing the early version and providing very good suggestions, such as: introduce bio_init_with_vec_table(), remove another unnecessary helpers for cleanup and so on. Any comments are welcome! V10: - no any code change, just add more guys and list into patch's CC list, as suggested by Christoph and Dave Chinner V9: - fix regression on iomap's sub-pagesize io vec, covered by patch 13 V8: - remove prepare patches which all are merged to linus tree - rebase on for-4.21/block - address comments on V7 - add patches of killing NO_SG_MERGE V7: - include Christoph and Mike's bio_clone_bioset() patches, which is actually prepare patches for multipage bvec - address Christoph's comments V6: - avoid to introduce lots of renaming, follow Jen's suggestion of using the name of chunk for multipage io vector - include Christoph's three prepare patches - decrease stack usage for using bio_for_each_chunk_segment_all() - address Kent's comment V5: - remove some of prepare patches, which have been merged already - add bio_clone_seg_bioset() to fix DM's bio clone, which is introduced by 18a25da84354c6b (dm: ensure bio submission follows a depth-first tree walk) - rebase on the latest block for-v4.18 V4: - rename bio_for_each_segment*() as bio_for_each_page*(), rename bio_segments() as bio_pages(), rename rq_for_each_segment() as rq_for_each_pages(), because these helpers never return real segment, and they always return single page bvec - introducing segment_for_each_page_all() - introduce new bio_for_each_segment*()/rq_for_each_segment()/bio_segments() for returning real multipage segment - rewrite segment_last_page() - rename bvec iterator helper as suggested by Christoph - replace comment with applying bio helpers as suggested by Christoph - document usage of bio iterator helpers - redefine BIO_MAX_PAGES as 256 to make the biggest bvec table accommodated in 4K page - move bio_alloc_pages() into bcache as suggested by Christoph V3: - rebase on v4.13-rc3 with for-next of block tree - run more xfstests: xfs/ext4 over NVMe, Sata, DM(linear), MD(raid1), and not see regressions triggered - add Reviewed-by on some btrfs patches - remove two MD patches because both are merged to linus tree already V2: - bvec table direct access in raid has been cleaned, so NO_MP flag is dropped - rebase on recent Neil Brown's change on bio and bounce code - reorganize the patchset V1: - against v4.10-rc1 and some cleanup in V0 are in -linus already - handle queue_virt_boundary() in mp bvec change and make NVMe happy - further BTRFS cleanup - remove QUEUE_FLAG_SPLIT_MP - rename for two new helpers of bio_for_each_segment_all() - fix bounce convertion - address comments in V0 [1], http://marc.info/?l=linux-kernel&m=141680246629547&w=2 [2], https://patchwork.kernel.org/patch/9451523/ [3], http://marc.info/?t=147735447100001&r=1&w=2 [4], http://marc.info/?l=linux-mm&m=147745525801433&w=2 [5], http://marc.info/?t=149569484500007&r=1&w=2 [6], http://marc.info/?t=149820215300004&r=1&w=2 Ming Lei (19): block: introduce multi-page page bvec helpers block: introduce bio_for_each_bvec() block: use bio_for_each_bvec() to compute multi-page bvec count block: use bio_for_each_bvec() to map sg block: introduce bvec_last_segment() fs/buffer.c: use bvec iterator to truncate the bio btrfs: use bvec_last_segment to get bio's last page btrfs: move bio_pages_all() to btrfs block: introduce bio_bvecs() block: loop: pass multi-page bvec to iov_iter bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages() block: allow bio_for_each_segment_all() to iterate over multi-page bvec iomap & xfs: only account for new added page block: enable multipage bvecs block: always define BIO_MAX_PAGES as 256 block: document usage of bio iterator helpers block: don't use bio->bi_vcnt to figure out segment number block: kill QUEUE_FLAG_NO_SG_MERGE block: kill BLK_MQ_F_SG_MERGE Documentation/block/biovecs.txt | 26 +++++ block/bio.c | 51 +++++++--- block/blk-merge.c | 199 +++++++++++++++++++++++++------------- block/blk-mq-debugfs.c | 2 - block/blk-mq.c | 3 - block/blk-zoned.c | 1 + block/bounce.c | 6 +- drivers/block/loop.c | 25 ++--- drivers/block/nbd.c | 2 +- drivers/block/rbd.c | 2 +- drivers/block/skd_main.c | 1 - drivers/block/xen-blkfront.c | 2 +- drivers/md/bcache/btree.c | 3 +- drivers/md/bcache/util.c | 2 +- drivers/md/dm-crypt.c | 3 +- drivers/md/dm-rq.c | 2 +- drivers/md/dm-table.c | 13 --- drivers/md/raid1.c | 3 +- drivers/mmc/core/queue.c | 3 +- drivers/scsi/scsi_lib.c | 2 +- drivers/staging/erofs/data.c | 3 +- drivers/staging/erofs/unzip_vle.c | 3 +- fs/block_dev.c | 6 +- fs/btrfs/compression.c | 8 +- fs/btrfs/disk-io.c | 3 +- fs/btrfs/extent_io.c | 29 ++++-- fs/btrfs/inode.c | 6 +- fs/btrfs/raid56.c | 3 +- fs/buffer.c | 5 +- fs/crypto/bio.c | 3 +- fs/direct-io.c | 4 +- fs/exofs/ore.c | 3 +- fs/exofs/ore_raid.c | 3 +- fs/ext4/page-io.c | 3 +- fs/ext4/readpage.c | 3 +- fs/f2fs/data.c | 9 +- fs/gfs2/lops.c | 6 +- fs/gfs2/meta_io.c | 3 +- fs/iomap.c | 28 ++++-- fs/mpage.c | 3 +- fs/xfs/xfs_aops.c | 15 ++- include/linux/bio.h | 94 ++++++++++++++---- include/linux/blk-mq.h | 1 - include/linux/blkdev.h | 1 - include/linux/bvec.h | 155 +++++++++++++++++++++++++++-- 45 files changed, 556 insertions(+), 195 deletions(-) Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-devel@redhat.com Cc: Alexander Viro Cc: linux-fsdevel@vger.kernel.org Cc: Shaohua Li Cc: linux-raid@vger.kernel.org Cc: linux-erofs@lists.ozlabs.org Cc: David Sterba Cc: linux-btrfs@vger.kernel.org Cc: Darrick J. Wong Cc: linux-xfs@vger.kernel.org Cc: Gao Xiang Cc: Christoph Hellwig Cc: Theodore Ts'o Cc: linux-ext4@vger.kernel.org Cc: Coly Li Cc: linux-bcache@vger.kernel.org Cc: Boaz Harrosh Cc: Bob Peterson Cc: cluster-devel@redhat.com