From patchwork Thu Mar 22 06:55:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Vagin X-Patchwork-Id: 10300903 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9D8F3600F6 for ; Thu, 22 Mar 2018 06:55:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9168629A4E for ; Thu, 22 Mar 2018 06:55:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 85C8B29A50; Thu, 22 Mar 2018 06:55:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C5F4729A4F for ; Thu, 22 Mar 2018 06:55:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752464AbeCVGzU (ORCPT ); Thu, 22 Mar 2018 02:55:20 -0400 Received: from mail-pg0-f65.google.com ([74.125.83.65]:39576 "EHLO mail-pg0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752455AbeCVGzR (ORCPT ); Thu, 22 Mar 2018 02:55:17 -0400 Received: by mail-pg0-f65.google.com with SMTP id a19so2918521pgw.6; Wed, 21 Mar 2018 23:55:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id; bh=xWPX2wQ0MzzLi7wDt6IpfQNww01U8cEZPpLIXESEKLo=; b=CSyR8GwPEEop1L95Q7temXO8eRCVvR6m49oIKLZZgIbygUz5eWjFk3DsTUu/KynXAO zRjMN6DS8YmmukHk4drI1lWUuyUEWT4X2KUIzVmF+qq/gZbARx9ynSuCr8wvN+kV38pI neLzIG9o4l0NL2yb0SNG2DMcD7KZQXJGgax8sToB6meaByEol2OFaUhjrh+jKlHslxyJ GY/yaLibgHCqWrL9tqqyjRYt39zlm7hK3t4Onb1LSOurksDhwrv3y+oodXst814m8v2X neh8wky9BgKfCbq27zYJYGyfCr4Fc8pRU/Iihm0NIPEAiMvvCPBki6bw77IFR5hbWj0N XqQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id; bh=xWPX2wQ0MzzLi7wDt6IpfQNww01U8cEZPpLIXESEKLo=; b=OM+bOytb4/im/UFwrKFmFB7e4jUS8sKh/maV+zDJeZE43NMtOv122aoIkYNVfV/kIe FwM9Qg/R0bRR7D+gSkvxSQW6PPTQWVe/usGcCQ4cmWPt09JlygbdEP6T/hDlQJZ0lm1k Pz/RSPznkD53n4wWc94+Ncs6bt9Pexniv9M8/rRYKA3wDti7FO9AhKIH1d6DA8n+YSRK tI0AVMMHAKh+Q+gAXE6/Ug05Q/uUiKqkwyRLt9MGBqs/TKkukqyPbZBE7PxS0AdxL1OY +7F2hKxRb+lhq2utM20hYheQqNFiL6oJACTVXDwPNkHmaFKQt66a0xfoL3fYYPl0hDAn ZUjA== X-Gm-Message-State: AElRT7HrYm7L3+nLptqqim07uGtEfgeYBNNQmGszS77FV6eWQ4r7YEbG D4X4qq6/01TZ6jA2sU9ZuTPhkfRF X-Google-Smtp-Source: AG47ELvPpysTlM8Nx3zrmhbnYGYNZO8RGBd+tWUjckZrsXd3RADOD8iAA7RjJOUNpCj4Pz7FziwbvA== X-Received: by 10.98.48.195 with SMTP id w186mr19537158pfw.174.1521701716893; Wed, 21 Mar 2018 23:55:16 -0700 (PDT) Received: from localhost.localdomain (c-73-140-212-29.hsd1.wa.comcast.net. [73.140.212.29]) by smtp.gmail.com with ESMTPSA id 75sm12449689pfl.169.2018.03.21.23.55.15 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 21 Mar 2018 23:55:16 -0700 (PDT) From: Andrei Vagin To: "Nicholas A. Bellinger" Cc: linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, Andrei Vagin , Christoph Hellwig , "Bryant G . Ly" Subject: [PATCH v2] target/file: add support of direct and async I/O Date: Wed, 21 Mar 2018 23:55:02 -0700 Message-Id: <20180322065502.25569-1-avagin@openvz.org> X-Mailer: git-send-email 2.13.6 Sender: target-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP There are two advantages: * Direct I/O allows to avoid the write-back cache, so it reduces affects to other processes in the system. * Async I/O allows to handle a few commands concurrently. DIO + AIO shows a better perfomance for random write operations: Mode: O_DSYNC Async: 1 $ ./fio --bs=4K --direct=1 --rw=randwrite --ioengine=libaio --iodepth=64 --name=/dev/sda --runtime=20 --numjobs=2 WRITE: bw=45.9MiB/s (48.1MB/s), 21.9MiB/s-23.0MiB/s (22.0MB/s-25.2MB/s), io=919MiB (963MB), run=20002-20020msec Mode: O_DSYNC Async: 0 $ ./fio --bs=4K --direct=1 --rw=randwrite --ioengine=libaio --iodepth=64 --name=/dev/sdb --runtime=20 --numjobs=2 WRITE: bw=1607KiB/s (1645kB/s), 802KiB/s-805KiB/s (821kB/s-824kB/s), io=31.8MiB (33.4MB), run=20280-20295msec Known issue: DIF (PI) emulation doesn't work when a target uses async I/O, because DIF metadata is saved in a separate file, and it is another non-trivial task how to synchronize writing in two files, so that a following read operation always returns a consisten metadata for a specified block. v2: fix comments from Christoph Hellwig Cc: "Nicholas A. Bellinger" Cc: Christoph Hellwig Cc: Bryant G. Ly Tested-by: Bryant G. Ly Signed-off-by: Andrei Vagin Signed-off-by: Christoph Hellwig Reviewed-by: Bryant G. Ly Reviewed-by: Mike Christie --- drivers/target/target_core_file.c | 137 ++++++++++++++++++++++++++++++++++---- drivers/target/target_core_file.h | 1 + 2 files changed, 124 insertions(+), 14 deletions(-) diff --git a/drivers/target/target_core_file.c b/drivers/target/target_core_file.c index 9b2c0c773022..16751ae55d7b 100644 --- a/drivers/target/target_core_file.c +++ b/drivers/target/target_core_file.c @@ -250,6 +250,84 @@ static void fd_destroy_device(struct se_device *dev) } } +struct target_core_file_cmd { + unsigned long len; + struct se_cmd *cmd; + struct kiocb iocb; +}; + +static void cmd_rw_aio_complete(struct kiocb *iocb, long ret, long ret2) +{ + struct target_core_file_cmd *cmd; + + cmd = container_of(iocb, struct target_core_file_cmd, iocb); + + if (ret != cmd->len) + target_complete_cmd(cmd->cmd, SAM_STAT_CHECK_CONDITION); + else + target_complete_cmd(cmd->cmd, SAM_STAT_GOOD); + + kfree(cmd); +} + +static sense_reason_t +fd_execute_rw_aio(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents, + enum dma_data_direction data_direction) +{ + int is_write = !(data_direction == DMA_FROM_DEVICE); + struct se_device *dev = cmd->se_dev; + struct fd_dev *fd_dev = FD_DEV(dev); + struct file *file = fd_dev->fd_file; + struct target_core_file_cmd *aio_cmd; + struct iov_iter iter = {}; + struct scatterlist *sg; + struct bio_vec *bvec; + ssize_t len = 0; + int ret = 0, i; + + aio_cmd = kmalloc(sizeof(struct target_core_file_cmd), GFP_KERNEL); + if (!aio_cmd) + return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE; + + bvec = kcalloc(sgl_nents, sizeof(struct bio_vec), GFP_KERNEL); + if (!bvec) { + kfree(aio_cmd); + return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE; + } + + for_each_sg(sgl, sg, sgl_nents, i) { + bvec[i].bv_page = sg_page(sg); + bvec[i].bv_len = sg->length; + bvec[i].bv_offset = sg->offset; + + len += sg->length; + } + + iov_iter_bvec(&iter, ITER_BVEC | is_write, bvec, sgl_nents, len); + + aio_cmd->cmd = cmd; + aio_cmd->len = len; + aio_cmd->iocb.ki_pos = cmd->t_task_lba * dev->dev_attrib.block_size; + aio_cmd->iocb.ki_filp = file; + aio_cmd->iocb.ki_complete = cmd_rw_aio_complete; + aio_cmd->iocb.ki_flags = IOCB_DIRECT; + + if (is_write && (cmd->se_cmd_flags & SCF_FUA)) + aio_cmd->iocb.ki_flags |= IOCB_DSYNC; + + if (is_write) + ret = call_write_iter(file, &aio_cmd->iocb, &iter); + else + ret = call_read_iter(file, &aio_cmd->iocb, &iter); + + kfree(bvec); + + if (ret != -EIOCBQUEUED) + cmd_rw_aio_complete(&aio_cmd->iocb, ret, 0); + + return 0; +} + static int fd_do_rw(struct se_cmd *cmd, struct file *fd, u32 block_size, struct scatterlist *sgl, u32 sgl_nents, u32 data_length, int is_write) @@ -527,7 +605,7 @@ fd_execute_unmap(struct se_cmd *cmd, sector_t lba, sector_t nolb) } static sense_reason_t -fd_execute_rw(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents, +fd_execute_rw_buffered(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents, enum dma_data_direction data_direction) { struct se_device *dev = cmd->se_dev; @@ -537,16 +615,6 @@ fd_execute_rw(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents, sense_reason_t rc; int ret = 0; /* - * We are currently limited by the number of iovecs (2048) per - * single vfs_[writev,readv] call. - */ - if (cmd->data_length > FD_MAX_BYTES) { - pr_err("FILEIO: Not able to process I/O of %u bytes due to" - "FD_MAX_BYTES: %u iovec count limitation\n", - cmd->data_length, FD_MAX_BYTES); - return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE; - } - /* * Call vectorized fileio functions to map struct scatterlist * physical memory addresses to struct iovec virtual memory. */ @@ -620,14 +688,39 @@ fd_execute_rw(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents, return 0; } +static sense_reason_t +fd_execute_rw(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents, + enum dma_data_direction data_direction) +{ + struct se_device *dev = cmd->se_dev; + struct fd_dev *fd_dev = FD_DEV(dev); + + /* + * We are currently limited by the number of iovecs (2048) per + * single vfs_[writev,readv] call. + */ + if (cmd->data_length > FD_MAX_BYTES) { + pr_err("FILEIO: Not able to process I/O of %u bytes due to" + "FD_MAX_BYTES: %u iovec count limitation\n", + cmd->data_length, FD_MAX_BYTES); + return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE; + } + + if (fd_dev->fbd_flags & FDBD_HAS_ASYNC_IO) + return fd_execute_rw_aio(cmd, sgl, sgl_nents, data_direction); + return fd_execute_rw_buffered(cmd, sgl, sgl_nents, data_direction); +} + enum { - Opt_fd_dev_name, Opt_fd_dev_size, Opt_fd_buffered_io, Opt_err + Opt_fd_dev_name, Opt_fd_dev_size, Opt_fd_buffered_io, + Opt_fd_async_io, Opt_err }; static match_table_t tokens = { {Opt_fd_dev_name, "fd_dev_name=%s"}, {Opt_fd_dev_size, "fd_dev_size=%s"}, {Opt_fd_buffered_io, "fd_buffered_io=%d"}, + {Opt_fd_async_io, "fd_async_io=%d"}, {Opt_err, NULL} }; @@ -693,6 +786,21 @@ static ssize_t fd_set_configfs_dev_params(struct se_device *dev, fd_dev->fbd_flags |= FDBD_HAS_BUFFERED_IO_WCE; break; + case Opt_fd_async_io: + ret = match_int(args, &arg); + if (ret) + goto out; + if (arg != 1) { + pr_err("bogus fd_async_io=%d value\n", arg); + ret = -EINVAL; + goto out; + } + + pr_debug("FILEIO: Using async I/O" + " operations for struct fd_dev\n"); + + fd_dev->fbd_flags |= FDBD_HAS_ASYNC_IO; + break; default: break; } @@ -709,10 +817,11 @@ static ssize_t fd_show_configfs_dev_params(struct se_device *dev, char *b) ssize_t bl = 0; bl = sprintf(b + bl, "TCM FILEIO ID: %u", fd_dev->fd_dev_id); - bl += sprintf(b + bl, " File: %s Size: %llu Mode: %s\n", + bl += sprintf(b + bl, " File: %s Size: %llu Mode: %s Async: %d\n", fd_dev->fd_dev_name, fd_dev->fd_dev_size, (fd_dev->fbd_flags & FDBD_HAS_BUFFERED_IO_WCE) ? - "Buffered-WCE" : "O_DSYNC"); + "Buffered-WCE" : "O_DSYNC", + !!(fd_dev->fbd_flags & FDBD_HAS_ASYNC_IO)); return bl; } diff --git a/drivers/target/target_core_file.h b/drivers/target/target_core_file.h index 53be5ffd3261..929b1ecd544e 100644 --- a/drivers/target/target_core_file.h +++ b/drivers/target/target_core_file.h @@ -22,6 +22,7 @@ #define FBDF_HAS_PATH 0x01 #define FBDF_HAS_SIZE 0x02 #define FDBD_HAS_BUFFERED_IO_WCE 0x04 +#define FDBD_HAS_ASYNC_IO 0x08 #define FDBD_FORMAT_UNIT_SIZE 2048 struct fd_dev {