From patchwork Thu Jun 25 09:01:08 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 6672761 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 8D5619F39B for ; Thu, 25 Jun 2015 09:01:27 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id B5D542053B for ; Thu, 25 Jun 2015 09:01:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CF5322053A for ; Thu, 25 Jun 2015 09:01:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751784AbbFYJBX (ORCPT ); Thu, 25 Jun 2015 05:01:23 -0400 Received: from mail-la0-f43.google.com ([209.85.215.43]:34226 "EHLO mail-la0-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750968AbbFYJBT (ORCPT ); Thu, 25 Jun 2015 05:01:19 -0400 Received: by lagx9 with SMTP id x9so40864636lag.1 for ; Thu, 25 Jun 2015 02:01:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id; bh=FdCbdwMo5Bh0RWLIrLxf4z8F9PxWngBob5Kq4BTDyNU=; b=dCvL6mmqg+X9zgNiTMjqFlMbphKsJ7dV2/Nvnq1I+R2L+zxJtlMWLBcnALfr3jTkff ZSH884+GXSwpkZqeca5dHAtXUUNi/OVwgICQ1Ejn7cvYyuCTaWD81f24wXyoYDHOAVdY FyeXqVzjvUs/xeJs2GOud6pd3p2y+lFiw4a62cyHud7uW3ex5xEL7FBmnMGI6EiIjy7r D9sOfZqD66H4+Mu9Cpi7Ox7I0hCuyhYJTdxcgkLLmyVxWmsoHHcqCMv2flSzWMqIvTmE NZ7pk0lx9GlDljhMrUsB+7/ynO81MstDA3GH3A5uO52HPR/P8KQl30ZuzSHGlRvwoAjB HZnw== X-Received: by 10.152.9.137 with SMTP id z9mr44134477laa.68.1435222877662; Thu, 25 Jun 2015 02:01:17 -0700 (PDT) Received: from localhost.localdomain ([109.110.66.238]) by mx.google.com with ESMTPSA id bd13sm3125819lab.5.2015.06.25.02.01.16 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 25 Jun 2015 02:01:17 -0700 (PDT) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Cc: Alex Elder Subject: [PATCH] rbd: bump queue_max_segments Date: Thu, 25 Jun 2015 12:01:08 +0300 Message-Id: <1435222868-34966-1-git-send-email-idryomov@gmail.com> X-Mailer: git-send-email 1.9.3 Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Spam-Status: No, score=-3.5 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_DNSWL_HI, RCVD_IN_SBL_CSS,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The default queue_limits::max_segments value (BLK_MAX_SEGMENTS = 128) unnecessarily limits bio sizes to 512k (assuming 4k pages). rbd, being a virtual block device, doesn't have any restrictions on the number of physical segments, so bump max_segments to max_hw_sectors, in theory allowing a sector per segment (although the only case this matters that I can think of is some readv/writev style thing). In practice this is going to give us 1M bios - the number of segments in a bio is limited in bio_get_nr_vecs() by BIO_MAX_PAGES = 256. Note that this doesn't result in any improvement on a typical direct sequential test. This is because on a box with a not too badly fragmented memory the default BLK_MAX_SEGMENTS is enough to see nice rbd object size sized requests. The only difference is the size of bios being merged - 512k vs 1M for something like $ dd if=/dev/zero of=/dev/rbd0 oflag=direct bs=$RBD_OBJ_SIZE $ dd if=/dev/rbd0 iflag=direct of=/dev/null bs=$RBD_OBJ_SIZE Signed-off-by: Ilya Dryomov Reviewed-by: Alex Elder --- drivers/block/rbd.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 89fe8a4bc02e..bc88fbcb9715 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -3791,6 +3791,7 @@ static int rbd_init_disk(struct rbd_device *rbd_dev) /* set io sizes to object size */ segment_size = rbd_obj_bytes(&rbd_dev->header); blk_queue_max_hw_sectors(q, segment_size / SECTOR_SIZE); + blk_queue_max_segments(q, segment_size / SECTOR_SIZE); blk_queue_max_segment_size(q, segment_size); blk_queue_io_min(q, segment_size); blk_queue_io_opt(q, segment_size);