From patchwork Thu Jun 14 13:38:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hannes Reinecke X-Patchwork-Id: 10464141 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 93BA4603B4 for ; Thu, 14 Jun 2018 13:38:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 80B82202DB for ; Thu, 14 Jun 2018 13:38:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7337525EF7; Thu, 14 Jun 2018 13:38:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1682B202DB for ; Thu, 14 Jun 2018 13:38:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754927AbeFNNik (ORCPT ); Thu, 14 Jun 2018 09:38:40 -0400 Received: from mx2.suse.de ([195.135.220.15]:52886 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755102AbeFNNih (ORCPT ); Thu, 14 Jun 2018 09:38:37 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay1.suse.de (charybdis-ext-too.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 3EB80ABBF; Thu, 14 Jun 2018 13:38:36 +0000 (UTC) From: Hannes Reinecke To: Jens Axboe Cc: linux-block@vger.kernel.org, Mel Gorman , Hannes Reinecke , Hannes Reinecke Subject: [PATCH] brd: Allow ramdisk to be allocated on selected NUMA node Date: Thu, 14 Jun 2018 15:38:32 +0200 Message-Id: <20180614133832.110947-1-hare@suse.de> X-Mailer: git-send-email 2.12.3 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For performance reasons we should be able to allocate all memory from a given NUMA node, so this patch adds a new parameter 'rd_numa_node' to allow the user to specify the NUMA node id. When restricing fio to use the same NUMA node I'm seeing a performance boost of more than 200%. Signed-off-by: Hannes Reinecke --- drivers/block/brd.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/drivers/block/brd.c b/drivers/block/brd.c index bb976598ee43..7142d836539e 100644 --- a/drivers/block/brd.c +++ b/drivers/block/brd.c @@ -36,6 +36,7 @@ */ struct brd_device { int brd_number; + int brd_numa_node; struct request_queue *brd_queue; struct gendisk *brd_disk; @@ -103,7 +104,7 @@ static struct page *brd_insert_page(struct brd_device *brd, sector_t sector) * restriction might be able to be lifted. */ gfp_flags = GFP_NOIO | __GFP_ZERO; - page = alloc_page(gfp_flags); + page = alloc_pages_node(brd->brd_numa_node, gfp_flags, 0); if (!page) return NULL; @@ -342,6 +343,10 @@ static int max_part = 1; module_param(max_part, int, 0444); MODULE_PARM_DESC(max_part, "Num Minors to reserve between devices"); +static int rd_numa_node = NUMA_NO_NODE; +module_param(rd_numa_node, int, 0444); +MODULE_PARM_DESC(rd_numa_node, "NUMA node number to allocate RAM disk on."); + MODULE_LICENSE("GPL"); MODULE_ALIAS_BLOCKDEV_MAJOR(RAMDISK_MAJOR); MODULE_ALIAS("rd"); @@ -363,7 +368,7 @@ __setup("ramdisk_size=", ramdisk_size); static LIST_HEAD(brd_devices); static DEFINE_MUTEX(brd_devices_mutex); -static struct brd_device *brd_alloc(int i) +static struct brd_device *brd_alloc(int i, int node) { struct brd_device *brd; struct gendisk *disk; @@ -372,10 +377,11 @@ static struct brd_device *brd_alloc(int i) if (!brd) goto out; brd->brd_number = i; + brd->brd_numa_node = node; spin_lock_init(&brd->brd_lock); INIT_RADIX_TREE(&brd->brd_pages, GFP_ATOMIC); - brd->brd_queue = blk_alloc_queue(GFP_KERNEL); + brd->brd_queue = blk_alloc_queue_node(GFP_KERNEL, node, NULL); if (!brd->brd_queue) goto out_free_dev; @@ -434,7 +440,7 @@ static struct brd_device *brd_init_one(int i, bool *new) goto out; } - brd = brd_alloc(i); + brd = brd_alloc(i, rd_numa_node); if (brd) { add_disk(brd->brd_disk); list_add_tail(&brd->brd_list, &brd_devices); @@ -495,7 +501,7 @@ static int __init brd_init(void) max_part = 1; for (i = 0; i < rd_nr; i++) { - brd = brd_alloc(i); + brd = brd_alloc(i, rd_numa_node); if (!brd) goto out_free; list_add_tail(&brd->brd_list, &brd_devices);