From patchwork Tue Oct 9 11:11:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matias Bjorling X-Patchwork-Id: 10632315 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 735E9174A for ; Tue, 9 Oct 2018 11:17:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 52E8C28A45 for ; Tue, 9 Oct 2018 11:17:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4714C28A73; Tue, 9 Oct 2018 11:17:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 342EC28A5E for ; Tue, 9 Oct 2018 11:17:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726775AbeJISai (ORCPT ); Tue, 9 Oct 2018 14:30:38 -0400 Received: from mail-pl1-f196.google.com ([209.85.214.196]:44869 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727333AbeJISai (ORCPT ); Tue, 9 Oct 2018 14:30:38 -0400 Received: by mail-pl1-f196.google.com with SMTP id p25-v6so658677pli.11 for ; Tue, 09 Oct 2018 04:14:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lightnvm-io.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AmFRqrNAlDrl6j89zyFw74SjJSrmvoigvbv7ouBR6SI=; b=QOfkiyQ9fVINpvKsv9zGV63BpJV/WL9bBl+HbpwZOerwm067gd9nWp7QrhiIOojeNJ KlBRuPl6G3YknkBO2tXuQZwL7XIu8RWSw4Uq7wsuqTPW7wCzEUSkpzYWzsAh+1jHpTco LqniOI9Zm2glzuZYjjlIW69RTq5CAMPRnEREZUxNTVxFT6xloUOatHnYRQkb9LhHBLwT QLKASGCx8Sn9mho5ZglFVZrgbWvlwDRCM6QwMTzb41LoC0BuiAH6AwedAukqo8LVM2e4 ut83llZhdI5ZOtGZCZOpo9rbURCUu5sq7L4FAcXXjyRGSPHocAZfl1faU4wdJwgs2zj/ Y1Uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AmFRqrNAlDrl6j89zyFw74SjJSrmvoigvbv7ouBR6SI=; b=FzuA2LJAKEqGqku9q4ODDytoajMI5NMuUagjE5A/vrgxSfyaS0kr/sKgdz3he57ChB vXPTsUxlJUmb5/T/Co9QvRBSkBAqCUnbbVTwdAZzFwPMntQWfPGLKFkY35vha1IrwuSj HwpQbbM3oA2XecnKbEzp0qzx+SXOnzs58MOewbO9vKfQX/nxSoyxdLMj0Gt2CEbtihQP scrv+uOKjYNvokb6vypmntyc6xjySqNzsl6umWy+im6IXU3sIhthxvcOmI5gD/oMHTkT +ETZfdypEurtHOwVUIcqKvewh0hc+QqIWMe1XFJxhFaF1dsDw5aTXvoDlKjCLX0SAaCg Cjeg== X-Gm-Message-State: ABuFfoghUrITa93DpgDeq5H20KWz9NLrP6Q8j/6EN3uVXk9D30Z4h+/2 dpCq/RtxTkglmby+zA++MhpacQ== X-Google-Smtp-Source: ACcGV62VbQ5vKp+vBY0Na8XqHjCauXjR3EYryMWqcvrMBiQYjAXEpxRQcDDfBDC178rtDrZhg6KnSg== X-Received: by 2002:a17:902:3341:: with SMTP id a59-v6mr16828620plc.138.1539083651576; Tue, 09 Oct 2018 04:14:11 -0700 (PDT) Received: from skyninja.hgst.com (rap-us.hgst.com. [199.255.44.250]) by smtp.gmail.com with ESMTPSA id p1-v6sm25342380pff.128.2018.10.09.04.14.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 09 Oct 2018 04:14:10 -0700 (PDT) From: =?utf-8?q?Matias_Bj=C3=B8rling?= To: axboe@fb.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, Hans Holmberg , =?utf-8?q?Matias_Bj=C3=B8rling?= Subject: [GIT PULL 17/45] lightnvm: pblk: allocate line map bitmaps using a mempool Date: Tue, 9 Oct 2018 13:11:47 +0200 Message-Id: <20181009111215.7653-18-mb@lightnvm.io> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181009111215.7653-1-mb@lightnvm.io> References: <20181009111215.7653-1-mb@lightnvm.io> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Hans Holmberg Line map bitmap allocations are fairly large and can fail. Allocation failures are fatal to pblk, stopping the write pipeline. To avoid this, allocate the bitmaps using a mempool instead. Mempool allocations never fail if called from a process context, and pblk *should* only allocate map bitmaps in process context, but keep the failure handling for robustness sake. Signed-off-by: Hans Holmberg Signed-off-by: Matias Bjørling --- drivers/lightnvm/pblk-core.c | 22 +++++++++++++++------- drivers/lightnvm/pblk-init.c | 18 ++++++++++++++++++ drivers/lightnvm/pblk-recovery.c | 2 +- drivers/lightnvm/pblk.h | 4 ++++ 4 files changed, 38 insertions(+), 8 deletions(-) diff --git a/drivers/lightnvm/pblk-core.c b/drivers/lightnvm/pblk-core.c index a31417682c90..e1207a4f9d54 100644 --- a/drivers/lightnvm/pblk-core.c +++ b/drivers/lightnvm/pblk-core.c @@ -1049,15 +1049,18 @@ static int pblk_line_init_metadata(struct pblk *pblk, struct pblk_line *line, static int pblk_line_alloc_bitmaps(struct pblk *pblk, struct pblk_line *line) { struct pblk_line_meta *lm = &pblk->lm; + struct pblk_line_mgmt *l_mg = &pblk->l_mg; - line->map_bitmap = kzalloc(lm->sec_bitmap_len, GFP_KERNEL); + line->map_bitmap = mempool_alloc(l_mg->bitmap_pool, GFP_KERNEL); if (!line->map_bitmap) return -ENOMEM; + memset(line->map_bitmap, 0, lm->sec_bitmap_len); + /* will be initialized using bb info from map_bitmap */ - line->invalid_bitmap = kmalloc(lm->sec_bitmap_len, GFP_KERNEL); + line->invalid_bitmap = mempool_alloc(l_mg->bitmap_pool, GFP_KERNEL); if (!line->invalid_bitmap) { - kfree(line->map_bitmap); + mempool_free(line->map_bitmap, l_mg->bitmap_pool); line->map_bitmap = NULL; return -ENOMEM; } @@ -1243,7 +1246,9 @@ int pblk_line_recov_alloc(struct pblk *pblk, struct pblk_line *line) void pblk_line_recov_close(struct pblk *pblk, struct pblk_line *line) { - kfree(line->map_bitmap); + struct pblk_line_mgmt *l_mg = &pblk->l_mg; + + mempool_free(line->map_bitmap, l_mg->bitmap_pool); line->map_bitmap = NULL; line->smeta = NULL; line->emeta = NULL; @@ -1261,8 +1266,11 @@ static void pblk_line_reinit(struct pblk_line *line) void pblk_line_free(struct pblk_line *line) { - kfree(line->map_bitmap); - kfree(line->invalid_bitmap); + struct pblk *pblk = line->pblk; + struct pblk_line_mgmt *l_mg = &pblk->l_mg; + + mempool_free(line->map_bitmap, l_mg->bitmap_pool); + mempool_free(line->invalid_bitmap, l_mg->bitmap_pool); pblk_line_reinit(line); } @@ -1741,7 +1749,7 @@ void pblk_line_close(struct pblk *pblk, struct pblk_line *line) list_add_tail(&line->list, move_list); - kfree(line->map_bitmap); + mempool_free(line->map_bitmap, l_mg->bitmap_pool); line->map_bitmap = NULL; line->smeta = NULL; line->emeta = NULL; diff --git a/drivers/lightnvm/pblk-init.c b/drivers/lightnvm/pblk-init.c index 8adc8ac8b03c..76a4a271b9cf 100644 --- a/drivers/lightnvm/pblk-init.c +++ b/drivers/lightnvm/pblk-init.c @@ -498,6 +498,9 @@ static void pblk_line_mg_free(struct pblk *pblk) pblk_mfree(l_mg->eline_meta[i]->buf, l_mg->emeta_alloc_type); kfree(l_mg->eline_meta[i]); } + + mempool_destroy(l_mg->bitmap_pool); + kmem_cache_destroy(l_mg->bitmap_cache); } static void pblk_line_meta_free(struct pblk_line_mgmt *l_mg, @@ -797,6 +800,17 @@ static int pblk_line_mg_init(struct pblk *pblk) goto fail_free_smeta; } + l_mg->bitmap_cache = kmem_cache_create("pblk_lm_bitmap", + lm->sec_bitmap_len, 0, 0, NULL); + if (!l_mg->bitmap_cache) + goto fail_free_smeta; + + /* the bitmap pool is used for both valid and map bitmaps */ + l_mg->bitmap_pool = mempool_create_slab_pool(PBLK_DATA_LINES * 2, + l_mg->bitmap_cache); + if (!l_mg->bitmap_pool) + goto fail_destroy_bitmap_cache; + /* emeta allocates three different buffers for managing metadata with * in-memory and in-media layouts */ @@ -849,6 +863,10 @@ static int pblk_line_mg_init(struct pblk *pblk) kfree(l_mg->eline_meta[i]->buf); kfree(l_mg->eline_meta[i]); } + + mempool_destroy(l_mg->bitmap_pool); +fail_destroy_bitmap_cache: + kmem_cache_destroy(l_mg->bitmap_cache); fail_free_smeta: for (i = 0; i < PBLK_DATA_LINES; i++) kfree(l_mg->sline_meta[i]); diff --git a/drivers/lightnvm/pblk-recovery.c b/drivers/lightnvm/pblk-recovery.c index 3bd2b6b0a359..eea901d7cebc 100644 --- a/drivers/lightnvm/pblk-recovery.c +++ b/drivers/lightnvm/pblk-recovery.c @@ -939,7 +939,7 @@ struct pblk_line *pblk_recov_l2p(struct pblk *pblk) list_move_tail(&line->list, move_list); spin_unlock(&l_mg->gc_lock); - kfree(line->map_bitmap); + mempool_free(line->map_bitmap, l_mg->bitmap_pool); line->map_bitmap = NULL; line->smeta = NULL; line->emeta = NULL; diff --git a/drivers/lightnvm/pblk.h b/drivers/lightnvm/pblk.h index 60c509a00574..9068b158de22 100644 --- a/drivers/lightnvm/pblk.h +++ b/drivers/lightnvm/pblk.h @@ -530,6 +530,10 @@ struct pblk_line_mgmt { struct pblk_emeta *eline_meta[PBLK_DATA_LINES]; unsigned long meta_bitmap; + /* Cache and mempool for map/invalid bitmaps */ + struct kmem_cache *bitmap_cache; + mempool_t *bitmap_pool; + /* Helpers for fast bitmap calculations */ unsigned long *bb_template; unsigned long *bb_aux;