From patchwork Fri Apr 8 09:38:11 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 8782351 Return-Path: X-Original-To: patchwork-linux-block@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 1292F9F659 for ; Fri, 8 Apr 2016 09:38:37 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 1E1AA2020F for ; Fri, 8 Apr 2016 09:38:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AC43820149 for ; Fri, 8 Apr 2016 09:38:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755900AbcDHJid (ORCPT ); Fri, 8 Apr 2016 05:38:33 -0400 Received: from mail-io0-f196.google.com ([209.85.223.196]:34202 "EHLO mail-io0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753462AbcDHJic (ORCPT ); Fri, 8 Apr 2016 05:38:32 -0400 Received: by mail-io0-f196.google.com with SMTP id z133so15550259iod.1; Fri, 08 Apr 2016 02:38:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=ZMQlnifBNlOUfsxsGiAXNT6k1TVTgGwHJU7452Riqnk=; b=QN6N0z3gWefYI/MCSbDkqhthnFto+w/QQnoQeZoEmduLwjnKO2DUIU2Su0g6Y1oR/I wlwpjbFskIVp8NUUubaOIskciJVEJpw32Ti4P2HnJDVE5Jc+/Bpyma5/gMOyoVWItVP8 0H9G2v//xRsErwK9Ok15QjGXLNE+3qWlLpj/sY+giRbQAEib2zgBeIa22W7Sg/iD124U 0UFp+YR0NneXjICIwTYml2euKLeouQH2fz5RlYm6z9qRT2VnWh5YatJOz0aidv8mm33c 97uN6zA5CPqvlTZJEFtuzw1vlosN8l4hMg3iqqd5IRN0aPp1EkvkmyRrKG+UnWOPodGs YbFg== X-Gm-Message-State: AD7BkJLUZSpHRRmB5GqIRuT+MQsw3OP+ebObQ9z5LidRMFIDsrBYWRzqcQYDpF+5i53YlQ== X-Received: by 10.107.10.16 with SMTP id u16mr9097735ioi.167.1460108311616; Fri, 08 Apr 2016 02:38:31 -0700 (PDT) Received: from localhost (45-125-195-13.ip4.readyserver.sg. [45.125.195.13]) by smtp.gmail.com with ESMTPSA id je6sm1446861igb.15.2016.04.08.02.38.29 (version=TLS1_2 cipher=AES128-SHA bits=128/128); Fri, 08 Apr 2016 02:38:30 -0700 (PDT) From: Ming Lei To: Jens Axboe , linux-kernel@vger.kernel.org Cc: linux-block@vger.kernel.org, linux-btrfs@vger.kernel.org, Christoph Hellwig , Ming Lei , Shaun Tancheff , Mikulas Patocka Subject: [RFC PATCH v1] block: avoid to call .bi_end_io() recursively Date: Fri, 8 Apr 2016 17:38:11 +0800 Message-Id: <1460108291-2387-1-git-send-email-ming.lei@canonical.com> X-Mailer: git-send-email 1.9.1 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Spam-Status: No, score=-7.8 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RCVD_IN_SBL, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP There were reports about heavy stack use by recursive calling .bi_end_io().[1][2][3] Also these patches[1] [2] [3] were posted for addressing the issue. And the idea is basically similar, all serializes the recursive calling of .bi_end_io() by percpu list. This patch still takes the same idea, but uses bio_list to implement it, which turns out more simple and the code becomes more readable meantime. xfstests(-g auto) is run with this patch and no regression is found on ext4, but when testing btrfs, generic/224 and generic/323 causes kernel oops. [1] http://marc.info/?t=121428502000004&r=1&w=2 [2] http://marc.info/?l=dm-devel&m=139595190620008&w=2 [3] http://marc.info/?t=145974644100001&r=1&w=2 Cc: Shaun Tancheff Cc: Christoph Hellwig Cc: Mikulas Patocka Signed-off-by: Ming Lei --- V1: - change to RFC - fix when unwind_bio_endio() is called recursively - run xfstest again: no regression found on ext4, but generic/323 and generic/224 cause kernel oops block/bio.c | 44 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 42 insertions(+), 2 deletions(-) diff --git a/block/bio.c b/block/bio.c index f124a0a..e2d0970 100644 --- a/block/bio.c +++ b/block/bio.c @@ -68,6 +68,8 @@ static DEFINE_MUTEX(bio_slab_lock); static struct bio_slab *bio_slabs; static unsigned int bio_slab_nr, bio_slab_max; +static DEFINE_PER_CPU(struct bio_list *, bio_end_list) = { NULL }; + static struct kmem_cache *bio_find_or_create_slab(unsigned int extra_size) { unsigned int sz = sizeof(struct bio) + extra_size; @@ -1737,6 +1739,45 @@ static inline bool bio_remaining_done(struct bio *bio) return false; } +/* disable local irq when manipulating the percpu bio_list */ +static void unwind_bio_endio(struct bio *bio) +{ + struct bio_list *bl; + unsigned long flags; + bool clear_list = false; + + preempt_disable(); + local_irq_save(flags); + + bl = this_cpu_read(bio_end_list); + if (!bl) { + struct bio_list bl_in_stack; + + bl = &bl_in_stack; + bio_list_init(bl); + this_cpu_write(bio_end_list, bl); + clear_list = true; + } else { + bio_list_add(bl, bio); + goto out; + } + + while (bio) { + local_irq_restore(flags); + + if (bio->bi_end_io) + bio->bi_end_io(bio); + + local_irq_save(flags); + bio = bio_list_pop(bl); + } + if (clear_list) + this_cpu_write(bio_end_list, NULL); + out: + local_irq_restore(flags); + preempt_enable(); +} + /** * bio_endio - end I/O on a bio * @bio: bio @@ -1765,8 +1806,7 @@ again: goto again; } - if (bio->bi_end_io) - bio->bi_end_io(bio); + unwind_bio_endio(bio); } EXPORT_SYMBOL(bio_endio);