From patchwork Thu Apr 9 21:45:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 11482501 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 193B814B4 for ; Thu, 9 Apr 2020 21:45:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DA11E2072F for ; Thu, 9 Apr 2020 21:45:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DA11E2072F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B4CE28E001D; Thu, 9 Apr 2020 17:45:41 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A86DE8E0003; Thu, 9 Apr 2020 17:45:41 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D8ED8E001D; Thu, 9 Apr 2020 17:45:41 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0039.hostedemail.com [216.40.44.39]) by kanga.kvack.org (Postfix) with ESMTP id 81AB88E0003 for ; Thu, 9 Apr 2020 17:45:41 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 526F82470 for ; Thu, 9 Apr 2020 21:45:41 +0000 (UTC) X-FDA: 76689648882.14.pull75_6bd76474d4c5f X-Spam-Summary: 2,0,0,c976883f82d22e82,d41d8cd98f00b204,mcgrof@gmail.com,,RULES_HIT:2:41:355:379:541:800:960:966:968:973:982:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1605:1606:1730:1747:1777:1792:2196:2198:2199:2200:2393:2553:2559:2562:2903:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3873:3874:4119:4250:4321:4385:4605:5007:6120:6261:6742:7875:7903:8660:9389:9592:10004:11026:11232:11473:11658:11914:12043:12048:12050:12291:12296:12297:12438:12517:12519:12555:12683:12895:13141:13146:13148:13230:13894:13972:21080:21212:21433:21444:21451:21627:21740:21972:21987:21990:30012:30054:30064:30070:30090:30091,0,RBL:209.85.215.196:@gmail.com:.lbl8.mailshell.net-66.100.201.100 62.50.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:14,LUA_SUMMARY:none X-HE-Tag: pull75_6bd76474d4c5f X-Filterd-Recvd-Size: 8322 Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Thu, 9 Apr 2020 21:45:40 +0000 (UTC) Received: by mail-pg1-f196.google.com with SMTP id p8so101533pgi.5 for ; Thu, 09 Apr 2020 14:45:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RyMpnPWw/Y2hF5e4RISVaGw9GiKX0YRBleN8UhVQook=; b=Q9VBE9H/jA+EvTlAhCQb5W/JPw0TyHyNi6BBbEBkoz0pw69a5qGOBZOBf3AKxifGTZ SFSpKYpw4DU7ySfM4e3OWekOTTWk/9bN0YLoM21CT3cOe+4OudMumqXXVNnDr45geJ5W vT2IRrvRh64RPShFcWtOj4IMoXBHVruq9LHSabQ1066YmCb3s2L7hpWsdxlw6IDrV6yR 2UcO6iL69dA/RHDseRWSXTI/tt3ttKn6OUZi2Wagzeie/nS21vy1q74byDmpQ6U4qFhO yyHbzQKku4xx5QR8Bvd0zR5ENlqvNJNX2WBPLbhNdUTDeLYDF8cP8nNJexyTPoruadRk CBYw== X-Gm-Message-State: AGi0PubP8QMpdnW66dH+51IrL6sawreL6mgdtVh1vWZzVzfOQL4JCLEh mVK0A2nWYts9vnH9mtgh7LI= X-Google-Smtp-Source: APiQypL2c8CK9HLPDrIBFDxSQIJ2+qEM1rtn9Tgz5zxuYIUXNV4bsvZbJNQbtegiTbkBK3QjOXl2Dw== X-Received: by 2002:a63:d512:: with SMTP id c18mr1420081pgg.347.1586468740119; Thu, 09 Apr 2020 14:45:40 -0700 (PDT) Received: from 42.do-not-panic.com (42.do-not-panic.com. [157.230.128.187]) by smtp.gmail.com with ESMTPSA id w134sm65241pfd.41.2020.04.09.14.45.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Apr 2020 14:45:37 -0700 (PDT) Received: by 42.do-not-panic.com (Postfix, from userid 1000) id 7115241DAB; Thu, 9 Apr 2020 21:45:32 +0000 (UTC) From: Luis Chamberlain To: axboe@kernel.dk, viro@zeniv.linux.org.uk, gregkh@linuxfoundation.org, rostedt@goodmis.org, mingo@redhat.com, jack@suse.cz, ming.lei@redhat.com, nstange@suse.de, akpm@linux-foundation.org Cc: mhocko@suse.com, yukuai3@huawei.com, linux-block@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Luis Chamberlain , Bart Van Assche , Omar Sandoval , Hannes Reinecke , Michal Hocko Subject: [RFC v2 5/5] block: revert back to synchronous request_queue removal Date: Thu, 9 Apr 2020 21:45:30 +0000 Message-Id: <20200409214530.2413-6-mcgrof@kernel.org> X-Mailer: git-send-email 2.23.0.rc1 In-Reply-To: <20200409214530.2413-1-mcgrof@kernel.org> References: <20200409214530.2413-1-mcgrof@kernel.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Commit dc9edc44de6c ("block: Fix a blk_exit_rl() regression") merged on v4.12 moved the work behind blk_release_queue() into a workqueue after a splat floated around which indicated some work on blk_release_queue() could sleep in blk_exit_rl(). This splat would be possible when a driver called blk_put_queue() or blk_cleanup_queue() (which calls blk_put_queue() as its final call) from an atomic context. blk_put_queue() puts decrements the refcount for the request_queue kobject, and upon reaching 0 blk_release_queue() is called. Although blk_exit_rl() is now removed through commit db6d9952356 ("block: remove request_list code"), we reserve the right to be able to sleep within blk_release_queue() context. There should be little reason to defer removal from atomic context these days, as you can always just increase your block device's reference count even in atomic context and leave the removal for the request_queue to the upper layers later. However if you really need to defer removal of the request_queue, you can set the queue flag QUEUE_FLAG_DEFER_REMOVAL now. Cc: Bart Van Assche Cc: Omar Sandoval Cc: Hannes Reinecke Cc: Nicolai Stange Cc: Greg Kroah-Hartman Cc: Michal Hocko Cc: yu kuai Signed-off-by: Luis Chamberlain --- block/blk-sysfs.c | 40 ++++++++++++++++++++++++++++++++-------- include/linux/blkdev.h | 3 +++ 2 files changed, 35 insertions(+), 8 deletions(-) diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 20f20b0fa0b9..2ae8c39c88ef 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -860,10 +860,9 @@ static void blk_exit_queue(struct request_queue *q) bdi_put(q->backing_dev_info); } - /** - * __blk_release_queue - release a request queue - * @work: pointer to the release_work member of the request queue to be released + * blk_release_queue_sync- release a request queue + * @q: pointer to the request queue to be released * * Description: * This function is called when a block device is being unregistered. The @@ -872,11 +871,27 @@ static void blk_exit_queue(struct request_queue *q) * the reference counter of the request queue. Once the reference counter * of the request queue reaches zero, blk_release_queue is called to release * all allocated resources of the request queue. + * + * There are two approaches to releasing the request queue, by default + * we reserve the right to sleep on release and so release is synchronous. + * If you know the path under which blk_cleanup_queue() or your last + * blk_put_queue() is called can be called in atomic context you want to + * ensure to defer the removal by setting the QUEUE_FLAG_DEFER_REMOVAL + * flag as follows upon initialization: + * + * blk_queue_flag_set(QUEUE_FLAG_DEFER_REMOVAL, q) + * + * Note that deferring removal may have implications for userspace. An + * example is if you are using an ioctl to allow removal of a block device, + * and the kernel returns immediately even though the device may only + * disappear after the full removal is completed. + * + * You should also be able to work around this by just increasing the + * refcount for the block device instead during your atomic operation, + * and so QUEUE_FLAG_DEFER_REMOVAL should almost never be required. */ -static void __blk_release_queue(struct work_struct *work) +static void blk_release_queue_sync(struct request_queue *q) { - struct request_queue *q = container_of(work, typeof(*q), release_work); - if (test_bit(QUEUE_FLAG_POLL_STATS, &q->queue_flags)) blk_stat_remove_callback(q, q->poll_cb); blk_stat_free_callback(q->poll_cb); @@ -905,13 +920,22 @@ static void __blk_release_queue(struct work_struct *work) call_rcu(&q->rcu_head, blk_free_queue_rcu); } +void __blk_release_queue(struct work_struct *work) +{ + struct request_queue *q = container_of(work, typeof(*q), release_work); + + blk_release_queue_sync(q); +} + static void blk_release_queue(struct kobject *kobj) { struct request_queue *q = container_of(kobj, struct request_queue, kobj); - INIT_WORK(&q->release_work, __blk_release_queue); - schedule_work(&q->release_work); + if (blk_queue_defer_removal(q)) + schedule_work(&q->release_work); + else + blk_release_queue_sync(q); } static const struct sysfs_ops queue_sysfs_ops = { diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 8b1cab52cef9..46fee1ef92e3 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -614,6 +614,7 @@ struct request_queue { #define QUEUE_FLAG_PCI_P2PDMA 25 /* device supports PCI p2p requests */ #define QUEUE_FLAG_ZONE_RESETALL 26 /* supports Zone Reset All */ #define QUEUE_FLAG_RQ_ALLOC_TIME 27 /* record rq->alloc_time_ns */ +#define QUEUE_FLAG_DEFER_REMOVAL 28 /* defer queue removal */ #define QUEUE_FLAG_MQ_DEFAULT ((1 << QUEUE_FLAG_IO_STAT) | \ (1 << QUEUE_FLAG_SAME_COMP)) @@ -648,6 +649,8 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q); #else #define blk_queue_rq_alloc_time(q) false #endif +#define blk_queue_defer_removal(q) \ + test_bit(QUEUE_FLAG_DEFER_REMOVAL, &(q)->queue_flags) #define blk_noretry_request(rq) \ ((rq)->cmd_flags & (REQ_FAILFAST_DEV|REQ_FAILFAST_TRANSPORT| \