From patchwork Thu Mar 2 16:13:30 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 9601355 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E42A260429 for ; Thu, 2 Mar 2017 18:17:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D9F7326C9B for ; Thu, 2 Mar 2017 18:17:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CEB05285D1; Thu, 2 Mar 2017 18:17:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1BB7226C9B for ; Thu, 2 Mar 2017 18:17:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755043AbdCBSRb (ORCPT ); Thu, 2 Mar 2017 13:17:31 -0500 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:38254 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755006AbdCBSRW (ORCPT ); Thu, 2 Mar 2017 13:17:22 -0500 Received: from pps.filterd (m0044008.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v22G979C015845; Thu, 2 Mar 2017 08:13:35 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=subject : to : references : cc : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=facebook; bh=0ukAC+yYKCCiLq/v9tDaWd0ayC4X4Dzkx4F+g5jMVyg=; b=bYkCA3arB1+M/Vge6f61HpBRDZg7Mf275MNluv6tqFUHM+DnWZSXIhWsJAyjnnlUj+iu u/bQU37A9k11pDMYB/j/OA0mHiMb6PJe+4mD+yy0UCR3ZaoOLE46iEI+FWLd+gXUwtDO 33Sl1/XXSSnc1E5KWTtN7g9X4XT/JYvyEBw= Received: from mail.thefacebook.com ([199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 28xjvrryuh-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 02 Mar 2017 08:13:35 -0800 Received: from NAM03-BY2-obe.outbound.protection.outlook.com (192.168.54.28) by o365-in.thefacebook.com (192.168.16.19) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 2 Mar 2017 08:13:34 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=0ukAC+yYKCCiLq/v9tDaWd0ayC4X4Dzkx4F+g5jMVyg=; b=F/+Mjej7ev8P2w28wTj0kPsiNQXTLIyEjUV7D3DkEOe8D8cHvaXBYhrJQcFVP5vGKlwV78iQ13Q1P4/C/PK11UeE+3IJwwHHVyye2b8NsAb6yzmtIOkpMbjN6Cu0GZX5mVTuJ4VR1ELGfMa1MlQNgbx5d+kK7DSyNqfoM9FV9LE= Received: from [192.168.1.154] (216.160.245.98) by CY4PR15MB1192.namprd15.prod.outlook.com (10.172.177.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.933.12; Thu, 2 Mar 2017 16:13:33 +0000 Subject: Re: [PATCH] blk-mq-sched: don't hold queue_lock when calling exit_icq To: Paolo Valente References: <73cd0cf484e8b75a771d908c172cd3a931dc00a3.1486751329.git.osandov@fb.com> <9fe1b3a9-2a80-8891-1044-f83558e28d15@fb.com> <1720FEB5-FBBA-4EAA-8292-E820AA15389D@linaro.org> <80de4554-a14b-4ce6-7c6e-ec66f70b14b1@fb.com> <6C03BF2A-0902-431A-A545-933F56D3E134@linaro.org> CC: Omar Sandoval , , From: Jens Axboe Message-ID: <75978d1d-8022-75c2-7799-aa65132fdcdd@fb.com> Date: Thu, 2 Mar 2017 09:13:30 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: <6C03BF2A-0902-431A-A545-933F56D3E134@linaro.org> X-Originating-IP: [216.160.245.98] X-ClientProxiedBy: CY4PR21CA0029.namprd21.prod.outlook.com (10.175.113.143) To CY4PR15MB1192.namprd15.prod.outlook.com (10.172.177.14) X-MS-Office365-Filtering-Correlation-Id: 83116c8b-0c1d-407a-13a8-08d461871379 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:CY4PR15MB1192; X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 3:SNBSiWx+4sbJ13EFQHxwaCtzRhX2wgtbIsD/S0GSbmoh5HKdnagmPjVXlgNKo4EsSqQvU9mlFzAEROSHcINLG3tvvV8aCEew7Q9yFnA9qWaidIufoY9/p8gnazLO0zMvKS6FKs95y4tx3KqdcHhlYMOmBiFcfPSp3zLz7hdu3ZWH9ENz7/+q8CwVa5m2pNNGGq6NEo6MzzC/r2SMB+KJzIf5UyGZrUvC61xgV1nm0f/ABQXq8cerKqXGivyaE+FC/Iin/QVapzN/d7R1OYGhtQ==; 25:/L4JgfuNJXOBIGTfiaRRZBqfBNQb8zfIMkkSjmqhtn1juSwUpn/T4U8EHSQLtRI7Af0nNc1gaIH2xbsuNTUZonpbmxXUtiYlLl2CkoB1QdqGyRZVg8uQz5+llX6XSTt+xxaiHyeFgm9oCV64/ZIwUH7P/HiKWxGKsXN+CHmyRCCoUUeaK5SolKzRmf4emxJ5NGrU+fJgtSWimvhTH3hdgNoNGJy3nOsC9KyzD54rDwuor6NxGk/de38L0N5bRnzw4KkO3nVK3efYEluUosD1a6O/fZrRfFWbersreEPNq2dGXt/lflFz4+Iv6PW5TiN9mgqmKkX3ypLl2FkbAHuAWgmtDK4QkVVzNxexBiJ9j4LcRTP8nuyNwvicAgDyc40Wj7X6Hy9gykc0sMu8h146YFolh7e/Dmv9zxwLjUwbogoG2uyPLtgl3U8yIMtc803mMsWdiBXaSKRU3dPWrrAcmw== X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 31:OwmS9yEpxmSuOhOLzUMxIORg9TgH99763ZjbjGme6AxIukkD9bx8MHglIFDg9IrJCCUl09niCPEmoNn7gLJKO+2qsYKQMWLjeSBEzr/hwPd9Dzwp2bMrujIc118R3EPNNLDniplnkLZ00+I79wcoeMb9oOqcM4CXcfc57m9sWNpUseEwcnFZUzcVPASai3HreHqRM7+Hyxp9o9d5hqMUYHsjoNhsL+qpuos81hdJAKk=; 20:lHz+ZBv6AwuSYKoeAU79IZtgKykjmaCsaG0ciuGuYnTVVJ5iRa5sx37tlNocnIYl7eV1Q3fAf/7BPP9iyzK/Qex0z3yW9BVUytQ7WPZmMOS/QS2gEy67VEJI+ZLG3C9o36I/wL2ZS+frzapoyaQ4qK4lB5EH3dJ/fqsYEHLvhX9CNtH+m40X0ue2DguFkVoSVLeZFDIcXBxu7khFxhvSEScq8FnHEAUUbhqZi/rGOOXeIRNouhmw8ydxv+a3lo6Vm5P3CWBlBnVtutFZne059wl8ONWoijmoheaOHYIonGrEJd5NlONrK2Bi8gMkJi4yVFMWEKoMHSQtw3JTbeVXuGzll/vpaDWIPHXnaiT6kau0LRlOzVMY24k4LNM0l/AfDLxfvWjO5z2p4k+BQI0h2qR7fKnLNhQvu5wjMlZ3rtINMa+KCETkbiJ368bpq1VOJv2tWnuai1ImZFqdmGx2/1dkhP8iMjOxa8v2sVKMMECQ9gQTc/RuPZ3zaIsSC+tr X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(67672495146484)(17755550239193); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040375)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6041248)(20161123558025)(20161123564025)(20161123555025)(20161123560025)(20161123562025)(6072148); SRVR:CY4PR15MB1192; BCL:0; PCL:0; RULEID:; SRVR:CY4PR15MB1192; X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 4:pgqcJKOYAnZv7bQlN6AJO7M8H366y3BRClKHpm5jjTQXzZqFxMQFdZQwR5Kuh4T6aeM9qDrUi5r2QB729AQKZTsvBciwT/CoBcWAAb+GU7b/OjspyR/QhkVO/LwmsemHcxQ3aX9SrDkI/WVGq59AAonCU1HrZF0IloxyRGglBs8p7AFi4821hGGIyoxQXpezJP5CnXKweTrtj3cn6ZdMccpjEZR3ag4QWFlwJqTIgI9m+X6v5XHmVgn9TQOZLEVcxuLcFCbRtmkkgb1l7A547M0Iac3hkPBI28QHNO10cAQx/w88Fp5PFQDa/OUsurYuIriK7e6MxyTUDMzJXsFESnIEvakHas3E15LfESEx9DsMggeqyoOEv10fV+5b91GMkp5eIDe+FFCiBKSvBvpMJOqfzwOqDcUzzkTzWNvq8nFHFxoqZFYOqk//OzO4C+9uxLqZe5Z/AeNhLLxhk4REobmuSaapVQICsWQ94+TxyEL3WR2VUWkCK2sO6YRx4EZv69ONa6IbNsV8Mpvx060Pvz21L55rKoFtIQa6bhvCxyjGgGf0IMO/FJI/6UnspEtLCH/1p/yUzNEEIs9nHLJzFWTHwa1mJul2VjTDMeAWxLviG0WGF0TIICiVwE+zWSi7uQBHWpTfQGHtWxaNqeJVL6dgRE6pBDBuxryDtWG8iu9Te0JNUqzOTy/NcZDQIXk/ X-Forefront-PRVS: 023495660C X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(4630300001)(6009001)(6049001)(7916002)(39830400002)(39450400003)(39410400002)(377454003)(24454002)(53546006)(90366009)(229853002)(2950100002)(189998001)(92566002)(6246003)(42186005)(38730400002)(47776003)(50466002)(31686004)(6916009)(4326008)(53936002)(110136004)(7736002)(8676002)(305945005)(81166006)(50986999)(31696002)(36756003)(4001350100001)(2906002)(23746002)(54906002)(25786008)(83506001)(5660300001)(54356999)(33646002)(76176999)(6486002)(6116002)(117156001)(3846002)(66066001)(64126003)(230700001)(93886004)(77096006)(86362001)(230783001); DIR:OUT; SFP:1102; SCL:1; SRVR:CY4PR15MB1192; H:[192.168.1.154]; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1; CY4PR15MB1192; 23:Q9o5MZ6i8zAUltrWV0UWArMxkSuaFQqSmrgkh?= =?Windows-1252?Q?GiG0ZOzu+3jpmFpmeeliGpipqMXMugYsp+9a1Z6mdGVolQISw8F9SQJ5?= =?Windows-1252?Q?oblggL3xDwVwgzWq28IMLFlZJ/VAXFFKrPhHoLsVF3lUWQB+4d1eJSDy?= =?Windows-1252?Q?IrcOFFFw5LiPcg+wS2uI0Vm+8JmYaFxGzQNLgEdrtWv/1iqXJ0/+c9Of?= =?Windows-1252?Q?VEsfOPVvSLmoyE2VpJ7x23WGldy9Kv4v0Kk9vKUOZreQNdHnD4bnxjPR?= =?Windows-1252?Q?mFGGdqDi0uBTcsh+cEy4WooSNggk37CJU+BTDciUzvHR2psKerzNMJKZ?= =?Windows-1252?Q?3sIKApO1uhaDDbYQfsA3JtwiS1mUHBI6sRoJYpeMxBN0WrgIa6o61srJ?= =?Windows-1252?Q?tZVUi4e8cn2230SStdsncJtiRQPaHwa5o9RCKBBIaETOBoansAJ8OCD+?= =?Windows-1252?Q?KzGBJrv/GHgjKRvPcu8+9xqjlo8XUB84jNn1NC0Wrpyl42fCwVmdB5x/?= =?Windows-1252?Q?GaPFQJCPVVfVKP+ENvSFtBDRWFBsbdE+1kvx51dvB0TBlV+B3ZtbE3y8?= =?Windows-1252?Q?toVhWX7ohB306PIR9rVdxDM84hc5syP9JrPQR0wzqeI2UxXMFG4hSsIC?= =?Windows-1252?Q?pK9/QUpdMLBHk39Wa5mCWAF1iaERATIYxMXuO8CoDMipTp6RTytVv0/1?= =?Windows-1252?Q?UH3cfLNpjDdFIU9F62o9s82UKik74KG4AF62SuPgHRQ5FUarqEpTGzmI?= =?Windows-1252?Q?YdtZFml5tZVQWS1JI5MK6+Pv1znUVvpht9CfTal1zMx3yx91/XnrsluT?= =?Windows-1252?Q?mpbH399NO9O4FQhwF2xIU02x4rkHyA/OpN1xwpUSASA3ERm7B/BZNWOU?= =?Windows-1252?Q?yeBHGG1W2Dcvl/OWiR9jxcKMvxJ2b1jAK894jnkvHD8F4Bo0gGf7TW9N?= =?Windows-1252?Q?PQ0V24VYNLjkjekpK3hUNQpQZlVh45FY6B+R6QBC/7ZgKl+C5vW1mKjM?= =?Windows-1252?Q?JHdHEhA90hWjQD5SuAWBnXStbI8YWTr7NNeoTgyl926/XPI9afWfgwO5?= =?Windows-1252?Q?Qe3PYYmixL7tqydPGAGTV616PJmpT+v21IZRyi4TDE0G/yBWCYkjQoau?= =?Windows-1252?Q?kTt/6HsaJsFI4zzy14zx+Td5p3oH4YpNGEyG7it2/NWjHjGnBwXsXzv3?= =?Windows-1252?Q?VcJicAmHEP9ZWuB8DnIq0ddKcocG66+546SkPViHgS+hOnH1tZ79KvLa?= =?Windows-1252?Q?6fL6E1CxGdXxu6glgExS6tNFSwI32JzS9Z55XzdYpTs7qoFJY4D+Os5a?= =?Windows-1252?Q?kybWsV8ekQkvwFLnmnW6jJXCepYVrMxB0suWemW7EolXhQXAqfncHode?= =?Windows-1252?Q?qeejVTHUoz8iUxMobUJRs4H5cviCegTcw=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 6:M2NVXvjhWiGmMwS5zeg2BdWAmh5FidV7jwAgcSHK9Raf8MorSvB++ToXNUVQC6Gn8vBTyRD9cjNQRtBsaUiFa9KMmE/aGF9dGmrVfhfmIkPOAq8FlAm8uOkXvtCUc5Z7rgV84hWD12uQlf5wak8jrVe5GZBou5wnk9bMDV2NOaAuc94jUmepVysig8AuBExkb6Cl3WicAcMgiWP4RXgXozrT2KGYbjYhb99rxoNLaBCljTXueNmQeDu9NhI+qN8ZA8g1LdCso5a927ZJqH0eDk/ikiM+RhkP/sGkY9As5idaIuT8YUtqw2+f7XUOnwblrNHFGO4wviTzYJP5bV7vU1j+Ka0h040fH8kcStFID2nvh2qI05lcFE3mV8OoO0JHFOHv+Td6KnsATgLGElhVxQ==; 5:suQnEfKiWrBsGdBy70D5TQD+INQXlCxlErHLdOR6+pU1LUN5XBxnzSAdtOPwpCyl4Jz+t9dSuLC6TDcolJXLbVU2UEdPmy5q1w6kA9pO1SDG29nJUVcPaB/km2VJJzNUUlmnMphr165HP/QURlDY0g==; 24:B8Rsn2fOs/xMy2QT0CFDcpDmixExWoAPEAYY89riFD7rRCz/eRWwaHaHvxMJzGa5Fyjm/t+Dud0YjN8+JAgYcctuqTHXcpOEchuPyaTMJRw= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 7:vuAu8Y/jpMJY8bsFlu1Eulx5IaPk8+KRczq4gZ2EnTEuwxgak9RMlVhi1a+I4mstHQPYp3LCzmdrlURXf7GJf+YIs+bsZye6jXwfpnTz/K+Wvz+10BFxodJ3MVf7I6j4r5lMnAGiXinTyZsodxVZop42Zj+vr17JQHW1MD19n/8dyhmoCRo1froFjXhA1PNJgBvyuCVNh8l6550mfUjnujEANTZME9uryOel0LeJEhqa8B1xu/+6zlnJWlUC1TAelGy75RqmaYkyEoaUa5af4nBQKfsXt0FL0lHL5xujm3ZXgAaRKPsMT9soskiaNfZis1M/Ii1/vWOZ9kIJpPpf1Q==; 20:bKj0Ia50WTP/ckmiJJUZBQU9+DbBc5hYEOHZzS+P5jqNd8UaZTqjdtZb10p+zWmw0+018HktmHlUkJbYQDOAPqmdfoJk6dMU7su0crNJk3eLPRQwfGgs+DX7Pkb+4oMcdkrzEhT/SH02J7mkCoY/O0Vb12Ma2RBx+pJEfG8Pq/k= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Mar 2017 16:13:33.9545 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR15MB1192 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-03-02_15:, , signatures=0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 03/02/2017 09:07 AM, Paolo Valente wrote: > >> Il giorno 02 mar 2017, alle ore 16:13, Jens Axboe ha scritto: >> >> On 03/02/2017 08:00 AM, Jens Axboe wrote: >>> On 03/02/2017 03:28 AM, Paolo Valente wrote: >>>> >>>>> Il giorno 15 feb 2017, alle ore 19:04, Jens Axboe ha scritto: >>>>> >>>>> On 02/15/2017 10:58 AM, Jens Axboe wrote: >>>>>> On 02/15/2017 10:24 AM, Paolo Valente wrote: >>>>>>> >>>>>>>> Il giorno 10 feb 2017, alle ore 19:32, Omar Sandoval ha scritto: >>>>>>>> >>>>>>>> From: Omar Sandoval >>>>>>>> >>>>>>>> None of the other blk-mq elevator hooks are called with this lock held. >>>>>>>> Additionally, it can lead to circular locking dependencies between >>>>>>>> queue_lock and the private scheduler lock. >>>>>>>> >>>>>>> >>>>>>> Hi Omar, >>>>>>> I'm sorry but it seems that a new potential deadlock has showed up. >>>>>>> See lockdep splat below. >>>>>>> >>>>>>> I've tried to think about different solutions than turning back to >>>>>>> deferring the body of exit_icq, but at no avail. >>>>>> >>>>>> Looks like a interaction between bfqd->lock and q->queue_lock. Since the >>>>>> core has no notion of you bfqd->lock, the naturally dependency here >>>>>> would be to nest bfqd->lock inside q->queue_lock. Is that possible for >>>>>> you? >>>>>> >>>>>> Looking at the code a bit, maybe it'd just be simpler to get rid of >>>>>> holding the queue lock for that spot. For the mq scheduler, we really >>>>>> don't want places where we invoke with that held already. Does the below >>>>>> work for you? >>>>> >>>>> Would need to remove one more lockdep assert. And only test this for >>>>> the mq parts, we'd need to spread a bit of love on the classic >>>>> scheduling icq exit path for this to work on that side. >>>>> >>>> >>>> >>>> Jens, >>>> here is the reply I anticipated in my previous email: after rebasing >>>> against master, I'm getting again the deadlock that this patch of >>>> yours solved (together with some changes in bfq-mq). I thought you added a >>>> sort of equivalent commit (now) to the mainline branch. Am I wrong? >>> >>> The patch I posted was never pulled to completion, it wasn't clear >>> to me if it fixed your issue or not. Maybe I missed a reply on >>> that? >>> >>> Let me take another stab at it today, I'll send you a version to test >>> on top of my for-linus branch. >> >> I took at look at my last posting, and I think it was only missing a >> lock grab in cfq, since we don't hold that for icq exit anymore. Paolo, >> it'd be great if you could verify that this works. Not for bfq-mq (we >> already know it does), but for CFQ. > > No luck, sorry :( > > It looks like to have just not to take q->queue_lock in cfq_exit_icq. I was worried about that. How about the below? We need to grab the lock at some point for legacy scheduling, but the ordering should be correct. Reviewed-by: Omar Sandoval diff --git a/block/blk-ioc.c b/block/blk-ioc.c index b12f9c87b4c3..6fd633b5d567 100644 --- a/block/blk-ioc.c +++ b/block/blk-ioc.c @@ -54,7 +54,7 @@ static void ioc_exit_icq(struct io_cq *icq) icq->flags |= ICQ_EXITED; } -/* Release an icq. Called with both ioc and q locked. */ +/* Release an icq. Called with ioc locked. */ static void ioc_destroy_icq(struct io_cq *icq) { struct io_context *ioc = icq->ioc; @@ -62,7 +62,6 @@ static void ioc_destroy_icq(struct io_cq *icq) struct elevator_type *et = q->elevator->type; lockdep_assert_held(&ioc->lock); - lockdep_assert_held(q->queue_lock); radix_tree_delete(&ioc->icq_tree, icq->q->id); hlist_del_init(&icq->ioc_node); @@ -222,24 +221,40 @@ void exit_io_context(struct task_struct *task) put_io_context_active(ioc); } +static void __ioc_clear_queue(struct list_head *icq_list) +{ + unsigned long flags; + + while (!list_empty(icq_list)) { + struct io_cq *icq = list_entry(icq_list->next, + struct io_cq, q_node); + struct io_context *ioc = icq->ioc; + + spin_lock_irqsave(&ioc->lock, flags); + ioc_destroy_icq(icq); + spin_unlock_irqrestore(&ioc->lock, flags); + } +} + /** * ioc_clear_queue - break any ioc association with the specified queue * @q: request_queue being cleared * - * Walk @q->icq_list and exit all io_cq's. Must be called with @q locked. + * Walk @q->icq_list and exit all io_cq's. */ void ioc_clear_queue(struct request_queue *q) { - lockdep_assert_held(q->queue_lock); + LIST_HEAD(icq_list); - while (!list_empty(&q->icq_list)) { - struct io_cq *icq = list_entry(q->icq_list.next, - struct io_cq, q_node); - struct io_context *ioc = icq->ioc; + spin_lock_irq(q->queue_lock); + list_splice_init(&q->icq_list, &icq_list); - spin_lock(&ioc->lock); - ioc_destroy_icq(icq); - spin_unlock(&ioc->lock); + if (q->mq_ops) { + spin_unlock_irq(q->queue_lock); + __ioc_clear_queue(&icq_list); + } else { + __ioc_clear_queue(&icq_list); + spin_unlock_irq(q->queue_lock); } } diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 002af836aa87..c44b321335f3 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -815,9 +815,7 @@ static void blk_release_queue(struct kobject *kobj) blkcg_exit_queue(q); if (q->elevator) { - spin_lock_irq(q->queue_lock); ioc_clear_queue(q); - spin_unlock_irq(q->queue_lock); elevator_exit(q->elevator); } diff --git a/block/elevator.c b/block/elevator.c index ac1c9f481a98..01139f549b5b 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -983,9 +983,7 @@ static int elevator_switch(struct request_queue *q, struct elevator_type *new_e) if (old_registered) elv_unregister_queue(q); - spin_lock_irq(q->queue_lock); ioc_clear_queue(q); - spin_unlock_irq(q->queue_lock); } /* allocate, init and register new elevator */