From patchwork Thu Feb 2 05:19:58 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 9551039 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6054460236 for ; Thu, 2 Feb 2017 05:20:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 508412838D for ; Thu, 2 Feb 2017 05:20:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 450652844A; Thu, 2 Feb 2017 05:20:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 792C22838D for ; Thu, 2 Feb 2017 05:20:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751057AbdBBFU1 (ORCPT ); Thu, 2 Feb 2017 00:20:27 -0500 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:44096 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751039AbdBBFUY (ORCPT ); Thu, 2 Feb 2017 00:20:24 -0500 Received: from pps.filterd (m0044008.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v125ImFm019763; Wed, 1 Feb 2017 21:20:22 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=subject : to : references : cc : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=facebook; bh=HU8+bAY310tRx2EruXgedncSLr66J2ISCY+jzSV5AOE=; b=fqm19M3rouT7CfcKerg9Zq+eQGeDdmbeV1g2yK8up2zg2L29wfxKe6dkTao4WKncjpTS tJIjOORfTaXAImllmy1dogB0PjGsz9RoHDYKIILR+eJ03EcETg1TNA1IHjXoMxs/R+z/ MCJoFMeVJLkXE7MYZ+p7zeOu2PctQUTmwYw= Received: from mail.thefacebook.com ([199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 28bsvm0r1c-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 01 Feb 2017 21:20:22 -0800 Received: from NAM01-BN3-obe.outbound.protection.outlook.com (192.168.54.28) by o365-in.thefacebook.com (192.168.16.20) with Microsoft SMTP Server (TLS) id 14.3.294.0; Wed, 1 Feb 2017 21:20:21 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=HU8+bAY310tRx2EruXgedncSLr66J2ISCY+jzSV5AOE=; b=in76Cpp0yYMjfXLES2zRmE2qxicshn6qERzX2moCRDcnGz/0O0uAUcxFsAo4adGHpIhTk4QdLAtB5Gy5uPaQg8iSi40/tuzPWiHTLgZSr2V6MO4ehgRd4nQyPirKbMxZ0pChGgf2IRD8AlhZJxBwiK2bfXca1JtCavhvQxygOxM= Received: from [192.168.1.176] (66.29.164.166) by CY4PR15MB1192.namprd15.prod.outlook.com (10.172.177.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.860.13; Thu, 2 Feb 2017 05:20:03 +0000 Subject: Re: [PATCH 7/8] mq-deadline: add blk-mq adaptation of the deadline IO scheduler To: Paolo Valente References: <1481933536-12844-1-git-send-email-axboe@fb.com> <1481933536-12844-8-git-send-email-axboe@fb.com> <84DA10A3-5055-4D48-A990-6CB5A5D34F0C@linaro.org> CC: Jens Axboe , , Linux-Kernal , From: Jens Axboe Message-ID: <21ff7888-ec08-0dab-a997-872ad0027fe2@fb.com> Date: Wed, 1 Feb 2017 22:19:58 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: <84DA10A3-5055-4D48-A990-6CB5A5D34F0C@linaro.org> X-Originating-IP: [66.29.164.166] X-ClientProxiedBy: BLUPR0601CA0017.namprd06.prod.outlook.com (10.163.210.27) To CY4PR15MB1192.namprd15.prod.outlook.com (10.172.177.14) X-MS-Office365-Filtering-Correlation-Id: 78873f5f-542c-451b-e8aa-08d44b2b2541 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:CY4PR15MB1192; X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 3:nBTtcc93FVyzl7ZCb15cfYoyN6Bi4wDYMo34G66UsfU/qk6dZ5jHZ8DP20GLh6F7uQPkFW0Qkhy1/UXkctlO5arDjiUDviMXjcjvqbKNirby0uIBklPOFQZyQeIKB5b7Nzy+Bvkq69/4CLKDKCg9wguNitgB4CO6ytybKFYssBknV6xYS1ndi0B1t3X+ZKeTT7qeYOq+KU4myb0sYZR1Vh8sd/EZAg/T5FNuchhIdvcIhK+1NdO07yEdMrwyII5nLZ4gPDYCPdjxd3pjGnOCgw==; 25:OuEo5IwPu60USpg0j17RccU0m83QDeq6X5T+cSlYKlKpZxz2QRug851d+1+b3p5NFfD996X0D+J2RTh5CAWTNxjpGaTgxpB7OMoI5B3bhx7Egs4LXXM2XQE4mcZSDmd+NGwmvyecfwcWPDdg7FBHi+unJ4R0QpBFJfv52wyMn0QU17mrB45WDDqjTMyuePTgq889l+xIycbgqPKzCG1tPOYhHYpsuxoiHDYJnPl2hMLKkeZYUefB97QSpvqNbPmaSgu9s9OI6gL4IuA08l+5FdO9rtNQY+QbgN0d2bBBcJr2aUej4BtiHziDcmWP9dszDyX3oPr0ecMVD44Q95upUH0FxfWT0YiBzTxzw1kuNL+nWypdIlUGUYR36s5Lr3C8+gxxn3WZF+J5pjekK95+6pEkG8jKwVkee53UufP9I2VYgMnq2732pTRG4KJ0Q/RNJFJvg0iF4eBuDUzRWltr8w== X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 31:ek9TjZX9rF+wLCaCU50T3wy3tS6ZQIeZO8pIAtQTw93dvNWdVHacLaYmamj+bLdPja7GkkQ88oZnqfRtoV0jZ4/kjZtryGvKrjewXXZgdK/SP+1wZdTbbXVPsyKjeTzqaJPiWbaQ8UAR+kVm9mo/cc6eJNn3rB6UNk29IHNQDIk+JjlVElHIrvvQhOcF/f8nI8UGdd5QBiceZAc3pHAExz2n10wV609H7ckvvH2s7KUDSQT88fd+RjNO8a6+SDMiWEgpkgOc+n1duvgODQbelA==; 20:l3ZRTgU502E3HkYmKsVSk2zRvO5aAEsRIo6IJjwbqfIlTTzBUo5QjUfGmx/7I+sWxVz/V1JlJWlyOdbPaesqG5c3d74GFMCcWI+0NhvjnflmLKo53NYV0L7eWqfTH+8wcAk3HLlOy87K+xPhfJ/J11OYisej3iCsTIt2po+nC1BG9Ip2BmK3qtFnof87dqNHOBKQb0FqoVJrPG1wMaWw2n0ixtCSRJi/MG+u/ZgjWeCEeHOPxgKSERVsX76EWRCZRUacLXjGqlaWfnIQas385ceiVPU+UBzM35nZrO4vrz8uTtkRPkE8Qy2ZfoFYTUNFripJEORnKBAkiV8yZncCOpQKOJDwAlCo0eJ6XNWXld1f75NUdEbGtO7MShBlap8teXsnb5TFpLbuRU+BiF257XqHVyJkI9YLFwce03/d0OCNu2tnQe/yIiToiNkxNgSRMAoJOtRrs3K2a2U0Knl+RpUNSRFAAqtj+Hh5WmeqyJ9CPhMq0fnI2HRlh8uGx3Hc X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040375)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6041248)(20161123555025)(20161123562025)(20161123560025)(20161123564025)(20161123558025)(6072148); SRVR:CY4PR15MB1192; BCL:0; PCL:0; RULEID:; SRVR:CY4PR15MB1192; X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 4:JXetK2BZvLeo2kSK+XpN7hxZm9IJ0aJsYs8KhUUduYZUbMlBsYde7MUCyIc4k8S1ZV93VwxVyujbATiIdOstJtrymAZYcdyPcVBQDaGVaPwbeD3Y0xnaND9uevH4EJ+XMMWZohc6OFBOEz7Yldz1RT2GOtZrXdxqXzFCP4eRsMhmGap8kMXPwx8TApAMwkVRNxAe5+00IOasRtYEjwUvGkmB/uY8fSlzfuo0b55QX8azWiPl1HH9shJsC843oIkxEUO08p2E2MEECeXSwjY78MQ13QpXRBRDsFc/4w2uYXraw8FRsFTFkbf6v/giwaT5rYxEVmmInLGC6dti/iSFFzn7YzHP7AOrBf81LezH+NDoiTHpU81ZBvAyiaPRLTWHpsvC00UK/4B1DH2d3GKIPnU2qrmqpYP/jiwo0r2mG6kRCnjIXeHzFLQImi9bW25CfwJspXkKJzx3t7FuRf2nw/JwA2OdyHqH4Jn/c9gXaeNp0cEO392ZiMxRVr4s5Ccv1aaiNFsEcYkFiyycfSapHxjSEfsrgP4g4X97yfULzjY/fpcCDqi/FZ+X+AbUNJ0eUVmnbCHbXCK8H8NwN/TeCsMpNm97gvroqXPgHtx4plg= X-Forefront-PRVS: 02065A9E77 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(4630300001)(6049001)(6009001)(7916002)(39450400003)(24454002)(199003)(377454003)(189002)(23746002)(230700001)(25786008)(6486002)(90366009)(4326007)(54906002)(77096006)(2906002)(38730400001)(229853002)(105586002)(106356001)(5660300001)(101416001)(31686004)(42186005)(8676002)(81156014)(81166006)(117156001)(33646002)(305945005)(7736002)(53936002)(68736007)(36756003)(50466002)(64126003)(92566002)(47776003)(65806001)(66066001)(189998001)(97736004)(4001350100001)(65956001)(31696002)(83506001)(6116002)(110136003)(50986999)(6666003)(6916009)(2950100002)(65826007)(86362001)(54356999)(3846002)(76176999); DIR:OUT; SFP:1102; SCL:1; SRVR:CY4PR15MB1192; H:[192.168.1.176]; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: fb.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1; CY4PR15MB1192; 23:yooHK50qfvZoK/a2k9jEZf39Pn84K0EC6nMjw?= =?Windows-1252?Q?FqzLPDaHWFQRxJbqp5Z5SkQOu/5yYf5V6B4FMMzMKeWMbkcNt8VwDZi8?= =?Windows-1252?Q?6Ikf4Kkol4ws0NYHPckyOSaJDq+0vwn06mKS7ue5A3nTued2UK27fab5?= =?Windows-1252?Q?uJxkCbdPGYufIWN8ZF1LQE/NUtfywTNiMwWOiFFJV3lLiqu5jz2Czy2a?= =?Windows-1252?Q?Sh6QgqaSK5dojpOfgkmjcI16eUouapvHfR7tm1XUjnDdFDQfJbtefypP?= =?Windows-1252?Q?5IeDoZ41aHeKT1sOoXXuKPPdn/gdQLYL7vEaYAA7XJjqC0R5oJUVIoyV?= =?Windows-1252?Q?zD8ZLrHyAnfHXzKgsZ6D8mZqVX3F7vGPtgKBikJgRwIRkQcZsTfG09kw?= =?Windows-1252?Q?7NXzbmtFiKpN19Uwi3tUfys2PTjjgG7Njkeckg+TPS20kk3IAYCC/Yup?= =?Windows-1252?Q?cTIwF6LQOUGPyIOT3acKUDha6FnZC1zQe/to+ZNj/29KtsmnNCNeFgca?= =?Windows-1252?Q?TSHvsva653XDSo0LVcargJMwZKxPbIAUsSIwM/r1WC1FrM2FyUP9BS2k?= =?Windows-1252?Q?PSmmokWo3a8AOdECqChqWtLAmPZviwFq7w52r+RTshuIbI3BcCQ/p7Qz?= =?Windows-1252?Q?24FRTR4CaTo3HCpkMn+F9bVCn/KA+ajIHjIZ6XXV3iOA3eoOh8fHhwHn?= =?Windows-1252?Q?5nHuG51GZpIK43sFtNV4uNKpj1UAlngvwG2WMbUfaFWOAzXlO+mFbMvV?= =?Windows-1252?Q?NeGrgcXlIMV3PnRkCRkwDu484NIAap7MA2jg34YZfjpSC1Bdpw/+rZSY?= =?Windows-1252?Q?xwy0bjKMl2L0IedFtCL3dfX4KrLB3I1VNijuYbeT2o2i2QD4pgiUd66T?= =?Windows-1252?Q?XohimeqdOKvE0JDe2cfQVVDc0tDc+e1h6v+xwQHGwuz/MLvpTz8LeV8W?= =?Windows-1252?Q?k4+IyNzzA5BFENskgZp+nTKj9iv+uS4joW8WO8Y3JXDneozk/bN/VZtw?= =?Windows-1252?Q?m5gG0cFGs4f46YlXVkwDbbJEOiWkZ3yhiUttPyq3CFHSQQZt3ktxIFWq?= =?Windows-1252?Q?yVGf79l51FWhWuSuPXzY4QkTIpQ1BCurpy/wfpXPxQg5WejPiAh1SIfT?= =?Windows-1252?Q?PiMZEbL0V85RGOSeGxjzPJsWgLOwY63GaS3G/2gMF3+54xk23vE/ahuP?= =?Windows-1252?Q?/nufrQrvTKihIZQXEnI1PBaLy9ipqiEiz2wZ88S31zKb+dJyccJ7KAfF?= =?Windows-1252?Q?OjKnUDCS9ceD9NJ+bTo/H2+HVXbuLyemIWJprqyzKyErY/juUWoKtT5L?= =?Windows-1252?Q?tR9luQCcKowmNAuAfLTjm5SOjkSgLgWMnde5tBeZ0RY8uL+Az/8ivYRC?= =?Windows-1252?Q?tBS0MEWchAP+ap+/zWshO4lUauyhzUPZaGGKL3q5yq997N0O6JKgB/fN?= =?Windows-1252?Q?1ahQTlDkOWplC7ywOTWP7JYZJP19WsGq1QOJF88QqAecxwB73lxf5zXN?= =?Windows-1252?Q?8cq+xW+NI86h98NDGN4rUjVJcFC?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 6:BE7NqYDDPhuHCwWFolKpnIJuutLfiUtKrxCjSJum7YpjM8oynIUeJGRPTyDoIpQ2K/5MlwJssJSemFjqUlPNr/5nsLgMCgWKZQPya8/do+mVyJTrBNCu0LrmLWe6R7ZrAWz6+owpuD6fIDplAXmyoJHkkKWpQaf7O0/F1PBuvkUYrqRwIGg7EhhL+i5Xycc8iyzbcdiKA6qRFI3Hwcw1PRCzx8aPN8sF5h8eTsQCj7XQOg/Zim4OyYjto9dHnoGeDU1qCXS8CtMcrhFQQksNAbirSIT6vEWWlILu4MDH8BugUczpfbj4pWuKkk84V76YABAEbVVtZ39etxzpeAtPiqYqXxsgaQBvQc+7YUP90O9ziHeS7PQfyRjOGg7pvqHF74B6hXOtNhXPBKtyVEdP6hjU0bFyooilY5vBUz9LNpo=; 5:Hv61vgS4uGQetSooC3sSbJdO4EeDHx/wWFsTsMoqBNd6tM5WKFrBUvg902rv1UDxClFB/Rf56lu+Bl+tMEHWdNxQx/OqfQtjzUigy9gi1ROg0oX7j5JfUt5s01dqEvsA1VGaziNrR04yGG4pRFDmTg==; 24:K9wmZMDAz+sTb5Xk6Y6MsdFuGsjxMDOEFfTWBzihsq15jPkzdRev38txGQl3L7M6lMZb4F98QOLyZqp78znaMNmp7Tofb7iHDsxYcZbcUxQ= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 7:etrlyorcukBpBxQLTtSNlZWX58LZhNnMxpLetMgeP/yQCJ8bheqpShyxPeM+GdmhUBO6jmsY3xculsZrCGDGzrrIw7A0MlixLkwIFkVTOmUjxbhXL/iDskSHORfCxGvvAWRkfRfoX5YsLsopqevqek1tUEHgJlHiQVhdU66xmMsbbjmBD67Bmir6r1xbd/dxWOsDOtaVlYt4uiAl6QfcKsld/uzNe4RE4Tgi0sNQcBjCCYh1srQnpIvKQtc6Kbuex7I3YCVawv8n8hqQqEaq3ozOmWvO3zjv0Asxha6gOeuZJtKHeg2BI692z1otrM+edKTb/+a5xDIpLeEFoBPCw81SnDFiKhNk1jcYfm3Bvz2h2l9M/TIdRRufv0U6SHRUUGt0fyoolCBqaVBLuZkV4m/pAmCqDKgNi7pyPnizZPfPAYLyJRCO5Ob8nGCRAz3yEdCFBOAqPodilwIN2LZHmI7YAjgIl7mx29s+uF4e5rbRCWNApw+sEIqjmk5N0r2DFL2nEUHvcBS1GXBeVFcBNiXbJ2KIUHoKkxOBJhZemOpOavNQou/qER9RfW+jPDq/; 20:rcrhBHwabyS0HLqqdnQOMOgCIbqGYFm2AlcMBBg+Ypot/h+wkyUIxKH3crtELOAf2BjvHG/cEn7I5UCc0bAnwd6ShFSNXahxhoDLI7bKfGWTRy2k26eE6ffxKrdy0NcNYpS8nowXcLoarVkwldVDztG9NQWK5CIGcx5fgpHDDhY= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2017 05:20:03.5636 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR15MB1192 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-02-02_03:, , signatures=0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 02/01/2017 04:11 AM, Paolo Valente wrote: >> +static bool dd_bio_merge(struct blk_mq_hw_ctx *hctx, struct bio *bio) >> +{ >> + struct request_queue *q = hctx->queue; >> + struct deadline_data *dd = q->elevator->elevator_data; >> + int ret; >> + >> + spin_lock(&dd->lock); >> + ret = blk_mq_sched_try_merge(q, bio); >> + spin_unlock(&dd->lock); >> + > > Hi Jens, > first, good news, bfq is passing my first sanity checks. Still, I > need a little more help for the following issue. There is a case that > would be impossible to handle without modifying code outside bfq. But > so far such a case never occurred, and I hope that it can never occur. > I'll try to briefly list all relevant details on this concern of mine, > so that you can quickly confirm my hope, or highlight where or what I > am missing. Remember my earlier advice - it's not a problem to change anything in the core, in fact I would be surprised if you did not need to. My foresight isn't THAT good! It's much better to fix up an inconsistency there, rather than work around it in the consumer of that API. > First, as done above for mq-deadline, invoking blk_mq_sched_try_merge > with the scheduler lock held is of course necessary (for example, to > protect q->last_merge). This may lead to put_rq_private invoked > with the lock held, in case of successful merge. Right, or some other lock with the same scope, as per my other email. > As a consequence, put_rq_private may be invoked: > (1) in IRQ context, no scheduler lock held, because of a completion: > can be handled by deferring work and lock grabbing, because the > completed request is not queued in the scheduler any more; > (2) in process context, scheduler lock held, because of the above > successful merge: must be handled immediately, for consistency, > because the request is still queued in the scheduler; > (3) in process context, no scheduler lock held, for some other reason: > some path apparently may lead to this case, although I've never seen > it to happen. Immediate handling, and hence locking, may be needed, > depending on whether the request is still queued in the scheduler. > > So, my main question is: is case (3) actually impossible? Should it > be possible, I guess we would have a problem, because of the > different lock state with respect to (2). I agree, there's some inconsistency there, if you potentially need to grab the lock in your put_rq_private handler. The problem case is #2, when we have the merge. I would probably suggest that the best way to handle that is to pass back the dropped request so we can put it outside of holding the lock. Let me see if I can come up with a good solution for this. We have to be consistent in how we invoke the scheduler functions, we can't have hooks that are called in unknown lock states. I also don't want you to have to add defer work handling in that kind of path, that will impact your performance and overhead. > Finally, I hope that it is certainly impossible to have a case (4): in > IRQ context, no lock held, but with the request in the scheduler. That should not be possible. Edit: since I'm on a flight and email won't send, I had a few minutes to hack this up. Totally untested, but something like the below should do it. Not super pretty... I'll play with this a bit more tomorrow. diff --git a/block/blk-core.c b/block/blk-core.c index c142de090c41..530a9a3f60c9 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1609,7 +1609,7 @@ static blk_qc_t blk_queue_bio(struct request_queue *q, struct bio *bio) { struct blk_plug *plug; int el_ret, where = ELEVATOR_INSERT_SORT; - struct request *req; + struct request *req, *free; unsigned int request_count = 0; unsigned int wb_acct; @@ -1650,15 +1650,21 @@ static blk_qc_t blk_queue_bio(struct request_queue *q, struct bio *bio) if (el_ret == ELEVATOR_BACK_MERGE) { if (bio_attempt_back_merge(q, req, bio)) { elv_bio_merged(q, req, bio); - if (!attempt_back_merge(q, req)) + free = attempt_back_merge(q, req); + if (!free) elv_merged_request(q, req, el_ret); + else + __blk_put_request(q, free); goto out_unlock; } } else if (el_ret == ELEVATOR_FRONT_MERGE) { if (bio_attempt_front_merge(q, req, bio)) { elv_bio_merged(q, req, bio); - if (!attempt_front_merge(q, req)) + free = attempt_front_merge(q, req); + if (!free) elv_merged_request(q, req, el_ret); + else + __blk_put_request(q, free); goto out_unlock; } } diff --git a/block/blk-merge.c b/block/blk-merge.c index 6aa43dec5af4..011b1c6e3cb4 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -661,29 +661,29 @@ static void blk_account_io_merge(struct request *req) /* * Has to be called with the request spinlock acquired */ -static int attempt_merge(struct request_queue *q, struct request *req, - struct request *next) +static struct request *attempt_merge(struct request_queue *q, + struct request *req, struct request *next) { if (!rq_mergeable(req) || !rq_mergeable(next)) - return 0; + return NULL; if (req_op(req) != req_op(next)) - return 0; + return NULL; /* * not contiguous */ if (blk_rq_pos(req) + blk_rq_sectors(req) != blk_rq_pos(next)) - return 0; + return NULL; if (rq_data_dir(req) != rq_data_dir(next) || req->rq_disk != next->rq_disk || req_no_special_merge(next)) - return 0; + return NULL; if (req_op(req) == REQ_OP_WRITE_SAME && !blk_write_same_mergeable(req->bio, next->bio)) - return 0; + return NULL; /* * If we are allowed to merge, then append bio list @@ -692,7 +692,7 @@ static int attempt_merge(struct request_queue *q, struct request *req, * counts here. */ if (!ll_merge_requests_fn(q, req, next)) - return 0; + return NULL; /* * If failfast settings disagree or any of the two is already @@ -732,30 +732,32 @@ static int attempt_merge(struct request_queue *q, struct request *req, if (blk_rq_cpu_valid(next)) req->cpu = next->cpu; - /* owner-ship of bio passed from next to req */ + /* + * owner-ship of bio passed from next to req, return 'next' for + * the caller to free + */ next->bio = NULL; - __blk_put_request(q, next); - return 1; + return next; } -int attempt_back_merge(struct request_queue *q, struct request *rq) +struct request *attempt_back_merge(struct request_queue *q, struct request *rq) { struct request *next = elv_latter_request(q, rq); if (next) return attempt_merge(q, rq, next); - return 0; + return NULL; } -int attempt_front_merge(struct request_queue *q, struct request *rq) +struct request *attempt_front_merge(struct request_queue *q, struct request *rq) { struct request *prev = elv_former_request(q, rq); if (prev) return attempt_merge(q, prev, rq); - return 0; + return NULL; } int blk_attempt_req_merge(struct request_queue *q, struct request *rq, @@ -767,7 +769,12 @@ int blk_attempt_req_merge(struct request_queue *q, struct request *rq, if (!e->type->ops.sq.elevator_allow_rq_merge_fn(q, rq, next)) return 0; - return attempt_merge(q, rq, next); + if (attempt_merge(q, rq, next)) { + __blk_put_request(q, next); + return 1; + } + + return 0; } bool blk_rq_merge_ok(struct request *rq, struct bio *bio) diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index 114814ec3d49..d93b56d53c4e 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -234,7 +234,8 @@ void blk_mq_sched_move_to_dispatch(struct blk_mq_hw_ctx *hctx, } EXPORT_SYMBOL_GPL(blk_mq_sched_move_to_dispatch); -bool blk_mq_sched_try_merge(struct request_queue *q, struct bio *bio) +bool blk_mq_sched_try_merge(struct request_queue *q, struct bio *bio, + struct request **merged_request) { struct request *rq; int ret; @@ -244,7 +245,8 @@ bool blk_mq_sched_try_merge(struct request_queue *q, struct bio *bio) if (!blk_mq_sched_allow_merge(q, rq, bio)) return false; if (bio_attempt_back_merge(q, rq, bio)) { - if (!attempt_back_merge(q, rq)) + *merged_request = attempt_back_merge(q, rq); + if (!*merged_request) elv_merged_request(q, rq, ret); return true; } @@ -252,7 +254,8 @@ bool blk_mq_sched_try_merge(struct request_queue *q, struct bio *bio) if (!blk_mq_sched_allow_merge(q, rq, bio)) return false; if (bio_attempt_front_merge(q, rq, bio)) { - if (!attempt_front_merge(q, rq)) + *merged_request = attempt_front_merge(q, rq); + if (!*merged_request) elv_merged_request(q, rq, ret); return true; } diff --git a/block/blk-mq-sched.h b/block/blk-mq-sched.h index 9478aaeb48c5..3643686a54b8 100644 --- a/block/blk-mq-sched.h +++ b/block/blk-mq-sched.h @@ -16,7 +16,8 @@ void blk_mq_sched_put_request(struct request *rq); void blk_mq_sched_request_inserted(struct request *rq); bool blk_mq_sched_bypass_insert(struct blk_mq_hw_ctx *hctx, struct request *rq); -bool blk_mq_sched_try_merge(struct request_queue *q, struct bio *bio); +bool blk_mq_sched_try_merge(struct request_queue *q, struct bio *bio, + struct request **merged_request); bool __blk_mq_sched_bio_merge(struct request_queue *q, struct bio *bio); bool blk_mq_sched_try_insert_merge(struct request_queue *q, struct request *rq); void blk_mq_sched_restart_queues(struct blk_mq_hw_ctx *hctx); diff --git a/block/blk.h b/block/blk.h index c1bd4bf9e645..918cea38d51e 100644 --- a/block/blk.h +++ b/block/blk.h @@ -204,8 +204,8 @@ int ll_back_merge_fn(struct request_queue *q, struct request *req, struct bio *bio); int ll_front_merge_fn(struct request_queue *q, struct request *req, struct bio *bio); -int attempt_back_merge(struct request_queue *q, struct request *rq); -int attempt_front_merge(struct request_queue *q, struct request *rq); +struct request *attempt_back_merge(struct request_queue *q, struct request *rq); +struct request *attempt_front_merge(struct request_queue *q, struct request *rq); int blk_attempt_req_merge(struct request_queue *q, struct request *rq, struct request *next); void blk_recalc_rq_segments(struct request *rq); diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 49583536698c..682fa64f55ff 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -371,12 +371,16 @@ static bool dd_bio_merge(struct blk_mq_hw_ctx *hctx, struct bio *bio) { struct request_queue *q = hctx->queue; struct deadline_data *dd = q->elevator->elevator_data; - int ret; + struct request *free = NULL; + bool ret; spin_lock(&dd->lock); - ret = blk_mq_sched_try_merge(q, bio); + ret = blk_mq_sched_try_merge(q, bio, &free); spin_unlock(&dd->lock); + if (free) + blk_mq_free_request(free); + return ret; }