From patchwork Fri Jan 27 08:04:06 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 9540881 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9B28860415 for ; Fri, 27 Jan 2017 09:36:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8CD98271BC for ; Fri, 27 Jan 2017 09:36:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8166020649; Fri, 27 Jan 2017 09:36:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DAC327FBB for ; Fri, 27 Jan 2017 09:36:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932369AbdA0Je7 (ORCPT ); Fri, 27 Jan 2017 04:34:59 -0500 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:55245 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932344AbdA0JeG (ORCPT ); Fri, 27 Jan 2017 04:34:06 -0500 Received: from pps.filterd (m0044008.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v0R7wlVv010114; Fri, 27 Jan 2017 00:04:14 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=subject : to : references : cc : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=facebook; bh=m4UICxx+YLSmVVxD0GZNmr3eVsFF6ieNRbvk1zp1PfE=; b=ChabYfk4TGFbZEdh/kTpICFXYeShiQKsF4d1YJx6Gc8O/5/1vp7hrRvsTWX7omjYGwer 3NGUR7cqUIPssxzWdWp6fxK3QPPCFu8s1xstR7LCK4MFBTcOotqX3lqG4/3MXlyqw0fx F3APWBU8AkJ/G0c6DHkUrssMg2qAUTDE5pk= Received: from mail.thefacebook.com ([199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 287w98rrem-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 27 Jan 2017 00:04:14 -0800 Received: from NAM03-DM3-obe.outbound.protection.outlook.com (192.168.54.28) by o365-in.thefacebook.com (192.168.16.24) with Microsoft SMTP Server (TLS) id 14.3.294.0; Fri, 27 Jan 2017 00:04:13 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=m4UICxx+YLSmVVxD0GZNmr3eVsFF6ieNRbvk1zp1PfE=; b=UApzDY29KpG6FnCKv2ujoLSw+2QQJWkvpaRcURQJwGbL+lfN2S8kXYbNSCyG9Y0dtscXj06t61btYbWpZYP/dS8bGdbv3oBX4r2Jjp6jCdW45QmTUY1Tv1AeJg01Jd6QhLggqb+XgA3amldJOFWm3T9+09tf1B5TpVPYuivy3s0= Received: from [192.168.1.176] (66.29.164.166) by CY4PR15MB1192.namprd15.prod.outlook.com (10.172.177.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.860.13; Fri, 27 Jan 2017 08:04:10 +0000 Subject: Re: [dm-devel] split scsi passthrough fields out of struct request V2 To: Bart Van Assche , "hch@lst.de" References: <1485365126-23210-1-git-send-email-hch@lst.de> <1485455329.2540.7.camel@sandisk.com> <1485456745.2540.9.camel@sandisk.com> <20170126185924.GA25289@lst.de> <71e22257-0592-fdd3-25e5-a78ceced2ab9@sandisk.com> <4054e944-b28d-1cd6-574f-6cd90e28c301@fb.com> <1485464486.2540.12.camel@sandisk.com> <6995c991-65a4-8dca-c36e-fb2eff277ca9@fb.com> <1485467235.2540.14.camel@sandisk.com> <1485472465.2540.19.camel@sandisk.com> <1485474426.2540.25.camel@sandisk.com> <1485477510.2540.27.camel@sandisk.com> <2d971693-b79d-c1b9-fb2a-f5dd04128c68@fb.com> <1485479738.2540.30.camel@sandisk.com> <37ab009a-bc2d-d2ae-a875-269ab563a430@fb.com> CC: "linux-scsi@vger.kernel.org" , "linux-raid@vger.kernel.org" , "dm-devel@redhat.com" , "linux-block@vger.kernel.org" , "snitzer@redhat.com" , "j-nomura@ce.jp.nec.com" From: Jens Axboe Message-ID: <9cbf0ce5-ed79-0252-fd2d-34bebaafffa3@fb.com> Date: Fri, 27 Jan 2017 01:04:06 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <37ab009a-bc2d-d2ae-a875-269ab563a430@fb.com> X-Originating-IP: [66.29.164.166] X-ClientProxiedBy: DM3PR14CA0120.namprd14.prod.outlook.com (10.166.157.46) To CY4PR15MB1192.namprd15.prod.outlook.com (10.172.177.14) X-MS-Office365-Filtering-Correlation-Id: 5bd3c1da-edb7-4e7c-eb8a-08d4468b13a2 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:CY4PR15MB1192; X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 3:BhGjUo745xXNUTg7uh3yVsdRGGyyL4+Ed2uSJuSAHyhB3wZjOVBao9uecHxLYBfneAqDMnhVKDyR5Uh/IU8NJCKOgHhguP70GIZT5cJYzyF/g5HwraLmg7gYqBzNa7Jm2STxitSr9ln8RDs5zKK4ubs+0XknEgA8jrqJH4/HOcbMp8y3RLsMQzYEyj4/9Cn5kPN3EyCdttMJE+ruQ9c+PcVCG3VCHnV4jSkschVVoQZ3L1QSSCpwg9MOpuSOShvQd7KX3L98UgVjqAWHlRa8RQ== X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 25:dXEKh3lW2Ppxujmq6AemQpW2Xls0dHjktGz8jMkzjYvfSFTiJvJWbZjfjLfNlr0iwCRjzUVhQXfDDwzkHYSJ89HrsKvrzi+wA8yeTMeN7SbeDOju5R157MyFSwWe1HqBNJwNaGTFSCai6VtOxGaP7ZdZelFQclrvBDzDD78DKe8MyctoAkzOMZCLU2pUeTuNhcoPSWyaRF5jzG1jSsir2v4eZVyMOKVnfBWRgul6J8QDmS/XdPALxUZ0TPt2MUXwpWjLowwXQrQpN7xxo8BGRQg//KPEBiQeCVs77LxZ5Dn8vnprwGztYneG/a0RdRrcLZjRPQJZykFD1KxWQzPeEcRAcFOHeg6euOIY4HDomplImq80buX9oJApILBo5JIebynEVxMYdOKgGUTkujCmWtYL0EewT7ULgfINCT5ONu6+KipHhzLITc57cVkVnOxNqzpahNGQOfOGKJozhL3hm2qAkiFa6++K1wuGsSUsRbKIzfB9DG4v8RMck2gLbXc9I8chycnOhDbbWiVJUOqTNbm5VDLgqYsT997yRxNdoWu90RDJboSXB9U1+1DUzJVEYcSJB4oPZdKuwKetceYKsXsELMSDQlwc95+AxZ03L2v7uQK9a+jZeF8qL+QuLQk18JwF5YmeGSbftya+cz7CS7+/azaORfiGiA1SCxKo/F+X3UiN2zdLngDpEuRUGvmyolEkHPUtKppa0FUbryCP0WEbzkuVAWl1JehzIUja/IU= X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 31:bh+c6pB9sV2QDc33xplooDGnc+cFO3P/PU+3kkyUpI66Y3U52LUqU+WPextYFZKtrXtB3I04YDsMh3jWmL75n6Y0RGnUtJeIv4On9xU7HW/ZxKuIKUJHGhpNfSIRgTW+k3fLZRxRZF3l/Hul7VH4X2Qy9UlBSQbENBD22U3NL3KhRBbaDy/0k3Lj/43jB9ulWUQG/2Bvat6uinxv95VLzHLIosYuN5HLVYncPSD9OfrQZrSIRH3nCh19cAuMX39iyc1rxqy8jMQQ7cPLXUkPOw==; 20:Stv2Mzc2u7YqVyxX3ZTlwNNRVRQqBTpafCsXeMbuEOCgMJy6Bq4fE3M8sJ63LlHDOBqccDbkf7hrCSNrUI8bmpgA3kEhuwRoJ+uqtyVb/lo09uYwkuxMccaSU5z/S7qE+40mMM1dURyjW+FIKy+I75Ow3dzd/PmFVfQyRFrtZgRsOnDZbEoS73AIVz99ENgB5mK4w8vX6tjP+CGwUdnvflATjCpdeQObtxynDO07hQsLEBc2M4zusU4vfwSsaHZqYw0FFzD7jT7rkDJ3BUcsSf4AlZLYKNOKbi/jpK59N0kmda06Osub3yurXW2ZJJNtFZks/aIZM0C63snXtZVrIUfM9xfzA41ulUw5Ieow1Ps7OqogVwTs7tvaSnqUYUwoqu3YfKNMWyDfCrZKdULw39MeCxiKZQt05gl+U9kMbZt/2Q/hQwzPBjHRWLgZUI/oBWi7f3js9jzFux1KwMZgpoMDxALBA9AypUhhwiV8DmW9nox30nAtvSEBBkZm+7dP X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(67672495146484); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040375)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6041248)(20161123564025)(20161123562025)(20161123560025)(20161123555025)(6072148)(6042181); SRVR:CY4PR15MB1192; BCL:0; PCL:0; RULEID:; SRVR:CY4PR15MB1192; X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 4:Nx8PJSCfBp6iT6n7mXJR6Ygg3EK+wUy54QZ9ZX0RxPzLxnSWiA8mE/sjaIgns3wtm9Qwig3T+UnaRNM75fiOrgxCQRo1fWslm8lpb7lk/ayYwMOjdghrqE2lVj5xdaoLvSXpcb834gayMVlx6FxPzvEC158uofTicr+8YTqD/co3mhdZQ6PDfXXNOYdX7hgYmANnt2CyEuCMqBKu8MDVFtZp58JH9S5VSwMBPC/6cKb4vFiz39AZDwTXIJOMMrp/s8Wr9clGbdHvVUcgmzWcMMbMppQF/5LZc5Xeiv6t1k/uIUwOcaywTgfAvii+W1eenkJPjRQbrGy6eJpHMyJJmWbHMdxXc5dLvmJghTtE/x4t9Z/diJFv09cx1uXP+bxkbbhYC78jjFuVhDOHJHz8/ZvhM6qDjQ28JGUkHUsHRF66YXTfoc7m4M0h9pi+z+S1/DrasqAqQmgbuSOBmW1s4b5W3kfe5JzMaljdy92k8XBx/FQWrSJRxdm48BA/ghztRLeCbfYTYS27q+K7TmahXmccxyxpiJ8Mc3fySjFOkgkAH9r/lPDnDQMZd08kM5WFAfel21aYpnzieiLQfYbtK274zQdHUMDr8/fQylAqwAeO7bQHyQv/XbXKUnbKgGMrLq181cp06a/dMtumV+IUVg== X-Forefront-PRVS: 0200DDA8BE X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(4630300001)(6009001)(6049001)(7916002)(24454002)(199003)(189002)(377454003)(377424004)(51234002)(230700001)(101416001)(2906002)(4326007)(42186005)(229853002)(31686004)(305945005)(38730400001)(47776003)(50986999)(66066001)(7736002)(65956001)(105586002)(65806001)(76176999)(6666003)(92566002)(2950100002)(8676002)(81166006)(3846002)(6116002)(81156014)(33646002)(93886004)(23746002)(117156001)(189998001)(54356999)(106356001)(2501003)(97736004)(5660300001)(5001770100001)(4001350100001)(25786008)(575784001)(65826007)(86362001)(54906002)(31696002)(64126003)(8656002)(36756003)(6486002)(77096006)(90366009)(1691005)(83506001)(53936002)(50466002)(68736007); DIR:OUT; SFP:1102; SCL:1; SRVR:CY4PR15MB1192; H:[192.168.1.176]; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: fb.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1; CY4PR15MB1192; 23:pRkzSFnZLu2f0a7BJFWZyoxZWdD0IRRgy0SSz?= =?Windows-1252?Q?BdkbsMm4ZyGx9rX6Awgsi2yn43khxrRrFqY01rPhNiN97yFMTO7Mvna1?= =?Windows-1252?Q?ZdDLoSRvzW9StTxJe2ggkvfNx6lCfM3mpFJVZREcfVF1wyT0J8clgRHU?= =?Windows-1252?Q?l1ZoFzj5tqf1/w5N/wq5HDVuWNgh6C4PBvSCTaZsDOeaqualHcB8aIul?= =?Windows-1252?Q?48u8vr7IBLfTcxEpEMfsLPzIwZgAJHNNH6onyKxQfLKT1nINPSNRlNoH?= =?Windows-1252?Q?YkkH67swORNQEunwySXgRQyAa8VzVpaWcFrzY7ztI9uB9S7r1q1c/xRs?= =?Windows-1252?Q?1FnP26FafpEaZP3YyxOQXBoRMkBFHPJxG4ZCBR7Fg4ogyjevxmJrtzAh?= =?Windows-1252?Q?CId+2n0e/JvO1q/Z/lPUQpN46qI3nOTrjHTMha/Jm8AnJnqM62WSXFgb?= =?Windows-1252?Q?RRN0P/xZrVN0V3JVF6oGCAaeyt10n7KAbDMIj82cd13IY987THJ+WpLY?= =?Windows-1252?Q?JSdfjJx32xwq3nmgfjrwBHSXy8HVHeeMqR6gjEW6N0wYTyiDhD+us9DK?= =?Windows-1252?Q?OCwlZIY7d+zCIrEuBB+RD0UQ15f6KlXHRRGSdo0+agurxZy0IUO22fZF?= =?Windows-1252?Q?FdtulFVGlS3K7bUW6+V6KlIjyQfZ9w3NS3Fehx9uMMriIY6mgxXMfLsy?= =?Windows-1252?Q?mjAjcCJUHXkNTQ+CG8ha/5CR31UjvXEndJitCxRU0f2gN6MrDDr3Eu3q?= =?Windows-1252?Q?ESwYPu1SqMLk1pibGBdfRo0cAJJqsfnCD9ndYpSBF0zxKeIUFm6womIO?= =?Windows-1252?Q?vRjegwTnzFdYc5TjSJ9f8k9wWxe24AlzGTTFgyxWf2DhOCzfRvzbeErV?= =?Windows-1252?Q?rRKGWxY9N/d+2SOh4PJK4Kz+/9Zh4xCPmI7eiYnTW8l5kpsUTyjHBiyY?= =?Windows-1252?Q?t/L8eKUTsF0btb545KC4DKQFK27fZzyOqE3jNufdxoTR4FCviqwomKRw?= =?Windows-1252?Q?a3jqTy682QA3cvNwOi1TsQ0p8JSYgielOjYT6/mjtvkMfrmow0A9qYZG?= =?Windows-1252?Q?wY7LKeWWlIUywQj7nmVib10ghv7BGsL45o28MK3xbYUMTNNG/qEj7K2M?= =?Windows-1252?Q?KTc9tmveOjrMg1Xjq6/j5rw6cX4mPzAsTW0HPCQ5dwBSjxBVhAk1y0IS?= =?Windows-1252?Q?utfkz91c2+w7Yq0dx6Hqie6TcEjL5NwcMEszgpCseOYs7XmT/AYzgIE+?= =?Windows-1252?Q?E6dlwbEtrYaGO00n2FvrOySryLFCwUA/xW4cBdxOEFkX9/uRh8WvuR5s?= =?Windows-1252?Q?CY5iezWq6cbx/DbXgRSVFX02/v54Ki6cKgTtZgicE4tpAJTB+iGbeB/G?= =?Windows-1252?Q?D+ibLIAog0agB2SWK9a9e0NjbPKdgxx9M3DYDzoRNqBTY9P8jPga0hCn?= =?Windows-1252?Q?G+RKePRHMiK/IdRqA20OiAuSItKSH+FDgYYf1+YaKu10p88Na+wORj8F?= =?Windows-1252?Q?HACj4+MZ1hCMwMcTR3/p1lp0HTe355x1DWy50aQByaH6Zu1eC03qAx8A?= =?Windows-1252?Q?uFNmNUtrDZ8G2o0ofPEgIaP9UHzf9N+35Neg/GnAWgm1W0ngsnCLUali?= =?Windows-1252?Q?qAVF493Efp8EYgxes8dkLwm+f2rvmperL8oMoMPvUU3?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 6:VQp7SPR4HLIuncuIhPb4m/0ovRCPFg2rXjbauIuP2H3jxzVjihuR/6mAjy20ccveJGUHXzp8McxYQRBLP55SWefZhQ5ME/QWuNOzbX3TmhHiUP3s7PTaRFEPNgRTZvG4QZ4MTRfj4h8oipnNqOSk8XL2tddB8H5lPir/ZU66C8KrBeYjHXWLJS3xyLhxj+AiQS6UE8JREu5oFLlx6tzkRbXdjnOkeJMOl6GvliJjzwKbew6DjEIaAJ/NPcucliYynQ26mD0Qb4K8CxaWJyR6NC0Vj3+SSnJpM53+DThNakSlNKCNmxmqeNQHneLWt/xtWlbOVVcP/49MjHYmQU8ytiQJC2Xayw2WswNOLJ9G2HxEua+Jpbp2nmkiuTpmfXtN6VgmtWkBMawvVsaXbj8OXd0x5UWLXh8PxyldSqGunVE=; 5:m3eiSlMER352zpiYRaHgf8k/v+ptVJjWJPx/M1wo0IbUJriD83HUDHR2sPrJbJ9IqrBCo612mlSGKJyHl7h5Eo9VfPZfPk4hEbn4tCsLUaNX04anIWIgc8mY0s3o9Xx3LLuUfCrgf/5kOgRcIL216A==; 24:qdaoAmXWPLr+KqGxr+I9RDDqdfId9vrbhxVKgh+MDk915bLFDKC9IsyBoTKACMCZ5AxH2fw0z680WdSlm/oWIUt62EtUMYN7ypZ6N90mlOQ= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1192; 7:9Eo3zkvzp/rYkRfDb1ZFvo120/cbmI2QVx9oZLgdiQYBueqCDC3Zvvu7skhVh2PyF325BYZqv7awuPW9dfgn26XyzMQzsDdhnaUxH/0jbvjnQaMX+zWwEQpLjaaxJ5VGr+CSYlfgU60YbSVIvbx6jANt2iSnzG6xn06yJx0ANVkNqcVNkU4pj7CsPCmghOz8bj/VJ+iI3aqRh7Ly4J0tmcHM8C7KFyZZcWvBhjZOaRLlz25BExlsTilaYnOJVbwsC6cD3Hz2nAswfdtG/HYcVz5uzhsh33TW8CduA+Rn/65ic37waMvP1Cwjvjs+ql+Ec58X+7O8N2cQA+f30M/gOoPJti/oz9YWeOHhEiqf/7WmuPkV3AL8V8ESpBVRVBooPSsoaFz2AI+G7G/UQxeU3av7Jq+tqsRiLWsM0hRWqVVZuqek/7/bRsnd4QUAejR+/p1hs7/JIUr3MYnfcQrsAA==; 20:ZSidFO7Gq7TnPYorc2pwwsKM9TRWpK+SzOVVbcW5Ugz9gcEPHCJBPmfzqpw6c7+sZJpp5exvs0HxtbiJH/8ncTqR6nyCZ3vyN/4HGUlYgnpJl6TpN9wYU/uG9nFg7g8I2rdS5WFhK6m0PGqzwA8C5lngwEcQVGZKZBE53WEPcSA= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jan 2017 08:04:10.1138 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR15MB1192 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-01-27_04:, , signatures=0 Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 01/26/2017 11:40 PM, Jens Axboe wrote: > On 01/26/2017 06:22 PM, Jens Axboe wrote: >> On 01/26/2017 06:15 PM, Bart Van Assche wrote: >>> On Thu, 2017-01-26 at 17:41 -0700, Jens Axboe wrote: >>>> On 01/26/2017 05:38 PM, Bart Van Assche wrote: >>>>> I see similar behavior with the blk-mq-sched branch of >>>>> git://git.kernel.dk/linux-block.git (git commit ID 0efe27068ecf): >>>>> booting happens much slower than usual and I/O hangs if I run the >>>>> srp-test software. >>>> >>>> Please don't run that, run for-4.11/block and merge it to master. >>>> Same behavior? >>> >>> I have not yet had the chance to run the srp-test software against that >>> kernel. But I already see that booting takes more than ten times longer >>> than usual. Note: as far as I know the dm-mpath driver is not involved >>> in the boot process of my test system. >> >> What's your boot device? I've been booting this on a variety of setups, >> no problems observed. It's booting my laptop, and on SCSI and SATA as >> well. What is your root drive? What is the queue depth of it? >> Controller? > > Are you using dm for your root device? > > I think I see what is going on. The scheduler framework put the > insertion of flushes on the side, whereas it's integrated "nicely" > on the legacy side. > > Can you try with this applied? This is on top of the previous two that > we already went through. Or, you can just pull: > > git://git.kernel.dk/linux-block for-4.11/next > > which is for-4.11/block with the next set of fixes on top that I haven't > pulled in yet. The previous patch had a bug if you didn't use a scheduler, here's a version that should work fine in both cases. I've also updated the above mentioned branch, so feel free to pull that as well and merge to master like before. commit 2f54ba92a274a7c1a5ceb34a56565f84f7b994b7 Author: Jens Axboe Date: Fri Jan 27 01:00:47 2017 -0700 blk-mq-sched: add flush insertion into blk_mq_sched_insert_request() Instead of letting the caller check this and handle the details of inserting a flush request, put the logic in the scheduler insertion function. This fixes direct flush insertion outside of the usual make_request_fn calls, like from dm via blk_insert_cloned_request(). Signed-off-by: Jens Axboe diff --git a/block/blk-core.c b/block/blk-core.c index a61f1407f4f6..78daf5b6d7cb 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -2129,7 +2129,7 @@ int blk_insert_cloned_request(struct request_queue *q, struct request *rq) if (q->mq_ops) { if (blk_queue_io_stat(q)) blk_account_io_start(rq, true); - blk_mq_sched_insert_request(rq, false, true, false); + blk_mq_sched_insert_request(rq, false, true, false, false); return 0; } diff --git a/block/blk-exec.c b/block/blk-exec.c index 86656fdfa637..ed1f10165268 100644 --- a/block/blk-exec.c +++ b/block/blk-exec.c @@ -66,7 +66,7 @@ void blk_execute_rq_nowait(struct request_queue *q, struct gendisk *bd_disk, * be reused after dying flag is set */ if (q->mq_ops) { - blk_mq_sched_insert_request(rq, at_head, true, false); + blk_mq_sched_insert_request(rq, at_head, true, false, false); return; } diff --git a/block/blk-flush.c b/block/blk-flush.c index d7de34ee39c2..4427896641ac 100644 --- a/block/blk-flush.c +++ b/block/blk-flush.c @@ -456,7 +456,7 @@ void blk_insert_flush(struct request *rq) if ((policy & REQ_FSEQ_DATA) && !(policy & (REQ_FSEQ_PREFLUSH | REQ_FSEQ_POSTFLUSH))) { if (q->mq_ops) - blk_mq_sched_insert_request(rq, false, true, false); + blk_mq_sched_insert_request(rq, false, true, false, false); else list_add_tail(&rq->queuelist, &q->queue_head); return; diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index c27613de80c5..5e91743e193a 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -336,6 +336,64 @@ void blk_mq_sched_restart_queues(struct blk_mq_hw_ctx *hctx) } } +/* + * Add flush/fua to the queue. If we fail getting a driver tag, then + * punt to the requeue list. Requeue will re-invoke us from a context + * that's safe to block from. + */ +static void blk_mq_sched_insert_flush(struct blk_mq_hw_ctx *hctx, + struct request *rq, bool can_block) +{ + if (blk_mq_get_driver_tag(rq, &hctx, can_block)) { + blk_insert_flush(rq); + blk_mq_run_hw_queue(hctx, true); + } else + blk_mq_add_to_requeue_list(rq, true, true); +} + +void blk_mq_sched_insert_request(struct request *rq, bool at_head, + bool run_queue, bool async, bool can_block) +{ + struct request_queue *q = rq->q; + struct elevator_queue *e = q->elevator; + struct blk_mq_ctx *ctx = rq->mq_ctx; + struct blk_mq_hw_ctx *hctx = blk_mq_map_queue(q, ctx->cpu); + + if (rq->tag == -1 && (rq->cmd_flags & (REQ_PREFLUSH | REQ_FUA))) { + blk_mq_sched_insert_flush(hctx, rq, can_block); + return; + } + + if (e && e->type->ops.mq.insert_requests) { + LIST_HEAD(list); + + list_add(&rq->queuelist, &list); + e->type->ops.mq.insert_requests(hctx, &list, at_head); + } else { + spin_lock(&ctx->lock); + __blk_mq_insert_request(hctx, rq, at_head); + spin_unlock(&ctx->lock); + } + + if (run_queue) + blk_mq_run_hw_queue(hctx, async); +} + +void blk_mq_sched_insert_requests(struct request_queue *q, + struct blk_mq_ctx *ctx, + struct list_head *list, bool run_queue_async) +{ + struct blk_mq_hw_ctx *hctx = blk_mq_map_queue(q, ctx->cpu); + struct elevator_queue *e = hctx->queue->elevator; + + if (e && e->type->ops.mq.insert_requests) + e->type->ops.mq.insert_requests(hctx, list, false); + else + blk_mq_insert_requests(hctx, ctx, list); + + blk_mq_run_hw_queue(hctx, run_queue_async); +} + static void blk_mq_sched_free_tags(struct blk_mq_tag_set *set, struct blk_mq_hw_ctx *hctx, unsigned int hctx_idx) diff --git a/block/blk-mq-sched.h b/block/blk-mq-sched.h index becbc7840364..9478aaeb48c5 100644 --- a/block/blk-mq-sched.h +++ b/block/blk-mq-sched.h @@ -21,6 +21,12 @@ bool __blk_mq_sched_bio_merge(struct request_queue *q, struct bio *bio); bool blk_mq_sched_try_insert_merge(struct request_queue *q, struct request *rq); void blk_mq_sched_restart_queues(struct blk_mq_hw_ctx *hctx); +void blk_mq_sched_insert_request(struct request *rq, bool at_head, + bool run_queue, bool async, bool can_block); +void blk_mq_sched_insert_requests(struct request_queue *q, + struct blk_mq_ctx *ctx, + struct list_head *list, bool run_queue_async); + void blk_mq_sched_dispatch_requests(struct blk_mq_hw_ctx *hctx); void blk_mq_sched_move_to_dispatch(struct blk_mq_hw_ctx *hctx, struct list_head *rq_list, @@ -62,45 +68,6 @@ static inline void blk_mq_sched_put_rq_priv(struct request_queue *q, e->type->ops.mq.put_rq_priv(q, rq); } -static inline void -blk_mq_sched_insert_request(struct request *rq, bool at_head, bool run_queue, - bool async) -{ - struct request_queue *q = rq->q; - struct elevator_queue *e = q->elevator; - struct blk_mq_ctx *ctx = rq->mq_ctx; - struct blk_mq_hw_ctx *hctx = blk_mq_map_queue(q, ctx->cpu); - - if (e && e->type->ops.mq.insert_requests) { - LIST_HEAD(list); - - list_add(&rq->queuelist, &list); - e->type->ops.mq.insert_requests(hctx, &list, at_head); - } else { - spin_lock(&ctx->lock); - __blk_mq_insert_request(hctx, rq, at_head); - spin_unlock(&ctx->lock); - } - - if (run_queue) - blk_mq_run_hw_queue(hctx, async); -} - -static inline void -blk_mq_sched_insert_requests(struct request_queue *q, struct blk_mq_ctx *ctx, - struct list_head *list, bool run_queue_async) -{ - struct blk_mq_hw_ctx *hctx = blk_mq_map_queue(q, ctx->cpu); - struct elevator_queue *e = hctx->queue->elevator; - - if (e && e->type->ops.mq.insert_requests) - e->type->ops.mq.insert_requests(hctx, list, false); - else - blk_mq_insert_requests(hctx, ctx, list); - - blk_mq_run_hw_queue(hctx, run_queue_async); -} - static inline bool blk_mq_sched_allow_merge(struct request_queue *q, struct request *rq, struct bio *bio) diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c index 1b156ca79af6..78bbacd129c9 100644 --- a/block/blk-mq-tag.c +++ b/block/blk-mq-tag.c @@ -106,6 +106,7 @@ unsigned int blk_mq_get_tag(struct blk_mq_alloc_data *data) struct sbq_wait_state *ws; DEFINE_WAIT(wait); unsigned int tag_offset; + bool drop_ctx; int tag; if (data->flags & BLK_MQ_REQ_RESERVED) { @@ -128,6 +129,7 @@ unsigned int blk_mq_get_tag(struct blk_mq_alloc_data *data) return BLK_MQ_TAG_FAIL; ws = bt_wait_ptr(bt, data->hctx); + drop_ctx = data->ctx == NULL; do { prepare_to_wait(&ws->wait, &wait, TASK_UNINTERRUPTIBLE); @@ -150,7 +152,8 @@ unsigned int blk_mq_get_tag(struct blk_mq_alloc_data *data) if (tag != -1) break; - blk_mq_put_ctx(data->ctx); + if (data->ctx) + blk_mq_put_ctx(data->ctx); io_schedule(); @@ -166,6 +169,9 @@ unsigned int blk_mq_get_tag(struct blk_mq_alloc_data *data) ws = bt_wait_ptr(bt, data->hctx); } while (1); + if (drop_ctx && data->ctx) + blk_mq_put_ctx(data->ctx); + finish_wait(&ws->wait, &wait); found_tag: diff --git a/block/blk-mq.c b/block/blk-mq.c index 4df397910251..888868b62018 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -568,13 +568,13 @@ static void blk_mq_requeue_work(struct work_struct *work) rq->rq_flags &= ~RQF_SOFTBARRIER; list_del_init(&rq->queuelist); - blk_mq_sched_insert_request(rq, true, false, false); + blk_mq_sched_insert_request(rq, true, false, false, true); } while (!list_empty(&rq_list)) { rq = list_entry(rq_list.next, struct request, queuelist); list_del_init(&rq->queuelist); - blk_mq_sched_insert_request(rq, false, false, false); + blk_mq_sched_insert_request(rq, false, false, false, true); } blk_mq_run_hw_queues(q, false); @@ -847,12 +847,11 @@ static inline unsigned int queued_to_index(unsigned int queued) return min(BLK_MQ_MAX_DISPATCH_ORDER - 1, ilog2(queued) + 1); } -static bool blk_mq_get_driver_tag(struct request *rq, - struct blk_mq_hw_ctx **hctx, bool wait) +bool blk_mq_get_driver_tag(struct request *rq, struct blk_mq_hw_ctx **hctx, + bool wait) { struct blk_mq_alloc_data data = { .q = rq->q, - .ctx = rq->mq_ctx, .hctx = blk_mq_map_queue(rq->q, rq->mq_ctx->cpu), .flags = wait ? 0 : BLK_MQ_REQ_NOWAIT, }; @@ -1395,7 +1394,7 @@ static void blk_mq_try_issue_directly(struct request *rq, blk_qc_t *cookie) } insert: - blk_mq_sched_insert_request(rq, false, true, true); + blk_mq_sched_insert_request(rq, false, true, true, false); } /* @@ -1446,10 +1445,12 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) cookie = request_to_qc_t(data.hctx, rq); if (unlikely(is_flush_fua)) { + blk_mq_put_ctx(data.ctx); blk_mq_bio_to_request(rq, bio); blk_mq_get_driver_tag(rq, NULL, true); blk_insert_flush(rq); - goto run_queue; + blk_mq_run_hw_queue(data.hctx, true); + goto done; } plug = current->plug; @@ -1502,7 +1503,7 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) blk_mq_put_ctx(data.ctx); blk_mq_bio_to_request(rq, bio); blk_mq_sched_insert_request(rq, false, true, - !is_sync || is_flush_fua); + !is_sync || is_flush_fua, true); goto done; } if (!blk_mq_merge_queue_io(data.hctx, data.ctx, rq, bio)) { @@ -1512,7 +1513,6 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) * latter allows for merging opportunities and more efficient * dispatching. */ -run_queue: blk_mq_run_hw_queue(data.hctx, !is_sync || is_flush_fua); } blk_mq_put_ctx(data.ctx); @@ -1568,10 +1568,12 @@ static blk_qc_t blk_sq_make_request(struct request_queue *q, struct bio *bio) cookie = request_to_qc_t(data.hctx, rq); if (unlikely(is_flush_fua)) { + blk_mq_put_ctx(data.ctx); blk_mq_bio_to_request(rq, bio); blk_mq_get_driver_tag(rq, NULL, true); blk_insert_flush(rq); - goto run_queue; + blk_mq_run_hw_queue(data.hctx, true); + goto done; } /* @@ -1612,7 +1614,7 @@ static blk_qc_t blk_sq_make_request(struct request_queue *q, struct bio *bio) blk_mq_put_ctx(data.ctx); blk_mq_bio_to_request(rq, bio); blk_mq_sched_insert_request(rq, false, true, - !is_sync || is_flush_fua); + !is_sync || is_flush_fua, true); goto done; } if (!blk_mq_merge_queue_io(data.hctx, data.ctx, rq, bio)) { @@ -1622,7 +1624,6 @@ static blk_qc_t blk_sq_make_request(struct request_queue *q, struct bio *bio) * latter allows for merging opportunities and more efficient * dispatching. */ -run_queue: blk_mq_run_hw_queue(data.hctx, !is_sync || is_flush_fua); } diff --git a/block/blk-mq.h b/block/blk-mq.h index d19b0e75a129..d34929968071 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -34,6 +34,8 @@ void blk_mq_wake_waiters(struct request_queue *q); bool blk_mq_dispatch_rq_list(struct blk_mq_hw_ctx *, struct list_head *); void blk_mq_flush_busy_ctxs(struct blk_mq_hw_ctx *hctx, struct list_head *list); bool blk_mq_hctx_has_pending(struct blk_mq_hw_ctx *hctx); +bool blk_mq_get_driver_tag(struct request *rq, struct blk_mq_hw_ctx **hctx, + bool wait); /* * Internal helpers for allocating/freeing the request map