From patchwork Thu Dec 22 16:23:24 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 9485289 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B701B600BA for ; Thu, 22 Dec 2016 16:23:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A0D0327F9C for ; Thu, 22 Dec 2016 16:23:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 92C1327FE4; Thu, 22 Dec 2016 16:23:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID, T_TVD_MIME_EPI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC05F27F9C for ; Thu, 22 Dec 2016 16:23:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S941227AbcLVQXi (ORCPT ); Thu, 22 Dec 2016 11:23:38 -0500 Received: from mail-sn1nam02on0047.outbound.protection.outlook.com ([104.47.36.47]:56561 "EHLO NAM02-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S941220AbcLVQXf (ORCPT ); Thu, 22 Dec 2016 11:23:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sandiskcorp.onmicrosoft.com; s=selector1-sandisk-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=RkBlwh4GsAVboAUlGy4oYcaRALsoJkrGDTIbxz6oR+U=; b=DFtktjXWbtU1Mpk75j9VW4qDl/3xLzr/+A4EzdamyuBKOxxRQBd7hU7aAW7/Mp53l3b/6kolGIP6fN+ws+wO/t6wugJsC6tw3CnIc6cl4cLlahRBISDh1r7DdUb3GAaEEA9WZNJPrjsLKbOY+Pkke7oQ3C6wAcC+gFo15iitTpg= Received: from BY2PR02CA0053.namprd02.prod.outlook.com (10.141.216.43) by BL2PR02MB386.namprd02.prod.outlook.com (10.141.91.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.803.11; Thu, 22 Dec 2016 16:23:31 +0000 Received: from BN1AFFO11FD043.protection.gbl (2a01:111:f400:7c10::105) by BY2PR02CA0053.outlook.office365.com (2a01:111:e400:2c40::43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.789.14 via Frontend Transport; Thu, 22 Dec 2016 16:23:31 +0000 Authentication-Results: spf=pass (sender IP is 74.221.232.54) smtp.mailfrom=sandisk.com; kernel.dk; dkim=none (message not signed) header.d=none;kernel.dk; dmarc=bestguesspass action=none header.from=sandisk.com; Received-SPF: Pass (protection.outlook.com: domain of sandisk.com designates 74.221.232.54 as permitted sender) receiver=protection.outlook.com; client-ip=74.221.232.54; helo=sacsmgep14.sandisk.com; Received: from sacsmgep14.sandisk.com (74.221.232.54) by BN1AFFO11FD043.mail.protection.outlook.com (10.58.52.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.789.10 via Frontend Transport; Thu, 22 Dec 2016 16:23:29 +0000 X-AuditID: ac1c2133-b8fff70000000ec6-c4-585c7921a1dd Received: from SACHUBIP01.sdcorp.global.sandisk.com (Unknown_Domain [172.28.1.254]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id 0F.79.03782.1297C585; Thu, 22 Dec 2016 17:08:51 -0800 (PST) Received: from ULS-OP-MBXIP03.sdcorp.global.sandisk.com ([fe80::f9ec:1e1b:1439:62d8]) by SACHUBIP01.sdcorp.global.sandisk.com ([10.181.10.103]) with mapi id 14.03.0319.002; Thu, 22 Dec 2016 08:23:25 -0800 From: Bart Van Assche To: "linux-kernel@vger.kernel.org" , "linux-block@vger.kernel.org" , "axboe@fb.com" , "axboe@kernel.dk" CC: "osandov@fb.com" , "paolo.valente@linaro.org" Subject: Re: [PATCHSET v4] blk-mq-scheduling framework Thread-Topic: [PATCHSET v4] blk-mq-scheduling framework Thread-Index: AQHSXG+3wEWjwr+YNkuLarcjHhg2Gw== Date: Thu, 22 Dec 2016 16:23:24 +0000 Message-ID: <1482423775.2682.6.camel@sandisk.com> References: <1481933536-12844-1-git-send-email-axboe@fb.com> In-Reply-To: <1481933536-12844-1-git-send-email-axboe@fb.com> Accept-Language: nl-NL, en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [172.28.1.254] MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrGKsWRmVeSWpSXmKPExsWyRobxn65yZUyEwYsDFhb/9xxjs1h9t5/N Yu8tbYvLu+awWRy+d5XF4tWfvUwObB4Tm9+xe1w+W+px59oeNo/Pm+QCWKK4bFJSczLLUov0 7RK4Mq41r2Ms2NrDUrF230qmBsbJ/5i7GDk5JARMJPpXHWMFsYUEljBJHL+i1cXIBWRfYpTY 0t4JlmATMJKYPWEPC0hCROAYo8S8DS/YQBLMAnESC7/eAJskLGAmcezjfbC4iIC5xOd5P5kh bD2JlRs/Ag3i4GARUJXY1KcBYvIKGEr8/W4MsddG4t+NyUwgNqeArcSk5klg1YwCshItr7kh FolL3HoynwniZBGJhxdPs0HYohIvH/9jhbAVJD6v+Ad1WIDEkedrweK8AoISJ2c+YZnAKDIL yahZSMpmISmbBbSZWUBTYv0ufYgSK4n7xw8yQ9iKElO6H7JD2CkSlya/ZoOwQySOnt0GZHMB 2fcZJW4sW84G0zzxwy+smh9vWc8E09zW84YJovkeo8StY70sMM0nZ87HojlM4u3LVqgFbhIL Ph9lh2i+yCgxeVYjO0zzlg0bUDQvYBRdxShWnJhcnJueWmBoolecmJeSWZytl5yfu4kRnOIU jXcw/tvgfohRgINRiYe3QiA6Qog1say4MvcQowrQoEcbVl9glGLJy89LVRLh/fIDKM2bklhZ lVqUH19UmpNafIhRmoNFSZxX/sDuUCGB9MSS1OzU1ILUIpgsEwenVAMjY+H5GZLm/+ZerTp9 4p3efuZjx1kubmVvfmSvvCxz0Z45j5oNF84KOmwXUdwTcofrkG1u7ksmYy35AMmbmro1DSrM K5/H5kfuLlgU3n4rUeTALJMV3xOmHrM/Fqz6lfXTX+81y7lf8ky/c2jigU8Lnh19LWdy7uN/ 6ZY/Fa4P3Fa3ydRqf9k+S4mlOCPRUIu5qDgRAFwMkwZ5AwAA X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:74.221.232.54; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(7916002)(39850400002)(39860400002)(39410400002)(39450400003)(39840400002)(2980300002)(438002)(377424004)(24454002)(189002)(199003)(2950100002)(8676002)(2476003)(305945005)(7736002)(2201001)(356003)(189998001)(30436002)(92566002)(5890100001)(4610100001)(38730400001)(84326002)(86362001)(626004)(2906002)(81156014)(512874002)(102836003)(6116002)(81166006)(3846002)(2501003)(68736007)(4810100001)(4326007)(50986999)(76176999)(54356999)(69596002)(99936001)(36756003)(230783001)(103116003)(568964002)(2900100001)(4001150100001)(97736004)(2270400002)(8936002)(5001770100001)(229853002)(106116001)(5660300001)(106466001)(33646002)(7099028); DIR:OUT; SFP:1101; SCL:1; SRVR:BL2PR02MB386; H:sacsmgep14.sandisk.com; FPR:; SPF:Pass; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BN1AFFO11FD043; 1:0n63gu1itqkBB0aLXoRkVQpaSaOxIubPym9avBFwagOouY3Au5Z9viJNRKqdhRgwYxnx3Rj/J19DVkOiKnX3teAUM9AJK35jNnZtj/JrKu+qUhjnx0TpfqewtkZpCl617qquD5Bs1QAQySulhnw+tDnFC6T2+mqG2HXFffrcKaE0OL8K3BgDyWENeV3pHqFUAUo0MdF1/LXnaW2TDhtM4xF+3GQJ1U69O63XgOJzM7t5F973Kj0djL3ejLY46k72YT0nohrMgKNU52Wj9ivm+R9vjGO7ZFFSxVhSoDLPzDW1N+JVXz27URP1kPFcLHxi+yKNxtnmla9VAqFE2KeTqcVDIq+6IGuHNNzSYk2SzoA2J30RK6SHGXg+aHlhmm2uvgG3Bhti+fmPjcnG+oI7T25Y4nvyC3FUIPhnAeA0Lxy/GRqQtlvAHVYwQdkow5gm3GJwhMtTDXWeTEDW6iDEbBSsnDlQVdwjcyCuwjH79dno1miYZQDmDsRQKI0ovL7VlL8kVDy8D+9G/xeIXNBhmpX8vUi+EICcyKZ+u8jDSSpYGkwYWwl+C1IwoSPABKScyDN72W7Wuyo3P/lZIr4Cmg== X-MS-Office365-Filtering-Correlation-Id: afc8229b-86b1-4e69-4826-08d42a86ddd5 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(8251501002); SRVR:BL2PR02MB386; X-Microsoft-Exchange-Diagnostics: 1; BL2PR02MB386; 3:s4WUAQ4dxPnnDRZWTfcV2ObNnsH6/clv7xpVixTegArEE+vmsEFjO2WP7OquQ/1kUcLZvH5KNQ39V6rJ8+MPEj4g0EYM9ODIFdjE2xMcs2mb4XQeieAnoCZ122OlnpLO1bTy4Jg4ihicHkB6AsqICQt6S/eCPKl18Sq5wXe3tQ8BrAJugk4fhst/5cXUMddfhSz3Ey3RTBD/vWELFC7tfm15i/rrZQ2ZV2bkYBXOein6G39/wt9TsAJOLbfFMkZozfN/TojvytGBvHWG+g7VLZE4G7IC75ZjOUfEnuuSC9540yI/xJewbxzHl2Gy0HGGZ/GphTYcMzfZoAFO8vnI5v3/6xmL5ENnNvP5e45zunEnzPLihWp8vmRKUtQSsB0f4l9spJZu7kJ6gaUqFaH4cQ==; 25:xqi8/HVH1LYle1ScFVaaOaQBr3MtW8JgSumi0kRJEGBg+gzY4obWoJ2BQJdbTYXmpQhW0Daey01ZdhaK67sNePSOOmV3cuBW1kt3Ze8LHfqUGx5g1jcts+eizuETVuR923Zii6x0X0g/njWlIm1LALwLtx9PwT2BAA3uu9aczOQweO5H8uVileQJdKb2IiyMvgI3Wblt/q51bJCiW9bicU1C/AMLKFztsgixvPac+NihPE0yYQOLm8UA4SmDYc7fEYkOfFup13U0IkjwWjZcj3F5YztIpTjRYa+PKKRtcppCb4UdwuJkW0LKqeoS7PHZIOYlQ+tWbkfQS5+NcVbYs6v/o3bOLWyp852pUo6h/DOVN7jGtCzy92j0Peflu7eiUcJsLp/0KIoyI47HJUgOAQHwB8fhyLcDLsHflMpjZxy6WpTCInvJvuWm9cWe56opaGijlmZoSF3Z9FJpfuTt+koL9YpVvEm1xgM4lbZwIAEFKNef9h9fkI27iZMOnvEWzCVMAAwDehDAgiX ZysGYwQ== X-Microsoft-Exchange-Diagnostics: 1; BL2PR02MB386; 31:gv/mfZmHS6gOnIDJutMjh9wndrntuBbftXXcCf37q7bGv56GLCi8ioPAAz8juL7Lf/f9W5CjeqeXJWtemDN8Sl7TLhEsS7nqKgUhxVSfirG8JNFBw8jb4kW9qG2+tBXq31OTryqzVEpYW5Wp7Y/R9+e7cNMlFMxssEgBFZhdn0ppRHTt0l296PUY74aGVgOFAXHOyUYIhEqECLIMabtc6tK/+omvAqty65eP9XcAQtxNzKC/gANXCuwjWV2qcaNbLM77TklH69DyvmLITxIq+dA5/D0opLpFe6KCmif8hEgN65qcXaAhlxnkCpHLZMvSnQw3snI/xftnNpnv0Eo6XZwnCdf5FFUOhK+BciS/oj4=; 20:LPDyn+pavlPUUE19ezH6L2uNaDl1keqOGCFjpWq9qnFRXjjfk5zZsvH+yNigAENpECATV+mitOs0BC+aVHGkEgJppcOP6pdBxQPCNqx4UhmxmZufUgcpjOZedTT78kHxncrGwO+j+4UwbEBx+ZDyjHpYJRc72/Q6t9QdKbtGE6WEj0/CNLf2FICI+r/dXeYGr60Uh3Q/8wDfh0+9xwmcH6LapY2bENvQxQ/MPVZ2GrX8rJhAu3BzmFbBKv6AyKmXHeumk/cGvBVz6AtDLloXvbECf8pCOMoZjE9+pAKZRrPMWJ5uOSRsxnQxumgkdrKKOvPiq2a3rPAuv3Om7ZXXboRkCgTCe8GfgKBXRUDLk4M9ugIPCqzkSeounpXKue/AmvFXKWwyE3DCfmkl745Zx7dV9/HS0f1/FAaCpJR23IBMjfrQol/B4XFWiqoE+t/lCJiejyelQAbjl0IhwHFlLp+KuRdGeCEljX0zpY2kZYoqcLQUtzPByt1/bnK3TGs3 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(788757137089)(42932892334569); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(102415395)(6040375)(601004)(2401047)(13017025)(13015025)(13018025)(13023025)(13024025)(8121501046)(5005006)(3002001)(10201501046)(6055026)(6041248)(20161123562025)(20161123560025)(20161123564025)(20161123555025)(6072148); SRVR:BL2PR02MB386; BCL:0; PCL:0; RULEID:; SRVR:BL2PR02MB386; X-Microsoft-Exchange-Diagnostics: 1; BL2PR02MB386; 4:Gd1sMNP3ptZbHwZGmpH6UkUsfBnaAjAu0hiiNHJ5OvBP1heGprfbvfZZNmCI8C/iOvrMGGzYZhPhYrOYoy8tJ4pJgHBfXa+h1E/K8wa8CRUMMqx6H7FuapBYPEbMCvS+TWeb+Q5r6vlwlH43/erqoG8gkV4Cjc8c2/yJJcxanmHC3e+LoDdiiELsOFSSNpipj4phGzPy0G0fcIseUM4SmhFdZn9ZbU75OkdTjd8kAKUZeE1YEB2qg47Fzto4mdSdUyF/nsbyZahqPBVqShd2VHZlrdFsxfSaSfsT+OyTy4EXrCo+PjCASf5TcfW7J2gTpJIe1R0HFmLmhRWQ1zSUzruwVnMQmgv3g4MqC+us4wCTJdKgxApSKw+IUe6lAkSUktycpoTm4GZfk0yXOq/FZUNolqj6b86J0PlBlasv2Rc7OqnO+a9abcdDuoweCtzlN37nK710LxQr0a0goOMTx6wNvJtipjsaUwHu7TFXk0X/QVhQjkDoIG+ET+yuIipha1s8cEHD15Xyx+JW1AOIhe1X+q9OvB+FRC7vdBdS6T5eZt5BLSGtTRYsfhIFhZwucSc8rACbitYbwLGjUHbdyhVLxkPQ1SOz9fCKn/0PTTopCyUD/V3wp1hys8rTVAM1dHH7STHy2WC+1NHX2MY5a4zzPQX7iQdN7+CwI2uNXXkmdnKzMtn3PUYSMWHFCl3CKgY5U92ig1o6mBG9oge+6XTKAT2EsNvTQscMff9OIwMEWgc76bbzG55++JLuMmclWazMGJm+DyiPskhKav64hQ== X-Forefront-PRVS: 01644DCF4A X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BL2PR02MB386; 23:gzlS9Yd/zQIYbSEd/NCSrp12d04sBOQpMi3LSPpvYJ?= =?us-ascii?Q?2Sb+NQCq76A7iXq+tcu7uau6cB1PXxiPG76xGATbve/EsZ2nx5KZQXNxvOer?= =?us-ascii?Q?WWlv7k69NbZSCNoB4jwaXDkxgsnqSW1T4mk2Gdn+RO5qdHm1wrVKMJNUocjg?= =?us-ascii?Q?CBmoqr5Xobra7jq4Qp2w8L9MjFAyusQSNzigWwFzbpgxMOrm/71jJDjp0UEy?= =?us-ascii?Q?HHVihc7kybiYiVGr9ApgOojBgscUbRpqEvdEjQSimxsKhGCkguaS9umHfHw8?= =?us-ascii?Q?MJtcStV6PRJQFmCBRlzHz3AEcxBiuOjnkKxL/yRTw0rcSXBwn8DhyviyoECk?= =?us-ascii?Q?sQL7FePB1F0hRney+eEUU5ksyN2jKDbbNX1eGXLIh57gm5TVFkMq9bIIc8Ib?= =?us-ascii?Q?NAK7dpcZfAiY2aRHdylDccDwBFNLupK6lqYXnZx7QnqC9tuwFqjsF4OCPaec?= =?us-ascii?Q?Y5te8IDp4Ptmm0oZmbKR9x/qbFlz5uJjHQkj3LXFQxtyMkq7qfmQz0gItRSw?= =?us-ascii?Q?TqsOgLaTShQTXV+cE3H/4TC+0N7Tr2EQwaaM38qSnthNxRu+5Gyn++vDmUVH?= =?us-ascii?Q?uw23NgrvrC+01D7hb40Iyn6aJgmeXLZ6tBOuBNEjid7Se+9OzKggCrZl4xqE?= =?us-ascii?Q?qjrWWdkboIxOopw5ILQ0W6/NdQ7RJgWNoUqrwbKh/7QrHZ6m/c8/uxLaAvBP?= =?us-ascii?Q?irBmCGSpdlD7m8nTFgD00CfVVlGALU19SSmqoyELN5bAv4zxo+3n44R09Jmn?= =?us-ascii?Q?TI4I9sEtQQcFXwOj7EGq/9+xPSiLvsOQOot46Z9Rel6li74zIyPLfazwrmfM?= =?us-ascii?Q?ww0EiCt5ghljr0tuVyfTMj4AjehSfXnOu2VV8g3kk6V9O2zzY2li5dD/nwin?= =?us-ascii?Q?2zqK7/jgBNjFgSP1i8QHfsLgsSBc3jPRtv4Vr2OYolkhB2uJP4EGABLbNUkL?= =?us-ascii?Q?D9JM7ilWa5YuY+9ACF3+1z4gCoynSgYttKvlI6XTiMVnwSwEPe6paDjMyM4O?= =?us-ascii?Q?SiAmz+LtPHLVZ62PTaVeXMv8dl+mgATlGsr4ce4GruXgwAMmskA80EXZKV9Y?= =?us-ascii?Q?6AdbXFlY0umGIalpIYkiZ88iTW7Z8xPc8p/CL/eRwOjyWvNqO8FVH+bb4IUR?= =?us-ascii?Q?MG+lTMsbnry+TemOy3H7tSnSuzulZJBwqOsaoP0mjRoawwV4X6Kgv/IOzjg/?= =?us-ascii?Q?xNncJ0HpnDa6qKBWH1wFgRbMYcyBEY/6z2zQ38zGWRYBePSdxYypivTsE6Ly?= =?us-ascii?Q?l22OMv8n1CUbFy/pB6ipow36vAVB74I3CweaNHOc4X448oc06cypJplPB4lz?= =?us-ascii?Q?TlfTGnWqkzbRk3WghifK8E5JNwgrtRty7ZWX0bt0J8XWggtp1swMBXWwIjFa?= =?us-ascii?Q?HOIW2itdETby8LmCVFMhP0kOStmkDaCFRyjut1CJNjuEdsYqop2i5CmACnKK?= =?us-ascii?Q?YS7OD1NwLajg09J3s8505vHVUpPjk4yJiiI1XNE9sb/mxJF9i0R2tTDpkfWa?= =?us-ascii?Q?3xtLcD1ur2kw=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; BL2PR02MB386; 6:5QYQDXd2UlX0fOG9ClrwHJvTLwm+eezFGDco/U5CwxdyOp3othSp69k28DGN8Rbofm83joynyYTFRMFBZeankMa7L2SjYtmE6PN5AT92M1ZokcJOp2OcqfW52PpcYvYUw6beXXo2/mF8CEZGt45qjFtcTzByV2TKDTmkQWm0ewu5Dm6EGpRj/7lOj4fOfuCWDfbJ0vnUfSHd7oJ2fc2dhVKhmpni7ymTnmtQgdb3MRV4Y951PPgQyGnpIXOJXt4C0cw7XzNG0nosDJZqllk9aRbJ9MWdmNTuHdgMfw9TTYu3ptS05l/8l4h+Tnf67WDVRW5bArC4P5t4YwXrQM2sAuscwz5GT5T2s3iPYmLXy8Ws9Op65hpjMN75F9FaGh6gvwPF2HACE6mhbhx7/AIMfykefEhSeW8Q5FMNICj58kH+oJbZJed4R35S7outxG3tLyRR9W+uesK/gHFZ+WOqEA==; 5://t4W7JhmVuQzpZKNh1mq5iyZjEgCPeceq4I9Zq0I5uyfTdFlJ99zJ2jb+Ox0gX3VR/htH8WEhEBcsmdSoAl12R7zZkEeU8B4qdUQ18skblHN485RfnMxI1OX7JGvzDKhLNnfQgru6ab5YPdocVXmVW//ODIm7ShMIwqi9dfQbw=; 24:7HQ7dxkaIfEfB0d0kd/4g+FB28DMaVCGSyCX3sypzCOhhp8jNEJXGAkQHVcprbVwDOSAw4zR1/llv6TI8cORI2cI6hTwdUfrbMyz4Vr+ESU= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BL2PR02MB386; 7:/MRn2i/y/rQIq8myJVO13kJLDMmFVw4gSqFb3ppp7DTTCw7bqOEmR+T/pbRbCiiBqFOybyLN1gkvGx44ePn4duUDGDgYZzF3Mwf8wDjJ2Y5FrT8H8IQdwZ2FPguU2u+mq8E8HuFSaSxgm0Hv5s0AZi7iA+o1C8edSnniTK+XJIhnC/Zbp2hKXumvWbo4SWJcSyHvySOJggYhtL1n8lTbWN2d8UlgM/f74L6QL4IKd/zRBxyXq1jH9ltKxKzWMP/QFVFgdH4SJJ1YoIV6Xs5x4NtQ0ztBMVsBFJNEBuaMyGweKufbYwLIqNcZwpC70vL+amU5yVP261Diw2etasR7fH0w5DZsRQ6pxY+pjHdYaRgDTj26rOkLfaeT0AJtjfp2ikrNnkUQba4Ln+l2+l5xOinoI/6ha5sUaNFZgdPVGxaNTkeIY7T3Unfg9c2/jxLpVlsxEpZp+eIE7/3a8AGZKQ==; 20:TuNv1zR96Wbol+LA+35bsuUyjbCMZVNI8qSE7ja/VqwyOkt23Oi2F0Gm30NgBcCQ4WV7lDn36EU/HEAVlrCSqnSJFSJZLHv5FWRJnBD6GBUORkNVofGDYVGv9zHP0wNzjU3RkU5w2omV4hVQuOcQYE0xdywr+7CP0G00DTX6roqbUwwb4vfSFYt6nu8+ZYZuQkZXf0uCMwXVZ1lyUrsr+a4LEza3O7NY5F08ydE68B67YBOOk2k0kqpoCsHygEpY X-OriginatorOrg: sandisk.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Dec 2016 16:23:29.4945 (UTC) X-MS-Exchange-CrossTenant-Id: fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d; Ip=[74.221.232.54]; Helo=[sacsmgep14.sandisk.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL2PR02MB386 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Fri, 2016-12-16 at 17:12 -0700, Jens Axboe wrote: > From the discussion last time, I looked into the feasibility of having > two sets of tags for the same request pool, to avoid having to copy > some of the request fields at dispatch and completion time. To do that, > we'd have to replace the driver tag map(s) with our own, and augment > that with tag map(s) on the side representing the device queue depth. > Queuing IO with the scheduler would allocate from the new map, and > dispatching would acquire the "real" tag. We would need to change > drivers to do this, or add an extra indirection table to map a real > tag to the scheduler tag. We would also need a 1:1 mapping between > scheduler and hardware tag pools, or additional info to track it. > Unless someone can convince me otherwise, I think the current approach > is cleaner. Hello Jens, Can you have a look at the attached patches? These implement the "two tags per request" approach without a table that maps one tag type to the other or any other ugly construct. __blk_mq_alloc_request() is modified such that it assigns rq->sched_tag and sched_tags->rqs[] instead of rq->tag and tags->rqs[]. rq->tag and tags->rqs[] are assigned just before dispatch by blk_mq_assign_drv_tag(). This approach results in significantly less code than the approach proposed in v4 of your blk-mq-sched patch series. Memory usage is lower because only a single set of requests is allocated. The runtime overhead is lower because request fields no longer have to be copied between the requests owned by the block driver and the requests owned by the I/O scheduler. I can boot a VM from the virtio-blk driver but otherwise the attached patches have not yet been tested. Thanks, Bart. From c49ec4e8b0e4135a87c9894597901539f3e3ca08 Mon Sep 17 00:00:00 2001 From: Bart Van Assche Date: Wed, 21 Dec 2016 12:39:33 +0100 Subject: [PATCH 3/3] blk-mq: Split driver and scheduler tags Add 'sched_tags' next to 'tags' in struct blk_mq_hw_ctx and also in struct blk_mq_tag_set. Add 'sched_tag' next to 'tag' in struct request. Modify blk_mq_update_nr_requests() such that it accepts values larger than the queue depth. Make __blk_mq_free_request() free both tags. Make blk_mq_alloc_tag_set() allocate both tag sets. Make blk_mq_free_tag_set() free both tag sets. Make blk_mq_dispatch_rq_list() allocate the driver tag. Modify blk_mq_update_nr_requests() such that it accepts a size that exceeds the queue depth. --- block/blk-flush.c | 9 ++- block/blk-mq.c | 160 +++++++++++++++++++++++++++++++++++-------------- block/blk-mq.h | 5 +- block/blk-tag.c | 1 + include/linux/blk-mq.h | 2 + include/linux/blkdev.h | 1 + 6 files changed, 129 insertions(+), 49 deletions(-) diff --git a/block/blk-flush.c b/block/blk-flush.c index 6a7c29d2eb3c..46d12bbfde85 100644 --- a/block/blk-flush.c +++ b/block/blk-flush.c @@ -170,6 +170,8 @@ static bool blk_flush_complete_seq(struct request *rq, struct list_head *pending = &fq->flush_queue[fq->flush_pending_idx]; bool queued = false, kicked; + BUG_ON(rq->tag < 0); + BUG_ON(rq->flush.seq & seq); rq->flush.seq |= seq; @@ -319,6 +321,8 @@ static bool blk_kick_flush(struct request_queue *q, struct blk_flush_queue *fq) if (q->mq_ops) { struct blk_mq_hw_ctx *hctx; + BUG_ON(first_rq->tag < 0); + flush_rq->mq_ctx = first_rq->mq_ctx; flush_rq->tag = first_rq->tag; fq->orig_rq = first_rq; @@ -452,8 +456,9 @@ void blk_insert_flush(struct request *rq) * processed directly without going through flush machinery. Queue * for normal execution. */ - if ((policy & REQ_FSEQ_DATA) && - !(policy & (REQ_FSEQ_PREFLUSH | REQ_FSEQ_POSTFLUSH))) { + if (((policy & REQ_FSEQ_DATA) && + !(policy & (REQ_FSEQ_PREFLUSH | REQ_FSEQ_POSTFLUSH))) || + (q->mq_ops && blk_mq_assign_drv_tag(rq) < 0)) { if (q->mq_ops) blk_mq_sched_insert_request(rq, false, true, false); else diff --git a/block/blk-mq.c b/block/blk-mq.c index b68b7fc43e46..48d7968d4ed9 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -220,20 +220,21 @@ EXPORT_SYMBOL_GPL(blk_mq_rq_ctx_init); struct request *__blk_mq_alloc_request(struct blk_mq_alloc_data *data, unsigned int op) { - struct blk_mq_tags *tags = data->hctx->tags; + struct blk_mq_tags *tags = data->hctx->sched_tags; struct request *rq; - unsigned int tag; + unsigned int sched_tag; - tag = blk_mq_get_tag(data, tags); - if (tag != BLK_MQ_TAG_FAIL) { - rq = tags->rqs[tag]; + sched_tag = blk_mq_get_tag(data, tags); + if (sched_tag != BLK_MQ_TAG_FAIL) { + rq = tags->rqs[sched_tag]; + rq->tag = -1; if (blk_mq_tag_busy(data->hctx)) { rq->rq_flags = RQF_MQ_INFLIGHT; atomic_inc(&data->hctx->nr_active); } - rq->tag = tag; + rq->sched_tag = sched_tag; blk_mq_rq_ctx_init(data->q, data->ctx, rq, op); return rq; } @@ -328,6 +329,7 @@ void __blk_mq_finish_request(struct blk_mq_hw_ctx *hctx, struct blk_mq_ctx *ctx, struct request *rq) { const int tag = rq->tag; + const int sched_tag = rq->sched_tag; struct request_queue *q = rq->q; ctx->rq_completed[rq_is_sync(rq)]++; @@ -340,7 +342,13 @@ void __blk_mq_finish_request(struct blk_mq_hw_ctx *hctx, struct blk_mq_ctx *ctx, clear_bit(REQ_ATOM_STARTED, &rq->atomic_flags); clear_bit(REQ_ATOM_POLL_SLEPT, &rq->atomic_flags); - blk_mq_put_tag(hctx, hctx->tags, ctx, tag); + if (tag >= 0) { + WARN_ON_ONCE(hctx->tags->rqs[tag] != rq); + hctx->tags->rqs[tag] = NULL; + blk_mq_put_tag(hctx, hctx->tags, ctx, tag); + } + if (sched_tag >= 0) + blk_mq_put_tag(hctx, hctx->sched_tags, ctx, sched_tag); blk_queue_exit(q); } @@ -844,6 +852,26 @@ static inline unsigned int queued_to_index(unsigned int queued) return min(BLK_MQ_MAX_DISPATCH_ORDER - 1, ilog2(queued) + 1); } +int blk_mq_assign_drv_tag(struct request *rq) +{ + struct request_queue *q = rq->q; + struct blk_mq_hw_ctx *hctx = blk_mq_map_queue(q, rq->mq_ctx->cpu); + struct blk_mq_alloc_data data = { + .q = rq->q, + .ctx = rq->mq_ctx, + .hctx = hctx, + }; + + rq->tag = blk_mq_get_tag(&data, hctx->tags); + if (rq->tag < 0) + goto out; + WARN_ON_ONCE(hctx->tags->rqs[rq->tag]); + hctx->tags->rqs[rq->tag] = rq; + +out: + return rq->tag; +} + bool blk_mq_dispatch_rq_list(struct blk_mq_hw_ctx *hctx, struct list_head *list) { struct request_queue *q = hctx->queue; @@ -866,6 +894,8 @@ bool blk_mq_dispatch_rq_list(struct blk_mq_hw_ctx *hctx, struct list_head *list) struct blk_mq_queue_data bd; rq = list_first_entry(list, struct request, queuelist); + if (rq->tag < 0 && blk_mq_assign_drv_tag(rq) < 0) + break; list_del_init(&rq->queuelist); bd.rq = rq; @@ -1296,7 +1326,8 @@ static void blk_mq_try_issue_directly(struct request *rq, blk_qc_t *cookie) goto insert; hctx = blk_mq_map_queue(q, rq->mq_ctx->cpu); - if (blk_mq_hctx_stopped(hctx)) + if (blk_mq_hctx_stopped(hctx) || + (rq->tag < 0 && blk_mq_assign_drv_tag(rq) < 0)) goto insert; new_cookie = blk_tag_to_qc_t(rq->tag, hctx->queue_num); @@ -1592,17 +1623,19 @@ void blk_mq_free_rq_map(struct blk_mq_tags *tags) } struct blk_mq_tags *blk_mq_alloc_rq_map(struct blk_mq_tag_set *set, - unsigned int hctx_idx) + unsigned int hctx_idx, + unsigned int nr_tags, + unsigned int reserved_tags) { struct blk_mq_tags *tags; - tags = blk_mq_init_tags(set->queue_depth, set->reserved_tags, + tags = blk_mq_init_tags(nr_tags, reserved_tags, set->numa_node, BLK_MQ_FLAG_TO_ALLOC_POLICY(set->flags)); if (!tags) return NULL; - tags->rqs = kzalloc_node(set->queue_depth * sizeof(struct request *), + tags->rqs = kzalloc_node(nr_tags * sizeof(struct request *), GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY, set->numa_node); if (!tags->rqs) { @@ -1800,6 +1833,7 @@ static int blk_mq_init_hctx(struct request_queue *q, cpuhp_state_add_instance_nocalls(CPUHP_BLK_MQ_DEAD, &hctx->cpuhp_dead); hctx->tags = set->tags[hctx_idx]; + hctx->sched_tags = set->sched_tags[hctx_idx]; /* * Allocate space for all possible cpus to avoid allocation at @@ -1881,6 +1915,38 @@ static void blk_mq_init_cpu_queues(struct request_queue *q, } } +static void __blk_mq_free_rq_map_i(struct blk_mq_tag_set *set, int hctx_idx) +{ + if (set->sched_tags[hctx_idx]) { + blk_mq_free_rqs(set, set->sched_tags[hctx_idx], hctx_idx); + blk_mq_free_rq_map(set->sched_tags[hctx_idx]); + set->sched_tags[hctx_idx] = NULL; + } + if (set->tags[hctx_idx]) { + blk_mq_free_rq_map(set->tags[hctx_idx]); + set->tags[hctx_idx] = NULL; + } +} + +static bool __blk_mq_alloc_rq_map_i(struct blk_mq_tag_set *set, int hctx_idx, + unsigned int nr_requests) +{ + int ret = 0; + + set->tags[hctx_idx] = blk_mq_alloc_rq_map(set, hctx_idx, + set->queue_depth, set->reserved_tags); + set->sched_tags[hctx_idx] = blk_mq_alloc_rq_map(set, hctx_idx, + nr_requests, 0); + if (set->sched_tags[hctx_idx]) + ret = blk_mq_alloc_rqs(set, set->sched_tags[hctx_idx], + hctx_idx); + if (!set->tags[hctx_idx] || !set->sched_tags[hctx_idx] || ret < 0) { + __blk_mq_free_rq_map_i(set, hctx_idx); + return false; + } + return true; +} + static void blk_mq_map_swqueue(struct request_queue *q, const struct cpumask *online_mask) { @@ -1909,23 +1975,15 @@ static void blk_mq_map_swqueue(struct request_queue *q, hctx_idx = q->mq_map[i]; /* unmapped hw queue can be remapped after CPU topo changed */ - if (!set->tags[hctx_idx]) { - set->tags[hctx_idx] = blk_mq_alloc_rq_map(set, - hctx_idx); - if (blk_mq_alloc_rqs(set, set->tags[hctx_idx], - hctx_idx) < 0) { - blk_mq_free_rq_map(set->tags[hctx_idx]); - set->tags[hctx_idx] = NULL; - } - + if (!set->tags[hctx_idx] && + !__blk_mq_alloc_rq_map_i(set, hctx_idx, q->nr_requests)) { /* * If tags initialization fail for some hctx, * that hctx won't be brought online. In this * case, remap the current ctx to hctx[0] which * is guaranteed to always have tags allocated */ - if (!set->tags[hctx_idx]) - q->mq_map[i] = 0; + q->mq_map[i] = 0; } ctx = per_cpu_ptr(q->queue_ctx, i); @@ -2318,26 +2376,20 @@ static int blk_mq_queue_reinit_prepare(unsigned int cpu) return 0; } -static int __blk_mq_alloc_rq_maps(struct blk_mq_tag_set *set) +static int __blk_mq_alloc_rq_maps(struct blk_mq_tag_set *set, + unsigned int nr_requests) { int i; - for (i = 0; i < set->nr_hw_queues; i++) { - set->tags[i] = blk_mq_alloc_rq_map(set, i); - if (!set->tags[i]) + for (i = 0; i < set->nr_hw_queues; i++) + if (!__blk_mq_alloc_rq_map_i(set, i, nr_requests)) goto out_unwind; - if (blk_mq_alloc_rqs(set, set->tags[i], i) < 0) - goto free_rq_map; - } return 0; out_unwind: - while (--i >= 0) { - blk_mq_free_rqs(set, set->tags[i], i); -free_rq_map: - blk_mq_free_rq_map(set->tags[i]); - } + while (--i >= 0) + __blk_mq_free_rq_map_i(set, i); return -ENOMEM; } @@ -2347,14 +2399,15 @@ static int __blk_mq_alloc_rq_maps(struct blk_mq_tag_set *set) * may reduce the depth asked for, if memory is tight. set->queue_depth * will be updated to reflect the allocated depth. */ -static int blk_mq_alloc_rq_maps(struct blk_mq_tag_set *set) +static int blk_mq_alloc_rq_maps(struct blk_mq_tag_set *set, + unsigned int nr_requests) { unsigned int depth; int err; depth = set->queue_depth; do { - err = __blk_mq_alloc_rq_maps(set); + err = __blk_mq_alloc_rq_maps(set, nr_requests); if (!err) break; @@ -2385,7 +2438,7 @@ static int blk_mq_alloc_rq_maps(struct blk_mq_tag_set *set) */ int blk_mq_alloc_tag_set(struct blk_mq_tag_set *set) { - int ret; + int ret = -ENOMEM; BUILD_BUG_ON(BLK_MQ_MAX_DEPTH > 1 << BLK_MQ_UNIQUE_TAG_BITS); @@ -2425,32 +2478,39 @@ int blk_mq_alloc_tag_set(struct blk_mq_tag_set *set) if (!set->tags) return -ENOMEM; - ret = -ENOMEM; + set->sched_tags = kzalloc_node(nr_cpu_ids * sizeof(struct blk_mq_tags *), + GFP_KERNEL, set->numa_node); + if (!set->sched_tags) + goto free_drv_tags; + set->mq_map = kzalloc_node(sizeof(*set->mq_map) * nr_cpu_ids, GFP_KERNEL, set->numa_node); if (!set->mq_map) - goto out_free_tags; + goto free_sched_tags; if (set->ops->map_queues) ret = set->ops->map_queues(set); else ret = blk_mq_map_queues(set); if (ret) - goto out_free_mq_map; + goto free_mq_map; - ret = blk_mq_alloc_rq_maps(set); + ret = blk_mq_alloc_rq_maps(set, set->queue_depth/*q->nr_requests*/); if (ret) - goto out_free_mq_map; + goto free_mq_map; mutex_init(&set->tag_list_lock); INIT_LIST_HEAD(&set->tag_list); return 0; -out_free_mq_map: +free_mq_map: kfree(set->mq_map); set->mq_map = NULL; -out_free_tags: +free_sched_tags: + kfree(set->sched_tags); + set->sched_tags = NULL; +free_drv_tags: kfree(set->tags); set->tags = NULL; return ret; @@ -2465,12 +2525,16 @@ void blk_mq_free_tag_set(struct blk_mq_tag_set *set) if (set->tags[i]) { blk_mq_free_rqs(set, set->tags[i], i); blk_mq_free_rq_map(set->tags[i]); + blk_mq_free_rq_map(set->sched_tags[i]); } } kfree(set->mq_map); set->mq_map = NULL; + kfree(set->sched_tags); + set->sched_tags = NULL; + kfree(set->tags); set->tags = NULL; } @@ -2482,14 +2546,18 @@ int blk_mq_update_nr_requests(struct request_queue *q, unsigned int nr) struct blk_mq_hw_ctx *hctx; int i, ret; - if (!set || nr > set->queue_depth) + if (!set) return -EINVAL; ret = 0; queue_for_each_hw_ctx(q, hctx, i) { if (!hctx->tags) continue; - ret = blk_mq_tag_update_depth(hctx->tags, nr); + ret = blk_mq_tag_update_depth(hctx->tags, + min(nr, set->queue_depth)); + if (ret) + break; + ret = blk_mq_tag_update_depth(hctx->sched_tags, nr); if (ret) break; } diff --git a/block/blk-mq.h b/block/blk-mq.h index 2e98dd8ccee2..0368c513c2ab 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -31,6 +31,7 @@ void blk_mq_freeze_queue(struct request_queue *q); void blk_mq_free_queue(struct request_queue *q); int blk_mq_update_nr_requests(struct request_queue *q, unsigned int nr); void blk_mq_wake_waiters(struct request_queue *q); +int blk_mq_assign_drv_tag(struct request *rq); bool blk_mq_dispatch_rq_list(struct blk_mq_hw_ctx *, struct list_head *); void blk_mq_flush_busy_ctxs(struct blk_mq_hw_ctx *hctx, struct list_head *list); @@ -41,7 +42,9 @@ void blk_mq_free_rqs(struct blk_mq_tag_set *set, struct blk_mq_tags *tags, unsigned int hctx_idx); void blk_mq_free_rq_map(struct blk_mq_tags *tags); struct blk_mq_tags *blk_mq_alloc_rq_map(struct blk_mq_tag_set *set, - unsigned int hctx_idx); + unsigned int hctx_idx, + unsigned int nr_tags, + unsigned int reserved_tags); int blk_mq_alloc_rqs(struct blk_mq_tag_set *set, struct blk_mq_tags *tags, unsigned int hctx_idx); diff --git a/block/blk-tag.c b/block/blk-tag.c index bae1decb6ec3..319a3a3eb1d7 100644 --- a/block/blk-tag.c +++ b/block/blk-tag.c @@ -272,6 +272,7 @@ void blk_queue_end_tag(struct request_queue *q, struct request *rq) list_del_init(&rq->queuelist); rq->rq_flags &= ~RQF_QUEUED; rq->tag = -1; + rq->sched_tag = -1; if (unlikely(bqt->tag_index[tag] == NULL)) printk(KERN_ERR "%s: tag %d is missing\n", diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 9255ccb043f2..377594bcda8d 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -36,6 +36,7 @@ struct blk_mq_hw_ctx { atomic_t wait_index; struct blk_mq_tags *tags; + struct blk_mq_tags *sched_tags; struct srcu_struct queue_rq_srcu; @@ -72,6 +73,7 @@ struct blk_mq_tag_set { void *driver_data; struct blk_mq_tags **tags; + struct blk_mq_tags **sched_tags; struct mutex tag_list_lock; struct list_head tag_list; diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 7c40fb838b44..112b57bce9e9 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -223,6 +223,7 @@ struct request { void *special; /* opaque pointer available for LLD use */ int tag; + int sched_tag; int errors; /* -- 2.11.0