From patchwork Sun May 13 18:07:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kalderon, Michal" X-Patchwork-Id: 10396345 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1E649602B1 for ; Sun, 13 May 2018 18:07:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0B2F028DC1 for ; Sun, 13 May 2018 18:07:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EFAFC2906F; Sun, 13 May 2018 18:07:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3994128DC1 for ; Sun, 13 May 2018 18:07:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751928AbeEMSHb (ORCPT ); Sun, 13 May 2018 14:07:31 -0400 Received: from mail-co1nam03on0046.outbound.protection.outlook.com ([104.47.40.46]:52496 "EHLO NAM03-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751878AbeEMSH3 (ORCPT ); Sun, 13 May 2018 14:07:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=TaLuDfPKZUMO2vsl20KRwpO5UBJ5mMam5fJQciITl4E=; b=ka4T7iIGObomn1XUV+aPYpvniVj4U1itJN6By44iXfhCzC/DNAbnBJGH7/jknH347nY+SNfXHONK/keoXEzci32NFmw49tzTVCYbHd4M9Cirp4RD2LgFvmLCOxGTVIBNTEsk/fFmkmkziOBb3ppwEl5bP5z1/k9mOTK2XV0vheU= Received: from SN4PR0701CA0001.namprd07.prod.outlook.com (2603:10b6:803:28::11) by BN3PR0701MB1218.namprd07.prod.outlook.com (2a01:111:e400:4012::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.755.16; Sun, 13 May 2018 18:07:26 +0000 Received: from BL2FFO11FD025.protection.gbl (2a01:111:f400:7c09::152) by SN4PR0701CA0001.outlook.office365.com (2603:10b6:803:28::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.755.16 via Frontend Transport; Sun, 13 May 2018 18:07:26 +0000 Authentication-Results: spf=pass (sender IP is 50.232.66.26) smtp.mailfrom=cavium.com; vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=bestguesspass action=none header.from=cavium.com; Received-SPF: Pass (protection.outlook.com: domain of cavium.com designates 50.232.66.26 as permitted sender) receiver=protection.outlook.com; client-ip=50.232.66.26; helo=CAEXCH02.caveonetworks.com; Received: from CAEXCH02.caveonetworks.com (50.232.66.26) by BL2FFO11FD025.mail.protection.outlook.com (10.173.161.104) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.20.735.16 via Frontend Transport; Sun, 13 May 2018 18:07:26 +0000 Received: from lb-tlvb-michal.il.qlogic.com.com (10.185.6.89) by CAEXCH02.caveonetworks.com (10.17.4.29) with Microsoft SMTP Server id 14.2.347.0; Sun, 13 May 2018 11:07:23 -0700 From: Michal Kalderon To: , , CC: , , Michal Kalderon , Ariel Elior Subject: [PATCH for-rc] RDMA/qedr: Fix doorbell bar mapping for dpi > 1 Date: Sun, 13 May 2018 21:07:07 +0300 Message-ID: <20180513180707.22003-1-Michal.Kalderon@cavium.com> X-Mailer: git-send-email 2.14.3 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:50.232.66.26; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(39850400004)(376002)(39380400002)(346002)(396003)(2980300002)(438002)(189003)(199004)(476003)(2616005)(1076002)(2906002)(54906003)(86362001)(575784001)(50226002)(110136005)(486006)(126002)(336012)(4326008)(16586007)(81166006)(106002)(8936002)(2201001)(26005)(72206003)(356003)(77096007)(59450400001)(8676002)(305945005)(316002)(81156014)(186003)(478600001)(51416003)(5660300001)(36756003)(106466001)(69596002)(47776003)(48376002)(50466002)(107886003)(6666003)(104016004); DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR0701MB1218; H:CAEXCH02.caveonetworks.com; FPR:; SPF:Pass; LANG:en; PTR:50-232-66-26-static.hfc.comcastbusiness.net; MX:1; A:1; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD025; 1:h57PMR2PMJyRUGQZoh3sAaBeFabDiUVikgeRBgvv48e0mdWbVepYY0PURfBl61cikjKbnkyh+oOqriK7dYc2oXCNsJVpaTRU5AqjqlafWB0hZbwW7hGhgDCExkPzPIV0 X-MS-PublicTrafficType: Email X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(4534165)(4627221)(201703031133081)(201702281549075)(5600026)(4608076)(2017052603328)(7153060); SRVR:BN3PR0701MB1218; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1218; 3:HdFA7OWBBsbfyS4+IuWkki0Q6YvPDqmjNo4rqRYwvi5OrJkm8lGZuQHVJdSQNF43Dop9e8XklUaaLzKCXqq9/dAgtCok+9ydAvZqNOLXR3q0UcncV3ICZz9fkK6DUwCHlcHZkdOvw6PJAND08gCGD9ql3f8vrFSAnQ/W9jRWaSm0PhhbxFpSKqYr7JYynYSLnGLNwPvLLL8PVuFio89s7bfdpuFA/HDcxCVbZH0I+vbKZ3RbspSr4sMHwRis7HPQd3FrRCNteq4dWAHR9XbK+kRV4C8rjeijcmwPSJMjnZQ0KFCnKZe4vG+I3zHbD6aXqnXBP48clZ73BZG1RoIc7XIJVglYyYAppDiAEzUJQw0=; 25:rGNeoTCJSbTV3sJTxv8iS0eGdQcVz9BE+0aLISIIDY1LnSNZhlPyye+Eul2UIiq2WBgZmkpkt01Siq0n4j0k//8FgxoQoPby4fSBv6HPHCLcGkKyQsoblD641eTfd0F7iPbuY15pVCLKPTn+OZzaC9oQl50GwAnvfAc3TBpcSXl+NYQYaGb693aAbWcUJLRVWtmLqSn8p8awFerqgP8n8MIPh29flFZ5Or7+3tJnnZOks1Y7Fh45FDpRcU4HCeeaYXpsTXVDCgU/eNF0hGtyjyj4Fd6ed77xjoaLwDOOy4dOhIlZ8TEdEFM5itRNyP0Y68Vw54kepOlW+htnC3CdVw== X-MS-TrafficTypeDiagnostic: BN3PR0701MB1218: X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1218; 31:NMuAHcTl6CgsjYYi36qhDTePxKlF3x/5D98H4pABs8w8kaVhfxXzUTU1zVj1ee7UkSSuxUIUPEZtad1Y7Gx9qUkSdtq9e/B6CLRDAd2nfv7WPAgUza4LZm9vcyGbacYZUC74vgRo3VOAYcXkbPCGWaQlt1cflETldttX7APCPmKhv6UhwN88w6mw2WUN5eAqf3LWM5fmEs4xeDCQUD2pDQrZ0XMjWN8CK2YfCXqo0RQ=; 20:udCETd6hxpt9zqvUnNUeTUVGNkLoy0NZxCwkUZQhmz12piusC1rpFw/hKfFDF9e5WJuG/podB8pYDQ1sJApmzZzOQKegpHDVKYSriD2t+fqW/2ndz7oeZO+SDFNS8UThG8zFyxZwN3FU/4DMrI+JR9NDLQB5mAd4zye9BH2xE3rtYDbV+GYHkuWUUhkUcW8e1YJIZtYZGlj77bN+PYopXq8PFZxbkipZ6iibgtDeAXtzce6I+weDgPL9JsSqKbhfI3ksANDoTMNqsRjlz1ljFiSVJyaM/lQSuYpho/OZRoFFW65LIy2JHZEA893EWIICFNEIMpPr0ltCvwBx/O6wlbhm4wFPEMHoCMg0W+6kyRqmaGj31CJpgkmHAPGkyoMcbUSj3TMGyzRqyojWpCezJhbUHTxe/5xdqgWCO4n6gg3ZXWVZA//Hae/TCrDiXE3mICgk3e8z0xoWJNlU1MsT/T9OQozX0+V9Tkg+cLx+bbsOfO1m4qgSP+BMCh3Ao4yO X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(278428928389397); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3231254)(944501410)(52105095)(93006095)(93004095)(3002001)(10201501046)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123560045)(20161123562045)(20161123564045)(6072148)(201708071742011); SRVR:BN3PR0701MB1218; BCL:0; PCL:0; RULEID:; SRVR:BN3PR0701MB1218; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1218; 4:0QIx/Fsc0SfuPgRUyl6yewnr+M/6+eh87irCmh4OtSSWb/2ptcOpB8uPW/1Nxm2DEXbgd789tVLkBWw5vaXyX2+KUPhHhrSeQH1UXp871D9HXxAGhmy/biUJe+fy9Wchvn7SrY21vEUsssdRs9CAa9PzmV0JvC9gAxWToOc3pclHaCfRZYaYQ02q0KsuyOxi/wTkFdhMz3pU/2Jk3JH0+La51YvAkIucafupnZ8TdCNbJ9lm75gWPWy514fi+1M/YB567AyMf6zOW1mEGd0jKD1h3iCljvkmXHAk5gejWwKwQh55xmso/CdgLBHQrNm9 X-Forefront-PRVS: 0671F32598 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR0701MB1218; 23:010wPiAhmdtrkS5nrIiY50vsz7x8PE3juiq1qgY?= =?us-ascii?Q?fNFFJQCJvOK7dQ+O6MFS7I6BE+CteHX1L9ClUlkug5llTox0WjmghnkHdFBC?= =?us-ascii?Q?PE5dRM/bTFqoQnnvKzoVUL2XM8MpH4yvgMKAv650MAzi/xet8xPU6V7jhGDv?= =?us-ascii?Q?+CtMA9ocQkRsx9WRff0fZQVU39WClGqu67ecJhYxo+CJPX2zALq6XXZJpbUj?= =?us-ascii?Q?0z0WWfmrYBt5ZNsM4/HWBElF8eGAk+pgoHx0Tw5JTUd1zs1utJNutV0IYP43?= =?us-ascii?Q?YISAHYENl3prPEkZ3ANvFw1Y3GKNbrPDxuk3uvfYCuR7tav8JRY9BumZLfCl?= =?us-ascii?Q?RVPCxdVvJkoqAW+Djx89aFQ1E+NCE++fD8bzaCZHW79jD8XGahsaLbKaGX5l?= =?us-ascii?Q?jCCzt2gIQ65DALEXoeYtg7Tvwa6BXTt9S5qOJhlhvqSArYRtIoizi88s3fdC?= =?us-ascii?Q?WUVswiBuNDu3BFlSlFZ8tAt7ayzzz7OlOX5hdsjR8ab1yTI246hR3wotp30q?= =?us-ascii?Q?rmSpZVatOH2RfbmOyRpelwHdpMhH738sJa93B+Bcpen98NwosUJzpbJiQM6i?= =?us-ascii?Q?hDo+DEl9H8HWhIynrmvwVNdvQ4nn+NJdw0FLwzp61bwvXiudnGiA2NAz/PFO?= =?us-ascii?Q?x27EUIQP8/2WzBhRvHl/S54bimLE0Un3ikzEm0HLHBppIuDbOk+HPIkIy7kD?= =?us-ascii?Q?eaOwrxwnCXJipc6AEsUmVUp7yoiByBbUneHNNDgSreiI3EijXpDQpRTzF6mt?= =?us-ascii?Q?DIzvgJN8CwXcnkbe/LkEifNUMekaUcqv4UPhFUa9zLC8irsG28erTytINSGE?= =?us-ascii?Q?i9L+OAWXTKuPkxWEPKTHtlBt7QjLK+Eid1yRKVejdPjDCT8IDklomcHxKZll?= =?us-ascii?Q?7hF492ZBmDpBsf/3ynXiKXzwjT0c4CaBG92La8J03iKWk4foXQ3IglM5mGZU?= =?us-ascii?Q?rTRJK0ZN5SLBEzO8atzz8yqdOG426rptvzVRqkdTgIDprRJJUy4uNAFdA12r?= =?us-ascii?Q?sgwkIGHd6EFA/IE/e1NLraE2/rY+quln7l2NGFefbAJk8tXoZseYevB9hXhV?= =?us-ascii?Q?kZvHIqDBYCAbB3zxcGNMdFjMsnZKvyoZa4/jMY+ByQnWRR0JxCx9sRfPoELH?= =?us-ascii?Q?Yito/fkYk/P0=3D?= X-Microsoft-Antispam-Message-Info: 6dmbWH6t3ihJfFnnp8ZYYNc5SSaeRLh50CrZqvLf9iFYSAPo3c8d82zlpuAoq83NigRX4/0TkN5LlHiA21vUjRZnUvu5zaqhUKW6E9wNx5nKKnkf2VB4L7EeUfg9ijxVFuNBJlFvtLMTTxCkrrpfDcqjfhmaqrKupogyJCIJztGcS2E7mjf4lRYxSTTf//MJ X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1218; 6:ph0IiQPJDJywqVpXJEHcLniJ2WODWVAGZbq9EIvXLAACcQwSFEClFn5ci39IFiWomnAzcunwQvCyM9jFFoCsPucqxThJJ2XQSP0Mt2nLLfoU1jNwLZ1ahvNjaTjiLIHzmFU2GvObpslD9oTzgVk237BVt6tlCBNLz4rNmjG2y+092j6uRvDF83bUklsCqnZOmW83OcwL2362Qr6ybafIzyjcZf5XzVIGs+uZqj+aZ/Oc7YP/ynjegd7BMZhc2SWyUCDH6HxxkPVqRHFaHbgcuo0T5AV2uN6ytTfzsVCl8ZXn8vE2hrfZNkp1DLe3aEjuSVjmzIU/QnZBaOCRxdDSgNV1Muip47Wn6YRSZ+vZ172TqSZOj3MqAzl5eh3qTuXzeL9LE4QvqyaNAwJK4Yspvs47IPq/yilRF1GgBmbyippnU3u2BjdG/R6e+mwCcprOCItNjxwVz4tkbl3VinUlOQ==; 5:W3jmy3N69U6vtkTnYThPnD3EQ5yI7etzYMgGifE9aNLMTk63pwPiKaw2rfaUIAsJqCVfpP5PK+ctfx6KW5TPQS1ZPI96a+2vwZdnvNes6ihylSudsmmURXIpecyVeGrWibXCbB1popNjl09vPw0W6qBZy5zyyv0k0qlRzNDny4Q=; 24:v47SCmvTsJJz4xpxjlsI57X0joBEohbisT6jCKZrB8H4VYlXI+Upnu+4pzuz3jpVKjoC/lQP3vsvJvXQjaqStizDwF3NieREUSPP0q0ED3Q= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1218; 7:vYiforyUi7HuuEGct/jTDUxc2hi2SHxudAhy0JodLfMxQdYqelTRYy29tlZj72Ygjuj1R+0vrtMwr2VWcgRdkwHDZfwDE6TsRHZg6u42e4IulR3/yH+aRhvEigrML5Et2IOQtSUaRuldo90MXWP5EUq28hw4cmCoJDDov72swRlcDL1/VN7a595fcQzp/5sA0CQW7pUqhmgxrkKiTe4XfJ+/dOSb+O82RFgvedh6DcRZnZE8imPfAs2uWZvX2IGI X-MS-Office365-Filtering-Correlation-Id: 5983cc5e-dc3e-4c6c-8617-08d5b8fc6264 X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 May 2018 18:07:26.1229 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5983cc5e-dc3e-4c6c-8617-08d5b8fc6264 X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=711e4ccf-2e9b-4bcf-a551-4094005b6194; Ip=[50.232.66.26]; Helo=[CAEXCH02.caveonetworks.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0701MB1218 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Each user_context receives a separate dpi value and thus a different address on the doorbell bar. The qedr_mmap function needs to validate the address and map the doorbell bar accordingly. The current implementation always checked against dpi=0 doorbell range leading to a wrong mapping for doorbell bar. (It entered an else case that mapped the address differently). qedr_mmap should only be used for doorbells, so the else was actually wrong in the first place. This only has an affect on arm architecture and not an issue on a x86 based architecture. This lead to doorbells not occurring on arm based systems and left applications that use more than one dpi (or several applications run simultaneously ) to hang. Fixes: ac1b36e55a51 ("qedr: Add support for user context verbs") Signed-off-by: Ariel Elior Signed-off-by: Michal Kalderon --- drivers/infiniband/hw/qedr/verbs.c | 59 ++++++++++++++++++-------------------- 1 file changed, 28 insertions(+), 31 deletions(-) diff --git a/drivers/infiniband/hw/qedr/verbs.c b/drivers/infiniband/hw/qedr/verbs.c index 7d3763b..27c0f00 100644 --- a/drivers/infiniband/hw/qedr/verbs.c +++ b/drivers/infiniband/hw/qedr/verbs.c @@ -401,49 +401,46 @@ int qedr_mmap(struct ib_ucontext *context, struct vm_area_struct *vma) { struct qedr_ucontext *ucontext = get_qedr_ucontext(context); struct qedr_dev *dev = get_qedr_dev(context->device); - unsigned long vm_page = vma->vm_pgoff << PAGE_SHIFT; - u64 unmapped_db = dev->db_phys_addr; + unsigned long phys_addr = vma->vm_pgoff << PAGE_SHIFT; unsigned long len = (vma->vm_end - vma->vm_start); - int rc = 0; - bool found; + unsigned long dpi_start; + + dpi_start = dev->db_phys_addr + (ucontext->dpi * ucontext->dpi_size); DP_DEBUG(dev, QEDR_MSG_INIT, - "qedr_mmap called vm_page=0x%lx vm_pgoff=0x%lx unmapped_db=0x%llx db_size=%x, len=%lx\n", - vm_page, vma->vm_pgoff, unmapped_db, dev->db_size, len); - if (vma->vm_start & (PAGE_SIZE - 1)) { - DP_ERR(dev, "Vma_start not page aligned = %ld\n", - vma->vm_start); + "mmap invoked with vm_start=0x%lx, vm_end=0x%lx,vm_pgoff=0x%lx; dpi_start=0x%lx dpi_size=0x%x\n", + vma->vm_start, vma->vm_end, vma->vm_pgoff, dpi_start, + ucontext->dpi_size); + + if ((vma->vm_start & (PAGE_SIZE - 1)) || (len & (PAGE_SIZE - 1))) { + DP_ERR(dev, + "failed mmap, adrresses must be page aligned: start=0x%lx, end=0x%lx\n", + vma->vm_start, vma->vm_end); return -EINVAL; } - found = qedr_search_mmap(ucontext, vm_page, len); - if (!found) { - DP_ERR(dev, "Vma_pgoff not found in mapped array = %ld\n", + if (!qedr_search_mmap(ucontext, phys_addr, len)) { + DP_ERR(dev, "failed mmap, vm_pgoff=0x%lx is not authorized\n", vma->vm_pgoff); return -EINVAL; } - DP_DEBUG(dev, QEDR_MSG_INIT, "Mapping doorbell bar\n"); - - if ((vm_page >= unmapped_db) && (vm_page <= (unmapped_db + - dev->db_size))) { - DP_DEBUG(dev, QEDR_MSG_INIT, "Mapping doorbell bar\n"); - if (vma->vm_flags & VM_READ) { - DP_ERR(dev, "Trying to map doorbell bar for read\n"); - return -EPERM; - } - - vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot); + if (phys_addr < dpi_start || + ((phys_addr + len) > (dpi_start + ucontext->dpi_size))) { + DP_ERR(dev, + "failed mmap, pages are outside of dpi; page address=0x%lx, dpi_start=0x%lx, dpi_size=0x%x\n", + phys_addr, dpi_start, ucontext->dpi_size); + return -EINVAL; + } - rc = io_remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff, - PAGE_SIZE, vma->vm_page_prot); - } else { - DP_DEBUG(dev, QEDR_MSG_INIT, "Mapping chains\n"); - rc = remap_pfn_range(vma, vma->vm_start, - vma->vm_pgoff, len, vma->vm_page_prot); + if (vma->vm_flags & VM_READ) { + DP_ERR(dev, "failed mmap, cannot map doorbell bar for read\n"); + return -EINVAL; } - DP_DEBUG(dev, QEDR_MSG_INIT, "qedr_mmap return code: %d\n", rc); - return rc; + + vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot); + return io_remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff, len, + vma->vm_page_prot); } struct ib_pd *qedr_alloc_pd(struct ib_device *ibdev,