From patchwork Thu Feb 21 00:20:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 10822889 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 19DAB1805 for ; Thu, 21 Feb 2019 00:21:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 090BF2FC7C for ; Thu, 21 Feb 2019 00:21:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F159D2FC81; Thu, 21 Feb 2019 00:21:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5702A2FC7C for ; Thu, 21 Feb 2019 00:21:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726536AbfBUAVK (ORCPT ); Wed, 20 Feb 2019 19:21:10 -0500 Received: from bombadil.infradead.org ([198.137.202.133]:52392 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726352AbfBUAVK (ORCPT ); Wed, 20 Feb 2019 19:21:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=bY/N6KjHzr5fLbW4UBhsAnN3RpUxiGMPh2dUDTqyXo0=; b=UMOCwPA2rrKyM1LcFZGuK/GZB ke0I5/UUKp/s8SslcyIUHp9e+r3InbL0k7TMLv4Ei6Yby2lDLf4KU+y2mgwimM6H+waZxyBk7E/F9 Cr0cF9sD6T51t2oSsHxEmF/h0LkhN+gEnSxZdkgVEGUzg16I7ylnD1J5FDtCnOYHISgoZG7NJcU3K 0x5yIUKw/iZtR3ItWTcSsVxBV2d7xsObgXIk4YRVSgmPVEWBQa+8W61tfGqn3EqfedW4jp+H5LmEZ 9CKTi072qj9A+8AGMjI1x8n0cRYz2P1GJTNWJQ41YfbqSKOykbjFDEH6kr4qQOe/klrsRNKa1yR3E KhcJKxHLg==; Received: from willy by bombadil.infradead.org with local (Exim 4.90_1 #2 (Red Hat Linux)) id 1gwc6v-0005u1-Cl; Thu, 21 Feb 2019 00:21:09 +0000 From: Matthew Wilcox To: Jason Gunthorpe Cc: Matthew Wilcox , linux-rdma@vger.kernel.org Subject: [PATCH 00/32] Convert the Infiniband subsystem to XArray Date: Wed, 20 Feb 2019 16:20:35 -0800 Message-Id: <20190221002107.22625-1-willy@infradead.org> X-Mailer: git-send-email 2.14.5 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP I intend to remove the IDR and the radix tree interfaces from Linux. Converting each user from either the IDR or radix tree interface varies from the trivial 1:1 replacement to a complete rewrite of the locking. Despite the best efforts of our automated testers (who have caught many of my mistakes), I cannot claim that my conversions of code are free from bugs. Please check these patches over carefully and test them; there may be off-by-one errors, locking mistakes, or various other failures on my part. drivers/infiniband/core/cm.c | 40 +--- drivers/infiniband/core/cma.c | 44 +--- drivers/infiniband/core/mad.c | 39 +--- drivers/infiniband/core/rdma_core.c | 78 +------- drivers/infiniband/core/rdma_core.h | 8 drivers/infiniband/core/sa_query.c | 43 +--- drivers/infiniband/core/ucm.c | 34 +-- drivers/infiniband/core/ucma.c | 82 +++------ drivers/infiniband/core/uverbs.h | 4 drivers/infiniband/core/uverbs_ioctl.c | 67 +------ drivers/infiniband/core/uverbs_uapi.c | 191 +++++++++------------ drivers/infiniband/hw/bnxt_re/ib_verbs.c | 2 drivers/infiniband/hw/cxgb3/iwch.c | 53 ++--- drivers/infiniband/hw/cxgb3/iwch.h | 38 ---- drivers/infiniband/hw/cxgb3/iwch_ev.c | 18 - drivers/infiniband/hw/cxgb3/iwch_mem.c | 2 drivers/infiniband/hw/cxgb3/iwch_provider.c | 16 - drivers/infiniband/hw/cxgb4/cm.c | 67 ++++--- drivers/infiniband/hw/cxgb4/cq.c | 6 drivers/infiniband/hw/cxgb4/device.c | 217 ++++++++++-------------- drivers/infiniband/hw/cxgb4/ev.c | 18 - drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 77 -------- drivers/infiniband/hw/cxgb4/mem.c | 16 - drivers/infiniband/hw/cxgb4/qp.c | 33 +-- drivers/infiniband/hw/hfi1/chip.c | 16 - drivers/infiniband/hw/hfi1/debugfs.c | 8 drivers/infiniband/hw/hfi1/driver.c | 10 - drivers/infiniband/hw/hfi1/hfi.h | 8 drivers/infiniband/hw/hfi1/init.c | 51 ----- drivers/infiniband/hw/hfi1/verbs.c | 8 drivers/infiniband/hw/hfi1/vnic_main.c | 15 - drivers/infiniband/hw/hns/hns_roce_cq.c | 33 +-- drivers/infiniband/hw/hns/hns_roce_device.h | 9 drivers/infiniband/hw/hns/hns_roce_qp.c | 51 +---- drivers/infiniband/hw/mlx4/cm.c | 36 +-- drivers/infiniband/hw/mlx4/cq.c | 2 drivers/infiniband/hw/mlx4/mlx4_ib.h | 5 drivers/infiniband/hw/mlx5/cq.c | 4 drivers/infiniband/hw/mlx5/mr.c | 10 - drivers/infiniband/hw/mlx5/srq.h | 5 drivers/infiniband/hw/mlx5/srq_cmd.c | 27 -- drivers/infiniband/hw/ocrdma/ocrdma_main.c | 16 - drivers/infiniband/hw/qedr/main.c | 10 - drivers/infiniband/hw/qedr/qedr.h | 11 - drivers/infiniband/hw/qedr/qedr_iw_cm.c | 10 - drivers/infiniband/hw/qedr/verbs.c | 36 --- drivers/infiniband/hw/qib/qib.h | 4 drivers/infiniband/hw/qib/qib_driver.c | 20 -- drivers/infiniband/hw/qib/qib_fs.c | 12 - drivers/infiniband/hw/qib/qib_iba7322.c | 4 drivers/infiniband/hw/qib/qib_init.c | 55 +----- drivers/infiniband/ulp/opa_vnic/opa_vnic_vema.c | 60 ++---- 52 files changed, 633 insertions(+), 1096 deletions(-) Substantive interface changes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - The IDR and radix tree required callers to handle their own locking. The XArray embeds a spinlock which is taken for modifications to the data structure; plain lookups occur under the RCU read lock or under the spinlock. - You can take the spinlock yourself (xa_lock() and friends) to protect related data. - idr_alloc() returned -ENOSPC, radix_tree_insert() returned -EEXIST. xa_insert() and xa_alloc() return -EBUSY. - The search keys which the radix tree calls "tags", the XArray calls "marks". - There is no preloading in the XArray API. If your locking is exceptionally complicated, you may need to use xa_reserve(), but there are only 6 callers of xa_reserve(), so it's quite uncommon. - The radix tree provided GFP flags as part of the tree definition; the XArray (like the IDR) passes GFP flags at the point of allocation. - radix_tree_insert() of a NULL pointer was not well-specified. The XArray treats it as reserving the entry (it reads back as NULL but a subsequent xa_insert() to that slot will fail). - xa_alloc_cyclic() returns 1 if the allocation wraps, unlike idr_alloc_cyclic() which provides no indication. - There is no equivalent to idr_for_each(); the xa_for_each() iterator is similar to idr_for_each_entry(). - idr_replace() has no exact equivalent. Some users relied on its exact semantics of only storing if the entry was non-NULL, but all users of idr_replace() were able to use xa_store(). - The family of radix tree gang lookup functions have been replaced with xa_extract(). Matthew Wilcox (32): mlx4: Convert srq_table->tree to XArray mlx5: Convert mlx5_srq_table to XArray mlx5: Convert mkey_table to XArray RDMA/hns: Convert cq_table to XArray RDMA/hns: Convert qp_table_tree to XArray infiniband/core: Convert uverbs to XArray IB/mad: Convert ib_mad_clients to XArray RDMA/cm: Convert local_id_table to XArray mlx4: Convert pv_id_table to XArray uverbs: Convert idr to XArray ib core: Convert query_idr to XArray cxgb3: Convert cqidr to XArray cxgb3: Convert qpidr to XArray cxgb3: Convert mmidr to XArray cxgb4: Convert cqidr to XArray cxgb4: Convert qpidr to XArray cxgb4: Convert mmidr to XArray cxgb4: Convert hwtid_idr to XArray cxgb4: Convert atid_idr to XArray cxgb4: Convert stid_idr to XArray hfi1: Convert hfi1_unit_table to XArray hfi1: Convert vesw_idr to XArray qedr: Convert qpidr to XArray qedr: Convert srqidr to XArray qib: Convert qib_unit_table to XArray opa_vnic: Convert vport_idr to XArray ocrdma: Convert ocrdma_dev_id to IDA ucm: Convert ctx_id_table to XArray ucma: Convert multicast_idr to XArray ucma: Convert ctx_idr to XArray cma: Convert portspace IDRs to XArray ib/bnxt: Remove mention of idr_alloc from comment