Message ID | 1543049362-97091-3-git-send-email-oulijun@huawei.com (mailing list archive) |
---|---|
State | Accepted |
Delegated to: | Jason Gunthorpe |
Headers | show |
Series | SRQ support for hip08 | expand |
On Sat, Nov 24, 2018 at 04:49:20PM +0800, Lijun Ou wrote: > diff --git a/drivers/infiniband/hw/hns/hns_roce_srq.c b/drivers/infiniband/hw/hns/hns_roce_srq.c > new file mode 100644 > index 0000000..d8a8613 > +++ b/drivers/infiniband/hw/hns/hns_roce_srq.c > @@ -0,0 +1,26 @@ > +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB > +/* > + * Copyright (c) 2018 Hisilicon Limited. > + */ > + > +#include <rdma/ib_umem.h> > +#include <rdma/hns-abi.h> > +#include "hns_roce_device.h" > +#include "hns_roce_cmd.h" > +#include "hns_roce_hem.h" > + > +int hns_roce_init_srq_table(struct hns_roce_dev *hr_dev) > +{ > + struct hns_roce_srq_table *srq_table = &hr_dev->srq_table; > + > + xa_init(&srq_table->xa); Is a call to xa_destroy missing? Jason
在 2018/11/30 8:06, Jason Gunthorpe 写道: > On Sat, Nov 24, 2018 at 04:49:20PM +0800, Lijun Ou wrote: > >> diff --git a/drivers/infiniband/hw/hns/hns_roce_srq.c b/drivers/infiniband/hw/hns/hns_roce_srq.c >> new file mode 100644 >> index 0000000..d8a8613 >> +++ b/drivers/infiniband/hw/hns/hns_roce_srq.c >> @@ -0,0 +1,26 @@ >> +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB >> +/* >> + * Copyright (c) 2018 Hisilicon Limited. >> + */ >> + >> +#include <rdma/ib_umem.h> >> +#include <rdma/hns-abi.h> >> +#include "hns_roce_device.h" >> +#include "hns_roce_cmd.h" >> +#include "hns_roce_hem.h" >> + >> +int hns_roce_init_srq_table(struct hns_roce_dev *hr_dev) >> +{ >> + struct hns_roce_srq_table *srq_table = &hr_dev->srq_table; >> + >> + xa_init(&srq_table->xa); > Is a call to xa_destroy missing? > > Jason > No. It is not needed. I have analyzed the usage again according to the extremely example
在 2018/11/30 8:06, Jason Gunthorpe 写道: > On Sat, Nov 24, 2018 at 04:49:20PM +0800, Lijun Ou wrote: > >> diff --git a/drivers/infiniband/hw/hns/hns_roce_srq.c b/drivers/infiniband/hw/hns/hns_roce_srq.c >> new file mode 100644 >> index 0000000..d8a8613 >> +++ b/drivers/infiniband/hw/hns/hns_roce_srq.c >> @@ -0,0 +1,26 @@ >> +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB >> +/* >> + * Copyright (c) 2018 Hisilicon Limited. >> + */ >> + >> +#include <rdma/ib_umem.h> >> +#include <rdma/hns-abi.h> >> +#include "hns_roce_device.h" >> +#include "hns_roce_cmd.h" >> +#include "hns_roce_hem.h" >> + >> +int hns_roce_init_srq_table(struct hns_roce_dev *hr_dev) >> +{ >> + struct hns_roce_srq_table *srq_table = &hr_dev->srq_table; >> + >> + xa_init(&srq_table->xa); > Is a call to xa_destroy missing? > > Jason Hi, Jason It uses xarray instead radix tree. the orign code as follows: spin_lock_int(&srq_table->lock); INIT_RADIX_TREE(&srq_table->tree, GFP_ATOMIC); So, we should use xa_init instead of it. I think that it should not use xa_destroy. thanks Lijun Ou >
On Fri, Nov 30, 2018 at 11:04:07AM +0800, oulijun wrote: > 在 2018/11/30 8:06, Jason Gunthorpe 写道: > > On Sat, Nov 24, 2018 at 04:49:20PM +0800, Lijun Ou wrote: > > > >> diff --git a/drivers/infiniband/hw/hns/hns_roce_srq.c b/drivers/infiniband/hw/hns/hns_roce_srq.c > >> new file mode 100644 > >> index 0000000..d8a8613 > >> +++ b/drivers/infiniband/hw/hns/hns_roce_srq.c > >> @@ -0,0 +1,26 @@ > >> +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB > >> +/* > >> + * Copyright (c) 2018 Hisilicon Limited. > >> + */ > >> + > >> +#include <rdma/ib_umem.h> > >> +#include <rdma/hns-abi.h> > >> +#include "hns_roce_device.h" > >> +#include "hns_roce_cmd.h" > >> +#include "hns_roce_hem.h" > >> + > >> +int hns_roce_init_srq_table(struct hns_roce_dev *hr_dev) > >> +{ > >> + struct hns_roce_srq_table *srq_table = &hr_dev->srq_table; > >> + > >> + xa_init(&srq_table->xa); > > Is a call to xa_destroy missing? > > > > Jason > Hi, Jason > It uses xarray instead radix tree. the orign code as follows: > spin_lock_int(&srq_table->lock); > INIT_RADIX_TREE(&srq_table->tree, GFP_ATOMIC); > > So, we should use xa_init instead of it. I think that it should not use xa_destroy. Well, radix tree didn't have a destroy, it was considered destroyed when empty, while xarray does have a destroy. Seems reasonable to call it even if you know the xarray is empty? Matthew? Jason
在 2018/11/30 12:42, Jason Gunthorpe 写道: > On Fri, Nov 30, 2018 at 11:04:07AM +0800, oulijun wrote: >> 在 2018/11/30 8:06, Jason Gunthorpe 写道: >>> On Sat, Nov 24, 2018 at 04:49:20PM +0800, Lijun Ou wrote: >>> >>>> diff --git a/drivers/infiniband/hw/hns/hns_roce_srq.c b/drivers/infiniband/hw/hns/hns_roce_srq.c >>>> new file mode 100644 >>>> index 0000000..d8a8613 >>>> +++ b/drivers/infiniband/hw/hns/hns_roce_srq.c >>>> @@ -0,0 +1,26 @@ >>>> +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB >>>> +/* >>>> + * Copyright (c) 2018 Hisilicon Limited. >>>> + */ >>>> + >>>> +#include <rdma/ib_umem.h> >>>> +#include <rdma/hns-abi.h> >>>> +#include "hns_roce_device.h" >>>> +#include "hns_roce_cmd.h" >>>> +#include "hns_roce_hem.h" >>>> + >>>> +int hns_roce_init_srq_table(struct hns_roce_dev *hr_dev) >>>> +{ >>>> + struct hns_roce_srq_table *srq_table = &hr_dev->srq_table; >>>> + >>>> + xa_init(&srq_table->xa); >>> Is a call to xa_destroy missing? >>> >>> Jason >> Hi, Jason >> It uses xarray instead radix tree. the orign code as follows: >> spin_lock_int(&srq_table->lock); >> INIT_RADIX_TREE(&srq_table->tree, GFP_ATOMIC); >> >> So, we should use xa_init instead of it. I think that it should not use xa_destroy. > Well, radix tree didn't have a destroy, it was considered destroyed > when empty, while xarray does have a destroy. Seems reasonable to call > it even if you know the xarray is empty? Matthew? > > Jason > Hi, Matthew I think that it is right for jason's views. I check the all patches from the address from jason again. all places is not call the xa_destroy. what is your opinion? thanks Lijun Ou
On Tue, Dec 04, 2018 at 07:38:03PM +0800, oulijun wrote: > 在 2018/11/30 12:42, Jason Gunthorpe 写道: > > On Fri, Nov 30, 2018 at 11:04:07AM +0800, oulijun wrote: > >> 在 2018/11/30 8:06, Jason Gunthorpe 写道: > >>> Is a call to xa_destroy missing? > >>> > >>> Jason > >> Hi, Jason > >> It uses xarray instead radix tree. the orign code as follows: > >> spin_lock_int(&srq_table->lock); > >> INIT_RADIX_TREE(&srq_table->tree, GFP_ATOMIC); > >> > >> So, we should use xa_init instead of it. I think that it should not use xa_destroy. > > Well, radix tree didn't have a destroy, it was considered destroyed > > when empty, while xarray does have a destroy. Seems reasonable to call > > it even if you know the xarray is empty? Matthew? > > > > Jason > > > Hi, Matthew > I think that it is right for jason's views. I check the all patches from the address from jason again. all places > is not call the xa_destroy. what is your opinion? There's no need to call xa_destroy() if you know the XArray is empty. It's just a convenience function.
On Tue, Dec 04, 2018 at 04:11:27AM -0800, Matthew Wilcox wrote: > On Tue, Dec 04, 2018 at 07:38:03PM +0800, oulijun wrote: > > 在 2018/11/30 12:42, Jason Gunthorpe 写道: > > > On Fri, Nov 30, 2018 at 11:04:07AM +0800, oulijun wrote: > > >> 在 2018/11/30 8:06, Jason Gunthorpe 写道: > > >>> Is a call to xa_destroy missing? > > >>> > > >>> Jason > > >> Hi, Jason > > >> It uses xarray instead radix tree. the orign code as follows: > > >> spin_lock_int(&srq_table->lock); > > >> INIT_RADIX_TREE(&srq_table->tree, GFP_ATOMIC); > > >> > > >> So, we should use xa_init instead of it. I think that it should not use xa_destroy. > > > Well, radix tree didn't have a destroy, it was considered destroyed > > > when empty, while xarray does have a destroy. Seems reasonable to call > > > it even if you know the xarray is empty? Matthew? > > > > > > Jason > > > > > Hi, Matthew > > I think that it is right for jason's views. I check the all patches from the address from jason again. all places > > is not call the xa_destroy. what is your opinion? > > There's no need to call xa_destroy() if you know the XArray is empty. > It's just a convenience function. Okay then, series applied to for-next thanks Jason
diff --git a/drivers/infiniband/hw/hns/Makefile b/drivers/infiniband/hw/hns/Makefile index cf03404..004c88b 100644 --- a/drivers/infiniband/hw/hns/Makefile +++ b/drivers/infiniband/hw/hns/Makefile @@ -7,7 +7,7 @@ ccflags-y := -Idrivers/net/ethernet/hisilicon/hns3 obj-$(CONFIG_INFINIBAND_HNS) += hns-roce.o hns-roce-objs := hns_roce_main.o hns_roce_cmd.o hns_roce_pd.o \ hns_roce_ah.o hns_roce_hem.o hns_roce_mr.o hns_roce_qp.o \ - hns_roce_cq.o hns_roce_alloc.o hns_roce_db.o + hns_roce_cq.o hns_roce_alloc.o hns_roce_db.o hns_roce_srq.o obj-$(CONFIG_INFINIBAND_HNS_HIP06) += hns-roce-hw-v1.o hns-roce-hw-v1-objs := hns_roce_hw_v1.o obj-$(CONFIG_INFINIBAND_HNS_HIP08) += hns-roce-hw-v2.o diff --git a/drivers/infiniband/hw/hns/hns_roce_alloc.c b/drivers/infiniband/hw/hns/hns_roce_alloc.c index 46f65f9..6300033 100644 --- a/drivers/infiniband/hw/hns/hns_roce_alloc.c +++ b/drivers/infiniband/hw/hns/hns_roce_alloc.c @@ -239,6 +239,8 @@ int hns_roce_buf_alloc(struct hns_roce_dev *hr_dev, u32 size, u32 max_direct, void hns_roce_cleanup_bitmap(struct hns_roce_dev *hr_dev) { + if (hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_SRQ) + hns_roce_cleanup_srq_table(hr_dev); hns_roce_cleanup_qp_table(hr_dev); hns_roce_cleanup_cq_table(hr_dev); hns_roce_cleanup_mr_table(hr_dev); diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h index 42ff400..5d79d80 100644 --- a/drivers/infiniband/hw/hns/hns_roce_device.h +++ b/drivers/infiniband/hw/hns/hns_roce_device.h @@ -205,6 +205,8 @@ enum { enum hns_roce_mtt_type { MTT_TYPE_WQE, MTT_TYPE_CQE, + MTT_TYPE_SRQWQE, + MTT_TYPE_IDX }; enum { @@ -340,6 +342,10 @@ struct hns_roce_mr_table { struct hns_roce_hem_table mtpt_table; struct hns_roce_buddy mtt_cqe_buddy; struct hns_roce_hem_table mtt_cqe_table; + struct hns_roce_buddy mtt_srqwqe_buddy; + struct hns_roce_hem_table mtt_srqwqe_table; + struct hns_roce_buddy mtt_idx_buddy; + struct hns_roce_hem_table mtt_idx_table; }; struct hns_roce_wq { @@ -454,6 +460,12 @@ struct hns_roce_cq_table { struct hns_roce_hem_table table; }; +struct hns_roce_srq_table { + struct hns_roce_bitmap bitmap; + struct xarray xa; + struct hns_roce_hem_table table; +}; + struct hns_roce_raq_table { struct hns_roce_buf_list *e_raq_buf; }; @@ -680,6 +692,8 @@ struct hns_roce_caps { u32 max_extend_sg; int num_qps; /* 256k */ int reserved_qps; + u32 max_srq_sg; + int num_srqs; u32 max_wqes; /* 16k */ u32 max_srqs; u32 max_srq_wrs; @@ -694,12 +708,16 @@ struct hns_roce_caps { int min_cqes; u32 min_wqes; int reserved_cqs; + int reserved_srqs; + u32 max_srqwqes; int num_aeq_vectors; /* 1 */ int num_comp_vectors; int num_other_vectors; int num_mtpts; u32 num_mtt_segs; u32 num_cqe_segs; + u32 num_srqwqe_segs; + u32 num_idx_segs; int reserved_mrws; int reserved_uars; int num_pds; @@ -713,6 +731,8 @@ struct hns_roce_caps { int irrl_entry_sz; int trrl_entry_sz; int cqc_entry_sz; + int srqc_entry_sz; + int idx_entry_sz; u32 pbl_ba_pg_sz; u32 pbl_buf_pg_sz; u32 pbl_hop_num; @@ -843,6 +863,7 @@ struct hns_roce_dev { struct hns_roce_uar_table uar_table; struct hns_roce_mr_table mr_table; struct hns_roce_cq_table cq_table; + struct hns_roce_srq_table srq_table; struct hns_roce_qp_table qp_table; struct hns_roce_eq_table eq_table; @@ -955,12 +976,14 @@ int hns_roce_buf_write_mtt(struct hns_roce_dev *hr_dev, int hns_roce_init_eq_table(struct hns_roce_dev *hr_dev); int hns_roce_init_cq_table(struct hns_roce_dev *hr_dev); int hns_roce_init_qp_table(struct hns_roce_dev *hr_dev); +int hns_roce_init_srq_table(struct hns_roce_dev *hr_dev); void hns_roce_cleanup_pd_table(struct hns_roce_dev *hr_dev); void hns_roce_cleanup_mr_table(struct hns_roce_dev *hr_dev); void hns_roce_cleanup_eq_table(struct hns_roce_dev *hr_dev); void hns_roce_cleanup_cq_table(struct hns_roce_dev *hr_dev); void hns_roce_cleanup_qp_table(struct hns_roce_dev *hr_dev); +void hns_roce_cleanup_srq_table(struct hns_roce_dev *hr_dev); int hns_roce_bitmap_alloc(struct hns_roce_bitmap *bitmap, unsigned long *obj); void hns_roce_bitmap_free(struct hns_roce_bitmap *bitmap, unsigned long obj, diff --git a/drivers/infiniband/hw/hns/hns_roce_hem.c b/drivers/infiniband/hw/hns/hns_roce_hem.c index f6faefe..388b040 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hem.c +++ b/drivers/infiniband/hw/hns/hns_roce_hem.c @@ -1041,6 +1041,15 @@ void hns_roce_cleanup_hem_table(struct hns_roce_dev *hr_dev, void hns_roce_cleanup_hem(struct hns_roce_dev *hr_dev) { + if ((hr_dev->caps.num_idx_segs)) + hns_roce_cleanup_hem_table(hr_dev, + &hr_dev->mr_table.mtt_idx_table); + if (hr_dev->caps.num_srqwqe_segs) + hns_roce_cleanup_hem_table(hr_dev, + &hr_dev->mr_table.mtt_srqwqe_table); + if (hr_dev->caps.srqc_entry_sz) + hns_roce_cleanup_hem_table(hr_dev, + &hr_dev->srq_table.table); hns_roce_cleanup_hem_table(hr_dev, &hr_dev->cq_table.table); if (hr_dev->caps.trrl_entry_sz) hns_roce_cleanup_hem_table(hr_dev, diff --git a/drivers/infiniband/hw/hns/hns_roce_hem.h b/drivers/infiniband/hw/hns/hns_roce_hem.h index e8850d5..a650278 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hem.h +++ b/drivers/infiniband/hw/hns/hns_roce_hem.h @@ -48,6 +48,8 @@ enum { /* UNMAP HEM */ HEM_TYPE_MTT, HEM_TYPE_CQE, + HEM_TYPE_SRQWQE, + HEM_TYPE_IDX, HEM_TYPE_IRRL, HEM_TYPE_TRRL, }; diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c index 2fec1f2..215f5ca 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c @@ -1276,11 +1276,14 @@ static int hns_roce_v2_profile(struct hns_roce_dev *hr_dev) caps->num_qps = HNS_ROCE_V2_MAX_QP_NUM; caps->max_wqes = HNS_ROCE_V2_MAX_WQE_NUM; caps->num_cqs = HNS_ROCE_V2_MAX_CQ_NUM; + caps->num_srqs = HNS_ROCE_V2_MAX_SRQ_NUM; caps->max_cqes = HNS_ROCE_V2_MAX_CQE_NUM; + caps->max_srqwqes = HNS_ROCE_V2_MAX_SRQWQE_NUM; caps->max_sq_sg = HNS_ROCE_V2_MAX_SQ_SGE_NUM; caps->max_extend_sg = HNS_ROCE_V2_MAX_EXTEND_SGE_NUM; caps->max_rq_sg = HNS_ROCE_V2_MAX_RQ_SGE_NUM; caps->max_sq_inline = HNS_ROCE_V2_MAX_SQ_INLINE; + caps->max_srq_sg = HNS_ROCE_V2_MAX_SRQ_SGE_NUM; caps->num_uars = HNS_ROCE_V2_UAR_NUM; caps->phy_num_uars = HNS_ROCE_V2_PHY_UAR_NUM; caps->num_aeq_vectors = HNS_ROCE_V2_AEQE_VEC_NUM; @@ -1289,6 +1292,8 @@ static int hns_roce_v2_profile(struct hns_roce_dev *hr_dev) caps->num_mtpts = HNS_ROCE_V2_MAX_MTPT_NUM; caps->num_mtt_segs = HNS_ROCE_V2_MAX_MTT_SEGS; caps->num_cqe_segs = HNS_ROCE_V2_MAX_CQE_SEGS; + caps->num_srqwqe_segs = HNS_ROCE_V2_MAX_SRQWQE_SEGS; + caps->num_idx_segs = HNS_ROCE_V2_MAX_IDX_SEGS; caps->num_pds = HNS_ROCE_V2_MAX_PD_NUM; caps->max_qp_init_rdma = HNS_ROCE_V2_MAX_QP_INIT_RDMA; caps->max_qp_dest_rdma = HNS_ROCE_V2_MAX_QP_DEST_RDMA; @@ -1299,8 +1304,10 @@ static int hns_roce_v2_profile(struct hns_roce_dev *hr_dev) caps->irrl_entry_sz = HNS_ROCE_V2_IRRL_ENTRY_SZ; caps->trrl_entry_sz = HNS_ROCE_V2_TRRL_ENTRY_SZ; caps->cqc_entry_sz = HNS_ROCE_V2_CQC_ENTRY_SZ; + caps->srqc_entry_sz = HNS_ROCE_V2_SRQC_ENTRY_SZ; caps->mtpt_entry_sz = HNS_ROCE_V2_MTPT_ENTRY_SZ; caps->mtt_entry_sz = HNS_ROCE_V2_MTT_ENTRY_SZ; + caps->idx_entry_sz = 4; caps->cq_entry_sz = HNS_ROCE_V2_CQE_ENTRY_SIZE; caps->page_size_cap = HNS_ROCE_V2_PAGE_SIZE_SUPPORTED; caps->reserved_lkey = 0; @@ -1308,6 +1315,7 @@ static int hns_roce_v2_profile(struct hns_roce_dev *hr_dev) caps->reserved_mrws = 1; caps->reserved_uars = 0; caps->reserved_cqs = 0; + caps->reserved_srqs = 0; caps->reserved_qps = HNS_ROCE_V2_RSV_QPS; caps->qpc_ba_pg_sz = 0; diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h index 24a4851..383bab5 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h @@ -50,9 +50,12 @@ #define HNS_ROCE_V2_MAX_SRQ_WR 0x8000 #define HNS_ROCE_V2_MAX_SRQ_SGE 0x100 #define HNS_ROCE_V2_MAX_CQ_NUM 0x8000 +#define HNS_ROCE_V2_MAX_SRQ_NUM 0x100000 #define HNS_ROCE_V2_MAX_CQE_NUM 0x10000 +#define HNS_ROCE_V2_MAX_SRQWQE_NUM 0x8000 #define HNS_ROCE_V2_MAX_RQ_SGE_NUM 0x100 #define HNS_ROCE_V2_MAX_SQ_SGE_NUM 0xff +#define HNS_ROCE_V2_MAX_SRQ_SGE_NUM 0x100 #define HNS_ROCE_V2_MAX_EXTEND_SGE_NUM 0x200000 #define HNS_ROCE_V2_MAX_SQ_INLINE 0x20 #define HNS_ROCE_V2_UAR_NUM 256 @@ -64,6 +67,8 @@ #define HNS_ROCE_V2_MAX_MTPT_NUM 0x8000 #define HNS_ROCE_V2_MAX_MTT_SEGS 0x1000000 #define HNS_ROCE_V2_MAX_CQE_SEGS 0x1000000 +#define HNS_ROCE_V2_MAX_SRQWQE_SEGS 0x1000000 +#define HNS_ROCE_V2_MAX_IDX_SEGS 0x1000000 #define HNS_ROCE_V2_MAX_PD_NUM 0x1000000 #define HNS_ROCE_V2_MAX_QP_INIT_RDMA 128 #define HNS_ROCE_V2_MAX_QP_DEST_RDMA 128 @@ -74,6 +79,7 @@ #define HNS_ROCE_V2_IRRL_ENTRY_SZ 64 #define HNS_ROCE_V2_TRRL_ENTRY_SZ 48 #define HNS_ROCE_V2_CQC_ENTRY_SZ 64 +#define HNS_ROCE_V2_SRQC_ENTRY_SZ 64 #define HNS_ROCE_V2_MTPT_ENTRY_SZ 64 #define HNS_ROCE_V2_MTT_ENTRY_SZ 64 #define HNS_ROCE_V2_CQE_ENTRY_SIZE 32 diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c index f6f288f..d183f13 100644 --- a/drivers/infiniband/hw/hns/hns_roce_main.c +++ b/drivers/infiniband/hw/hns/hns_roce_main.c @@ -651,8 +651,58 @@ static int hns_roce_init_hem(struct hns_roce_dev *hr_dev) goto err_unmap_trrl; } + if (hr_dev->caps.srqc_entry_sz) { + ret = hns_roce_init_hem_table(hr_dev, &hr_dev->srq_table.table, + HEM_TYPE_SRQC, + hr_dev->caps.srqc_entry_sz, + hr_dev->caps.num_srqs, 1); + if (ret) { + dev_err(dev, + "Failed to init SRQ context memory, aborting.\n"); + goto err_unmap_cq; + } + } + + if (hr_dev->caps.num_srqwqe_segs) { + ret = hns_roce_init_hem_table(hr_dev, + &hr_dev->mr_table.mtt_srqwqe_table, + HEM_TYPE_SRQWQE, + hr_dev->caps.mtt_entry_sz, + hr_dev->caps.num_srqwqe_segs, 1); + if (ret) { + dev_err(dev, + "Failed to init MTT srqwqe memory, aborting.\n"); + goto err_unmap_srq; + } + } + + if (hr_dev->caps.num_idx_segs) { + ret = hns_roce_init_hem_table(hr_dev, + &hr_dev->mr_table.mtt_idx_table, + HEM_TYPE_IDX, + hr_dev->caps.idx_entry_sz, + hr_dev->caps.num_idx_segs, 1); + if (ret) { + dev_err(dev, + "Failed to init MTT idx memory, aborting.\n"); + goto err_unmap_srqwqe; + } + } + return 0; +err_unmap_srqwqe: + if (hr_dev->caps.num_srqwqe_segs) + hns_roce_cleanup_hem_table(hr_dev, + &hr_dev->mr_table.mtt_srqwqe_table); + +err_unmap_srq: + if (hr_dev->caps.srqc_entry_sz) + hns_roce_cleanup_hem_table(hr_dev, &hr_dev->srq_table.table); + +err_unmap_cq: + hns_roce_cleanup_hem_table(hr_dev, &hr_dev->cq_table.table); + err_unmap_trrl: if (hr_dev->caps.trrl_entry_sz) hns_roce_cleanup_hem_table(hr_dev, @@ -732,8 +782,21 @@ static int hns_roce_setup_hca(struct hns_roce_dev *hr_dev) goto err_cq_table_free; } + if (hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_SRQ) { + ret = hns_roce_init_srq_table(hr_dev); + if (ret) { + dev_err(dev, + "Failed to init share receive queue table.\n"); + goto err_qp_table_free; + } + } + return 0; +err_qp_table_free: + if (hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_SRQ) + hns_roce_cleanup_qp_table(hr_dev); + err_cq_table_free: hns_roce_cleanup_cq_table(hr_dev); diff --git a/drivers/infiniband/hw/hns/hns_roce_srq.c b/drivers/infiniband/hw/hns/hns_roce_srq.c new file mode 100644 index 0000000..d8a8613 --- /dev/null +++ b/drivers/infiniband/hw/hns/hns_roce_srq.c @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* + * Copyright (c) 2018 Hisilicon Limited. + */ + +#include <rdma/ib_umem.h> +#include <rdma/hns-abi.h> +#include "hns_roce_device.h" +#include "hns_roce_cmd.h" +#include "hns_roce_hem.h" + +int hns_roce_init_srq_table(struct hns_roce_dev *hr_dev) +{ + struct hns_roce_srq_table *srq_table = &hr_dev->srq_table; + + xa_init(&srq_table->xa); + + return hns_roce_bitmap_init(&srq_table->bitmap, hr_dev->caps.num_srqs, + hr_dev->caps.num_srqs - 1, + hr_dev->caps.reserved_srqs, 0); +} + +void hns_roce_cleanup_srq_table(struct hns_roce_dev *hr_dev) +{ + hns_roce_bitmap_cleanup(&hr_dev->srq_table.bitmap); +}
This patch inits hem resource for SRQ table, includes SRQWQE and SRQWQE index resource. Signed-off-by: Lijun Ou <oulijun@huawei.com> --- V2->V3: 1. Use xa_init instead of INIT_RADIX_TREE --- drivers/infiniband/hw/hns/Makefile | 2 +- drivers/infiniband/hw/hns/hns_roce_alloc.c | 2 + drivers/infiniband/hw/hns/hns_roce_device.h | 23 +++++++++++ drivers/infiniband/hw/hns/hns_roce_hem.c | 9 +++++ drivers/infiniband/hw/hns/hns_roce_hem.h | 2 + drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 8 ++++ drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 6 +++ drivers/infiniband/hw/hns/hns_roce_main.c | 63 +++++++++++++++++++++++++++++ drivers/infiniband/hw/hns/hns_roce_srq.c | 26 ++++++++++++ 9 files changed, 140 insertions(+), 1 deletion(-) create mode 100644 drivers/infiniband/hw/hns/hns_roce_srq.c