diff mbox series

[for-next,v4,3/4] RDMA/efa: User/kernel compatibility handshake mechanism

Message ID 20200722140312.3651-4-galpress@amazon.com (mailing list archive)
State Mainlined
Commit a5d87b698547233321466b2dc91271f5855a4df6
Delegated to: Jason Gunthorpe
Headers show
Series Add support for 0xefa1 device | expand

Commit Message

Gal Pressman July 22, 2020, 2:03 p.m. UTC 
Introduce a mechanism that performs an handshake between the userspace
provider and kernel driver which verifies that the user supports all
required features in order to operate correctly.

The handshake verifies the needed functionality by comparing the
reported device caps and the provider caps. If the device reports a
non-zero capability the appropriate comp mask is required from the
userspace provider in order to allocate the context.

Reviewed-by: Shadi Ammouri <sammouri@amazon.com>
Reviewed-by: Yossi Leybovich <sleybo@amazon.com>
Signed-off-by: Gal Pressman <galpress@amazon.com>
---
 drivers/infiniband/hw/efa/efa_verbs.c | 40 +++++++++++++++++++++++++++
 include/uapi/rdma/efa-abi.h           | 10 +++++++
 2 files changed, 50 insertions(+)

Comments

Jason Gunthorpe July 27, 2020, 6:56 p.m. UTC | #1 
On Wed, Jul 22, 2020 at 05:03:11PM +0300, Gal Pressman wrote:
> Introduce a mechanism that performs an handshake between the userspace
> provider and kernel driver which verifies that the user supports all
> required features in order to operate correctly.
> 
> The handshake verifies the needed functionality by comparing the
> reported device caps and the provider caps. If the device reports a
> non-zero capability the appropriate comp mask is required from the
> userspace provider in order to allocate the context.
> 
> Reviewed-by: Shadi Ammouri <sammouri@amazon.com>
> Reviewed-by: Yossi Leybovich <sleybo@amazon.com>
> Signed-off-by: Gal Pressman <galpress@amazon.com>
>  drivers/infiniband/hw/efa/efa_verbs.c | 40 +++++++++++++++++++++++++++
>  include/uapi/rdma/efa-abi.h           | 10 +++++++
>  2 files changed, 50 insertions(+)
> 
> diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c
> index 26102ab333b2..fda175836fb6 100644
> +++ b/drivers/infiniband/hw/efa/efa_verbs.c
> @@ -1501,11 +1501,39 @@ static int efa_dealloc_uar(struct efa_dev *dev, u16 uarn)
>  	return efa_com_dealloc_uar(&dev->edev, &params);
>  }
>  
> +#define EFA_CHECK_USER_COMP(_dev, _comp_mask, _attr, _mask, _attr_str) \
> +	(_attr_str = (!(_dev)->dev_attr._attr || ((_comp_mask) & (_mask))) ? \
> +		     NULL : #_attr)
> +
> +static int efa_user_comp_handshake(const struct ib_ucontext *ibucontext,
> +				   const struct efa_ibv_alloc_ucontext_cmd *cmd)
> +{
> +	struct efa_dev *dev = to_edev(ibucontext->device);
> +	char *attr_str;
> +
> +	if (EFA_CHECK_USER_COMP(dev, cmd->comp_mask, max_tx_batch,
> +				EFA_ALLOC_UCONTEXT_CMD_COMP_TX_BATCH, attr_str))
> +		goto err;
> +
> +	if (EFA_CHECK_USER_COMP(dev, cmd->comp_mask, min_sq_depth,
> +				EFA_ALLOC_UCONTEXT_CMD_COMP_MIN_SQ_WR,
> +				attr_str))
> +		goto err;

But this patch should be first, the kernel should never return a
non-zero value unless these input bits are set

Jason
Gal Pressman July 28, 2020, 11:50 a.m. UTC | #2 
On 27/07/2020 21:56, Jason Gunthorpe wrote:
> On Wed, Jul 22, 2020 at 05:03:11PM +0300, Gal Pressman wrote:
>> Introduce a mechanism that performs an handshake between the userspace
>> provider and kernel driver which verifies that the user supports all
>> required features in order to operate correctly.
>>
>> The handshake verifies the needed functionality by comparing the
>> reported device caps and the provider caps. If the device reports a
>> non-zero capability the appropriate comp mask is required from the
>> userspace provider in order to allocate the context.
>>
>> Reviewed-by: Shadi Ammouri <sammouri@amazon.com>
>> Reviewed-by: Yossi Leybovich <sleybo@amazon.com>
>> Signed-off-by: Gal Pressman <galpress@amazon.com>
>>  drivers/infiniband/hw/efa/efa_verbs.c | 40 +++++++++++++++++++++++++++
>>  include/uapi/rdma/efa-abi.h           | 10 +++++++
>>  2 files changed, 50 insertions(+)
>>
>> diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c
>> index 26102ab333b2..fda175836fb6 100644
>> +++ b/drivers/infiniband/hw/efa/efa_verbs.c
>> @@ -1501,11 +1501,39 @@ static int efa_dealloc_uar(struct efa_dev *dev, u16 uarn)
>>  	return efa_com_dealloc_uar(&dev->edev, &params);
>>  }
>>  
>> +#define EFA_CHECK_USER_COMP(_dev, _comp_mask, _attr, _mask, _attr_str) \
>> +	(_attr_str = (!(_dev)->dev_attr._attr || ((_comp_mask) & (_mask))) ? \
>> +		     NULL : #_attr)
>> +
>> +static int efa_user_comp_handshake(const struct ib_ucontext *ibucontext,
>> +				   const struct efa_ibv_alloc_ucontext_cmd *cmd)
>> +{
>> +	struct efa_dev *dev = to_edev(ibucontext->device);
>> +	char *attr_str;
>> +
>> +	if (EFA_CHECK_USER_COMP(dev, cmd->comp_mask, max_tx_batch,
>> +				EFA_ALLOC_UCONTEXT_CMD_COMP_TX_BATCH, attr_str))
>> +		goto err;
>> +
>> +	if (EFA_CHECK_USER_COMP(dev, cmd->comp_mask, min_sq_depth,
>> +				EFA_ALLOC_UCONTEXT_CMD_COMP_MIN_SQ_WR,
>> +				attr_str))
>> +		goto err;
> 
> But this patch should be first, the kernel should never return a
> non-zero value unless these input bits are set

But that's exactly what this patch does, it can only fail in case
max_tx_batch/min_sq_depth is turned on by the device.

Anyway, the order doesn't matter as long as the pciid patch is last.
Jason Gunthorpe July 28, 2020, 12:28 p.m. UTC | #3 
On Tue, Jul 28, 2020 at 02:50:18PM +0300, Gal Pressman wrote:
> On 27/07/2020 21:56, Jason Gunthorpe wrote:
> > On Wed, Jul 22, 2020 at 05:03:11PM +0300, Gal Pressman wrote:
> >> Introduce a mechanism that performs an handshake between the userspace
> >> provider and kernel driver which verifies that the user supports all
> >> required features in order to operate correctly.
> >>
> >> The handshake verifies the needed functionality by comparing the
> >> reported device caps and the provider caps. If the device reports a
> >> non-zero capability the appropriate comp mask is required from the
> >> userspace provider in order to allocate the context.
> >>
> >> Reviewed-by: Shadi Ammouri <sammouri@amazon.com>
> >> Reviewed-by: Yossi Leybovich <sleybo@amazon.com>
> >> Signed-off-by: Gal Pressman <galpress@amazon.com>
> >>  drivers/infiniband/hw/efa/efa_verbs.c | 40 +++++++++++++++++++++++++++
> >>  include/uapi/rdma/efa-abi.h           | 10 +++++++
> >>  2 files changed, 50 insertions(+)
> >>
> >> diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c
> >> index 26102ab333b2..fda175836fb6 100644
> >> +++ b/drivers/infiniband/hw/efa/efa_verbs.c
> >> @@ -1501,11 +1501,39 @@ static int efa_dealloc_uar(struct efa_dev *dev, u16 uarn)
> >>  	return efa_com_dealloc_uar(&dev->edev, &params);
> >>  }
> >>  
> >> +#define EFA_CHECK_USER_COMP(_dev, _comp_mask, _attr, _mask, _attr_str) \
> >> +	(_attr_str = (!(_dev)->dev_attr._attr || ((_comp_mask) & (_mask))) ? \
> >> +		     NULL : #_attr)
> >> +
> >> +static int efa_user_comp_handshake(const struct ib_ucontext *ibucontext,
> >> +				   const struct efa_ibv_alloc_ucontext_cmd *cmd)
> >> +{
> >> +	struct efa_dev *dev = to_edev(ibucontext->device);
> >> +	char *attr_str;
> >> +
> >> +	if (EFA_CHECK_USER_COMP(dev, cmd->comp_mask, max_tx_batch,
> >> +				EFA_ALLOC_UCONTEXT_CMD_COMP_TX_BATCH, attr_str))
> >> +		goto err;
> >> +
> >> +	if (EFA_CHECK_USER_COMP(dev, cmd->comp_mask, min_sq_depth,
> >> +				EFA_ALLOC_UCONTEXT_CMD_COMP_MIN_SQ_WR,
> >> +				attr_str))
> >> +		goto err;
> > 
> > But this patch should be first, the kernel should never return a
> > non-zero value unless these input bits are set
> 
> But that's exactly what this patch does, it can only fail in case
> max_tx_batch/min_sq_depth is turned on by the device.

My point is the series is out of order, the introduction of the two
uapi parts should be in the same patch

> Anyway, the order doesn't matter as long as the pciid patch is last.

Oh?

Jason
Gal Pressman July 28, 2020, 1:07 p.m. UTC | #4 
On 28/07/2020 15:28, Jason Gunthorpe wrote:
> On Tue, Jul 28, 2020 at 02:50:18PM +0300, Gal Pressman wrote:
>> On 27/07/2020 21:56, Jason Gunthorpe wrote:
>>> On Wed, Jul 22, 2020 at 05:03:11PM +0300, Gal Pressman wrote:
>>>> Introduce a mechanism that performs an handshake between the userspace
>>>> provider and kernel driver which verifies that the user supports all
>>>> required features in order to operate correctly.
>>>>
>>>> The handshake verifies the needed functionality by comparing the
>>>> reported device caps and the provider caps. If the device reports a
>>>> non-zero capability the appropriate comp mask is required from the
>>>> userspace provider in order to allocate the context.
>>>>
>>>> Reviewed-by: Shadi Ammouri <sammouri@amazon.com>
>>>> Reviewed-by: Yossi Leybovich <sleybo@amazon.com>
>>>> Signed-off-by: Gal Pressman <galpress@amazon.com>
>>>>  drivers/infiniband/hw/efa/efa_verbs.c | 40 +++++++++++++++++++++++++++
>>>>  include/uapi/rdma/efa-abi.h           | 10 +++++++
>>>>  2 files changed, 50 insertions(+)
>>>>
>>>> diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c
>>>> index 26102ab333b2..fda175836fb6 100644
>>>> +++ b/drivers/infiniband/hw/efa/efa_verbs.c
>>>> @@ -1501,11 +1501,39 @@ static int efa_dealloc_uar(struct efa_dev *dev, u16 uarn)
>>>>  	return efa_com_dealloc_uar(&dev->edev, &params);
>>>>  }
>>>>  
>>>> +#define EFA_CHECK_USER_COMP(_dev, _comp_mask, _attr, _mask, _attr_str) \
>>>> +	(_attr_str = (!(_dev)->dev_attr._attr || ((_comp_mask) & (_mask))) ? \
>>>> +		     NULL : #_attr)
>>>> +
>>>> +static int efa_user_comp_handshake(const struct ib_ucontext *ibucontext,
>>>> +				   const struct efa_ibv_alloc_ucontext_cmd *cmd)
>>>> +{
>>>> +	struct efa_dev *dev = to_edev(ibucontext->device);
>>>> +	char *attr_str;
>>>> +
>>>> +	if (EFA_CHECK_USER_COMP(dev, cmd->comp_mask, max_tx_batch,
>>>> +				EFA_ALLOC_UCONTEXT_CMD_COMP_TX_BATCH, attr_str))
>>>> +		goto err;
>>>> +
>>>> +	if (EFA_CHECK_USER_COMP(dev, cmd->comp_mask, min_sq_depth,
>>>> +				EFA_ALLOC_UCONTEXT_CMD_COMP_MIN_SQ_WR,
>>>> +				attr_str))
>>>> +		goto err;
>>>
>>> But this patch should be first, the kernel should never return a
>>> non-zero value unless these input bits are set
>>
>> But that's exactly what this patch does, it can only fail in case
>> max_tx_batch/min_sq_depth is turned on by the device.
> 
> My point is the series is out of order, the introduction of the two
> uapi parts should be in the same patch
> 
>> Anyway, the order doesn't matter as long as the pciid patch is last.
> 
> Oh?

0xefa0 devices will not turn on these bits, so as long as the 0xefa1 patch is
last this order is fine.
diff mbox series

Patch

diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c
index 26102ab333b2..fda175836fb6 100644
--- a/drivers/infiniband/hw/efa/efa_verbs.c
+++ b/drivers/infiniband/hw/efa/efa_verbs.c
@@ -1501,11 +1501,39 @@  static int efa_dealloc_uar(struct efa_dev *dev, u16 uarn)
 	return efa_com_dealloc_uar(&dev->edev, &params);
 }
 
+#define EFA_CHECK_USER_COMP(_dev, _comp_mask, _attr, _mask, _attr_str) \
+	(_attr_str = (!(_dev)->dev_attr._attr || ((_comp_mask) & (_mask))) ? \
+		     NULL : #_attr)
+
+static int efa_user_comp_handshake(const struct ib_ucontext *ibucontext,
+				   const struct efa_ibv_alloc_ucontext_cmd *cmd)
+{
+	struct efa_dev *dev = to_edev(ibucontext->device);
+	char *attr_str;
+
+	if (EFA_CHECK_USER_COMP(dev, cmd->comp_mask, max_tx_batch,
+				EFA_ALLOC_UCONTEXT_CMD_COMP_TX_BATCH, attr_str))
+		goto err;
+
+	if (EFA_CHECK_USER_COMP(dev, cmd->comp_mask, min_sq_depth,
+				EFA_ALLOC_UCONTEXT_CMD_COMP_MIN_SQ_WR,
+				attr_str))
+		goto err;
+
+	return 0;
+
+err:
+	ibdev_dbg(&dev->ibdev, "Userspace handshake failed for %s attribute\n",
+		  attr_str);
+	return -EOPNOTSUPP;
+}
+
 int efa_alloc_ucontext(struct ib_ucontext *ibucontext, struct ib_udata *udata)
 {
 	struct efa_ucontext *ucontext = to_eucontext(ibucontext);
 	struct efa_dev *dev = to_edev(ibucontext->device);
 	struct efa_ibv_alloc_ucontext_resp resp = {};
+	struct efa_ibv_alloc_ucontext_cmd cmd = {};
 	struct efa_com_alloc_uar_result result;
 	int err;
 
@@ -1514,6 +1542,18 @@  int efa_alloc_ucontext(struct ib_ucontext *ibucontext, struct ib_udata *udata)
 	 * we will ack input fields in our response.
 	 */
 
+	err = ib_copy_from_udata(&cmd, udata,
+				 min(sizeof(cmd), udata->inlen));
+	if (err) {
+		ibdev_dbg(&dev->ibdev,
+			  "Cannot copy udata for alloc_ucontext\n");
+		goto err_out;
+	}
+
+	err = efa_user_comp_handshake(ibucontext, &cmd);
+	if (err)
+		goto err_out;
+
 	err = efa_com_alloc_uar(&dev->edev, &result);
 	if (err)
 		goto err_out;
diff --git a/include/uapi/rdma/efa-abi.h b/include/uapi/rdma/efa-abi.h
index 7ef2306f8dd4..507a2862bedb 100644
--- a/include/uapi/rdma/efa-abi.h
+++ b/include/uapi/rdma/efa-abi.h
@@ -20,6 +20,16 @@ 
  * hex bit offset of the field.
  */
 
+enum {
+	EFA_ALLOC_UCONTEXT_CMD_COMP_TX_BATCH  = 1 << 0,
+	EFA_ALLOC_UCONTEXT_CMD_COMP_MIN_SQ_WR = 1 << 1,
+};
+
+struct efa_ibv_alloc_ucontext_cmd {
+	__u32 comp_mask;
+	__u8 reserved_20[4];
+};
+
 enum efa_ibv_user_cmds_supp_udata {
 	EFA_USER_CMDS_SUPP_UDATA_QUERY_DEVICE = 1 << 0,
 	EFA_USER_CMDS_SUPP_UDATA_CREATE_AH    = 1 << 1,