| Message ID | 1471633611-41155-1-git-send-email-jarod@redhat.com (mailing list archive) |
|---|---|
| State | Not Applicable |
| Headers | show |
On 8/19/2016 2:06 PM, Jarod Wilson wrote: > The ocrdma driver supports a maximum inline send size of 224, while the > current code sets this size to 236. Add an ocrdma-specific check when > setting max inline size to eliminate the need to always pass -I 224 when > running ib_send_lat on ocrdma hardware. <soap box> We really should fix the RDMA uAPI to allow querying the device to get the max inline size for various SQ WRs (it might vary for send vs write, for example), instead of requiring applications to guess or hardcoding benchmarks based on device name... </soap box> > CC: Gil Rockah <gilr@mellanox.com> > Signed-off-by: Jarod Wilson <jarod@redhat.com> > --- > src/perftest_parameters.c | 8 ++++++++ > src/perftest_parameters.h | 3 +++ > 2 files changed, 11 insertions(+) > > diff --git a/src/perftest_parameters.c b/src/perftest_parameters.c > index 3ffddd7..65ce731 100755 > --- a/src/perftest_parameters.c > +++ b/src/perftest_parameters.c > @@ -1465,6 +1465,14 @@ static void ctx_set_max_inline(struct ibv_context *context,struct perftest_param > } > } > > + if (strncmp("ocrdma", user_param->ib_devname, 6) == 0) { > + if (user_param->inline_size > DEF_INLINE_SEND_MAX_OCRDMA) { > + fprintf(stdout, "lowering inline size to ocrdma supported maximum of %d\n", > + DEF_INLINE_SEND_MAX_OCRDMA); > + user_param->inline_size = DEF_INLINE_SEND_MAX_OCRDMA; > + } > + } > + > return; > } > /****************************************************************************** > diff --git a/src/perftest_parameters.h b/src/perftest_parameters.h > index 1122792..a566f42 100755 > --- a/src/perftest_parameters.h > +++ b/src/perftest_parameters.h > @@ -133,6 +133,9 @@ > #define DEF_INLINE_SEND_UD (188) > #define DEF_INLINE_DC (150) > > +/* OCRDMA has an inline max of 224 */ > +#define DEF_INLINE_SEND_MAX_OCRDMA (224) > + > /* Max and Min allowed values for perftest parameters. */ > #define MIN_TOS (0) > #define MAX_TOS (256) -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Aug 19, 2016 at 03:45:57PM -0500, Steve Wise wrote: > On 8/19/2016 2:06 PM, Jarod Wilson wrote: > >The ocrdma driver supports a maximum inline send size of 224, while the > >current code sets this size to 236. Add an ocrdma-specific check when > >setting max inline size to eliminate the need to always pass -I 224 when > >running ib_send_lat on ocrdma hardware. > > <soap box> > We really should fix the RDMA uAPI to allow querying the device to > get the max inline size for various SQ WRs (it might vary for send > vs write, for example), instead of requiring applications to guess > or hardcoding benchmarks based on device name... > </soap box> This was a nice quick hack Friday afternoon hack to get it working with existing kernels in our testing infra, but having a way to ask the hardware would definitely be vastly superior.
> On Fri, Aug 19, 2016 at 03:45:57PM -0500, Steve Wise wrote: > > On 8/19/2016 2:06 PM, Jarod Wilson wrote: > > >The ocrdma driver supports a maximum inline send size of 224, while the > > >current code sets this size to 236. Add an ocrdma-specific check when > > >setting max inline size to eliminate the need to always pass -I 224 when > > >running ib_send_lat on ocrdma hardware. > > > > <soap box> > > We really should fix the RDMA uAPI to allow querying the device to > > get the max inline size for various SQ WRs (it might vary for send > > vs write, for example), instead of requiring applications to guess > > or hardcoding benchmarks based on device name... > > </soap box> > > This was a nice quick hack Friday afternoon hack to get it working > with existing kernels in our testing infra, but having a way to ask > the hardware would definitely be vastly superior. > I hear ya. Maybe as part of the new verbs API work we could provide a way for applications to query the max inline... -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/src/perftest_parameters.c b/src/perftest_parameters.c index 3ffddd7..65ce731 100755 --- a/src/perftest_parameters.c +++ b/src/perftest_parameters.c @@ -1465,6 +1465,14 @@ static void ctx_set_max_inline(struct ibv_context *context,struct perftest_param } } + if (strncmp("ocrdma", user_param->ib_devname, 6) == 0) { + if (user_param->inline_size > DEF_INLINE_SEND_MAX_OCRDMA) { + fprintf(stdout, "lowering inline size to ocrdma supported maximum of %d\n", + DEF_INLINE_SEND_MAX_OCRDMA); + user_param->inline_size = DEF_INLINE_SEND_MAX_OCRDMA; + } + } + return; } /****************************************************************************** diff --git a/src/perftest_parameters.h b/src/perftest_parameters.h index 1122792..a566f42 100755 --- a/src/perftest_parameters.h +++ b/src/perftest_parameters.h @@ -133,6 +133,9 @@ #define DEF_INLINE_SEND_UD (188) #define DEF_INLINE_DC (150) +/* OCRDMA has an inline max of 224 */ +#define DEF_INLINE_SEND_MAX_OCRDMA (224) + /* Max and Min allowed values for perftest parameters. */ #define MIN_TOS (0) #define MAX_TOS (256)
The ocrdma driver supports a maximum inline send size of 224, while the current code sets this size to 236. Add an ocrdma-specific check when setting max inline size to eliminate the need to always pass -I 224 when running ib_send_lat on ocrdma hardware. CC: Gil Rockah <gilr@mellanox.com> Signed-off-by: Jarod Wilson <jarod@redhat.com> --- src/perftest_parameters.c | 8 ++++++++ src/perftest_parameters.h | 3 +++ 2 files changed, 11 insertions(+)