From patchwork Fri Oct 21 20:01:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015352 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49A6BC38A2D for ; Fri, 21 Oct 2022 20:02:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229841AbiJUUCl (ORCPT ); Fri, 21 Oct 2022 16:02:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48276 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230016AbiJUUCN (ORCPT ); Fri, 21 Oct 2022 16:02:13 -0400 Received: from mail-oa1-x2e.google.com (mail-oa1-x2e.google.com [IPv6:2001:4860:4864:20::2e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 71CA125ED0C for ; Fri, 21 Oct 2022 13:02:02 -0700 (PDT) Received: by mail-oa1-x2e.google.com with SMTP id 586e51a60fabf-1324e7a1284so4872741fac.10 for ; Fri, 21 Oct 2022 13:02:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zJoK0HylogCNImnm+D5kGSnKHCyCbyRc9oJ5qG2Py+Q=; b=H/BjvspHQG0mhW5wyawtPUc+ycoN/6/YIUig5QgtkN5SHBKyidBOoKgB322+UBk9/L IKpxMMDt+xHp//HwsuDOpkt0YqEWagj214OYsFQlN2NgfWfqXWBZjwLgDI76c2mZNPsX LmuDtJIzA28noShfaJiRVTlPVl0HRvQy+8Ybp+qKNfse40X1Up8w16m9YOCGRAYlo6PI /gAQv8Xqo+OthqVxQ062VWJ4L+3OE85UPQe8/xMzkoXuuoqApt/Nm/YWEw1uJ/QCEtqJ zM/GOr95hq7xBYE6DpSyAoL8kV6iHiuauv0rkrQnbQ3STeT6LFnFrz/uYM+c4wTYntkM xrhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zJoK0HylogCNImnm+D5kGSnKHCyCbyRc9oJ5qG2Py+Q=; b=olAlBPPY8fFuT+1gi7VHv/FpuUGUnHQEPGze4N1j+RsUblIMiixD5Khny5/INErVnd I6RVPx37OkFN5sHPC/ejTYiMOr9gl5FUbDcB1nYY96T2o+T6GdabdUvc6dENP8gIAeew 3r3kR1KQ6TE6DbbolUzR40e2ddSMHN8jsSive5+F2Qy1QPiGREeeCpycb3bt72nCpRKX Q9Czo7QbacJbGVsa1vN6poHQA5/2S6Uin0bQJwk13HPn4b7yGLecWGgGH38elRnxcQ+i DkijLcoXv/REVSM+3FFyRewKUhY6pfcZwZXmIo8D+wca1LZ/bfVS55gWISnRfXbDO0zk SzlQ== X-Gm-Message-State: ACrzQf2KCgvsQm6oX8QslrWFIv/QnJuTvn04V3ipJUkXFieRO5efAAy3 VQichjuqCbCCYiOhNkVd8lk= X-Google-Smtp-Source: AMsMyM5E3yPZLztP3je+iRgWDEZ6PnnD0rjXR7y7evHvBFvQJCgDUS92K4I+k+NpDiS8br9kqa4R3A== X-Received: by 2002:a05:6870:2191:b0:132:a213:3657 with SMTP id l17-20020a056870219100b00132a2133657mr13758141oae.13.1666382521813; Fri, 21 Oct 2022 13:02:01 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:01 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 01/18] RDMA/rxe: Remove redundant header files Date: Fri, 21 Oct 2022 15:01:02 -0500 Message-Id: <20221021200118.2163-2-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Remove unneeded include files. Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_task.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c index ec2b7de1c497..3fbaba9eec39 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.c +++ b/drivers/infiniband/sw/rxe/rxe_task.c @@ -4,10 +4,6 @@ * Copyright (c) 2015 System Fabric Works, Inc. All rights reserved. */ -#include -#include -#include - #include "rxe.h" int __rxe_do_task(struct rxe_task *task) From patchwork Fri Oct 21 20:01:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015353 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1C60C433FE for ; Fri, 21 Oct 2022 20:02:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230016AbiJUUCm (ORCPT ); Fri, 21 Oct 2022 16:02:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48280 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230254AbiJUUCN (ORCPT ); Fri, 21 Oct 2022 16:02:13 -0400 Received: from mail-oa1-x36.google.com (mail-oa1-x36.google.com [IPv6:2001:4860:4864:20::36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 33E3B26205F for ; Fri, 21 Oct 2022 13:02:04 -0700 (PDT) Received: by mail-oa1-x36.google.com with SMTP id 586e51a60fabf-13b103a3e5dso4928839fac.2 for ; Fri, 21 Oct 2022 13:02:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yh+FwTHc00Q0kWWf36fdCBxCjYor7euxP3hgKCKswJg=; b=YrAD+5Y2RnKoj8aqhfLJ+pNPVG99vZPrwN+OMkr9v8J/SucdWKQrhVc9exrEoc5tbr 6lO/2x4fiK01wEaeXi8w8G37X7RaHI7/bUHW1srffDtr0Td8eEgSxMZfQf9VJRUGY0Lu Bze72ecsaA+a0Zy+SSlBLiBGJ0eHvLODikOtMf0kIwt0ani2uQes2p5pTAlzpY7e9+tF TQJwgadvsCgC9p98818K5DvrPojw5092GvSK5m5ONq5tx1pWDzkHwkg0bsTiFR9sU/ik CssAxVgWiVrXpdSCWlvg02X1D/NXKJCnh3/SISN81TebdGIYBuJE65tLLjVVKv32hH6s cm8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yh+FwTHc00Q0kWWf36fdCBxCjYor7euxP3hgKCKswJg=; b=wpZ/eSLl4XS45fKxbzuf4d3mtCW9y/jjo9oQL2QmE413YzHcLXqGSzpHsUeHxlWA5B 8s0NwNgIUiNtsNZuxrKqwDXl9CJRdRCZLa82vBpyJXgGHteeL4KHW7romVj0Rnhe4AmE thsw35CxEJJjgbvBsgiGBsLwTznpNs2os5J/78nLSsDszHeQmJKyqyh2p1YSJCL98AXH lK2Q+Tyb2iS2uZJOiKwDxCO/stJ3Z+b8BPAJJo1Ez63aWAY0je4nbuXjX79X0Z8YMVcY xAX6SfhbG9PbY73ginFh3F6MmQawOPrSbhx3C6aDRIASs7YR/uVZ1boZwO0rZMORdaIJ 10DA== X-Gm-Message-State: ACrzQf1Fb5SutKtU1sFifcPSgKLiEewk1MyKJHJXl+pKaUuWR1Xoj5ts YZJSPJMtzBTjfXy6mnO2wVg= X-Google-Smtp-Source: AMsMyM6FXtk9DrySLrDKQXHiOgXeAAVnI6jOLin8jLoYkqcrH6nTzxta4kKuuqM1Hkw4z+tLUJqmhQ== X-Received: by 2002:a05:6870:3452:b0:127:74a6:365c with SMTP id i18-20020a056870345200b0012774a6365cmr30012330oah.169.1666382523226; Fri, 21 Oct 2022 13:02:03 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:02 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 02/18] RDMA/rxe: Remove init of task locks from rxe_qp.c Date: Fri, 21 Oct 2022 15:01:03 -0500 Message-Id: <20221021200118.2163-3-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The calls to spin_lock_init() for the tasklet spinlocks in rxe_qp_init_misc() are redundant since they are intiialized in rxe_init_task(). This patch removes them. Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_qp.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index a62bab88415c..57c3f05ad15b 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -172,10 +172,6 @@ static void rxe_qp_init_misc(struct rxe_dev *rxe, struct rxe_qp *qp, spin_lock_init(&qp->state_lock); - spin_lock_init(&qp->req.task.state_lock); - spin_lock_init(&qp->resp.task.state_lock); - spin_lock_init(&qp->comp.task.state_lock); - spin_lock_init(&qp->sq.sq_lock); spin_lock_init(&qp->rq.producer_lock); spin_lock_init(&qp->rq.consumer_lock); From patchwork Fri Oct 21 20:01:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015354 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3DCAC433FE for ; Fri, 21 Oct 2022 20:02:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230190AbiJUUCr (ORCPT ); Fri, 21 Oct 2022 16:02:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230308AbiJUUCO (ORCPT ); Fri, 21 Oct 2022 16:02:14 -0400 Received: from mail-oa1-x2a.google.com (mail-oa1-x2a.google.com [IPv6:2001:4860:4864:20::2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0055526206D for ; Fri, 21 Oct 2022 13:02:04 -0700 (PDT) Received: by mail-oa1-x2a.google.com with SMTP id 586e51a60fabf-1321a1e94b3so4942982fac.1 for ; Fri, 21 Oct 2022 13:02:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MfHzARTUOwlxk5UNCo6aVjcUHyvPDj/ohKpbqfaDQFY=; b=f3ZpvGDLIi4SdSSRQ7TUp3F9tXc8P5ppdQlx/UvIiMmQ2VYWFYa+xPdd6lR/Dmj0Ba J1AtTgwSJ1z2hbPHfkzUoj4X2i4gm8kRG7JaXn0eeB1var/QKAw2edlWh+ZS91ii3Y2c HNN9MxQX4GB/PU7gEIzS7X9SBe6aUFeWMt+tDeYZTPE3hHOUYEGVRnIdUoCBeiNDrWus pmr/B0vuHWqvWD/slMnsZL9/dxG1Yvc/5bfEYOhlnu+OhIG+/mEznjNyj4c4gZOh/3rw 9H7vWa/piZ3thGXxR4LUUCNRgee8TiYSJrPtxqS+rWDkIyv6/dhFGyNieIKlwC9d6X1W EyrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MfHzARTUOwlxk5UNCo6aVjcUHyvPDj/ohKpbqfaDQFY=; b=A9QW40Mu8DUg9jgvbZSEOv9c81y8I7lcJVsg3GVdz5praMI5bQl5jOToPpQ/vQ1B35 Se3maV/C+dItpSmAhKID8831USh5Y03dH6VTMPEuPerb9sxWjbVHnG5bir/6gYf6oxv8 uTBj8PEdhvLgaVvxFvtizmQxSCiQ15Fa6C63v9+nfyBVXg70TmOEkWIEWF0PB4k230DL iFdKI6u51+r3PJ/rZSA7tkCm4SpaWZFLnoHu2Okz8sR523RLr8alh50FspdwIITaNEbx U1ExnSai3AAfhYvyrneDd7LbD9wCz2StyZVNVkdMBlk027tac8OavziS2ULBNiSRbwoD Hkpg== X-Gm-Message-State: ACrzQf1BQFmGh38dOmmO5nYwrF52VhoVC2yXs/RHOwKrekK+NMc9aUcf PAtxtwx+d+ch6v22Q+eoimM= X-Google-Smtp-Source: AMsMyM7cOuikro8415Tfh2beIGx7L0/qRjWZl/dE1yCrimMy/bT7QWWOWMT2Oqh/mOJumm03XotdGQ== X-Received: by 2002:a05:6871:8a3:b0:13b:18ef:e8df with SMTP id r35-20020a05687108a300b0013b18efe8dfmr4529044oaq.181.1666382524365; Fri, 21 Oct 2022 13:02:04 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:04 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 03/18] RDMA/rxe: Removed unused name from rxe_task struct Date: Fri, 21 Oct 2022 15:01:04 -0500 Message-Id: <20221021200118.2163-4-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The name field in struct rxe_task is never used. This patch removes it. Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_qp.c | 9 +++------ drivers/infiniband/sw/rxe/rxe_task.c | 4 +--- drivers/infiniband/sw/rxe/rxe_task.h | 4 +--- 3 files changed, 5 insertions(+), 12 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index 57c3f05ad15b..03bd9f3e9956 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -238,10 +238,8 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, skb_queue_head_init(&qp->req_pkts); - rxe_init_task(&qp->req.task, qp, - rxe_requester, "req"); - rxe_init_task(&qp->comp.task, qp, - rxe_completer, "comp"); + rxe_init_task(&qp->req.task, qp, rxe_requester); + rxe_init_task(&qp->comp.task, qp, rxe_completer); qp->qp_timeout_jiffies = 0; /* Can't be set for UD/UC in modify_qp */ if (init->qp_type == IB_QPT_RC) { @@ -288,8 +286,7 @@ static int rxe_qp_init_resp(struct rxe_dev *rxe, struct rxe_qp *qp, skb_queue_head_init(&qp->resp_pkts); - rxe_init_task(&qp->resp.task, qp, - rxe_responder, "resp"); + rxe_init_task(&qp->resp.task, qp, rxe_responder); qp->resp.opcode = OPCODE_NONE; qp->resp.msn = 0; diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c index 3fbaba9eec39..0cbba455fefd 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.c +++ b/drivers/infiniband/sw/rxe/rxe_task.c @@ -90,12 +90,10 @@ void rxe_do_task(struct tasklet_struct *t) task->ret = ret; } -int rxe_init_task(struct rxe_task *task, - void *arg, int (*func)(void *), char *name) +int rxe_init_task(struct rxe_task *task, void *arg, int (*func)(void *)) { task->arg = arg; task->func = func; - snprintf(task->name, sizeof(task->name), "%s", name); task->destroyed = false; tasklet_setup(&task->tasklet, rxe_do_task); diff --git a/drivers/infiniband/sw/rxe/rxe_task.h b/drivers/infiniband/sw/rxe/rxe_task.h index 7f612a1c68a7..b3dfd970d1dc 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.h +++ b/drivers/infiniband/sw/rxe/rxe_task.h @@ -25,7 +25,6 @@ struct rxe_task { void *arg; int (*func)(void *arg); int ret; - char name[16]; bool destroyed; }; @@ -34,8 +33,7 @@ struct rxe_task { * arg => parameter to pass to fcn * func => function to call until it returns != 0 */ -int rxe_init_task(struct rxe_task *task, - void *arg, int (*func)(void *), char *name); +int rxe_init_task(struct rxe_task *task, void *arg, int (*func)(void *)); /* cleanup task */ void rxe_cleanup_task(struct rxe_task *task); From patchwork Fri Oct 21 20:01:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015355 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23353C3A59D for ; Fri, 21 Oct 2022 20:02:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230131AbiJUUCs (ORCPT ); Fri, 21 Oct 2022 16:02:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48304 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230309AbiJUUCO (ORCPT ); Fri, 21 Oct 2022 16:02:14 -0400 Received: from mail-oa1-x33.google.com (mail-oa1-x33.google.com [IPv6:2001:4860:4864:20::33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C42C32623CB for ; Fri, 21 Oct 2022 13:02:06 -0700 (PDT) Received: by mail-oa1-x33.google.com with SMTP id 586e51a60fabf-13aeccf12fbso4863957fac.11 for ; Fri, 21 Oct 2022 13:02:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wdnveJIHom1+SWg66s8BAF1Vf7hnIpTnaAtvPa7O3rg=; b=JBen8dtGwqFl1578W1lCZfXPpFCfAMuF0yivx74yUd1pxzMSjvu1olhLiMsiM9lkPT iJT26w7BPoUgt4WNVyS2lzhuP5Ampi/SuezueH1cccAzoYa+t6IfxAevhKj2r5E0sCy6 aPkVIZ2mIP4kSiM8qe1t/KMmUJnCfPjutTKZZystL7HHm0FWQJv+6SFFUu3uvcy0WI94 iBQKplXEJkSOowGppMeYxUXBUIEaOmUZy+uJxglKM9XoQgCBRBgWGD/wD4bv/PLLT6EP UkMmu9W+o6thudg1ZH0E9Md/Rl3Xf2jPSq5Lj7SA0OGLbXsYkM1/suOvpVmWWGyylQSe KbVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wdnveJIHom1+SWg66s8BAF1Vf7hnIpTnaAtvPa7O3rg=; b=P1hdYjYCsUhrsqWWjPBX0vcxP9kxq9kYtnSNRAiJ2GzF7T/Fp/Yh706M5sXV890b9R L+u6cLC1hbjpa0oeVIYyw4NvkK+p5oSbXy2vcBYLQjq2IbUFkzXypZTtdmw+ZdnjKX9y jZ7gdOCAii8KKlaOjBWbA1PJfEprUL9I+TIyEJroZa11SxJ9VCUQ4M2EwOfIS/W0c6cP MB3k/r9Vh9o6A5I7N/57zscDkQ1zpDocCU2bkQ9u7x7hSXNVHo7ZtT3sHp9NV3IktxlK Lqh+ALo7mA4tFmuvWQTj1nKt77TDM0m8lWZGEY3omd9Ds74GsrXF9VPxyZZNRHEez1Ci e3Rg== X-Gm-Message-State: ACrzQf3NDq2znQAKlmY+RyB8Tx9REm+rHZz+Zcl6+p43od7sBiUwz/kG LD/1AeF8CvP8/Aq5tv21yuU= X-Google-Smtp-Source: AMsMyM5eou4kTkg7fEiMzw0ngC2YNaOE45pISu3/C59qOf0dCfZ4GU9vlczyS4/4kK3BvRiltvz4XQ== X-Received: by 2002:a05:6870:8199:b0:13a:e10a:4b2 with SMTP id k25-20020a056870819900b0013ae10a04b2mr8271079oae.110.1666382525817; Fri, 21 Oct 2022 13:02:05 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:05 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 04/18] RDMA/rxe: Split rxe_run_task() into two subroutines Date: Fri, 21 Oct 2022 15:01:05 -0500 Message-Id: <20221021200118.2163-5-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Split rxe_run_task(task, sched) into rxe_run_task(task) and rxe_sched_task(task). Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_comp.c | 19 +++++++++++-------- drivers/infiniband/sw/rxe/rxe_net.c | 4 ++-- drivers/infiniband/sw/rxe/rxe_qp.c | 10 +++++----- drivers/infiniband/sw/rxe/rxe_req.c | 10 +++++----- drivers/infiniband/sw/rxe/rxe_resp.c | 5 ++++- drivers/infiniband/sw/rxe/rxe_task.c | 15 ++++++++++----- drivers/infiniband/sw/rxe/rxe_task.h | 7 +++---- drivers/infiniband/sw/rxe/rxe_verbs.c | 8 ++++---- 8 files changed, 44 insertions(+), 34 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index fb0c008af78c..d2a250123617 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -118,7 +118,7 @@ void retransmit_timer(struct timer_list *t) if (qp->valid) { qp->comp.timeout = 1; - rxe_run_task(&qp->comp.task, 1); + rxe_sched_task(&qp->comp.task); } } @@ -132,7 +132,10 @@ void rxe_comp_queue_pkt(struct rxe_qp *qp, struct sk_buff *skb) if (must_sched != 0) rxe_counter_inc(SKB_TO_PKT(skb)->rxe, RXE_CNT_COMPLETER_SCHED); - rxe_run_task(&qp->comp.task, must_sched); + if (must_sched) + rxe_sched_task(&qp->comp.task); + else + rxe_run_task(&qp->comp.task); } static inline enum comp_state get_wqe(struct rxe_qp *qp, @@ -305,7 +308,7 @@ static inline enum comp_state check_ack(struct rxe_qp *qp, qp->comp.psn = pkt->psn; if (qp->req.wait_psn) { qp->req.wait_psn = 0; - rxe_run_task(&qp->req.task, 0); + rxe_run_task(&qp->req.task); } } return COMPST_ERROR_RETRY; @@ -452,7 +455,7 @@ static void do_complete(struct rxe_qp *qp, struct rxe_send_wqe *wqe) */ if (qp->req.wait_fence) { qp->req.wait_fence = 0; - rxe_run_task(&qp->req.task, 0); + rxe_run_task(&qp->req.task); } } @@ -466,7 +469,7 @@ static inline enum comp_state complete_ack(struct rxe_qp *qp, if (qp->req.need_rd_atomic) { qp->comp.timeout_retry = 0; qp->req.need_rd_atomic = 0; - rxe_run_task(&qp->req.task, 0); + rxe_run_task(&qp->req.task); } } @@ -512,7 +515,7 @@ static inline enum comp_state complete_wqe(struct rxe_qp *qp, if (qp->req.wait_psn) { qp->req.wait_psn = 0; - rxe_run_task(&qp->req.task, 1); + rxe_sched_task(&qp->req.task); } } @@ -646,7 +649,7 @@ int rxe_completer(void *arg) if (qp->req.wait_psn) { qp->req.wait_psn = 0; - rxe_run_task(&qp->req.task, 1); + rxe_sched_task(&qp->req.task); } state = COMPST_DONE; @@ -714,7 +717,7 @@ int rxe_completer(void *arg) RXE_CNT_COMP_RETRY); qp->req.need_retry = 1; qp->comp.started_retry = 1; - rxe_run_task(&qp->req.task, 0); + rxe_run_task(&qp->req.task); } goto done; diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index 35f327b9d4b8..c36cad9c7a66 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -345,7 +345,7 @@ static void rxe_skb_tx_dtor(struct sk_buff *skb) if (unlikely(qp->need_req_skb && skb_out < RXE_INFLIGHT_SKBS_PER_QP_LOW)) - rxe_run_task(&qp->req.task, 1); + rxe_sched_task(&qp->req.task); rxe_put(qp); } @@ -429,7 +429,7 @@ int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_info *pkt, if ((qp_type(qp) != IB_QPT_RC) && (pkt->mask & RXE_END_MASK)) { pkt->wqe->state = wqe_state_done; - rxe_run_task(&qp->comp.task, 1); + rxe_sched_task(&qp->comp.task); } rxe_counter_inc(rxe, RXE_CNT_SENT_PKTS); diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index 03bd9f3e9956..3f6d62a80bea 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -536,10 +536,10 @@ static void rxe_qp_drain(struct rxe_qp *qp) if (qp->req.state != QP_STATE_DRAINED) { qp->req.state = QP_STATE_DRAIN; if (qp_type(qp) == IB_QPT_RC) - rxe_run_task(&qp->comp.task, 1); + rxe_sched_task(&qp->comp.task); else __rxe_do_task(&qp->comp.task); - rxe_run_task(&qp->req.task, 1); + rxe_sched_task(&qp->req.task); } } } @@ -553,13 +553,13 @@ void rxe_qp_error(struct rxe_qp *qp) qp->attr.qp_state = IB_QPS_ERR; /* drain work and packet queues */ - rxe_run_task(&qp->resp.task, 1); + rxe_sched_task(&qp->resp.task); if (qp_type(qp) == IB_QPT_RC) - rxe_run_task(&qp->comp.task, 1); + rxe_sched_task(&qp->comp.task); else __rxe_do_task(&qp->comp.task); - rxe_run_task(&qp->req.task, 1); + rxe_sched_task(&qp->req.task); } /* called by the modify qp verb */ diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index f63771207970..41f1d84f0acb 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -105,7 +105,7 @@ void rnr_nak_timer(struct timer_list *t) /* request a send queue retry */ qp->req.need_retry = 1; qp->req.wait_for_rnr_timer = 0; - rxe_run_task(&qp->req.task, 1); + rxe_sched_task(&qp->req.task); } static struct rxe_send_wqe *req_next_wqe(struct rxe_qp *qp) @@ -608,7 +608,7 @@ static int rxe_do_local_ops(struct rxe_qp *qp, struct rxe_send_wqe *wqe) * which can lead to a deadlock. So go ahead and complete * it now. */ - rxe_run_task(&qp->comp.task, 1); + rxe_sched_task(&qp->comp.task); return 0; } @@ -733,7 +733,7 @@ int rxe_requester(void *arg) qp->req.wqe_index); wqe->state = wqe_state_done; wqe->status = IB_WC_SUCCESS; - rxe_run_task(&qp->comp.task, 0); + rxe_run_task(&qp->comp.task); goto done; } payload = mtu; @@ -795,7 +795,7 @@ int rxe_requester(void *arg) rollback_state(wqe, qp, &rollback_wqe, rollback_psn); if (err == -EAGAIN) { - rxe_run_task(&qp->req.task, 1); + rxe_sched_task(&qp->req.task); goto exit; } @@ -817,7 +817,7 @@ int rxe_requester(void *arg) qp->req.wqe_index = queue_next_index(qp->sq.queue, qp->req.wqe_index); wqe->state = wqe_state_error; qp->req.state = QP_STATE_ERROR; - rxe_run_task(&qp->comp.task, 0); + rxe_run_task(&qp->comp.task); exit: ret = -EAGAIN; out: diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index ed5a09e86417..2a2a50de51b2 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -91,7 +91,10 @@ void rxe_resp_queue_pkt(struct rxe_qp *qp, struct sk_buff *skb) must_sched = (pkt->opcode == IB_OPCODE_RC_RDMA_READ_REQUEST) || (skb_queue_len(&qp->req_pkts) > 1); - rxe_run_task(&qp->resp.task, must_sched); + if (must_sched) + rxe_sched_task(&qp->resp.task); + else + rxe_run_task(&qp->resp.task); } static inline enum resp_states get_req(struct rxe_qp *qp, diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c index 0cbba455fefd..442b7348acdc 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.c +++ b/drivers/infiniband/sw/rxe/rxe_task.c @@ -123,15 +123,20 @@ void rxe_cleanup_task(struct rxe_task *task) tasklet_kill(&task->tasklet); } -void rxe_run_task(struct rxe_task *task, int sched) +void rxe_run_task(struct rxe_task *task) { if (task->destroyed) return; - if (sched) - tasklet_schedule(&task->tasklet); - else - rxe_do_task(&task->tasklet); + rxe_do_task(&task->tasklet); +} + +void rxe_sched_task(struct rxe_task *task) +{ + if (task->destroyed) + return; + + tasklet_schedule(&task->tasklet); } void rxe_disable_task(struct rxe_task *task) diff --git a/drivers/infiniband/sw/rxe/rxe_task.h b/drivers/infiniband/sw/rxe/rxe_task.h index b3dfd970d1dc..590b1c1d7e7c 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.h +++ b/drivers/infiniband/sw/rxe/rxe_task.h @@ -52,10 +52,9 @@ int __rxe_do_task(struct rxe_task *task); */ void rxe_do_task(struct tasklet_struct *t); -/* run a task, else schedule it to run as a tasklet, The decision - * to run or schedule tasklet is based on the parameter sched. - */ -void rxe_run_task(struct rxe_task *task, int sched); +void rxe_run_task(struct rxe_task *task); + +void rxe_sched_task(struct rxe_task *task); /* keep a task from scheduling */ void rxe_disable_task(struct rxe_task *task); diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 88825edc7dce..f2f82efbaf6d 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -695,9 +695,9 @@ static int rxe_post_send_kernel(struct rxe_qp *qp, const struct ib_send_wr *wr, wr = next; } - rxe_run_task(&qp->req.task, 1); + rxe_sched_task(&qp->req.task); if (unlikely(qp->req.state == QP_STATE_ERROR)) - rxe_run_task(&qp->comp.task, 1); + rxe_sched_task(&qp->comp.task); return err; } @@ -719,7 +719,7 @@ static int rxe_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr, if (qp->is_user) { /* Utilize process context to do protocol processing */ - rxe_run_task(&qp->req.task, 0); + rxe_run_task(&qp->req.task); return 0; } else return rxe_post_send_kernel(qp, wr, bad_wr); @@ -759,7 +759,7 @@ static int rxe_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr, spin_unlock_irqrestore(&rq->producer_lock, flags); if (qp->resp.state == QP_STATE_ERROR) - rxe_run_task(&qp->resp.task, 1); + rxe_sched_task(&qp->resp.task); err1: return err; From patchwork Fri Oct 21 20:01:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015356 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75B94C38A2D for ; Fri, 21 Oct 2022 20:02:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230230AbiJUUCu (ORCPT ); Fri, 21 Oct 2022 16:02:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48366 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230327AbiJUUCS (ORCPT ); Fri, 21 Oct 2022 16:02:18 -0400 Received: from mail-oi1-x22f.google.com (mail-oi1-x22f.google.com [IPv6:2607:f8b0:4864:20::22f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B2D332623DA for ; Fri, 21 Oct 2022 13:02:07 -0700 (PDT) Received: by mail-oi1-x22f.google.com with SMTP id g130so4382116oia.13 for ; Fri, 21 Oct 2022 13:02:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BFM9klmBW4ISA+589VLKd91C/jSg7NSbf9UoDxhQrS4=; b=KfyVo5KZTRiLP5gj/wAR0mzYfmYplM3DAS8l5DFGmqGxXC8RuE5UXKvY7xpHhNnJwY QyWS9jTUcJPXUxv/J/6sql82v9qeJsm0lZTyBbVcvTA/zM/HUnk0tclHlr0vG6vHJnXJ 4yxqzH73IgETmLjBJCS8M2G1CHriv73Imee9bW8QKcszgFVUZ7f46w4ATs9RCUXTjy3R ET8KCbmyV29B6zDjHBEXVcmkr2SAjyVUE97PDWM4EKBU7ensBfHEs1sDlXtg6lCJAry0 ghh2QeW+P3QXoy+1CBgmpgrIR/b09ieHKv+9z50OTQO5AaVYqcVmAL6WkNFW7DXdaixe EyBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BFM9klmBW4ISA+589VLKd91C/jSg7NSbf9UoDxhQrS4=; b=iqL8fIvIet8aDnzTT6tgw2gFTY3KRL2B7S+KtViCbzOwi8SqaLe2nFLjkTGgqB0XsY WTsvOJYrNdh0NurT4iqwT6/+H9AMPABbcZ51OLoxdXHFyrFkETH9lFfc1Zjv2PJadD/t uFweg4zXULifLERaN9BIB5m8Ykinfgppt/5vpzP3VWALI0ooc4THFQUbTZKsaMbx5WrY BpMArUGM9LSCSX7Q3UFedacXpkstJWE+mfGsY5EVbHk+l29TNKGmhnbC7PqqF7p/g8Ma SYkl2DNvHxGNV1Ej8nWOo8hznlW1nIOHSdFARuqxs1kQLuPa9CrMKROHaRZM15FDB2jq q5Fw== X-Gm-Message-State: ACrzQf24EYqzqR5l6/S4QwHm/7JlUosgD2LL7jXa0sCEtYz3lwDOxZcG j8LWQg4v+gzWBEPAMN6xOAs= X-Google-Smtp-Source: AMsMyM7iHzwj3/GIDFYtJ5j58Q5WoTKcUw4nw72rVezMJP2KrC83GGlAkDCaINN8NIf7iqlTcXVDfA== X-Received: by 2002:a05:6808:1209:b0:353:92d1:2a0 with SMTP id a9-20020a056808120900b0035392d102a0mr11011315oil.51.1666382526970; Fri, 21 Oct 2022 13:02:06 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:06 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 05/18] RDMA/rxe: Make rxe_do_task static Date: Fri, 21 Oct 2022 15:01:06 -0500 Message-Id: <20221021200118.2163-6-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The subroutine rxe_do_task() is only called in rxe_task.c. This patch makes it static and renames it do_task(). Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_task.c | 6 +++--- drivers/infiniband/sw/rxe/rxe_task.h | 8 -------- 2 files changed, 3 insertions(+), 11 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c index 442b7348acdc..fb953f5195b8 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.c +++ b/drivers/infiniband/sw/rxe/rxe_task.c @@ -24,7 +24,7 @@ int __rxe_do_task(struct rxe_task *task) * a second caller finds the task already running * but looks just after the last call to func */ -void rxe_do_task(struct tasklet_struct *t) +static void do_task(struct tasklet_struct *t) { int cont; int ret; @@ -96,7 +96,7 @@ int rxe_init_task(struct rxe_task *task, void *arg, int (*func)(void *)) task->func = func; task->destroyed = false; - tasklet_setup(&task->tasklet, rxe_do_task); + tasklet_setup(&task->tasklet, do_task); task->state = TASK_STATE_START; spin_lock_init(&task->state_lock); @@ -128,7 +128,7 @@ void rxe_run_task(struct rxe_task *task) if (task->destroyed) return; - rxe_do_task(&task->tasklet); + do_task(&task->tasklet); } void rxe_sched_task(struct rxe_task *task) diff --git a/drivers/infiniband/sw/rxe/rxe_task.h b/drivers/infiniband/sw/rxe/rxe_task.h index 590b1c1d7e7c..99e0173e5c46 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.h +++ b/drivers/infiniband/sw/rxe/rxe_task.h @@ -44,14 +44,6 @@ void rxe_cleanup_task(struct rxe_task *task); */ int __rxe_do_task(struct rxe_task *task); -/* - * common function called by any of the main tasklets - * If there is any chance that there is additional - * work to do someone must reschedule the task before - * leaving - */ -void rxe_do_task(struct tasklet_struct *t); - void rxe_run_task(struct rxe_task *task); void rxe_sched_task(struct rxe_task *task); From patchwork Fri Oct 21 20:01:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015357 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 245A8C433FE for ; Fri, 21 Oct 2022 20:02:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230262AbiJUUCv (ORCPT ); Fri, 21 Oct 2022 16:02:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48500 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230382AbiJUUCW (ORCPT ); Fri, 21 Oct 2022 16:02:22 -0400 Received: from mail-oa1-x2b.google.com (mail-oa1-x2b.google.com [IPv6:2001:4860:4864:20::2b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82E9F26207F for ; Fri, 21 Oct 2022 13:02:08 -0700 (PDT) Received: by mail-oa1-x2b.google.com with SMTP id 586e51a60fabf-1364357a691so4894230fac.7 for ; Fri, 21 Oct 2022 13:02:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xWKHDmzwNMn+q2ooSjlIQRw6Jqrwp6NxfB1YrN3XR2o=; b=Bj2fkOdp4w2S0KWGbLDd6U3o8bNcCLSNt74349XcBihVS7F6mT3hDZgMUljs/k0imp yQTQj62frWES3QHSyS+tbIvxYjUFK+/GeLheIKapWnklOAbpOFgglx2wK/t9oZl1RBi+ /ekPQ3bONWAGED61YcNX1zrVj2Z+EKUtAx8NTLOuRGLe66pbRbBc/JYeMSXNdEOyomvn ZpRa4ogqzge9doJp912Dk2W47C0HsxkDm4ijv6jJD0iholr9nVDwSOQRNrM07blv0pys 048J8eYtPYLMfAas5T1A6BsDh06X7o4Ots6/s0UGfYhSZZHnsiVSiYJsxDoSFzMz/I98 JFmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xWKHDmzwNMn+q2ooSjlIQRw6Jqrwp6NxfB1YrN3XR2o=; b=kU88FqdrPtFwB6iVjpPY83JWoH6HsMYidO/FIk9LCQbIS1H1wPBnvo415dJFONd8Zt x2MPq/XhoTRbAfbpC6Kl85tNquJZSezdii9MEIDo0vZcuJqKeozDyeA04PSMGRhwt+B7 5WOVSDXF2Gr3aZfmyRZi5N/WhJBy9yJMk7cJtrKCfHv72rWTNgZCwPnBwv9yFnweilU3 T+cK+V8GNnE1nmlFOwx6SN3DOoWnpBAQ46sZEGbUIBw0qA7jiSXGCQ7wclK4ME5u+VMN yerE/OAmAKRi57DVy7A8ONvopTscjR3lENqqNr1paMn2rlq7WAc7JtBF8onIjMFGTVBs PVkw== X-Gm-Message-State: ACrzQf38nmTDZG9iWP6rWU5JXg5l2uqZM6FmJzkHMjU+XhCEwcDwOhu0 0m4E2Klzi5zjZWZUd6YFVz6ymAD3QzM= X-Google-Smtp-Source: AMsMyM4sC7Gh3EthPSJP8SivhPQGbMtQVfPtlzQmg9i5KfMVrgmSnzdGzckWdD+hVzwtzBkCRO2Y3w== X-Received: by 2002:a05:6870:1611:b0:132:d73d:8f7f with SMTP id b17-20020a056870161100b00132d73d8f7fmr29951067oae.250.1666382528220; Fri, 21 Oct 2022 13:02:08 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:07 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 06/18] RDMA/rxe: Rename task->state_lock to task->lock Date: Fri, 21 Oct 2022 15:01:07 -0500 Message-Id: <20221021200118.2163-7-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Rename task-state_lock to task->lock Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_task.c | 18 +++++++++--------- drivers/infiniband/sw/rxe/rxe_task.h | 2 +- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c index fb953f5195b8..0208d833a41b 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.c +++ b/drivers/infiniband/sw/rxe/rxe_task.c @@ -31,22 +31,22 @@ static void do_task(struct tasklet_struct *t) struct rxe_task *task = from_tasklet(task, t, tasklet); unsigned int iterations = RXE_MAX_ITERATIONS; - spin_lock_bh(&task->state_lock); + spin_lock_bh(&task->lock); switch (task->state) { case TASK_STATE_START: task->state = TASK_STATE_BUSY; - spin_unlock_bh(&task->state_lock); + spin_unlock_bh(&task->lock); break; case TASK_STATE_BUSY: task->state = TASK_STATE_ARMED; fallthrough; case TASK_STATE_ARMED: - spin_unlock_bh(&task->state_lock); + spin_unlock_bh(&task->lock); return; default: - spin_unlock_bh(&task->state_lock); + spin_unlock_bh(&task->lock); pr_warn("%s failed with bad state %d\n", __func__, task->state); return; } @@ -55,7 +55,7 @@ static void do_task(struct tasklet_struct *t) cont = 0; ret = task->func(task->arg); - spin_lock_bh(&task->state_lock); + spin_lock_bh(&task->lock); switch (task->state) { case TASK_STATE_BUSY: if (ret) { @@ -84,7 +84,7 @@ static void do_task(struct tasklet_struct *t) pr_warn("%s failed with bad state %d\n", __func__, task->state); } - spin_unlock_bh(&task->state_lock); + spin_unlock_bh(&task->lock); } while (cont); task->ret = ret; @@ -99,7 +99,7 @@ int rxe_init_task(struct rxe_task *task, void *arg, int (*func)(void *)) tasklet_setup(&task->tasklet, do_task); task->state = TASK_STATE_START; - spin_lock_init(&task->state_lock); + spin_lock_init(&task->lock); return 0; } @@ -115,9 +115,9 @@ void rxe_cleanup_task(struct rxe_task *task) task->destroyed = true; do { - spin_lock_bh(&task->state_lock); + spin_lock_bh(&task->lock); idle = (task->state == TASK_STATE_START); - spin_unlock_bh(&task->state_lock); + spin_unlock_bh(&task->lock); } while (!idle); tasklet_kill(&task->tasklet); diff --git a/drivers/infiniband/sw/rxe/rxe_task.h b/drivers/infiniband/sw/rxe/rxe_task.h index 99e0173e5c46..7b88129702ac 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.h +++ b/drivers/infiniband/sw/rxe/rxe_task.h @@ -21,7 +21,7 @@ enum { struct rxe_task { struct tasklet_struct tasklet; int state; - spinlock_t state_lock; /* spinlock for task state */ + spinlock_t lock; void *arg; int (*func)(void *arg); int ret; From patchwork Fri Oct 21 20:01:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015358 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12C25C3A59D for ; Fri, 21 Oct 2022 20:02:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230408AbiJUUCx (ORCPT ); Fri, 21 Oct 2022 16:02:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230383AbiJUUCW (ORCPT ); Fri, 21 Oct 2022 16:02:22 -0400 Received: from mail-oi1-x22a.google.com (mail-oi1-x22a.google.com [IPv6:2607:f8b0:4864:20::22a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B83D263B61 for ; Fri, 21 Oct 2022 13:02:10 -0700 (PDT) Received: by mail-oi1-x22a.google.com with SMTP id u15so4457444oie.2 for ; Fri, 21 Oct 2022 13:02:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EU9/rBYN2hXS/v49sgDZV0QWnssuEz3K4CW9gz24luE=; b=PmDQW56WN4OSQGG0PUJgyQJJdBPIXDHZ+eF1UP7UaDHMenw0+LTQDfwoDrCQkjjFmH fzLm3lxr3MFCVFkfkMVX/VlB0yAY3gmQUH9FLNyqIPOrtPuvOnBjRbUkGPMDNMbJcsqt j9OOO652ELr28L5zN6wkp/RilqC4YaO/MbETHtJOjdOyx1UZIpDSI9/7i4NMohQZ7FP8 hYgkNfD5Nc+4gw/6gZxznRjGTdPqw6uJU8VH9fa+L0MyjsnlqWwDeh7ComVWg+GCCXC+ r8SzJc4dX84FX0PBmRePXHuKR3kDMoUSchd3E71DPyI9f4bFTFCwk+nnkuFKnAkb5eJR Ks0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EU9/rBYN2hXS/v49sgDZV0QWnssuEz3K4CW9gz24luE=; b=USht4Mljbq3Mk76FPlC/kk65slB+Ckpxtt33wCAFUALWq+Qegey2KvDxzZCf3XXDna LaS8ym2PYRzvRQUYaWYJArUNLcDAos0DIEcpROdFJixSsVuAJPB6AOS2I0ut8ZTXUN+E PyNaRBai2nSplj9+oZndLdUfVDW/usr1h1E/Ty0MmHGI6NfciR2hAOBy/IkIRfUVpZOv yGZwWaQ6lj+AuuXmCzGv3wY9g0ysop+GkKUoLR5vpDeBcLB+UTpiX1Ujc6Yz/7lnzgoQ LB8X4lZtIyOg8sKVWixdWxLh+aHYIQjbM6XkooAAa50Kz5VCqA78eJN1CqIIO9wIkCl9 SJBw== X-Gm-Message-State: ACrzQf0y13AhMKYbkGQQrB/lF2Db+yNN1OGJLHqJ+xlLUw/qZd6G8pv1 0RX3zMU9SBDpI1dtYFSy6ZA= X-Google-Smtp-Source: AMsMyM5uy4XXmoBGcbA54qkf4g+GlEzgqIn9bYL/Wd2u0OaVzwlYUhTwO10NaJnzz3Vy/XCDRIt9yg== X-Received: by 2002:a05:6808:aa6:b0:354:94e2:ddc with SMTP id r6-20020a0568080aa600b0035494e20ddcmr25222242oij.192.1666382529531; Fri, 21 Oct 2022 13:02:09 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:09 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 07/18] RDMA/rxe: Make task interface pluggable Date: Fri, 21 Oct 2022 15:01:08 -0500 Message-Id: <20221021200118.2163-8-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Make the internal interface to the task operations pluggable and add a new 'inline' type. Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_qp.c | 8 +- drivers/infiniband/sw/rxe/rxe_task.c | 160 ++++++++++++++++++++++----- drivers/infiniband/sw/rxe/rxe_task.h | 44 +++++--- 3 files changed, 165 insertions(+), 47 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index 3f6d62a80bea..b5e108794aa1 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -238,8 +238,10 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, skb_queue_head_init(&qp->req_pkts); - rxe_init_task(&qp->req.task, qp, rxe_requester); - rxe_init_task(&qp->comp.task, qp, rxe_completer); + rxe_init_task(&qp->req.task, qp, rxe_requester, RXE_TASK_TYPE_TASKLET); + rxe_init_task(&qp->comp.task, qp, rxe_completer, + (qp_type(qp) == IB_QPT_RC) ? RXE_TASK_TYPE_TASKLET : + RXE_TASK_TYPE_INLINE); qp->qp_timeout_jiffies = 0; /* Can't be set for UD/UC in modify_qp */ if (init->qp_type == IB_QPT_RC) { @@ -286,7 +288,7 @@ static int rxe_qp_init_resp(struct rxe_dev *rxe, struct rxe_qp *qp, skb_queue_head_init(&qp->resp_pkts); - rxe_init_task(&qp->resp.task, qp, rxe_responder); + rxe_init_task(&qp->resp.task, qp, rxe_responder, RXE_TASK_TYPE_TASKLET); qp->resp.opcode = OPCODE_NONE; qp->resp.msn = 0; diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c index 0208d833a41b..8dfbfa164eff 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.c +++ b/drivers/infiniband/sw/rxe/rxe_task.c @@ -24,12 +24,11 @@ int __rxe_do_task(struct rxe_task *task) * a second caller finds the task already running * but looks just after the last call to func */ -static void do_task(struct tasklet_struct *t) +static void do_task(struct rxe_task *task) { + unsigned int iterations = RXE_MAX_ITERATIONS; int cont; int ret; - struct rxe_task *task = from_tasklet(task, t, tasklet); - unsigned int iterations = RXE_MAX_ITERATIONS; spin_lock_bh(&task->lock); switch (task->state) { @@ -90,28 +89,21 @@ static void do_task(struct tasklet_struct *t) task->ret = ret; } -int rxe_init_task(struct rxe_task *task, void *arg, int (*func)(void *)) +static void disable_task(struct rxe_task *task) { - task->arg = arg; - task->func = func; - task->destroyed = false; - - tasklet_setup(&task->tasklet, do_task); - - task->state = TASK_STATE_START; - spin_lock_init(&task->lock); + /* todo */ +} - return 0; +static void enable_task(struct rxe_task *task) +{ + /* todo */ } -void rxe_cleanup_task(struct rxe_task *task) +/* busy wait until any previous tasks are done */ +static void cleanup_task(struct rxe_task *task) { bool idle; - /* - * Mark the task, then wait for it to finish. It might be - * running in a non-tasklet (direct call) context. - */ task->destroyed = true; do { @@ -119,32 +111,144 @@ void rxe_cleanup_task(struct rxe_task *task) idle = (task->state == TASK_STATE_START); spin_unlock_bh(&task->lock); } while (!idle); +} - tasklet_kill(&task->tasklet); +/* silently treat schedule as inline for inline tasks */ +static void inline_sched(struct rxe_task *task) +{ + do_task(task); } -void rxe_run_task(struct rxe_task *task) +static void inline_run(struct rxe_task *task) { - if (task->destroyed) - return; + do_task(task); +} - do_task(&task->tasklet); +static void inline_disable(struct rxe_task *task) +{ + disable_task(task); } -void rxe_sched_task(struct rxe_task *task) +static void inline_enable(struct rxe_task *task) { - if (task->destroyed) - return; + enable_task(task); +} + +static void inline_cleanup(struct rxe_task *task) +{ + cleanup_task(task); +} + +static const struct rxe_task_ops inline_ops = { + .sched = inline_sched, + .run = inline_run, + .enable = inline_enable, + .disable = inline_disable, + .cleanup = inline_cleanup, +}; +static void inline_init(struct rxe_task *task) +{ + task->ops = &inline_ops; +} + +/* use tsklet_xxx to avoid name collisions with tasklet_xxx */ +static void tsklet_sched(struct rxe_task *task) +{ tasklet_schedule(&task->tasklet); } -void rxe_disable_task(struct rxe_task *task) +static void tsklet_do_task(struct tasklet_struct *tasklet) { + struct rxe_task *task = container_of(tasklet, typeof(*task), tasklet); + + do_task(task); +} + +static void tsklet_run(struct rxe_task *task) +{ + do_task(task); +} + +static void tsklet_disable(struct rxe_task *task) +{ + disable_task(task); tasklet_disable(&task->tasklet); } -void rxe_enable_task(struct rxe_task *task) +static void tsklet_enable(struct rxe_task *task) { tasklet_enable(&task->tasklet); + enable_task(task); +} + +static void tsklet_cleanup(struct rxe_task *task) +{ + cleanup_task(task); + tasklet_kill(&task->tasklet); +} + +static const struct rxe_task_ops tsklet_ops = { + .sched = tsklet_sched, + .run = tsklet_run, + .enable = tsklet_enable, + .disable = tsklet_disable, + .cleanup = tsklet_cleanup, +}; + +static void tsklet_init(struct rxe_task *task) +{ + tasklet_setup(&task->tasklet, tsklet_do_task); + task->ops = &tsklet_ops; +} + +int rxe_init_task(struct rxe_task *task, void *arg, int (*func)(void *), + enum rxe_task_type type) +{ + task->arg = arg; + task->func = func; + task->destroyed = false; + task->type = type; + task->state = TASK_STATE_START; + + spin_lock_init(&task->lock); + + switch (type) { + case RXE_TASK_TYPE_INLINE: + inline_init(task); + break; + case RXE_TASK_TYPE_TASKLET: + tsklet_init(task); + break; + default: + pr_debug("%s: invalid task type = %d\n", __func__, type); + return -EINVAL; + } + + return 0; +} + +void rxe_sched_task(struct rxe_task *task) +{ + task->ops->sched(task); +} + +void rxe_run_task(struct rxe_task *task) +{ + task->ops->run(task); +} + +void rxe_enable_task(struct rxe_task *task) +{ + task->ops->enable(task); +} + +void rxe_disable_task(struct rxe_task *task) +{ + task->ops->disable(task); +} + +void rxe_cleanup_task(struct rxe_task *task) +{ + task->ops->cleanup(task); } diff --git a/drivers/infiniband/sw/rxe/rxe_task.h b/drivers/infiniband/sw/rxe/rxe_task.h index 7b88129702ac..31963129ff7a 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.h +++ b/drivers/infiniband/sw/rxe/rxe_task.h @@ -7,6 +7,21 @@ #ifndef RXE_TASK_H #define RXE_TASK_H +struct rxe_task; + +struct rxe_task_ops { + void (*sched)(struct rxe_task *task); + void (*run)(struct rxe_task *task); + void (*disable)(struct rxe_task *task); + void (*enable)(struct rxe_task *task); + void (*cleanup)(struct rxe_task *task); +}; + +enum rxe_task_type { + RXE_TASK_TYPE_INLINE = 0, + RXE_TASK_TYPE_TASKLET = 1, +}; + enum { TASK_STATE_START = 0, TASK_STATE_BUSY = 1, @@ -19,24 +34,19 @@ enum { * called again. */ struct rxe_task { - struct tasklet_struct tasklet; - int state; - spinlock_t lock; - void *arg; - int (*func)(void *arg); - int ret; - bool destroyed; + struct tasklet_struct tasklet; + int state; + spinlock_t lock; + void *arg; + int (*func)(void *arg); + int ret; + bool destroyed; + const struct rxe_task_ops *ops; + enum rxe_task_type type; }; -/* - * init rxe_task structure - * arg => parameter to pass to fcn - * func => function to call until it returns != 0 - */ -int rxe_init_task(struct rxe_task *task, void *arg, int (*func)(void *)); - -/* cleanup task */ -void rxe_cleanup_task(struct rxe_task *task); +int rxe_init_task(struct rxe_task *task, void *arg, int (*func)(void *), + enum rxe_task_type type); /* * raw call to func in loop without any checking @@ -54,4 +64,6 @@ void rxe_disable_task(struct rxe_task *task); /* allow task to run */ void rxe_enable_task(struct rxe_task *task); +void rxe_cleanup_task(struct rxe_task *task); + #endif /* RXE_TASK_H */ From patchwork Fri Oct 21 20:01:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015359 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D927C433FE for ; Fri, 21 Oct 2022 20:02:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230383AbiJUUCy (ORCPT ); Fri, 21 Oct 2022 16:02:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48520 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230389AbiJUUCW (ORCPT ); Fri, 21 Oct 2022 16:02:22 -0400 Received: from mail-oa1-x36.google.com (mail-oa1-x36.google.com [IPv6:2001:4860:4864:20::36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 87D5D263B67 for ; Fri, 21 Oct 2022 13:02:11 -0700 (PDT) Received: by mail-oa1-x36.google.com with SMTP id 586e51a60fabf-12c8312131fso4910968fac.4 for ; Fri, 21 Oct 2022 13:02:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RlLIib9VYaRcJOHWfnIenhWFOSeBBzkMaeQ6xNMElCo=; b=WtdKxcNOSO7Q0zAYjpADfP1UvL0cI1lyxzuUOIf+Q80Z+fswgjuzudNVmpDdn3Bbx/ sJGECRF/4psLq4nV/DtFdk+7EEoFs9entbIZgdYAo9EPhvlVw9I/CdN5wXPrHBEySszh OeE3WkU+Oe5YtLXTl+tKVBAEnmTWM4Z4uWgjG2hf+o3eReRcKpb4TGR1v77T1p6aqPe2 1RpmtmPhFEvERU8a3daUyjKCNK+stkD/LlL8L60O5TmV4+JIcet2Q40iZbdNZNpUI3Rw JlHMuUt4BKe3mO4/daSlFm+QW/Uyu94lkTGbmgrS45RDw46bfkPNTZAZtE/OqnBtaHZO bAvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RlLIib9VYaRcJOHWfnIenhWFOSeBBzkMaeQ6xNMElCo=; b=1HN8y9s/nlSrR3Rmlm172r50JybLxzmPypcwcrUX0lMi+8gv7zGfOKgpmEK9PoDzXe /8Jofalk8QgmzVXxEIKrhsghbrOr+qO4oN9a5IwJLBjDpXV0rULxPWT9UyOPZ16RPDKj jlDgrxVsRFXJbU1dW47juT9yUp1vSq+ViIrfJU+eIKFUH6yCsXkxicymj+AaIbOjdRKG f2+EKICfxQsduGWtSMFcjXk6Ml4oCjnuIS9/SFdI6Nr9o94jrjb+LXBptSN3iwcPjK4r 6FVR0ARmT9KH0jISbxBn/j1H4+VXs97BfrMWDKiI4gV2fQIDzhn+COd4QG8IvQE3C8Bm Ok8A== X-Gm-Message-State: ACrzQf0L2kcymlQUXhOf/qs5PXnb2Z9njmKBf3y0ljYmx/ecpnDsZ3EZ g8P9NxHhk7uXoiDeBb83BWtZ6iCCrk4= X-Google-Smtp-Source: AMsMyM4kjdgbZkAPm+2Tr7habAHupofVksCQQnVMw5EuBqPYuv7A8OVaVVyYD0MHNc9b6pe335RwNQ== X-Received: by 2002:a05:6870:5b89:b0:132:1241:fb5f with SMTP id em9-20020a0568705b8900b001321241fb5fmr12926761oab.74.1666382530952; Fri, 21 Oct 2022 13:02:10 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:10 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 08/18] RDMA/rxe: Split rxe_drain_resp_pkts() Date: Fri, 21 Oct 2022 15:01:09 -0500 Message-Id: <20221021200118.2163-9-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Split rxe_drain_resp_pkts() into two subroutines which perform separate functions. Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_comp.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index d2a250123617..5d434cce2b69 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -524,17 +524,21 @@ static inline enum comp_state complete_wqe(struct rxe_qp *qp, return COMPST_GET_WQE; } -static void rxe_drain_resp_pkts(struct rxe_qp *qp, bool notify) +static void rxe_drain_resp_pkts(struct rxe_qp *qp) { struct sk_buff *skb; - struct rxe_send_wqe *wqe; - struct rxe_queue *q = qp->sq.queue; while ((skb = skb_dequeue(&qp->resp_pkts))) { rxe_put(qp); kfree_skb(skb); ib_device_put(qp->ibqp.device); } +} + +static void rxe_drain_send_queue(struct rxe_qp *qp, bool notify) +{ + struct rxe_send_wqe *wqe; + struct rxe_queue *q = qp->sq.queue; while ((wqe = queue_head(q, q->type))) { if (notify) { @@ -565,6 +569,7 @@ int rxe_completer(void *arg) struct sk_buff *skb = NULL; struct rxe_pkt_info *pkt = NULL; enum comp_state state; + bool notify; int ret; if (!rxe_get(qp)) @@ -572,8 +577,9 @@ int rxe_completer(void *arg) if (!qp->valid || qp->comp.state == QP_STATE_ERROR || qp->comp.state == QP_STATE_RESET) { - rxe_drain_resp_pkts(qp, qp->valid && - qp->comp.state == QP_STATE_ERROR); + notify = qp->valid && (qp->comp.state == QP_STATE_ERROR); + rxe_drain_resp_pkts(qp); + rxe_drain_send_queue(qp, notify); goto exit; } From patchwork Fri Oct 21 20:01:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015360 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D970C38A2D for ; Fri, 21 Oct 2022 20:02:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230104AbiJUUCz (ORCPT ); Fri, 21 Oct 2022 16:02:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48538 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230394AbiJUUCX (ORCPT ); Fri, 21 Oct 2022 16:02:23 -0400 Received: from mail-oa1-x32.google.com (mail-oa1-x32.google.com [IPv6:2001:4860:4864:20::32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E16DE2625DF for ; Fri, 21 Oct 2022 13:02:12 -0700 (PDT) Received: by mail-oa1-x32.google.com with SMTP id 586e51a60fabf-1322d768ba7so4912591fac.5 for ; Fri, 21 Oct 2022 13:02:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rlVoip2hDPGQa9+wV269vJ7C/vC9zRk/r8HX4gvpR+w=; b=ff4BbOAjcITZQ4ckiWv0ZCnNUiuUiHBKGy7ol70Iy5usZASm+PCcsG7Jp8xNzaFc53 rng6Jo8l1EqlPOyQz/U60NqU8RJhabDC0Cq25llxBz32tKKGZwtY7HQRuLef92h8fXih UWpKW9VWBZN5y4tPUbeCFoYG0NjQ+MV5u1FfJYrckqMpue5jqfo7DPgKjO/wptJTaNWs JRsm0CyH8bPDf8iNm2dJV5pmra+N7BMTreUcFXJPfjlUMVein1uwAK1IMAbDaDaHFUBq KAz/ijV3xFV49x75AlzEt2BVCzw50uyDB2UY4so5ZI+9XFkm7S1Kn+OOs6nXW6W2Hi8H CQCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rlVoip2hDPGQa9+wV269vJ7C/vC9zRk/r8HX4gvpR+w=; b=CR/k2JDRhf4bQt9jNVJ2Ivl5N5GYDeukKD1mDkSnrG9Iqnisr7SfLJaWbKf2tOIAsq xKjOECU/nr504/b5phMTbJkp6Y54uSf+ufpzKbAzglnamgqhib+8BliWRwbATm98bDqT H8PXSS7YJlJz2P40U8jeiUS9klgN/bzWavvOsYSZTtA1rghkzbrIP8QUCfg5XzjVs8AX TDrMsIee/Six9pkcur07dEJS0SeFYoMUztz6evxtQNHZgZU3/8ETNc0pZ3u4syCWdxsZ ssFjgAVHfI3MEWQuXG39kPXvn/Cw1GVp/D8bqd9j6vOxDh3Fw1aGx17b5K+e/O1ulnfL VN5A== X-Gm-Message-State: ACrzQf3YJjAeaKgEs7XbqRRc/Qr0VVP8/USYL6ZIBe+WjPrj+g2YUDzF qfw0bsjq5eKXWWrT2nfWr6E= X-Google-Smtp-Source: AMsMyM7hGYNDneCxj/mO3OQjj6HuZzyLIAuQu9PdiqZFWX5kSoAFOuY/IRR2DL1tZeXbRITei1bgtQ== X-Received: by 2002:a05:6870:b68c:b0:132:b864:2aa2 with SMTP id cy12-20020a056870b68c00b00132b8642aa2mr30117901oab.130.1666382532322; Fri, 21 Oct 2022 13:02:12 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:11 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 09/18] RDMA/rxe: Simplify reset state handling in rxe_resp.c Date: Fri, 21 Oct 2022 15:01:10 -0500 Message-Id: <20221021200118.2163-10-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Make rxe_responder() more like rxe_completer() and take qp reset handling out of the state machine. Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_resp.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index 2a2a50de51b2..dd11dea70bbf 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -40,7 +40,6 @@ enum resp_states { RESPST_ERR_LENGTH, RESPST_ERR_CQ_OVERFLOW, RESPST_ERROR, - RESPST_RESET, RESPST_DONE, RESPST_EXIT, }; @@ -75,7 +74,6 @@ static char *resp_state_name[] = { [RESPST_ERR_LENGTH] = "ERR_LENGTH", [RESPST_ERR_CQ_OVERFLOW] = "ERR_CQ_OVERFLOW", [RESPST_ERROR] = "ERROR", - [RESPST_RESET] = "RESET", [RESPST_DONE] = "DONE", [RESPST_EXIT] = "EXIT", }; @@ -1278,8 +1276,9 @@ int rxe_responder(void *arg) switch (qp->resp.state) { case QP_STATE_RESET: - state = RESPST_RESET; - break; + rxe_drain_req_pkts(qp, false); + qp->resp.wqe = NULL; + goto exit; default: state = RESPST_GET_REQ; @@ -1438,11 +1437,6 @@ int rxe_responder(void *arg) goto exit; - case RESPST_RESET: - rxe_drain_req_pkts(qp, false); - qp->resp.wqe = NULL; - goto exit; - case RESPST_ERROR: qp->resp.goto_error = 0; pr_debug("qp#%d moved to error state\n", qp_num(qp)); From patchwork Fri Oct 21 20:01:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015361 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71BB0ECDFA1 for ; Fri, 21 Oct 2022 20:02:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230229AbiJUUCz (ORCPT ); Fri, 21 Oct 2022 16:02:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47938 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229692AbiJUUCX (ORCPT ); Fri, 21 Oct 2022 16:02:23 -0400 Received: from mail-ot1-x334.google.com (mail-ot1-x334.google.com [IPv6:2607:f8b0:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C2762623DF for ; Fri, 21 Oct 2022 13:02:14 -0700 (PDT) Received: by mail-ot1-x334.google.com with SMTP id t4-20020a9d7f84000000b00661c3d864f9so2446205otp.10 for ; Fri, 21 Oct 2022 13:02:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OXG5CI1MZEwriHR4ZXnpg5KIBA98wlCNTK6at1ExAAU=; b=Tlbl/h1OPudUiVLcq1Q65GL/u9XtK54TzKm4T0lB+rTQyZuxbds0f2Lka2mGSJIPaQ rWdy6OQqF54eiaua8+uZZCLSgYlh81G/ENyypMpkLqpH9SHfqzM7/mFp6WP+JHcY0IRt 6bcwuzIyMyfLrW2QNeciVXwqRSU7cRPWaGdk2KygeVbh3mALDwHpb1kkFtq5g1nVsBMw 2HGTs44K3w/+j232RrVinGTuL+BSVtAcTKe0ybFgp4yMxw3pCNA5xD9b9AN079I409fH RPK/He3mkDGUkb5j2fP/2XCmKntbv2jkEBWDI0Jh17WIdwQV+K0DFvOa++W6SXItWXQB zHFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OXG5CI1MZEwriHR4ZXnpg5KIBA98wlCNTK6at1ExAAU=; b=Xlcqi/BGplf2IKOCsNKY0DAECxpD9VN1C4PPbbUH2qF08xi6yCAhepvXVJaitboNGX MHUCm8ZMaxXO0vwuErRzpUC4NolBM9g7kef7Jlt9X9ssEel6quhx2g86v/KowhrhIWa7 T3wCm3B5VvMPPYRVxH/1WFSFKOIZTwoPDiNuP44jIttczNhN16k0C4D/6D+pYl8BpTT+ i6+xQPH/wtlLx/u9i8RgwjXlMx+ZiOz+Ukuo+OwtTeq70DVUaM/qUkP3ASJmwXf10ajS cZYgTWZw93fj4sBPcfv0dPlfvZCCq4eEyzQn+udOg277GyoZ6clmUgY7M7IGPgi23sMf +kGg== X-Gm-Message-State: ACrzQf0s49N1Ym5YPVYIHlquAiXfmbxThyq0oELMLfTE0FFL2OVs6yV7 1GZo2+H1Oq0LrNs5NB/MvhA5djbydHI= X-Google-Smtp-Source: AMsMyM5hgLouUldBDuX1D4ovGpDfPLaBOQwv1DGTjzKnXpIbXH7Kmk2S8J3nat+YLRMvZdPt5ok2ng== X-Received: by 2002:a9d:4a1:0:b0:65c:4497:6cd9 with SMTP id 30-20020a9d04a1000000b0065c44976cd9mr10388050otm.188.1666382533647; Fri, 21 Oct 2022 13:02:13 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:13 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 10/18] RDMA/rxe: Handle qp error in rxe_resp.c Date: Fri, 21 Oct 2022 15:01:11 -0500 Message-Id: <20221021200118.2163-11-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Split rxe_drain_req_pkts() into two subroutines which perform separate tasks. Change qp error and reset states and !qp->valid in the same way as rxe_comp.c. Flush recv wqes for qp in error. Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_resp.c | 64 ++++++++++++++++++++-------- 1 file changed, 47 insertions(+), 17 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index dd11dea70bbf..0bcdd1154641 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -1025,7 +1025,6 @@ static enum resp_states do_complete(struct rxe_qp *qp, return RESPST_CLEANUP; } - static int send_common_ack(struct rxe_qp *qp, u8 syndrome, u32 psn, int opcode, const char *msg) { @@ -1240,22 +1239,56 @@ static enum resp_states do_class_d1e_error(struct rxe_qp *qp) } } -static void rxe_drain_req_pkts(struct rxe_qp *qp, bool notify) +static void rxe_drain_req_pkts(struct rxe_qp *qp) { struct sk_buff *skb; - struct rxe_queue *q = qp->rq.queue; while ((skb = skb_dequeue(&qp->req_pkts))) { rxe_put(qp); kfree_skb(skb); ib_device_put(qp->ibqp.device); } +} + +int complete_flush(struct rxe_qp *qp, struct rxe_recv_wqe *wqe) +{ + struct rxe_cqe cqe; + struct ib_wc *wc = &cqe.ibwc; + struct ib_uverbs_wc *uwc = &cqe.uibwc; + + memset(&cqe, 0, sizeof(cqe)); - if (notify) - return; + if (qp->rcq->is_user) { + uwc->status = IB_WC_WR_FLUSH_ERR; + uwc->qp_num = qp->ibqp.qp_num; + uwc->wr_id = wqe->wr_id; + } else { + wc->status = IB_WC_WR_FLUSH_ERR; + wc->qp = &qp->ibqp; + wc->wr_id = wqe->wr_id; + } - while (!qp->srq && q && queue_head(q, q->type)) + if (rxe_cq_post(qp->rcq, &cqe, 0)) + return -ENOMEM; + + return 0; +} + +/* drain the receive queue. Complete each wqe with a flush error + * if notify is true or until a cq overflow occurs. + */ +static void rxe_drain_recv_queue(struct rxe_qp *qp, bool notify) +{ + struct rxe_recv_wqe *wqe; + struct rxe_queue *q = qp->rq.queue; + + while ((wqe = queue_head(q, q->type))) { + if (notify && complete_flush(qp, wqe)) + notify = 0; queue_advance_consumer(q, q->type); + } + + qp->resp.wqe = NULL; } int rxe_responder(void *arg) @@ -1264,6 +1297,7 @@ int rxe_responder(void *arg) struct rxe_dev *rxe = to_rdev(qp->ibqp.device); enum resp_states state; struct rxe_pkt_info *pkt = NULL; + bool notify; int ret; if (!rxe_get(qp)) @@ -1271,20 +1305,16 @@ int rxe_responder(void *arg) qp->resp.aeth_syndrome = AETH_ACK_UNLIMITED; - if (!qp->valid) - goto exit; - - switch (qp->resp.state) { - case QP_STATE_RESET: - rxe_drain_req_pkts(qp, false); - qp->resp.wqe = NULL; + if (!qp->valid || qp->resp.state == QP_STATE_ERROR || + qp->resp.state == QP_STATE_RESET) { + notify = qp->valid && (qp->resp.state == QP_STATE_ERROR); + rxe_drain_req_pkts(qp); + rxe_drain_recv_queue(qp, notify); goto exit; - - default: - state = RESPST_GET_REQ; - break; } + state = RESPST_GET_REQ; + while (1) { pr_debug("qp#%d state = %s\n", qp_num(qp), resp_state_name[state]); From patchwork Fri Oct 21 20:01:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015362 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCBC6C3A59D for ; Fri, 21 Oct 2022 20:02:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229692AbiJUUC4 (ORCPT ); Fri, 21 Oct 2022 16:02:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47948 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230398AbiJUUCX (ORCPT ); Fri, 21 Oct 2022 16:02:23 -0400 Received: from mail-oa1-x2a.google.com (mail-oa1-x2a.google.com [IPv6:2001:4860:4864:20::2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8344C25ED03 for ; Fri, 21 Oct 2022 13:02:15 -0700 (PDT) Received: by mail-oa1-x2a.google.com with SMTP id 586e51a60fabf-136b5dd6655so4916376fac.3 for ; Fri, 21 Oct 2022 13:02:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jywcoq6WfpSz2CinBO3sC4x3ilzoShyeAxGkJ7NOWiY=; b=X9H9TsVo6ZqZveeuCZUJ7oLE5bS0cVsma/da9EzY10+P95nYFwn3rZj8HDkBaavYcA oKxGeMwk+aR6yqAi91VEVarnDywDUCDMMmvJwMNcJxXaaUR55CRWMz6Zs3SRlsPtroGa jTuLeAk+Ofic2N4NJsoecSnrWoVcgq1JdsSMSksJlylrKe1GgMhA2cTw6+ZVs3Dt8cTi qYqDsEsLC7dEaMOcV/LsTmVu48hEOneNavD1B45qXXxR8/2fuz0HrC/2Ixe8DVYRDaHm Q+0zRGdOiHKdQP/38p/HNZQcGQrVukNGzHHNw8ZMOhMQcoyqXGphhGtcec8NFjJINmda ISDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jywcoq6WfpSz2CinBO3sC4x3ilzoShyeAxGkJ7NOWiY=; b=NM81aAu8oGF/plunadQTaz+Y16BvvBYhIiSANAl1JI5FETUNUe8pBv6JSjS+XOKlv4 7upRzC+oDXJtyP/cPgM1610+xwoqr7pNNiJU5ZtkmpU5rJnCgnZWDt4eCDA5qp/RJxdB w7tdlCpXvdCq0lVrahw4ahf31Hoz2RPQ1KsVRWaFrC0iU3WgFvo+prlcrz1n7d1KJsfN 2XQZEf3rg8E4qNttq9dA0r3FsLvRMpAsunt+miNG1IfJf25YCDapriIlj7XIdAQM8TwD VU/rQByX9f1Nm8Wd6nBR/7AcfDzSS24esdxX+5omBGH8tt2Nyaoyh7A8cmOisXVMVz4A t06g== X-Gm-Message-State: ACrzQf0FlNzVMUssrITi+nlXhNAXi1W4LSAThpb7upAz0d7vflEwhfFu r2jYcOnc6q7t0R+H3XgeabI= X-Google-Smtp-Source: AMsMyM5nIYDYBrQ/TBT9lYmXB5cPRMMZuDkf/PpCMOScB59FMGRbJ+KVq33kmdZ1WOJqRKyfhp70Qg== X-Received: by 2002:a05:6870:586:b0:132:f48a:b487 with SMTP id m6-20020a056870058600b00132f48ab487mr13563107oap.51.1666382534921; Fri, 21 Oct 2022 13:02:14 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:14 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 11/18] RDMA/rxe: Cleanup comp tasks in rxe_qp.c Date: Fri, 21 Oct 2022 15:01:12 -0500 Message-Id: <20221021200118.2163-12-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Take advantage of inline task behavior to cleanup code in rxe_qp.c for completer tasks. Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_qp.c | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index b5e108794aa1..3691eb97c576 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -480,8 +480,7 @@ static void rxe_qp_reset(struct rxe_qp *qp) /* stop request/comp */ if (qp->sq.queue) { - if (qp_type(qp) == IB_QPT_RC) - rxe_disable_task(&qp->comp.task); + rxe_disable_task(&qp->comp.task); rxe_disable_task(&qp->req.task); } @@ -524,9 +523,7 @@ static void rxe_qp_reset(struct rxe_qp *qp) rxe_enable_task(&qp->resp.task); if (qp->sq.queue) { - if (qp_type(qp) == IB_QPT_RC) - rxe_enable_task(&qp->comp.task); - + rxe_enable_task(&qp->comp.task); rxe_enable_task(&qp->req.task); } } @@ -537,10 +534,7 @@ static void rxe_qp_drain(struct rxe_qp *qp) if (qp->sq.queue) { if (qp->req.state != QP_STATE_DRAINED) { qp->req.state = QP_STATE_DRAIN; - if (qp_type(qp) == IB_QPT_RC) - rxe_sched_task(&qp->comp.task); - else - __rxe_do_task(&qp->comp.task); + rxe_sched_task(&qp->comp.task); rxe_sched_task(&qp->req.task); } } @@ -556,11 +550,7 @@ void rxe_qp_error(struct rxe_qp *qp) /* drain work and packet queues */ rxe_sched_task(&qp->resp.task); - - if (qp_type(qp) == IB_QPT_RC) - rxe_sched_task(&qp->comp.task); - else - __rxe_do_task(&qp->comp.task); + rxe_sched_task(&qp->comp.task); rxe_sched_task(&qp->req.task); } From patchwork Fri Oct 21 20:01:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015363 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A77D8C433FE for ; Fri, 21 Oct 2022 20:02:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229872AbiJUUC5 (ORCPT ); Fri, 21 Oct 2022 16:02:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230403AbiJUUCX (ORCPT ); Fri, 21 Oct 2022 16:02:23 -0400 Received: from mail-oa1-x2c.google.com (mail-oa1-x2c.google.com [IPv6:2001:4860:4864:20::2c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 245322623E3 for ; Fri, 21 Oct 2022 13:02:17 -0700 (PDT) Received: by mail-oa1-x2c.google.com with SMTP id 586e51a60fabf-13af2d12469so4857592fac.13 for ; Fri, 21 Oct 2022 13:02:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NLhGIriOec+yW5+uAGQBdRbRCWaKcwXXxn/8Qw9nAnA=; b=Gdb9wfXNftCxxsDuinU58kvGLKUuOrkGzWN6ri5MnRjbkCy+o3Bu8HHogMkjSIQlWw aJSZKGRHgknTzT+CaIm0IKYbJppSFKxiN30osk70qYXGAWD9SmH0fRhhl56xgYEs494P LxdQEJZAeAzdlCYoLhRtfE9Dv3Hkv9DnVqCOLJHUlU2MSvBm0oNO+Hg2DRKynHmsu1DM rt3QL+wzwV2ELBxhbIMhbAzZWklPskcmKTas7LA7BMdCN3c6KqJeN77J/mP8UsHbO+NX QmW4mq7f7XjE+sdIuCTwPNkW9VmIfFCJkSQTdrpLy8E3K5e0d7Hiq3NdC6yjIpbc3zeU mGkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NLhGIriOec+yW5+uAGQBdRbRCWaKcwXXxn/8Qw9nAnA=; b=tXuoKUiySq6zSZY0RzPfgk6N9xLlMDZ4cly79i4dW8Ek4Cv4BkL2IYxHlLrj9k06NM 2l3ifgzhevmAHOSwXJYKYo/YnXs0Ku71zYKjQW1hT6YzdWvsHRwdsnUznEkKzI7LQCUi e3jQWgF2AvJoLET8HkkKA4TDkhkpikqnPnJUaBq2iXhfYBPWxKX7h7F3ENky/03fcDtH 7jKXbstqcz54NpobDOUlCtCMDNdFaeyTW7xj1NO62utlGoQasfC4E4TpSaSziPHo/9tx mbEFSrT2IsimhWxdKuzOYvhErerO/DqZKcO6tebuDDvvUNj0dMBcCb1zgGO5c6X/PHXC CK9g== X-Gm-Message-State: ACrzQf3991gNSmqdTl081jW7+H2WrfgwVGeJGqDGYd7jaAhgvfMtfl9L +QlHvrl0FmAFuIqy1ZYYLwQ= X-Google-Smtp-Source: AMsMyM5RovetoJULFu4i2K4/yv4ZAJ99ZF8ppcp+3QSzd6T9sTRmziHnmDJ2G7C+sFrg/gPQrURH8g== X-Received: by 2002:a05:6870:d6a3:b0:131:8546:5f5d with SMTP id z35-20020a056870d6a300b0013185465f5dmr14001894oap.216.1666382536321; Fri, 21 Oct 2022 13:02:16 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:15 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 12/18] RDMA/rxe: Remove __rxe_do_task() Date: Fri, 21 Oct 2022 15:01:13 -0500 Message-Id: <20221021200118.2163-13-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The subroutine __rxe_do_task is not thread safe. It is only used in the rxe_qp_reset() and rxe_qp_do_cleanup() routines. After changes in error handling in the tasklet functions the queues can be drained by calling them once outside of the tasklet code. This allows __rxe_do_task() to be removed. Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_qp.c | 60 ++++++++++++++-------------- drivers/infiniband/sw/rxe/rxe_task.c | 13 ------ drivers/infiniband/sw/rxe/rxe_task.h | 6 --- 3 files changed, 29 insertions(+), 50 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index 3691eb97c576..50f6b8b8ad9d 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -477,28 +477,23 @@ static void rxe_qp_reset(struct rxe_qp *qp) { /* stop tasks from running */ rxe_disable_task(&qp->resp.task); - - /* stop request/comp */ - if (qp->sq.queue) { - rxe_disable_task(&qp->comp.task); - rxe_disable_task(&qp->req.task); - } + rxe_disable_task(&qp->comp.task); + rxe_disable_task(&qp->req.task); /* move qp to the reset state */ qp->req.state = QP_STATE_RESET; qp->comp.state = QP_STATE_RESET; qp->resp.state = QP_STATE_RESET; - /* let state machines reset themselves drain work and packet queues - * etc. - */ - __rxe_do_task(&qp->resp.task); + /* drain work and packet queues */ + rxe_responder(qp); + rxe_completer(qp); + rxe_requester(qp); - if (qp->sq.queue) { - __rxe_do_task(&qp->comp.task); - __rxe_do_task(&qp->req.task); + if (qp->rq.queue) + rxe_queue_reset(qp->rq.queue); + if (qp->sq.queue) rxe_queue_reset(qp->sq.queue); - } /* cleanup attributes */ atomic_set(&qp->ssn, 0); @@ -521,11 +516,8 @@ static void rxe_qp_reset(struct rxe_qp *qp) /* reenable tasks */ rxe_enable_task(&qp->resp.task); - - if (qp->sq.queue) { - rxe_enable_task(&qp->comp.task); - rxe_enable_task(&qp->req.task); - } + rxe_enable_task(&qp->comp.task); + rxe_enable_task(&qp->req.task); } /* drain the send queue */ @@ -543,15 +535,25 @@ static void rxe_qp_drain(struct rxe_qp *qp) /* move the qp to the error state */ void rxe_qp_error(struct rxe_qp *qp) { + /* stop tasks from running */ + rxe_disable_task(&qp->resp.task); + rxe_disable_task(&qp->comp.task); + rxe_disable_task(&qp->req.task); + qp->req.state = QP_STATE_ERROR; qp->resp.state = QP_STATE_ERROR; qp->comp.state = QP_STATE_ERROR; qp->attr.qp_state = IB_QPS_ERR; /* drain work and packet queues */ - rxe_sched_task(&qp->resp.task); - rxe_sched_task(&qp->comp.task); - rxe_sched_task(&qp->req.task); + rxe_responder(qp); + rxe_completer(qp); + rxe_requester(qp); + + /* reenable tasks */ + rxe_enable_task(&qp->resp.task); + rxe_enable_task(&qp->comp.task); + rxe_enable_task(&qp->req.task); } /* called by the modify qp verb */ @@ -770,24 +772,20 @@ static void rxe_qp_do_cleanup(struct work_struct *work) qp->valid = 0; qp->qp_timeout_jiffies = 0; - rxe_cleanup_task(&qp->resp.task); if (qp_type(qp) == IB_QPT_RC) { del_timer_sync(&qp->retrans_timer); del_timer_sync(&qp->rnr_nak_timer); } + rxe_cleanup_task(&qp->resp.task); rxe_cleanup_task(&qp->req.task); rxe_cleanup_task(&qp->comp.task); - /* flush out any receive wr's or pending requests */ - if (qp->req.task.func) - __rxe_do_task(&qp->req.task); - - if (qp->sq.queue) { - __rxe_do_task(&qp->comp.task); - __rxe_do_task(&qp->req.task); - } + /* drain any receive wr's or pending requests */ + rxe_responder(qp); + rxe_completer(qp); + rxe_requester(qp); if (qp->sq.queue) rxe_queue_cleanup(qp->sq.queue); diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c index 8dfbfa164eff..120693c9a795 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.c +++ b/drivers/infiniband/sw/rxe/rxe_task.c @@ -6,19 +6,6 @@ #include "rxe.h" -int __rxe_do_task(struct rxe_task *task) - -{ - int ret; - - while ((ret = task->func(task->arg)) == 0) - ; - - task->ret = ret; - - return ret; -} - /* * this locking is due to a potential race where * a second caller finds the task already running diff --git a/drivers/infiniband/sw/rxe/rxe_task.h b/drivers/infiniband/sw/rxe/rxe_task.h index 31963129ff7a..d594468fcf56 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.h +++ b/drivers/infiniband/sw/rxe/rxe_task.h @@ -48,12 +48,6 @@ struct rxe_task { int rxe_init_task(struct rxe_task *task, void *arg, int (*func)(void *), enum rxe_task_type type); -/* - * raw call to func in loop without any checking - * can call when tasklets are disabled - */ -int __rxe_do_task(struct rxe_task *task); - void rxe_run_task(struct rxe_task *task); void rxe_sched_task(struct rxe_task *task); From patchwork Fri Oct 21 20:01:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015364 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB8DDC38A2D for ; Fri, 21 Oct 2022 20:02:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230333AbiJUUC6 (ORCPT ); Fri, 21 Oct 2022 16:02:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47664 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230260AbiJUUCX (ORCPT ); Fri, 21 Oct 2022 16:02:23 -0400 Received: from mail-oa1-x2f.google.com (mail-oa1-x2f.google.com [IPv6:2001:4860:4864:20::2f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7CA525E8A8 for ; Fri, 21 Oct 2022 13:02:18 -0700 (PDT) Received: by mail-oa1-x2f.google.com with SMTP id 586e51a60fabf-1322fa1cf6fso4897668fac.6 for ; Fri, 21 Oct 2022 13:02:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=moCS8bMrE9y/ygj3tPnfai4xVg2bxmPG46HvIHu3Rcc=; b=D3dz7SN8vxDv3ri4+4tFDTIDnVCa5mOOzEgccIEvIVABT9vSlL2Fr8ft+EvrqjnlzC Nhx156SKpVZZzWGg+6kwQitq+LgiaetsAL97/SSo1PLqmuRjrIk2s8kxG6bGeOho8XOM D3NsocGR3d4+myNDOa3bCd9uanJ6Daf6l9VWR1E+NzceRYC2DHrX1cGYXx/bGItwKI8f t5CVwurDIHX2wT28+sbiOorCJ1aiiYce7+eBN2vYhd5MJ3fT8EUfikAlbmVW5d+D5cSj eejA5NOrbv8kwe5i0eYF4xwVt+66knwKdN4+c99B+F8tLUasB5YOLCadax15fjancEF8 gS3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=moCS8bMrE9y/ygj3tPnfai4xVg2bxmPG46HvIHu3Rcc=; b=QUAomo6t6vrHQvRGP5JffTjaKNZ2atf/m001e+qMt6EABN97rIwIL7McvZWCnynnHD pJjkyg+fjwGEkaE+wFzUMya51cs2A402UyT0QltJ1dS6nartv1DXlevlM3ZfrFUvjuKZ BNC8h+5fJNGUWf1pX1E9ykBMb/kr0Df5XbjU/84NTncFcSXijuXNTuOCyQz38q0hbP0R +mP3iW5r23xJitZe9DoxOeNp/rxWHpDNd+3dSuwjTtoQ8I2xwWpZD59rszNCtTKtGkIx Qlq/f7FfdXuxjd4f1KYt/LHxSY0mtSmWQ9gqBUU92eGbPCvQ9PLcjDFHMtzMDouy6DBI SKhA== X-Gm-Message-State: ACrzQf3bqeEHOHo2BSqnWovj0xg+WOEBMBU5KnGZLNtqnh0gk0QpJGPc O1QaTcEGNVEC9SrNhJHwm+o= X-Google-Smtp-Source: AMsMyM7w7Iv5uCcUmiJXwS9kifFFrU7+WFm4kVRGLAHQa/nWBQzhT/+wKf+/eFEJZ1WxJi9Do1onuw== X-Received: by 2002:a05:6870:b027:b0:136:662d:7cf5 with SMTP id y39-20020a056870b02700b00136662d7cf5mr31199175oae.110.1666382537518; Fri, 21 Oct 2022 13:02:17 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:17 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 13/18] RDMA/rxe: Make tasks schedule each other Date: Fri, 21 Oct 2022 15:01:14 -0500 Message-Id: <20221021200118.2163-14-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Replace rxe_run_task() by rxe_sched_task() when tasks call each other. These are not performance critical and mainly involve error paths but they run the risk of causing deadlocks. Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_comp.c | 8 ++++---- drivers/infiniband/sw/rxe/rxe_req.c | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index 5d434cce2b69..6c15c9307660 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -308,7 +308,7 @@ static inline enum comp_state check_ack(struct rxe_qp *qp, qp->comp.psn = pkt->psn; if (qp->req.wait_psn) { qp->req.wait_psn = 0; - rxe_run_task(&qp->req.task); + rxe_sched_task(&qp->req.task); } } return COMPST_ERROR_RETRY; @@ -455,7 +455,7 @@ static void do_complete(struct rxe_qp *qp, struct rxe_send_wqe *wqe) */ if (qp->req.wait_fence) { qp->req.wait_fence = 0; - rxe_run_task(&qp->req.task); + rxe_sched_task(&qp->req.task); } } @@ -469,7 +469,7 @@ static inline enum comp_state complete_ack(struct rxe_qp *qp, if (qp->req.need_rd_atomic) { qp->comp.timeout_retry = 0; qp->req.need_rd_atomic = 0; - rxe_run_task(&qp->req.task); + rxe_sched_task(&qp->req.task); } } @@ -723,7 +723,7 @@ int rxe_completer(void *arg) RXE_CNT_COMP_RETRY); qp->req.need_retry = 1; qp->comp.started_retry = 1; - rxe_run_task(&qp->req.task); + rxe_sched_task(&qp->req.task); } goto done; diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 41f1d84f0acb..fba7572e1d0c 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -733,7 +733,7 @@ int rxe_requester(void *arg) qp->req.wqe_index); wqe->state = wqe_state_done; wqe->status = IB_WC_SUCCESS; - rxe_run_task(&qp->comp.task); + rxe_sched_task(&qp->comp.task); goto done; } payload = mtu; @@ -817,7 +817,7 @@ int rxe_requester(void *arg) qp->req.wqe_index = queue_next_index(qp->sq.queue, qp->req.wqe_index); wqe->state = wqe_state_error; qp->req.state = QP_STATE_ERROR; - rxe_run_task(&qp->comp.task); + rxe_sched_task(&qp->comp.task); exit: ret = -EAGAIN; out: From patchwork Fri Oct 21 20:01:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015369 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC927C433FE for ; Fri, 21 Oct 2022 20:03:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230283AbiJUUDC (ORCPT ); Fri, 21 Oct 2022 16:03:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47648 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230285AbiJUUCk (ORCPT ); Fri, 21 Oct 2022 16:02:40 -0400 Received: from mail-oa1-x2a.google.com (mail-oa1-x2a.google.com [IPv6:2001:4860:4864:20::2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E378325ED10 for ; Fri, 21 Oct 2022 13:02:39 -0700 (PDT) Received: by mail-oa1-x2a.google.com with SMTP id 586e51a60fabf-13af2d12469so4859026fac.13 for ; Fri, 21 Oct 2022 13:02:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p3kPzmPDjjNhDNGku1w9RH09kAppbd1qvrZUMK/RXkU=; b=jKgSmILPz4yCvkYYjr7Ai3tnR5DNcvUIQ+C/+WvekqSI6vSR5o/D6AIlZQ6sBS90dd j512nCKxEqpMaFRgdT6POdw6dFUEz6AWGuuT2sMdEi1Askuit3bdJrR1s/zwCU6irFNt hWXSuvVxoqdOyGqM/amu4AtvpZdrep2/5hoN/J9ij36i0PcqGcRLGUsPyiX8vl1E07Gs pSZ3ZWYLVxN3whXgp0WaWlAlEfeckpMvPag4UU45CKbbyMuyG4ALQ8uAbMfreVJYGm6N FOIn+IJt7LLpcMiBDb0zD4fhtu0lme971uQL5+w5dOLpTtB3d8dWnwF53d9Nf5bG5OFL lxAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p3kPzmPDjjNhDNGku1w9RH09kAppbd1qvrZUMK/RXkU=; b=dX7SsSOaCbdSYZqXlfWGe1jN3FyZGJph/XSJh7ckH4mfJG9E19vXYjZncSUJ5upfNj ykBdurr2wg1aDLMVqeTNkG8I8znNlebQv/XiWUhLGryyCNz/MaBKjBzuiD80x5TDiFwn CqZXJcUAfqPgouheaTRfnzdHOBDqOHkFjflnycOL9cNGwVsHIjDHp9cQGp2T7olOTxpC HTdDmCuVHe/FEV164cAss3Fmi0hi2F2O66pAn9/Y4xKa4Zd/Dkn9vJqW1wRYOaDjb+8X BMXemLhyqXZzsshR12jm0laK4avl8f3F4nn07PZ2wPNWsmYFpZq2hS/Dp58WEJh2Ys2X F2tg== X-Gm-Message-State: ACrzQf0Q8QbemuYkgrWtcOllE5N/g3yiM2TWY7MAG290VVNTl4682jQ3 y+4ACaOwxLoU7PhlDejLCJ3G4guS/HY= X-Google-Smtp-Source: AMsMyM5VaQ3iBsF7dm3mVuoqS2WC/h4qZlRWOEu+3JpXqZlYTcT0RJ31pfSoCCVHEQILJ1qVcjYhSg== X-Received: by 2002:a05:6870:538d:b0:136:3cc4:78fa with SMTP id h13-20020a056870538d00b001363cc478famr30749320oan.278.1666382538746; Fri, 21 Oct 2022 13:02:18 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:18 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 14/18] RDMA/rxe: Implement disable/enable_task() Date: Fri, 21 Oct 2022 15:01:15 -0500 Message-Id: <20221021200118.2163-15-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Implement common disable_task() and enable_task() routines by adding a new PAUSED state to the do_task() state machine. These replace tasklet_disable and tasklet_enable with code that can be shared with all the task types. Move rxe_sched_task to re-schedule the task outside of the locks to avoid a deadlock. Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_task.c | 107 ++++++++++++++++----------- drivers/infiniband/sw/rxe/rxe_task.h | 1 + 2 files changed, 66 insertions(+), 42 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c index 120693c9a795..d824de82f2ae 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.c +++ b/drivers/infiniband/sw/rxe/rxe_task.c @@ -6,36 +6,46 @@ #include "rxe.h" -/* - * this locking is due to a potential race where - * a second caller finds the task already running - * but looks just after the last call to func - */ -static void do_task(struct rxe_task *task) +static bool task_is_idle(struct rxe_task *task) { - unsigned int iterations = RXE_MAX_ITERATIONS; - int cont; - int ret; + if (task->destroyed) + return false; spin_lock_bh(&task->lock); switch (task->state) { case TASK_STATE_START: task->state = TASK_STATE_BUSY; spin_unlock_bh(&task->lock); - break; - + return true; case TASK_STATE_BUSY: task->state = TASK_STATE_ARMED; fallthrough; case TASK_STATE_ARMED: - spin_unlock_bh(&task->lock); - return; - + case TASK_STATE_PAUSED: + break; default: + WARN_ON(1); + break; + } + spin_unlock_bh(&task->lock); + + return false; +} + +static void do_task(struct rxe_task *task) +{ + unsigned int iterations = RXE_MAX_ITERATIONS; + bool resched = false; + int cont; + int ret; + + /* flush out pending tasks */ + spin_lock_bh(&task->lock); + if (task->state == TASK_STATE_PAUSED) { spin_unlock_bh(&task->lock); - pr_warn("%s failed with bad state %d\n", __func__, task->state); return; } + spin_unlock_bh(&task->lock); do { cont = 0; @@ -43,47 +53,52 @@ static void do_task(struct rxe_task *task) spin_lock_bh(&task->lock); switch (task->state) { + case TASK_STATE_START: case TASK_STATE_BUSY: if (ret) { task->state = TASK_STATE_START; - } else if (iterations--) { + } else if (task->type == RXE_TASK_TYPE_INLINE || + iterations--) { cont = 1; } else { - /* reschedule the tasklet and exit - * the loop to give up the cpu - */ - tasklet_schedule(&task->tasklet); task->state = TASK_STATE_START; + resched = true; } break; - - /* someone tried to run the task since the last time we called - * func, so we will call one more time regardless of the - * return value - */ case TASK_STATE_ARMED: task->state = TASK_STATE_BUSY; cont = 1; break; - + case TASK_STATE_PAUSED: + break; default: - pr_warn("%s failed with bad state %d\n", __func__, - task->state); + WARN_ON(1); + break; } spin_unlock_bh(&task->lock); } while (cont); + if (resched) + rxe_sched_task(task); + task->ret = ret; } static void disable_task(struct rxe_task *task) { - /* todo */ + spin_lock_bh(&task->lock); + task->state = TASK_STATE_PAUSED; + spin_unlock_bh(&task->lock); } static void enable_task(struct rxe_task *task) { - /* todo */ + spin_lock_bh(&task->lock); + task->state = TASK_STATE_START; + spin_unlock_bh(&task->lock); + + /* restart task in case */ + rxe_run_task(task); } /* busy wait until any previous tasks are done */ @@ -95,7 +110,8 @@ static void cleanup_task(struct rxe_task *task) do { spin_lock_bh(&task->lock); - idle = (task->state == TASK_STATE_START); + idle = (task->state == TASK_STATE_START || + task->state == TASK_STATE_PAUSED); spin_unlock_bh(&task->lock); } while (!idle); } @@ -103,22 +119,26 @@ static void cleanup_task(struct rxe_task *task) /* silently treat schedule as inline for inline tasks */ static void inline_sched(struct rxe_task *task) { - do_task(task); + if (task_is_idle(task)) + do_task(task); } static void inline_run(struct rxe_task *task) { - do_task(task); + if (task_is_idle(task)) + do_task(task); } static void inline_disable(struct rxe_task *task) { - disable_task(task); + if (!task->destroyed) + disable_task(task); } static void inline_enable(struct rxe_task *task) { - enable_task(task); + if (!task->destroyed) + enable_task(task); } static void inline_cleanup(struct rxe_task *task) @@ -142,31 +162,34 @@ static void inline_init(struct rxe_task *task) /* use tsklet_xxx to avoid name collisions with tasklet_xxx */ static void tsklet_sched(struct rxe_task *task) { - tasklet_schedule(&task->tasklet); + if (task_is_idle(task)) + tasklet_schedule(&task->tasklet); } static void tsklet_do_task(struct tasklet_struct *tasklet) { struct rxe_task *task = container_of(tasklet, typeof(*task), tasklet); - do_task(task); + if (!task->destroyed) + do_task(task); } static void tsklet_run(struct rxe_task *task) { - do_task(task); + if (task_is_idle(task)) + do_task(task); } static void tsklet_disable(struct rxe_task *task) { - disable_task(task); - tasklet_disable(&task->tasklet); + if (!task->destroyed) + disable_task(task); } static void tsklet_enable(struct rxe_task *task) { - tasklet_enable(&task->tasklet); - enable_task(task); + if (!task->destroyed) + enable_task(task); } static void tsklet_cleanup(struct rxe_task *task) diff --git a/drivers/infiniband/sw/rxe/rxe_task.h b/drivers/infiniband/sw/rxe/rxe_task.h index d594468fcf56..792832786456 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.h +++ b/drivers/infiniband/sw/rxe/rxe_task.h @@ -26,6 +26,7 @@ enum { TASK_STATE_START = 0, TASK_STATE_BUSY = 1, TASK_STATE_ARMED = 2, + TASK_STATE_PAUSED = 3, }; /* From patchwork Fri Oct 21 20:01:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015366 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DC80C433FE for ; Fri, 21 Oct 2022 20:03:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230172AbiJUUDA (ORCPT ); Fri, 21 Oct 2022 16:03:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230070AbiJUUCZ (ORCPT ); Fri, 21 Oct 2022 16:02:25 -0400 Received: from mail-ot1-x32b.google.com (mail-ot1-x32b.google.com [IPv6:2607:f8b0:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A31D29CBBD for ; Fri, 21 Oct 2022 13:02:21 -0700 (PDT) Received: by mail-ot1-x32b.google.com with SMTP id r13-20020a056830418d00b0065601df69c0so2454426otu.7 for ; Fri, 21 Oct 2022 13:02:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IgZtbe5QbJIdt0GKspEyQP1OPD6Un66ARePr7nadvDw=; b=Q0PNn0kI4ZVFivFv+eVhleAKa1HRWzZuX3dhIWS4xAYylmsDaJfMUidi4B/SwHucus Y7EjvVX+GDDS7EEWhkMmygxWj4bP6u03wurzbWS+OKXJ9t3pogAjZL6VX6OrprWWVTj2 QoKKvgisj448IvmxGpmftK5nNHwqQWAbPmSYfsv+Zl/s4N8n3pGmFlfqslYJ+S8sGlUs Uc/nrEO2DlwGcmnPKp/i7WwRRyK1uCn3Yi5XQfUU+z7wl2UymbLH46Snb2IaHtglcA2F nQ9v55A4hBRCYRV8sA+Y88oGunpc0x5TEIqVnwYxDpjXzaXmHQbegQ5JqhAzc4retQ3J bTeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IgZtbe5QbJIdt0GKspEyQP1OPD6Un66ARePr7nadvDw=; b=dK5kvyIJlfmv833zZinFOfXyDroFC4FjcURfkg0P12JC19zq1fsHU35G5FREoeJIPs hPl/C2snkwBo4yqoU3IMh72ZHYGkb514w1x69q7+4SDslftfFSvAcSVSHcEVycsG/3Et rAplN5JHfMxZbzLrspGbK2WLVbAIB05Enn3Hr/nGZcCVsdGb/2rq5slKXR9a+lzd8kcR ycuOQoxkFPtSwkAv6z/JV7Qabi75PeRtgNqgWhPTE9XcJ8MZTNrUWdhV72kHWcyqp4c6 7U71wZ6Xpfcl43p5IM75u+Bn0P8yR2zxs5PbDiACaI9isGLmLq+13YsWJF/nAIt534YL BNeA== X-Gm-Message-State: ACrzQf3uF6y86I6r6aN49KgadP9dSlqWfyIzSZsK0j17DSQ1iaASbcze oNxBUITugFYSFp6nArXfmhk= X-Google-Smtp-Source: AMsMyM4Cu1Tn6V0znBbM+s0LAcIBGHYqwm5opbrRAZC/TrGK36Y0y03hRSJzGwwymLjD2qJBTDeB/A== X-Received: by 2002:a05:6830:148a:b0:661:94cb:32b1 with SMTP id s10-20020a056830148a00b0066194cb32b1mr10909647otq.174.1666382539947; Fri, 21 Oct 2022 13:02:19 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:19 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 15/18] RDMA/rxe: Replace TASK_STATE_START by TASK_STATE_IDLE Date: Fri, 21 Oct 2022 15:01:16 -0500 Message-Id: <20221021200118.2163-16-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Replace the enum TASK_STATE_START by TASK_STATE_IDLE. Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_task.c | 14 +++++++------- drivers/infiniband/sw/rxe/rxe_task.h | 2 +- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c index d824de82f2ae..0fd0d97e8272 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.c +++ b/drivers/infiniband/sw/rxe/rxe_task.c @@ -13,7 +13,7 @@ static bool task_is_idle(struct rxe_task *task) spin_lock_bh(&task->lock); switch (task->state) { - case TASK_STATE_START: + case TASK_STATE_IDLE: task->state = TASK_STATE_BUSY; spin_unlock_bh(&task->lock); return true; @@ -53,15 +53,15 @@ static void do_task(struct rxe_task *task) spin_lock_bh(&task->lock); switch (task->state) { - case TASK_STATE_START: + case TASK_STATE_IDLE: case TASK_STATE_BUSY: if (ret) { - task->state = TASK_STATE_START; + task->state = TASK_STATE_IDLE; } else if (task->type == RXE_TASK_TYPE_INLINE || iterations--) { cont = 1; } else { - task->state = TASK_STATE_START; + task->state = TASK_STATE_IDLE; resched = true; } break; @@ -94,7 +94,7 @@ static void disable_task(struct rxe_task *task) static void enable_task(struct rxe_task *task) { spin_lock_bh(&task->lock); - task->state = TASK_STATE_START; + task->state = TASK_STATE_IDLE; spin_unlock_bh(&task->lock); /* restart task in case */ @@ -110,7 +110,7 @@ static void cleanup_task(struct rxe_task *task) do { spin_lock_bh(&task->lock); - idle = (task->state == TASK_STATE_START || + idle = (task->state == TASK_STATE_IDLE || task->state == TASK_STATE_PAUSED); spin_unlock_bh(&task->lock); } while (!idle); @@ -219,7 +219,7 @@ int rxe_init_task(struct rxe_task *task, void *arg, int (*func)(void *), task->func = func; task->destroyed = false; task->type = type; - task->state = TASK_STATE_START; + task->state = TASK_STATE_IDLE; spin_lock_init(&task->lock); diff --git a/drivers/infiniband/sw/rxe/rxe_task.h b/drivers/infiniband/sw/rxe/rxe_task.h index 792832786456..0146307fc517 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.h +++ b/drivers/infiniband/sw/rxe/rxe_task.h @@ -23,7 +23,7 @@ enum rxe_task_type { }; enum { - TASK_STATE_START = 0, + TASK_STATE_IDLE = 0, TASK_STATE_BUSY = 1, TASK_STATE_ARMED = 2, TASK_STATE_PAUSED = 3, From patchwork Fri Oct 21 20:01:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015365 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A4FFECDFA1 for ; Fri, 21 Oct 2022 20:03:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229894AbiJUUC7 (ORCPT ); Fri, 21 Oct 2022 16:02:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230217AbiJUUCZ (ORCPT ); Fri, 21 Oct 2022 16:02:25 -0400 Received: from mail-ot1-x336.google.com (mail-ot1-x336.google.com [IPv6:2607:f8b0:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D14DF25ED0F for ; Fri, 21 Oct 2022 13:02:22 -0700 (PDT) Received: by mail-ot1-x336.google.com with SMTP id 101-20020a9d0bee000000b00661b54d945fso2433017oth.13 for ; Fri, 21 Oct 2022 13:02:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6Z4qyWIMoBUjYRI1MMwuXnIkq79/rYZQuSMRXsWdTRk=; b=I2i0yQM736BQRWzDjjJtYRuHvzDXKuvj0XJxmJpj6SW6Bx8ZV+qQahYr3YRKCdV1U9 1KzqUW7PRSvbLzhIXZxfzBJUZg5exxnM1aD6Jxf0/RN8furpjtEPmcpmxkbZlSsyrn9R s3aY80sMB87f2AQnkAGgZsjN/Z7DCtb6mTne0wYYbnjt2CVbPsy5AW1Gw3uCe/AIpsE+ V6Rp42X0Jz2G5KUEWeW52lhqjvfkWn76qCnWTWfsVmyk0ytPBy7cudQYWnQpedkPbVHv LIrmddG44EutYeOltWtTy9/rdtU7wJkS/p8kZfLNM9kvjECSyqhvVnulLiYpzDT4mdYH fn4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6Z4qyWIMoBUjYRI1MMwuXnIkq79/rYZQuSMRXsWdTRk=; b=kF+aMXAYfrc4jCe5hMm9czFHB7uFzGGuzZ2Cwu6WbbT+Thxvs2C8VOiMuo60O8xOq0 t4avFCC9ySliVX3ZLAe5ob3KUPehJ5c8Y7j9taYs7MJoC6VNS/d7zXnJGxZG0SWJPZGU FqL7vDL4e0U+cOG3DER+AMwru6GjZ76/4ge60QOnaDTUhJiCqaQgvserssa7JyAlvWoG KD2gk0/LfBO6jerVR1HowcceM5sKvkHEtKKxflspchvyom39r69nYQDSIFYeWuS7lkD6 fol36yiNd5r/gASk5Cy+RxpNR1Fhpubuy1hDqorSm0yj+eOWPlh6x54m6dJdBQKZg+yf QLjQ== X-Gm-Message-State: ACrzQf3EjzpIroXGwA6+n4QcVuAZ9VK3tK8YAXjSU7RQqG01MdZkIY3W 2ZXnAfMkTDdxk4d7gRnrojs= X-Google-Smtp-Source: AMsMyM7eWBocsY68HKL3tVnnxCRSA/K89JiQ2tkk8f5PDsXzqJb5ADCkKSWhAaEmPVHbAnG2Q0K95g== X-Received: by 2002:a05:6830:2b09:b0:661:c464:3c6f with SMTP id l9-20020a0568302b0900b00661c4643c6fmr10646538otv.90.1666382541235; Fri, 21 Oct 2022 13:02:21 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:20 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 16/18] RDMA/rxe: Replace task->destroyed by task state INVALID. Date: Fri, 21 Oct 2022 15:01:17 -0500 Message-Id: <20221021200118.2163-17-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Add a new state TASK_STATE_INVALID to replace the flag task->destroyed. Make changes to task->state proteted by task->lock now including TASK_STATE_INVALID. Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_task.c | 49 ++++++++++------------------ drivers/infiniband/sw/rxe/rxe_task.h | 3 +- 2 files changed, 19 insertions(+), 33 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c index 0fd0d97e8272..da175f2a0dbf 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.c +++ b/drivers/infiniband/sw/rxe/rxe_task.c @@ -8,9 +8,6 @@ static bool task_is_idle(struct rxe_task *task) { - if (task->destroyed) - return false; - spin_lock_bh(&task->lock); switch (task->state) { case TASK_STATE_IDLE: @@ -19,12 +16,8 @@ static bool task_is_idle(struct rxe_task *task) return true; case TASK_STATE_BUSY: task->state = TASK_STATE_ARMED; - fallthrough; - case TASK_STATE_ARMED: - case TASK_STATE_PAUSED: break; default: - WARN_ON(1); break; } spin_unlock_bh(&task->lock); @@ -41,7 +34,8 @@ static void do_task(struct rxe_task *task) /* flush out pending tasks */ spin_lock_bh(&task->lock); - if (task->state == TASK_STATE_PAUSED) { + if (task->state == TASK_STATE_PAUSED || + task->state == TASK_STATE_INVALID) { spin_unlock_bh(&task->lock); return; } @@ -69,10 +63,7 @@ static void do_task(struct rxe_task *task) task->state = TASK_STATE_BUSY; cont = 1; break; - case TASK_STATE_PAUSED: - break; default: - WARN_ON(1); break; } spin_unlock_bh(&task->lock); @@ -87,14 +78,16 @@ static void do_task(struct rxe_task *task) static void disable_task(struct rxe_task *task) { spin_lock_bh(&task->lock); - task->state = TASK_STATE_PAUSED; + if (task->state != TASK_STATE_INVALID) + task->state = TASK_STATE_PAUSED; spin_unlock_bh(&task->lock); } static void enable_task(struct rxe_task *task) { spin_lock_bh(&task->lock); - task->state = TASK_STATE_IDLE; + if (task->state != TASK_STATE_INVALID) + task->state = TASK_STATE_IDLE; spin_unlock_bh(&task->lock); /* restart task in case */ @@ -104,16 +97,16 @@ static void enable_task(struct rxe_task *task) /* busy wait until any previous tasks are done */ static void cleanup_task(struct rxe_task *task) { - bool idle; - - task->destroyed = true; + bool busy; do { spin_lock_bh(&task->lock); - idle = (task->state == TASK_STATE_IDLE || - task->state == TASK_STATE_PAUSED); + busy = (task->state == TASK_STATE_BUSY || + task->state == TASK_STATE_ARMED); + if (!busy) + task->state = TASK_STATE_INVALID; spin_unlock_bh(&task->lock); - } while (!idle); + } while (busy); } /* silently treat schedule as inline for inline tasks */ @@ -131,14 +124,12 @@ static void inline_run(struct rxe_task *task) static void inline_disable(struct rxe_task *task) { - if (!task->destroyed) - disable_task(task); + disable_task(task); } static void inline_enable(struct rxe_task *task) { - if (!task->destroyed) - enable_task(task); + enable_task(task); } static void inline_cleanup(struct rxe_task *task) @@ -168,10 +159,7 @@ static void tsklet_sched(struct rxe_task *task) static void tsklet_do_task(struct tasklet_struct *tasklet) { - struct rxe_task *task = container_of(tasklet, typeof(*task), tasklet); - - if (!task->destroyed) - do_task(task); + do_task(container_of(tasklet, struct rxe_task, tasklet)); } static void tsklet_run(struct rxe_task *task) @@ -182,14 +170,12 @@ static void tsklet_run(struct rxe_task *task) static void tsklet_disable(struct rxe_task *task) { - if (!task->destroyed) - disable_task(task); + disable_task(task); } static void tsklet_enable(struct rxe_task *task) { - if (!task->destroyed) - enable_task(task); + enable_task(task); } static void tsklet_cleanup(struct rxe_task *task) @@ -217,7 +203,6 @@ int rxe_init_task(struct rxe_task *task, void *arg, int (*func)(void *), { task->arg = arg; task->func = func; - task->destroyed = false; task->type = type; task->state = TASK_STATE_IDLE; diff --git a/drivers/infiniband/sw/rxe/rxe_task.h b/drivers/infiniband/sw/rxe/rxe_task.h index 0146307fc517..2c4ef4d339f1 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.h +++ b/drivers/infiniband/sw/rxe/rxe_task.h @@ -27,6 +27,7 @@ enum { TASK_STATE_BUSY = 1, TASK_STATE_ARMED = 2, TASK_STATE_PAUSED = 3, + TASK_STATE_INVALID = 4, }; /* @@ -41,7 +42,7 @@ struct rxe_task { void *arg; int (*func)(void *arg); int ret; - bool destroyed; + bool invalid; const struct rxe_task_ops *ops; enum rxe_task_type type; }; From patchwork Fri Oct 21 20:01:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015367 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 375FEC38A2D for ; Fri, 21 Oct 2022 20:03:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230070AbiJUUDA (ORCPT ); Fri, 21 Oct 2022 16:03:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48734 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230328AbiJUUC1 (ORCPT ); Fri, 21 Oct 2022 16:02:27 -0400 Received: from mail-oo1-xc2e.google.com (mail-oo1-xc2e.google.com [IPv6:2607:f8b0:4864:20::c2e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C510726109E for ; Fri, 21 Oct 2022 13:02:23 -0700 (PDT) Received: by mail-oo1-xc2e.google.com with SMTP id j6-20020a4ab1c6000000b004809a59818cso597726ooo.0 for ; Fri, 21 Oct 2022 13:02:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cJWw3ZWMyyqJyXyBd0hMBqS86p0cuWLJRsxZt7tbY50=; b=Ea+0oy3gaxaeDylQ3Gn/lcdFomaCjTK/fuYibIXnWG6o2yvxfVfp7mnoXzBpC3VsDH sdywuG2OKqby4RJGpUyrenp1A1z/nnVKRGwihQcwPonJbElZNxgDWNF21RPua/Y+XzS+ yKFG1beycOqIsOltVIbQaAp5gV8Hr1pBe2jXt3Dxn2YfCfaM6+y71YlXdjuZC9Zvq6bB kkRpEENwGgDtbqiMGV5jY1ZeiFpqWJoJ/somrvBYyyDpyyWbGBDKhfmsHpkjUouXZ+uc IS6zFPsGPpM1Ki2fS4gAJHaGzsGwbVQ0YtBp/N8qm/WhXboozmk/2ttoBbx5m4GAZGPa T87A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cJWw3ZWMyyqJyXyBd0hMBqS86p0cuWLJRsxZt7tbY50=; b=Zt3Dv5Rd1YbgWDh+BnNeFwZgTDoabIFr22y7KNuNeJFlx4/wRNt29RsdhiGDMuqlKV YUwM6Cw+wftSWUmihHuSY1kjGpJwGHkw2jUUubs4rZhGGyHS7zV5MoEIHe7yF6y1FGYU IvsNZkw4+sAgAxwmnhlnitafMShmhGKT4BIGyo1gXPHYkJUEpR2fW3EV+P1Z25UdNDiW ENMkgVkplKGlAtlVCAV6/EugNlZp1OjeGsRza9KS3Kh64NZ7580GtiCOwQW6d1trNGmV KmCnVx4nd3rUruu+tDK/JG42pKslgPZ+0zMH0HlmDGo9R30f9UHdt8hi1l6tp/Jy2jFm PBKw== X-Gm-Message-State: ACrzQf3dGaaYbxm2bnG7BEXgDkT2UMFQUXBZxNSdxanMx9hgEV0bw4AS kJylIvu1DuFXNPmeV9I3LhY= X-Google-Smtp-Source: AMsMyM5+KCQVFxiWOrhiN8HZ/i698R6AW6J1kSPhtiNeRwsYXvUEgBnUtTDJlDEInrU3uvcJmk1uHA== X-Received: by 2002:a4a:97a5:0:b0:475:fa71:731e with SMTP id w34-20020a4a97a5000000b00475fa71731emr9712464ooi.38.1666382542534; Fri, 21 Oct 2022 13:02:22 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:22 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 17/18] RDMA/rxe: Add workqueue support for tasks Date: Fri, 21 Oct 2022 15:01:18 -0500 Message-Id: <20221021200118.2163-18-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Add a third task type RXE_TASK_TYPE_WORKQUEUE to rxe_task.c. Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe.c | 9 +++- drivers/infiniband/sw/rxe/rxe_task.c | 69 ++++++++++++++++++++++++++++ drivers/infiniband/sw/rxe/rxe_task.h | 10 +++- 3 files changed, 86 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index 51daac5c4feb..6d80218334ca 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -210,10 +210,16 @@ static int __init rxe_module_init(void) { int err; - err = rxe_net_init(); + err = rxe_alloc_wq(); if (err) return err; + err = rxe_net_init(); + if (err) { + rxe_destroy_wq(); + return err; + } + rdma_link_register(&rxe_link_ops); pr_info("loaded\n"); return 0; @@ -224,6 +230,7 @@ static void __exit rxe_module_exit(void) rdma_link_unregister(&rxe_link_ops); ib_unregister_driver(RDMA_DRIVER_RXE); rxe_net_exit(); + rxe_destroy_wq(); pr_info("unloaded\n"); } diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c index da175f2a0dbf..9b8c9d28ee46 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.c +++ b/drivers/infiniband/sw/rxe/rxe_task.c @@ -6,6 +6,22 @@ #include "rxe.h" +static struct workqueue_struct *rxe_wq; + +int rxe_alloc_wq(void) +{ + rxe_wq = alloc_workqueue("rxe_wq", WQ_CPU_INTENSIVE, WQ_MAX_ACTIVE); + if (!rxe_wq) + return -ENOMEM; + + return 0; +} + +void rxe_destroy_wq(void) +{ + destroy_workqueue(rxe_wq); +} + static bool task_is_idle(struct rxe_task *task) { spin_lock_bh(&task->lock); @@ -198,6 +217,53 @@ static void tsklet_init(struct rxe_task *task) task->ops = &tsklet_ops; } +static void work_sched(struct rxe_task *task) +{ + if (task_is_idle(task)) + queue_work(rxe_wq, &task->work); +} + +static void work_do_task(struct work_struct *work) +{ + do_task(container_of(work, struct rxe_task, work)); +} + +static void work_run(struct rxe_task *task) +{ + if (task_is_idle(task)) + do_task(task); +} + +static void work_enable(struct rxe_task *task) +{ + enable_task(task); +} + +static void work_disable(struct rxe_task *task) +{ + disable_task(task); + flush_workqueue(rxe_wq); +} + +static void work_cleanup(struct rxe_task *task) +{ + cleanup_task(task); +} + +static const struct rxe_task_ops work_ops = { + .sched = work_sched, + .run = work_run, + .enable = work_enable, + .disable = work_disable, + .cleanup = work_cleanup, +}; + +static void work_init(struct rxe_task *task) +{ + INIT_WORK(&task->work, work_do_task); + task->ops = &work_ops; +} + int rxe_init_task(struct rxe_task *task, void *arg, int (*func)(void *), enum rxe_task_type type) { @@ -215,6 +281,9 @@ int rxe_init_task(struct rxe_task *task, void *arg, int (*func)(void *), case RXE_TASK_TYPE_TASKLET: tsklet_init(task); break; + case RXE_TASK_TYPE_WORKQUEUE: + work_init(task); + break; default: pr_debug("%s: invalid task type = %d\n", __func__, type); return -EINVAL; diff --git a/drivers/infiniband/sw/rxe/rxe_task.h b/drivers/infiniband/sw/rxe/rxe_task.h index 2c4ef4d339f1..d1156b935635 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.h +++ b/drivers/infiniband/sw/rxe/rxe_task.h @@ -20,6 +20,7 @@ struct rxe_task_ops { enum rxe_task_type { RXE_TASK_TYPE_INLINE = 0, RXE_TASK_TYPE_TASKLET = 1, + RXE_TASK_TYPE_WORKQUEUE = 2, }; enum { @@ -36,7 +37,10 @@ enum { * called again. */ struct rxe_task { - struct tasklet_struct tasklet; + union { + struct tasklet_struct tasklet; + struct work_struct work; + }; int state; spinlock_t lock; void *arg; @@ -47,6 +51,10 @@ struct rxe_task { enum rxe_task_type type; }; +int rxe_alloc_wq(void); + +void rxe_destroy_wq(void); + int rxe_init_task(struct rxe_task *task, void *arg, int (*func)(void *), enum rxe_task_type type); From patchwork Fri Oct 21 20:01:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13015368 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 223C4C3A59D for ; Fri, 21 Oct 2022 20:03:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230180AbiJUUDB (ORCPT ); Fri, 21 Oct 2022 16:03:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48820 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229776AbiJUUC2 (ORCPT ); Fri, 21 Oct 2022 16:02:28 -0400 Received: from mail-ot1-x331.google.com (mail-ot1-x331.google.com [IPv6:2607:f8b0:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8EEC6262DC7 for ; Fri, 21 Oct 2022 13:02:25 -0700 (PDT) Received: by mail-ot1-x331.google.com with SMTP id r8-20020a056830120800b00661a0a236efso2469479otp.4 for ; Fri, 21 Oct 2022 13:02:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iEGocqs8ZhTGDBgstqxV42UMAnOhyyzVCvP5BXHM/pk=; b=LqIHF4yjUeEOaLT12gPTdzP0ZmFnlOMZTj6weoUmBcrba4SoNoQyOnLJ44XL0Qpmw0 nsr0BE7GFPd1/v9y6gx/SZfMhjUt4l3L2Nb1rnAha+gajQvHEBEovLqZM3FATRLrojIj FyemMMTQkV6lFfuhyZvCOpO78kDBDi22ud+OHTn9hjaJlvcQ/kL9hdJ1XbF3D1z757Ze VFyc8j6+hbSQe8V4G3Q6LDw+yNGxFnOttRWqvH9waWPIVYDcmhRqLUBZYwHvwBpdROaz iv3oroflAjaBBzo4fPtd0T3iWFHQ7l0Dkm+TZaHEfRdkYnfNG4R4M75oWBAevfldHYlj siTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iEGocqs8ZhTGDBgstqxV42UMAnOhyyzVCvP5BXHM/pk=; b=NWPkDWByT2jfPzPqyf73vX274aNW9Fc5YwHF9mLBc2d/3n0XtQqOAK+40s36xu/KDz gepoftvx/liq58JN/7MlzAPz7NZG13xBytCAvduIIgdss1hN5m2j9Afc+kgtNt96jZLy aQhfQ3qbqZLgq0kIbjdr38BwEPG+WvXr1Z+S34b2N5Ar+Aw47td/K+S9P0Ahkn/6CmU0 NKOjLsC+IgUkxGFeL/x2BOfTQu1wOcAwiLfnIbXycklfSCD6dLAMKCM8PPFQYjkpBERr l+U+/d2Unx8gH0RhU8FEd5Ui86iHH+kwsjairLKIVINpV0kuoqDk6IbZm7jimhniXKR/ FYfQ== X-Gm-Message-State: ACrzQf1gF6IusqeZdowY8sH7kslcc9vJTHuZA6J4XCBNoigy1tkcFFkO EAGicqgJNJiIpdPI0Cb14ouSmsrHNXc= X-Google-Smtp-Source: AMsMyM6shc3K0jWIYMGLMZWKIgTM6cSFB5MKCBmCaIURG0E63fhD9BPNiZ/pts0lCltGUR13KU6N4w== X-Received: by 2002:a05:6830:6307:b0:660:be1d:a753 with SMTP id cg7-20020a056830630700b00660be1da753mr10885243otb.54.1666382543845; Fri, 21 Oct 2022 13:02:23 -0700 (PDT) Received: from ubuntu-22.tx.rr.com (2603-8081-140c-1a00-a860-f1d2-9e17-7593.res6.spectrum.com. [2603:8081:140c:1a00:a860:f1d2:9e17:7593]) by smtp.googlemail.com with ESMTPSA id s23-20020a056870631700b00132f141ef2dsm10674684oao.56.2022.10.21.13.02.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Oct 2022 13:02:23 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, ian.ziemba@hpe.com, matsuda-daisuke@fujitsu.com, lizhijian@fujitsu.com, haris.phnx@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 18/18] RDMA/rxe: Add parameters to control task type Date: Fri, 21 Oct 2022 15:01:19 -0500 Message-Id: <20221021200118.2163-19-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221021200118.2163-1-rpearsonhpe@gmail.com> References: <20221021200118.2163-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Add modparams to control the task types for req, comp, and resp tasks. It is expected that the work queue version will take the place of the tasklet version once this patch series is accepted and moved upstream. However, for now it is convenient to keep the option of easily switching between the versions to help benchmarking and debugging. Signed-off-by: Ian Ziemba Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_qp.c | 6 +++--- drivers/infiniband/sw/rxe/rxe_task.c | 8 ++++++++ drivers/infiniband/sw/rxe/rxe_task.h | 4 ++++ 3 files changed, 15 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index 50f6b8b8ad9d..673d52271062 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -238,9 +238,9 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, skb_queue_head_init(&qp->req_pkts); - rxe_init_task(&qp->req.task, qp, rxe_requester, RXE_TASK_TYPE_TASKLET); + rxe_init_task(&qp->req.task, qp, rxe_requester, rxe_req_task_type); rxe_init_task(&qp->comp.task, qp, rxe_completer, - (qp_type(qp) == IB_QPT_RC) ? RXE_TASK_TYPE_TASKLET : + (qp_type(qp) == IB_QPT_RC) ? rxe_comp_task_type : RXE_TASK_TYPE_INLINE); qp->qp_timeout_jiffies = 0; /* Can't be set for UD/UC in modify_qp */ @@ -288,7 +288,7 @@ static int rxe_qp_init_resp(struct rxe_dev *rxe, struct rxe_qp *qp, skb_queue_head_init(&qp->resp_pkts); - rxe_init_task(&qp->resp.task, qp, rxe_responder, RXE_TASK_TYPE_TASKLET); + rxe_init_task(&qp->resp.task, qp, rxe_responder, rxe_resp_task_type); qp->resp.opcode = OPCODE_NONE; qp->resp.msn = 0; diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c index 9b8c9d28ee46..4568c4a05e85 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.c +++ b/drivers/infiniband/sw/rxe/rxe_task.c @@ -6,6 +6,14 @@ #include "rxe.h" +int rxe_req_task_type = RXE_TASK_TYPE_TASKLET; +int rxe_comp_task_type = RXE_TASK_TYPE_TASKLET; +int rxe_resp_task_type = RXE_TASK_TYPE_TASKLET; + +module_param_named(req_task_type, rxe_req_task_type, int, 0664); +module_param_named(comp_task_type, rxe_comp_task_type, int, 0664); +module_param_named(resp_task_type, rxe_resp_task_type, int, 0664); + static struct workqueue_struct *rxe_wq; int rxe_alloc_wq(void) diff --git a/drivers/infiniband/sw/rxe/rxe_task.h b/drivers/infiniband/sw/rxe/rxe_task.h index d1156b935635..5a2ac7ada05b 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.h +++ b/drivers/infiniband/sw/rxe/rxe_task.h @@ -7,6 +7,10 @@ #ifndef RXE_TASK_H #define RXE_TASK_H +extern int rxe_req_task_type; +extern int rxe_comp_task_type; +extern int rxe_resp_task_type; + struct rxe_task; struct rxe_task_ops {