From patchwork Wed Sep 20 08:41:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13392299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 288E1CE79AB for ; Wed, 20 Sep 2023 08:53:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233406AbjITIxP (ORCPT ); Wed, 20 Sep 2023 04:53:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41590 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232813AbjITIxP (ORCPT ); Wed, 20 Sep 2023 04:53:15 -0400 Received: from mail-pf1-x432.google.com (mail-pf1-x432.google.com [IPv6:2607:f8b0:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C1F394 for ; Wed, 20 Sep 2023 01:53:09 -0700 (PDT) Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-690b8859c46so2116358b3a.3 for ; Wed, 20 Sep 2023 01:53:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1695199989; x=1695804789; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=4IoBmgwb4Y/zKSNxhQXnlJXuUPoyV9n/NbIMbYX70Us=; b=eY2jAaHwHqllpSm4N2n2zs2Fn0LS73thoi/yRCb8syNEJdM+mdmfcL//pau2NS4QmL 7IXKEpRyVp4P/gbTpdTvDgeFh4z2zPtm67MTlZaPdpVxxjIG3cA5xa4fxDsyY2uKEmZb KMNa+sK1/pFBJbDI8J/lybdUuQVP1C2ilgozQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695199989; x=1695804789; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=4IoBmgwb4Y/zKSNxhQXnlJXuUPoyV9n/NbIMbYX70Us=; b=LLNTXcz8rPglsWLtBF5Nhb68FoRgsH6XRr3SKGx0qDzzOROeNWJPp5yPq6pLVncBry dGCcI3qzriI7Bto1HWRNdxiJpOB5NKzyIfdIPP0lLZPC+HZ6fmotvL7+J8VA+PkMiceI fROo7b0KsuYRNZDN32W05zOlB725T5Kn234zJ+f+HQaWtJhext6drgAjKvsJO55wQZgi oecqVan7cwO9J/ajrApbDoiXXqd6p2bpABiazH8A9whoa9IJFm3VvKUbWQ0BRcXa0YTA BoEFJLSS5UUcOatdVzeBtDYhDNAU911bgnNWkIf6vBxnN0mh/HT8zL6VBk6i2xRREABG O7bg== X-Gm-Message-State: AOJu0Yzlbdwg3s8e4b2X2aEZPkAxHD8V1CmxdjEhVMsYrgu45o8we1Q7 /7Hu8n2ABx7j5cyScnXIHv0pVuspVEIMO8k+Bbo= X-Google-Smtp-Source: AGHT+IES+s1vZTtkDGyF2XL66lKGeGEOcnRE65NlfKrU53LIB772hBXvAqpte0J9jCOMpicNQHojrw== X-Received: by 2002:a05:6a20:dd87:b0:159:e0b9:bd25 with SMTP id kw7-20020a056a20dd8700b00159e0b9bd25mr1592325pzb.57.1695199988720; Wed, 20 Sep 2023 01:53:08 -0700 (PDT) Received: from dhcp-10-192-206-197.iig.avagotech.net.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id v12-20020a17090ae98c00b00276d039aecasm543099pjy.13.2023.09.20.01.53.06 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 20 Sep 2023 01:53:07 -0700 (PDT) From: Selvin Xavier To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, Selvin Xavier , Saravanan Vajravel Subject: [PATCH for-rc 1/2] RDMA/bnxt_re: Fix the handling of control path response data Date: Wed, 20 Sep 2023 01:41:19 -0700 Message-Id: <1695199280-13520-2-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1695199280-13520-1-git-send-email-selvin.xavier@broadcom.com> References: <1695199280-13520-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Flag that indicate control path command completion should be cleared only after copying the command response data. As soon as the is_in_used flag is clear, the waiting thread can proceed with wrong response data. This wrong data is causing multiple issues like wrong lkey used in data traffic and wrong AH Id etc. Use a memory barrier to ensure that the response data is copied and visible to the process waiting on a different cpu core before clearing the is_in_used flag. Clear the is_in_used after copying the command response. Fixes: bcfee4ce3e01 ("RDMA/bnxt_re: remove redundant cmdq_bitmap") Signed-off-by: Saravanan Vajravel Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/qplib_rcfw.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c index c8c4017..e47b4ca 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c +++ b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c @@ -665,7 +665,6 @@ static int bnxt_qplib_process_qp_event(struct bnxt_qplib_rcfw *rcfw, blocked = cookie & RCFW_CMD_IS_BLOCKING; cookie &= RCFW_MAX_COOKIE_VALUE; crsqe = &rcfw->crsqe_tbl[cookie]; - crsqe->is_in_used = false; if (WARN_ONCE(test_bit(FIRMWARE_STALL_DETECTED, &rcfw->cmdq.flags), @@ -681,8 +680,14 @@ static int bnxt_qplib_process_qp_event(struct bnxt_qplib_rcfw *rcfw, atomic_dec(&rcfw->timeout_send); if (crsqe->is_waiter_alive) { - if (crsqe->resp) + if (crsqe->resp) { memcpy(crsqe->resp, qp_event, sizeof(*qp_event)); + /* Insert write memory barrier to ensure that + * response data is copied before clearing the + * flags + */ + smp_wmb(); + } if (!blocked) wait_cmds++; } @@ -694,6 +699,8 @@ static int bnxt_qplib_process_qp_event(struct bnxt_qplib_rcfw *rcfw, if (!is_waiter_alive) crsqe->resp = NULL; + crsqe->is_in_used = false; + hwq->cons += req_size; /* This is a case to handle below scenario -