From patchwork Mon Sep 30 23:16:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11167817 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 45807912 for ; Mon, 30 Sep 2019 23:17:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2EB6F2168B for ; Mon, 30 Sep 2019 23:17:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731781AbfI3XRU (ORCPT ); Mon, 30 Sep 2019 19:17:20 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:38393 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727118AbfI3XRU (ORCPT ); Mon, 30 Sep 2019 19:17:20 -0400 Received: by mail-pg1-f194.google.com with SMTP id x10so8243255pgi.5 for ; Mon, 30 Sep 2019 16:17:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=r7m6YScsy0R9Q7oVPrZM5vUGUw2WYUyTOdi6FlEBqQ4=; b=AiEO5lF7mIiRMMKWE9Rp3/45+NalyakPXXtCUqqEetM2hndvVT6gc9ji+qrBMICf3+ u9z8GWJT4VIB5aQ3t0hKxboAe0LsJN+DbZlLoBg1VjNAOJtY0NtRc4wYv4bwGwC3lKen Kuj2YOfTo2nQHB0ISFN8wbmnA1h7IPtQXe2Omel0avmTdHbrANkAE9+rk9CdqaQN6Ll+ 1r3APwC5yZwQgJPWvokM/N6s68Qa3w5A97DEp8ervXs6UsdouBKaJwoLyATRhchbSuiS T05bEVniSiB8JZwJWqpoeIGebqLGb9jbFpzXYUzGxwX0WrUbVumsmaKSio2om3reAkmJ F0ag== X-Gm-Message-State: APjAAAUh4aAijf8dD3TbexU0hONgC+aeqDuU/0CslhjWKJmtun/+skfY WF53O139jkldwNnwZb6zi+k= X-Google-Smtp-Source: APXvYqyixEs8LdmnesVA5hr1pCHE0Q9FxQwgOy9rJ7uh43xIpKQowFOMend5u98X9vFaPkSrUI96tA== X-Received: by 2002:a62:1d12:: with SMTP id d18mr24438990pfd.53.1569885439799; Mon, 30 Sep 2019 16:17:19 -0700 (PDT) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id l7sm585406pjy.12.2019.09.30.16.17.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2019 16:17:19 -0700 (PDT) From: Bart Van Assche To: Jason Gunthorpe Cc: Leon Romanovsky , Doug Ledford , linux-rdma@vger.kernel.org, Bart Van Assche Subject: [PATCH 01/15] RDMA/ucma: Reduce the number of rdma_destroy_id() calls Date: Mon, 30 Sep 2019 16:16:53 -0700 Message-Id: <20190930231707.48259-2-bvanassche@acm.org> X-Mailer: git-send-email 2.23.0.444.g18eeb5a265-goog In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org> References: <20190930231707.48259-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Instead of calling rdma_destroy_id() after waiting for the context completion finished, call rdma_destroy_id() from inside the ucma_put_ctx() function. This patch reduces the number of rdma_destroy_id() calls but does not change the behavior of this code. Signed-off-by: Bart Van Assche --- drivers/infiniband/core/ucma.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c index 0274e9b704be..30c09864fd9e 100644 --- a/drivers/infiniband/core/ucma.c +++ b/drivers/infiniband/core/ucma.c @@ -160,8 +160,14 @@ static struct ucma_context *ucma_get_ctx(struct ucma_file *file, int id) static void ucma_put_ctx(struct ucma_context *ctx) { - if (atomic_dec_and_test(&ctx->ref)) - complete(&ctx->comp); + if (!atomic_dec_and_test(&ctx->ref)) + return; + /* + * rdma_destroy_id() ensures that no event handlers are inflight + * for that id before releasing it. + */ + rdma_destroy_id(ctx->cm_id); + complete(&ctx->comp); } /* @@ -199,8 +205,6 @@ static void ucma_close_id(struct work_struct *work) */ ucma_put_ctx(ctx); wait_for_completion(&ctx->comp); - /* No new events will be generated after destroying the id. */ - rdma_destroy_id(ctx->cm_id); } static struct ucma_context *ucma_alloc_ctx(struct ucma_file *file) @@ -628,7 +632,6 @@ static ssize_t ucma_destroy_id(struct ucma_file *file, const char __user *inbuf, xa_unlock(&ctx_table); ucma_put_ctx(ctx); wait_for_completion(&ctx->comp); - rdma_destroy_id(ctx->cm_id); } else { xa_unlock(&ctx_table); } @@ -1756,10 +1759,6 @@ static int ucma_close(struct inode *inode, struct file *filp) xa_unlock(&ctx_table); ucma_put_ctx(ctx); wait_for_completion(&ctx->comp); - /* rdma_destroy_id ensures that no event handlers are - * inflight for that id before releasing it. - */ - rdma_destroy_id(ctx->cm_id); } else { xa_unlock(&ctx_table); } From patchwork Mon Sep 30 23:16:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11167819 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 037881599 for ; Mon, 30 Sep 2019 23:17:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E08382168B for ; Mon, 30 Sep 2019 23:17:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731806AbfI3XRW (ORCPT ); Mon, 30 Sep 2019 19:17:22 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:33322 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729057AbfI3XRW (ORCPT ); Mon, 30 Sep 2019 19:17:22 -0400 Received: by mail-pg1-f194.google.com with SMTP id q1so219118pgb.0; Mon, 30 Sep 2019 16:17:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ocb0Btdcve+bO+wvymS9VkmrTcFGK8V6696tRMbHc+k=; b=BhCKdjvdhTBZ69uqLHX2rGhrP36s1pp3LebDq630hlx5pZd1TJU2aRe3anlZwZiRp/ dPE5Qs3uUSvt9BIhLynmIdZR+pcKG3bzSPL96tfKysMZkP9kwMc+rjXh4ToFiNqvT6NC iUGtYjJaiOilU71VexFOrjk5qiBbq9Q2P4XIgLZ3FP6FMG7KhT1eoTkxXuO1GH0aKM11 wHUJY0b0Z8MDl7ScLdgRp/XoAAJ8HtsxhgSpRyGhXGN9gsYWdiF+CmNVw0Me5tJGSKSj 1q9BYcosEACI0JScPOzFtV+gPDlX0AC9P9frw2XCieCHmFOByFFtVuCrPq8zksVnMdco 9+Wg== X-Gm-Message-State: APjAAAVy29WaWbqZ+2ptOMnkFdkcoKZKZsLePPDWHUl6tYW81YnCyQj8 yul+f2I1mPmEKje7oPzVe9k= X-Google-Smtp-Source: APXvYqxqlWVNGWrOUIUEPZcvOX2yzzKdpmHAFcs908qNTdxO1EMIqimea5iYW00BRDuKt2JUsKyy1g== X-Received: by 2002:aa7:9104:: with SMTP id 4mr24407619pfh.176.1569885441335; Mon, 30 Sep 2019 16:17:21 -0700 (PDT) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id l7sm585406pjy.12.2019.09.30.16.17.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2019 16:17:20 -0700 (PDT) From: Bart Van Assche To: Jason Gunthorpe Cc: Leon Romanovsky , Doug Ledford , linux-rdma@vger.kernel.org, Bart Van Assche , Or Gerlitz , Steve Wise , Sagi Grimberg , Bernard Metzler , Krishnamraju Eraparaju , stable@vger.kernel.org Subject: [PATCH 02/15] RDMA/iwcm: Fix a lock inversion issue Date: Mon, 30 Sep 2019 16:16:54 -0700 Message-Id: <20190930231707.48259-3-bvanassche@acm.org> X-Mailer: git-send-email 2.23.0.444.g18eeb5a265-goog In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org> References: <20190930231707.48259-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org This patch fixes the lock inversion complaint: ============================================ WARNING: possible recursive locking detected 5.3.0-rc7-dbg+ #1 Not tainted -------------------------------------------- kworker/u16:6/171 is trying to acquire lock: 00000000035c6e6c (&id_priv->handler_mutex){+.+.}, at: rdma_destroy_id+0x78/0x4a0 [rdma_cm] but task is already holding lock: 00000000bc7c307d (&id_priv->handler_mutex){+.+.}, at: iw_conn_req_handler+0x151/0x680 [rdma_cm] other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(&id_priv->handler_mutex); lock(&id_priv->handler_mutex); *** DEADLOCK *** May be due to missing lock nesting notation 3 locks held by kworker/u16:6/171: #0: 00000000e2eaa773 ((wq_completion)iw_cm_wq){+.+.}, at: process_one_work+0x472/0xac0 #1: 000000001efd357b ((work_completion)(&work->work)#3){+.+.}, at: process_one_work+0x476/0xac0 #2: 00000000bc7c307d (&id_priv->handler_mutex){+.+.}, at: iw_conn_req_handler+0x151/0x680 [rdma_cm] stack backtrace: CPU: 3 PID: 171 Comm: kworker/u16:6 Not tainted 5.3.0-rc7-dbg+ #1 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 Workqueue: iw_cm_wq cm_work_handler [iw_cm] Call Trace: dump_stack+0x8a/0xd6 __lock_acquire.cold+0xe1/0x24d lock_acquire+0x106/0x240 __mutex_lock+0x12e/0xcb0 mutex_lock_nested+0x1f/0x30 rdma_destroy_id+0x78/0x4a0 [rdma_cm] iw_conn_req_handler+0x5c9/0x680 [rdma_cm] cm_work_handler+0xe62/0x1100 [iw_cm] process_one_work+0x56d/0xac0 worker_thread+0x7a/0x5d0 kthread+0x1bc/0x210 ret_from_fork+0x24/0x30 Cc: Or Gerlitz Cc: Steve Wise Cc: Sagi Grimberg Cc: Bernard Metzler Cc: Krishnamraju Eraparaju Cc: Fixes: de910bd92137 ("RDMA/cma: Simplify locking needed for serialization of callbacks"; v2.6.27). Signed-off-by: Bart Van Assche --- drivers/infiniband/core/cma.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 0e3cf3461999..d78f67623f24 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -2396,9 +2396,10 @@ static int iw_conn_req_handler(struct iw_cm_id *cm_id, conn_id->cm_id.iw = NULL; cma_exch(conn_id, RDMA_CM_DESTROYING); mutex_unlock(&conn_id->handler_mutex); + mutex_unlock(&listen_id->handler_mutex); cma_deref_id(conn_id); rdma_destroy_id(&conn_id->id); - goto out; + return ret; } mutex_unlock(&conn_id->handler_mutex); From patchwork Mon Sep 30 23:16:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11167821 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7D47B1599 for ; Mon, 30 Sep 2019 23:17:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 663032168B for ; Mon, 30 Sep 2019 23:17:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729748AbfI3XRY (ORCPT ); Mon, 30 Sep 2019 19:17:24 -0400 Received: from mail-pl1-f196.google.com ([209.85.214.196]:38125 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729057AbfI3XRY (ORCPT ); Mon, 30 Sep 2019 19:17:24 -0400 Received: by mail-pl1-f196.google.com with SMTP id w8so4039159plq.5 for ; Mon, 30 Sep 2019 16:17:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jaWZU2wf7vKjpO8vNoRcYWOSkiD/1uNaWrL8xtDVNMM=; b=O+OHxO+oZZQGXnO9qzy2rdQYAsg6dt6yUTdsfMmIBzUgiDQ1kG9Qf/0b+kPEWWsunw p4Yvp/nhAaMzP7Z617lWHfM32V67nyqUK77Yo5c/QbTB8YRm7SJ4S4QcO7275Bn/T+e8 p/j35vSPyPTZ7b06AmZfWy1hG/kbHne0PwOaJrKh5jfomEpompoeSrzP1Kanrw+/GeCg XwkU6I/j1MeVNSq+h9N0Td+AyopWC2m7nLAJK7uPihWb0MD2z/d0HNWAW89/talhsnMl PQJ+whXDtgZgPYbgzlGTHVD0FN6/cEhHW4bPE9lDBFVoXAhcsh/NRFiZM9B5WetXhVoD 0zvA== X-Gm-Message-State: APjAAAW4ZxhNjh5AmQ99CcGDjDl280XKfDoXA7ZMo21YS51twGbAQ4By AFckkI4k+lZhWjdMN6i41SU= X-Google-Smtp-Source: APXvYqwtHPVerkFurN9QtSWN2DIankIfA8Ee2Grm0PdzvSawhCtL1SAyS9TQuPfS7dWrskH24mpSuw== X-Received: by 2002:a17:902:8e86:: with SMTP id bg6mr12411916plb.212.1569885442565; Mon, 30 Sep 2019 16:17:22 -0700 (PDT) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id l7sm585406pjy.12.2019.09.30.16.17.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2019 16:17:21 -0700 (PDT) From: Bart Van Assche To: Jason Gunthorpe Cc: Leon Romanovsky , Doug Ledford , linux-rdma@vger.kernel.org, Bart Van Assche , Bernard Metzler Subject: [PATCH 03/15] RDMA/siw: Simplify several debug messages Date: Mon, 30 Sep 2019 16:16:55 -0700 Message-Id: <20190930231707.48259-4-bvanassche@acm.org> X-Mailer: git-send-email 2.23.0.444.g18eeb5a265-goog In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org> References: <20190930231707.48259-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Do not print the remote address if it is not used. Use %pISp instead of %pI4 %d. Cc: Bernard Metzler Signed-off-by: Bart Van Assche Reviewed-by: Bernard Metzler --- drivers/infiniband/sw/siw/siw_cm.c | 36 ++++++------------------------ 1 file changed, 7 insertions(+), 29 deletions(-) diff --git a/drivers/infiniband/sw/siw/siw_cm.c b/drivers/infiniband/sw/siw/siw_cm.c index 8c1931a57f4a..5a75deb9870b 100644 --- a/drivers/infiniband/sw/siw/siw_cm.c +++ b/drivers/infiniband/sw/siw/siw_cm.c @@ -1373,22 +1373,8 @@ int siw_connect(struct iw_cm_id *id, struct iw_cm_conn_param *params) rv = -EINVAL; goto error; } - if (v4) - siw_dbg_qp(qp, - "pd_len %d, laddr %pI4 %d, raddr %pI4 %d\n", - pd_len, - &((struct sockaddr_in *)(laddr))->sin_addr, - ntohs(((struct sockaddr_in *)(laddr))->sin_port), - &((struct sockaddr_in *)(raddr))->sin_addr, - ntohs(((struct sockaddr_in *)(raddr))->sin_port)); - else - siw_dbg_qp(qp, - "pd_len %d, laddr %pI6 %d, raddr %pI6 %d\n", - pd_len, - &((struct sockaddr_in6 *)(laddr))->sin6_addr, - ntohs(((struct sockaddr_in6 *)(laddr))->sin6_port), - &((struct sockaddr_in6 *)(raddr))->sin6_addr, - ntohs(((struct sockaddr_in6 *)(raddr))->sin6_port)); + siw_dbg_qp(qp, "pd_len %d, laddr %pISp, raddr %pISp\n", pd_len, laddr, + raddr); rv = sock_create(v4 ? AF_INET : AF_INET6, SOCK_STREAM, IPPROTO_TCP, &s); if (rv < 0) @@ -1935,7 +1921,7 @@ static void siw_drop_listeners(struct iw_cm_id *id) /* * siw_create_listen - Create resources for a listener's IWCM ID @id * - * Listens on the socket addresses id->local_addr and id->remote_addr. + * Listens on the socket address id->local_addr. * * If the listener's @id provides a specific local IP address, at most one * listening socket is created and associated with @id. @@ -1959,7 +1945,7 @@ int siw_create_listen(struct iw_cm_id *id, int backlog) */ if (id->local_addr.ss_family == AF_INET) { struct in_device *in_dev = in_dev_get(dev); - struct sockaddr_in s_laddr, *s_raddr; + struct sockaddr_in s_laddr; const struct in_ifaddr *ifa; if (!in_dev) { @@ -1967,12 +1953,8 @@ int siw_create_listen(struct iw_cm_id *id, int backlog) goto out; } memcpy(&s_laddr, &id->local_addr, sizeof(s_laddr)); - s_raddr = (struct sockaddr_in *)&id->remote_addr; - siw_dbg(id->device, - "laddr %pI4:%d, raddr %pI4:%d\n", - &s_laddr.sin_addr, ntohs(s_laddr.sin_port), - &s_raddr->sin_addr, ntohs(s_raddr->sin_port)); + siw_dbg(id->device, "laddr %pISp\n", &s_laddr); rtnl_lock(); in_dev_for_each_ifa_rtnl(ifa, in_dev) { @@ -1992,17 +1974,13 @@ int siw_create_listen(struct iw_cm_id *id, int backlog) } else if (id->local_addr.ss_family == AF_INET6) { struct inet6_dev *in6_dev = in6_dev_get(dev); struct inet6_ifaddr *ifp; - struct sockaddr_in6 *s_laddr = &to_sockaddr_in6(id->local_addr), - *s_raddr = &to_sockaddr_in6(id->remote_addr); + struct sockaddr_in6 *s_laddr = &to_sockaddr_in6(id->local_addr); if (!in6_dev) { rv = -ENODEV; goto out; } - siw_dbg(id->device, - "laddr %pI6:%d, raddr %pI6:%d\n", - &s_laddr->sin6_addr, ntohs(s_laddr->sin6_port), - &s_raddr->sin6_addr, ntohs(s_raddr->sin6_port)); + siw_dbg(id->device, "laddr %pISp\n", &s_laddr); rtnl_lock(); list_for_each_entry(ifp, &in6_dev->addr_list, if_list) { From patchwork Mon Sep 30 23:16:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11167825 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E0D917E1 for ; Mon, 30 Sep 2019 23:17:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 06F8E2168B for ; Mon, 30 Sep 2019 23:17:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729057AbfI3XR0 (ORCPT ); Mon, 30 Sep 2019 19:17:26 -0400 Received: from mail-pl1-f193.google.com ([209.85.214.193]:38125 "EHLO mail-pl1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731831AbfI3XR0 (ORCPT ); Mon, 30 Sep 2019 19:17:26 -0400 Received: by mail-pl1-f193.google.com with SMTP id w8so4039190plq.5 for ; Mon, 30 Sep 2019 16:17:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=peXAg/IElw6iW3nPywzZ2ouBwYeCgHvHncmueQCplok=; b=KntH65pzJid7HM5kicPiDXSTaS+g3R3CbkX+Is6suKaX+DukFeYx/Khf3FI015KX8N ZjUZ90/VT/1jciSqQRkdue9OXr6cZ06i0jLAQQWAo3BaNFO7dFG000SjcqDdJwpQ8+y0 5lYtkHfokiJkcwQu3Isl5k0hanoLN+auGIJsqUe9ipn2zehNMlup5a2++9k1FeWxOSqm r+3SD2Ifwc+rFaPRY0Umomw7gAtpGU++NVCd5uExs+O8AOR69dz3opHHko8mrkyswKoA zZauCIwwUI1wPHncW9johXLxfC1vi+6X5NM1PBDwd7j3I3/EvBVWhU0BCcFURT8uL1BH dX7A== X-Gm-Message-State: APjAAAUG9lHBKwZRu737acZ1yA/BpXwu90bT/MeOAWdt5TpFfuzHr4FS 6hRfHidvvvOPI9NetVUgiMM= X-Google-Smtp-Source: APXvYqxwN15QedeYoD4jV1TYgJ7EcVP5gBEcAb4hxLYbcY/Cj29lG+Vm9H7mf+dt6ogeWCZFwJZwhw== X-Received: by 2002:a17:902:209:: with SMTP id 9mr22200270plc.1.1569885443904; Mon, 30 Sep 2019 16:17:23 -0700 (PDT) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id l7sm585406pjy.12.2019.09.30.16.17.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2019 16:17:22 -0700 (PDT) From: Bart Van Assche To: Jason Gunthorpe Cc: Leon Romanovsky , Doug Ledford , linux-rdma@vger.kernel.org, Bart Van Assche , Bernard Metzler Subject: [PATCH 04/15] RDMA/siw: Fix port number endianness in a debug message Date: Mon, 30 Sep 2019 16:16:56 -0700 Message-Id: <20190930231707.48259-5-bvanassche@acm.org> X-Mailer: git-send-email 2.23.0.444.g18eeb5a265-goog In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org> References: <20190930231707.48259-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org sin_port and sin6_port are big endian member variables. Convert these port numbers into CPU endianness before printing. Cc: Bernard Metzler Fixes: 6c52fdc244b5 ("rdma/siw: connection management") Signed-off-by: Bart Van Assche Reviewed-by: Bernard Metzler --- drivers/infiniband/sw/siw/siw_cm.c | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/drivers/infiniband/sw/siw/siw_cm.c b/drivers/infiniband/sw/siw/siw_cm.c index 5a75deb9870b..3bccfef40e7e 100644 --- a/drivers/infiniband/sw/siw/siw_cm.c +++ b/drivers/infiniband/sw/siw/siw_cm.c @@ -1853,14 +1853,7 @@ static int siw_listen_address(struct iw_cm_id *id, int backlog, list_add_tail(&cep->listenq, (struct list_head *)id->provider_data); cep->state = SIW_EPSTATE_LISTENING; - if (addr_family == AF_INET) - siw_dbg(id->device, "Listen at laddr %pI4 %u\n", - &(((struct sockaddr_in *)laddr)->sin_addr), - ((struct sockaddr_in *)laddr)->sin_port); - else - siw_dbg(id->device, "Listen at laddr %pI6 %u\n", - &(((struct sockaddr_in6 *)laddr)->sin6_addr), - ((struct sockaddr_in6 *)laddr)->sin6_port); + siw_dbg(id->device, "Listen at laddr %pISp\n", laddr); return 0; From patchwork Mon Sep 30 23:16:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11167823 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AA755912 for ; Mon, 30 Sep 2019 23:17:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 930402168B for ; Mon, 30 Sep 2019 23:17:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731860AbfI3XR0 (ORCPT ); Mon, 30 Sep 2019 19:17:26 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:44091 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729057AbfI3XRZ (ORCPT ); Mon, 30 Sep 2019 19:17:25 -0400 Received: by mail-pf1-f196.google.com with SMTP id q21so6472393pfn.11 for ; Mon, 30 Sep 2019 16:17:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gb3+EeVbu+Tx4lpzyZCMvzRZ5k8a3eMcXGGIPrVaBOw=; b=boyJ1QXpsw+oaOmDGGOoKb4dh5vPsqFrMkKIlrh26T88+svTnRYz+oialc/WXk7pcr qQtCH9JsSyD1eansWBY7TYhNTMimFu49NSP7KP6qdN82TaK2JuHk45zppE7Fqg2oLzEY TbgGB+poa8fN5pd2igQhYhlLN8S/ihi44/f9tu4mhOUx6VJTfgC6kCiYiH+cPOHl0/zY XB1T0PUQoRbal4CP5zI8v5m8I2/O/nGpbOzje0vkHP7PXE6fblnPg8+xxaJA8EEfpVnV oD/pWgdDZsCmj0wcjRb55UHZYU2Vr5HMEJeajoUH/Xly90r2lSqZdfMyOMchxNcuSWfs vz/w== X-Gm-Message-State: APjAAAVsJF9VL1WdNS4n3JmzbnZtpmglnVpTDduXenYdqTaUC9wFyZES FrGVr1sMrcmYG24T14ztB0o= X-Google-Smtp-Source: APXvYqxFCF4RbLa7npVllL5F1qD8k60AkulDBhQaZ8EtjHsgAtN/VeI0zAJ9proDb1WsW7oWxEHkNw== X-Received: by 2002:a63:3e8a:: with SMTP id l132mr23892pga.301.1569885445149; Mon, 30 Sep 2019 16:17:25 -0700 (PDT) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id l7sm585406pjy.12.2019.09.30.16.17.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2019 16:17:24 -0700 (PDT) From: Bart Van Assche To: Jason Gunthorpe Cc: Leon Romanovsky , Doug Ledford , linux-rdma@vger.kernel.org, Bart Van Assche , Bernard Metzler Subject: [PATCH 05/15] RDMA/siw: Make node GUIDs valid EUI-64 identifiers Date: Mon, 30 Sep 2019 16:16:57 -0700 Message-Id: <20190930231707.48259-6-bvanassche@acm.org> X-Mailer: git-send-email 2.23.0.444.g18eeb5a265-goog In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org> References: <20190930231707.48259-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From the IBTA: "GUID (Global Unique Identifier): A globally unique EUI-64 compliant identifier." Make sure that siw GUIDs are valid EUI-64 identifiers. Cc: Bernard Metzler Signed-off-by: Bart Van Assche --- drivers/infiniband/sw/siw/siw_main.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/sw/siw/siw_main.c b/drivers/infiniband/sw/siw/siw_main.c index 05a92f997f60..d1a1b7aa7d83 100644 --- a/drivers/infiniband/sw/siw/siw_main.c +++ b/drivers/infiniband/sw/siw/siw_main.c @@ -16,6 +16,7 @@ #include #include +#include #include #include #include @@ -350,15 +351,19 @@ static struct siw_device *siw_device_create(struct net_device *netdev) sdev->netdev = netdev; if (netdev->type != ARPHRD_LOOPBACK) { - memcpy(&base_dev->node_guid, netdev->dev_addr, 6); + addrconf_addr_eui48((unsigned char *)&base_dev->node_guid, + netdev->dev_addr); } else { /* * The loopback device does not have a HW address, * but connection mangagement lib expects gid != 0 */ - size_t gidlen = min_t(size_t, strlen(base_dev->name), 6); + size_t len = min_t(size_t, strlen(base_dev->name), 6); + char addr[6] = { }; - memcpy(&base_dev->node_guid, base_dev->name, gidlen); + memcpy(addr, base_dev->name, len); + addrconf_addr_eui48((unsigned char *)&base_dev->node_guid, + addr); } base_dev->uverbs_cmd_mask = (1ull << IB_USER_VERBS_CMD_QUERY_DEVICE) | From patchwork Mon Sep 30 23:16:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11167827 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 468811599 for ; Mon, 30 Sep 2019 23:17:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2E8D22168B for ; Mon, 30 Sep 2019 23:17:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731890AbfI3XR2 (ORCPT ); Mon, 30 Sep 2019 19:17:28 -0400 Received: from mail-pl1-f177.google.com ([209.85.214.177]:40064 "EHLO mail-pl1-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731840AbfI3XR2 (ORCPT ); Mon, 30 Sep 2019 19:17:28 -0400 Received: by mail-pl1-f177.google.com with SMTP id d22so4499001pll.7 for ; Mon, 30 Sep 2019 16:17:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ykBs8a3TuaMfCRpJEdk54DvGqAFLnU+m5ckDb+h6hQk=; b=qGkbr74uxd88RjrtOdJjnTuAQ+eznpWZMkgV51UGi1AnftpB+GFFPBLpx2cpgZVVyj BrjCdCikhla42a9edV9BtTh4r+LIf4soTWNkNf20HW7obftZqLM1Aq6Sodpom9ruQVWC SoHeHCr8vg3pBNDkslHYYe1y1HfhGmCnm78DPJ9m3FExvT8i+0qR7h1lMH2rtq/uhoS3 nYUcPlV3xZq/w8JCKO2en9eiGrZxCg1zfjb6YkDT2I4aZsumZ8RQZnjl38FoAv/CMBHc ltvB1eeFZbcpbg6c7eT1Ddzv7HlcRsYg8+Xk6zhe/zXMVnjf+/buCVIFjJSjmkgfwcUU Tx2g== X-Gm-Message-State: APjAAAVRuQBN5dLCPy4A7xk6UJNJrRiCgGGEud21V78WiUbwflHJJyX+ qtYZzm3k7DgoVv88eShCpIlZe2ucxpI= X-Google-Smtp-Source: APXvYqw2viLEInRgOzAbkBZuPPa+jDl5dmUMpsPlVkL2eknBFxYSUI01+dhKdJZBlV3WSxD9V1tBxg== X-Received: by 2002:a17:902:b7ca:: with SMTP id v10mr21722308plz.54.1569885446428; Mon, 30 Sep 2019 16:17:26 -0700 (PDT) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id l7sm585406pjy.12.2019.09.30.16.17.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2019 16:17:25 -0700 (PDT) From: Bart Van Assche To: Jason Gunthorpe Cc: Leon Romanovsky , Doug Ledford , linux-rdma@vger.kernel.org, Bart Van Assche , Honggang LI , Laurence Oberman Subject: [PATCH 06/15] RDMA/srp: Remove two casts Date: Mon, 30 Sep 2019 16:16:58 -0700 Message-Id: <20190930231707.48259-7-bvanassche@acm.org> X-Mailer: git-send-email 2.23.0.444.g18eeb5a265-goog In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org> References: <20190930231707.48259-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org This patch does not change any functionality. Cc: Honggang LI Cc: Laurence Oberman Signed-off-by: Bart Van Assche --- drivers/infiniband/ulp/srp/ib_srp.c | 4 ++-- drivers/infiniband/ulp/srp/ib_srp.h | 2 ++ 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index b5960351bec0..f015dc4ce22c 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -352,8 +352,8 @@ static int srp_new_rdma_cm_id(struct srp_rdma_ch *ch) init_completion(&ch->done); ret = rdma_resolve_addr(new_cm_id, target->rdma_cm.src_specified ? - (struct sockaddr *)&target->rdma_cm.src : NULL, - (struct sockaddr *)&target->rdma_cm.dst, + &target->rdma_cm.src.sa : NULL, + &target->rdma_cm.dst.sa, SRP_PATH_REC_TIMEOUT_MS); if (ret) { pr_err("No route available from %pIS to %pIS (%d)\n", diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h index b2861cd2087a..af9922550ae1 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.h +++ b/drivers/infiniband/ulp/srp/ib_srp.h @@ -245,11 +245,13 @@ struct srp_target_port { union { struct sockaddr_in ip4; struct sockaddr_in6 ip6; + struct sockaddr sa; struct sockaddr_storage ss; } src; union { struct sockaddr_in ip4; struct sockaddr_in6 ip6; + struct sockaddr sa; struct sockaddr_storage ss; } dst; bool src_specified; From patchwork Mon Sep 30 23:16:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11167829 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8B04217E1 for ; Mon, 30 Sep 2019 23:17:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 73F602168B for ; Mon, 30 Sep 2019 23:17:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731840AbfI3XR3 (ORCPT ); Mon, 30 Sep 2019 19:17:29 -0400 Received: from mail-pf1-f194.google.com ([209.85.210.194]:44096 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731831AbfI3XR2 (ORCPT ); Mon, 30 Sep 2019 19:17:28 -0400 Received: by mail-pf1-f194.google.com with SMTP id q21so6472478pfn.11 for ; Mon, 30 Sep 2019 16:17:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e2yyKJ+48yJsyLDfmpXnIm0mNhIybbQPF2WIfgTuD98=; b=MxDzYDft+B8e6AsF6ziYh2ZSujZW8CTSEowaVJfNu0FG6lHK0hsSoIJNKqwJ6qyfVL xzcfjhcQw+pLFwj3HcxRUeKjq5VXC6kBQl+0dM+79p7X2EL4Q3+sbNZ0NfzE+xcFRUs8 NQzk1GBWr5fO7OHKs5+Q7UYei9QGrYUOacj1hDP9XRC3rwAnH1ZDH0jRRRbTrWq7Ik29 m6xgn9bf/cy8iou1wnqw9bi4cqbe0ZL7M0s+JnkC0ZqxGZ0o21yhTVf4Khcpf3DSE5/r qSccSzvk0xPKqOk5hk+37mCnwcz6tUNGKMNsyuZv1MLGiQIderpaFXTo7dyzbuSITCUH ciqg== X-Gm-Message-State: APjAAAXqg4xbzD1SpvPRaNDdBaiS/PVCsUw22ns11CkkVQZdEKlCIhjk 05RvLrRC9hpAlempyBHMuk0= X-Google-Smtp-Source: APXvYqzeSecd5KwPqgD9ItA2eaoclpKvABdHoDsmi9b8NFlZh9VP69oSQbIDNd6YMOOWsRYuKBcKDA== X-Received: by 2002:a65:68c4:: with SMTP id k4mr26642797pgt.180.1569885447963; Mon, 30 Sep 2019 16:17:27 -0700 (PDT) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id l7sm585406pjy.12.2019.09.30.16.17.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2019 16:17:26 -0700 (PDT) From: Bart Van Assche To: Jason Gunthorpe Cc: Leon Romanovsky , Doug Ledford , linux-rdma@vger.kernel.org, Bart Van Assche , Honggang LI , Laurence Oberman Subject: [PATCH 07/15] RDMA/srp: Honor the max_send_sge device attribute Date: Mon, 30 Sep 2019 16:16:59 -0700 Message-Id: <20190930231707.48259-8-bvanassche@acm.org> X-Mailer: git-send-email 2.23.0.444.g18eeb5a265-goog In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org> References: <20190930231707.48259-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Instead of assuming that max_send_sge >= 3, restrict the number of scatter gather elements to what is supported by the RDMA adapter. Cc: Honggang LI Cc: Laurence Oberman Signed-off-by: Bart Van Assche --- drivers/infiniband/ulp/srp/ib_srp.c | 7 +++++-- drivers/infiniband/ulp/srp/ib_srp.h | 1 + 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index f015dc4ce22c..d77e7dd3e745 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -552,6 +552,7 @@ static int srp_create_ch_ib(struct srp_rdma_ch *ch) { struct srp_target_port *target = ch->target; struct srp_device *dev = target->srp_host->srp_dev; + const struct ib_device_attr *attr = &dev->dev->attrs; struct ib_qp_init_attr *init_attr; struct ib_cq *recv_cq, *send_cq; struct ib_qp *qp; @@ -583,12 +584,14 @@ static int srp_create_ch_ib(struct srp_rdma_ch *ch) init_attr->cap.max_send_wr = m * target->queue_size; init_attr->cap.max_recv_wr = target->queue_size + 1; init_attr->cap.max_recv_sge = 1; - init_attr->cap.max_send_sge = SRP_MAX_SGE; + init_attr->cap.max_send_sge = min(SRP_MAX_SGE, attr->max_send_sge); init_attr->sq_sig_type = IB_SIGNAL_REQ_WR; init_attr->qp_type = IB_QPT_RC; init_attr->send_cq = send_cq; init_attr->recv_cq = recv_cq; + ch->max_imm_sge = min(init_attr->cap.max_send_sge - 1U, 255U); + if (target->using_rdma_cm) { ret = rdma_create_qp(ch->rdma_cm.cm_id, dev->pd, init_attr); qp = ch->rdma_cm.cm_id->qp; @@ -1838,7 +1841,7 @@ static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_rdma_ch *ch, return -EIO; if (ch->use_imm_data && - count <= SRP_MAX_IMM_SGE && + count <= ch->max_imm_sge && SRP_IMM_DATA_OFFSET + data_len <= ch->max_it_iu_len && scmnd->sc_data_direction == DMA_TO_DEVICE) { struct srp_imm_buf *buf; diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h index af9922550ae1..f38fbb00d0e8 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.h +++ b/drivers/infiniband/ulp/srp/ib_srp.h @@ -161,6 +161,7 @@ struct srp_rdma_ch { }; uint32_t max_it_iu_len; uint32_t max_ti_iu_len; + u8 max_imm_sge; bool use_imm_data; /* Everything above this point is used in the hot path of From patchwork Mon Sep 30 23:17:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11167831 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0C122912 for ; Mon, 30 Sep 2019 23:17:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E95552168B for ; Mon, 30 Sep 2019 23:17:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731893AbfI3XRa (ORCPT ); Mon, 30 Sep 2019 19:17:30 -0400 Received: from mail-pf1-f195.google.com ([209.85.210.195]:43884 "EHLO mail-pf1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731914AbfI3XRa (ORCPT ); Mon, 30 Sep 2019 19:17:30 -0400 Received: by mail-pf1-f195.google.com with SMTP id a2so6478139pfo.10 for ; Mon, 30 Sep 2019 16:17:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FUxp3rgyLhBSgFTM4FI1qwScxaiOfWHDKvnwbYfupyo=; b=Pqp0XvWhoXrvqex9W2NrDzkO4WQnR6iNkasXpUG9RZMzsFbC2vUyjZOIHT39B57eLF QSRJUb2PzrlDb5lBEvoadWfpWTfc1Tq/9HcNSPxstDONSc/Gp+YaMpShuTQXukjNC8N8 V16sT7UOs42R0jsBSS2bNhAVIXUHRGsSJNkJj6aqnMP2clSsPVXsuwFcxuO+V8ZT7IpO RUVMxwoL+itQE7Vy9moindcxJsnOFFlIChXw4ME6Pd8DSSiYQRWOyxwsbQTIMtmSuRIO 8Ltqq31RxZf7xMCslo2bS1uwH1sT2PTiUoWxzNYhdQYm3YwRj7EPKQ11xqZ8xiHBsEyS +KrA== X-Gm-Message-State: APjAAAU6513vf2wOY6HHCodsrsHC69LmiEuYQ0K7fEut0kVC10XM0tyG CWuD3Bb2lqmLjmtbxfUWfIg= X-Google-Smtp-Source: APXvYqxj3s/9k2zGfkixN6pb3Hxwf1zSc0xacbQpHLstynMhv1H63B8MtSek1tfISbmh1/YhuCoKig== X-Received: by 2002:a63:dc13:: with SMTP id s19mr26607109pgg.272.1569885449492; Mon, 30 Sep 2019 16:17:29 -0700 (PDT) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id l7sm585406pjy.12.2019.09.30.16.17.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2019 16:17:28 -0700 (PDT) From: Bart Van Assche To: Jason Gunthorpe Cc: Leon Romanovsky , Doug Ledford , linux-rdma@vger.kernel.org, Bart Van Assche , Honggang LI , Laurence Oberman Subject: [PATCH 08/15] RDMA/srp: Make route resolving error messages more informative Date: Mon, 30 Sep 2019 16:17:00 -0700 Message-Id: <20190930231707.48259-9-bvanassche@acm.org> X-Mailer: git-send-email 2.23.0.444.g18eeb5a265-goog In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org> References: <20190930231707.48259-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The IPv6 scope ID is essential when setting up an iWARP connection between IPv6 link-local addresses. Report the scope ID in error messages. Cc: Honggang LI Cc: Laurence Oberman Signed-off-by: Bart Van Assche --- drivers/infiniband/ulp/srp/ib_srp.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index d77e7dd3e745..6fddd14b6bd9 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -356,7 +356,7 @@ static int srp_new_rdma_cm_id(struct srp_rdma_ch *ch) &target->rdma_cm.dst.sa, SRP_PATH_REC_TIMEOUT_MS); if (ret) { - pr_err("No route available from %pIS to %pIS (%d)\n", + pr_err("No route available from %pISpsc to %pISpsc (%d)\n", &target->rdma_cm.src, &target->rdma_cm.dst, ret); goto out; } @@ -366,7 +366,7 @@ static int srp_new_rdma_cm_id(struct srp_rdma_ch *ch) ret = ch->status; if (ret) { - pr_err("Resolving address %pIS failed (%d)\n", + pr_err("Resolving address %pISpsc failed (%d)\n", &target->rdma_cm.dst, ret); goto out; } From patchwork Mon Sep 30 23:17:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11167833 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5AAB31599 for ; Mon, 30 Sep 2019 23:17:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 43B2B2168B for ; Mon, 30 Sep 2019 23:17:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731914AbfI3XRb (ORCPT ); Mon, 30 Sep 2019 19:17:31 -0400 Received: from mail-pg1-f193.google.com ([209.85.215.193]:46316 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731852AbfI3XRb (ORCPT ); Mon, 30 Sep 2019 19:17:31 -0400 Received: by mail-pg1-f193.google.com with SMTP id a3so8205408pgm.13 for ; Mon, 30 Sep 2019 16:17:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YJictFiV/NUBeu9ierblkmoLW1qMoxZQbWHlCeAkVdU=; b=EHDSTrxTua6ThHdLbqOwZSLhiZbKVVQxUmjAjAE1mKb8p1zxV4a4IFwxiVH1eWNlE/ 4eHIHhZJivV6/a5/MRNeksOxOC3XIb0jcKpJtv1TTAqUj2TXY2yBhZtGSMiHBBcHE8je DZy0wS1ywan/mnt6h15Oquc4IfxKDo+PEtpegUP5UXDXJORBaLlFU5SOgtIHSxJwBDfZ +PiXvdTE65SI7paPDD/Y5Sfw1MG+qutKrhsM+cQ35UGofvJybJ0xYh1Ges8DQS/qMzio n3exo3uO3XTFNEwjMK+r9Geoc0RKIsPSHWxNGlZQu4tNlOH5VkuIVUKFx+dN4x9UDJSn 5sBg== X-Gm-Message-State: APjAAAXg/NcwJntG3jk46N+6TQIz/o6c7d7X5a578ZWn/fpBa51hFW4m gTut0sMixzGJzKIEaVKYWRg= X-Google-Smtp-Source: APXvYqyJDR9ZbzawqsgdO4uOLesUxB9xAHUqATIcoK75UysUoNsryCcI3jjuQWpRvalMdfH+sGT9Tg== X-Received: by 2002:a17:90a:3450:: with SMTP id o74mr1971221pjb.5.1569885450851; Mon, 30 Sep 2019 16:17:30 -0700 (PDT) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id l7sm585406pjy.12.2019.09.30.16.17.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2019 16:17:29 -0700 (PDT) From: Bart Van Assche To: Jason Gunthorpe Cc: Leon Romanovsky , Doug Ledford , linux-rdma@vger.kernel.org, Bart Van Assche , Honggang LI , Laurence Oberman Subject: [PATCH 09/15] RDMA/srpt: Fix handling of SR-IOV and iWARP ports Date: Mon, 30 Sep 2019 16:17:01 -0700 Message-Id: <20190930231707.48259-10-bvanassche@acm.org> X-Mailer: git-send-email 2.23.0.444.g18eeb5a265-goog In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org> References: <20190930231707.48259-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Management datagrams (MADs) are not supported by SR-IOV VFs nor by iWARP ports. Support SR-IOV VFs and iWARP ports by only logging an error message if MAD handler registration fails. Cc: Honggang LI Cc: Laurence Oberman Signed-off-by: Bart Van Assche --- drivers/infiniband/ulp/srpt/ib_srpt.c | 41 +++++++++++++-------------- 1 file changed, 19 insertions(+), 22 deletions(-) diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c index e25c70a56be6..4f99a5e040c3 100644 --- a/drivers/infiniband/ulp/srpt/ib_srpt.c +++ b/drivers/infiniband/ulp/srpt/ib_srpt.c @@ -556,24 +556,16 @@ static int srpt_refresh_port(struct srpt_port *sport) struct ib_port_attr port_attr; int ret; - memset(&port_modify, 0, sizeof(port_modify)); - port_modify.set_port_cap_mask = IB_PORT_DEVICE_MGMT_SUP; - port_modify.clr_port_cap_mask = 0; - - ret = ib_modify_port(sport->sdev->device, sport->port, 0, &port_modify); - if (ret) - goto err_mod_port; - ret = ib_query_port(sport->sdev->device, sport->port, &port_attr); if (ret) - goto err_query_port; + return ret; sport->sm_lid = port_attr.sm_lid; sport->lid = port_attr.lid; ret = rdma_query_gid(sport->sdev->device, sport->port, 0, &sport->gid); if (ret) - goto err_query_port; + return ret; sport->port_guid_wwn.priv = sport; srpt_format_guid(sport->port_guid, sizeof(sport->port_guid), @@ -584,6 +576,20 @@ static int srpt_refresh_port(struct srpt_port *sport) be64_to_cpu(sport->gid.global.subnet_prefix), be64_to_cpu(sport->gid.global.interface_id)); + if (rdma_protocol_iwarp(sport->sdev->device, sport->port)) + return 0; + + memset(&port_modify, 0, sizeof(port_modify)); + port_modify.set_port_cap_mask = IB_PORT_DEVICE_MGMT_SUP; + port_modify.clr_port_cap_mask = 0; + + ret = ib_modify_port(sport->sdev->device, sport->port, 0, &port_modify); + if (ret) { + pr_warn("%s-%d: enabling device management failed (%d). Note: this is expected if SR-IOV is enabled.\n", + dev_name(&sport->sdev->device->dev), sport->port, ret); + return 0; + } + if (!sport->mad_agent) { memset(®_req, 0, sizeof(reg_req)); reg_req.mgmt_class = IB_MGMT_CLASS_DEVICE_MGMT; @@ -599,23 +605,14 @@ static int srpt_refresh_port(struct srpt_port *sport) srpt_mad_recv_handler, sport, 0); if (IS_ERR(sport->mad_agent)) { - ret = PTR_ERR(sport->mad_agent); + pr_err("%s-%d: MAD agent registration failed (%ld). Note: this is expected if SR-IOV is enabled.\n", + dev_name(&sport->sdev->device->dev), sport->port, + PTR_ERR(sport->mad_agent)); sport->mad_agent = NULL; - goto err_query_port; } } return 0; - -err_query_port: - - port_modify.set_port_cap_mask = 0; - port_modify.clr_port_cap_mask = IB_PORT_DEVICE_MGMT_SUP; - ib_modify_port(sport->sdev->device, sport->port, 0, &port_modify); - -err_mod_port: - - return ret; } /** From patchwork Mon Sep 30 23:17:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11167835 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9763B1599 for ; Mon, 30 Sep 2019 23:17:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 802672168B for ; Mon, 30 Sep 2019 23:17:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731960AbfI3XRd (ORCPT ); Mon, 30 Sep 2019 19:17:33 -0400 Received: from mail-pg1-f193.google.com ([209.85.215.193]:34562 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731852AbfI3XRc (ORCPT ); Mon, 30 Sep 2019 19:17:32 -0400 Received: by mail-pg1-f193.google.com with SMTP id y35so8263485pgl.1 for ; Mon, 30 Sep 2019 16:17:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LeCtAwbRRdyPPNVebY8Q0aEvrjvJ6tMNcrmSbGUu0bw=; b=c42UF1cA06hzwEGsOR6oYS3aP+tFOBki82fC+y59yb/43yBMsNB2FC4163vVVD90g7 Iy27Hy+JyRq4c++TY6Hs2Nvt5EjvTdUACf/jzqJ8tUKak9mcmUEe3KnTRRJMQ7DfPfHA rvDqWz4HLx3N2HDXKQAUEuYFLclAF38yPMke3GZMsUh93dvfQHIFlUKrpSbKu+ct8Op2 e81dEX6rhoQVHOBlOFD/d84QgTI4L49DCnteSs/wd8Un4QHKLmXuLIeWhY+C+LRM8BnW BW2msodSHShePGNP82LMZB/lnW4IU7TMm7X8qefUgHaDWozxFEirFKR8Whu96MlexXpq btOA== X-Gm-Message-State: APjAAAU79GQBH5+z7JpnhVyjPchHS2C9ONIIR1zFvCRUn6ow1F9k2Qjf /GD04kfg/8jz79xNIhnYD/8= X-Google-Smtp-Source: APXvYqxOp7KwP/KgURdTIwky+Mt00O/azIij69OyBMQ5qPiiPpgLehDHR8AR+9vF6y0qxZng6Vqx+g== X-Received: by 2002:a17:90a:65c8:: with SMTP id i8mr1906599pjs.51.1569885452171; Mon, 30 Sep 2019 16:17:32 -0700 (PDT) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id l7sm585406pjy.12.2019.09.30.16.17.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2019 16:17:31 -0700 (PDT) From: Bart Van Assche To: Jason Gunthorpe Cc: Leon Romanovsky , Doug Ledford , linux-rdma@vger.kernel.org, Bart Van Assche , Honggang LI , Laurence Oberman Subject: [PATCH 10/15] RDMA/srpt: Fix handling of iWARP logins Date: Mon, 30 Sep 2019 16:17:02 -0700 Message-Id: <20190930231707.48259-11-bvanassche@acm.org> X-Mailer: git-send-email 2.23.0.444.g18eeb5a265-goog In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org> References: <20190930231707.48259-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The path_rec pointer is NULL set for IB and RoCE logins but not for iWARP logins. Hence check the path_rec pointer before dereferencing it. Cc: Honggang LI Cc: Laurence Oberman Signed-off-by: Bart Van Assche --- drivers/infiniband/ulp/srpt/ib_srpt.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c index 4f99a5e040c3..fbfadeedc195 100644 --- a/drivers/infiniband/ulp/srpt/ib_srpt.c +++ b/drivers/infiniband/ulp/srpt/ib_srpt.c @@ -2502,6 +2502,7 @@ static int srpt_rdma_cm_req_recv(struct rdma_cm_id *cm_id, struct srpt_device *sdev; struct srp_login_req req; const struct srp_login_req_rdma *req_rdma; + struct sa_path_rec *path_rec = cm_id->route.path_rec; char src_addr[40]; sdev = ib_get_client_data(cm_id->device, &srpt_client); @@ -2527,7 +2528,7 @@ static int srpt_rdma_cm_req_recv(struct rdma_cm_id *cm_id, &cm_id->route.addr.src_addr); return srpt_cm_req_recv(sdev, NULL, cm_id, cm_id->port_num, - cm_id->route.path_rec->pkey, &req, src_addr); + path_rec ? path_rec->pkey : 0, &req, src_addr); } static void srpt_cm_rej_recv(struct srpt_rdma_ch *ch, From patchwork Mon Sep 30 23:17:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11167837 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 21FAF912 for ; Mon, 30 Sep 2019 23:17:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id F405F2168B for ; Mon, 30 Sep 2019 23:17:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731968AbfI3XRe (ORCPT ); Mon, 30 Sep 2019 19:17:34 -0400 Received: from mail-pg1-f193.google.com ([209.85.215.193]:34565 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731852AbfI3XRe (ORCPT ); Mon, 30 Sep 2019 19:17:34 -0400 Received: by mail-pg1-f193.google.com with SMTP id y35so8263521pgl.1 for ; Mon, 30 Sep 2019 16:17:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eGtTfGcD29RzTaSchvjdg6mH4L5jyhUU8qp7VAgaELI=; b=AdPfdHQ1zCNN11aq5vS3+ueLqB3vFB/hz9u2Gj+tpfxyli4NOWjsRWed9a9DMyGObd GVbv3CcibyMCwaO74CKTJajM+HgA3MmMRVDPD72eBGJBl88MI+4E8ioasLEL0RKendGh bEUbwvugL9FMWmMxhYZG0TzBNo2DPjlk/HBx9yeYrNxcntm9sahgNApgCZGCxp4HaVml /gEG6qqhVhoxsCk6S8pfPLtnFuuap70Flxd+uMne37gWAKVG6rLnUqKN/mReOqzV2xT1 SO6Y99KresRbZLhqQ9oq9/AqDQpaIMICkiLvVoiQifsUiL9esZNt6zx6xeIRDqVpp2rz cuGg== X-Gm-Message-State: APjAAAVvgIcO5yBAB4NaGneOu/JpJiFdUyzZDL3V1PbngOutmJ80Njdv fI1HXy4dz9zXV3Ug8w1y/WY= X-Google-Smtp-Source: APXvYqwZfeNYDemKwG8xmGfA1peUGhvsys6cMFRnfnxJUOYz1D1d39mgaK8xq3vlTOVNpeGMX74Lqg== X-Received: by 2002:aa7:8436:: with SMTP id q22mr24330861pfn.74.1569885453620; Mon, 30 Sep 2019 16:17:33 -0700 (PDT) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id l7sm585406pjy.12.2019.09.30.16.17.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2019 16:17:32 -0700 (PDT) From: Bart Van Assche To: Jason Gunthorpe Cc: Leon Romanovsky , Doug Ledford , linux-rdma@vger.kernel.org, Bart Van Assche , Honggang LI , Laurence Oberman Subject: [PATCH 11/15] RDMA/srpt: Improve a debug message Date: Mon, 30 Sep 2019 16:17:03 -0700 Message-Id: <20190930231707.48259-12-bvanassche@acm.org> X-Mailer: git-send-email 2.23.0.444.g18eeb5a265-goog In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org> References: <20190930231707.48259-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The ib_srpt driver uses two different identifiers while registering a session with the LIO core. Report both identifiers if the modified pr_debug() statement is enabled. Cc: Honggang LI Cc: Laurence Oberman Signed-off-by: Bart Van Assche --- drivers/infiniband/ulp/srpt/ib_srpt.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c index fbfadeedc195..dabaea301328 100644 --- a/drivers/infiniband/ulp/srpt/ib_srpt.c +++ b/drivers/infiniband/ulp/srpt/ib_srpt.c @@ -2293,7 +2293,8 @@ static int srpt_cm_req_recv(struct srpt_device *const sdev, be64_to_cpu(*(__be64 *)nexus->i_port_id), be64_to_cpu(*(__be64 *)(nexus->i_port_id + 8))); - pr_debug("registering session %s\n", ch->sess_name); + pr_debug("registering src addr %s or i_port_id %s\n", ch->sess_name, + i_port_id); tag_num = ch->rq_size; tag_size = 1; /* ib_srpt does not use se_sess->sess_cmd_map */ From patchwork Mon Sep 30 23:17:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11167839 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BDF5B912 for ; Mon, 30 Sep 2019 23:17:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A73B12168B for ; Mon, 30 Sep 2019 23:17:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731991AbfI3XRg (ORCPT ); Mon, 30 Sep 2019 19:17:36 -0400 Received: from mail-pf1-f194.google.com ([209.85.210.194]:46457 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731852AbfI3XRg (ORCPT ); Mon, 30 Sep 2019 19:17:36 -0400 Received: by mail-pf1-f194.google.com with SMTP id q5so6466457pfg.13 for ; Mon, 30 Sep 2019 16:17:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ov1JeuWLrT5nJIZWkm/784ePsDn9xbIvZ12ZQdk1C00=; b=RdM8ivNfIp5pIIIj22BC517kmp8tSLipKUhoJUEqebiRgxbXjHRXUNAfEFfPJvQpDc wcNcHd8eqqgtHmNoCIzR9kv9TZC12EmsUiSe10tWX/M0GwqKdWbUqV0isycu7GtFmwrz 5hDSrP/UqbB9GL5K9omW+WvJ07vUSQya+2Dqae9dab3cyPqQJrRzRERU3DO3nvIl355h ZdYZxPfSKIbpKrydrr9Bbp7RzPgI+M3aiMMZhmkzigL/lxgpNwzGKS/mJQE+hklCqTKr Y8w7cxkoTnX/myesjGX3sGMpSQYVaMHDKMD0inKxwCjCknGwiYww4RPfAb9twSY55E/k 2GAA== X-Gm-Message-State: APjAAAWuP+SZKhN4FVNXD61vyFkHwHPY5b2iJeTd8iVjOVZfXUnvA23H Ji37i9Tbp8ReIbaws8ECDqk= X-Google-Smtp-Source: APXvYqyNYoCTctAhbu9foUYt3bDNqUSc/dwGTG2jFKTTTO7EJfDZ49syEsXJVGiNlcu7syIg2ntF7g== X-Received: by 2002:a17:90a:b114:: with SMTP id z20mr1926562pjq.113.1569885455103; Mon, 30 Sep 2019 16:17:35 -0700 (PDT) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id l7sm585406pjy.12.2019.09.30.16.17.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2019 16:17:34 -0700 (PDT) From: Bart Van Assche To: Jason Gunthorpe Cc: Leon Romanovsky , Doug Ledford , linux-rdma@vger.kernel.org, Bart Van Assche , Honggang LI , Laurence Oberman Subject: [PATCH 12/15] RDMA/srpt: Rework the approach for closing an RDMA channel Date: Mon, 30 Sep 2019 16:17:04 -0700 Message-Id: <20190930231707.48259-13-bvanassche@acm.org> X-Mailer: git-send-email 2.23.0.444.g18eeb5a265-goog In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org> References: <20190930231707.48259-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Instead of relying on a waitqueue, report when the identity of an RDMA channel can be reused through a completion. Cc: Honggang LI Cc: Laurence Oberman Signed-off-by: Bart Van Assche --- drivers/infiniband/ulp/srpt/ib_srpt.c | 30 +++++++-------------------- drivers/infiniband/ulp/srpt/ib_srpt.h | 3 +++ 2 files changed, 10 insertions(+), 23 deletions(-) diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c index dabaea301328..88e11a250c96 100644 --- a/drivers/infiniband/ulp/srpt/ib_srpt.c +++ b/drivers/infiniband/ulp/srpt/ib_srpt.c @@ -1928,41 +1928,22 @@ static int srpt_disconnect_ch(struct srpt_rdma_ch *ch) return ret; } -static bool srpt_ch_closed(struct srpt_port *sport, struct srpt_rdma_ch *ch) -{ - struct srpt_nexus *nexus; - struct srpt_rdma_ch *ch2; - bool res = true; - - rcu_read_lock(); - list_for_each_entry(nexus, &sport->nexus_list, entry) { - list_for_each_entry(ch2, &nexus->ch_list, list) { - if (ch2 == ch) { - res = false; - goto done; - } - } - } -done: - rcu_read_unlock(); - - return res; -} - /* Send DREQ and wait for DREP. */ static void srpt_disconnect_ch_sync(struct srpt_rdma_ch *ch) { + DECLARE_COMPLETION_ONSTACK(closed); struct srpt_port *sport = ch->sport; pr_debug("ch %s-%d state %d\n", ch->sess_name, ch->qp->qp_num, ch->state); + ch->closed = &closed; + mutex_lock(&sport->mutex); srpt_disconnect_ch(ch); mutex_unlock(&sport->mutex); - while (wait_event_timeout(sport->ch_releaseQ, srpt_ch_closed(sport, ch), - 5 * HZ) == 0) + while (wait_for_completion_timeout(&closed, 5 * HZ) == 0) pr_info("%s(%s-%d state %d): still waiting ...\n", __func__, ch->sess_name, ch->qp->qp_num, ch->state); @@ -2089,6 +2070,9 @@ static void srpt_release_channel_work(struct work_struct *w) list_del_rcu(&ch->list); mutex_unlock(&sport->mutex); + if (ch->closed) + complete(ch->closed); + srpt_destroy_ch_ib(ch); srpt_free_ioctx_ring((struct srpt_ioctx **)ch->ioctx_ring, diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.h b/drivers/infiniband/ulp/srpt/ib_srpt.h index ee9f20e9177a..d0955bc0343d 100644 --- a/drivers/infiniband/ulp/srpt/ib_srpt.h +++ b/drivers/infiniband/ulp/srpt/ib_srpt.h @@ -264,6 +264,8 @@ enum rdma_ch_state { * @zw_cqe: Zero-length write CQE. * @rcu: RCU head. * @kref: kref for this channel. + * @closed: Completion object that will be signaled as soon as a new + * channel object with the same identity can be created. * @rq_size: IB receive queue size. * @max_rsp_size: Maximum size of an RSP response message in bytes. * @sq_wr_avail: number of work requests available in the send queue. @@ -306,6 +308,7 @@ struct srpt_rdma_ch { struct ib_cqe zw_cqe; struct rcu_head rcu; struct kref kref; + struct completion *closed; int rq_size; u32 max_rsp_size; atomic_t sq_wr_avail; From patchwork Mon Sep 30 23:17:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11167841 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A96371599 for ; Mon, 30 Sep 2019 23:17:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 882A920842 for ; Mon, 30 Sep 2019 23:17:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731996AbfI3XRi (ORCPT ); Mon, 30 Sep 2019 19:17:38 -0400 Received: from mail-pg1-f196.google.com ([209.85.215.196]:36669 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731985AbfI3XRh (ORCPT ); Mon, 30 Sep 2019 19:17:37 -0400 Received: by mail-pg1-f196.google.com with SMTP id t14so8253615pgs.3 for ; Mon, 30 Sep 2019 16:17:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oNma/HAel59HGi4qPVyzpJng0X2up6DTL+uZMvwQCEI=; b=IaXXnxUS+/eac+CAUvgvGY+f/uV09JUsqJHWqy3F5oyIEZtdWy5q1fNvMzcvmAKQOc J003IU3L5vvvdhCXYPx2wGkis5cPvPd/LXhYqDGSLCD1o0e4R9WR2p86aewK3coHcHyx /E9TTtBP/xEVHsNVpc/kirR7nlsSBQ4uftUMt8SdYVQgEUlaGi9zdHkLOGrPPyb4vsLH xyi9fjZ8qFo9oisaKb7wVldnft5IMu6X/Nn6PRdvuCLC7YChsOumWJGzTRYJ9v8K60jK pBDVOOQUnD0OeyQKalouLy1L6xjpgIcPD3HUys3ELNwWvBuFHgDSIXv34XTq6NbMIUby lN8g== X-Gm-Message-State: APjAAAVE0Vp2xdCuIWRUyIyCIhKEA1duq8TQdWRZfpXKtUV0eCNVbdx8 xm8ewWn0xytEyo3yhOW2Vlc= X-Google-Smtp-Source: APXvYqxCkUW7nwaBSpMNrTL47GfC+TKgrkfVWwcxrfYk40U6sfM4aj/rm9xC+1QK6r34ik/EVAuf8w== X-Received: by 2002:aa7:91d4:: with SMTP id z20mr18724223pfa.131.1569885456960; Mon, 30 Sep 2019 16:17:36 -0700 (PDT) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id l7sm585406pjy.12.2019.09.30.16.17.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2019 16:17:35 -0700 (PDT) From: Bart Van Assche To: Jason Gunthorpe Cc: Leon Romanovsky , Doug Ledford , linux-rdma@vger.kernel.org, Bart Van Assche , Honggang LI , Laurence Oberman Subject: [PATCH 13/15] RDMA/srpt: Rework the code that waits until an RDMA port is no longer in use Date: Mon, 30 Sep 2019 16:17:05 -0700 Message-Id: <20190930231707.48259-14-bvanassche@acm.org> X-Mailer: git-send-email 2.23.0.444.g18eeb5a265-goog In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org> References: <20190930231707.48259-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The current implementation does not wait until srpt_release_channel() has finished and hence can trigger a use-after-free. Rework srpt_release_sport() such that it waits until srpt_release_channel() has finished. This patch fixes the following KASAN complaint: ================================================================== BUG: KASAN: use-after-free in srpt_free_ioctx.part.23+0x42/0x100 [ib_srpt] Read of size 8 at addr ffff888115c71100 by task kworker/4:3/807 CPU: 4 PID: 807 Comm: kworker/4:3 Not tainted 5.3.0-dbg+ #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 Workqueue: events srpt_release_channel_work [ib_srpt] Call Trace: dump_stack+0x86/0xca print_address_description+0x74/0x32d __kasan_report.cold.6+0x1b/0x36 kasan_report+0x12/0x17 __asan_load8+0x54/0x90 srpt_free_ioctx.part.23+0x42/0x100 [ib_srpt] srpt_free_ioctx_ring.part.24+0x50/0x80 [ib_srpt] srpt_release_channel_work+0x2ad/0x390 [ib_srpt] process_one_work+0x51a/0xa60 worker_thread+0x67/0x5b0 kthread+0x1dc/0x200 ret_from_fork+0x24/0x30 Allocated by task 984: save_stack+0x21/0x90 __kasan_kmalloc.constprop.9+0xc7/0xd0 kasan_kmalloc+0x9/0x10 __kmalloc+0x153/0x370 srpt_add_one+0x4f/0x570 [ib_srpt] add_client_context+0x251/0x290 [ib_core] ib_register_client+0x1da/0x220 [ib_core] iblock_get_alignment_offset_lbas+0x6b/0x100 [target_core_iblock] do_one_initcall+0xcd/0x43a do_init_module+0x103/0x380 load_module+0x3b77/0x3eb0 __do_sys_finit_module+0x12d/0x1b0 __x64_sys_finit_module+0x43/0x50 do_syscall_64+0x6b/0x2d0 entry_SYSCALL_64_after_hwframe+0x49/0xbe Freed by task 1128: save_stack+0x21/0x90 __kasan_slab_free+0x139/0x190 kasan_slab_free+0xe/0x10 slab_free_freelist_hook+0x67/0x1e0 kfree+0xcb/0x2a0 srpt_remove_one+0x569/0x5b0 [ib_srpt] remove_client_context+0x9a/0xe0 [ib_core] disable_device+0x106/0x1b0 [ib_core] __ib_unregister_device+0x5f/0xf0 [ib_core] ib_unregister_device_and_put+0x48/0x60 [ib_core] nldev_dellink+0x120/0x180 [ib_core] rdma_nl_rcv+0x287/0x480 [ib_core] netlink_unicast+0x2cc/0x370 netlink_sendmsg+0x3b1/0x630 __sys_sendto+0x1db/0x290 __x64_sys_sendto+0x80/0xa0 do_syscall_64+0x6b/0x2d0 entry_SYSCALL_64_after_hwframe+0x49/0xbe The buggy address belongs to the object at ffff888115c71100 which belongs to the cache kmalloc-4k of size 4096 The buggy address is located 0 bytes inside of 4096-byte region [ffff888115c71100, ffff888115c72100) The buggy address belongs to the page: page:ffffea0004571c00 refcount:1 mapcount:0 mapping:ffff88811ac0de00 index:0xffff888115c70000 compound_mapcount: 0 flags: 0x2fff000000010200(slab|head) raw: 2fff000000010200 ffffea00045ac408 ffffea0004593208 ffff88811ac0de00 raw: ffff888115c70000 0000000000070002 00000001ffffffff 0000000000000000 page dumped because: kasan: bad access detected Memory state around the buggy address: ffff888115c71000: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc ffff888115c71080: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc >ffff888115c71100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ^ ffff888115c71180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ffff888115c71200: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ================================================================== Cc: Honggang LI Cc: Laurence Oberman Signed-off-by: Bart Van Assche --- drivers/infiniband/ulp/srpt/ib_srpt.c | 42 +++++++++++++-------------- drivers/infiniband/ulp/srpt/ib_srpt.h | 6 ++-- 2 files changed, 25 insertions(+), 23 deletions(-) diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c index 88e11a250c96..5e402f7b9692 100644 --- a/drivers/infiniband/ulp/srpt/ib_srpt.c +++ b/drivers/infiniband/ulp/srpt/ib_srpt.c @@ -2023,10 +2023,17 @@ static void srpt_set_enabled(struct srpt_port *sport, bool enabled) __srpt_close_all_ch(sport); } +static void srpt_drop_sport_ref(struct srpt_port *sport) +{ + if (atomic_dec_return(&sport->refcount) == 0 && sport->freed_channels) + complete(sport->freed_channels); +} + static void srpt_free_ch(struct kref *kref) { struct srpt_rdma_ch *ch = container_of(kref, struct srpt_rdma_ch, kref); + srpt_drop_sport_ref(ch->sport); kfree_rcu(ch, rcu); } @@ -2087,8 +2094,6 @@ static void srpt_release_channel_work(struct work_struct *w) kmem_cache_destroy(ch->req_buf_cache); - wake_up(&sport->ch_releaseQ); - kref_put(&ch->kref, srpt_free_ch); } @@ -2307,6 +2312,12 @@ static int srpt_cm_req_recv(struct srpt_device *const sdev, goto destroy_ib; } + /* + * Once a session has been created destruction of srpt_rdma_ch objects + * will decrement sport->refcount. Hence increment sport->refcount now. + */ + atomic_inc(&sport->refcount); + mutex_lock(&sport->mutex); if ((req->req_flags & SRP_MTCH_ACTION) == SRP_MULTICHAN_SINGLE) { @@ -2889,39 +2900,29 @@ static void srpt_refresh_port_work(struct work_struct *work) srpt_refresh_port(sport); } -static bool srpt_ch_list_empty(struct srpt_port *sport) -{ - struct srpt_nexus *nexus; - bool res = true; - - rcu_read_lock(); - list_for_each_entry(nexus, &sport->nexus_list, entry) - if (!list_empty(&nexus->ch_list)) - res = false; - rcu_read_unlock(); - - return res; -} - /** * srpt_release_sport - disable login and wait for associated channels * @sport: SRPT HCA port. */ static int srpt_release_sport(struct srpt_port *sport) { + DECLARE_COMPLETION_ONSTACK(c); struct srpt_nexus *nexus, *next_n; struct srpt_rdma_ch *ch; WARN_ON_ONCE(irqs_disabled()); + sport->freed_channels = &c; + mutex_lock(&sport->mutex); srpt_set_enabled(sport, false); mutex_unlock(&sport->mutex); - while (wait_event_timeout(sport->ch_releaseQ, - srpt_ch_list_empty(sport), 5 * HZ) <= 0) { - pr_info("%s_%d: waiting for session unregistration ...\n", - dev_name(&sport->sdev->device->dev), sport->port); + while (atomic_read(&sport->refcount) > 0 && + wait_for_completion_timeout(&c, 5 * HZ) <= 0) { + pr_info("%s_%d: waiting for unregistration of %d sessions ...\n", + dev_name(&sport->sdev->device->dev), sport->port, + atomic_read(&sport->refcount)); rcu_read_lock(); list_for_each_entry(nexus, &sport->nexus_list, entry) { list_for_each_entry(ch, &nexus->ch_list, list) { @@ -3130,7 +3131,6 @@ static void srpt_add_one(struct ib_device *device) for (i = 1; i <= sdev->device->phys_port_cnt; i++) { sport = &sdev->port[i - 1]; INIT_LIST_HEAD(&sport->nexus_list); - init_waitqueue_head(&sport->ch_releaseQ); mutex_init(&sport->mutex); sport->sdev = sdev; sport->port = i; diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.h b/drivers/infiniband/ulp/srpt/ib_srpt.h index d0955bc0343d..f3df791dd877 100644 --- a/drivers/infiniband/ulp/srpt/ib_srpt.h +++ b/drivers/infiniband/ulp/srpt/ib_srpt.h @@ -381,7 +381,8 @@ struct srpt_port_attrib { * @port_gid_tpg: TPG associated with target port GID. * @port_gid_wwn: WWN associated with target port GID. * @port_attrib: Port attributes that can be accessed through configfs. - * @ch_releaseQ: Enables waiting for removal from nexus_list. + * @refcount: Number of objects associated with this port. + * @freed_channels: Completion that will be signaled once @refcount becomes 0. * @mutex: Protects nexus_list. * @nexus_list: Nexus list. See also srpt_nexus.entry. */ @@ -401,7 +402,8 @@ struct srpt_port { struct se_portal_group port_gid_tpg; struct se_wwn port_gid_wwn; struct srpt_port_attrib port_attrib; - wait_queue_head_t ch_releaseQ; + atomic_t refcount; + struct completion *freed_channels; struct mutex mutex; struct list_head nexus_list; }; From patchwork Mon Sep 30 23:17:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11167843 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E4E351599 for ; Mon, 30 Sep 2019 23:17:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C408C2168B for ; Mon, 30 Sep 2019 23:17:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732011AbfI3XRj (ORCPT ); Mon, 30 Sep 2019 19:17:39 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:33345 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731985AbfI3XRj (ORCPT ); Mon, 30 Sep 2019 19:17:39 -0400 Received: by mail-pg1-f194.google.com with SMTP id q1so219535pgb.0 for ; Mon, 30 Sep 2019 16:17:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZxzmWX7e/SWmX32qz7KUHQIKpenE+NZIccmezNq7Fu4=; b=Ez5u3cM2k8Lo/6tTO2gOe95ajZqE+SvZW4tZJaM2JC0HiKaiNLiRxHfy60qFb5YR70 Eoz0aGG6Ar9u5XDzBlA4ufpN/YTp1tdGlhB1+xMLdLi26stwCrWtkXkEltiN82qjcc3e PWYvGSVfbssfi4QoE6Wwq6b/X0VXbgpQkOTT1Hmzok7EKs/6VyHRXBEfAk12s3armcMH qIE3hFata7aFWCktP3K7WjEIFSfPjPzQ3TlDWREEphHYL4+MxII+zD7gjSCt3H+QZgwM PcwgH8Ezji6iJBkIOQ1tSFr9G63QNhFUN6GCrGr3B5sVD8/7Dw8NlbDUmSErzWZTEDu4 e6Xw== X-Gm-Message-State: APjAAAVmdhwWP2atmax1+mRKB8/xeUcRa045ib39hIrgUGedKn83HbXW mrDmUdWtCUQAeelpOn1Mo10= X-Google-Smtp-Source: APXvYqwFZetuCGjw2KYGmQMZrCJYeDw4utpXRg6s7VvQ2/R9qxMl44beHoB4huMyDJXl03Xtx2penw== X-Received: by 2002:a17:90a:17c5:: with SMTP id q63mr1958415pja.106.1569885458270; Mon, 30 Sep 2019 16:17:38 -0700 (PDT) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id l7sm585406pjy.12.2019.09.30.16.17.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2019 16:17:37 -0700 (PDT) From: Bart Van Assche To: Jason Gunthorpe Cc: Leon Romanovsky , Doug Ledford , linux-rdma@vger.kernel.org, Bart Van Assche , Honggang LI , Laurence Oberman Subject: [PATCH 14/15] RDMA/srpt: Make the code for handling port identities more systematic Date: Mon, 30 Sep 2019 16:17:06 -0700 Message-Id: <20190930231707.48259-15-bvanassche@acm.org> X-Mailer: git-send-email 2.23.0.444.g18eeb5a265-goog In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org> References: <20190930231707.48259-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Introduce a new data structure for the information about an RDMA port name. This patch does not change any functionality. Cc: Honggang LI Cc: Laurence Oberman Signed-off-by: Bart Van Assche --- drivers/infiniband/ulp/srpt/ib_srpt.c | 64 +++++++++++++++++---------- drivers/infiniband/ulp/srpt/ib_srpt.h | 25 +++++++---- 2 files changed, 57 insertions(+), 32 deletions(-) diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c index 5e402f7b9692..0582b3d4ec4d 100644 --- a/drivers/infiniband/ulp/srpt/ib_srpt.c +++ b/drivers/infiniband/ulp/srpt/ib_srpt.c @@ -567,11 +567,12 @@ static int srpt_refresh_port(struct srpt_port *sport) if (ret) return ret; - sport->port_guid_wwn.priv = sport; - srpt_format_guid(sport->port_guid, sizeof(sport->port_guid), + sport->port_guid_id.wwn.priv = sport; + srpt_format_guid(sport->port_guid_id.name, + sizeof(sport->port_guid_id.name), &sport->gid.global.interface_id); - sport->port_gid_wwn.priv = sport; - snprintf(sport->port_gid, sizeof(sport->port_gid), + sport->port_gid_id.wwn.priv = sport; + snprintf(sport->port_gid_id.name, sizeof(sport->port_gid_id.name), "0x%016llx%016llx", be64_to_cpu(sport->gid.global.subnet_prefix), be64_to_cpu(sport->gid.global.interface_id)); @@ -2287,17 +2288,17 @@ static int srpt_cm_req_recv(struct srpt_device *const sdev, tag_num = ch->rq_size; tag_size = 1; /* ib_srpt does not use se_sess->sess_cmd_map */ - if (sport->port_guid_tpg.se_tpg_wwn) - ch->sess = target_setup_session(&sport->port_guid_tpg, tag_num, + if (sport->port_guid_id.tpg.se_tpg_wwn) + ch->sess = target_setup_session(&sport->port_guid_id.tpg, tag_num, tag_size, TARGET_PROT_NORMAL, ch->sess_name, ch, NULL); - if (sport->port_gid_tpg.se_tpg_wwn && IS_ERR_OR_NULL(ch->sess)) - ch->sess = target_setup_session(&sport->port_gid_tpg, tag_num, + if (sport->port_gid_id.tpg.se_tpg_wwn && IS_ERR_OR_NULL(ch->sess)) + ch->sess = target_setup_session(&sport->port_gid_id.tpg, tag_num, tag_size, TARGET_PROT_NORMAL, i_port_id, ch, NULL); /* Retry without leading "0x" */ - if (sport->port_gid_tpg.se_tpg_wwn && IS_ERR_OR_NULL(ch->sess)) - ch->sess = target_setup_session(&sport->port_gid_tpg, tag_num, + if (sport->port_gid_id.tpg.se_tpg_wwn && IS_ERR_OR_NULL(ch->sess)) + ch->sess = target_setup_session(&sport->port_gid_id.tpg, tag_num, tag_size, TARGET_PROT_NORMAL, i_port_id + 2, ch, NULL); if (IS_ERR_OR_NULL(ch->sess)) { @@ -2959,10 +2960,10 @@ static struct se_wwn *__srpt_lookup_wwn(const char *name) for (i = 0; i < dev->phys_port_cnt; i++) { sport = &sdev->port[i]; - if (strcmp(sport->port_guid, name) == 0) - return &sport->port_guid_wwn; - if (strcmp(sport->port_gid, name) == 0) - return &sport->port_gid_wwn; + if (strcmp(sport->port_guid_id.name, name) == 0) + return &sport->port_guid_id.wwn; + if (strcmp(sport->port_gid_id.name, name) == 0) + return &sport->port_gid_id.wwn; } } @@ -3241,14 +3242,33 @@ static struct srpt_port *srpt_tpg_to_sport(struct se_portal_group *tpg) return tpg->se_tpg_wwn->priv; } -static char *srpt_get_fabric_wwn(struct se_portal_group *tpg) +static struct srpt_port_id *srpt_tpg_to_sport_id(struct se_portal_group *tpg) { struct srpt_port *sport = srpt_tpg_to_sport(tpg); - WARN_ON_ONCE(tpg != &sport->port_guid_tpg && - tpg != &sport->port_gid_tpg); - return tpg == &sport->port_guid_tpg ? sport->port_guid : - sport->port_gid; + if (tpg == &sport->port_guid_id.tpg) + return &sport->port_guid_id; + if (tpg == &sport->port_gid_id.tpg) + return &sport->port_gid_id; + WARN_ON_ONCE(true); + return NULL; +} + +static struct srpt_port_id *srpt_wwn_to_sport_id(struct se_wwn *wwn) +{ + struct srpt_port *sport = wwn->priv; + + if (wwn == &sport->port_guid_id.wwn) + return &sport->port_guid_id; + if (wwn == &sport->port_gid_id.wwn) + return &sport->port_gid_id; + WARN_ON_ONCE(true); + return NULL; +} + +static char *srpt_get_fabric_wwn(struct se_portal_group *tpg) +{ + return srpt_tpg_to_sport_id(tpg)->name; } static u16 srpt_get_tag(struct se_portal_group *tpg) @@ -3705,13 +3725,9 @@ static struct se_portal_group *srpt_make_tpg(struct se_wwn *wwn, const char *name) { struct srpt_port *sport = wwn->priv; - struct se_portal_group *tpg; + struct se_portal_group *tpg = &srpt_wwn_to_sport_id(wwn)->tpg; int res; - WARN_ON_ONCE(wwn != &sport->port_guid_wwn && - wwn != &sport->port_gid_wwn); - tpg = wwn == &sport->port_guid_wwn ? &sport->port_guid_tpg : - &sport->port_gid_tpg; res = core_tpg_register(wwn, tpg, SCSI_PROTOCOL_SRP); if (res) return ERR_PTR(res); diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.h b/drivers/infiniband/ulp/srpt/ib_srpt.h index f3df791dd877..f8bd95302ac0 100644 --- a/drivers/infiniband/ulp/srpt/ib_srpt.h +++ b/drivers/infiniband/ulp/srpt/ib_srpt.h @@ -363,13 +363,26 @@ struct srpt_port_attrib { bool use_srq; }; +/** + * struct srpt_port_id - information about an RDMA port name + * @tpg: TPG associated with the RDMA port. + * @wwn: WWN associated with the RDMA port. + * @name: ASCII representation of the port name. + * + * Multiple sysfs directories can be associated with a single RDMA port. This + * data structure represents a single (port, name) pair. + */ +struct srpt_port_id { + struct se_portal_group tpg; + struct se_wwn wwn; + char name[64]; +}; + /** * struct srpt_port - information associated by SRPT with a single IB port * @sdev: backpointer to the HCA information. * @mad_agent: per-port management datagram processing information. * @enabled: Whether or not this target port is enabled. - * @port_guid: ASCII representation of Port GUID - * @port_gid: ASCII representation of Port GID * @port: one-based port number. * @sm_lid: cached value of the port's sm_lid. * @lid: cached value of the port's lid. @@ -390,17 +403,13 @@ struct srpt_port { struct srpt_device *sdev; struct ib_mad_agent *mad_agent; bool enabled; - u8 port_guid[24]; - u8 port_gid[64]; u8 port; u32 sm_lid; u32 lid; union ib_gid gid; struct work_struct work; - struct se_portal_group port_guid_tpg; - struct se_wwn port_guid_wwn; - struct se_portal_group port_gid_tpg; - struct se_wwn port_gid_wwn; + struct srpt_port_id port_guid_id; + struct srpt_port_id port_gid_id; struct srpt_port_attrib port_attrib; atomic_t refcount; struct completion *freed_channels; From patchwork Mon Sep 30 23:17:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 11167845 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 727BF912 for ; Mon, 30 Sep 2019 23:17:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5ADF32168B for ; Mon, 30 Sep 2019 23:17:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732018AbfI3XRk (ORCPT ); Mon, 30 Sep 2019 19:17:40 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:41727 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731985AbfI3XRk (ORCPT ); Mon, 30 Sep 2019 19:17:40 -0400 Received: by mail-pf1-f196.google.com with SMTP id q7so6483017pfh.8 for ; Mon, 30 Sep 2019 16:17:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bUSAgCJXEgMrQIQdTaSDbcphVPmwFXISsrnMoNw0POQ=; b=l2KvaFBMtoLA83sVwToJBNcbs9SGi6mi2FYmApeco4GVafU6xK/kdr0BGb4S6XqOQG pwYOL94lpWevIvKepIdOhqhywPhgPibI7CDM+emH3lLX8oRhAGhdoKpzbYJ8KNmYCoTZ VZG5rrHfzoLrKcFMWy6WEKlkllgPm+y13enRdkqiTCmp53I2kiKFKqXyGZ+92jmkhcla Dvta0harqm8MUPR7nJexUOV3o+DWWeZNRf1gcHATDOgk8kqlxvWugFu+ZJ+m+t/NyN/a qSgw374IU+lva1bbQrKYHU+2RdjbDiWXKoAQMk4UoQMKbFADn2kFwKb5ChY29ZW4Zjpa R5vQ== X-Gm-Message-State: APjAAAUxPznkIm5Y9l2d93G2KsPy+4bOqe7iGCNJmTsHmXoGGxZsdcz/ YcjZ9906XMdNMTV3Yhh48og= X-Google-Smtp-Source: APXvYqwDBL/13dFIL8m7i0Hr/g/u8erz1zeFUGl8h78AYw+2rwdQekto+D6tZlW8RWd4hVUUZ4PXIQ== X-Received: by 2002:a63:1c22:: with SMTP id c34mr26254605pgc.435.1569885459527; Mon, 30 Sep 2019 16:17:39 -0700 (PDT) Received: from desktop-bart.svl.corp.google.com ([2620:15c:2cd:202:4308:52a3:24b6:2c60]) by smtp.gmail.com with ESMTPSA id l7sm585406pjy.12.2019.09.30.16.17.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2019 16:17:38 -0700 (PDT) From: Bart Van Assche To: Jason Gunthorpe Cc: Leon Romanovsky , Doug Ledford , linux-rdma@vger.kernel.org, Bart Van Assche Subject: [PATCH 15/15] RDMA/srpt: Postpone HCA removal until after configfs directory removal Date: Mon, 30 Sep 2019 16:17:07 -0700 Message-Id: <20190930231707.48259-16-bvanassche@acm.org> X-Mailer: git-send-email 2.23.0.444.g18eeb5a265-goog In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org> References: <20190930231707.48259-1-bvanassche@acm.org> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org A shortcoming of the SCSI target core is that it does not have an API for removing tpg or wwn objects. Wait until these directories have been removed before allowing HCA removal to finish. See also Bart Van Assche, "Re: Why using configfs as the only interface is wrong for a storage target", 2011-02-07 (https://www.spinics.net/lists/linux-scsi/msg50248.html). This patch fixes the following kernel crash: ================================================================== BUG: KASAN: use-after-free in __configfs_open_file.isra.4+0x1a8/0x400 Read of size 8 at addr ffff88811880b690 by task restart-lio-srp/1215 CPU: 1 PID: 1215 Comm: restart-lio-srp Not tainted 5.3.0-dbg+ #3 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 Call Trace: dump_stack+0x86/0xca print_address_description+0x74/0x32d __kasan_report.cold.6+0x1b/0x36 kasan_report+0x12/0x17 __asan_load8+0x54/0x90 __configfs_open_file.isra.4+0x1a8/0x400 configfs_open_file+0x13/0x20 do_dentry_open+0x2b1/0x770 vfs_open+0x58/0x60 path_openat+0x5fa/0x14b0 do_filp_open+0x115/0x180 do_sys_open+0x1d4/0x2a0 __x64_sys_openat+0x59/0x70 do_syscall_64+0x6b/0x2d0 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x7f2f2bd3fcce Code: 25 00 00 41 00 3d 00 00 41 00 74 48 48 8d 05 19 d7 0d 00 8b 00 85 c0 75 69 89 f2 b8 01 01 00 00 48 89 fe bf 9c ff ff ff 0f 05 <48> 3d 00 f0 ff ff 0f 87 a6 00 00 00 48 8b 4c 24 28 64 48 33 0c 25 RSP: 002b:00007ffd155f7850 EFLAGS: 00000246 ORIG_RAX: 0000000000000101 RAX: ffffffffffffffda RBX: 0000564609ba88e0 RCX: 00007f2f2bd3fcce RDX: 0000000000000241 RSI: 0000564609ba8cf0 RDI: 00000000ffffff9c RBP: 00007ffd155f7950 R08: 0000000000000000 R09: 0000000000000020 R10: 00000000000001b6 R11: 0000000000000246 R12: 0000000000000000 R13: 0000000000000003 R14: 0000000000000001 R15: 0000564609ba8cf0 Allocated by task 995: save_stack+0x21/0x90 __kasan_kmalloc.constprop.9+0xc7/0xd0 kasan_kmalloc+0x9/0x10 __kmalloc+0x153/0x370 srpt_add_one+0x4f/0x561 [ib_srpt] add_client_context+0x251/0x290 [ib_core] ib_register_client+0x1da/0x220 [ib_core] iblock_get_alignment_offset_lbas+0x6b/0x100 [target_core_iblock] do_one_initcall+0xcd/0x43a do_init_module+0x103/0x380 load_module+0x3b77/0x3eb0 __do_sys_finit_module+0x12d/0x1b0 __x64_sys_finit_module+0x43/0x50 do_syscall_64+0x6b/0x2d0 entry_SYSCALL_64_after_hwframe+0x49/0xbe Freed by task 1221: save_stack+0x21/0x90 __kasan_slab_free+0x139/0x190 kasan_slab_free+0xe/0x10 slab_free_freelist_hook+0x67/0x1e0 kfree+0xcb/0x2a0 srpt_remove_one+0x596/0x670 [ib_srpt] remove_client_context+0x9a/0xe0 [ib_core] disable_device+0x106/0x1b0 [ib_core] __ib_unregister_device+0x5f/0xf0 [ib_core] ib_unregister_driver+0x11a/0x170 [ib_core] 0xffffffffa087f666 __x64_sys_delete_module+0x1f8/0x2c0 do_syscall_64+0x6b/0x2d0 entry_SYSCALL_64_after_hwframe+0x49/0xbe The buggy address belongs to the object at ffff88811880b300 which belongs to the cache kmalloc-4k of size 4096 The buggy address is located 912 bytes inside of 4096-byte region [ffff88811880b300, ffff88811880c300) The buggy address belongs to the page: page:ffffea0004620200 refcount:1 mapcount:0 mapping:ffff88811ac0de00 index:0x0 compound_mapcount: 0 flags: 0x2fff000000010200(slab|head) raw: 2fff000000010200 dead000000000100 dead000000000122 ffff88811ac0de00 raw: 0000000000000000 0000000000070007 00000001ffffffff 0000000000000000 page dumped because: kasan: bad access detected Memory state around the buggy address: ffff88811880b580: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ffff88811880b600: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >ffff88811880b680: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ^ ffff88811880b700: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ffff88811880b780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ================================================================== Signed-off-by: Bart Van Assche --- drivers/infiniband/ulp/srpt/ib_srpt.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c index 0582b3d4ec4d..daf811abf40a 100644 --- a/drivers/infiniband/ulp/srpt/ib_srpt.c +++ b/drivers/infiniband/ulp/srpt/ib_srpt.c @@ -2921,7 +2921,7 @@ static int srpt_release_sport(struct srpt_port *sport) while (atomic_read(&sport->refcount) > 0 && wait_for_completion_timeout(&c, 5 * HZ) <= 0) { - pr_info("%s_%d: waiting for unregistration of %d sessions ...\n", + pr_info("%s_%d: waiting for unregistration of %d sessions and configfs directories ...\n", dev_name(&sport->sdev->device->dev), sport->port, atomic_read(&sport->refcount)); rcu_read_lock(); @@ -3732,6 +3732,8 @@ static struct se_portal_group *srpt_make_tpg(struct se_wwn *wwn, if (res) return ERR_PTR(res); + atomic_inc(&sport->refcount); + return tpg; } @@ -3745,6 +3747,7 @@ static void srpt_drop_tpg(struct se_portal_group *tpg) sport->enabled = false; core_tpg_deregister(tpg); + srpt_drop_sport_ref(sport); } /**