From patchwork Thu Jun 24 03:28:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olga Kornievskaia X-Patchwork-Id: 12340727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79FA3C49EA6 for ; Thu, 24 Jun 2021 03:28:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5D3AE613BA for ; Thu, 24 Jun 2021 03:28:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229822AbhFXDbQ (ORCPT ); Wed, 23 Jun 2021 23:31:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230002AbhFXDbQ (ORCPT ); Wed, 23 Jun 2021 23:31:16 -0400 Received: from mail-il1-x134.google.com (mail-il1-x134.google.com [IPv6:2607:f8b0:4864:20::134]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 991A6C061756 for ; Wed, 23 Jun 2021 20:28:57 -0700 (PDT) Received: by mail-il1-x134.google.com with SMTP id i17so4674053ilj.11 for ; Wed, 23 Jun 2021 20:28:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ee336bxKIuOlvF/K5YoW63n+TrKhakpxaOkSHP+m6V8=; b=FonW707Y1W5GpBPoA1S3MPdDGZSccKChkes8/XwIOhVF+nvoHm2BBGL7pgjZaHZsd5 yDk2vfcFpQg4jv7WlBh/4/qMKilZGQBFMYVzmMLTONM6DD79uTAWAWxslE26sBLW0NxH k9fLOVi2WMUCO/CzxBirhe9XfelaaZqpAmsQ6xi4BfHe0uLwQT+F9PGt2/Z8A1Ci9qaF 3zzBjIQ2/BcHos/uLeE8oVvUP+ilXyY1HBJBZkl8T60MLs+RpD9kYw8Un2AanlyR57uf Fhht8RmF5PYYlCRZb/k1hOaDZDhuMujpNIEzLcIhke9fB8gns085wQ9b9dN3bXiwyF7T GGSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ee336bxKIuOlvF/K5YoW63n+TrKhakpxaOkSHP+m6V8=; b=EDi4MFtOCF1hEwpx3mDjDZEC0UTk4mYJjxOXedHUfULTn6EXrO1i3YANKDm+9Gmf6J /AeQL/grx6oZ0XLu5q//D6sZPNA0q2nE82jJxVJU03GJBelyLm6phBNQSV0b7K166Ano nHEV1RM0fgmyguIsOyMgnSP4WTu9m3CEUCxwr8ieu0EZIi8Ir0VHshsy9b4aBbzGPk8v rS0lzRRWCKqZ+8yjOF+fnO/OU8H2R58/DrKuJuXqbWZ7kfyTTCJAMEse746r3jcdXOcj XSHWPRKW8BaGKeaYLTC2Zz4Cix7H/I5We/oH36Kez1S8zKrKBQQhl+LSJhpQF8hrbhVf dk0w== X-Gm-Message-State: AOAM530FThLdhg05RO+YoDqRACr3jXWVm4PSNSp9X/UvM7HiWft6jG3N kRVmCTbBSQyfdgRa+b1Ec2h1VaJ1noc4+w== X-Google-Smtp-Source: ABdhPJz42lrXOqyBFyc1hU0uTETF4BeT/HivI0e37hsHcH3SapA5Tj9uzt2iRyD+gztFQRxbI46rQg== X-Received: by 2002:a05:6e02:13ad:: with SMTP id h13mr2032009ilo.128.1624505337089; Wed, 23 Jun 2021 20:28:57 -0700 (PDT) Received: from kolga-mac-1.attlocal.net ([2600:1700:6a10:2e90:fd18:15dc:e0e4:e39e]) by smtp.gmail.com with ESMTPSA id g4sm1026780ilk.37.2021.06.23.20.28.56 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jun 2021 20:28:56 -0700 (PDT) From: Olga Kornievskaia To: trond.myklebust@hammerspace.com, anna.schumaker@netapp.com Cc: linux-nfs@vger.kernel.org Subject: [PATCH v2 1/8] SUNRPC mark the first transport Date: Wed, 23 Jun 2021 23:28:46 -0400 Message-Id: <20210624032853.4776-2-olga.kornievskaia@gmail.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20210624032853.4776-1-olga.kornievskaia@gmail.com> References: <20210624032853.4776-1-olga.kornievskaia@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Olga Kornievskaia When an RPC client gets created it's first transport is special and should be marked a main transport. Signed-off-by: Olga Kornievskaia --- include/linux/sunrpc/xprt.h | 1 + net/sunrpc/clnt.c | 1 + 2 files changed, 2 insertions(+) diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index 13a4eaf385cf..692e5946c029 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -293,6 +293,7 @@ struct rpc_xprt { struct rcu_head rcu; const struct xprt_class *xprt_class; struct rpc_sysfs_xprt *xprt_sysfs; + bool main; /*mark if this is the 1st transport */ }; #if defined(CONFIG_SUNRPC_BACKCHANNEL) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 4d8fd9d9c264..7ca946567e13 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -412,6 +412,7 @@ static struct rpc_clnt * rpc_new_client(const struct rpc_create_args *args, } rpc_clnt_set_transport(clnt, xprt, timeout); + xprt->main = true; xprt_iter_init(&clnt->cl_xpi, xps); xprt_switch_put(xps); From patchwork Thu Jun 24 03:28:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olga Kornievskaia X-Patchwork-Id: 12340731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F7C0C49EA7 for ; Thu, 24 Jun 2021 03:29:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E948A6137D for ; Thu, 24 Jun 2021 03:28:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230006AbhFXDbR (ORCPT ); Wed, 23 Jun 2021 23:31:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36176 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230002AbhFXDbR (ORCPT ); Wed, 23 Jun 2021 23:31:17 -0400 Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC62BC061574 for ; Wed, 23 Jun 2021 20:28:58 -0700 (PDT) Received: by mail-io1-xd2f.google.com with SMTP id k11so6156489ioa.5 for ; Wed, 23 Jun 2021 20:28:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Oa9Vgi1+mE9cwK7RVRVacr6n6EHbCKrO9HpKfsQaeJ8=; b=klurHPo09Nq6BBVWMy6MogC7x9jKuQwSaxvOovVosG/E4oA20+kGtRf7I2G1BetZvx tEf7O6KhyYTb3f9RkLLSHN0J0fgqdn+Ci4BVP/DqIMJGag0EHcx29/b8S0OESgg6PBvw gpMXn0L5zsDbLN2i3pGcymHTP+EFeEGihXFgNY+XVaT/z9V4r3owCHOm8/SiCBL8L0rq CDN0ckYzHVcnWVCSsL0ASTpyrJeK38keflsepEbVdY1paYq4cAIfhG5ak/9gYvIkJKN4 gxhz8h7bDVwUdlcejaOuPsiWPVU3fFWChf9+V26s5HvodO3I9n3SH7seH5jIRH3JyY/B HZKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Oa9Vgi1+mE9cwK7RVRVacr6n6EHbCKrO9HpKfsQaeJ8=; b=i16La65suS/A5C5GFwA8A1tijsXl/NSuHPWG3VHYsmDnP4TLWrYVwpWwDHX1zdhUMF HfpiGePQTECwgt7bi8gDD+Z/7lWQ32B7dkI4Ux1xi3rrdlT8zPFFw8eAabRLFfiiTgu2 8rpI39sh9lJQsAWcax9N0a97FVWPnLMkYl1wwWssBzS+IRlLEkDZd6gvQL8A99gn2UiD VqsL8b1d/Op40/4iBbZZBHlab6Ud8+ch4kU45C81MIvkUdBfqvQEWRF+8Xq1zPMhe09H u2Jodm5dJj4oUqdYbJkzw2+9tPPxUnC+gXfp6cbEaKIeg739La4cflGXAim/ilIj1s6m m94w== X-Gm-Message-State: AOAM532vrksc5irOBErq1KEQtG7QOj2omGH1R+74L2l0syu+W5XUH50B NePUCJ2pzcE8v7E1EDEkQYSConU5SnUPnA== X-Google-Smtp-Source: ABdhPJweLdCk/0LcgQCF0NAD3PxfliygYkl01VGmlvXjDwevTopf6b8kN0nsCA1IeNIKnJKyNvGntw== X-Received: by 2002:a6b:7d05:: with SMTP id c5mr2296776ioq.148.1624505338152; Wed, 23 Jun 2021 20:28:58 -0700 (PDT) Received: from kolga-mac-1.attlocal.net ([2600:1700:6a10:2e90:fd18:15dc:e0e4:e39e]) by smtp.gmail.com with ESMTPSA id g4sm1026780ilk.37.2021.06.23.20.28.57 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jun 2021 20:28:57 -0700 (PDT) From: Olga Kornievskaia To: trond.myklebust@hammerspace.com, anna.schumaker@netapp.com Cc: linux-nfs@vger.kernel.org Subject: [PATCH v2 2/8] SUNRPC display xprt's main value in sysfs's xprt_info Date: Wed, 23 Jun 2021 23:28:47 -0400 Message-Id: <20210624032853.4776-3-olga.kornievskaia@gmail.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20210624032853.4776-1-olga.kornievskaia@gmail.com> References: <20210624032853.4776-1-olga.kornievskaia@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Olga Kornievskaia Display in sysfs in the information about the xprt if this is a main transport or not. Signed-off-by: Olga Kornievskaia --- net/sunrpc/sysfs.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c index 45c0cf18360d..6ef5469fe998 100644 --- a/net/sunrpc/sysfs.c +++ b/net/sunrpc/sysfs.c @@ -103,10 +103,10 @@ static ssize_t rpc_sysfs_xprt_info_show(struct kobject *kobj, ret = sprintf(buf, "last_used=%lu\ncur_cong=%lu\ncong_win=%lu\n" "max_num_slots=%u\nmin_num_slots=%u\nnum_reqs=%u\n" "binding_q_len=%u\nsending_q_len=%u\npending_q_len=%u\n" - "backlog_q_len=%u\n", xprt->last_used, xprt->cong, - xprt->cwnd, xprt->max_reqs, xprt->min_reqs, + "backlog_q_len=%u\nmain_xprt=%d\n", xprt->last_used, + xprt->cong, xprt->cwnd, xprt->max_reqs, xprt->min_reqs, xprt->num_reqs, xprt->binding.qlen, xprt->sending.qlen, - xprt->pending.qlen, xprt->backlog.qlen); + xprt->pending.qlen, xprt->backlog.qlen, xprt->main); xprt_put(xprt); return ret + 1; } From patchwork Thu Jun 24 03:28:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olga Kornievskaia X-Patchwork-Id: 12340733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAA0DC49EA5 for ; Thu, 24 Jun 2021 03:29:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B1F44610C7 for ; Thu, 24 Jun 2021 03:29:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229959AbhFXDbS (ORCPT ); Wed, 23 Jun 2021 23:31:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36180 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230002AbhFXDbS (ORCPT ); Wed, 23 Jun 2021 23:31:18 -0400 Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF6B0C061574 for ; Wed, 23 Jun 2021 20:28:59 -0700 (PDT) Received: by mail-io1-xd29.google.com with SMTP id o5so6171721iob.4 for ; Wed, 23 Jun 2021 20:28:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8GxraR82L9fNcxFEXCeukO6xiXhWsxYlbSPpiRziGFo=; b=JYFTRdX26WR0i1iPAHtCjl3Xa7ozCi8GoGIGTouI5e4NJfbVF12hjbHA/l/z4N9x3P MhBSIOEQGqvIXIAzCptHrxY1isNSwWyBl1eLKCNdZvSaw4we73N26VgqFk/kSC68qPFb sKwDnbF4b0mxwzERIOccEo9YMKevdG0wkbnS30DE07r+AAYDopGe4/AUzVYYXT33kduz XHaMWyueUWCu1p5/h9e9vQ3j+U82OlYepBKGBy82Jhpa6ogQbSsjLseY8JOEMiOwnoj2 mnysZEGo9FG+24ZGG8xS5U/4hybUup3QHA+mHICwUVtSWgHM060j/lT7udWV811Q7rj8 zqng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8GxraR82L9fNcxFEXCeukO6xiXhWsxYlbSPpiRziGFo=; b=KcV/2J2SujDk2eSmZF9dokrnpC0Cn1ywUXotcUOCl9XH6Hmot41RT9ZxKTTKuhL8Uy wMRLSF+sK2yJhgIfVVvf7aPlQ8G2uUXwK8kbypNv18uwuQfwdVNtbqvZiKotnw3o3Sfg hwUFUZqDHwDWMI40MVDJ4SIbt57eDYT59WveSwOiYOa4vuTqbF9sY4BOmFl/+UxV/Nd/ n+TuTAotSsrqnlE/e42xgEmDXkAiCVk2byZpqQlSFzHlEmzjZ9+mZT/Z6Gcq0Vf7JA3S GmuT4AoCu5zLyHYHOQRGtdC0aJTaH2TItHllKB1zjJtlHAWC/IFKX9KDguwPVQDvCw/r i82Q== X-Gm-Message-State: AOAM530V21iy5pY2UHOIkMNkoDltBw31gdQx78sasO1aX6iczREwo6jh O1jl9OhmBWnetGCTZ3FX2FU= X-Google-Smtp-Source: ABdhPJxTlh9gMzOtzeFIRH8vr036KCBguCdbOnUrdO2j2GfV/IG9/XOpR/8Z4p9aH2ebtM0DCJUwHA== X-Received: by 2002:a5e:8513:: with SMTP id i19mr2268167ioj.50.1624505339229; Wed, 23 Jun 2021 20:28:59 -0700 (PDT) Received: from kolga-mac-1.attlocal.net ([2600:1700:6a10:2e90:fd18:15dc:e0e4:e39e]) by smtp.gmail.com with ESMTPSA id g4sm1026780ilk.37.2021.06.23.20.28.58 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jun 2021 20:28:58 -0700 (PDT) From: Olga Kornievskaia To: trond.myklebust@hammerspace.com, anna.schumaker@netapp.com Cc: linux-nfs@vger.kernel.org Subject: [PATCH v2 3/8] SUNRPC query transport's source port Date: Wed, 23 Jun 2021 23:28:48 -0400 Message-Id: <20210624032853.4776-4-olga.kornievskaia@gmail.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20210624032853.4776-1-olga.kornievskaia@gmail.com> References: <20210624032853.4776-1-olga.kornievskaia@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Olga Kornievskaia Provide ability to query transport's source port. Signed-off-by: Olga Kornievskaia --- include/linux/sunrpc/xprtsock.h | 1 + net/sunrpc/xprtsock.c | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/include/linux/sunrpc/xprtsock.h b/include/linux/sunrpc/xprtsock.h index 3c1423ee74b4..8c2a712cb242 100644 --- a/include/linux/sunrpc/xprtsock.h +++ b/include/linux/sunrpc/xprtsock.h @@ -10,6 +10,7 @@ int init_socket_xprt(void); void cleanup_socket_xprt(void); +unsigned short get_srcport(struct rpc_xprt *); #define RPC_MIN_RESVPORT (1U) #define RPC_MAX_RESVPORT (65535U) diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index 2ad4d0df45fe..4611845ec1eb 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -1653,6 +1653,13 @@ static int xs_get_srcport(struct sock_xprt *transport) return port; } +unsigned short get_srcport(struct rpc_xprt *xprt) +{ + struct sock_xprt *sock = container_of(xprt, struct sock_xprt, xprt); + return sock->srcport; +} +EXPORT_SYMBOL(get_srcport); + static unsigned short xs_next_srcport(struct sock_xprt *transport, unsigned short port) { if (transport->srcport != 0) From patchwork Thu Jun 24 03:28:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olga Kornievskaia X-Patchwork-Id: 12340735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02F05C48BC2 for ; Thu, 24 Jun 2021 03:29:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E2893610C7 for ; Thu, 24 Jun 2021 03:29:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230013AbhFXDbU (ORCPT ); Wed, 23 Jun 2021 23:31:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36188 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230010AbhFXDbU (ORCPT ); Wed, 23 Jun 2021 23:31:20 -0400 Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C380CC061574 for ; Wed, 23 Jun 2021 20:29:00 -0700 (PDT) Received: by mail-io1-xd30.google.com with SMTP id k16so6100530ios.10 for ; Wed, 23 Jun 2021 20:29:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=X1fkEW2LhHY59nCgB5W4dhaAN6w2ef4pK1UTWYJd0tg=; b=tltkt7fqSPU8930cZG2fSePLQK0/RxAzmLjFCqj/wIJu32lhwqkaoR34UqgsNxCDW7 Ph9ljVksi0dUUClTkqUhB0R0+dtS3E4YYxawKyfuPKBK5+hkXNDaEPfHWYhgEdki3Ewk QfMUKCFAe7cue8K47pxB60JF9ywGf8FTsArZjTLECbT75v8GDguGr5vkoNs1vBM561dp 2NXZfDOAaYq6+u2LPNTvGEYcEk497Zip0juByUgPpt6x4fzu0oiPqnMTGF29kGIgLm8v 7WpzQxC5jSzFFFb01tW/z5F+i5JoBnOkmUSLRiaP2Ocjn26syBOuaJuM/4j0KaqXblpE yIcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=X1fkEW2LhHY59nCgB5W4dhaAN6w2ef4pK1UTWYJd0tg=; b=lD4tqafrgm7XxmkJBty7EJSycJ7inGgIsx7moCAnxmTdB0QakoHC5ZGruc2tjGX+LW 3cbfSPEjlge7VnOL8TBajLjIQmeHtDJ7sZrU58yRS2LkKBbGNm1f6Hnw3H7hrYghG/+7 Pp2RN5k62sITxHOFXEm63N1N36so4Z9pY2bJqH1H2ZtQhg7XYv6AEmRJ2iNe0oIM2o9J C4fVqTWmuLf+xr48fKsiUjyGQ18k3pCLZAa+HQyYyF+RaL7CXcld0S3xSfYDKQBzTWLd ZxeSS/4R7t0H09gkpSIjel9GiY5IzhjU3RMFqrTzdKBiUQUpcmSVS2HTkzDJcHKvh8nD /OwA== X-Gm-Message-State: AOAM530+VzoefWYa7z6qBx0bmk+5f0D6UJ0iqOZHqBkZJL00F2lrnlGs ddgdCJaENruldLEoeVN9fl4= X-Google-Smtp-Source: ABdhPJw3LrfWpiWRIzeYIMBpJODcBnIb+aZLJXp3ruE1ywe44sJyTR6/hXJt80IGQX7c27Al1ujExg== X-Received: by 2002:a6b:292:: with SMTP id 140mr2319583ioc.11.1624505340254; Wed, 23 Jun 2021 20:29:00 -0700 (PDT) Received: from kolga-mac-1.attlocal.net ([2600:1700:6a10:2e90:fd18:15dc:e0e4:e39e]) by smtp.gmail.com with ESMTPSA id g4sm1026780ilk.37.2021.06.23.20.28.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jun 2021 20:28:59 -0700 (PDT) From: Olga Kornievskaia To: trond.myklebust@hammerspace.com, anna.schumaker@netapp.com Cc: linux-nfs@vger.kernel.org Subject: [PATCH v2 4/8] SUNRPC for TCP display xprt's source port in sysfs xprt_info Date: Wed, 23 Jun 2021 23:28:49 -0400 Message-Id: <20210624032853.4776-5-olga.kornievskaia@gmail.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20210624032853.4776-1-olga.kornievskaia@gmail.com> References: <20210624032853.4776-1-olga.kornievskaia@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Olga Kornievskaia Using TCP connection's source port it is useful to match connections seen on the network traces to the xprts used by the linux nfs client. Signed-off-by: Olga Kornievskaia --- net/sunrpc/sysfs.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c index 6ef5469fe998..816f543d4237 100644 --- a/net/sunrpc/sysfs.c +++ b/net/sunrpc/sysfs.c @@ -5,6 +5,7 @@ #include #include #include +#include #include "sysfs.h" @@ -103,10 +104,13 @@ static ssize_t rpc_sysfs_xprt_info_show(struct kobject *kobj, ret = sprintf(buf, "last_used=%lu\ncur_cong=%lu\ncong_win=%lu\n" "max_num_slots=%u\nmin_num_slots=%u\nnum_reqs=%u\n" "binding_q_len=%u\nsending_q_len=%u\npending_q_len=%u\n" - "backlog_q_len=%u\nmain_xprt=%d\n", xprt->last_used, - xprt->cong, xprt->cwnd, xprt->max_reqs, xprt->min_reqs, - xprt->num_reqs, xprt->binding.qlen, xprt->sending.qlen, - xprt->pending.qlen, xprt->backlog.qlen, xprt->main); + "backlog_q_len=%u\nmain_xprt=%d\nsrc_port=%u\n", + xprt->last_used, xprt->cong, xprt->cwnd, xprt->max_reqs, + xprt->min_reqs, xprt->num_reqs, xprt->binding.qlen, + xprt->sending.qlen, xprt->pending.qlen, + xprt->backlog.qlen, xprt->main, + (xprt->xprt_class->ident == XPRT_TRANSPORT_TCP) ? + get_srcport(xprt) : 0); xprt_put(xprt); return ret + 1; } From patchwork Thu Jun 24 03:28:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olga Kornievskaia X-Patchwork-Id: 12340737 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82467C49EA7 for ; Thu, 24 Jun 2021 03:29:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6CD026137D for ; Thu, 24 Jun 2021 03:29:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230010AbhFXDbU (ORCPT ); Wed, 23 Jun 2021 23:31:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36192 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230002AbhFXDbU (ORCPT ); Wed, 23 Jun 2021 23:31:20 -0400 Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB6E1C061756 for ; Wed, 23 Jun 2021 20:29:01 -0700 (PDT) Received: by mail-io1-xd33.google.com with SMTP id a6so6209460ioe.0 for ; Wed, 23 Jun 2021 20:29:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DaXOH9viw4AlOcNpHDVEcNVeb68SygmlHfuqV9aPO9I=; b=cohvp/bUsdBeM4aNzTbLJ9lFg1z+RNEFgzMUx1BMm7+xmg9OAVqXbv5g66rdrVnBrG 1k4SusJN1Xq1Oish6Jj0vcoBnO0G9RyO0ofby3tqcOz/pnZFCjDhhrpY1TLDRHsdTY/X bADJs17R+8v+K8fyOgGJ2m2hUl9/WN5iOd7WzFRqXFJsCaik0ovwzsWjGxW4ryzTRCHj /4R3G90dQtT1VlVmq5QLT95UC1UENhyhZeUMZD8EAts+8XeQ2U/OU6ifLGGNjZQk7yXE 8xQDLtF1lIhyRV3uWUORwcxFOfA/CmvZmABOW+Y6+IXo/3fZV8o2ms5qu7qGDXdtLtWS FOJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DaXOH9viw4AlOcNpHDVEcNVeb68SygmlHfuqV9aPO9I=; b=U36YwMLua+CRih1KNKh5rtwnLeTtj/YEjgYMHa9lTphn9De0RIgf2uCFFfzGh5oRTb mLU3ditNyoYzftyNFAtTxXZ1BoqAEH8ycTCrzSiIuhdRmntqTLBMIAFnovsFAL1fkEtO SRwsYTUL181Icrg9MyUqIfkPxLn19jkZeH2FewvNdqWtPrARf5MGkwcl0AjyZBB45bM/ CBTL4JYukMbiC1vx5w2ryOi/Y6+/CGZaDOpcHD8WLae1gg78uAesihEHhVvRqi9H04pa 8N1jhWDNcerysl+CPmmN+rNqckpj5Y+9MvZTRNv9UNDeU15oK/KI66R+fdlHTak5CwsT 87gA== X-Gm-Message-State: AOAM530p2g8lDqFS9OmgM5hZGSgEHzP3l5H2U+Q8t/uftsUM7aNufQN4 wM39LXMXU/5UT9X90G9AHwE= X-Google-Smtp-Source: ABdhPJyRQ2ozDl6FOB6dQXcEHRI4e89qvU4R3wUlJEz+M92byxE0eMXPB8GFruzy7ul3bCA642NOxA== X-Received: by 2002:a6b:fe0e:: with SMTP id x14mr2300373ioh.79.1624505341306; Wed, 23 Jun 2021 20:29:01 -0700 (PDT) Received: from kolga-mac-1.attlocal.net ([2600:1700:6a10:2e90:fd18:15dc:e0e4:e39e]) by smtp.gmail.com with ESMTPSA id g4sm1026780ilk.37.2021.06.23.20.29.00 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jun 2021 20:29:00 -0700 (PDT) From: Olga Kornievskaia To: trond.myklebust@hammerspace.com, anna.schumaker@netapp.com Cc: linux-nfs@vger.kernel.org Subject: [PATCH v2 5/8] SUNRPC: take a xprt offline using sysfs Date: Wed, 23 Jun 2021 23:28:50 -0400 Message-Id: <20210624032853.4776-6-olga.kornievskaia@gmail.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20210624032853.4776-1-olga.kornievskaia@gmail.com> References: <20210624032853.4776-1-olga.kornievskaia@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Olga Kornievskaia Using sysfs's xprt_state attribute, mark a particular transport offline. It will not be picked during the round-robin selection. It's not allowed to take the main (1st created transport associated with the rpc_client) offline. Also bring a transport back online via sysfs by writing "online" and that would allow for this transport to be picked during the round- robin selection. Signed-off-by: Olga Kornievskaia --- include/linux/sunrpc/xprt.h | 1 + net/sunrpc/sysfs.c | 66 ++++++++++++++++++++++++++++++++++--- net/sunrpc/sysfs.h | 1 + net/sunrpc/xprtmultipath.c | 6 ++-- 4 files changed, 68 insertions(+), 6 deletions(-) diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index 692e5946c029..b8ed7fa1b4ca 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -427,6 +427,7 @@ void xprt_release_write(struct rpc_xprt *, struct rpc_task *); #define XPRT_BOUND (4) #define XPRT_BINDING (5) #define XPRT_CLOSING (6) +#define XPRT_OFFLINE (7) #define XPRT_CONGESTED (9) #define XPRT_CWND_WAIT (10) #define XPRT_WRITE_SPACE (11) diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c index 816f543d4237..e66888cc4c14 100644 --- a/net/sunrpc/sysfs.c +++ b/net/sunrpc/sysfs.c @@ -68,6 +68,15 @@ rpc_sysfs_xprt_kobj_get_xprt(struct kobject *kobj) return xprt_get(x->xprt); } +static inline struct rpc_xprt_switch * +rpc_sysfs_xprt_kobj_get_xprt_switch(struct kobject *kobj) +{ + struct rpc_sysfs_xprt *x = container_of(kobj, + struct rpc_sysfs_xprt, kobject); + + return xprt_switch_get(x->xprt_switch); +} + static inline struct rpc_xprt_switch * rpc_sysfs_xprt_switch_kobj_get_xprt(struct kobject *kobj) { @@ -122,7 +131,7 @@ static ssize_t rpc_sysfs_xprt_state_show(struct kobject *kobj, struct rpc_xprt *xprt = rpc_sysfs_xprt_kobj_get_xprt(kobj); ssize_t ret; int locked, connected, connecting, close_wait, bound, binding, - closing, congested, cwnd_wait, write_space; + closing, congested, cwnd_wait, write_space, offline; if (!xprt) return 0; @@ -140,8 +149,9 @@ static ssize_t rpc_sysfs_xprt_state_show(struct kobject *kobj, congested = test_bit(XPRT_CONGESTED, &xprt->state); cwnd_wait = test_bit(XPRT_CWND_WAIT, &xprt->state); write_space = test_bit(XPRT_WRITE_SPACE, &xprt->state); + offline = test_bit(XPRT_OFFLINE, &xprt->state); - ret = sprintf(buf, "state=%s %s %s %s %s %s %s %s %s %s\n", + ret = sprintf(buf, "state=%s %s %s %s %s %s %s %s %s %s %s\n", locked ? "LOCKED" : "", connected ? "CONNECTED" : "", connecting ? "CONNECTING" : "", @@ -151,7 +161,8 @@ static ssize_t rpc_sysfs_xprt_state_show(struct kobject *kobj, closing ? "CLOSING" : "", congested ? "CONGESTED" : "", cwnd_wait ? "CWND_WAIT" : "", - write_space ? "WRITE_SPACE" : ""); + write_space ? "WRITE_SPACE" : "", + offline ? "OFFLINE" : ""); } xprt_put(xprt); @@ -235,6 +246,52 @@ static ssize_t rpc_sysfs_xprt_dstaddr_store(struct kobject *kobj, goto out; } +static ssize_t rpc_sysfs_xprt_state_change(struct kobject *kobj, + struct kobj_attribute *attr, + const char *buf, size_t count) +{ + struct rpc_xprt *xprt = rpc_sysfs_xprt_kobj_get_xprt(kobj); + int offline = 0, online = 0; + struct rpc_xprt_switch *xps = rpc_sysfs_xprt_kobj_get_xprt_switch(kobj); + + if (!xprt) + return 0; + + if (!strncmp(buf, "offline", 7)) + offline = 1; + else if (!strncmp(buf, "online", 6)) + online = 1; + else + return -EINVAL; + + if (wait_on_bit_lock(&xprt->state, XPRT_LOCKED, TASK_KILLABLE)) { + count = -EINTR; + goto out_put; + } + if (xprt->main) { + count = -EINVAL; + goto release_tasks; + } + if (offline) { + set_bit(XPRT_OFFLINE, &xprt->state); + spin_lock(&xps->xps_lock); + xps->xps_nactive--; + spin_unlock(&xps->xps_lock); + } else if (online) { + clear_bit(XPRT_OFFLINE, &xprt->state); + spin_lock(&xps->xps_lock); + xps->xps_nactive++; + spin_unlock(&xps->xps_lock); + } + +release_tasks: + xprt_release_write(xprt, NULL); +out_put: + xprt_put(xprt); + xprt_switch_put(xps); + return count; +} + int rpc_sysfs_init(void) { rpc_sunrpc_kset = kset_create_and_add("sunrpc", NULL, kernel_kobj); @@ -305,7 +362,7 @@ static struct kobj_attribute rpc_sysfs_xprt_info = __ATTR(xprt_info, 0444, rpc_sysfs_xprt_info_show, NULL); static struct kobj_attribute rpc_sysfs_xprt_change_state = __ATTR(xprt_state, - 0644, rpc_sysfs_xprt_state_show, NULL); + 0644, rpc_sysfs_xprt_state_show, rpc_sysfs_xprt_state_change); static struct attribute *rpc_sysfs_xprt_attrs[] = { &rpc_sysfs_xprt_dstaddr.attr, @@ -468,6 +525,7 @@ void rpc_sysfs_xprt_setup(struct rpc_xprt_switch *xprt_switch, if (rpc_xprt) { xprt->xprt_sysfs = rpc_xprt; rpc_xprt->xprt = xprt; + rpc_xprt->xprt_switch = xprt_switch; kobject_uevent(&rpc_xprt->kobject, KOBJ_ADD); } } diff --git a/net/sunrpc/sysfs.h b/net/sunrpc/sysfs.h index ff10451de6fa..6620cebd1037 100644 --- a/net/sunrpc/sysfs.h +++ b/net/sunrpc/sysfs.h @@ -22,6 +22,7 @@ struct rpc_sysfs_xprt_switch { struct rpc_sysfs_xprt { struct kobject kobject; struct rpc_xprt *xprt; + struct rpc_xprt_switch *xprt_switch; }; int rpc_sysfs_init(void); diff --git a/net/sunrpc/xprtmultipath.c b/net/sunrpc/xprtmultipath.c index 584349c8cad4..7d40cdf81274 100644 --- a/net/sunrpc/xprtmultipath.c +++ b/net/sunrpc/xprtmultipath.c @@ -65,7 +65,8 @@ static void xprt_switch_remove_xprt_locked(struct rpc_xprt_switch *xps, { if (unlikely(xprt == NULL)) return; - xps->xps_nactive--; + if (!test_bit(XPRT_OFFLINE, &xprt->state)) + xps->xps_nactive--; xps->xps_nxprts--; if (xps->xps_nxprts == 0) xps->xps_net = NULL; @@ -231,7 +232,8 @@ void xprt_iter_default_rewind(struct rpc_xprt_iter *xpi) static bool xprt_is_active(const struct rpc_xprt *xprt) { - return kref_read(&xprt->kref) != 0; + return (kref_read(&xprt->kref) != 0 && + !test_bit(XPRT_OFFLINE, &xprt->state)); } static From patchwork Thu Jun 24 03:28:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olga Kornievskaia X-Patchwork-Id: 12340739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E591C49EA6 for ; Thu, 24 Jun 2021 03:29:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8222C610C7 for ; Thu, 24 Jun 2021 03:29:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230014AbhFXDbW (ORCPT ); Wed, 23 Jun 2021 23:31:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230002AbhFXDbV (ORCPT ); Wed, 23 Jun 2021 23:31:21 -0400 Received: from mail-il1-x132.google.com (mail-il1-x132.google.com [IPv6:2607:f8b0:4864:20::132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02E75C061574 for ; Wed, 23 Jun 2021 20:29:03 -0700 (PDT) Received: by mail-il1-x132.google.com with SMTP id i12so4660547ila.13 for ; Wed, 23 Jun 2021 20:29:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Oy8L62XnZY4pyFFfTZKcTqYbSTRclV9kgxUhLZk7FgA=; b=A8A86WoP1JXQ/BAhQHd0uj5spyPpRXXierjqo+wvauAD2/BlAPGrRhEHpUR15fwf07 e1PD1CxM/h4R4QfECvc2vgzyhsBBmh3b5kOEguplOzzKIQfgm7jvBoe1kAKwcCeCn0mi woShwOK0Po9ZqKjaVp6wBEvglTiruvRxTYg/yKLi1KyVnr8tLWdOPoSS//iyu4DAWArW 2BTslOiD+xL3ifyRERcgUQYaffISSXA+IOssH9DcKVaMB02wgZu5LwjDO2Nn+5K00tsc wG0T+ro/Gvo1yEjxVkvI+zw1KC0lf2/Xb0eF1IdKUuo6iP6NNkg8b9VnjIEF/qaHFerJ kAfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Oy8L62XnZY4pyFFfTZKcTqYbSTRclV9kgxUhLZk7FgA=; b=nBj+vE1kFoJwQKXGuc20ski7cfwn+zPCXI+yRV7qHp3ITUivBZSS+1sUZ8bdmU1bPW uCj/tAJuPXPiLnz4aDjGjGpKNr0uT9GduhRa/uws+fUSo/zaujJsgEFuM2vQ4bzIdG74 xkEoxFOepV2Mc6Y+2iDbpwKQi7zSGxahjy7lFK45LcHv9F1KJhLrABoY3PFpYl44jFCS WT06U9t/aLQWOLh8kn8/m6OZlB5GFKWyMPoYprs/TIsiQggONkjkuiKZSqPxY9f7SPcY aA1ZW3oitanldfZDu6nP8AdvmP78e4Iv4Imjj+ECNGPPOtkrol3IxsllxTwBCH4n26cc Durg== X-Gm-Message-State: AOAM531syf0Jc0mxYCcndnyKRvhyDcyl6SPaP1rMphAVkTKNh/98i44t 8xl6VUyp9r+FVBWmhH/ZKUHUiD8TvZNJ6A== X-Google-Smtp-Source: ABdhPJwn34yVtSbCAGZRCvz0iVjF4T/hIO8Ogi8h86GxgLe+pqgPlOspGAHIvrIP0NzeUJM7e7wSIA== X-Received: by 2002:a92:cdab:: with SMTP id g11mr2059546ild.240.1624505342394; Wed, 23 Jun 2021 20:29:02 -0700 (PDT) Received: from kolga-mac-1.attlocal.net ([2600:1700:6a10:2e90:fd18:15dc:e0e4:e39e]) by smtp.gmail.com with ESMTPSA id g4sm1026780ilk.37.2021.06.23.20.29.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jun 2021 20:29:01 -0700 (PDT) From: Olga Kornievskaia To: trond.myklebust@hammerspace.com, anna.schumaker@netapp.com Cc: linux-nfs@vger.kernel.org Subject: [PATCH v2 6/8] NFSv4.1 identify and mark RPC tasks that can move between transports Date: Wed, 23 Jun 2021 23:28:51 -0400 Message-Id: <20210624032853.4776-7-olga.kornievskaia@gmail.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20210624032853.4776-1-olga.kornievskaia@gmail.com> References: <20210624032853.4776-1-olga.kornievskaia@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Olga Kornievskaia In preparation for when we can re-try a task on a different transport, identify and mark such RPC tasks as moveable. Only 4.1+ operarations can be re-tried on a different transport. Signed-off-by: Olga Kornievskaia --- fs/nfs/nfs4proc.c | 38 +++++++++++++++++++++++++++++++----- fs/nfs/pagelist.c | 8 ++++++-- fs/nfs/write.c | 6 +++++- include/linux/sunrpc/sched.h | 2 ++ 4 files changed, 46 insertions(+), 8 deletions(-) diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c index e653654c10bc..d3ee3700c9dd 100644 --- a/fs/nfs/nfs4proc.c +++ b/fs/nfs/nfs4proc.c @@ -1155,7 +1155,11 @@ static int nfs4_call_sync_sequence(struct rpc_clnt *clnt, struct nfs4_sequence_args *args, struct nfs4_sequence_res *res) { - return nfs4_do_call_sync(clnt, server, msg, args, res, 0); + unsigned short task_flags = 0; + + if (server->nfs_client->cl_minorversion) + task_flags = RPC_TASK_MOVEABLE; + return nfs4_do_call_sync(clnt, server, msg, args, res, task_flags); } @@ -2569,6 +2573,9 @@ static int nfs4_run_open_task(struct nfs4_opendata *data, }; int status; + if (server->nfs_client->cl_minorversion) + task_setup_data.flags |= RPC_TASK_MOVEABLE; + kref_get(&data->kref); data->rpc_done = false; data->rpc_status = 0; @@ -3749,6 +3756,9 @@ int nfs4_do_close(struct nfs4_state *state, gfp_t gfp_mask, int wait) }; int status = -ENOMEM; + if (server->nfs_client->cl_minorversion) + task_setup_data.flags |= RPC_TASK_MOVEABLE; + nfs4_state_protect(server->nfs_client, NFS_SP4_MACH_CRED_CLEANUP, &task_setup_data.rpc_client, &msg); @@ -4188,6 +4198,9 @@ static int _nfs4_proc_getattr(struct nfs_server *server, struct nfs_fh *fhandle, }; unsigned short task_flags = 0; + if (nfs4_has_session(server->nfs_client)) + task_flags = RPC_TASK_MOVEABLE; + /* Is this is an attribute revalidation, subject to softreval? */ if (inode && (server->flags & NFS_MOUNT_SOFTREVAL)) task_flags |= RPC_TASK_TIMEOUT; @@ -4307,6 +4320,9 @@ static int _nfs4_proc_lookup(struct rpc_clnt *clnt, struct inode *dir, }; unsigned short task_flags = 0; + if (server->nfs_client->cl_minorversion) + task_flags = RPC_TASK_MOVEABLE; + /* Is this is an attribute revalidation, subject to softreval? */ if (nfs_lookup_is_soft_revalidate(dentry)) task_flags |= RPC_TASK_TIMEOUT; @@ -6538,7 +6554,7 @@ static int _nfs4_proc_delegreturn(struct inode *inode, const struct cred *cred, .rpc_client = server->client, .rpc_message = &msg, .callback_ops = &nfs4_delegreturn_ops, - .flags = RPC_TASK_ASYNC | RPC_TASK_TIMEOUT, + .flags = RPC_TASK_ASYNC | RPC_TASK_TIMEOUT | RPC_TASK_MOVEABLE, }; int status = 0; @@ -6856,6 +6872,11 @@ static struct rpc_task *nfs4_do_unlck(struct file_lock *fl, .workqueue = nfsiod_workqueue, .flags = RPC_TASK_ASYNC, }; + struct nfs_client *client = + NFS_SERVER(lsp->ls_state->inode)->nfs_client; + + if (client->cl_minorversion) + task_setup_data.flags |= RPC_TASK_MOVEABLE; nfs4_state_protect(NFS_SERVER(lsp->ls_state->inode)->nfs_client, NFS_SP4_MACH_CRED_CLEANUP, &task_setup_data.rpc_client, &msg); @@ -7130,6 +7151,10 @@ static int _nfs4_do_setlk(struct nfs4_state *state, int cmd, struct file_lock *f .flags = RPC_TASK_ASYNC | RPC_TASK_CRED_NOREF, }; int ret; + struct nfs_client *client = NFS_SERVER(state->inode)->nfs_client; + + if (client->cl_minorversion) + task_setup_data.flags |= RPC_TASK_MOVEABLE; dprintk("%s: begin!\n", __func__); data = nfs4_alloc_lockdata(fl, nfs_file_open_context(fl->fl_file), @@ -9186,7 +9211,7 @@ static struct rpc_task *_nfs41_proc_sequence(struct nfs_client *clp, .rpc_client = clp->cl_rpcclient, .rpc_message = &msg, .callback_ops = &nfs41_sequence_ops, - .flags = RPC_TASK_ASYNC | RPC_TASK_TIMEOUT, + .flags = RPC_TASK_ASYNC | RPC_TASK_TIMEOUT | RPC_TASK_MOVEABLE, }; struct rpc_task *ret; @@ -9509,7 +9534,8 @@ nfs4_proc_layoutget(struct nfs4_layoutget *lgp, long *timeout) .rpc_message = &msg, .callback_ops = &nfs4_layoutget_call_ops, .callback_data = lgp, - .flags = RPC_TASK_ASYNC | RPC_TASK_CRED_NOREF, + .flags = RPC_TASK_ASYNC | RPC_TASK_CRED_NOREF | + RPC_TASK_MOVEABLE, }; struct pnfs_layout_segment *lseg = NULL; struct nfs4_exception exception = { @@ -9650,6 +9676,7 @@ int nfs4_proc_layoutreturn(struct nfs4_layoutreturn *lrp, bool sync) .rpc_message = &msg, .callback_ops = &nfs4_layoutreturn_call_ops, .callback_data = lrp, + .flags = RPC_TASK_MOVEABLE, }; int status = 0; @@ -9804,6 +9831,7 @@ nfs4_proc_layoutcommit(struct nfs4_layoutcommit_data *data, bool sync) .rpc_message = &msg, .callback_ops = &nfs4_layoutcommit_ops, .callback_data = data, + .flags = RPC_TASK_MOVEABLE, }; struct rpc_task *task; int status = 0; @@ -10131,7 +10159,7 @@ static int nfs41_free_stateid(struct nfs_server *server, .rpc_client = server->client, .rpc_message = &msg, .callback_ops = &nfs41_free_stateid_ops, - .flags = RPC_TASK_ASYNC, + .flags = RPC_TASK_ASYNC | RPC_TASK_MOVEABLE, }; struct nfs_free_stateid_data *data; struct rpc_task *task; diff --git a/fs/nfs/pagelist.c b/fs/nfs/pagelist.c index cf9cc62ec48e..cc232d1f16f2 100644 --- a/fs/nfs/pagelist.c +++ b/fs/nfs/pagelist.c @@ -954,6 +954,7 @@ static int nfs_generic_pg_pgios(struct nfs_pageio_descriptor *desc) { struct nfs_pgio_header *hdr; int ret; + unsigned short task_flags = 0; hdr = nfs_pgio_header_alloc(desc->pg_rw_ops); if (!hdr) { @@ -962,14 +963,17 @@ static int nfs_generic_pg_pgios(struct nfs_pageio_descriptor *desc) } nfs_pgheader_init(desc, hdr, nfs_pgio_header_free); ret = nfs_generic_pgio(desc, hdr); - if (ret == 0) + if (ret == 0) { + if (NFS_SERVER(hdr->inode)->nfs_client->cl_minorversion) + task_flags = RPC_TASK_MOVEABLE; ret = nfs_initiate_pgio(NFS_CLIENT(hdr->inode), hdr, hdr->cred, NFS_PROTO(hdr->inode), desc->pg_rpc_callops, desc->pg_ioflags, - RPC_TASK_CRED_NOREF); + RPC_TASK_CRED_NOREF | task_flags); + } return ret; } diff --git a/fs/nfs/write.c b/fs/nfs/write.c index 3bf82178166a..eae9bf114041 100644 --- a/fs/nfs/write.c +++ b/fs/nfs/write.c @@ -1810,6 +1810,7 @@ nfs_commit_list(struct inode *inode, struct list_head *head, int how, struct nfs_commit_info *cinfo) { struct nfs_commit_data *data; + unsigned short task_flags = 0; /* another commit raced with us */ if (list_empty(head)) @@ -1820,8 +1821,11 @@ nfs_commit_list(struct inode *inode, struct list_head *head, int how, /* Set up the argument struct */ nfs_init_commit(data, head, NULL, cinfo); atomic_inc(&cinfo->mds->rpcs_out); + if (NFS_SERVER(inode)->nfs_client->cl_minorversion) + task_flags = RPC_TASK_MOVEABLE; return nfs_initiate_commit(NFS_CLIENT(inode), data, NFS_PROTO(inode), - data->mds_ops, how, RPC_TASK_CRED_NOREF); + data->mds_ops, how, + RPC_TASK_CRED_NOREF | task_flags); } /* diff --git a/include/linux/sunrpc/sched.h b/include/linux/sunrpc/sched.h index df696efdd675..a237b8dbf608 100644 --- a/include/linux/sunrpc/sched.h +++ b/include/linux/sunrpc/sched.h @@ -121,6 +121,7 @@ struct rpc_task_setup { */ #define RPC_TASK_ASYNC 0x0001 /* is an async task */ #define RPC_TASK_SWAPPER 0x0002 /* is swapping in/out */ +#define RPC_TASK_MOVEABLE 0x0004 /* nfs4.1+ rpc tasks */ #define RPC_TASK_NULLCREDS 0x0010 /* Use AUTH_NULL credential */ #define RPC_CALL_MAJORSEEN 0x0020 /* major timeout seen */ #define RPC_TASK_ROOTCREDS 0x0040 /* force root creds */ @@ -139,6 +140,7 @@ struct rpc_task_setup { #define RPC_IS_SOFT(t) ((t)->tk_flags & (RPC_TASK_SOFT|RPC_TASK_TIMEOUT)) #define RPC_IS_SOFTCONN(t) ((t)->tk_flags & RPC_TASK_SOFTCONN) #define RPC_WAS_SENT(t) ((t)->tk_flags & RPC_TASK_SENT) +#define RPC_IS_MOVEABLE(t) ((t)->tk_flags & RPC_TASK_MOVEABLE) #define RPC_TASK_RUNNING 0 #define RPC_TASK_QUEUED 1 From patchwork Thu Jun 24 03:28:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olga Kornievskaia X-Patchwork-Id: 12340741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FF65C49EA5 for ; Thu, 24 Jun 2021 03:29:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 36CE86137D for ; Thu, 24 Jun 2021 03:29:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230018AbhFXDbW (ORCPT ); Wed, 23 Jun 2021 23:31:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36202 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230002AbhFXDbW (ORCPT ); Wed, 23 Jun 2021 23:31:22 -0400 Received: from mail-il1-x12b.google.com (mail-il1-x12b.google.com [IPv6:2607:f8b0:4864:20::12b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0EAE9C061574 for ; Wed, 23 Jun 2021 20:29:04 -0700 (PDT) Received: by mail-il1-x12b.google.com with SMTP id v5so4704108ilo.5 for ; Wed, 23 Jun 2021 20:29:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L1TGSDioUL5W9prSNZhTqw+0YOv25eoLuadjd+CUCHc=; b=jEDi9v7l8NruJ/19hLaddZDDbqLH8WyBe8sPCfyc7nd4GT8gN4U0n8v3uo38ADdA24 2LKiKfy1hG6OaJPNS5DhpfGN/gwDQyrCG8Ir+L1UgChDrZ4wUdnwiw2VM9OfGLVosb9P ReBAsz1NCA49b2A8OUCPlvCZI9qj1f3waWJA9MYBIhCrH4D4q9WNeCxiYZw8baf6g1IV aJopW2Cqnr9Q5+S1+Tjy8W1dapefFIo53VZ1H/4Z8DxfEUzX8TGU+VZFyg1D4lkhqOLt 4GUOXLnimfT10TFl8DPxUipGnqd5sj3jBwb/3Rzg9PBCF5bJJPoyYYoxq2YA8QRa/ehA 0rlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L1TGSDioUL5W9prSNZhTqw+0YOv25eoLuadjd+CUCHc=; b=aMyq3gWv5WMLLalkK0CC4MBLBVZVHtzgUKgcb3ykDNQRmRObUBb3eRPnky/g02bA+7 K+v+ejb9KP9WqUfP+JKYxKZFJVuMa8NjjMBgQUafq/oaB/hnDGl7EOj4J2uLoGEbE9xA jgiLiKrG7E3SrK5YhPuiJzNyesA9Y8HZMEcJjvm9zinzOZ7qg/0JkKRbUVGm4kv3tRx0 jAKx4FLB868O8O1nqermv469XuyqGqhr7IG1ipOLX/YU0vzRrCeBRFSZMw+TyljRmv/J huYEA7CdIGSguVlbs2d66pNcPCQg9tMncS9bSd9Rb+iLPtQYdG1RhfnoGZluyrMEJVz6 TD6w== X-Gm-Message-State: AOAM5305+YeYzafQlPpMGVf1IbiHu4/AU+i656uxetwBbpUQjYStwQ4t OXY4DOwMuu8C+DKWH6zYH0w= X-Google-Smtp-Source: ABdhPJx3dZwjSOZrNCMFhZ5v7c3pt+guCOB1k7bcfboNboWUOSCWiu9OW7Yohql19hVX/iWhWa14vQ== X-Received: by 2002:a92:d852:: with SMTP id h18mr1880001ilq.299.1624505343444; Wed, 23 Jun 2021 20:29:03 -0700 (PDT) Received: from kolga-mac-1.attlocal.net ([2600:1700:6a10:2e90:fd18:15dc:e0e4:e39e]) by smtp.gmail.com with ESMTPSA id g4sm1026780ilk.37.2021.06.23.20.29.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jun 2021 20:29:02 -0700 (PDT) From: Olga Kornievskaia To: trond.myklebust@hammerspace.com, anna.schumaker@netapp.com Cc: linux-nfs@vger.kernel.org Subject: [PATCH v2 7/8] sunrpc: display xprt's queuelen of assigned tasks via sysfs Date: Wed, 23 Jun 2021 23:28:52 -0400 Message-Id: <20210624032853.4776-8-olga.kornievskaia@gmail.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20210624032853.4776-1-olga.kornievskaia@gmail.com> References: <20210624032853.4776-1-olga.kornievskaia@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Olga Kornievskaia Once a task grabs a trasnport it's reflected in the queuelen of the rpc_xprt structure. Add display of that value in the xprt's info file in sysfs. Signed-off-by: Olga Kornievskaia --- net/sunrpc/sysfs.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c index e66888cc4c14..754bd010ea29 100644 --- a/net/sunrpc/sysfs.c +++ b/net/sunrpc/sysfs.c @@ -113,13 +113,15 @@ static ssize_t rpc_sysfs_xprt_info_show(struct kobject *kobj, ret = sprintf(buf, "last_used=%lu\ncur_cong=%lu\ncong_win=%lu\n" "max_num_slots=%u\nmin_num_slots=%u\nnum_reqs=%u\n" "binding_q_len=%u\nsending_q_len=%u\npending_q_len=%u\n" - "backlog_q_len=%u\nmain_xprt=%d\nsrc_port=%u\n", + "backlog_q_len=%u\nmain_xprt=%d\nsrc_port=%u\n" + "tasks_queuelen=%ld\n", xprt->last_used, xprt->cong, xprt->cwnd, xprt->max_reqs, xprt->min_reqs, xprt->num_reqs, xprt->binding.qlen, xprt->sending.qlen, xprt->pending.qlen, xprt->backlog.qlen, xprt->main, (xprt->xprt_class->ident == XPRT_TRANSPORT_TCP) ? - get_srcport(xprt) : 0); + get_srcport(xprt) : 0, + atomic_long_read(&xprt->queuelen)); xprt_put(xprt); return ret + 1; } From patchwork Thu Jun 24 03:28:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olga Kornievskaia X-Patchwork-Id: 12340743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7E18C48BC2 for ; Thu, 24 Jun 2021 03:29:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B24B66137D for ; Thu, 24 Jun 2021 03:29:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230002AbhFXDbY (ORCPT ); Wed, 23 Jun 2021 23:31:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36210 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229818AbhFXDbX (ORCPT ); Wed, 23 Jun 2021 23:31:23 -0400 Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3069FC061574 for ; Wed, 23 Jun 2021 20:29:05 -0700 (PDT) Received: by mail-io1-xd29.google.com with SMTP id k16so6100680ios.10 for ; Wed, 23 Jun 2021 20:29:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oHBCy9w3wXSZMZ1zRa/auMjewfvBPVGOk6hJ8eHii8c=; b=UV0UJrBhLakck9qC+n+xG00ki+whU4F+nwBDx/xmdxQKfZmSh/3eMPau4wBGzsuNkz AIAnxsp1MUdoiX0OGRfn/d7wyIflMLpjcqfeJ09c66O1TbhHRPOnnYiNrE6ylUaSOZof frxRE9s2Wjef4JPVgsl4samq8GriBvGuSWf7jiwJzvJxEXUrb1Iiz70/i+FD/gLVsdGr voCRYT7afOBM4/cLknwDPizsjNS9p0ESqR2pFyWCv2FuyQRPy5sI/ptmafGT8GNA21RX AkkUY/WLLUJUahVwWnaW8PNvRUEMP3Lh39xmazIhMpKlZduoVEqC49wJ1Mbx4GeD6KwO Vaxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oHBCy9w3wXSZMZ1zRa/auMjewfvBPVGOk6hJ8eHii8c=; b=TaO+ntETd3Jis0dTOL1q9RneQTguiRcwf3dbSeBBLskox24wUWd0eJzOin/OQDwXh2 HWd3rU5OhBPYnHVceMOsxieD+HYWqs+ATFcLXdMqm5R2iIA1jxtSzy3PMeWQ1lrQEyML mIYO4wQEwthZBUnwTc521+baPT2DTQpMV2IuEMRMo6s4drsG9JfGZbmT2RZBnTHNLsPD +RKhQ8UsepAhi4v6R+eq6WkL8MuneQiAbHjD5fkSEXOTT0i9lwEQGSBD5km75o9ilnyM wCzulJl0LD74TIyCQVkRF7JS3KcMv8bWHeVfp+vvY4wg7anxl/IUO3zC89ZecfU2C5Wr 8pTg== X-Gm-Message-State: AOAM531s8T7PxxoGbNQ54nDT36k6JNcDHRrt07M3Y3jchCF7g6cKkh+8 MZw6EgEke53ZTmhuXHRZisjRrADAcA0oQA== X-Google-Smtp-Source: ABdhPJwGlotvefyTNIi/IWFiztLGCD9jtABKv6bf+Lct45beoAFjgFrGAp5bHkkHIFRWp419QQunvA== X-Received: by 2002:a6b:4418:: with SMTP id r24mr2272875ioa.123.1624505344638; Wed, 23 Jun 2021 20:29:04 -0700 (PDT) Received: from kolga-mac-1.attlocal.net ([2600:1700:6a10:2e90:fd18:15dc:e0e4:e39e]) by smtp.gmail.com with ESMTPSA id g4sm1026780ilk.37.2021.06.23.20.29.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jun 2021 20:29:04 -0700 (PDT) From: Olga Kornievskaia To: trond.myklebust@hammerspace.com, anna.schumaker@netapp.com Cc: linux-nfs@vger.kernel.org Subject: [PATCH v2 8/8] sunrpc: remove an offlined xprt using sysfs Date: Wed, 23 Jun 2021 23:28:53 -0400 Message-Id: <20210624032853.4776-9-olga.kornievskaia@gmail.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20210624032853.4776-1-olga.kornievskaia@gmail.com> References: <20210624032853.4776-1-olga.kornievskaia@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Olga Kornievskaia Once a transport has been put offline, this transport can be also removed from the list of transports. Any tasks that have been stuck on this transport would find the next available active transport and be re-tried. This transport would be removed from the xprt_switch list and freed. Signed-off-by: Olga Kornievskaia --- include/linux/sunrpc/xprt.h | 1 + net/sunrpc/clnt.c | 24 ++++++++++++++++++++++++ net/sunrpc/sysfs.c | 26 ++++++++++++++++++++++---- 3 files changed, 47 insertions(+), 4 deletions(-) diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index b8ed7fa1b4ca..c8c39f22d3b1 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -428,6 +428,7 @@ void xprt_release_write(struct rpc_xprt *, struct rpc_task *); #define XPRT_BINDING (5) #define XPRT_CLOSING (6) #define XPRT_OFFLINE (7) +#define XPRT_REMOVE (8) #define XPRT_CONGESTED (9) #define XPRT_CWND_WAIT (10) #define XPRT_WRITE_SPACE (11) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 7ca946567e13..f8ec29b06d0d 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -2106,6 +2106,30 @@ call_connect_status(struct rpc_task *task) case -ENOTCONN: case -EAGAIN: case -ETIMEDOUT: + if (!(task->tk_flags & RPC_TASK_NO_ROUND_ROBIN) && + (task->tk_flags & RPC_TASK_MOVEABLE) && + test_bit(XPRT_REMOVE, &xprt->state)) { + struct rpc_xprt *saved = task->tk_xprt; + struct rpc_xprt_switch *xps; + + rcu_read_lock(); + xps = xprt_switch_get(rcu_dereference(clnt->cl_xpi.xpi_xpswitch)); + rcu_read_unlock(); + if (xps->xps_nxprts > 1) { + long value; + + xprt_release(task); + value = atomic_long_dec_return(&xprt->queuelen); + if (value == 0) + rpc_xprt_switch_remove_xprt(xps, saved); + xprt_put(saved); + task->tk_xprt = NULL; + task->tk_action = call_start; + } + xprt_switch_put(xps); + if (!task->tk_xprt) + return; + } goto out_retry; case -ENOBUFS: rpc_delay(task, HZ >> 2); diff --git a/net/sunrpc/sysfs.c b/net/sunrpc/sysfs.c index 754bd010ea29..34b60db5321f 100644 --- a/net/sunrpc/sysfs.c +++ b/net/sunrpc/sysfs.c @@ -133,7 +133,7 @@ static ssize_t rpc_sysfs_xprt_state_show(struct kobject *kobj, struct rpc_xprt *xprt = rpc_sysfs_xprt_kobj_get_xprt(kobj); ssize_t ret; int locked, connected, connecting, close_wait, bound, binding, - closing, congested, cwnd_wait, write_space, offline; + closing, congested, cwnd_wait, write_space, offline, remove; if (!xprt) return 0; @@ -152,8 +152,9 @@ static ssize_t rpc_sysfs_xprt_state_show(struct kobject *kobj, cwnd_wait = test_bit(XPRT_CWND_WAIT, &xprt->state); write_space = test_bit(XPRT_WRITE_SPACE, &xprt->state); offline = test_bit(XPRT_OFFLINE, &xprt->state); + remove = test_bit(XPRT_REMOVE, &xprt->state); - ret = sprintf(buf, "state=%s %s %s %s %s %s %s %s %s %s %s\n", + ret = sprintf(buf, "state=%s %s %s %s %s %s %s %s %s %s %s %s\n", locked ? "LOCKED" : "", connected ? "CONNECTED" : "", connecting ? "CONNECTING" : "", @@ -164,7 +165,8 @@ static ssize_t rpc_sysfs_xprt_state_show(struct kobject *kobj, congested ? "CONGESTED" : "", cwnd_wait ? "CWND_WAIT" : "", write_space ? "WRITE_SPACE" : "", - offline ? "OFFLINE" : ""); + offline ? "OFFLINE" : "", + remove ? "REMOVE" : ""); } xprt_put(xprt); @@ -253,7 +255,7 @@ static ssize_t rpc_sysfs_xprt_state_change(struct kobject *kobj, const char *buf, size_t count) { struct rpc_xprt *xprt = rpc_sysfs_xprt_kobj_get_xprt(kobj); - int offline = 0, online = 0; + int offline = 0, online = 0, remove = 0; struct rpc_xprt_switch *xps = rpc_sysfs_xprt_kobj_get_xprt_switch(kobj); if (!xprt) @@ -263,6 +265,8 @@ static ssize_t rpc_sysfs_xprt_state_change(struct kobject *kobj, offline = 1; else if (!strncmp(buf, "online", 6)) online = 1; + else if (!strncmp(buf, "remove", 6)) + remove = 1; else return -EINVAL; @@ -284,6 +288,20 @@ static ssize_t rpc_sysfs_xprt_state_change(struct kobject *kobj, spin_lock(&xps->xps_lock); xps->xps_nactive++; spin_unlock(&xps->xps_lock); + } else if (remove) { + if (test_bit(XPRT_OFFLINE, &xprt->state)) { + set_bit(XPRT_REMOVE, &xprt->state); + xprt_force_disconnect(xprt); + if (test_bit(XPRT_CONNECTED, &xprt->state)) { + if (!xprt->sending.qlen && + !xprt->pending.qlen && + !xprt->backlog.qlen && + !atomic_long_read(&xprt->queuelen)) + rpc_xprt_switch_remove_xprt(xps, xprt); + } + } else { + count = -EINVAL; + } } release_tasks: