From patchwork Fri May 18 13:04:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Pen X-Patchwork-Id: 10410599 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A0BFD602CB for ; Fri, 18 May 2018 13:06:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8F7152897C for ; Fri, 18 May 2018 13:06:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 83E6428984; Fri, 18 May 2018 13:06:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C299F2897C for ; Fri, 18 May 2018 13:06:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752579AbeERNGP (ORCPT ); Fri, 18 May 2018 09:06:15 -0400 Received: from mail-wr0-f194.google.com ([209.85.128.194]:44916 "EHLO mail-wr0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752646AbeERNFn (ORCPT ); Fri, 18 May 2018 09:05:43 -0400 Received: by mail-wr0-f194.google.com with SMTP id y15-v6so9133633wrg.11 for ; Fri, 18 May 2018 06:05:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=profitbricks-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=cjzdCJ8mHOzcaw7hXdcmJrtk6Jct6s3mY7fc606a6qg=; b=qI5jWixvN3Tj/wKZJ0QGDu5bCqPQxvMN31L/2uRr1Ly8t78lTqCeOF1cS2RS0oXfGh W82Bu69EjbyCMCg2wlDn2Kz1rag8y/NEGgBGo2pZuciO6uMqfBKDOUT03i5wVvxQs1tl mCYIEglf+jnXLw0x0UNUdPfPy1+DMJdJ4amjGFXD+PvtA5tt91l0y7Gu7MzbGJ43t6H6 BdNNbb0Inu5sVEQ+I4VleFVpBk6wON/7qkIPamFkeBK23dpmOUqzk5mCiL50HgxS4Dr2 eP2PgVRCF/syFkvwB0h7kj6KgJ4G8SOIZdtsAPlYTRXzR6WyLseGYMtdqUiGPLUdhyEv C4VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=cjzdCJ8mHOzcaw7hXdcmJrtk6Jct6s3mY7fc606a6qg=; b=qaeVUvh6GskZgG4eoQT1zoVg0rIhha2twG39DhygkJrPhsg+74X5qX86H+1hBEMJwc lOnblx+JjaqaDPgG8QqGOeUjRAs5JStxvusqxcVQhJrPD04sxpewOTJojdyHu0xN+BSj CXCyxkj3Qm3M56P9Vw8YL7ncxvPnAi7ZfWKN+KsRRvMWhg9/8O6mNulaCoVmG4Es+WTH 46Axx6IS0mLSJEChF5rox2Mm/8cq6nD2vpbUBckTIE9aO+eKp//H5JV1RmfvoU7tURlG erCNalyr3JoXy+KoebHMQZ+tZsdcbdL8sjpo93nLaZxVGcmIGeGCBEWWizuIeNYwPXqC bcGQ== X-Gm-Message-State: ALKqPweFrWJO8+sypPbxlkPLnlQNOpcRF28GDJ1O5zzHKs7I9PxkowPd 8217sR4/DuaQhgQEhc9nUPOrnBcn X-Google-Smtp-Source: AB8JxZpNOFgLmwC+s3f3ETWMx8HCogWi+tqTRpn/4tMgbsu0Sg9JwqMt8eliCskJ7ZMbCxbwB6pCIA== X-Received: by 2002:adf:86ac:: with SMTP id 41-v6mr7292501wrx.260.1526648741824; Fri, 18 May 2018 06:05:41 -0700 (PDT) Received: from pb.pb.local ([62.217.45.26]) by smtp.gmail.com with ESMTPSA id m35-v6sm7639980wrm.51.2018.05.18.06.05.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 18 May 2018 06:05:41 -0700 (PDT) From: Roman Pen To: linux-block@vger.kernel.org, linux-rdma@vger.kernel.org Cc: Jens Axboe , Christoph Hellwig , Sagi Grimberg , Bart Van Assche , Or Gerlitz , Doug Ledford , Swapnil Ingle , Danil Kipnis , Jack Wang , Roman Pen Subject: [PATCH v2 13/26] ibtrs: server: sysfs interface functions Date: Fri, 18 May 2018 15:04:00 +0200 Message-Id: <20180518130413.16997-14-roman.penyaev@profitbricks.com> X-Mailer: git-send-email 2.13.1 In-Reply-To: <20180518130413.16997-1-roman.penyaev@profitbricks.com> References: <20180518130413.16997-1-roman.penyaev@profitbricks.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This is the sysfs interface to IBTRS sessions on server side: /sys/devices/virtual/ibtrs-server// *** IBTRS session accepted from a client peer | |- paths// *** established paths from a client in a session | |- disconnect | *** disconnect path | |- hca_name | *** HCA name | |- hca_port | *** HCA port | |- stats/ *** current path statistics | |- rdma |- reset_all |- wc_completions Signed-off-by: Roman Pen Signed-off-by: Danil Kipnis Cc: Jack Wang --- drivers/infiniband/ulp/ibtrs/ibtrs-srv-sysfs.c | 271 +++++++++++++++++++++++++ 1 file changed, 271 insertions(+) create mode 100644 drivers/infiniband/ulp/ibtrs/ibtrs-srv-sysfs.c diff --git a/drivers/infiniband/ulp/ibtrs/ibtrs-srv-sysfs.c b/drivers/infiniband/ulp/ibtrs/ibtrs-srv-sysfs.c new file mode 100644 index 000000000000..96d9d9f08e0e --- /dev/null +++ b/drivers/infiniband/ulp/ibtrs/ibtrs-srv-sysfs.c @@ -0,0 +1,271 @@ +/* + * InfiniBand Transport Layer + * + * Copyright (c) 2014 - 2017 ProfitBricks GmbH. All rights reserved. + * Authors: Fabian Holler + * Jack Wang + * Kleber Souza + * Danil Kipnis + * Roman Penyaev + * Milind Dumbare + * + * Copyright (c) 2017 - 2018 ProfitBricks GmbH. All rights reserved. + * Authors: Danil Kipnis + * Roman Penyaev + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version 2 + * of the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, see . + */ + +#undef pr_fmt +#define pr_fmt(fmt) KBUILD_MODNAME " L" __stringify(__LINE__) ": " fmt + +#include "ibtrs-pri.h" +#include "ibtrs-srv.h" +#include "ibtrs-log.h" + +extern struct class *ibtrs_dev_class; + +static struct kobj_type ktype = { + .sysfs_ops = &kobj_sysfs_ops, +}; + +static ssize_t ibtrs_srv_disconnect_show(struct kobject *kobj, + struct kobj_attribute *attr, + char *page) +{ + return scnprintf(page, PAGE_SIZE, "Usage: echo 1 > %s\n", + attr->attr.name); +} + +static ssize_t ibtrs_srv_disconnect_store(struct kobject *kobj, + struct kobj_attribute *attr, + const char *buf, size_t count) +{ + struct ibtrs_srv_sess *sess; + char str[MAXHOSTNAMELEN]; + + sess = container_of(kobj, struct ibtrs_srv_sess, kobj); + if (!sysfs_streq(buf, "1")) { + ibtrs_err(sess, "%s: invalid value: '%s'\n", + attr->attr.name, buf); + return -EINVAL; + } + + sockaddr_to_str((struct sockaddr *)&sess->s.dst_addr, str, sizeof(str)); + + ibtrs_info(sess, "disconnect for path %s requested\n", str); + ibtrs_srv_queue_close(sess); + + return count; +} + +static struct kobj_attribute ibtrs_srv_disconnect_attr = + __ATTR(disconnect, 0644, + ibtrs_srv_disconnect_show, ibtrs_srv_disconnect_store); + +static ssize_t ibtrs_srv_hca_port_show(struct kobject *kobj, + struct kobj_attribute *attr, + char *page) +{ + struct ibtrs_srv_sess *sess; + struct ibtrs_con *usr_con; + + sess = container_of(kobj, typeof(*sess), kobj); + usr_con = sess->s.con[0]; + + return scnprintf(page, PAGE_SIZE, "%u\n", + usr_con->cm_id->port_num); +} + +static struct kobj_attribute ibtrs_srv_hca_port_attr = + __ATTR(hca_port, 0444, ibtrs_srv_hca_port_show, NULL); + +static ssize_t ibtrs_srv_hca_name_show(struct kobject *kobj, + struct kobj_attribute *attr, + char *page) +{ + struct ibtrs_srv_sess *sess; + + sess = container_of(kobj, struct ibtrs_srv_sess, kobj); + + return scnprintf(page, PAGE_SIZE, "%s\n", + sess->s.dev->ib_dev->name); +} + +static struct kobj_attribute ibtrs_srv_hca_name_attr = + __ATTR(hca_name, 0444, ibtrs_srv_hca_name_show, NULL); + +static struct attribute *ibtrs_srv_sess_attrs[] = { + &ibtrs_srv_hca_name_attr.attr, + &ibtrs_srv_hca_port_attr.attr, + &ibtrs_srv_disconnect_attr.attr, + NULL, +}; + +static struct attribute_group ibtrs_srv_sess_attr_group = { + .attrs = ibtrs_srv_sess_attrs, +}; + +STAT_ATTR(struct ibtrs_srv_sess, rdma, + ibtrs_srv_stats_rdma_to_str, + ibtrs_srv_reset_rdma_stats); + +STAT_ATTR(struct ibtrs_srv_sess, wc_completion, + ibtrs_srv_stats_wc_completion_to_str, + ibtrs_srv_reset_wc_completion_stats); + +STAT_ATTR(struct ibtrs_srv_sess, reset_all, + ibtrs_srv_reset_all_help, + ibtrs_srv_reset_all_stats); + +static struct attribute *ibtrs_srv_stats_attrs[] = { + &rdma_attr.attr, + &wc_completion_attr.attr, + &reset_all_attr.attr, + NULL, +}; + +static struct attribute_group ibtrs_srv_stats_attr_group = { + .attrs = ibtrs_srv_stats_attrs, +}; + +static void ibtrs_srv_dev_release(struct device *dev) +{ + /* Nobody plays with device references, so nop */ +} + +static int ibtrs_srv_create_once_sysfs_root_folders(struct ibtrs_srv_sess *sess) +{ + struct ibtrs_srv *srv = sess->srv; + int err = 0; + + mutex_lock(&srv->paths_mutex); + if (srv->dev_ref++) { + /* + * Just increase device reference. We can't use get_device() + * because we need to unregister device when ref goes to 0, + * not just to put it. + */ + goto unlock; + } + srv->dev.class = ibtrs_dev_class; + srv->dev.release = ibtrs_srv_dev_release; + dev_set_name(&srv->dev, "%s", sess->s.sessname); + + err = device_register(&srv->dev); + if (unlikely(err)) { + pr_err("device_register(): %d\n", err); + goto unlock; + } + err = kobject_init_and_add(&srv->kobj_paths, &ktype, + &srv->dev.kobj, "paths"); + if (unlikely(err)) { + pr_err("kobject_init_and_add(): %d\n", err); + device_unregister(&srv->dev); + goto unlock; + } +unlock: + mutex_unlock(&srv->paths_mutex); + + return err; +} + +static void ibtrs_srv_destroy_once_sysfs_root_folders(struct ibtrs_srv_sess *sess) +{ + struct ibtrs_srv *srv = sess->srv; + + mutex_lock(&srv->paths_mutex); + if (!--srv->dev_ref) { + kobject_put(&srv->kobj_paths); + device_unregister(&srv->dev); + } + mutex_unlock(&srv->paths_mutex); +} + +static int ibtrs_srv_create_stats_files(struct ibtrs_srv_sess *sess) +{ + int err; + + err = kobject_init_and_add(&sess->kobj_stats, &ktype, + &sess->kobj, "stats"); + if (unlikely(err)) { + ibtrs_err(sess, "kobject_init_and_add(): %d\n", err); + return err; + } + err = sysfs_create_group(&sess->kobj_stats, + &ibtrs_srv_stats_attr_group); + if (unlikely(err)) { + ibtrs_err(sess, "sysfs_create_group(): %d\n", err); + goto err; + } + + return 0; + +err: + kobject_put(&sess->kobj_stats); + + return err; +} + +int ibtrs_srv_create_sess_files(struct ibtrs_srv_sess *sess) +{ + struct ibtrs_srv *srv = sess->srv; + char str[MAXHOSTNAMELEN]; + int err; + + sockaddr_to_str((struct sockaddr *)&sess->s.dst_addr, str, sizeof(str)); + + err = ibtrs_srv_create_once_sysfs_root_folders(sess); + if (unlikely(err)) + return err; + + err = kobject_init_and_add(&sess->kobj, &ktype, &srv->kobj_paths, + "%s", str); + if (unlikely(err)) { + ibtrs_err(sess, "kobject_init_and_add(): %d\n", err); + goto destroy_root; + } + err = sysfs_create_group(&sess->kobj, &ibtrs_srv_sess_attr_group); + if (unlikely(err)) { + ibtrs_err(sess, "sysfs_create_group(): %d\n", err); + goto put_kobj; + } + err = ibtrs_srv_create_stats_files(sess); + if (unlikely(err)) + goto remove_group; + + return 0; + +remove_group: + sysfs_remove_group(&sess->kobj, &ibtrs_srv_sess_attr_group); +put_kobj: + kobject_del(&sess->kobj); + kobject_put(&sess->kobj); +destroy_root: + ibtrs_srv_destroy_once_sysfs_root_folders(sess); + + return err; +} + +void ibtrs_srv_destroy_sess_files(struct ibtrs_srv_sess *sess) +{ + if (sess->kobj.state_in_sysfs) { + kobject_del(&sess->kobj_stats); + kobject_put(&sess->kobj_stats); + kobject_del(&sess->kobj); + kobject_put(&sess->kobj); + + ibtrs_srv_destroy_once_sysfs_root_folders(sess); + } +}