From patchwork Fri Mar 24 10:45:26 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 9642607 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8D4946020B for ; Fri, 24 Mar 2017 10:48:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A471527F07 for ; Fri, 24 Mar 2017 10:48:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 993D928236; Fri, 24 Mar 2017 10:48:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0D4632823D for ; Fri, 24 Mar 2017 10:48:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756977AbdCXKr6 (ORCPT ); Fri, 24 Mar 2017 06:47:58 -0400 Received: from mail-wr0-f169.google.com ([209.85.128.169]:34245 "EHLO mail-wr0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S935703AbdCXKrs (ORCPT ); Fri, 24 Mar 2017 06:47:48 -0400 Received: by mail-wr0-f169.google.com with SMTP id l43so6748039wre.1 for ; Fri, 24 Mar 2017 03:47:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=profitbricks-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=iYKLhDoKJcRCvhaJlak8B0s/r2L0kjnOgPgD8MXrA/w=; b=DZ3N/M39c1ywydqoIMG4HkFPpGt8M2L8ou1MtAVLCEVkxyggXYrld/ntBjmcp+h1fV Fn2GtDpu8AMpkUR2eta+psY2+XMEcVM2qJzjg5DKBaCEnAFOogpO89ixRlzJUzoYH5/M FBjpJ4A2AYqLQ/iIxCEDdHqOaLhZKWeIseU9Oc1eF8iKMPUs3y7WabcM0ilP8NTFSV89 319Kb9xAFr29O/zlKmgFws/VtdoAY1z3H/W2pJfcI2DQSTMaHmR9MzaNd9JzbWrRXdoH T/FypSldhTvhoLysvushL55IGmwngaLx2tksr+hZC2k7Zu8cL2CXgl2tL94UIIO1tALg Bv6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=iYKLhDoKJcRCvhaJlak8B0s/r2L0kjnOgPgD8MXrA/w=; b=NqjA2j0uDDhbRmviZXiIato1A2D6d8+gwzqIRSlNJoLoNaUfwrFj9AA+x1Gql0T2OE 3Dv7bshG5WvLklqBq6pIOYe8N+TxWdABmP3HfTtENNZIMnez6DCHbF+AWQ++jNGzA8k5 6B+tRDjjKJ582yyRILYN/S6vWxGaUblKuq6uJnqTt6Gkd31p1K892qsxeCk9YXfWJ1B+ gXdz9tAjZPyrFVWze/WskyVMEFZe9nJb9PhUXZPWs6nI7DVCF7dZezvODcH3UjrUwZov cIhOdxQrimEu9EX7wpBOODh/rRwg36lL3nHPXTe7lPs7pUFg6lZvW92yOydXg8L7RgBe NiPA== X-Gm-Message-State: AFeK/H1y+jGKnQ64Sh/rwbNOPn0JKohxhS6mWmAK83jKwhVMUo59J5iCx86NtmKh28vzZ8gX X-Received: by 10.223.133.228 with SMTP id 33mr7817395wru.0.1490352461349; Fri, 24 Mar 2017 03:47:41 -0700 (PDT) Received: from jinpu-GA-870A-USB3.pb.local ([62.217.45.26]) by smtp.gmail.com with ESMTPSA id s17sm2404458wrc.25.2017.03.24.03.47.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 24 Mar 2017 03:47:40 -0700 (PDT) From: Jack Wang X-Google-Original-From: Jack Wang To: linux-block@vger.kernel.org, linux-rdma@vger.kernel.org Cc: dledford@redhat.com, axboe@kernel.dk, hch@lst.de, mail@fholler.de, Milind.dumbare@gmail.com, yun.wang@profitbricks.com, Jack Wang , Kleber Souza , Danil Kipnis , Roman Pen Subject: [PATCH 11/28] ibtrs_srv: add header shared in ibtrs_server Date: Fri, 24 Mar 2017 11:45:26 +0100 Message-Id: <1490352343-20075-12-git-send-email-jinpu.wangl@profitbricks.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1490352343-20075-1-git-send-email-jinpu.wangl@profitbricks.com> References: <1490352343-20075-1-git-send-email-jinpu.wangl@profitbricks.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jack Wang Signed-off-by: Jack Wang Signed-off-by: Kleber Souza Signed-off-by: Danil Kipnis Signed-off-by: Roman Pen --- .../ulp/ibtrs_server/ibtrs_srv_internal.h | 201 +++++++++++++++++++++ 1 file changed, 201 insertions(+) create mode 100644 drivers/infiniband/ulp/ibtrs_server/ibtrs_srv_internal.h diff --git a/drivers/infiniband/ulp/ibtrs_server/ibtrs_srv_internal.h b/drivers/infiniband/ulp/ibtrs_server/ibtrs_srv_internal.h new file mode 100644 index 0000000..79130a1 --- /dev/null +++ b/drivers/infiniband/ulp/ibtrs_server/ibtrs_srv_internal.h @@ -0,0 +1,201 @@ +/* + * InfiniBand Transport Layer + * + * Copyright (c) 2014 - 2017 ProfitBricks GmbH. All rights reserved. + * Authors: Fabian Holler < mail@fholler.de> + * Jack Wang + * Kleber Souza + * Danil Kipnis + * Roman Pen + * Milind Dumbare + * + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions, and the following disclaimer, + * without modification. + * 2. Redistributions in binary form must reproduce at minimum a disclaimer + * substantially similar to the "NO WARRANTY" disclaimer below + * ("Disclaimer") and any redistribution must be conditioned upon + * including a substantially similar Disclaimer requirement for further + * binary redistribution. + * 3. Neither the names of the above-listed copyright holders nor the names + * of any contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * Alternatively, this software may be distributed under the terms of the + * GNU General Public License ("GPL") version 2 as published by the Free + * Software Foundation. + * + * NO WARRANTY + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING + * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGES. + * + */ + +#ifndef _IBTRS_SRV_INTERNAL_H +#define _IBTRS_SRV_INTERNAL_H + +#include + +enum ssm_state { + SSM_STATE_IDLE, + SSM_STATE_CONNECTED, + SSM_STATE_CLOSING, + SSM_STATE_CLOSED +}; + +/* + * Describes the rdma buffer managed by client and used for his rdma writes + * Rdma info has to be sent in OPEN_RESP message to the client. + */ +struct ibtrs_rcv_buf { + dma_addr_t rdma_addr; + void *buf; +}; + +/* to indicate that memory chunk was not allocated from a N-order contiguous + * pages area + */ +#define IBTRS_MEM_CHUNK_NOORDER -1 + +struct ibtrs_mem_chunk { + struct list_head list; + int order; + void *addr; +}; + +struct ibtrs_rcv_buf_pool { + struct list_head list; + struct list_head chunk_list; + struct ibtrs_rcv_buf *rcv_bufs; +}; + +struct ibtrs_stats_wc_comp { + atomic_t max_wc_cnt; + atomic64_t calls; + atomic64_t total_wc_cnt; +}; + +struct ibtrs_srv_stats_rdma_stats { + atomic64_t cnt_read; + atomic64_t size_total_read; + atomic64_t cnt_write; + atomic64_t size_total_write; + + atomic_t inflight; + atomic64_t inflight_total; +}; + +struct ibtrs_srv_stats_user_ib_msgs { + atomic64_t recv_msg_cnt; + atomic64_t sent_msg_cnt; + atomic64_t recv_size; + atomic64_t sent_size; +}; + +struct ibtrs_srv_stats { + struct ibtrs_srv_stats_rdma_stats rdma_stats; + struct ibtrs_srv_stats_user_ib_msgs user_ib_msgs; + atomic_t apm_cnt; + struct ibtrs_stats_wc_comp wc_comp; +}; + +struct ibtrs_session { + struct list_head list; + enum ssm_state state; + struct kref kref; + struct workqueue_struct *sm_wq; /* event processing */ + struct workqueue_struct *msg_wq; + struct ibtrs_device *dev; /* ib dev with mempool */ + struct rdma_cm_id *cm_id; /* cm_id used to create the session */ + struct mutex lock; /* to protect con_list */ + int cur_cq_vector; + struct list_head con_list; + struct ibtrs_iu *rdma_info_iu; + struct ibtrs_iu *dummy_rx_iu; + struct ibtrs_iu **usr_rx_ring; + struct ibtrs_ops_id **ops_ids; + /* lock for tx_bufs */ + spinlock_t tx_bufs_lock ____cacheline_aligned; + struct list_head tx_bufs; + u16 tx_bufs_used; + unsigned int est_cnt; /* number of established connections */ + unsigned int active_cnt; /* number of active (not closed) + * connections + */ + u8 con_cnt; + u8 ver; + bool state_in_sysfs; + bool session_announced_to_user; + struct ibtrs_rcv_buf_pool *rcv_buf_pool; + wait_queue_head_t bufs_wait; + u8 off_len; /* number of bits for offset in + * one client buffer. + * 32 - ilog2(sess->queue_depth) + */ + u32 off_mask; /* mask to get offset in client buf + * out of the imm field + */ + u16 queue_depth; + u16 wq_size; + u8 uuid[IBTRS_UUID_SIZE]; + struct ibtrs_heartbeat heartbeat; + struct delayed_work check_heartbeat_dwork; + struct delayed_work send_heartbeat_dwork; + void *priv; + struct kobject kobj; + struct kobject kobj_stats; + char addr[IBTRS_ADDRLEN]; /* client address */ + char hostname[MAXHOSTNAMELEN]; + u8 primary_port_num; + struct ibtrs_srv_stats stats; + wait_queue_head_t mu_iu_wait_q; + wait_queue_head_t mu_buf_wait_q; + atomic_t peer_usr_msg_bufs; +}; + +void ibtrs_srv_queue_close(struct ibtrs_session *sess); + +u8 ibtrs_srv_get_sess_primary_port_num(struct ibtrs_session *sess); + +int ibtrs_srv_current_hca_port_to_str(struct ibtrs_session *sess, + char *buf, size_t len); +int ibtrs_srv_failover_hca_port_to_str(struct ibtrs_session *sess, + char *buf, size_t len); +const char *ibtrs_srv_get_sess_hca_name(struct ibtrs_session *sess); +int ibtrs_srv_migrate(struct ibtrs_session *sess, u8 port_num); +int ibtrs_srv_reset_rdma_stats(struct ibtrs_session *sess, bool enable); +ssize_t ibtrs_srv_stats_rdma_to_str(struct ibtrs_session *sess, + char *page, size_t len); +int ibtrs_srv_reset_user_ib_msgs_stats(struct ibtrs_session *sess, bool enable); +int ibtrs_srv_stats_user_ib_msgs_to_str(struct ibtrs_session *sess, char *buf, + size_t len); +int ibtrs_srv_reset_apm_stats(struct ibtrs_session *sess, bool enable); +int ibtrs_srv_stats_apm_to_str(struct ibtrs_session *sess, char *buf, + size_t len); +int ibtrs_srv_reset_wc_completion_stats(struct ibtrs_session *sess, + bool enable); +int ibtrs_srv_stats_wc_completion_to_str(struct ibtrs_session *sess, char *buf, + size_t len); +int ibtrs_srv_reset_all_stats(struct ibtrs_session *sess, bool enable); +ssize_t ibtrs_srv_reset_all_help(struct ibtrs_session *sess, + char *page, size_t len); +int heartbeat_timeout_validate(int timeout); + +int ibtrs_srv_sess_get(struct ibtrs_session *sess); + +void ibtrs_srv_sess_put(struct ibtrs_session *sess); + +#endif