From patchwork Sun Dec 12 15:07:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12672303 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 58AB3C433FE for ; Sun, 12 Dec 2021 15:08:13 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id DDB1921F493; Sun, 12 Dec 2021 07:08:10 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C64E521CBC4 for ; Sun, 12 Dec 2021 07:08:06 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id B139810084B2; Sun, 12 Dec 2021 10:08:04 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id A3389E07E3; Sun, 12 Dec 2021 10:08:04 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 12 Dec 2021 10:07:52 -0500 Message-Id: <1639321683-22909-2-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> References: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 01/12] lustre: llite: do not take mod rpc slot for getxattr X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vladimir Saveliev , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Vladimir Saveliev The following scenario may lead to client eviction: clientA clientB MDS threadA1: write to file F1, get and hold DoM MDC LDLM lock L1: ->cl_io_loop() ->cl_io_lock() : ->mdc_lock_granted() ->lock->l_writers++ [hold ref until write done] threadA2-A8: create files F2-F8: ->ll_file_open() ->mdc_enqueue_base() ->ldlm_cli_enqueue() ->ptlrpc_get_mod_rpc_slot() ->ptlrpc_queue_wait() [hold RPC slot until create done] OST(s) in recovery. MDS waiting on OST(s) to precreate new objects. threadA1: -> cl_io_start() -> __generic_file_aio_write() -> file_remove_suid() -> ll_xattr_cache_refill() -> mdc_xattr_common() -> ptlrpc_get_mod_rpc_slot() [blocked waiting for RPC slot] threadB1: write file F1, enqueue DoM MDC lock L1 MDS sends blocking AST to clientA for lock L1 ldlm_threadA3: cannot cancel busy lock L1: -> ldlm_handle_bl_callback() ["Lock L1 referenced, will be cancelled later"] MDS evicts clientA for not cancelling lock L1 threadA1: never completes write: ->cl_io_end() ->cl_io_unlock() ->osc_lock_cancel() ->lock->l_writers--; The fix is to add IT_GETXATTR to list of operations which do not need mod rpc slot. Tests to illustrate the issue is added. wait_for_function(): total sleep time (wait) is to be equal to max when 1 is returned. HPE-bug-id: LUS-7271 WC-bug-id: https://jira.whamcloud.com/browse/LU-12347 Lustre-commit: eb64594e4473af85 ("LU-12347 llite: do not take mod rpc slot for getxattr") Signed-off-by: Vladimir Saveliev Reviewed-on: https://review.whamcloud.com/44151 Reviewed-by: Andreas Dilger Reviewed-by: Andrew Perepechko Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/obd_support.h | 1 + fs/lustre/llite/xattr_cache.c | 2 ++ fs/lustre/mdc/mdc_locks.c | 2 +- 3 files changed, 4 insertions(+), 1 deletion(-) diff --git a/fs/lustre/include/obd_support.h b/fs/lustre/include/obd_support.h index 540e1e0..d57c25c 100644 --- a/fs/lustre/include/obd_support.h +++ b/fs/lustre/include/obd_support.h @@ -484,6 +484,7 @@ #define OBD_FAIL_LLITE_RACE_MOUNT 0x1417 #define OBD_FAIL_LLITE_PAGE_ALLOC 0x1418 #define OBD_FAIL_LLITE_OPEN_DELAY 0x1419 +#define OBD_FAIL_LLITE_XATTR_PAUSE 0x1420 #define OBD_FAIL_FID_INDIR 0x1501 #define OBD_FAIL_FID_INLMA 0x1502 diff --git a/fs/lustre/llite/xattr_cache.c b/fs/lustre/llite/xattr_cache.c index b044c89..7c1f5b7 100644 --- a/fs/lustre/llite/xattr_cache.c +++ b/fs/lustre/llite/xattr_cache.c @@ -396,6 +396,8 @@ static int ll_xattr_cache_refill(struct inode *inode) u32 *xsizes; int rc, i; + CFS_FAIL_TIMEOUT(OBD_FAIL_LLITE_XATTR_PAUSE, cfs_fail_val ?: 2); + rc = ll_xattr_find_get_lock(inode, &oit, &req); if (rc) goto err_req; diff --git a/fs/lustre/mdc/mdc_locks.c b/fs/lustre/mdc/mdc_locks.c index 66f0039..2c344d7 100644 --- a/fs/lustre/mdc/mdc_locks.c +++ b/fs/lustre/mdc/mdc_locks.c @@ -886,7 +886,7 @@ static inline bool mdc_skip_mod_rpc_slot(const struct lookup_intent *it) { if (it && (it->it_op == IT_GETATTR || it->it_op == IT_LOOKUP || - it->it_op == IT_READDIR || + it->it_op == IT_READDIR || it->it_op == IT_GETXATTR || (it->it_op == IT_LAYOUT && !(it->it_flags & MDS_FMODE_WRITE)))) return true; return false; From patchwork Sun Dec 12 15:07:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12672301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1D73FC433F5 for ; Sun, 12 Dec 2021 15:08:14 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8241321F4A2; Sun, 12 Dec 2021 07:08:11 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3566721CBC4 for ; Sun, 12 Dec 2021 07:08:07 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id B524710084BE; Sun, 12 Dec 2021 10:08:04 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id A44C7E07E5; Sun, 12 Dec 2021 10:08:04 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 12 Dec 2021 10:07:53 -0500 Message-Id: <1639321683-22909-3-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> References: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 02/12] lnet: uapi: move out kernel only code. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" Userland doesn't use the new IPv6 function in the UAPI nidstr.h. So move this to the kernel header lib-types.h. Normally this wouldn't matter but the python wrappers break with the LUTF project. With the move to Netlink in the future for the user land API we shouldn't need these functions anyways. WC-bug-id: https://jira.whamcloud.com/browse/LU-10391 Lustre-commit: 0631caae7d804720d ("LU-10391 uapi: move out kernel only code.") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/44844 Reviewed-by: Neil Brown Reviewed-by: Chris Horn Reviewed-by: Amir Shehata Reviewed-by: Oleg Drokin --- include/linux/lnet/lib-types.h | 16 ++++++++++++++++ include/uapi/linux/lnet/nidstr.h | 13 ------------- 2 files changed, 16 insertions(+), 13 deletions(-) diff --git a/include/linux/lnet/lib-types.h b/include/linux/lnet/lib-types.h index 7631044..eb736a5 100644 --- a/include/linux/lnet/lib-types.h +++ b/include/linux/lnet/lib-types.h @@ -51,6 +51,22 @@ #include #include +char *libcfs_nidstr_r(const struct lnet_nid *nid, + char *buf, size_t buf_size); + +static inline char *libcfs_nidstr(const struct lnet_nid *nid) +{ + return libcfs_nidstr_r(nid, libcfs_next_nidstring(), + LNET_NIDSTR_SIZE); +} + +int libcfs_strnid(struct lnet_nid *nid, const char *str); +char *libcfs_idstr(struct lnet_processid *id); + +int cfs_match_nid_net(struct lnet_nid *nid, u32 net, + struct list_head *net_num_list, + struct list_head *addr); + /* Max payload size */ #define LNET_MAX_PAYLOAD LNET_MTU diff --git a/include/uapi/linux/lnet/nidstr.h b/include/uapi/linux/lnet/nidstr.h index 482cfb2..80be2eb 100644 --- a/include/uapi/linux/lnet/nidstr.h +++ b/include/uapi/linux/lnet/nidstr.h @@ -90,27 +90,14 @@ static inline char *libcfs_nid2str(lnet_nid_t nid) LNET_NIDSTR_SIZE); } -char *libcfs_nidstr_r(const struct lnet_nid *nid, - char *buf, __kernel_size_t buf_size); -static inline char *libcfs_nidstr(const struct lnet_nid *nid) -{ - return libcfs_nidstr_r(nid, libcfs_next_nidstring(), - LNET_NIDSTR_SIZE); -} - __u32 libcfs_str2net(const char *str); lnet_nid_t libcfs_str2nid(const char *str); -int libcfs_strnid(struct lnet_nid *nid, const char *str); int libcfs_str2anynid(lnet_nid_t *nid, const char *str); char *libcfs_id2str(struct lnet_process_id id); -char *libcfs_idstr(struct lnet_processid *id); void cfs_free_nidlist(struct list_head *list); int cfs_parse_nidlist(char *str, int len, struct list_head *list); int cfs_print_nidlist(char *buffer, int count, struct list_head *list); int cfs_match_nid(lnet_nid_t nid, struct list_head *list); -int cfs_match_nid_net(struct lnet_nid *nid, __u32 net, - struct list_head *net_num_list, - struct list_head *addr); int cfs_match_net(__u32 net_id, __u32 net_type, struct list_head *net_num_list); From patchwork Sun Dec 12 15:07:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12672321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EB883C433F5 for ; Sun, 12 Dec 2021 15:09:18 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4276221F549; Sun, 12 Dec 2021 07:08:29 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6C6BD21CBC4 for ; Sun, 12 Dec 2021 07:08:07 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id B6DA610084C0; Sun, 12 Dec 2021 10:08:04 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id A93CBE07E6; Sun, 12 Dec 2021 10:08:04 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 12 Dec 2021 10:07:54 -0500 Message-Id: <1639321683-22909-4-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> References: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 03/12] lustre: ptlrpc: Do not unlink difficult reply until sent X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Chris Horn , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Chris Horn If a difficult reply is queued in LNet, or the PUT for it is otherwise delayed, then it is possible for the commit callback to unlink the reply MD which will abort the send. This results in client hitting "slow reply" timeout for the associated RPC and an unnecessary reconnect (and possibly resend). This patch replaces the rs_on_net flag with rs_sent and rs_unlinked. These flags indicate whether the send event for the reply MD has been generated, and whether the MD has been unlinked, respectively. If rs_sent is set, but rs_unlinked has not been set, then ptlrpc_hr is free to unlink the reply MD as a result of the commit callback. The reply-ack will simply be dropped by the server. If ptlrpc_hr is processing the reply because of commit callback, and rs_sent has not been set, then ptlrpc_hr will not unlink the reply MD. This means that the reply_out_callback must also be modified to check for this case when the send event occurs. Otherwise, if the ACK never arrives from the client, then the MD would never be unlinked. Thus when the send event occurs, and rs_handled is set, the reply_out_callback will schedule the reply for handling by ptlrpc_hr. HPE-bug-id: LUS-10505 WC-bug-id: https://jira.whamcloud.com/browse/LU-15068 Lustre-commit: 5c156b48425aae245 ("LU-15068 ptlrpc: Do not unlink difficult reply until sent") Signed-off-by: Chris Horn Reviewed-on: https://review.whamcloud.com/45138 Reviewed-by: Andriy Skulysh Reviewed-by: Alexey Lyashkov Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_net.h | 3 ++- fs/lustre/ldlm/ldlm_lib.c | 11 +++++++---- fs/lustre/ptlrpc/events.c | 19 ++++++++++++++++--- fs/lustre/ptlrpc/pack_generic.c | 2 +- fs/lustre/ptlrpc/service.c | 10 +++++++--- 5 files changed, 33 insertions(+), 12 deletions(-) diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index f72f7c6..78df59b 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -492,7 +492,8 @@ struct ptlrpc_reply_state { unsigned long rs_scheduled:1; /* being handled? */ unsigned long rs_scheduled_ever:1; /* any schedule attempts? */ unsigned long rs_handled:1; /* been handled yet? */ - unsigned long rs_on_net:1; /* reply_out_callback pending? */ + unsigned long rs_sent:1; /* Got LNET_EVENT_SEND? */ + unsigned long rs_unlinked:1; /* Reply MD unlinked? */ unsigned long rs_prealloc:1; /* rs from prealloc list */ unsigned long rs_committed:1; /* the transaction was committed * and the rs was dispatched diff --git a/fs/lustre/ldlm/ldlm_lib.c b/fs/lustre/ldlm/ldlm_lib.c index 90bad71..9aa87d1 100644 --- a/fs/lustre/ldlm/ldlm_lib.c +++ b/fs/lustre/ldlm/ldlm_lib.c @@ -813,7 +813,8 @@ void target_send_reply(struct ptlrpc_request *req, int rc, int fail_id) LASSERT(!rs->rs_scheduled); LASSERT(!rs->rs_scheduled_ever); LASSERT(!rs->rs_handled); - LASSERT(!rs->rs_on_net); + LASSERT(!rs->rs_sent); + LASSERT(!rs->rs_unlinked); LASSERT(!rs->rs_export); LASSERT(list_empty(&rs->rs_obd_list)); LASSERT(list_empty(&rs->rs_exp_list)); @@ -822,7 +823,8 @@ void target_send_reply(struct ptlrpc_request *req, int rc, int fail_id) /* disable reply scheduling while I'm setting up */ rs->rs_scheduled = 1; - rs->rs_on_net = 1; + rs->rs_sent = 0; + rs->rs_unlinked = 0; rs->rs_xid = req->rq_xid; rs->rs_transno = req->rq_transno; rs->rs_export = exp; @@ -856,13 +858,14 @@ void target_send_reply(struct ptlrpc_request *req, int rc, int fail_id) * would have been +1 ref for the net, which * reply_out_callback leaves alone) */ - rs->rs_on_net = 0; + rs->rs_sent = 1; + rs->rs_unlinked = 1; ptlrpc_rs_addref(rs); } spin_lock(&rs->rs_lock); if (rs->rs_transno <= exp->exp_last_committed || - (!rs->rs_on_net && !rs->rs_no_ack) || + (rs->rs_unlinked && !rs->rs_no_ack) || list_empty(&rs->rs_exp_list) || /* completed already */ list_empty(&rs->rs_obd_list)) { CDEBUG(D_HA, "Schedule reply immediately\n"); diff --git a/fs/lustre/ptlrpc/events.c b/fs/lustre/ptlrpc/events.c index 559d811..dbf9f9d 100644 --- a/fs/lustre/ptlrpc/events.c +++ b/fs/lustre/ptlrpc/events.c @@ -401,6 +401,7 @@ void reply_out_callback(struct lnet_event *ev) struct ptlrpc_cb_id *cbid = ev->md_user_ptr; struct ptlrpc_reply_state *rs = cbid->cbid_arg; struct ptlrpc_service_part *svcpt = rs->rs_svcpt; + bool need_schedule = false; LASSERT(ev->type == LNET_EVENT_SEND || ev->type == LNET_EVENT_ACK || @@ -415,16 +416,28 @@ void reply_out_callback(struct lnet_event *ev) return; } - LASSERT(rs->rs_on_net); + if (ev->type == LNET_EVENT_SEND) { + spin_lock(&rs->rs_lock); + rs->rs_sent = 1; + /* If transaction was committed before the SEND, and the ACK + * is lost, then we need to schedule so ptlrpc_hr can unlink + * the MD. + */ + if (rs->rs_handled) + need_schedule = true; + spin_unlock(&rs->rs_lock); + } + + if (ev->unlinked || need_schedule) { + LASSERT(rs->rs_sent); - if (ev->unlinked) { /* Last network callback. The net's ref on 'rs' stays put * until ptlrpc_handle_rs() is done with it */ spin_lock(&svcpt->scp_rep_lock); spin_lock(&rs->rs_lock); - rs->rs_on_net = 0; + rs->rs_unlinked = ev->unlinked; if (!rs->rs_no_ack || rs->rs_transno <= rs->rs_export->exp_obd->obd_last_committed || diff --git a/fs/lustre/ptlrpc/pack_generic.c b/fs/lustre/ptlrpc/pack_generic.c index 62e060d..23b36de 100644 --- a/fs/lustre/ptlrpc/pack_generic.c +++ b/fs/lustre/ptlrpc/pack_generic.c @@ -458,7 +458,7 @@ void lustre_free_reply_state(struct ptlrpc_reply_state *rs) LASSERT(atomic_read(&rs->rs_refcount) == 0); LASSERT(!rs->rs_difficult || rs->rs_handled); - LASSERT(!rs->rs_on_net); + LASSERT(!rs->rs_difficult || rs->rs_unlinked); LASSERT(!rs->rs_scheduled); LASSERT(!rs->rs_export); LASSERT(rs->rs_nlocks == 0); diff --git a/fs/lustre/ptlrpc/service.c b/fs/lustre/ptlrpc/service.c index 2917ca3..dbe2347 100644 --- a/fs/lustre/ptlrpc/service.c +++ b/fs/lustre/ptlrpc/service.c @@ -1940,10 +1940,14 @@ static int ptlrpc_handle_rs(struct ptlrpc_reply_state *rs) libcfs_nid2str(exp->exp_connection->c_peer.nid)); } - if ((!been_handled && rs->rs_on_net) || nlocks > 0) { + if ((rs->rs_sent && !rs->rs_unlinked) || nlocks > 0) { spin_unlock(&rs->rs_lock); - if (!been_handled && rs->rs_on_net) { + /* We can unlink if the LNET_EVENT_SEND has occurred. + * If rs_unlinked is set then MD is already unlinked and no + * need to do so here. + */ + if ((rs->rs_sent && !rs->rs_unlinked)) { LNetMDUnlink(rs->rs_md_h); /* Ignore return code; we're racing with completion */ } @@ -1957,7 +1961,7 @@ static int ptlrpc_handle_rs(struct ptlrpc_reply_state *rs) rs->rs_scheduled = 0; - if (!rs->rs_on_net) { + if (rs->rs_unlinked) { /* Off the net */ spin_unlock(&rs->rs_lock); From patchwork Sun Dec 12 15:07:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12672315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 91B37C433F5 for ; Sun, 12 Dec 2021 15:09:02 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3D19E21F492; Sun, 12 Dec 2021 07:08:22 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B7B7E21CBFF for ; Sun, 12 Dec 2021 07:08:07 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id B7FF810084CE; Sun, 12 Dec 2021 10:08:04 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id AC45CE080D; Sun, 12 Dec 2021 10:08:04 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 12 Dec 2021 10:07:55 -0500 Message-Id: <1639321683-22909-5-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> References: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 04/12] lustre: obdclass: make niduuid for lustre_stop_mgc() static X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" The process to create a proper string for niduuid can be made simpler and avoid a memory allocation. WC-bug-id: https://jira.whamcloud.com/browse/LU-9325 Lustre-commit: 85b400b67b0d8d493 ("LU-9325 obdclass: make niduuid for lustre_stop_mgc() static") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/33617 Reviewed-by: Arshad Hussain Reviewed-by: Andreas Dilger Reviewed-by: Alex Zhuravlev Reviewed-by: Oleg Drokin --- fs/lustre/obdclass/obd_mount.c | 25 ++++++++----------------- 1 file changed, 8 insertions(+), 17 deletions(-) diff --git a/fs/lustre/obdclass/obd_mount.c b/fs/lustre/obdclass/obd_mount.c index 19684fb..58ca72d 100644 --- a/fs/lustre/obdclass/obd_mount.c +++ b/fs/lustre/obdclass/obd_mount.c @@ -431,8 +431,8 @@ static int lustre_stop_mgc(struct super_block *sb) { struct lustre_sb_info *lsi = s2lsi(sb); struct obd_device *obd; - char *niduuid = NULL, *ptr = NULL; - int i, rc = 0, len = 0; + char niduuid[MAX_OBD_NAME + 6], *ptr = NULL; + int i, rc = 0; if (!lsi) return -ENOENT; @@ -467,24 +467,17 @@ static int lustre_stop_mgc(struct super_block *sb) CDEBUG(D_MOUNT, "disconnect failed %d\n", rc); } - /* Save the obdname for cleaning the nid uuids, which are obdname_XX */ - len = strlen(obd->obd_name) + 6; - niduuid = kzalloc(len, GFP_NOFS); - if (niduuid) { - strcpy(niduuid, obd->obd_name); - ptr = niduuid + strlen(niduuid); - } + /* + * Cache the obdname for cleaning the nid uuids, which are + * obdname_XX before calling class_manual_cleanup + */ + strcpy(niduuid, obd->obd_name); + ptr = niduuid + strlen(niduuid); rc = class_manual_cleanup(obd); if (rc) goto out; - /* Clean the nid uuids */ - if (!niduuid) { - rc = -ENOMEM; - goto out; - } - for (i = 0; i < lsi->lsi_lmd->lmd_mgs_failnodes; i++) { sprintf(ptr, "_%x", i); rc = do_lcfg(LUSTRE_MGC_OBDNAME, 0, LCFG_DEL_UUID, @@ -494,8 +487,6 @@ static int lustre_stop_mgc(struct super_block *sb) niduuid, rc); } out: - kfree(niduuid); - /* class_import_put will get rid of the additional connections */ mutex_unlock(&mgc_start_lock); return rc; From patchwork Sun Dec 12 15:07:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12672305 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4778DC433F5 for ; Sun, 12 Dec 2021 15:08:21 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 154B421CB7E; Sun, 12 Dec 2021 07:08:14 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3208421CBFF for ; Sun, 12 Dec 2021 07:08:08 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id BF19010084DD; Sun, 12 Dec 2021 10:08:04 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id AF3F6E080E; Sun, 12 Dec 2021 10:08:04 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 12 Dec 2021 10:07:56 -0500 Message-Id: <1639321683-22909-6-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> References: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 05/12] lnet: Allow specifying a source NID for lnetctl ping X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Chris Horn , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Chris Horn Add a new --source option for lnetctl ping command. This allows the user to specify a local NI from which to send the ping. This also ensures that the specified destination NID is also used. Otherwise, pings to multi-rail peers may end up going to a different peer NI based on the multi-rail selection algorithm. The ability to specify a source NI, and thus fix the destination NI, is a great help in troubleshooting communication issues between multi-rail peers. Add test to exercise lnetctl ping --source option. HPE-bug-id: LUS-10296 WC-bug-id: https://jira.whamcloud.com/browse/LU-14939 Lustre-commit: 48ef9982c474a02c4 ("LU-14939 lnet: Allow specifying a source NID for lnetctl ping") Signed-off-by: Chris Horn Reviewed-on: https://review.whamcloud.com/44727 Reviewed-by: Serguei Smirnov Reviewed-by: Andriy Skulysh Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/uapi/linux/lnet/lnet-dlc.h | 1 + net/lnet/lnet/api-ni.c | 29 ++++++++++++++++++++++------- 2 files changed, 23 insertions(+), 7 deletions(-) diff --git a/include/uapi/linux/lnet/lnet-dlc.h b/include/uapi/linux/lnet/lnet-dlc.h index 2ca70eb..415968a 100644 --- a/include/uapi/linux/lnet/lnet-dlc.h +++ b/include/uapi/linux/lnet/lnet-dlc.h @@ -132,6 +132,7 @@ struct lnet_ioctl_ping_data { __u32 mr_info; struct lnet_process_id ping_id; struct lnet_process_id __user *ping_buf; + lnet_nid_t ping_src; }; struct lnet_ioctl_config_data { diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index 3ed3f0b..550f035 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -205,8 +205,9 @@ static void lnet_set_lnd_timeout(void) */ static atomic_t lnet_dlc_seq_no = ATOMIC_INIT(0); -static int lnet_ping(struct lnet_process_id id, signed long timeout, - struct lnet_process_id __user *ids, int n_ids); +static int lnet_ping(struct lnet_process_id id, lnet_nid_t src_nid, + signed long timeout, struct lnet_process_id __user *ids, + int n_ids); static int lnet_discover(struct lnet_process_id id, u32 force, struct lnet_process_id __user *ids, int n_ids); @@ -4267,7 +4268,7 @@ u32 lnet_get_dlc_seq_locked(void) else timeout = msecs_to_jiffies(data->ioc_u32[1]); - rc = lnet_ping(id, timeout, data->ioc_pbuf1, + rc = lnet_ping(id, LNET_NID_ANY, timeout, data->ioc_pbuf1, data->ioc_plen1 / sizeof(struct lnet_process_id)); if (rc < 0) @@ -4281,6 +4282,19 @@ u32 lnet_get_dlc_seq_locked(void) struct lnet_ioctl_ping_data *ping = arg; struct lnet_peer *lp; signed long timeout; + lnet_nid_t src_nid = LNET_NID_ANY; + + /* Check if the supplied ping data supports source nid + * NB: This check is sufficient if lnet_ioctl_ping_data has + * additional fields added, but if they are re-ordered or + * fields removed then this will break. It is expected that + * these ioctls will be replaced with netlink implementation, so + * it is probably not worth coming up with a more robust version + * compatibility scheme. + */ + if (ping->ping_hdr.ioc_len >= + sizeof(struct lnet_ioctl_ping_data)) + src_nid = ping->ping_src; /* If timeout is negative then set default of 3 minutes */ if (((s32)ping->op_param) <= 0 || @@ -4289,7 +4303,7 @@ u32 lnet_get_dlc_seq_locked(void) else timeout = msecs_to_jiffies(ping->op_param); - rc = lnet_ping(ping->ping_id, timeout, + rc = lnet_ping(ping->ping_id, src_nid, timeout, ping->ping_buf, ping->ping_count); if (rc < 0) @@ -4526,8 +4540,9 @@ struct ping_data { complete(&pd->completion); } -static int lnet_ping(struct lnet_process_id id, signed long timeout, - struct lnet_process_id __user *ids, int n_ids) +static int lnet_ping(struct lnet_process_id id, lnet_nid_t src_nid, + signed long timeout, struct lnet_process_id __user *ids, + int n_ids) { struct lnet_md md = { NULL }; struct ping_data pd = { 0 }; @@ -4572,7 +4587,7 @@ static int lnet_ping(struct lnet_process_id id, signed long timeout, goto fail_ping_buffer_decref; } - rc = LNetGet(LNET_NID_ANY, pd.mdh, id, + rc = LNetGet(src_nid, pd.mdh, id, LNET_RESERVED_PORTAL, LNET_PROTO_PING_MATCHBITS, 0, false); if (rc) { From patchwork Sun Dec 12 15:07:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12672307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 66C02C433EF for ; Sun, 12 Dec 2021 15:08:25 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 2D56621CB8E; Sun, 12 Dec 2021 07:08:15 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EF14521CBFF for ; Sun, 12 Dec 2021 07:08:07 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id BE95D10084D8; Sun, 12 Dec 2021 10:08:04 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id B232DE080F; Sun, 12 Dec 2021 10:08:04 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 12 Dec 2021 10:07:57 -0500 Message-Id: <1639321683-22909-7-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> References: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 06/12] lnet: Fix source specified send to different net X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Chris Horn , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Chris Horn The destination NI is fixed for all source-specified sends. Thus, in order for a source-specified send to be considered "local", i.e. a send that does not require a route, the destination NID must be on the same net as the specified source. HPE-bug-id: LUS-10303 WC-bug-id: https://jira.whamcloud.com/browse/LU-14940 Lustre-commit: 3e3563f719ce89de2 ("LU-14940 lnet: Fix source specified send to different net") Signed-off-by: Chris Horn Reviewed-on: https://review.whamcloud.com/44728 Reviewed-by: Serguei Smirnov Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/lib-move.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/net/lnet/lnet/lib-move.c b/net/lnet/lnet/lib-move.c index 088a754..a411724 100644 --- a/net/lnet/lnet/lib-move.c +++ b/net/lnet/lnet/lib-move.c @@ -2733,15 +2733,19 @@ struct lnet_ni * /* Identify the different send cases */ - if (src_nid == LNET_NID_ANY) + if (src_nid == LNET_NID_ANY) { send_case |= SRC_ANY; - else + if (lnet_get_net_locked(LNET_NIDNET(dst_nid))) + send_case |= LOCAL_DST; + else + send_case |= REMOTE_DST; + } else { send_case |= SRC_SPEC; - - if (lnet_get_net_locked(LNET_NIDNET(dst_nid))) - send_case |= LOCAL_DST; - else - send_case |= REMOTE_DST; + if (LNET_NIDNET(src_nid) == LNET_NIDNET(dst_nid)) + send_case |= LOCAL_DST; + else + send_case |= REMOTE_DST; + } final_hop = false; if (msg->msg_routing && (send_case & LOCAL_DST)) From patchwork Sun Dec 12 15:07:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12672313 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 69EA7C433F5 for ; Sun, 12 Dec 2021 15:08:59 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C29AB21E07C; Sun, 12 Dec 2021 07:08:20 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 7C40721F46A for ; Sun, 12 Dec 2021 07:08:08 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id C33A110084E6; Sun, 12 Dec 2021 10:08:04 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id B52BBE0811; Sun, 12 Dec 2021 10:08:04 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 12 Dec 2021 10:07:58 -0500 Message-Id: <1639321683-22909-8-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> References: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 07/12] lnet: Fix source specified to routed destination X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Chris Horn , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Chris Horn If a source NI is specified for a send then we should not modify the destination NID that was passed to lnet_send(). HPE-bug-id: LUS-10301 WC-bug-id: https://jira.whamcloud.com/browse/LU-14941 Lustre-commit: 98da4ace43a6c4c59 ("LU-14941 lnet: Fix source specified to routed destination") Signed-off-by: Chris Horn Reviewed-on: https://review.whamcloud.com/44730 Reviewed-by: Serguei Smirnov Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/lib-move.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/net/lnet/lnet/lib-move.c b/net/lnet/lnet/lib-move.c index a411724..caffa30 100644 --- a/net/lnet/lnet/lib-move.c +++ b/net/lnet/lnet/lib-move.c @@ -1989,13 +1989,19 @@ struct lnet_ni * } if (!route_found) { - if (sd->sd_msg->msg_routing) { + if (sd->sd_msg->msg_routing || src_nid != LNET_NID_ANY) { /* If I'm routing this message then I need to find the * next hop based on the destination NID + * + * We also find next hop based on the destination NID + * if the source NI was specified */ best_rnet = lnet_find_rnet_locked(LNET_NIDNET(sd->sd_dst_nid)); if (!best_rnet) { - CERROR("Unable to route message to %s - Route table may be misconfigured\n", + CERROR("Unable to send message from %s to %s - Route table may be misconfigured\n", + src_nid != LNET_NID_ANY ? + libcfs_nid2str(src_nid) : + "any local NI", libcfs_nid2str(sd->sd_dst_nid)); return -EHOSTUNREACH; } From patchwork Sun Dec 12 15:07:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12672323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6497EC433F5 for ; Sun, 12 Dec 2021 15:09:22 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 2240421EBED; Sun, 12 Dec 2021 07:08:32 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B35F821F46A for ; Sun, 12 Dec 2021 07:08:08 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id C546010084EB; Sun, 12 Dec 2021 10:08:04 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id B809DE0813; Sun, 12 Dec 2021 10:08:04 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 12 Dec 2021 10:07:59 -0500 Message-Id: <1639321683-22909-9-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> References: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 08/12] lustre: obdclass: cosmetic changes in pool handling X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Sergey Cheremencev , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sergey Cheremencev cosmetic changes in pool handling: - make tgt_pool_free void - set default rc value in lu_tgt_check_index. HPE-bug-id: LUS-9547 WC-bug-id: https://jira.whamcloud.com/browse/LU-15110 Lustre-commit: d01cae3ec8758fc27 ("LU-15110 quota: cosmetic changes in PQ") Signed-off-by: Sergey Cheremencev Reviewed-on: https://review.whamcloud.com/45258 Reviewed-by: Petros Koutoupis Reviewed-by: Shaun Tancheff Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lu_object.h | 2 +- fs/lustre/obdclass/lu_tgt_pool.c | 15 +++++++-------- 2 files changed, 8 insertions(+), 9 deletions(-) diff --git a/fs/lustre/include/lu_object.h b/fs/lustre/include/lu_object.h index 146398a..3fb40c6 100644 --- a/fs/lustre/include/lu_object.h +++ b/fs/lustre/include/lu_object.h @@ -1423,7 +1423,7 @@ struct lu_tgt_pool { int lu_tgt_pool_init(struct lu_tgt_pool *op, unsigned int count); int lu_tgt_pool_add(struct lu_tgt_pool *op, u32 idx, unsigned int min_count); int lu_tgt_pool_remove(struct lu_tgt_pool *op, u32 idx); -int lu_tgt_pool_free(struct lu_tgt_pool *op); +void lu_tgt_pool_free(struct lu_tgt_pool *op); int lu_tgt_check_index(int idx, struct lu_tgt_pool *osts); int lu_tgt_pool_extend(struct lu_tgt_pool *op, unsigned int min_count); diff --git a/fs/lustre/obdclass/lu_tgt_pool.c b/fs/lustre/obdclass/lu_tgt_pool.c index 17bae54..38ab83d 100644 --- a/fs/lustre/obdclass/lu_tgt_pool.c +++ b/fs/lustre/obdclass/lu_tgt_pool.c @@ -197,14 +197,15 @@ int lu_tgt_pool_remove(struct lu_tgt_pool *op, u32 idx) int lu_tgt_check_index(int idx, struct lu_tgt_pool *osts) { - int rc = 0, i; + int i, rc = -ENOENT; down_read(&osts->op_rw_sem); for (i = 0; i < osts->op_count; i++) { - if (osts->op_array[i] == idx) + if (osts->op_array[i] == idx) { + rc = 0; goto out; + } } - rc = -ENOENT; out: up_read(&osts->op_rw_sem); return rc; @@ -219,13 +220,11 @@ int lu_tgt_check_index(int idx, struct lu_tgt_pool *osts) * deleted from memory. * * @op pool to be freed. - * - * Return: 0 on success or if pool was already freed */ -int lu_tgt_pool_free(struct lu_tgt_pool *op) +void lu_tgt_pool_free(struct lu_tgt_pool *op) { if (op->op_size == 0) - return 0; + return; down_write(&op->op_rw_sem); @@ -235,6 +234,6 @@ int lu_tgt_pool_free(struct lu_tgt_pool *op) op->op_size = 0; up_write(&op->op_rw_sem); - return 0; + return; } EXPORT_SYMBOL(lu_tgt_pool_free); From patchwork Sun Dec 12 15:08:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12672317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 95DBCC433F5 for ; Sun, 12 Dec 2021 15:09:08 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B989D21CBC5; Sun, 12 Dec 2021 07:08:23 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EBB7721F46A for ; Sun, 12 Dec 2021 07:08:08 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id C56B910084F4; Sun, 12 Dec 2021 10:08:04 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id BC313E07E0; Sun, 12 Dec 2021 10:08:04 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 12 Dec 2021 10:08:00 -0500 Message-Id: <1639321683-22909-10-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> References: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 09/12] lustre: llite: properly detect SELinux disabled case X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson Usually, security_dentry_init_security() returns -EOPNOTSUPP when SELinux is disabled. But on some kernels it returns 0 when SELinux is disabled, and in this case the security context is empty. So in both cases make sure the security context name is not set, which means "SELinux is disabled" for the rest of the code. WC-bug-id: https://jira.whamcloud.com/browse/LU-15184 Lustre-commit: 42661f7ba106b7d2e ("LU-15184 llite: properly detect SELinux disabled case") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/45501 Reviewed-by: Jian Yu Reviewed-by: Shaun Tancheff Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/xattr_security.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/fs/lustre/llite/xattr_security.c b/fs/lustre/llite/xattr_security.c index e4fb64a..f14021d 100644 --- a/fs/lustre/llite/xattr_security.c +++ b/fs/lustre/llite/xattr_security.c @@ -60,7 +60,13 @@ int ll_dentry_init_security(struct dentry *dentry, int mode, struct qstr *name, rc = security_dentry_init_security(dentry, mode, name, secctx, secctx_size); - if (rc == -EOPNOTSUPP) + /* Usually, security_dentry_init_security() returns -EOPNOTSUPP when + * SELinux is disabled. + * But on some kernels (e.g. rhel 8.5) it returns 0 when SELinux is + * disabled, and in this case the security context is empty. + */ + if (rc == -EOPNOTSUPP || (rc == 0 && *secctx_size == 0)) + /* do nothing */ return 0; if (rc < 0) return rc; From patchwork Sun Dec 12 15:08:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12672325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DF827C433F5 for ; Sun, 12 Dec 2021 15:09:25 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8E6AD21F596; Sun, 12 Dec 2021 07:08:34 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 678D221CBC5 for ; Sun, 12 Dec 2021 07:08:09 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id C9C2C10084F6; Sun, 12 Dec 2021 10:08:04 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id BF545E07E3; Sun, 12 Dec 2021 10:08:04 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 12 Dec 2021 10:08:01 -0500 Message-Id: <1639321683-22909-11-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> References: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 10/12] lnet: o2iblnd: Default map_on_demand to 1 X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Chris Horn , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Chris Horn On kernels that provide global MR we default to using that exclusively even if FMR/FastReg is available. This causes an interop issue if the active side of a connection request has a higher fragment count than the passive side because FMR/FastReg may be needed to map the higher fragment count. We should change the default map_on_demand to 1 so that FMR/FastReg is used by default. map_on)demand can still be set to 0 if needed. WC-bug-id: https://jira.whamcloud.com/browse/LU-15186 Lustre-commit: 21fdd616bd4784e4e ("LU-15186 o2iblnd: Default map_on_demand to 1") Signed-off-by: Chris Horn Reviewed-on: https://review.whamcloud.com/45431 Reviewed-by: Alexey Lyashkov Reviewed-by: Andriy Skulysh Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/klnds/o2iblnd/o2iblnd_modparams.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c b/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c index 81cde1b..022ed02 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c @@ -141,8 +141,7 @@ * the behavior when transmit with GAPS verses contiguous. */ -#define IBLND_DEFAULT_MAP_ON_DEMAND 1 -static int map_on_demand = IBLND_DEFAULT_MAP_ON_DEMAND; +static int map_on_demand = 1; module_param(map_on_demand, int, 0444); MODULE_PARM_DESC(map_on_demand, "map on demand (obsolete)"); From patchwork Sun Dec 12 15:08:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12672311 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A584FC433EF for ; Sun, 12 Dec 2021 15:08:50 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1128321C9B2; Sun, 12 Dec 2021 07:08:18 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 2EF7D21F46A for ; Sun, 12 Dec 2021 07:08:09 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id C997E10084F5; Sun, 12 Dec 2021 10:08:04 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id C2881E07E5; Sun, 12 Dec 2021 10:08:04 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 12 Dec 2021 10:08:02 -0500 Message-Id: <1639321683-22909-12-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> References: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 11/12] lustre: pcc: disable PCC for encrypted files X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Qian Yingjin When files are encrypted in Lustre using fscrypt, they should normally not be accessible to users without the proper encyrption key. However, if a user has then encryption key loaded when they read a file, it may be decrypted in memory and saved to the PCC backend in unencrypted form. Due to the above reason, we just disable PCC caching for encrypted files. DDN-bug-id: EX-3571 WC-bug-id: https://jira.whamcloud.com/browse/LU-15217 Lustre-commit: f8c79eea11ac96019 ("LU-15217 pcc: disable PCC for encrypted files") Signed-off-by: Qian Yingjin Reviewed-on: https://review.whamcloud.com/45545 Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/file.c | 3 +++ fs/lustre/llite/pcc.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index d3374232..898db80 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -3598,6 +3598,9 @@ static long ll_file_unlock_lease(struct file *file, struct ll_ioc_lease *ioc, if (ioc->lil_count != 1) return -EINVAL; + if (IS_ENCRYPTED(inode)) + return -EOPNOTSUPP; + arg += sizeof(*ioc); if (copy_from_user(¶m.pa_archive_id, (void __user *)arg, sizeof(u32))) { diff --git a/fs/lustre/llite/pcc.c b/fs/lustre/llite/pcc.c index 8430fff..85114b8 100644 --- a/fs/lustre/llite/pcc.c +++ b/fs/lustre/llite/pcc.c @@ -1472,6 +1472,9 @@ int pcc_file_open(struct inode *inode, struct file *file) if (!S_ISREG(inode->i_mode)) return 0; + if (IS_ENCRYPTED(inode)) + return 0; + pcc_inode_lock(inode); pcci = ll_i2pcci(inode); From patchwork Sun Dec 12 15:08:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12672319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3D98CC433F5 for ; Sun, 12 Dec 2021 15:09:15 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 84CCF21F524; Sun, 12 Dec 2021 07:08:26 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9F3CD21F487 for ; Sun, 12 Dec 2021 07:08:09 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id CB0E410084F7; Sun, 12 Dec 2021 10:08:04 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id C6299E07E6; Sun, 12 Dec 2021 10:08:04 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sun, 12 Dec 2021 10:08:03 -0500 Message-Id: <1639321683-22909-13-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> References: <1639321683-22909-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 12/12] lustre: llite: avoid needless large stats alloc X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Zarochentsev , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andreas Dilger Allocate the ll_rw_extents_info (5896 bytes), ll_rw_offset_info (6400 bytes), and ll_rw_process_info (640 bytes) structs only when these stats are enabled, which is very rarely. WC-bug-id: https://jira.whamcloud.com/browse/LU-15217 Lustre-commit: 9490fd9bb84dc277b ("LU-13601 llite: avoid needless large stats alloc") Signed-off-by: Andreas Dilger Signed-off-by: Alexander Zarochentsev Reviewed-on: https://review.whamcloud.com/40901 Reviewed-by: Bobi Jam Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/llite_internal.h | 7 +- fs/lustre/llite/llite_lib.c | 10 +- fs/lustre/llite/lproc_llite.c | 238 ++++++++++++++++++++++++++++----------- 3 files changed, 177 insertions(+), 78 deletions(-) diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index ce7431f..01672b8 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -701,12 +701,12 @@ struct ll_sb_info { struct cl_device *ll_cl; /* Statistics */ - struct ll_rw_extents_info ll_rw_extents_info; + struct ll_rw_extents_info *ll_rw_extents_info; int ll_extent_process_count; unsigned int ll_offset_process_count; + struct ll_rw_process_info *ll_rw_process_info; + struct ll_rw_process_info *ll_rw_offset_info; ktime_t ll_process_stats_init; - struct ll_rw_process_info ll_rw_process_info[LL_PROCESS_HIST_MAX]; - struct ll_rw_process_info ll_rw_offset_info[LL_OFFSET_HIST_MAX]; unsigned int ll_rw_offset_entry_count; int ll_stats_track_id; enum stats_track_type ll_stats_track_type; @@ -997,6 +997,7 @@ int cl_get_grouplock(struct cl_object *obj, unsigned long gid, int nonblock, void ll_rw_stats_tally(struct ll_sb_info *sbi, pid_t pid, struct ll_file_data *file, loff_t pos, size_t count, int rw); +void ll_free_rw_stats_info(struct ll_sb_info *sbi); enum { LPROC_LL_READ_BYTES, diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index 147e680..dddbe7a 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -85,7 +85,6 @@ static struct ll_sb_info *ll_init_sbi(void) unsigned long lru_page_max; struct sysinfo si; int rc; - int i; sbi = kzalloc(sizeof(*sbi), GFP_NOFS); if (!sbi) @@ -161,14 +160,6 @@ static struct ll_sb_info *ll_init_sbi(void) set_bit(LL_SBI_LRU_RESIZE, sbi->ll_flags); set_bit(LL_SBI_LAZYSTATFS, sbi->ll_flags); - for (i = 0; i <= LL_PROCESS_HIST_MAX; i++) { - struct per_process_info *pp_ext; - - pp_ext = &sbi->ll_rw_extents_info.pp_extents[i]; - spin_lock_init(&pp_ext->pp_r_hist.oh_lock); - spin_lock_init(&pp_ext->pp_w_hist.oh_lock); - } - /* metadata statahead is enabled by default */ sbi->ll_sa_running_max = LL_SA_RUNNING_DEF; sbi->ll_sa_max = LL_SA_RPC_DEF; @@ -241,6 +232,7 @@ static void ll_free_sbi(struct super_block *sb) kvfree(items); sbi->ll_foreign_symlink_upcall_items = NULL; } + ll_free_rw_stats_info(sbi); pcc_super_fini(&sbi->ll_pcc_super); kfree(sbi); } diff --git a/fs/lustre/llite/lproc_llite.c b/fs/lustre/llite/lproc_llite.c index a7eb8e1..ce715b4 100644 --- a/fs/lustre/llite/lproc_llite.c +++ b/fs/lustre/llite/lproc_llite.c @@ -1924,15 +1924,16 @@ void ll_debugfs_unregister_super(struct super_block *sb) lprocfs_free_stats(&sbi->ll_stats); } -static void ll_display_extents_info(struct ll_rw_extents_info *io_extents, +static void ll_display_extents_info(struct ll_rw_extents_info *rw_extents, struct seq_file *seq, int which) { unsigned long read_tot = 0, write_tot = 0, read_cum, write_cum; unsigned long start, end, r, w; char *unitp = "KMGTPEZY"; int i, units = 10; - struct per_process_info *pp_info = &io_extents->pp_extents[which]; + struct per_process_info *pp_info; + pp_info = &rw_extents->pp_extents[which]; read_cum = 0; write_cum = 0; start = 0; @@ -1968,39 +1969,103 @@ static void ll_display_extents_info(struct ll_rw_extents_info *io_extents, static int ll_rw_extents_stats_pp_seq_show(struct seq_file *seq, void *v) { struct ll_sb_info *sbi = seq->private; - struct ll_rw_extents_info *io_extents = &sbi->ll_rw_extents_info; + struct ll_rw_extents_info *rw_extents = sbi->ll_rw_extents_info; int k; - if (!sbi->ll_rw_stats_on) { - seq_puts(seq, "disabled\n" - "write anything in this file to activate, then '0' or 'disabled' to deactivate\n"); + if (!sbi->ll_rw_stats_on || !rw_extents) { + seq_puts(seq, "disabled\n write anything in this file to activate, then '0' or 'disabled' to deactivate\n"); return 0; } - lprocfs_stats_header(seq, ktime_get(), io_extents->pp_init, 25, ":", 1); + + spin_lock(&sbi->ll_pp_extent_lock); + lprocfs_stats_header(seq, ktime_get(), rw_extents->pp_init, 25, ":", 1); seq_printf(seq, "%15s %19s | %20s\n", " ", "read", "write"); seq_printf(seq, "%13s %14s %4s %4s | %14s %4s %4s\n", - "extents", "calls", "%", "cum%", - "calls", "%", "cum%"); - spin_lock(&sbi->ll_pp_extent_lock); + "extents", "calls", "%", "cum%", "calls", "%", "cum%"); + for (k = 0; k < LL_PROCESS_HIST_MAX; k++) { - if (io_extents->pp_extents[k].pid != 0) { + if (rw_extents->pp_extents[k].pid != 0) { seq_printf(seq, "\nPID: %d\n", - io_extents->pp_extents[k].pid); - ll_display_extents_info(io_extents, seq, k); + rw_extents->pp_extents[k].pid); + ll_display_extents_info(rw_extents, seq, k); } } spin_unlock(&sbi->ll_pp_extent_lock); return 0; } +static int alloc_rw_stats_info(struct ll_sb_info *sbi) +{ + struct ll_rw_extents_info *rw_extents; + struct ll_rw_process_info *offset; + struct ll_rw_process_info *process; + int i, rc = 0; + + rw_extents = kzalloc(sizeof(*rw_extents), GFP_KERNEL); + if (!rw_extents) + return -ENOMEM; + + for (i = 0; i <= LL_PROCESS_HIST_MAX; i++) { + spin_lock_init(&rw_extents->pp_extents[i].pp_r_hist.oh_lock); + spin_lock_init(&rw_extents->pp_extents[i].pp_w_hist.oh_lock); + } + + spin_lock(&sbi->ll_pp_extent_lock); + if (!sbi->ll_rw_extents_info) + sbi->ll_rw_extents_info = rw_extents; + spin_unlock(&sbi->ll_pp_extent_lock); + /* another writer allocated the struct before we got the lock */ + if (sbi->ll_rw_extents_info != rw_extents) + kfree(rw_extents); + + process = kzalloc(sizeof(*process) * LL_PROCESS_HIST_MAX, + GFP_KERNEL); + if (!process) { + rc = -ENOMEM; + goto out; + } + offset = kzalloc(sizeof(*offset) * LL_OFFSET_HIST_MAX, + GFP_KERNEL); + if (!offset) { + rc = -ENOMEM; + goto out_free; + } + + spin_lock(&sbi->ll_process_lock); + if (!sbi->ll_rw_process_info) + sbi->ll_rw_process_info = process; + if (!sbi->ll_rw_offset_info) + sbi->ll_rw_offset_info = offset; + spin_unlock(&sbi->ll_process_lock); + + /* another writer allocated the structs before we got the lock */ + if (sbi->ll_rw_offset_info != offset) + kfree(offset); + if (sbi->ll_rw_process_info != process) { +out_free: + kfree(process); + } +out: + return rc; +} + +void ll_free_rw_stats_info(struct ll_sb_info *sbi) +{ + kfree(sbi->ll_rw_extents_info); + sbi->ll_rw_extents_info = NULL; + kfree(sbi->ll_rw_offset_info); + sbi->ll_rw_offset_info = NULL; + kfree(sbi->ll_rw_process_info); + sbi->ll_rw_process_info = NULL; +} + static ssize_t ll_rw_extents_stats_pp_seq_write(struct file *file, const char __user *buf, - size_t len, - loff_t *off) + size_t len, loff_t *off) { struct seq_file *seq = file->private_data; struct ll_sb_info *sbi = seq->private; - struct ll_rw_extents_info *io_extents = &sbi->ll_rw_extents_info; + struct ll_rw_extents_info *rw_extents; int i; s64 value; @@ -2009,19 +2074,31 @@ static ssize_t ll_rw_extents_stats_pp_seq_write(struct file *file, value = ll_stats_pid_write(buf, len); - if (value == 0) + if (value == 0) { sbi->ll_rw_stats_on = 0; - else + } else { + if (!sbi->ll_rw_extents_info) { + int rc = alloc_rw_stats_info(sbi); + + if (rc) + return rc; + } sbi->ll_rw_stats_on = 1; + } + spin_lock(&sbi->ll_pp_extent_lock); - io_extents->pp_init = ktime_get(); - for (i = 0; i < LL_PROCESS_HIST_MAX; i++) { - io_extents->pp_extents[i].pid = 0; - lprocfs_oh_clear(&io_extents->pp_extents[i].pp_r_hist); - lprocfs_oh_clear(&io_extents->pp_extents[i].pp_w_hist); + rw_extents = sbi->ll_rw_extents_info; + if (rw_extents) { + rw_extents->pp_init = ktime_get(); + for (i = 0; i < LL_PROCESS_HIST_MAX; i++) { + rw_extents->pp_extents[i].pid = 0; + lprocfs_oh_clear(&rw_extents->pp_extents[i].pp_r_hist); + lprocfs_oh_clear(&rw_extents->pp_extents[i].pp_w_hist); + } } spin_unlock(&sbi->ll_pp_extent_lock); + return len; } @@ -2030,22 +2107,22 @@ static ssize_t ll_rw_extents_stats_pp_seq_write(struct file *file, static int ll_rw_extents_stats_seq_show(struct seq_file *seq, void *v) { struct ll_sb_info *sbi = seq->private; - struct ll_rw_extents_info *io_extents = &sbi->ll_rw_extents_info; + struct ll_rw_extents_info *rw_extents = sbi->ll_rw_extents_info; - if (!sbi->ll_rw_stats_on) { - seq_puts(seq, "disabled\n" - "write anything in this file to activate, then '0' or 'disabled' to deactivate\n"); + if (!sbi->ll_rw_stats_on || !rw_extents) { + seq_puts(seq, "disabled\n write anything in this file to activate, then '0' or 'disabled' to deactivate\n"); return 0; } - lprocfs_stats_header(seq, ktime_get(), io_extents->pp_init, 25, ":", 1); + spin_lock(&sbi->ll_lock); + lprocfs_stats_header(seq, ktime_get(), rw_extents->pp_init, 25, ":", 1); seq_printf(seq, "%15s %19s | %20s\n", " ", "read", "write"); seq_printf(seq, "%13s %14s %4s %4s | %14s %4s %4s\n", "extents", "calls", "%", "cum%", "calls", "%", "cum%"); - spin_lock(&sbi->ll_lock); - ll_display_extents_info(io_extents, seq, LL_PROCESS_HIST_MAX); + + ll_display_extents_info(rw_extents, seq, LL_PROCESS_HIST_MAX); spin_unlock(&sbi->ll_lock); return 0; @@ -2057,7 +2134,7 @@ static ssize_t ll_rw_extents_stats_seq_write(struct file *file, { struct seq_file *seq = file->private_data; struct ll_sb_info *sbi = seq->private; - struct ll_rw_extents_info *io_extents = &sbi->ll_rw_extents_info; + struct ll_rw_extents_info *rw_extents; int i; s64 value; @@ -2066,17 +2143,27 @@ static ssize_t ll_rw_extents_stats_seq_write(struct file *file, value = ll_stats_pid_write(buf, len); - if (value == 0) + if (value == 0) { sbi->ll_rw_stats_on = 0; - else + } else { + if (!sbi->ll_rw_extents_info) { + int rc = alloc_rw_stats_info(sbi); + + if (rc) + return rc; + } sbi->ll_rw_stats_on = 1; + } spin_lock(&sbi->ll_pp_extent_lock); - io_extents->pp_init = ktime_get(); - for (i = 0; i <= LL_PROCESS_HIST_MAX; i++) { - io_extents->pp_extents[i].pid = 0; - lprocfs_oh_clear(&io_extents->pp_extents[i].pp_r_hist); - lprocfs_oh_clear(&io_extents->pp_extents[i].pp_w_hist); + rw_extents = sbi->ll_rw_extents_info; + if (rw_extents) { + rw_extents->pp_init = ktime_get(); + for (i = 0; i <= LL_PROCESS_HIST_MAX; i++) { + rw_extents->pp_extents[i].pid = 0; + lprocfs_oh_clear(&rw_extents->pp_extents[i].pp_r_hist); + lprocfs_oh_clear(&rw_extents->pp_extents[i].pp_w_hist); + } } spin_unlock(&sbi->ll_pp_extent_lock); @@ -2094,17 +2181,21 @@ void ll_rw_stats_tally(struct ll_sb_info *sbi, pid_t pid, struct ll_rw_process_info *offset; int *off_count = &sbi->ll_rw_offset_entry_count; int *process_count = &sbi->ll_offset_process_count; - struct ll_rw_extents_info *io_extents = &sbi->ll_rw_extents_info; + struct ll_rw_extents_info *rw_extents; if (!sbi->ll_rw_stats_on) return; - process = sbi->ll_rw_process_info; - offset = sbi->ll_rw_offset_info; spin_lock(&sbi->ll_pp_extent_lock); + rw_extents = sbi->ll_rw_extents_info; + if (!rw_extents) { + spin_unlock(&sbi->ll_pp_extent_lock); + return; + } + /* Extent statistics */ for (i = 0; i < LL_PROCESS_HIST_MAX; i++) { - if (io_extents->pp_extents[i].pid == pid) { + if (rw_extents->pp_extents[i].pid == pid) { cur = i; break; } @@ -2115,24 +2206,29 @@ void ll_rw_stats_tally(struct ll_sb_info *sbi, pid_t pid, sbi->ll_extent_process_count = (sbi->ll_extent_process_count + 1) % LL_PROCESS_HIST_MAX; cur = sbi->ll_extent_process_count; - io_extents->pp_extents[cur].pid = pid; - lprocfs_oh_clear(&io_extents->pp_extents[cur].pp_r_hist); - lprocfs_oh_clear(&io_extents->pp_extents[cur].pp_w_hist); + rw_extents->pp_extents[cur].pid = pid; + lprocfs_oh_clear(&rw_extents->pp_extents[cur].pp_r_hist); + lprocfs_oh_clear(&rw_extents->pp_extents[cur].pp_w_hist); } for (i = 0; (count >= 1 << (LL_HIST_START + i)) && (i < (LL_HIST_MAX - 1)); i++) ; if (rw == 0) { - io_extents->pp_extents[cur].pp_r_hist.oh_buckets[i]++; - io_extents->pp_extents[LL_PROCESS_HIST_MAX].pp_r_hist.oh_buckets[i]++; + rw_extents->pp_extents[cur].pp_r_hist.oh_buckets[i]++; + rw_extents->pp_extents[LL_PROCESS_HIST_MAX].pp_r_hist.oh_buckets[i]++; } else { - io_extents->pp_extents[cur].pp_w_hist.oh_buckets[i]++; - io_extents->pp_extents[LL_PROCESS_HIST_MAX].pp_w_hist.oh_buckets[i]++; + rw_extents->pp_extents[cur].pp_w_hist.oh_buckets[i]++; + rw_extents->pp_extents[LL_PROCESS_HIST_MAX].pp_w_hist.oh_buckets[i]++; } spin_unlock(&sbi->ll_pp_extent_lock); spin_lock(&sbi->ll_process_lock); + process = sbi->ll_rw_process_info; + offset = sbi->ll_rw_offset_info; + if (!process || !offset) + goto out_unlock; + /* Offset statistics */ for (i = 0; i < LL_PROCESS_HIST_MAX; i++) { if (process[i].rw_pid == pid) { @@ -2143,8 +2239,7 @@ void ll_rw_stats_tally(struct ll_sb_info *sbi, pid_t pid, process[i].rw_largest_extent = count; process[i].rw_offset = 0; process[i].rw_last_file = file; - spin_unlock(&sbi->ll_process_lock); - return; + goto out_unlock; } if (process[i].rw_last_file_pos != pos) { *off_count = @@ -2173,8 +2268,7 @@ void ll_rw_stats_tally(struct ll_sb_info *sbi, pid_t pid, if (process[i].rw_largest_extent < count) process[i].rw_largest_extent = count; process[i].rw_last_file_pos = pos + count; - spin_unlock(&sbi->ll_process_lock); - return; + goto out_unlock; } } *process_count = (*process_count + 1) % LL_PROCESS_HIST_MAX; @@ -2186,14 +2280,16 @@ void ll_rw_stats_tally(struct ll_sb_info *sbi, pid_t pid, process[*process_count].rw_largest_extent = count; process[*process_count].rw_offset = 0; process[*process_count].rw_last_file = file; + +out_unlock: spin_unlock(&sbi->ll_process_lock); } static int ll_rw_offset_stats_seq_show(struct seq_file *seq, void *v) { struct ll_sb_info *sbi = seq->private; - struct ll_rw_process_info *offset = sbi->ll_rw_offset_info; - struct ll_rw_process_info *process = sbi->ll_rw_process_info; + struct ll_rw_process_info *offset; + struct ll_rw_process_info *process; int i; if (!sbi->ll_rw_stats_on) { @@ -2204,12 +2300,13 @@ static int ll_rw_offset_stats_seq_show(struct seq_file *seq, void *v) spin_lock(&sbi->ll_process_lock); lprocfs_stats_header(seq, ktime_get(), sbi->ll_process_stats_init, 25, ":", true); - seq_printf(seq, "%3s %10s %14s %14s %17s %17s %14s\n", "R/W", "PID", "RANGE START", "RANGE END", "SMALLEST EXTENT", "LARGEST EXTENT", "OFFSET"); + /* We stored the discontiguous offsets here; print them first */ - for (i = 0; i < LL_OFFSET_HIST_MAX; i++) { + offset = sbi->ll_rw_offset_info; + for (i = 0; offset && i < LL_OFFSET_HIST_MAX; i++) { if (offset[i].rw_pid != 0) seq_printf(seq, "%3c %10d %14llu %14llu %17lu %17lu %14lld\n", @@ -2221,8 +2318,10 @@ static int ll_rw_offset_stats_seq_show(struct seq_file *seq, void *v) (unsigned long)offset[i].rw_largest_extent, offset[i].rw_offset); } + /* Then print the current offsets for each process */ - for (i = 0; i < LL_PROCESS_HIST_MAX; i++) { + process = sbi->ll_rw_process_info; + for (i = 0; process && i < LL_PROCESS_HIST_MAX; i++) { if (process[i].rw_pid != 0) seq_printf(seq, "%3c %10d %14llu %14llu %17lu %17lu %14lld\n", @@ -2245,8 +2344,6 @@ static ssize_t ll_rw_offset_stats_seq_write(struct file *file, { struct seq_file *seq = file->private_data; struct ll_sb_info *sbi = seq->private; - struct ll_rw_process_info *process_info = sbi->ll_rw_process_info; - struct ll_rw_process_info *offset_info = sbi->ll_rw_offset_info; s64 value; if (len == 0) @@ -2254,19 +2351,28 @@ static ssize_t ll_rw_offset_stats_seq_write(struct file *file, value = ll_stats_pid_write(buf, len); - if (value == 0) + if (value == 0) { sbi->ll_rw_stats_on = 0; - else + } else { + if (!sbi->ll_rw_process_info || !sbi->ll_rw_offset_info) { + int rc = alloc_rw_stats_info(sbi); + + if (rc) + return rc; + } sbi->ll_rw_stats_on = 1; + } spin_lock(&sbi->ll_process_lock); sbi->ll_offset_process_count = 0; sbi->ll_rw_offset_entry_count = 0; sbi->ll_process_stats_init = ktime_get(); - memset(process_info, 0, sizeof(struct ll_rw_process_info) * - LL_PROCESS_HIST_MAX); - memset(offset_info, 0, sizeof(struct ll_rw_process_info) * - LL_OFFSET_HIST_MAX); + if (sbi->ll_rw_process_info) + memset(sbi->ll_rw_process_info, 0, + sizeof(struct ll_rw_process_info) * LL_PROCESS_HIST_MAX); + if (sbi->ll_rw_offset_info) + memset(sbi->ll_rw_offset_info, 0, + sizeof(struct ll_rw_process_info) * LL_OFFSET_HIST_MAX); spin_unlock(&sbi->ll_process_lock); return len;